Getting you up to date

Its been a long while since I last posted, previously I was working with Linux and getting trying to get minimize events handled but to no avail.

After the Linux task, I moved on to creating a library dependency check system to see if any of GateWare's library would need a certain library enabled in order to run correctly. The check is a preprocessor check that runs on build time. There are two types of messages the user will get, either a warning or an error. This depends on what the user defines. By default the user will get warning messages but if defined, the user can force error messages or disable the checks all together.

I worked on the dependency checks for two weeks after the last post, or the last two weeks of June, which lead into a week long Summer break. When break was over I would be working on UWP.

During the break(last week) I did some research and had a meeting with the Chase Richards, who started the UWP portion of GW, to see where he left off. As always he was very informal and helpful.

At the beginning of this week GW had a meeting and we talked about where I would start and plan things out. During the meeting I brought up an outline I had made from my meeting with Chase and we started building a work-path from there. 

In my opinion the meeting went really smoothly, besides the fact that I am the only dev currently. I believe having the meeting with Chase proved to be a very good choice because it  provided me with an outline of what to do and it let me gather the correct information to justify the points in my outline.

From the meeting my work-path would be as follows:

- Get UWP branch up to date with development

- Port missing UWP libraries

- Isolate UWP code and iOS code in app files

This whole week I have been working on getting UWP up to date from 6 months ago. The first thing I was going to do was go step by step and merging up the Git timeline. The reason behind it was i wanted to see the changes happening before my eyes to I know what merge conflicts to choose. After two merges I decided to just attempt to merge up to our development branch. It seemed to work at first, when the Input and Graphics unit tests were disabled. If they were enabled however I would get linker errors that came from some tests.

Since it worked in a certain case I decided to keep the merge. The next thing to do was fix the linker errors, which caused a lot of headache. After some time staring at the code, I realized that UWP uses different API's than GW does for user input and for console windows. The reason for the errors was because the tests were enabled but we could not feed it the right data because UWP uses its own api. The solution to this was to disable the Input and Graphics tests if UWP is being used, even if the user enables them. This is done by this:

#if !defined(WINAPI_FAMILY) && !(WINAPI_FAMILY == WINAPI_FAMILY_APP)

#endif

This checks if the UWP api is being used.

I just had a meeting with Lari and Chase earlier today over messaging. We all are on the same page about where GW and UWP is at. Now that I have a stable build i will be testing for bugs and try to see if i can make my patch more efficient. Other than that the upcoming week I will start to work on porting missing libraries.

Link to predefined macro:

https://social.msdn.microsoft.com/Forums/en-US/51f29cfe-d07f-4898-9962-945ed7b66d7f/predefined-macro-to-identify-uwp?forum=wpdevelop

Linux Minimize Event - end of my attempt

So far this week I was working on the Minimize event not being reported on Linux, I had trouble setting up my Linux OS on my second internal. This bug is not an easy bug to fix because 'PropertyNotify' is not being returned when the event of Minimizing the window happens, while it is maximized. Since it is not being returned, it does not go into our case statement that checks what state the window is in and prints us information. 

I noticed that if we tried to minimize while max'd we get an 'Unmap' return rather than the 'PropertyNotify' so I tried to add case statements for map and unmap. I was then going to try and differentiate between when a window is closed, or minimized. I dont think it would have been that bad because I believe there were checks for if the window is being closed or is closed that happens before we listen to events.

 Since this bug was taking too much time and I am still an intern, we decided to move me onto another ticket. My ticket now is to create output messages that tell the user if a certain library is dependent on another library. This should not stop the program from running (dummies will run). The messages should let the user know the tests will run, but the tests that do not have their dependencies will possibly fail or not yield good results.

My time here at GW so far has been a little intense with the bugs that got complicated. Although I could not fix the Mac audio problem or the Minimize event not being reported on linux, I got a lot of experience with working in each environment (Linux was friendly as expected). Working on these different bugs was pretty cool too, I learned some new stuff just by going through the code itself.

Mountains under those hills

 Since my last post I was working with Vulkan and Mac, which had a few problems that didn't seem that hard to fix. That didnt turn out that way, turns out Vulkan has bugs of their own that affect us. There is also a sound problem that seems to be only happening on one of our remote machines, this turned out to be a thread related problem so I backed off  from that ticket and moved to work on a Linus Mint ticket.

At the end of last week, I was working on a system event problem that is found on Linux Mint OS. Starting off on this problem was slow because I had to read up on the X11 library (not to be confused with DX11). From there I had to figure out how the code was being worked in our project. 

The main problem of this ticket was that no event seems to be being reported when the window is maximized(everything else is being reported when not maximized). Thats because property_notify is not the flag being returned when we try to minimize the window. 

The reason for property_notify not being returned is because when you minimize, the resolution stays the same as before you minimized so the system doesn't register a change to the window. 

I did find that map and unmap notify were being returned when I did something to the window(minimize and restore/expose). My plan is to somehow detect if the minimize button is pressed through the map/unmap notify and then change the windows resolution property. From there we go into property notify if we can and run that case.

I was doing my testing/coding via remote machine and ran into a problem where my terminal would sometimes act weird. It might have been because of the project itself not displaying output.

To resolve this someone suggested to install Mint and dual boot. I finally got Mint installed but not the way I want it. The process is usually easy I would assume, but I was installing it on a second internal ssd. The problem was that I could not find a proper tutorial as different tutorials would leave out some small information, do things a different way or chose a different option when setting things up. Another problem I ran into was the actual dual boot process. After my first install my computer would use Grub(Linux boot launcher) instead of Windows. From there I logged into Windows and noticed some of my computer settings were off like my resolution and my time was incorrect as well. I tried changing the boot order via BIOS but in doing so the Linux OS was unavaliable(not showing on list). After all that I looked into getting dual boot to work but it turns out the Mint installer has a bug where it installs the boot loader to the first efi it finds, the one way to bypass that is to unmount the internal drive you do not want Mint on. I chose not to do this because I am using a laptop. With that said I will bite the bullet and just use the Grub booter for now until i have the time to properly set up Mint the way i want.

 

Well, besides the problems that got elevated to out of my reach, I think things are going well. I believe that I am learning quite a few valuable things. For example documentation! Finding help for X11 can be sometimes rare and the X11 docs can be frustrating as it is not really up to par compared to other documentations. While I was going through the X11 documentation there are a lot of places where they don't really tell or show you how something works, it would just give a description and a list of things that are related to it. Other than that I think i gained more knowledge and experience working outside of the Windows OS and about BIOS.

A lot of people are probably mad about the Vulkan update

 This is my second post and it has been quite interesting since my last post.

Since my last post I have been working on patching up problems that a Vulkan update brought to us when we updated to Vulkan 1.2.176. The problem is the 'VK_HKR_portability_subset' device extension not being included when the device is getting set up.

This problem is not from our side, more on the Vulkan side. Fortunately Khronos, who works on Vulkan is aware of this problem is working on a fix.

For now the work around was including a beta header file 'vulkan_beta.h' so the said extension earlier will be defined and we can use it. Although this is not the ideal solution, it was the only solution to keep GW's code from not changing dramatically as Khronos is working on a fix.

We had multiple errors that were related to the 'VK_HKR_portability_subset' extension. When I added the extension from the beta.h file, only one error was left. This error was a parameter error from me trying to use W::GRAPHICS::GVulkanSurface::Create function. What I did was try to increase the number of device extensions to ‘2’ and sending in a vector of ‘const char*’s but it failed.

From working on this I got a better understanding on how programs work on multiple platforms as well as some insight on how different programs work together.

As of now I will be dropping the ticket and will be moving on to fix the GW::Audio problem for Mac.

Hello Gateware!

 Hello,

My name is Lam Truong and I have recently joined the Gateware Team! I am starting off as a Gen Dev and will possibly transition to other things like porting.
Within the first week, I have gotten Gateware to work on Linux, Windows, and Mac with a new update from Vulkan, ver 1.2.176.1. With the perfect timing of the update and me having little to no experience with a Mac, I had the opportunity to get the FULL Mac experience. This week I will be working on getting sound re-enabled on Mac continuing my experience.
Besides a few hiccups I am enjoying my time here and I am sure that will continue!

The Tales of Vulkan on iOS: Simple mistakes

    At the end of last week, I had thought that I would be done with Vulkan for good. Turns out a simple mistake I made back when I ported GWindow got me stuck on Vulkan. To understand what got me stuck, we need to first learn about UIKit's UIViews and CALayers. A NSView, which macOS uses for content in windows, can optionally be what is called "layer-backed" and CALayers can be added to the View to get special rendering effects, like gradients, fades, and animation. Metal subclasses CALayer into CAMetalLayer to get their GPU accelerated rendering, and this is true for macOS and iOS. Where they diverge is how you actually get a CAMetalLayer onto the views. On macOS you can just add it at runtime as long as you are on the UIThread. iOS has more restrictions with this but we will get to that later. It is critical to know that to make a Vulkan surface for iOS you need to pass a UIView that has a CAMetalLayer attached or a CAMetalLayer instance itself.

    UIViews cannot be not layer-backed like NSViews can. This means that UIViews need to have their Layer defined under the "layerClass" method when you need to be different than the default CALayer. Now when I got Vulkan rendering before, I did just this, but I soon found that we have an issue with that. Like Ozzie, I ran into the duplicate symbols issue when trying to link to the Vulkan Surface file. This had not happened when I had the UIView subclass in a different file. This usually wouldn't be such a big deal, just use the Objective-C runtime library to get around the file-per-implementation rule. Altough this could be done, I could not find a way for Vulkan to detect the Metal support I was implementing at runtime. I even tried changing the methods of the meta-class of the subclassed UIView with no luck. Vulkan just would not recognize the Metal support.

    This is where I had started to panic. I had effectively 5 days until my presentation and I did not have Vulkan rendering anymore. I started to look for alternatives to passing the UIView to Vulkan and found that you could just pass the CAMetalLayer instance itself. I tried casting the CALayer that was attached to the runtime view, but even that would recognize as not Metal compatible by Vulkan. I had run out of ideas and looked for a alternative to subclassing UIView altogether. It was lucky that I started to do this, since I found a subclass of UIView that (presumably) was not created using the Objective-C Runtime Library. This class was MTKView, which stands for Metal Kit View. You can create an instance of this class and it will have a CAMetalLayer attached by default. This saved my skin, as it made the Vulkan surface creation method happy. But there was one other problem, the view would only render the top left quadrant of the screen. I thought that it was back to the lab with this so I went back.

    

    I tried adding a sublayer to the default UIView and rendering with that, but that didn't work and made the problem worse, even though it did render. Finally I had to call for some help. I first got Ozzie in a call, to see if I was doing anything wrong with the Objective-C runtime, but it turns out I made the classes perfectly (somehow) and it really was Vulkan not recognizing the View. Then I called Lari, the Graphics man himself. He instantly recognized the issue as a viewport scaling issue. We got to work messing with the scaling options with the sublayer implementation. That ended up not working too well, so we went back to the MTKView implementation. Here is where we made a big discovery. iOS uses an odd method of getting pixel data. They use a width and height of pixels, but it is a little different with retina-enabled devices. They also use a "Scale Factor" property. This is different per-device, usually the larger the device, the larger the scaling factor. The iPhone 8 has a scaling factor of 2 while an iPhone 6s+ has a scaling factor of 3. Older phones like the iPhone 3 and 4 have a factor of 1. 

    With all of this information we finally came to the conclusion that the View and Layers all had the correct scaling and resolution applied. But there was one object that did not, the Vulkan Swapchain. I did not mention that neither Vulkan, nor Metal had any errors when rendering. This is because the Vulkan swapchain was half the resolution it should have been. By simply doubling the width and height of the swapchain, we fixed the top left corner rendering issue. The problem was actually in GWindow all along. It did not account for the scaling of retina devices. Once I resolved that mistake the Vulkan Surface worked like a charm. 

References:

UIKit

https://developer.apple.com/documentation/uikit?language=objc

MetalKit

https://developer.apple.com/documentation/metalkit/

MoltenVK

https://github.com/KhronosGroup/MoltenVK

UIView

https://developer.apple.com/documentation/uikit/uiview?language=objc

MTKView

https://developer.apple.com/documentation/metalkit/mtkview

NSView

https://developer.apple.com/documentation/appkit/nsview

CALayer

https://developer.apple.com/documentation/quartzcore/calayer?language=objc

CAMetalLayer

https://developer.apple.com/documentation/quartzcore/cametallayer

Duplicate Symbols Issue, By Ozzie
https://fsgateware.blogspot.com/2020/07/duplicate-symbols.html

Objective-C Runtime Library

https://developer.apple.com/documentation/objectivec/objective-c_runtime?language=objc

Vulkan on iOS part 2

 After being able to link Vulkan libraries in our Xcode iOS project, we are finally able to write Vulkan code for iOS. To use Vulkan with iOS, we need to give Vulkan the pixel buffer area for the screen. Usually this is done through something like HWND for DirectX, but with iOS we cannot get that low to a raw window pointer. So Vulkan asks for a UIView pointer, but not just any UIView. The one that you give to Vulkan must have a backing layer if CAMetalLayer or something that derives from CAMetalLayer. In MacOS this is an easy thing to achieve, just grab the NSWindow and say that it's backing layer is Metal, but in iOS, it gets a bit messy.

 iOS UIviews are layer-backed, meaning that they are given a layer upon creation and you absolutely cannot change it after (unless you write a custom View controller). This creates a conflict for Gateware. We want the user to be able to create a UIView themselves if they wish and still be able to use Vulkan, but we cannot change what they have specified. It is possible to run 2 controllers at the same time, or even just 2 views on the same controller. This solution has its drawbacks too. Since Gateware is usually used for games, this extra view or controller would be unecessarily using system resources. There must be a solution where the user can create a window but have Gateware put Vulkan into it.

 This is where I had an idea, if we cannot change the current view, can we change the current view controller? The answer was simple, as long as you present the view once you are done swapping them, the newly created view would be the one that is visible, and on top. So I had decided to create a Gateware View and View controller that had support for Metal. Next was just to detect whether or not the users View was Metal capable and overwrite it if not. This allows easy access for people wanting to develop Vulkan for their games and a level of control for people that already have a game looking to use Gateware.

  I am glad to say that the effort paid off. After fiddling around with some of the desktop Unit Tests, (and some incorrect file IO pathing) I was able to run them on iOS Simulator and an iPhone 7 running iOS 13. It feels nice to finally be done with Vulkan and to move on to Audio.

 References:

MoltenVK 

https://github.com/KhronosGroup/MoltenVK

UIKit UIViewController 

 https://developer.apple.com/documentation/uikit/uiviewcontroller

UIKit UIView 

 https://developer.apple.com/documentation/uikit/uiview

Metal CAMetalLayer 

https://developer.apple.com/documentation/quartzcore/cametallayer