I Broke Vulkan

 So now that I've finished cleaning up all the warnings on Windows, Linux, and Mac, we're getting pretty close to a new version release. Part of that process is merging branches back into the main development branch. I merged my warnings branch back in and set to work testing and updating the templates to the new version.

The first project I opened was RedScreen, our Vulkan template. It built and rand fine but then if you tried to resize the window or maximize it, this happened:

Good ol' Vulkan puking up errors that nobody can understand. 

Basically, what it's saying is that the swap chain needs to be recreated from scratch every time you resize the frame buffer (the window), but it wasn't happening.

Testing with the previous version of Gateware.h showed that it worked fine there, so clearly somebody screwed things up recently. Lari initially thought he broke it with a fix that he added earlier in the day, but I tested the commit right before his, and the problem persisted. 

So now I'm searching through commit after commit, trying to find all the instances where the Vulkan core is modified. And lo and behold!

Culprit located. In my very first commit to the gateware repo, I found this line with an unused return value. I seem to have brain farted, because instead of just removing the assignment, I removed the entire line. We need GetSurfaceData to actually set the proper variables when the window/surface is resized. This problem was there for over a month, it just wasn't caught until now because we don't have unit tests that test if the Vulkan surface is successfully re-sizable. 

So I added GetSurfaceData back in, tested RedScreen again and...

All is well in the Vulkan template again.

Apple's Uncanny Entry Point

     The week before last I had encountered a problem, one that I believe very few developers, if any, have tried to solve. To understand the problem we must first understand the goal. First Gateware code is meant to be compatible with 3 (currently) platforms, Windows, Linux, and macOS. Over the last few months, Chase and I have been porting Gateware to less desktop based environments, Windows UWP for him, and iOS for me. There is a subtle difference between desktop and app ecosystems that most developers don't think of off the top of their heads; entry points. Entry points serve as the beginning of any program, and most (if not all) programs need an entry point to start out. Windows (Win32), Linux, and macOS all use some form of

 int main(int argc, char** argv); 

for an entry point, but things get a little funky with app based programs.

    UWP and, if you configure it correctly,  Win32 take a different function as an entry point

 int WINAPI wWinMain(...);  

and while this may be convenient for developers of those platforms to get the extra information from the parameters of this function, it makes the life of a cross-platform developer much more difficult. To account for this change in entry, we made GApp, a class that would define whatever entry point was required on a platform, and call the usual main that the user would write. This keeps the code simple as the users of Gateware would only have to write one main for every platform. For UWP this was pretty simple, you take main as a parameter to a global constructor and run it from there, making sure to define WinMain at the same time. For iOS, this is a different story.  

    iOS does traditional main, so there's no issue right? Well there is a caveat with that. To get the storyboard to render, one must call the

 int UIApplicationMain(int argc, char * _Nullable *argv, NSString *principalClassName, NSString *delegateClassName);  

function. Well then just call this function in GApp and we're good right? Nope. Not only must this function be called on the main operating thread, it must also be called in main, or in any function that main calls (assuming that it isn't on another thread). Well, this is an issue, how can we call this function and run the code that the user supplies? You cannot multithread this function on the same thread, as it overrides regular main and as such it will never run the regular main code. And you cannot dispatch this function, as dispatch will favor UIApplicationMain over traditional main and traditional main will be put on the back burner.

    So is that it? Is there any way to have main run on the same thread put asynchronous to UIApplicationMain? I do not know. But there is an unexpected solution. With UWP, GApp defines the main entry point and calls the user-defined main later, so why can't we do the same with iOS? So we defined the traditional main entry point, and later in the file, did this

 #define main Gateware_main  

and it worked. As with the UWP GApp, the iOS GApp constructor takes the main function as a parameter and calls it later, but this time the main function that the user writes is not called, as it is now recognized by the compiler as "Gateware_main" and as such doesn't run it. Instead the compiler will see the GApp main as the entry point and run that. So we can have both functions. The GApp main handles everything for the iOS side, and then later calls the users main so that their functionality still works. This works as UIApplicationMain does require to be on the main thread and called by main, so that condition is checked, and there are none for the users main, other than it is expected to be the start of the program, so we run it as soon as the app is done launching.

     

3 Months with an Undiscovered Bug

     Since my last blog post, GWindow has finally been finished. Well maybe not finished, but it is in a very good place so that I can start porting other important libraries so someone could begin to make a game for Xbox. This was a decision that had to be made. I can't spend the rest of my last month on this project getting one library to 100% when other ones are still sitting 0. With GWindow mostly out of the way, I began to work on the DirectX 11 port. This is where I found the bug that has been there for 3 months and I only now discovered it.

The Problem:

    When I went to use the macro IID_PPV_ARGS, I kept receiving the error that it was undefined, when I know for a fact that it is. This led me down the rabbit hole of trying to figure out why this was the way it was. After a little research and cross referencing other C++/WinRT projects, I came to find out that C++/WinRT overrides a function that the macro is representing. This override is in base.h, which is generated at runtime with  C++/WinRT. Turns out that this whole time, my UWP Gateware project was never generating any of the header files it needs for the language projection from C++ to WinRT APIs. How was it even running then you ask, well it was using the WinRT header files included in the Windows 10 SDK, which are from the first iteration of C++/WinRT and that base.h did not have that override function I needed for the macro.

The Solution:

    Lari had the idea to check the the .sln and .vcxproj files to see if anything in those differ from what was in the projects that I know work. Eureka! There was a discrepancy in the .vcxproj file. It did not include references to C++/WinRT while the working projects did. I needed to come up with a way to get those in there automatically without having me and future devs go in there manually and fix this. What I ended up doing was creating a PowerShell script that takes in the file after CMake compiles it and edits it. I tried my best to write it in a way that if more files are put into the project that my script will not need to be rewritten. Here is that script: 

     This script adds back in the references to the .vcxproj file and allows the project to generate the the header files needed. In the end, the macro worked again and I can now continue my DirectX 11 port. Speaking of the DirectX port it is coming along great so far, minus a few threading issues. I'll save that for next week when its done.

Apple Heard You Like Braces

    After cleaning up all of the user-visible warnings on Linux earlier this week, I moved on to doing the same for Mac. For the uninitiated, Apple uses a custom version of the Clang compiler. This compiler is very picky about how you should write your code, and it will throw all kinds of warnings at you. In this case, I was getting something like 400 instances of -Wmissing-braces.

    All of these warnings came from one file... the implementation for GCollision. GCollision is very large; some 13,000+ lines. GCollision also uses lines, vectors, and matrices that are defined in GMath like this:

The union is just there to let us access the two vectors (points) with an index if we wanted to loop or something like that. Using a GLINE is pretty straightforward. Here is an example of a declaration of a GLINE within some function: 

    The first line in the function sets data in the first vector of the line. The second line does the same for the second vector. All is well, right? 

    Not according to Apple:

    "Suggest braces around initialization of subobject"

    Apparently, this is caused by the way we declared our line/vector/matrix classes. Apple clang treats the class itself as one scope, the union as another scope, and the struct within the union as a third scope. 

    The solution to this warning is to add braces for each scope. So for our GLINE here, we have to add them for the scopes of the line class itself, and then also for the vector classes within the final scope.

    It looks like this...

    ...and it hurts me.

    After figuring out the solution, I now had to add braces like this to all ~400 instances of this warning in GCollision, by hand, on a remote Mac three timezones away.

    In the end it took me about four and a half hours.

    Why, Apple? Why did you make me do this? *cries in braces*

The Sandbox environment in iOS

     Over the last week I have been porting GFile to iOS. As such I have been getting used to the environment that iOS places app developers in. iOS apps only allow developers to access certain directories when doing file I/O. Any directory accessed outside of these specified locations will give you a "permission denied" error. The app needs to be given access to directories outside of it's "sandbox" by the user, and even then you still do not have access to the whole device. With these restrictions I need to conform the GFile implementation as to not access files that the operating system does not allow to be accessed.

    I first started out the GFile implementation by copying what was done on Mac, and mold it into something that would work on iOS. At the moment I have some of the functionality working on iOS, but not all of it. First thing was to get opening a file working, this was as simple as checking if the file was in the sandbox and give it to the user, if the file was not in the sandbox then I would return an error stating that the user did not have permissions to access the file.

    Another thing to consider was the directories that the user has access to, specifically giving those directories names. After a long meeting with the other App porting developer, we had come up with a few names for Gateware to use that are not platform-dependent like the names used by Apple and Microsoft. Here are some examples of directory name changes:

Library -- UserFolder

tmp -- TempFolder 

Library/Caches -- CacheFolder

    Overall the GFile port has been going smoothly, I hope to complete this by Wednesday. 

Porting GWindow to UWP

    Porting GWindow seemed like the obvious next step to take after porting GFile. GWindow is very important to Gateware and is essential for the library. Getting it to work on UWP and have parity with the rest of the Gateware platforms is crucial. This will be a short update as the GWindow port is still ongoing with many other things that will need to be done, such as the unit tests.

The Problem:

    I knew going in that GWindow was going to be a handful, what I didn't expect was for most of the library to be deemed unsupported by the UWP platform. Most of the imported stuff like WndProc is inaccessible on UWP because of the sandboxing that happens within CoreWindow, that makes functions such as ProcessWindowEvents useless. This left me with the question of "What is GWindow going to be used for in UWP?". 

The Solution:

    I beat my head against a metaphorical brick wall for about 3 days trying to figure this out before I just decided to only do the functions that made sense for UWP. The purpose of each implementation of the GWindow interface is to provide functionality for that platform, so that is what I did. Only a few functions made the cut, such as GetClientWidth, GetClientHeight, and GetWindowHandle.

    Figuring that out was the easy part, it was the revelation that to get those functions working, I would need access to the CoreWindow. This is made difficult by the fact it needs to be accessed from the main UI thread. This was solved by these two functions:

    Calling the second function, allows you to just pass a lambda expression that contains what you would like to happen on the UI thread. This is where I would get access to the CoreWindow and do the actions that I needed. 

To Be Done:

    Currently, the GetWindowHandle function is still not working. That is down to the return value that the function has, 'void*'. As of right now, I have not successfully gotten CoreWindow to cast to it correctly.

Rectifying XCode thinking an iOS project is a 32-Bit MacOS project

     I was planning on porting GFile to iOS, but my plans were derailed by XCode. Last week I had been trying to get XCode to recognize two separate targets in one project. I had achieved that task, but I had one other issue. XCode wanted to compile the iOS target as a 32-bit Mac executable. As such the symbols for iOS were not recognized and the build failed. And so I was stuck once again. The Unit Tests for Gateware's pure C++ code, which should run on anything that can compile C++11, was not even compiling due to this error. 

    Most of the compilation errors were from mismatching architectures. As such I resolved to fix this by creating a new iOS project via XCode and looking at what build settings were enabled by default, and matching those settings on the Gateware iOS target. After a lot of fiddling with the settings I had gotten the compilation errors down to just 6 parsing errors. These errors were just from the iOS specific classes. I had to change one final setting to resolve this error, the "hide Clang debug symbols" setting. 

    Once all of the settings were in order, I had one final task to complete to get the unit test app running in simulator. The "info.plist" file tells the iDevice what information to give to the installed app. I had CMake setup the app bundle name and version through a .plist template that CMake provides. After setting these two entries were filled, the app ran on simulator, and the unit tests for the Gateware Core and Math libraries ran and passed. 

    I hope to get more libraries ported over to iOS and this step in the process will help immensely with development and testing of the libraries.