Month 1 - Week 4 - Adding MSAA to DX11

 So, last week was spent primarily focusing on getting the setup process for Windows more fine-tuned for users to contribute towards the project. This was a rather important task, and a lot of time was spent getting everything done correctly, but last week was also the week that it was finally completed and merged. With that, this week was spent working on my first real feature. This feature is MSAA support for DX11. To implement this, it needs the whole package, unit testing, research, and actual implementation code to get working. A lot of time was poured into this, and a lot of research into MSAA in general. While I could explain the full workload that was taken for this, I will instead explain two problems I faced with the implementation, and what I did to figure it out and get it working properly.

One of the first steps was adding the values for MSAA to the allowed mask variable, but I also needed the sample count for whatever was passed. This process was fairly straightforward and provided little issue in the actual implementation.

The second step was going through and altering the swap chain description and all texture descriptions to reflect the multisampling. This was also fairly straightforward as shown above. However, it was the code after this that proved to be more problematic.

Now, this image shows the solution, but to get the fix for this was a little troublesome. To explain what this is doing, I need to break down what I think is happening and what resulted in the code above. When I was initially implementing MSAA, I was running into a problem where the stencil view was failing to create, this was a constant error and had no obvious source other than the stencil view being null. Through the internet and discussing the problem with my mentor, I came to realize that the stencil view may not have a large enough buffer to hold all the pixels needed for the multisampling to work correctly. I looked around online and found a simple fix. By changing the view dimension from D3D11_DSV_DIMENSION_TEXTURE2D to D3D11_DSV_DIMENSION_TEXTURE2DMS, we can create a buffer that can store the appropriate amount of pixels. All we need to do is check if the sample is greater than one, one being no MSAA, if it is, then we set the view dimension of the stencil view to the proper setting so the multisampling works. After everything is implemented here, we now have MSAA.

This, of course, looks over the immense amount of work that was invested into the tests. There were almost 3K+ lines of code added just for the tests to ensure that MSAA was working properly, and there were many issues that came along the way with it.

A good example of the test code needing problems fixed is the one above. There was a problem I encountered with the tests I was doing when trying to do a test for MSAA x16. Although x16 MSAA has been around for a long time, modern GPUs will sometimes still lack the support for this level of MSAA, so what happens in the test is it will try creating an x16 surface for DX11, and immediately fail. A failed unit test is something we don't want, especially when it's a hardware limitation. So the way around this is to check for it. Normally, the REQUIRE statement would contain the create function call, and then we'd check the return immediately to see if it is a success, however, this has to be changed. What has to be done now is call to the surface, attempt to create it, and then check if the result is first a HARDWARE_UNAVAILABLE error. If this error is returned, then we can simply skip it because this isn't the fault of the test itself, but rather the hardware. Now, if the hardware is supported, this return from create will not occur, and we can REQUIRE the result is a success and nothing else. This problem took me a long time to handle. I had to research, redesign, and approach how to fix it. On a quick glance, the fix was probably easy to spot, but for me, just learning how everything works, I spent much longer on it than I would like to admit.

With that, however, everything else flowed smoothly and the implementation was rather easy, minus all of the writing time required for it. It was interesting learning all the internals of the library to get this working, and ensuring both the desktop version and the UWP version worked. All there is to do now is wait and see how the code review goes, and if any corrections are needed to be made.

Month 1 - Week 3 - Finishing Touches with Setup Process

 Last week I touched on the setup process and worked towards taking steps to make it better. To do this I made a bat script that ran a PowerShell script which installed Chocolatey, and then installed the NuGet package to work with CMake. This was done to avoid the issue of NuGet not being installed when the UWP CMake needed it for proper project generation. There were two catches with this approach. 1) To install Chocolatey, the user needs to be an administrator, and 2) the user has to be an administrator to even install the package as Chocolatey will force a prompt on the user if not. This prompt can be bypassed but locked on a thirty-second timer, even with arguments to attempt auto-accepting it.

Due to this issue in particular, I spent the early part of this week working on finding a solution to the problem. The reason for this being a problem is due to the runner for UWP and Windows compilation. Both runners for these builds can't perform input to bypass these prompts, so the prompt is either cut off from performing the build completely or has to wait thirty seconds for the prompt to auto-accept.

I don't have pictures for the iterations on the work put into this, but some of the iterations involved trying to bypass the prompts through PowerShell directly, and some involved completely remaking the scripts in an attempt to fix it, but eventually, I settled on a solution.

Chocolatey had to be replaced.

At first, this sounds like a large process. My requirements for a package manager are 1) it needs to allow no input, 2) it must allow running without an administrator, 3) must be able to install packages with no prompts, and 4) must be updated regularly so the version of the package isn't old. This search for the right package manager could've been long and difficult, but there is a package manager right now that is perfect for the task. It was a package manager I had been researching just a couple of days before encountering the issues I did with Chocolatey. This solution is WinGet.

As the image above explains, WinGet is a command line tool that is tightly integrated into Windows 10 and 11. This tool allows users to discover, install, upgrade, remove, and configure applications. This is the client that interfaces with the Windows Package Manager service. The big perk with this is it is not only already installed by Windows, but it doesn't prompt for administrator access unless the application requires it, which is much more preferable since NuGet doesn't need this to install. With this, I went to work on the code immediately.

The first step is updating the bat file. Originally, the file would open PowerShell, and then open another PowerShell within with the argument to open with a UAC prompt. This has been altered to now just open the PowerShell and execute the script without any extra work. This has the benefit of making the script look much more concise now.

The next part is to update the PowerShell script. The first thing needed is to test the `winget` command to see if it works. We can print the version number, and then check the exit code, this exit code will show whether or not it is. If it isn't, the user has to install the App Installer program from the Microsoft Store, which is side-loaded with `winget`. After this check is done, and if it passes, it will then use WinGet to install NuGet, using a couple arguments with it. The first argument `-e` is checking for packages with the exact name as requested, the second and third will auto-accept any package or source agreements, with the final forcing a direct run of the command and continuing with non-security related issues. This will ensure no prompts are given, and that the package will install smoothly for the user automatically. However, there is a final step.

WinGet, upon installation of the package, will require the shell to be completely restarted for the path environment variable to update. This is required for the NuGet process to be used and seen by the shell. But, the runner can't do this, and the user would need to run two scripts to make this efficient. To get around this, we can do some PowerShell magic. The image above shows us setting the environment variable `Path`, which is temporary for the instance to an updated path. We do this by accessing the system environment and getting the environment variable of the same name. This allows us, without restarting, to use NuGet freely throughout the setup.

With that, the runner is now able to do the setup script without any issues. This pull request was merged, and will hopefully make setting up on Windows easier, especially for UWP where this specific NuGet requirement was needed.

Next week, I will be focusing heavily on getting work done with another issue involving DirectX11 MSAA support and will be making posts for that as well.

Month 1 - Week 2 - Project Setup Updated

I've started working on updating the setup process for Windows users developing on Gateware. Due to recent UWP issues that we were experiencing the previous week, we found that NuGet was required for the project and wasn't being handled appropriately when it wasn't found. I was assigned to start working on updating the script used to set the project up for use. The previous version of the script was a single bat file that would create needed directories and then call CMake to generate the projects.

The code for it was simple, but what was needed for the change was to convert this bat file into a PowerShell script. Specifically, the new process needed to get Chocolatey, install NuGet with it, and then perform the same operations as the bat file.

There were a couple issues with this, however. The first was the execution of the script. The script is stored in a ps1 file, this file is the default for PowerShell scripts, and to run this script can be tricky. Traditionally, PowerShell wants the script to be signed to run without further scripting, however, the signing process is confusing and a little strenuous to my knowledge, so I looked at the other route. This other route is setting the execution policy of the PowerShell instance to bypass. Doing this allows for the script to be run without needing the script to be signed. this was the route I ended up taking for this. The second issue that I ran into was for Chocolatey installation. Chocolatey requires that PowerShell is run as administrator to install itself correctly. This doesn't sound like an issue at first, but let's take a step back and visualize how a user would set up the project on the old and theoretical new.

If I wanted to generate the project back then, I could just start the bat file, wait for CMake to generate the projects, and then I would be ready to work. That's if I have NuGet installed and configured in the path beforehand. Overall though, NuGet configuring would only need to be done once, and then the process from there is quick for each following setup on that system.

Now on the new system, I would have to open PowerShell as administrator, and then call the command `Set-ExecutionPolicy Bypass -Scope Process -Force` before finally calling our script. This isn't just something that has to be done once either. This process would have to be done every time the user needs to set up the project. Of course, we want this done easily, so this is where I started problem-solving.

Firstly, apologies for the text being hard to read, since commands in bat can't be broken into multiple lines, it had to be written rather long.

Now, this image is of the bat file replacement I wrote. What this bat file does is execute an instance of PowerShell that will execute another instance of PowerShell, except this second instance will run as administrator and be fed two commands. One command is to set the location to the active directory used by the bat file, and the second is to execute our script. Pretty straightforward, but there were problems with this.

You may have already noted that two PowerShell instances to run one as administrator has a bit of code smell, and you would be right. However, when I attempted to use the much more straightforward `runas` command that the bat script already has, it seemed to have trouble executing the PowerShell process with any arguments fed to it. This is problematic as I want the bat to handle everything for the user besides the UAC prompt to run as administrator. The only workaround I know of is this solution I made above. It's not the best by any means, but it gets the script executed.

Now, moving to the PowerShell script itself, we had to do a couple things immediately. The first is ensuring the script is being run as administrator. The code for this is a little lengthy, but surprisingly readable all things considered. After this check, we then check an absolute path to see if Chocolatey is installed. The reason I'm using an absolute path here is that Chocolatey will install here always. I don't think Chocolatey lets you modify where it installs, however, if it does, we can update this down the road to support other paths. For now, it's using the default location. If the directory where Chocolatey installs doesn't exist, we call on the installer and install Chocolatey for the user. After this installation, we immediately install the required package, NuGet.

From here, the rest of the code is simply a translation of the old bat script to PowerShell. A lot of the code stayed the same, except for the use of `errorlevel` which was switched out with `$LASTEXITCODE` since the previous was specific to bat scripting and doesn't exist in PowerShell. 

Overall, the process of implementing this new system took around 2-3 hours and was relatively error-free besides the issues I mentioned. I had to take a little bit of time to design and ensure the developer running the bat file would have to interact as little as possible so that the process could be mostly automatic. I feel that the result is good and works well, but there is still the code review that will be needed for this to pass, as well as testing on another system to see if it works on other systems that are not my own. Once these pass through, the new Windows setup script should be merged and a part of the project.

Month 1 - Week 1 - Introduction

As per FSO, my first blog post will be an introduction to myself, my journey, and my goals here. This will help others get to know more about me, the type of person I am, some of my personal achievements, and what I want to do during my time here.

So, my name is Alexander Cusaac. A little bit of history about myself. I started programming when I was just nine years old after finding an old book on Java my brother had--Java for Dummies if I remember. They had just started covering Java 8 at the time and I was hooked from there. From 9-13 years old I spent a long of time just writing small programs, making little IO programs, number guessing games, and messing with other languages like C#. It was around the time I was 14 I started understanding the more complex ideas of programming and started taking steps to really understand them. By the time I went into FSU (Full Sail University), I had made games, GUI software, extensions and mods, a simple OS, and more. Even made a compiler at one point just to understand and learn Rust--a rather unique and different programming language. As of right now, I've been programming for over 13 years, and plan to add many more years as I make programming my career.

I do have a GitHub if you want to look through any projects I have publically available. There's a decent bit to look through, but a lot of the projects are rather old and outside of my quality standards today.

To wrap this up, my goals. My goals with working on Gateware are to, hopefully, fix many issues that are currently on the backlog and generally work on bringing more stability to the project. I may work on adding new features to the project, but that won't be known until later in the following weeks/months. For now, I'm looking forward to fixing a lot of issues and getting the ball rolling on making the project more stable and sound for those using the library.

With that, this is my simple introduction. Happy to be aboard the project.

Month 4-Week 3 of Gateware: The Build Process is Finally Complete!

)    Well, we’re finally here. The GSource UWP branch has been merged into main and the GCompiler UWP branch will be merged into the main here in the next few days. Gateware’s UWP implementation is finally being released into the world, and while it may not be the prettiest, it still feels good. 

    Last week, we had an issue with the Unit Tests weren't being waited on by the yml, and the yml could tell whether or not Unit Tests. Well since then, the unit test yml has grown considerably. 

    That top block of code will first check to see if the application is already installed (from a previous test or elsewhere), and if it is, then uninstall the app. It then goes through the normal steps that we have from last time, except now, if we find a certain file (I'll talk about the importance of this file later), and if it exists, then delete it. Once we've then got the application launched and running, we then startup a while loop that will look for that file from earlier, and if it can find the file in time, then it will pass; if not, then it fails.

    Now, you may be asking, "What's up with this file?". Well, since UWP applications aren't able to write out to the console, we instead have it write out to a file. If all of the Unit Tests pass, then the file will be created; if any of the tests fail, then the file is not created. Simple as that. At the time being, the output file, unfortunately, does not have any real information.  We would be able to one day redirect what would normally go to the console to go to an output file, and then we could just have the yml print out the file. But that will just take longer to do than the amount of time I have left here.

    

    And with that, the GSource UWP branch was pretty much ready to get merged into main. We still had a little bit of tidying to do, but once that was taken care of, we had the pipeline run one more time and then merged the branches, and the merge passed all the tests!

    Not, it was GCompiler's turn. GCompiler's whole process went much faster than GSource since the vast majority of the changes we had to make we stuff we had already done in GScource. For the build step, all we did just copy over the build step from Win32 and then add a few lines so that it would run the CMake to build the UWP version of the SinglerHeaderTestMain project. And speaking of which, we needed an actual CMake to create the UWP version of the project, but because of how similar it is to the GSource test project, I was pretty much able to just copy it over, get rid of the Dummy project, and then change a few names. 

    The test step is exactly like the test step from GSource, just with some slightly different pathing and project name. And since the test step was also looking for a success file, we just needed to add it to the main of the test project to create the file once all the Unit Tests pass. And with that done, GCompiler is ready to be merged into main. The pull request has been submitted, and it passes all the tests, so it will be merged anytime now.

    And with that, Gateware-UWP is gonna be launched to the public and will be ready to be used at any time. There are still some improvements that can be made, but for now, it is working; just could be working a little nicer. But that's it! There's just a tiny bit more cleaning I'm going to do, but those shouldn't really have any functionality changes and will be really quick to do. So other than that, I'm pretty much done with Gateware as far as my time as a student goes. For my final week, I'm going to be trying to add some more functionality to my showcase project to hopefully have it be more than just a spinning cube with some music.

Month 4-Week 2 of Gateware: The Build Process is (Almost) Done

     We are so close to having the build process finished. It has presented a fair share of issues and has forced me to learn a lot of new things that I didn't think I would have to when I first began this journey. While we do have the vast majority of the build and unit test process finished for the GSource branch, we're still missing what is arguably the most important part.

    We got the whole build and process set up so that we now have to project being created, built, deployed, installed, and launched. 

    The meat of what's going on is happening here. The creation of the project in the build phase is pretty standard. compared to the platforms, but the main differences are here in the testing portion. Firstly, we are navigating into the build folder so that we have access to the solution file. We then use Push-Location to save the directory that we are currently in. We do this because the next line of code opens up the Visual Studio Developer Powershell, and doing so changes our working directory to some other random directory. But we are then able to use Pop-Location to get us back into the directory we had saved. We then use the devenv command to deploy the project (it also rebuilds the project), which will generate the AppX folder. After this point, we thought we would then have to take the AppX folder and compress it into a .appx package and then use the package to install the application onto the machine. But through some testing, I discovered that the deploy command actually goes ahead and installs the application onto your machine. And so then we were able to skip those two other steps, which I'm very happy about since that install step was giving me a lot of trouble. Finally, the bottom command then goes ahead and launches the application.

    The problem we are having is that after the application is launched, the yml just ends there. On the other platforms, the Unit Test is a console application, so when it launches the executable, it opens in the command line, and it can't continue until the unit test completes. It is also then able to see whether or not all the unit tests have passed. But UWP isn't able to run as a console application, so the yml doesn't wait for it to finish. So right now, the unit tests only pass and fail based on whether or not the application is able to be launched, not on the individual unit tests. 

    So now we have to find a way to not only get the yml to wait for the application to go through all the unit tests but also be able to read the unit tests and determine when and which unit test fails. The Start-Process has several commands that are actually perfect for what we are looking for it to do, but unfortunately, they all seem to be incompatible with UWP applications. We are so close to the end; I can taste it. But this is looking to be one of the more major issues and could potentially take some time to complete.

Month 4-Week 1 of Gateware: The Beginning of the End

    We are in the final stretch now, and it's all coming to an end. The showcase project for the presentation was completed in time for the presentation on Monday, though there still is a bit of a problem with it, and it definitely leaves more to be desired. We also began looking into setting up the automatic build process so the Windows runners and build and run the UWP implementation. At first, this really seemed to be a huge issue, and we were kind of at a standstill for a bit, but I believe we've made a bit of a breakthrough, and there should really only be one more hurdle to get over.

    For the showcase project, I ended up taking a DX11 texturing example that just had a spinning 2D plane, and I was able to turn it into a cube that you could spin around with either the keyboard or a controller. I was then able to add in some file I/O so that we could save the position of the cube when the application closes and then read it back when the application is opened. The last thing I added was that music now plays when the application opens. It may not be much, but the application does show off DX11 rendering, both controller and keyboard input, file I/O, audio, and it runs on Xbox.

    One of the lab instructors here also creating something to load in GLTF files which would be able to get us much more advanced 3D models and basic materials for the time being. We're hoping that it will also be able to give us more data in the future, like textures and normals. I want to try and implement this in the showcase project, but before I'm able to start working on that, I need to get this build process down. 

    Speaking of which, the build process in presenting a very interesting issue because, of course, UWP has to be that way. For all of the other platforms, we can simply just have the yml run the CMake the create the project, have CMake then build the project to create .exe files, and then we can just run the .exe files. But UWP can't be that simple because that would just be too easy. While this same process does make a .exe file for UWP, we can't actually use it since UWP just doesn't work that way. Instead, we have to do a couple extra steps to get it working. 

    It starts off the same by running the CMake file to create the project in the build folder and then having CMake build the project. But then, from here, we have to find a way to actually deploy the application so that it will create the AppX install folder. Once we have that install folder, we can then use it to create a .appx file, which can then be used to fully side-load the project so it can run separately from Visual Studio. I believe we currently have all of these steps figured out individually, except for the deployment step to actually get the AppX folder created. But once we have that figured out, we can begin to start fitting all the pieces together and get the Windows runner building and running UWP.

    As long as progress is able to keep moving like it has today and yesterday, I believe we can get this more or less wrapped up by the end of next week, and then I'll be able to go in-depth on how we're tackling every step of this build, install, and run process. But until then, we'll just have to keep our fingers crossed and hope that no more major roadblocks appear.

Month 3-Week 4 of Gateware: Creating a Showcase Project for Future Students.

     This week's blog post is probably not going to be quite as long as the past couple of weeks, as not quite as much has happened in the past week. This upcoming Monday, the new capstone tracks are going to begin rolling out, and with them, we will be trying to entice more students to make their own game engine using Gateware. And a big selling point we want to provide for using Gateware is that it is compatible with UWP, meaning it can run on various Microsoft devices, including newer generations of Xbox. So Lari And Carlos are wanting me to have an example project made and running on Xbox by the big presentation on Monday.

    I began working on a showcase project last Friday a bit, but so far, I have had to completely restart two or three times so far for various. One of the first restarts happened because I was using an assignment from a previous class and building off of that and adding extra functionality to show off more of the capabilities of Gateware, but I later learned that they don't want me to use previous assignments in case future students somehow find the template and use it complete their projects. After that, I started using examples already provided to students and building off of them. The last restart, however, happened because I learned a little too late that the example uses external libraries that aren't compatible with UWP, so we had to scrap that one.

    But now we're on to the most recent showcase project, and so far, it is looking far more promising than the previous attempts. I was able to create a CMake to build the project as a UWP project, and just today, I was finally able to get it running on an Xbox, with some slight problems, though. Currently, the project is just a cube floating in space that you are able to spin around with some controls, but I'm hoping to at least add some file I/O and some music playing to show off those capabilities. The problem I mentioned earlier is that when the application runs on Xbox, the image gets extremely blown up, so it's like the camera is super zoomed in. I haven't had much time to look into it yet, so maybe it's not really that big of an issue.

    In more exciting news, we were able to manually compile the UWP version of Gateware into its single header form! It took some time to get working and some finagling with GCompiler to get it working, but now it is. Also, because of some of the said finagling, GCompiler will hopefully now be more accepting of the UWP version of Gateware when the time for automatic compiling comes along, but we're still a week or two away from that. 

    Well's pretty much it for this week. Not quite as exciting as the past couple of weeks, but we've still got some progress being made, and we're still moving forward. Hopefully, next week's post will have a bit more excitement to it as we enter into the final month of my capstone with Gateware. Hopefully, I'll be able to get this UWP thing fully integrated into the main branch before I check out. Fingers crossed. 

Month 3-Week 3 of Gateware: Getting Close to Gateware UWP's Launch!

     Well, the storm I talked about last week has been averted; for the moment, at least. Since UWP applications can run on a wide array of devices, it has many limitations on what it can and cannot do for security reasons. One of these limitations is that it cannot access any files outside its own install folder. This is quite a big contrast to traditional desktop applications that can access files just about anywhere on the device it is on. This creates a huge discrepancy between the capabilities of GFile on Linux, MacOS, and Win32 and of GFile on UWP. We could heavily limit the capabilities of GFile so that it's consistent across all platforms; we don't want to limit users that are developing games and applications for desktop use only. So we've just decided to leave it as is for now, but we have plans to revisit this in the future to allow the user to put GFile in a sort of mode that will make it more consistent across all platforms.

    But the original problem that brought this to our attention still remains. A few of our unit tests, such as GFile, GAudio, and GBlitter, use external resource files to conduct their unit tests, which of course, exist outside of the install folder since the install folder isn't created until the application is built and runs the first time. So we have to find some way to get the files from the external resource folder to our local resource folder inside of the install folder. Luckily, Chase Richards has already done this for the files in GFile and GAudio, and I just needed to copy what he did for those and then do it for the GBlitter files. 

First, we grab all the files we want to copy and store them into a Global Resource. For GBlitter, we are copying both the png and tga files.

We then want to copy those Global Resources into stand CMake variables.

Next, we want to group these files altogether into one variable (we'll come back to this later).

Next, we want to have the application copy these files into the deployment location (install folder).

Finally, we'll take the grouped variable from earlier and add those to the executable and source group.

    While this does get the job done, it can be very annoying to do this every time you want to add something new to the resource folder. So I began looking into a way to copy over the entire resource folder in one go, and I got about halfway there. I managed to find a way to copy an entire directory with everything inside of it and paste it wherever I wanted, except for the local install folder. Since the install folder is created when the application runs, not when it is built, CMake doesn't have direct access to the install folder. That's why we had to use the "set_property" function above. But I couldn't find a way to use that function along with the "add_custom_target" function that I'm using to copy the whole directory. So, for now, I will just be leaving it in the CMake file and let someone who knows more about CMake than me come along and finish what I started.

    After this all got sorted out, the UWP implementation was finally passing all of the unit tests. Well, except for GWindow. So we'll go and tackle that real quick. While it took a bit to figure out exactly what the problem with the GWindow unit tests was, we did eventually find it. The problem is that since the functionality of GWindow is heavily limited to mobile applications (I know, big shocker), it isn't actually able to pass the vast majority of the unit tests made for GWindow. So someone in the past had created a separate group of GWindow unit tests that only get ran Gateware is being run in an application environment. While these tests only got compiled and ran on UWP, the standard unit tests were still getting compiled. So I was just able to separate the desktop unit test and the application unit tests so that they're only ran on the associated environment. Now, the UWP libraries are completed, and it's passing all of the Unit Tests! Now we need to go back and fix what is wrong with the other platforms. 

    Since this week's post is already so long and these last problems aren't really that major, I'm going to try and run through these fairly quickly. GAudio unit tests were failing on Linux because it is, for some reason, considered part of the APP Windows API Family and was running some code that is meant only for UWP, so I was just able to specify that the platform also needed to be a Windows platform and that fixed that problem. The MacOS CMake had some code that was specific to iOS, but since the iOS implementation is incomplete, it was causing some errors, so I commented it out (that way, when someone revisits iOS in the future, that code will still be there for them) and now MacOS was getting to the end of the build process. The final problem was the .yml file that tells the runners how to build and run the unit tests. On the UWP branch, a certain folder inside of all the platforms' build folders had its name changed, and we just needed to update the .yml file to look for the new file name. 

    And now, probably for the first time in about three years since its development began, the UWP branch finally has green check marks all across the board! 

    But the work is not over yet. I now need to begin working on a demo application that uses Gateware so that we can get it running on an Xbox console as a proof of concept for future capstone students that want to create a game for consoles. And then, after that, it will be working on the UWP build process so that it can finally be merged into the main branch after being on its own for about three years.

Month 3-Week 2 of Gateware: The End of GRasterSurface (Out of the Frying Pan and into the Fire)

    After nearly three weeks of working on it, GRasterSurface is finally complete! It's been a bumpy rider full of head scratches, reworks, and "hmmm"s. But it's finally over and ready to go. After finishing that, I could check to see if the universal implementation of GBlitter still works, and... of course, it doesn't. In fact, it has opened up a massive rabbit of possibly major flaws in multiple UWP libraries. But before we talk about the raging storm that lies ahead of us, let's take a moment to reflect on the success of GRasterSurface.

    After last week's blog post, I realized that there was a major problem with UpdateSurfaceSubset(). To fully understand this problem, I first need to explain what exactly we were doing before. Whenever the Resize event for the window gets called, we have to create a new texture to match the new screen width and height, and since creating a new texture can be very expensive, this is the only time we ever create a new texture. 

    So now we have our texture, but how do we change the texture? That's where Map() and Unmap() come in. The map function allows you to use a mapped subresource to pull information from the GPU, manipulate it, and then send it back to the GPU. A limitation of mobile platforms like UWP is that when you pull the information from the GPU, you have to immediately discard it. This makes it so that you can't keep any previous information that the texture did have.

    So anytime we want to update the texture (Clear, UpdateSurface, UpdateSurfaceSubset, and SmartUpdateSurface), we must use Map/Unmap with the discard. The problem that this created became apparent when we got to UpdateSurfaceSubset. How this function works is the user is able to specify a rectangle in the texture and then gives the function to update only that rectangle. However, because of the discard that happens with the Map, changes to the rest of the texture aren't kept. So when the UpdateSurfaceSubset is drawn, the changes rectangle is simply overlayed on top of the previously drawn frame, which kinda sucks. 

    So to counteract this, we need to have a local buffer that we update in all of the functions that write to the texture and then only update the texture on the GPU once we are ready to draw to the screen. On the one hand, this has made it so that we can't really do any hardware acceleration when it comes to the SmartUpdateSurface (though it shouldn't out weight all the time we save with the hardware-accelerated drawing), but on the other hand, we're able to largely just copy code from the Win32 functions that modify the texture and just make a few modifications to fit our needs since we don't need both a front and back buffer. 

    So that's pretty much it for GRasterSurface. So glad to finally have that behind me. But now it's time to face the storm that lies ahead. As much as I'm sure you're just dying to hear what the problem is, you'll have to wait a bit longer. The extents of this problem have yet to be fully explored, so I can't say for certain what is going on and how we plan to fix it cause we don't know. But I promise you'll hear all about it next week cause it is likely going to be all that I do next week. Until then, wish me luck!

Month 3-Week 1 of Gateware: GRasterSurface Cont.

     Another week of GRasterSurface has come and gone, but much more progress has been made toward finishing it. We've finally got it passing all of the tests* and drawing images to the screen. The only thing that should be left to do is modify SmartUpdateSurface() so that it can utilize hardware acceleration, and then we don't have to worry about unnecessary thread locking and unlocking.

    As I mentioned last week, I decided to just implement GRasterSurface using DirectX11 to avoid using weird external libraries, and it'll be a much more efficient implementation. I started by doing some research into how to set up the DX11 surface along with everything that goes along with it, like the shaders, buffers, descriptions, and the texture itself. The two main resources that proved to be helpful in getting all of this set up were the API examples from Full Sail's 3D Content Creation class and a Stack Overflow post. 

    The class examples have an example of a renderer that utilizes texturing, and this provided the main foundation for setting up everything. I had to modify the vertex shader a bit since we don't have to worry about matrix math and putting things in world space, but the pixel shader was able to work right out of the box. 

                      

    Along with the standard setup process, it also showed the process for sending a Texture2D to the GPU using a ShaderResourceView and SamplerState. This part was definitely much more of a mystery to me beforehand since I haven't done any sort of texturing in graphics before, let alone in DirectX11, which I have virtually no experience with. But the main problem with this example is how it is actually making its Texture2D. The example is creating textures from DDS files which requires a completely different method of creating the texture from what you would do to create a texture from an array of pixels.
    

    So this is where the second main resource comes into play. Since I couldn't use the DDS method that is used in the example, I did some research into how to create a Texture2D from an array of pixels, and I stumbled across the Stack Overflow post linked below. This showed how to set up the texture description, create a Texture2D using that, and create a ShaderResourceView using the texture. However, creating a Texture2D every time we have a new image to display can be very expensive, so we're only re-creating it when the window is resized. So how do we get new images to the Texture2D?

https://stackoverflow.com/questions/41627317/directx-11-how-to-create-a-very-simple-2d-texture

    This is where my third, secret resource comes from; people! Lari Norri was able to help me out with being able to Map and Unmap the texture's data so that we can update the texture in a very low-cost way. This was able to get a lot of the universal functions finished, like the lock/unlock functions and UpdateSurface and UpdateSurfaceSubset. I was also able to get some help from some other people at Full Sail with the occasional DirectX11 thing.

    So now GRasterSurface is just about finished, but as you may remember from the beginning of this post, SmartUpdateSurface() still needs to be done and that's also kind of the asterisk to all the unit tests passed. Since SmartUpdateSurface() isn't doing anything at the moment, GRasterSurface can't really pass the tests that require that function, so they're commented out at the moment. But I'm hoping to be able to get SmartUpdateSurface done by early next week so that I can stay on schedule with what I've got planned for this month. But only time will tell.

Month 2-Week 4 of Gateware: GRasterSurface

    This week has pretty much been dedicated entirely to creating the UWP implementation for GRasterSurface. When I first began looking into implementing GRasterSurface, I said in my daily standup for that day, I said: "it's looking like it's either going to be somewhat fairly smooth, or it is going to be very difficult and require rewriting somewhere between a lot and all of the code." And after a week of working on it, it's turning out to be the latter option.

    When I initially started working on GRasterSurface, everything seemed to work except for the SetDIBitsToDevice(), which is kind of an important function since it's what actually does the drawing to the screen. But UWP doesn't have access to this function. So simple enough, just find whatever the Win/RT version of this function is and just use that, right? Wrong! WinRT doesn't really have a direct comparison to this function. Instead, you have to go through a whole process of creating an Image in Xaml and then linking it back to a Bitmap in C++. But with how Gateware is set up, we can't really have any Xaml files, so we aren't able to create the image.

    After that, we tried using DirectX12 with the use of SpriteBatches, but, as it turns out, that isn't available in UWP, so we weren't able to go down that road. I then began looking into just creating two triangles and then drawing a 2D texture onto them to simulate drawing a 2D image. But after going down this road a bit, it became apparent that trying to do this in DirectX12 is a bit overkill for just drawing a 2D image and we had also discovered a new library called Direct2D, which is an abstraction of DirectX11, and it looked very promising.

    Until we actually started to implement it. In the original examples we looked into, you didn't have to create a DirectX11 device; all you had to do was create a D2D factory, device, render target, and bitmap, link them together, send the data to the bitmap, and then use the bitmap to draw to the render target. But after plugging the code into GRasterSurface and molding it to our needs, a few problems arose. In the example, they needed an HWND to be able to create the render target, as shown in the example, but being in UWP, we don't have an HWND object. So now, to set up the whole Direct2D pipeline, we now have to set up a DirectX11 instance as well with everything that comes with that, and then take a whole other route to be able to create the bitmap. But as I began diving deeper and deeper into that rabbit hole, every time I tried to create an object, I had to create 2 or 3 more to be able to create the original one. But even to be able to create those 2 or three other ones, I had to create 2 or 3  more objects for each of those. It was like a dependencies hydra where every time I tried to create one object, I would have to create two more in its place! And I tried to just work my way down this chain, but it eventually just became too much to keep track of, and also, a new problem with Direct2D became apparent.

    Every time we wanted to update the D2D bitmap, we would have to call a CopyFromMemory() function. Which isn't too bad on its own, but we're already doing two or three copies further up the chain and adding another one could really lead to the system getting bogged down. But if we were to use a DirectX11 Texture2D, we would be able to eliminate the extra copy and just directly stream the data to the GPU. So now, after being frustrated with all these "fancy" libraries not really working and wasting my time, I'm just going to go back to the roots of it all and just draw two triangles in DirectX11 and then texture them with a Texture2D.  I haven't yet been able to start as of writing this post, but I should have it done by my next post in a week. Hopefully I'll have better news then.

Month 2-Week 3 of Gateware: Win32 GInput/GBufferedInput bug fixes and UWP GBufferedInput

    This week has primarily been focused on fixing a hidden bug in both Win32 GInput and GBufferedInput and developing the UWP implementation for GBufferedInput. 

    To understand the fix for the Win32 GInput bug, we first need to understand how the original GInput worked. So, when you create a Win32 application, you are able to create your own Windows Procedure (or WinProc for short) that allows you to access the events of a window in a specified manner. In GInput, we have to capture the input events so that we can relay that information back to the user, and so we do this by creating a WinProc. However, the user that is utilizing Gateware is also able to create a WinProc of their own, and only one WinProc can be "set" at a time. So what we do is we first grab a reference to the original WinProc set by the user, set our own WinProc as the "active" one, and then we have it set up so that at the end of our WinProc, it calls the original and passes along all the original information. Also, since we can have multiple GInputs, so the reference to the original WinProc (along with all the other variables in GINPUT_GLOBAL) is static so that they can be shared across different GInputs. While this all may seem fine and dandy on paper, it in fact, seems to have never been tested and, indeed, does not work.

    When the first GInput is created, the currently active WinProc, WinProcOG, in this case, gets stored in the static reference, and WinProc1 gets set as the active WinProc. Now when the second GInput is created, the currently active WinProc, now WinProc1, will get stored into the static reference, and WinProc2 becomes active. So, WinProc2 is called first, and then WinProc2 calls the WinProc in the static reference, which is WinProc1. The problem here is that our GWinProc function is static, so, therefore, WinProc1 and WinProc2 are exactly the same. So when WinProc2 calls on WinProc1, it's actually calling on itself and is creating an infinite recursion loop which creates a stack overflow.

    After trying out a few different solutions, Lari and I were finally able to land on one that worked. Instead of having the GINPUT_GLOBAL be static, we created a static map that had the GLOBAL (which holds the reference for the WinProc) as the value and the window as the key. This way, each window had its own corresponding WinProc and input values. We then made it so that when a GInput is created, it checks the map to see if the window it is being created on already has an entry in the map. If it does not have an entry, we make one and then go along with the creation process as normal. If one already exists, however, all we do is grab a reference to the entry and then increment the number of GInputs that are created for that window. And then, when a GInput is deleted, we just decrement the counter until it reaches zero, at which point we go ahead and reactive the WinProcOG and remove the entry for that window.

    The day after we got this fix implemented into GInput, we realized that GBufferedInput has the same problem, but luckily since it's the same problem, it takes the same fix, and we were able to get that sorted out in one day.

    Next up was implementing GBufferedInput for UWP. At first, it was a little confusing how GBufferedInput actually worked, but after Lari explained it to me, it turns out that GBufferedInput is actually more similar to how UWP handles input than GInput. While GInput is all about getting the input, storing all the data, and then giving it to the user when asked, GBufferedInput pretty much takes the input events, does a little bit of processing on them to simplify things, and then forwards it on to the user to do whatever they want with it. This second method is very similar to how UWP input works. In UWP, you are able to create create a custom function and then tie it to whatever event you want. Once it's tied, UWP just takes the event arguments and passes them onto your function so you can do whatever you want with them. 

    And with that, GInput is mostly done for UWP. There are still a few minor things that need to be hammered out, but it shouldn't be anything major. Next up will be doing GRasterSurface and then figuring out why GBlitter was disabled. Once that's all done, UWP should be just about ready to go and be merged into the main branch. We'll still have to figure out how to get the runners to recognize its folder structure so that the pipeline can actually pass.

Month 2-Week 2 of Gateware: UWP GInput

    This entire week has been working entirely on UWP GInput, so this week's blog post is going to pretty much just be taking a deep dive into the process of getting it working and how it works.

    I first studied what the Win32 implementation for GInput was doing and how it worked. It would register the mouse and keyboard as raw input devices and then use the custom-made function GWinProc to be called an event every time an event was triggered. These events themselves don't really have any information other than telling us that an Input event happened. We then have to access the data of the raw input devices to actually get the relevant data that we need to figure out what actually happened and then store that data so that it can be called on later.

    Looking at other UWP implementations, I kind of expected UWP to just be very similar to Win32 with just a few changes, but apparently, that's only true in a few cases. I originally tried to just copy over the Win32 implementation and then make some slight adjustments, but after way too much back and forth with it, I decided to start from the ground up and mostly ignore the overall structure of the Win32 implementation moving forward.

    The UWP system has three major differences from the Win32 system. Firstly, there is no raw input or raw input devices to pull any information; secondly, UWP has a much more expansive event system where each type of action has its own event; and thirdly, the events actually hold all the information that we need in their parameters. The hard part was then trying to figure out how to actually use this event system. Every resource that I found on the subject seems to handle the event system in a different way, and I couldn't get any of them to work. It seems to be a threading issue where the event had to be called on the main UI thread, but I couldn't figure out any way to access that main thread. But then, I finally found the UWP implementation of GWindow. The GWindow events are actually quite similar to the GInput events that I need to use. So, I was able to copy over the general structure of the GWindow Create() function and use it in my GInput Create(). I also copied over the function CallOnMainViewUiThreadAsync(). This function played a large role in getting everything up and running. Essentially, you can just feed it a block of code, and it will run it like it's running on the main thread. I was then able to connect my specially-made functions to the Input events so that they would be called every time one of those events was called.

    So the overall structure of UWP GInput is that on Create(), it binds all of the private functions to the UWP input system events. Then whenever one of the events is triggered, the respective private function is called, and the event arguments are passed to it. The function then processes the data and stores it in GInputGlobal(), which essentially holds the current state of the mouse and keyboard. The user can then call on any of the available Get____() functions, which accesses the data from GInputGlobal() and returns it to the user. Then, in the destructor of GInput, all the functions are decoupled from the events.

    One problem that I haven't been able to figure out yet is finding a way to simulate input events with UWP. I plan on looking into it more, but at the time being, it is looking to be quite impossible. This may just have to be a task for future Gateware developers.

Month 2-Week 1 of Gateware: Finally Moving on to UWP

    Mac is finally behind me (for now), and UWP and Windows lie ahead! The first task that stood before me was bringing the UWP branch that Chase Richards and Lam Truong had worked on and merging the main branch into it to begin the updating process. This, of course, led to several conflicts that needed to be sorted out. Luckily, TotoiseGit has a really nice GUI that's good for sorting out conflicts and letting you edit the files outside of the conflicting areas as well. So after getting all of those conflicts sorted out, the Win32 version, of course, still wouldn't compile. But it turned out that all that was needed was changing some file pathing since UWP required some different folder structuring. And within the first day and a half, I had the Desktop version of Gateware compiling and passing all of the tests.

    But the Win32 version is only half the battle; now it was on to the UWP. And surprisingly, it wasn't that bad. The only thing that really gave issues was the UWP implementation of GFile, but to fix that, I really just had to copy-paste some code from the Win32 implementation of GFile. So I was actually able to get the UWP compiling and passing all the tests in the same day I was able to finish the Win32 implementation. Man, only two days in, and I'm flying through this! This UWP stuff is gonna be a breeze!

    That's what I thought at the end of the second day; now, here I am at the end of day 5, and, on the surface, nothing has happened since then. First, allow me to set the stage as to what is halting progress.

    As I mentioned before, UWP requires some very different folder structuring since it needs one solution for Win32 and another for the UWP. Now for a human being with an ever-changing brain, this is no problem. But for a machine that is running off of a specific set of commands, looking for a specific set of folders and files, this can be a huge issue. And it just so happens that the runners that we use to test the different implementations of Gateware are that kind of machine. So, until we can find a way to get those runners to recognize this different folder structure, the UWP branch cannot pass the pipeline tests and, therefore, cannot be merged into the main branch.

    But this is only the first problem. The second problem has to do with the Windows SDK versions (just as a heads up, I am in Windows 11 throughout this entire process). So to develop a UWP application, you need to download the UWP development tool through Visual Studio. When you install the UWP tool, it also installs Windows 11 SDK 10.0.22000.0, which is an SDK for Windows 11. If you try to delete this SDK, it will also delete the UWP tool, no matter what. So there is no way to have the UWP tool, without this version of the Windows SDK. Also, an additional piece of information to keep in mind; you cannot use a Windows 11 SDK while running Windows 10.

    Well this is all cool and all, but what does this have to do with Gateware? Well, when CMake is run, it has to create a UWP project, and a UWP project requires a Windows SDK version. But CMake can't just give it a random version, so it finds the latest version that you have installed and uses that in the project settings. And now, you suddenly have a UWP project that is targeting Windows 11, even if you're on Windows 10. Now, if you're a human being with an ever-changing brain, you can simply just go into the project properties and change the platform version to a Windows 10 SDK that you have installed, granted that would get very annoying since you'd have to do that every time you ran CMake. But, once again, our runners are not humans being with an ever-changing brain and can't just change the platform version as they wish.

    So Lari and I began trying to find a way to get CMake to find the Windows SDK version you had installed that was both older than the current OS version you were running and the newest out of that whole bunch. But after two whole days, we could not find a strong and reliable way to do it without creating more work for the user. So we came to the final solution. Just use Windows 11. While it will still be possible to use Gateware UWP on Windows 10, there will be hoops that will be needed to be jumped through, so it will be recommended to just use Windows 11. As I am typing this right now, I am using a freshly installed Windows 11, and we are currently looking into updating the Windows runner to Windows 11. 

Week 4 of Gateware: End of the First Month

     Well, here we are at the end of the month. Much learned, but still far more to learn. The problem we were having with the MacOS crashing last week was that the CMake code we were using to link the Vulkan SDK was outdated and since then, CMake has built-in support for linking with newer Vulkan SDK's, so we just needed to update that bit of code to get it working. So after that was done, it was finally compiling, but it was now crashing on the VulkanSurface tests.

    So in the newer Vukan SDK's for MacOS, they force the extension "VK_KHR_PORTABIITY_subset" to be enabled. This extension allows for applications to exist on non-conformant platforms and since MoltenVK isn't fully conformant, this extension needs to be enabled. And just for those who don't know (like me up until two weeks ago), MoltenVK is needed so that Vulkan can can map to Apple's Metal graphics framework. To opt-in to this extension, you need to modify the VkInstanceCreateInfo by adding the VK_INSTANCE_CREATE_ENUMERATE_PROTABILITY_BIT_KHR bit to the flag parameters and add the VK_KHR_PORTABILITY_ENUMERATE_EXTENSION_NAME extension to the ppEnabledExtensionNames array along with incrementing the enabledExtensionCount variable. Links and a screenshot of this information will be posted below.
Link: https://vulkan.lunarg.com/doc/sdk/1.3.243.0/mac/getting_started.html

While this is a very simple thing to implement in an ordinary application, Gateware is no ordinary application and took a bit more finessing. 

    GVulkanSurface uses a file called GVulkanSurface_core to implement the majority of the VulkanSurface implementation and thus makes the OS-specific files much smaller. But since our VkInstanceCreateInfo is in the core file, we can't just make all those changes there since Linux and Windows are comformant platforms and would cause another crashing error. So instead, after the original VkInstanceCreateInfo is created and setup, we have a new function called "PlatformOverrideInstanceCreate," which allows us to pass in the VkInstanceCreateInfo, along with some of its original variables to be cleaned up later, and then modify them all in the function to opt-in to the extension. We then of course had to include dummy versions of this function into the Linux and Windows version of VulkanSurface so that the function can still be called on other Operating Sytems.

    Since this extension only exists on newer versions of Vulkan, we aren't able to get Gateware to compile on older SDK's with the new Mac implementation. With this said, we will be dropping support for Vulkan SDK's on MacOS older than 1.3.216.

    Since we have been doing a lot of updating on the Mac side of things, we are also trying to update our Mac testing machines to the latest MacOS that they can support, which is Monterey. However, this has caused a previously discovered bug to pop up again. GWindow causes crashing on Monterey (and likely other newer MacOS's as well). While we aren't entirely certain what the problem is, we believe it has something to do with multi-threading. I wasn't really able to discover anything new with this issue, just tried to understand what was already found and made. I would like to return and try to fix this problem later on before I graduate, but we'll see when we get there. For now, this is the link to the issue in GitLab. https://gitlab.com/gateware-development/gsource/-/issues/229.

Week 3 of Gateware: MacOS

     For the past 5 days, I have been dealing with the donor MacBook. Originally, I was using it to fix a compiler issue involving GController unit tests, but in the process of doing so, I found another issue where GSource doesn't seem to compile using the latest Vulkan SDK version. After some research, we believe that the issue is that the OS on the donor MacBook is simply too outdated and we are hoping that updating the OS will solve the issue. So I just had to update the OS and the problem should be fixed. Simple, right? Of course not. Since the MacBook also had Linux and Windows operating systems on it, there wasn't enough space to install the update. So I had to factory reset the Mac and re-install the OS, but for whatever reason, it couldn't download the installer and install the OS. So I had to get access to the second donor MacBook, download a new MacOS (Monterey), transfer it to a USB drive, convert that USB drive to a bootable installer, plug it into the original MacBook, and then I can finally install a newer OS. And I am happy to say that it did successfully install. ☺

    Now with the new OS installed, I can start re-installing all the necessary software to compile and run GSource and everything was moving smoothly until I got to XCode. To use XCode, you need to sign into your Apple ID account, which we have. However, two-factor authentication has been set up for the account and we currently have no way around it. I'm hoping to soon find who set up the two-factor authentication so we can get rid of it, but if we can't find that person, things might get a little more complicated.

    It might be asking too much, but I'm hoping that once we've got this Vulkan compiler issues sorted out, I'll be able to figure out the other bug fixes fairly quickly so that I can get started on the UWP tasks and just have to worry about the Windows development environment. But as everything in the development world, that probably won't be the case.

Week 2 of Gateware: Research

    My big plan for Gateware or the next several months that I was going to be working on was sort of a two-parter. Firstly, I was going to complete the work of previous developers to get Gateware running as a UWP application so that it could run on additional Microsoft devices such as the Xbox, Windows Mixed Reality Headset, Hololens, and so on. Secondly, I wanted to convert the UWP code over to WinUI 3 to future-proof Gateware, as it seemed that Microsoft was dropping support for UWP and replacing it with WinUI 3. The problems arose with this second part, as the previous statement only seems to be partly true.

     From what I can tell, when people are talking about "UWP" in various articles and websites, they seem to be talking about two different things. The first thing they could be talking about is the UWP application framework. This is the overall structure of a project that makes a UWP app a UWP app. The second thing they could be talking about is UWP/XAML. This is a library/language used to create UI elements inside of a UWP application. Up until WinUI 3 came out, UWP/XAML was the only way to create UI elements in UWP apps, and that was really the only use that UWP/XAML had. 

    But now, with the introduction of WinUI 3, Microsoft seems to be trying to replace UWP/XAML (not the UWP application framework) with WinUI 3. But since people have always just used "UWP" to refer to both parts of the UWP ecosystem, this now seems to have created some confusion. Or at least it has really confused me. 

    The main thing to be learned from this is that WinUI 3 only brings about new libraries and API's for creating UI elements, which is something that Gateware does not have/need any time soon. Since Gateware uses graphics API's to draw to the screen, Gateware users can just create UI elements using those API's. Because of this, WinUI 3 has to use being in the current Gateware. I can see it possibly having some use cases in the future, as Gateware has more features added, but until then, there is not really anything to be done.

    Next Wednesday (April 19, 2023), Microsoft will be holding their monthly WinUI 3 Community Call where they has a QnA towards the end. I plan on attending this Community call to try and learn more about WinUI 3 and UWP, just to confirm that WinUI 3 holds no prospects for Gateware in present time.

    In the meantime, I will be working on various fixes that involves Linux and MacOS to not just get more Gateware experience, but to also get some experience in working with those other OS's. I do still plan on completing the UWP work to get Gateware running on Xbox, but that will be after I have confirmed that there is nothing to be done concerning WinUI 3.