Month 3-Week 4 of Gateware: Creating a Showcase Project for Future Students.

     This week's blog post is probably not going to be quite as long as the past couple of weeks, as not quite as much has happened in the past week. This upcoming Monday, the new capstone tracks are going to begin rolling out, and with them, we will be trying to entice more students to make their own game engine using Gateware. And a big selling point we want to provide for using Gateware is that it is compatible with UWP, meaning it can run on various Microsoft devices, including newer generations of Xbox. So Lari And Carlos are wanting me to have an example project made and running on Xbox by the big presentation on Monday.

    I began working on a showcase project last Friday a bit, but so far, I have had to completely restart two or three times so far for various. One of the first restarts happened because I was using an assignment from a previous class and building off of that and adding extra functionality to show off more of the capabilities of Gateware, but I later learned that they don't want me to use previous assignments in case future students somehow find the template and use it complete their projects. After that, I started using examples already provided to students and building off of them. The last restart, however, happened because I learned a little too late that the example uses external libraries that aren't compatible with UWP, so we had to scrap that one.

    But now we're on to the most recent showcase project, and so far, it is looking far more promising than the previous attempts. I was able to create a CMake to build the project as a UWP project, and just today, I was finally able to get it running on an Xbox, with some slight problems, though. Currently, the project is just a cube floating in space that you are able to spin around with some controls, but I'm hoping to at least add some file I/O and some music playing to show off those capabilities. The problem I mentioned earlier is that when the application runs on Xbox, the image gets extremely blown up, so it's like the camera is super zoomed in. I haven't had much time to look into it yet, so maybe it's not really that big of an issue.

    In more exciting news, we were able to manually compile the UWP version of Gateware into its single header form! It took some time to get working and some finagling with GCompiler to get it working, but now it is. Also, because of some of the said finagling, GCompiler will hopefully now be more accepting of the UWP version of Gateware when the time for automatic compiling comes along, but we're still a week or two away from that. 

    Well's pretty much it for this week. Not quite as exciting as the past couple of weeks, but we've still got some progress being made, and we're still moving forward. Hopefully, next week's post will have a bit more excitement to it as we enter into the final month of my capstone with Gateware. Hopefully, I'll be able to get this UWP thing fully integrated into the main branch before I check out. Fingers crossed. 

Month 3-Week 3 of Gateware: Getting Close to Gateware UWP's Launch!

     Well, the storm I talked about last week has been averted; for the moment, at least. Since UWP applications can run on a wide array of devices, it has many limitations on what it can and cannot do for security reasons. One of these limitations is that it cannot access any files outside its own install folder. This is quite a big contrast to traditional desktop applications that can access files just about anywhere on the device it is on. This creates a huge discrepancy between the capabilities of GFile on Linux, MacOS, and Win32 and of GFile on UWP. We could heavily limit the capabilities of GFile so that it's consistent across all platforms; we don't want to limit users that are developing games and applications for desktop use only. So we've just decided to leave it as is for now, but we have plans to revisit this in the future to allow the user to put GFile in a sort of mode that will make it more consistent across all platforms.

    But the original problem that brought this to our attention still remains. A few of our unit tests, such as GFile, GAudio, and GBlitter, use external resource files to conduct their unit tests, which of course, exist outside of the install folder since the install folder isn't created until the application is built and runs the first time. So we have to find some way to get the files from the external resource folder to our local resource folder inside of the install folder. Luckily, Chase Richards has already done this for the files in GFile and GAudio, and I just needed to copy what he did for those and then do it for the GBlitter files. 

First, we grab all the files we want to copy and store them into a Global Resource. For GBlitter, we are copying both the png and tga files.

We then want to copy those Global Resources into stand CMake variables.

Next, we want to group these files altogether into one variable (we'll come back to this later).

Next, we want to have the application copy these files into the deployment location (install folder).

Finally, we'll take the grouped variable from earlier and add those to the executable and source group.

    While this does get the job done, it can be very annoying to do this every time you want to add something new to the resource folder. So I began looking into a way to copy over the entire resource folder in one go, and I got about halfway there. I managed to find a way to copy an entire directory with everything inside of it and paste it wherever I wanted, except for the local install folder. Since the install folder is created when the application runs, not when it is built, CMake doesn't have direct access to the install folder. That's why we had to use the "set_property" function above. But I couldn't find a way to use that function along with the "add_custom_target" function that I'm using to copy the whole directory. So, for now, I will just be leaving it in the CMake file and let someone who knows more about CMake than me come along and finish what I started.

    After this all got sorted out, the UWP implementation was finally passing all of the unit tests. Well, except for GWindow. So we'll go and tackle that real quick. While it took a bit to figure out exactly what the problem with the GWindow unit tests was, we did eventually find it. The problem is that since the functionality of GWindow is heavily limited to mobile applications (I know, big shocker), it isn't actually able to pass the vast majority of the unit tests made for GWindow. So someone in the past had created a separate group of GWindow unit tests that only get ran Gateware is being run in an application environment. While these tests only got compiled and ran on UWP, the standard unit tests were still getting compiled. So I was just able to separate the desktop unit test and the application unit tests so that they're only ran on the associated environment. Now, the UWP libraries are completed, and it's passing all of the Unit Tests! Now we need to go back and fix what is wrong with the other platforms. 

    Since this week's post is already so long and these last problems aren't really that major, I'm going to try and run through these fairly quickly. GAudio unit tests were failing on Linux because it is, for some reason, considered part of the APP Windows API Family and was running some code that is meant only for UWP, so I was just able to specify that the platform also needed to be a Windows platform and that fixed that problem. The MacOS CMake had some code that was specific to iOS, but since the iOS implementation is incomplete, it was causing some errors, so I commented it out (that way, when someone revisits iOS in the future, that code will still be there for them) and now MacOS was getting to the end of the build process. The final problem was the .yml file that tells the runners how to build and run the unit tests. On the UWP branch, a certain folder inside of all the platforms' build folders had its name changed, and we just needed to update the .yml file to look for the new file name. 

    And now, probably for the first time in about three years since its development began, the UWP branch finally has green check marks all across the board! 

    But the work is not over yet. I now need to begin working on a demo application that uses Gateware so that we can get it running on an Xbox console as a proof of concept for future capstone students that want to create a game for consoles. And then, after that, it will be working on the UWP build process so that it can finally be merged into the main branch after being on its own for about three years.

Month 3-Week 2 of Gateware: The End of GRasterSurface (Out of the Frying Pan and into the Fire)

    After nearly three weeks of working on it, GRasterSurface is finally complete! It's been a bumpy rider full of head scratches, reworks, and "hmmm"s. But it's finally over and ready to go. After finishing that, I could check to see if the universal implementation of GBlitter still works, and... of course, it doesn't. In fact, it has opened up a massive rabbit of possibly major flaws in multiple UWP libraries. But before we talk about the raging storm that lies ahead of us, let's take a moment to reflect on the success of GRasterSurface.

    After last week's blog post, I realized that there was a major problem with UpdateSurfaceSubset(). To fully understand this problem, I first need to explain what exactly we were doing before. Whenever the Resize event for the window gets called, we have to create a new texture to match the new screen width and height, and since creating a new texture can be very expensive, this is the only time we ever create a new texture. 

    So now we have our texture, but how do we change the texture? That's where Map() and Unmap() come in. The map function allows you to use a mapped subresource to pull information from the GPU, manipulate it, and then send it back to the GPU. A limitation of mobile platforms like UWP is that when you pull the information from the GPU, you have to immediately discard it. This makes it so that you can't keep any previous information that the texture did have.

    So anytime we want to update the texture (Clear, UpdateSurface, UpdateSurfaceSubset, and SmartUpdateSurface), we must use Map/Unmap with the discard. The problem that this created became apparent when we got to UpdateSurfaceSubset. How this function works is the user is able to specify a rectangle in the texture and then gives the function to update only that rectangle. However, because of the discard that happens with the Map, changes to the rest of the texture aren't kept. So when the UpdateSurfaceSubset is drawn, the changes rectangle is simply overlayed on top of the previously drawn frame, which kinda sucks. 

    So to counteract this, we need to have a local buffer that we update in all of the functions that write to the texture and then only update the texture on the GPU once we are ready to draw to the screen. On the one hand, this has made it so that we can't really do any hardware acceleration when it comes to the SmartUpdateSurface (though it shouldn't out weight all the time we save with the hardware-accelerated drawing), but on the other hand, we're able to largely just copy code from the Win32 functions that modify the texture and just make a few modifications to fit our needs since we don't need both a front and back buffer. 

    So that's pretty much it for GRasterSurface. So glad to finally have that behind me. But now it's time to face the storm that lies ahead. As much as I'm sure you're just dying to hear what the problem is, you'll have to wait a bit longer. The extents of this problem have yet to be fully explored, so I can't say for certain what is going on and how we plan to fix it cause we don't know. But I promise you'll hear all about it next week cause it is likely going to be all that I do next week. Until then, wish me luck!

Month 3-Week 1 of Gateware: GRasterSurface Cont.

     Another week of GRasterSurface has come and gone, but much more progress has been made toward finishing it. We've finally got it passing all of the tests* and drawing images to the screen. The only thing that should be left to do is modify SmartUpdateSurface() so that it can utilize hardware acceleration, and then we don't have to worry about unnecessary thread locking and unlocking.

    As I mentioned last week, I decided to just implement GRasterSurface using DirectX11 to avoid using weird external libraries, and it'll be a much more efficient implementation. I started by doing some research into how to set up the DX11 surface along with everything that goes along with it, like the shaders, buffers, descriptions, and the texture itself. The two main resources that proved to be helpful in getting all of this set up were the API examples from Full Sail's 3D Content Creation class and a Stack Overflow post. 

    The class examples have an example of a renderer that utilizes texturing, and this provided the main foundation for setting up everything. I had to modify the vertex shader a bit since we don't have to worry about matrix math and putting things in world space, but the pixel shader was able to work right out of the box. 

                      

    Along with the standard setup process, it also showed the process for sending a Texture2D to the GPU using a ShaderResourceView and SamplerState. This part was definitely much more of a mystery to me beforehand since I haven't done any sort of texturing in graphics before, let alone in DirectX11, which I have virtually no experience with. But the main problem with this example is how it is actually making its Texture2D. The example is creating textures from DDS files which requires a completely different method of creating the texture from what you would do to create a texture from an array of pixels.
    

    So this is where the second main resource comes into play. Since I couldn't use the DDS method that is used in the example, I did some research into how to create a Texture2D from an array of pixels, and I stumbled across the Stack Overflow post linked below. This showed how to set up the texture description, create a Texture2D using that, and create a ShaderResourceView using the texture. However, creating a Texture2D every time we have a new image to display can be very expensive, so we're only re-creating it when the window is resized. So how do we get new images to the Texture2D?

https://stackoverflow.com/questions/41627317/directx-11-how-to-create-a-very-simple-2d-texture

    This is where my third, secret resource comes from; people! Lari Norri was able to help me out with being able to Map and Unmap the texture's data so that we can update the texture in a very low-cost way. This was able to get a lot of the universal functions finished, like the lock/unlock functions and UpdateSurface and UpdateSurfaceSubset. I was also able to get some help from some other people at Full Sail with the occasional DirectX11 thing.

    So now GRasterSurface is just about finished, but as you may remember from the beginning of this post, SmartUpdateSurface() still needs to be done and that's also kind of the asterisk to all the unit tests passed. Since SmartUpdateSurface() isn't doing anything at the moment, GRasterSurface can't really pass the tests that require that function, so they're commented out at the moment. But I'm hoping to be able to get SmartUpdateSurface done by early next week so that I can stay on schedule with what I've got planned for this month. But only time will tell.

Month 2-Week 4 of Gateware: GRasterSurface

    This week has pretty much been dedicated entirely to creating the UWP implementation for GRasterSurface. When I first began looking into implementing GRasterSurface, I said in my daily standup for that day, I said: "it's looking like it's either going to be somewhat fairly smooth, or it is going to be very difficult and require rewriting somewhere between a lot and all of the code." And after a week of working on it, it's turning out to be the latter option.

    When I initially started working on GRasterSurface, everything seemed to work except for the SetDIBitsToDevice(), which is kind of an important function since it's what actually does the drawing to the screen. But UWP doesn't have access to this function. So simple enough, just find whatever the Win/RT version of this function is and just use that, right? Wrong! WinRT doesn't really have a direct comparison to this function. Instead, you have to go through a whole process of creating an Image in Xaml and then linking it back to a Bitmap in C++. But with how Gateware is set up, we can't really have any Xaml files, so we aren't able to create the image.

    After that, we tried using DirectX12 with the use of SpriteBatches, but, as it turns out, that isn't available in UWP, so we weren't able to go down that road. I then began looking into just creating two triangles and then drawing a 2D texture onto them to simulate drawing a 2D image. But after going down this road a bit, it became apparent that trying to do this in DirectX12 is a bit overkill for just drawing a 2D image and we had also discovered a new library called Direct2D, which is an abstraction of DirectX11, and it looked very promising.

    Until we actually started to implement it. In the original examples we looked into, you didn't have to create a DirectX11 device; all you had to do was create a D2D factory, device, render target, and bitmap, link them together, send the data to the bitmap, and then use the bitmap to draw to the render target. But after plugging the code into GRasterSurface and molding it to our needs, a few problems arose. In the example, they needed an HWND to be able to create the render target, as shown in the example, but being in UWP, we don't have an HWND object. So now, to set up the whole Direct2D pipeline, we now have to set up a DirectX11 instance as well with everything that comes with that, and then take a whole other route to be able to create the bitmap. But as I began diving deeper and deeper into that rabbit hole, every time I tried to create an object, I had to create 2 or 3 more to be able to create the original one. But even to be able to create those 2 or three other ones, I had to create 2 or 3  more objects for each of those. It was like a dependencies hydra where every time I tried to create one object, I would have to create two more in its place! And I tried to just work my way down this chain, but it eventually just became too much to keep track of, and also, a new problem with Direct2D became apparent.

    Every time we wanted to update the D2D bitmap, we would have to call a CopyFromMemory() function. Which isn't too bad on its own, but we're already doing two or three copies further up the chain and adding another one could really lead to the system getting bogged down. But if we were to use a DirectX11 Texture2D, we would be able to eliminate the extra copy and just directly stream the data to the GPU. So now, after being frustrated with all these "fancy" libraries not really working and wasting my time, I'm just going to go back to the roots of it all and just draw two triangles in DirectX11 and then texture them with a Texture2D.  I haven't yet been able to start as of writing this post, but I should have it done by my next post in a week. Hopefully I'll have better news then.

Month 2-Week 3 of Gateware: Win32 GInput/GBufferedInput bug fixes and UWP GBufferedInput

    This week has primarily been focused on fixing a hidden bug in both Win32 GInput and GBufferedInput and developing the UWP implementation for GBufferedInput. 

    To understand the fix for the Win32 GInput bug, we first need to understand how the original GInput worked. So, when you create a Win32 application, you are able to create your own Windows Procedure (or WinProc for short) that allows you to access the events of a window in a specified manner. In GInput, we have to capture the input events so that we can relay that information back to the user, and so we do this by creating a WinProc. However, the user that is utilizing Gateware is also able to create a WinProc of their own, and only one WinProc can be "set" at a time. So what we do is we first grab a reference to the original WinProc set by the user, set our own WinProc as the "active" one, and then we have it set up so that at the end of our WinProc, it calls the original and passes along all the original information. Also, since we can have multiple GInputs, so the reference to the original WinProc (along with all the other variables in GINPUT_GLOBAL) is static so that they can be shared across different GInputs. While this all may seem fine and dandy on paper, it in fact, seems to have never been tested and, indeed, does not work.

    When the first GInput is created, the currently active WinProc, WinProcOG, in this case, gets stored in the static reference, and WinProc1 gets set as the active WinProc. Now when the second GInput is created, the currently active WinProc, now WinProc1, will get stored into the static reference, and WinProc2 becomes active. So, WinProc2 is called first, and then WinProc2 calls the WinProc in the static reference, which is WinProc1. The problem here is that our GWinProc function is static, so, therefore, WinProc1 and WinProc2 are exactly the same. So when WinProc2 calls on WinProc1, it's actually calling on itself and is creating an infinite recursion loop which creates a stack overflow.

    After trying out a few different solutions, Lari and I were finally able to land on one that worked. Instead of having the GINPUT_GLOBAL be static, we created a static map that had the GLOBAL (which holds the reference for the WinProc) as the value and the window as the key. This way, each window had its own corresponding WinProc and input values. We then made it so that when a GInput is created, it checks the map to see if the window it is being created on already has an entry in the map. If it does not have an entry, we make one and then go along with the creation process as normal. If one already exists, however, all we do is grab a reference to the entry and then increment the number of GInputs that are created for that window. And then, when a GInput is deleted, we just decrement the counter until it reaches zero, at which point we go ahead and reactive the WinProcOG and remove the entry for that window.

    The day after we got this fix implemented into GInput, we realized that GBufferedInput has the same problem, but luckily since it's the same problem, it takes the same fix, and we were able to get that sorted out in one day.

    Next up was implementing GBufferedInput for UWP. At first, it was a little confusing how GBufferedInput actually worked, but after Lari explained it to me, it turns out that GBufferedInput is actually more similar to how UWP handles input than GInput. While GInput is all about getting the input, storing all the data, and then giving it to the user when asked, GBufferedInput pretty much takes the input events, does a little bit of processing on them to simplify things, and then forwards it on to the user to do whatever they want with it. This second method is very similar to how UWP input works. In UWP, you are able to create create a custom function and then tie it to whatever event you want. Once it's tied, UWP just takes the event arguments and passes them onto your function so you can do whatever you want with them. 

    And with that, GInput is mostly done for UWP. There are still a few minor things that need to be hammered out, but it shouldn't be anything major. Next up will be doing GRasterSurface and then figuring out why GBlitter was disabled. Once that's all done, UWP should be just about ready to go and be merged into the main branch. We'll still have to figure out how to get the runners to recognize its folder structure so that the pipeline can actually pass.

Month 2-Week 2 of Gateware: UWP GInput

    This entire week has been working entirely on UWP GInput, so this week's blog post is going to pretty much just be taking a deep dive into the process of getting it working and how it works.

    I first studied what the Win32 implementation for GInput was doing and how it worked. It would register the mouse and keyboard as raw input devices and then use the custom-made function GWinProc to be called an event every time an event was triggered. These events themselves don't really have any information other than telling us that an Input event happened. We then have to access the data of the raw input devices to actually get the relevant data that we need to figure out what actually happened and then store that data so that it can be called on later.

    Looking at other UWP implementations, I kind of expected UWP to just be very similar to Win32 with just a few changes, but apparently, that's only true in a few cases. I originally tried to just copy over the Win32 implementation and then make some slight adjustments, but after way too much back and forth with it, I decided to start from the ground up and mostly ignore the overall structure of the Win32 implementation moving forward.

    The UWP system has three major differences from the Win32 system. Firstly, there is no raw input or raw input devices to pull any information; secondly, UWP has a much more expansive event system where each type of action has its own event; and thirdly, the events actually hold all the information that we need in their parameters. The hard part was then trying to figure out how to actually use this event system. Every resource that I found on the subject seems to handle the event system in a different way, and I couldn't get any of them to work. It seems to be a threading issue where the event had to be called on the main UI thread, but I couldn't figure out any way to access that main thread. But then, I finally found the UWP implementation of GWindow. The GWindow events are actually quite similar to the GInput events that I need to use. So, I was able to copy over the general structure of the GWindow Create() function and use it in my GInput Create(). I also copied over the function CallOnMainViewUiThreadAsync(). This function played a large role in getting everything up and running. Essentially, you can just feed it a block of code, and it will run it like it's running on the main thread. I was then able to connect my specially-made functions to the Input events so that they would be called every time one of those events was called.

    So the overall structure of UWP GInput is that on Create(), it binds all of the private functions to the UWP input system events. Then whenever one of the events is triggered, the respective private function is called, and the event arguments are passed to it. The function then processes the data and stores it in GInputGlobal(), which essentially holds the current state of the mouse and keyboard. The user can then call on any of the available Get____() functions, which accesses the data from GInputGlobal() and returns it to the user. Then, in the destructor of GInput, all the functions are decoupled from the events.

    One problem that I haven't been able to figure out yet is finding a way to simulate input events with UWP. I plan on looking into it more, but at the time being, it is looking to be quite impossible. This may just have to be a task for future Gateware developers.