Month 2-Week 4 of Gateware: GRasterSurface

    This week has pretty much been dedicated entirely to creating the UWP implementation for GRasterSurface. When I first began looking into implementing GRasterSurface, I said in my daily standup for that day, I said: "it's looking like it's either going to be somewhat fairly smooth, or it is going to be very difficult and require rewriting somewhere between a lot and all of the code." And after a week of working on it, it's turning out to be the latter option.

    When I initially started working on GRasterSurface, everything seemed to work except for the SetDIBitsToDevice(), which is kind of an important function since it's what actually does the drawing to the screen. But UWP doesn't have access to this function. So simple enough, just find whatever the Win/RT version of this function is and just use that, right? Wrong! WinRT doesn't really have a direct comparison to this function. Instead, you have to go through a whole process of creating an Image in Xaml and then linking it back to a Bitmap in C++. But with how Gateware is set up, we can't really have any Xaml files, so we aren't able to create the image.

    After that, we tried using DirectX12 with the use of SpriteBatches, but, as it turns out, that isn't available in UWP, so we weren't able to go down that road. I then began looking into just creating two triangles and then drawing a 2D texture onto them to simulate drawing a 2D image. But after going down this road a bit, it became apparent that trying to do this in DirectX12 is a bit overkill for just drawing a 2D image and we had also discovered a new library called Direct2D, which is an abstraction of DirectX11, and it looked very promising.

    Until we actually started to implement it. In the original examples we looked into, you didn't have to create a DirectX11 device; all you had to do was create a D2D factory, device, render target, and bitmap, link them together, send the data to the bitmap, and then use the bitmap to draw to the render target. But after plugging the code into GRasterSurface and molding it to our needs, a few problems arose. In the example, they needed an HWND to be able to create the render target, as shown in the example, but being in UWP, we don't have an HWND object. So now, to set up the whole Direct2D pipeline, we now have to set up a DirectX11 instance as well with everything that comes with that, and then take a whole other route to be able to create the bitmap. But as I began diving deeper and deeper into that rabbit hole, every time I tried to create an object, I had to create 2 or 3 more to be able to create the original one. But even to be able to create those 2 or three other ones, I had to create 2 or 3  more objects for each of those. It was like a dependencies hydra where every time I tried to create one object, I would have to create two more in its place! And I tried to just work my way down this chain, but it eventually just became too much to keep track of, and also, a new problem with Direct2D became apparent.

    Every time we wanted to update the D2D bitmap, we would have to call a CopyFromMemory() function. Which isn't too bad on its own, but we're already doing two or three copies further up the chain and adding another one could really lead to the system getting bogged down. But if we were to use a DirectX11 Texture2D, we would be able to eliminate the extra copy and just directly stream the data to the GPU. So now, after being frustrated with all these "fancy" libraries not really working and wasting my time, I'm just going to go back to the roots of it all and just draw two triangles in DirectX11 and then texture them with a Texture2D.  I haven't yet been able to start as of writing this post, but I should have it done by my next post in a week. Hopefully I'll have better news then.

Month 2-Week 3 of Gateware: Win32 GInput/GBufferedInput bug fixes and UWP GBufferedInput

    This week has primarily been focused on fixing a hidden bug in both Win32 GInput and GBufferedInput and developing the UWP implementation for GBufferedInput. 

    To understand the fix for the Win32 GInput bug, we first need to understand how the original GInput worked. So, when you create a Win32 application, you are able to create your own Windows Procedure (or WinProc for short) that allows you to access the events of a window in a specified manner. In GInput, we have to capture the input events so that we can relay that information back to the user, and so we do this by creating a WinProc. However, the user that is utilizing Gateware is also able to create a WinProc of their own, and only one WinProc can be "set" at a time. So what we do is we first grab a reference to the original WinProc set by the user, set our own WinProc as the "active" one, and then we have it set up so that at the end of our WinProc, it calls the original and passes along all the original information. Also, since we can have multiple GInputs, so the reference to the original WinProc (along with all the other variables in GINPUT_GLOBAL) is static so that they can be shared across different GInputs. While this all may seem fine and dandy on paper, it in fact, seems to have never been tested and, indeed, does not work.

    When the first GInput is created, the currently active WinProc, WinProcOG, in this case, gets stored in the static reference, and WinProc1 gets set as the active WinProc. Now when the second GInput is created, the currently active WinProc, now WinProc1, will get stored into the static reference, and WinProc2 becomes active. So, WinProc2 is called first, and then WinProc2 calls the WinProc in the static reference, which is WinProc1. The problem here is that our GWinProc function is static, so, therefore, WinProc1 and WinProc2 are exactly the same. So when WinProc2 calls on WinProc1, it's actually calling on itself and is creating an infinite recursion loop which creates a stack overflow.

    After trying out a few different solutions, Lari and I were finally able to land on one that worked. Instead of having the GINPUT_GLOBAL be static, we created a static map that had the GLOBAL (which holds the reference for the WinProc) as the value and the window as the key. This way, each window had its own corresponding WinProc and input values. We then made it so that when a GInput is created, it checks the map to see if the window it is being created on already has an entry in the map. If it does not have an entry, we make one and then go along with the creation process as normal. If one already exists, however, all we do is grab a reference to the entry and then increment the number of GInputs that are created for that window. And then, when a GInput is deleted, we just decrement the counter until it reaches zero, at which point we go ahead and reactive the WinProcOG and remove the entry for that window.

    The day after we got this fix implemented into GInput, we realized that GBufferedInput has the same problem, but luckily since it's the same problem, it takes the same fix, and we were able to get that sorted out in one day.

    Next up was implementing GBufferedInput for UWP. At first, it was a little confusing how GBufferedInput actually worked, but after Lari explained it to me, it turns out that GBufferedInput is actually more similar to how UWP handles input than GInput. While GInput is all about getting the input, storing all the data, and then giving it to the user when asked, GBufferedInput pretty much takes the input events, does a little bit of processing on them to simplify things, and then forwards it on to the user to do whatever they want with it. This second method is very similar to how UWP input works. In UWP, you are able to create create a custom function and then tie it to whatever event you want. Once it's tied, UWP just takes the event arguments and passes them onto your function so you can do whatever you want with them. 

    And with that, GInput is mostly done for UWP. There are still a few minor things that need to be hammered out, but it shouldn't be anything major. Next up will be doing GRasterSurface and then figuring out why GBlitter was disabled. Once that's all done, UWP should be just about ready to go and be merged into the main branch. We'll still have to figure out how to get the runners to recognize its folder structure so that the pipeline can actually pass.

Month 2-Week 2 of Gateware: UWP GInput

    This entire week has been working entirely on UWP GInput, so this week's blog post is going to pretty much just be taking a deep dive into the process of getting it working and how it works.

    I first studied what the Win32 implementation for GInput was doing and how it worked. It would register the mouse and keyboard as raw input devices and then use the custom-made function GWinProc to be called an event every time an event was triggered. These events themselves don't really have any information other than telling us that an Input event happened. We then have to access the data of the raw input devices to actually get the relevant data that we need to figure out what actually happened and then store that data so that it can be called on later.

    Looking at other UWP implementations, I kind of expected UWP to just be very similar to Win32 with just a few changes, but apparently, that's only true in a few cases. I originally tried to just copy over the Win32 implementation and then make some slight adjustments, but after way too much back and forth with it, I decided to start from the ground up and mostly ignore the overall structure of the Win32 implementation moving forward.

    The UWP system has three major differences from the Win32 system. Firstly, there is no raw input or raw input devices to pull any information; secondly, UWP has a much more expansive event system where each type of action has its own event; and thirdly, the events actually hold all the information that we need in their parameters. The hard part was then trying to figure out how to actually use this event system. Every resource that I found on the subject seems to handle the event system in a different way, and I couldn't get any of them to work. It seems to be a threading issue where the event had to be called on the main UI thread, but I couldn't figure out any way to access that main thread. But then, I finally found the UWP implementation of GWindow. The GWindow events are actually quite similar to the GInput events that I need to use. So, I was able to copy over the general structure of the GWindow Create() function and use it in my GInput Create(). I also copied over the function CallOnMainViewUiThreadAsync(). This function played a large role in getting everything up and running. Essentially, you can just feed it a block of code, and it will run it like it's running on the main thread. I was then able to connect my specially-made functions to the Input events so that they would be called every time one of those events was called.

    So the overall structure of UWP GInput is that on Create(), it binds all of the private functions to the UWP input system events. Then whenever one of the events is triggered, the respective private function is called, and the event arguments are passed to it. The function then processes the data and stores it in GInputGlobal(), which essentially holds the current state of the mouse and keyboard. The user can then call on any of the available Get____() functions, which accesses the data from GInputGlobal() and returns it to the user. Then, in the destructor of GInput, all the functions are decoupled from the events.

    One problem that I haven't been able to figure out yet is finding a way to simulate input events with UWP. I plan on looking into it more, but at the time being, it is looking to be quite impossible. This may just have to be a task for future Gateware developers.

Month 2-Week 1 of Gateware: Finally Moving on to UWP

    Mac is finally behind me (for now), and UWP and Windows lie ahead! The first task that stood before me was bringing the UWP branch that Chase Richards and Lam Truong had worked on and merging the main branch into it to begin the updating process. This, of course, led to several conflicts that needed to be sorted out. Luckily, TotoiseGit has a really nice GUI that's good for sorting out conflicts and letting you edit the files outside of the conflicting areas as well. So after getting all of those conflicts sorted out, the Win32 version, of course, still wouldn't compile. But it turned out that all that was needed was changing some file pathing since UWP required some different folder structuring. And within the first day and a half, I had the Desktop version of Gateware compiling and passing all of the tests.

    But the Win32 version is only half the battle; now it was on to the UWP. And surprisingly, it wasn't that bad. The only thing that really gave issues was the UWP implementation of GFile, but to fix that, I really just had to copy-paste some code from the Win32 implementation of GFile. So I was actually able to get the UWP compiling and passing all the tests in the same day I was able to finish the Win32 implementation. Man, only two days in, and I'm flying through this! This UWP stuff is gonna be a breeze!

    That's what I thought at the end of the second day; now, here I am at the end of day 5, and, on the surface, nothing has happened since then. First, allow me to set the stage as to what is halting progress.

    As I mentioned before, UWP requires some very different folder structuring since it needs one solution for Win32 and another for the UWP. Now for a human being with an ever-changing brain, this is no problem. But for a machine that is running off of a specific set of commands, looking for a specific set of folders and files, this can be a huge issue. And it just so happens that the runners that we use to test the different implementations of Gateware are that kind of machine. So, until we can find a way to get those runners to recognize this different folder structure, the UWP branch cannot pass the pipeline tests and, therefore, cannot be merged into the main branch.

    But this is only the first problem. The second problem has to do with the Windows SDK versions (just as a heads up, I am in Windows 11 throughout this entire process). So to develop a UWP application, you need to download the UWP development tool through Visual Studio. When you install the UWP tool, it also installs Windows 11 SDK 10.0.22000.0, which is an SDK for Windows 11. If you try to delete this SDK, it will also delete the UWP tool, no matter what. So there is no way to have the UWP tool, without this version of the Windows SDK. Also, an additional piece of information to keep in mind; you cannot use a Windows 11 SDK while running Windows 10.

    Well this is all cool and all, but what does this have to do with Gateware? Well, when CMake is run, it has to create a UWP project, and a UWP project requires a Windows SDK version. But CMake can't just give it a random version, so it finds the latest version that you have installed and uses that in the project settings. And now, you suddenly have a UWP project that is targeting Windows 11, even if you're on Windows 10. Now, if you're a human being with an ever-changing brain, you can simply just go into the project properties and change the platform version to a Windows 10 SDK that you have installed, granted that would get very annoying since you'd have to do that every time you ran CMake. But, once again, our runners are not humans being with an ever-changing brain and can't just change the platform version as they wish.

    So Lari and I began trying to find a way to get CMake to find the Windows SDK version you had installed that was both older than the current OS version you were running and the newest out of that whole bunch. But after two whole days, we could not find a strong and reliable way to do it without creating more work for the user. So we came to the final solution. Just use Windows 11. While it will still be possible to use Gateware UWP on Windows 10, there will be hoops that will be needed to be jumped through, so it will be recommended to just use Windows 11. As I am typing this right now, I am using a freshly installed Windows 11, and we are currently looking into updating the Windows runner to Windows 11.