Month 4 - Blog 7 - Gateware Development

Time is coming to an end with GNodeFactory and I have had an immense pleasure being a part of it. As I am writing this a fair few additions have been made to the project. In order to organize this into a succinct post, listing out each item will help:

GNodeFactoryF Allocation Strategy being updated (again..)

    The update to the allocation strategy this time went from having pools of data that could theoretically be endless to having a fixed maximum size pool. This would mean that when the user allocates nodes, they are either filling up existing pools, or if needed having more pools being made at a fixed size and having the handles returned. Handles still work with cross-pool relationships which was exciting to not have to go track and fix more bugs.

    Currently the node count per allocated pool is 4096 which should be modified as testing happens finding out the best "bang for buck" allocation size for the system. While users typically shouldn't need to allocate a "large" number of nodes (i.e. one million nodes), GNodeFactory should still be able to maintain performance at those large node counts, within reason.

FindSphereNodes being implemented, then multithreaded

    Lari and I discussed that at least one spatial query method would be implemented before my time with Gateware was up. Looking at the workload amongst my program requirements I initially wanted to leave spatial query methods to a future developer. Since Lari wouldn't consider GnodeFactory "completed" until at least one was implemented, FindSphereNodes is now here.

    Initially I had waited to do multithreading until the entire library was created, then I would go back and multithread where reasonable. In this case since we need to look down all the pools we can tell GConcurrent to run some work for each pool to find the node in parallel. The idea is that if multiple threads are working to find the node within their pool, the time spent overall would be less if it was linear. Functionality was tested using a separate solar systems simulation. Below is that simulation where I would press `1` to toggle FindSphereNodes within 100m of the camera.

ResolveHierarchy being multithreaded

    One of the largest features within GNodeFactory is the ability to resolve the hierarchies transforms. In order to do this, every pool will be tested to find out which nodes are dirty. Then based on those nodes, run work in parallel to recompute them. Since node roots don't overlap, every node has exactly one parent. This means you can follow a parent from anywhere resulting in a single root. Two roots can never share a node, letting each root get its own thread with no locking between them.

    The write lock is taken once at the top and the workers run under it without ever locking again, which also fixed an earlier version that deadlocked itself by re-locking on every level of the recursion. ResolveSubHierarchy stays single threaded since the handles a user passes in can overlap, and two threads resolving overlapping subtrees would trip over each other.

FindAll Child, Sibling, and Parent Nodes being multithreaded

    These three take a list of nodes to search from, so instead of splitting the work up by pool, I split it up by search. Each search is on its own anyway, so each one gets its own task and its own list to write results into.

    Once they all finish, those lists get copied into the real output array in order. That order matters because the output is one flat array with a count for each search, and a thread has no way to know where its results go until the searches before it are done. Copying them in afterwards means everything ends up exactly where it did before.

    That said, this one probably wasn't worth it. It only helps if you're searching from a lot of nodes at once, and most of the time you're only searching from one, so you get a single task and pay for the threading without getting anything back. It does keep the Find methods consistent with each other, at least.

Find Capsule, Frustrum, AABB, and OBB Nodes 

    Once FindSphereNodes was done, the rest were easy. Every node already has a sphere around it, which is just the collider radius sitting wherever the node resolved to. So all four of these are really asking the same thing, does this sphere touch that shape? GCollision already had those tests written, so each method ended up being the same scan as the sphere one with a single line swapped out in the middle.

    AABB needed its min and max corners turned into a center and extent first, since that's the form the test wants. The frustum is the odd one out, because it isn't one shape, it's six planes. A node is outside if its sphere sits fully behind any one of them, so it checks all six and stops at the first miss. That assumes the planes point inward, which is normal, but worth knowing if a frustum ever comes from somewhere unexpected.

As for everything else, I don't really have any major updates towards GNodeFactoryF. GNodeFactoryD however will be getting made soon since it will be mostly a copy and paste from the float implementation. Since that covers most of the past two weeks, I hope everyone who gets the chance to use GNodeFactory can create some awesome projects with it!

Month 3 - Blog 6 - Gateware Development

GNodeFactory... a factory to manage large amounts of nodes.

An initial task that was started throughout these previous weeks was to implement most of the Get and Find methods regarding GNodeFactory. Without these methods being implemented the developer or user will not be able to actually retrieve information in the system. Say they lost a child node's handle, without the methods getting the handle back easily would become a lot more difficult. That being said the following methods were implemented to help provide this functionality: 

• GetImmediateParentNodes
• GetImmediateChildNodes
• GetImmediateSiblingNodes
• GetFactoryInfo
• GetFactorySize
• FindAllParentNodes
• FindAllChildNodes
• FindAllSiblingNodes

This grouping of functionality was fairly straightforward with its approach. Simply access the pool, then access the state of the given node. Repeat for the number of requested nodes. This takes into account that all the data has been validated beforehand. Since swapping to using a GNodeHandle it makes determining the pool and node a bit easier since they are in a union as an unsigned long long. 

Finding the nodes however was a little bit more involved since it required using a DFS Traversal to gather the nodes. Since the user might not know the amount of parents or children that a node could have, the number of nodes gathered is also given as an out parameter. Traversal can get a bit cost heavy, to avoid this there is a thread_local vector maximized to the number of possible nodes we have. This way we will never run out of space to allocate nodes and there is always enough space to fit all existing nodes in it. For FindAllChildNodes, siblings are pushed onto the stack first, then children. Since it's a stack, children come out first when we popped, which helps with traversal since they are still in order.

Aside from the Finding and Getting, in a previous meeting Lari pointed out that AllocateNodes didn't properly allocate to the right pools. Embarrassing as that was, he suggested to use a GPool Structure which was similar. Instead of having 4 pools (states, metas, internals, and nodes) there would be a single factory of GPool. So instead of trying to manage both data oriented and parallelization. We shifted to 100% parallelization and attempting to maximize the efficiency of that. 

Allocation was adjusted to create a pool of an ideal size (currently 4096) and push nodes into those pools if there is room. If the user asks for a larger count than 4096 then a pool of that size is made. Otherwise if there is room found in an existing pool to use that, or make a new pool of ideal if none is found.

GHandle also was introduced to help identify nodes and pools properly. Instead of the node and pool being individual memebers, they are now a union as a "handle" which can be passed as an unsigned long long. This makes it so the user can store handles quicker and easier for the end user/developer.

The next milestone will be nearing the end of GNodeFactory's development time with me as I hope to implement the Resolving of the Hierarchy and a visual demonstration of it working!

Month 3 - Blog 5 - Gateware Development

 These last two weeks have been quite hectic to say the least with Gateware. As development continues with GNodeFactory (GNF) a couple of hurdles came along. First was how to actually store the data within GTL's Factory architecture since I was still getting the hang of it. GNF is supposed to be structured as a factory containing multiple root factories. These root factories will help manage threads and computations making operations faster than single threaded executions. I initially was not able to fully grasp how the implementation would look, but after some time with a single root factory I slowly was able to understand and created the below implementation so far. Previously there was going to be a pool for the State and Meta information in their own factories as well, but after discussing some storage and accessing viewpoints with Lari we decided it was optimal to keep the information within the Internal Node structure. This makes looking up the information faster since there is no need to perform additional access operations. 

This led into the next task for implementing allocate and deallocate for GNF. Whenever Allocate is called, it will attempt to recycle an existing pool identifier before creating a new one. With this design I choose to create a pool of root nodes wherever is available first. Deallocating performs the inverse by validating the data is available to remove then cleans up any associated data pools and recycling their identifiers. Since this task was done in the previous week portion for this blog post, I have now code reviewed and merge requested the Allocation/Deallocation of GNF with Lari.

Moving forwards to the bulk of this previous weeks work: Parenting and Unparenting. There were some challenges at first to Parenting and Unparenting, first was making sure the calculation was one with the proper inverse performed on the parent nodes rotation. One issue that I found while writing the unit tests was that if I wanted a duplicate index to unparent then the operation should technically fail since I shouldn't be able to unparent an already unparented node. Provided that most of the implementation still stays relatively the same, I believe Parenting and Unparenting are nearly complete as of this post being written.

This milestone also covered reading and writing to the nodes, matrices, and meta data while keeping reading of the states non-writeable. State information regarding the nodes would be purely manipulated internally since there should be no reason to modify a parent/child/sibling/depth/flags associated within the nodes. Unit tests to validate that these operations are performing as suspected are also being written since validation and unit tests are required for very part of Gateware.

Moving forward into the next two weeks I will be focusing more so on implementing the resolving of the hierarchy and ensuring that all data within GNF uses the flags properly. There will be many.. many more unit tests to cover with more implementations and samples near the end of that.

Month 2 - Blog 4 - Gateware Development

Over the past two weeks, I have worked on getting the new GNodeFactory library set up and having a basic skeleton structure with boilerplate implemented. An initial work item for adding the initial GNodeFactory boilerplate files was made. Within it, the Universal/Dummy Implementation, Class Header, and Unit Test files were all incorporated. Since most of the functionality within GNodeFactory is shared between the floating and double precision types they can exist within a shared implementation class. A few structures needed to be added within GMathDefines being the GNode, GNodeState, and GNodeMeta. Each of these structures will help manage the basic information required within the factory. GNode will contain the transformation/rotation/collision information. GNodeState would incorporate how to traverse the hierarchy with access to the parent, child, and siblings. Lastly, GNodeMeta data will be utilized for future libraries within Gateware were the data can be managed by downstream systems.

Since GNodeFactory is part of Gateware it also requires Unit Tests to be made using the Catch2 API as all previous libraries have. For the Unit Tests I wanted to make them clear as to what functionality I would be testing with descriptors of the precision type. Some prior libraries use the SECTION macro to differentiate tests, so I opted to also use that with distinguish precision.

Lastly, I have been working on a spike solution for GNodeFactory as I wait for Merge Requests and Code Reviews to take place. I have made some significant progress with the spike solution as I was able to get some users to test out the work in progress library and give some feedback. The spike solution is a significantly condensed version of GNodeFactory but aims to stick as true as it can to the goal of the library. Within the spike solution I am opting to use return types instead of pass by references/pointers which Gateware utilizes. This is for simplicity on what the library will do in the meantime. Below are some implementations of the spike solution:

As we get into the next couple of weeks I will aim to implement the spike solutions research into the GNodeFactory library itself. Currently awaiting code reviews and feedback from maintains, advisors, and instructors. 

Month 2 - Blog 3 - Gateware Development

Another two weeks have past and that means another blog update. This time with progress not only on the GSample repository but also the GNodeFactory implementation. Some of the samples I have worked on in the past two weeks were:

• GFile
• GLogic
• GBlitter
• GController
• GInterface

Most of these samples will help when it comes to unit testing for GNodeFactory since stress testing the library will be conducted over a period of a week. This will need to involve automated data analysis which GFile and GLogic can help assist with. GInterface will be utilized (Specifically GInterfaceInterface) within GNodeFactory to inherit functionality helping manage the libraries lifetime and tasks.

GBlitter, while unable to really assist with GNodeFactory itself, has reinforced the concept of descriptive code. That will help me make GNodeFactory as understandable as possible for users. The GBlitter sample features a slime sprite spinning around on some grass tiles. By using TGA files, GBlitter can import and have the tiles defined. 

I have also made progress with drafting more of the official proposal out. A section stating the allocation and management of nodes is still to be written but will be accomplished before the next milestone. I have started to implement the interface for GNodeFactory within the source. Since it is a math library it will utilize both Float and Double operation types. Also since this library is in progress it will be marked with the "In Development Flag".

Month 1 - Blog 2 - Gateware Development

 The past two weeks have been hectic to say the least. Being filled with development and preparing for my presentation, my nerves were definitely on the line. Since my ongoing task was to create samples for the new GSamples repository, I added some more tasks for samples expanding on the following Gateware modules:
- GMatrix
- GVector
- GQuaternion
- GDaemon
- GThreadShared
- GConcurrent

So far most of theses samples have been merged into the main GSamples repository which is very exciting. They didn't come with the fun hiccups of learning more about CMake, like with GAudio and GBufferedInput, but did offer more learning into how threading and concurrency work. 

Developing GDaemon's sample created an interesting point about the documentation provided in the Gateware Doxygen resource. My assumption was to make a daemon, then add event responders to the daemon. This ended up not being the intended functionality and needed to be swapped out for something more focused on GDaemon itself. Swapping to demonstrate the Fibonacci Sequence using a daemon seemed more suitable, so one daemon was used to run iterations of the sequence while another toggled the sequence daemon on and off. Both of these daemons had event responders to print out the status of the daemons, while also running at specific intervals.

The rest of the samples went smoothly, with little to no issues arising. This helped the fact that I had to work on my GNodeFactory Proposal Presentation. This was a requirement of the CSMS degree in which I had to get the project approved by the Committee members. The past week I worked with Lari to present my materials to the Gateware Maintainers, which was insightful. They provided a ton of feedback which I tried to incorporate as best I could into the upcoming presentation. After the meeting with Lari and the maintainers, I had my meeting with the Committee. They really liked the project and were interested in a ton of the unit testing and research ideology. Overall I think the past two weeks have been really fun, not only for my confidence, but for learning more with modern systems.

Month 1 - Blog 1 - Gateware Development

So being onboarded into Gateware was fun to say the least. My tasks for onboarding into Gateware were simple at first, recursively clone the repositories and verify they build. Since I have a dual-boot machine I can test Linux (Ubuntu) and Windows implementations fairly quickly. Mac on the other hand wanted to cause a bit of trouble by not being updateable. After eventually getting the required software installed and validated everything was cloned properly, I started working on the GSamples. 

GSamples is a new repository that aims to help developers who are unfamiliar with how Gateware operates into showing how to utilize the library. There were some samples already provided such as the GVulkanSurface and GOpenGLSurface which helped give me a jumpstart into what was expected. The first sample I was to work on was for the audio interfaces GSound and GMusic. As I started to make the samples, I found that isPlaying wasn't implemented into the main API which was what I used to identify if the sample should stop or continue playing. Since the Audio Interfaces needed a running application to continue playing the sound, this created a bit of a snag. Though thankfully Alex Cusaac mentioned opting to use sleep_for and yield instead. 

So safe to say I was out of the clear for the time being. Turns out testing all available platforms is really important. The audio I happened to use for the sample had a formatting issue which exposed an underlying offset bit read in the GMusic implementation. With that fix pushed, I was able to continue validating the GMusic and GSound samples. These samples helped me understand how the Gateware Developers use a Continuous Integration / Continuous Development (CI/CD) Pipeline to always test and push updated releases.

While I wasn't familiar with CI/CD I have grown to like this format of development. The next couple of samples I worked on were GInput and GBuffered Input. Since I wanted to print out the ASCII equivalent of the key pressed, CI/CD allowed me to pull the latest version of Gateware and implement that fix. 

My future work with Gateware will be to inevitably incorporate the GNodeFactory module. This module will help manage large 3D Transformation Hierarchies like scene graphs. For now I am researching into how I can parallelize and optimize operations in the factory before implementing them. To accomplish this task, I was requested to look into GConcurrent, GThreadShared, and GDaemon. These modules in Gateware are also on the coding chopping block for samples to create. Additionally GVector, GMatrix, GQuaternion are also going to be implemented over the next two weeks. 

Month 2 - Part 1

Over the past week I was working on a Vulkan validation error that was caused by some weird environment setup on my machine. The error was complaining about not knowing what a specific struct type was when chaining features on in our CreateVkDevice() method in this section of the code:

What was happening was the requested API got set to 1.3 but 1.4 was defined so when it would hit the #if defined(VK_API_VERSION_1_4) it would try to chain on the 1.4 feature struct but was using the 1.3 validation layers which were too old to recognize this newer struct.

Validation layers in Vulkan are meant as a debug tool to tell you when your pipeline/set up is off and helps debug issues early before they become a problem in release. Despite the message, everything got set up correctly and I couldn't reproduce the issue reliably so Lari and I decided that it would be best to move on as it was probably a user error in how I set up the project and my environment variables.

I then moved on to an issue reported to us by Justin Edwards:
VUID-VkDeviceCreateInfo-pNext-02830(ERROR / SPEC): msgNum: 555635515 - Validation Error: [ VUID-VkDeviceCreateInfo-pNext-02830 ] Object 0: handle = 0x25a8407cc70, type = VK_OBJECT_TYPE_INSTANCE; | MessageID = 0x211e533b | vkCreateDevice(): If the pNext chain includes a VkPhysicalDeviceVulkan12Features structure, then it must not include a VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_DESCRIPTOR_INDEXING_FEATURES structure

Which is complaining about 12 including the VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_DESCRIPTOR_INDEXING_FEATURES structure which comes from this part in our code:

Where before we didn't have the:

#if defined(VK_API_VERSION_1_2)

features12.descriptorIndexing = VK_TRUE;

#else

but with the new if define we're basically saying if we're on 1.2 or higher then we don't want to add that flag which ended up fixing the issue.

To ensure this fix was working we asked Justin for his exact setup and I created a test case mimicking this which looks like:

Which also accounts for the other versions of Vulkan. This test reproduced his errors without the fix and didn't produce them with it which proved our fix was working. He then downloaded the new Gateware version and tested it, confirming that it cleared that error but still produced this:

I'll be looking into what's causing this validation error along with tackling these tasks:

Which should provide some good learning opportunities for me.

Month 1 - Part 2

Over the past couple of weeks I've been working on a Vulkan issue that could happen if the user had Vulkan tooling installed that was on a lower version of the API than their drivers claimed they could support. I figured this out when I tried to build the project requesting Vulkan SDK v1.3.296.0 while my drivers supported 1.4+. The error message I got was:

VUID-VkDeviceCreateInfo-pNext-pNext(ERROR / SPEC): msgNum: -1876993556 - Validation Error: [ VUID-VkDeviceCreateInfo-pNext-pNext ] | MessageID = 0x901f59ec | vkCreateDevice(): pCreateInfo->pNext chain includes a structure with unknown VkStructureType (55). This error is based on the Valid Usage documentation for version 296 of the Vulkan header. It is possible that you are using a struct from a private extension or an extension that was added to a later version of the Vulkan header, in which case the use of pCreateInfo->pNext is undefined and may not work correctly with validation enabled. The Vulkan spec states: Each pNext member of any structure (including this one) in the pNext chain must be either NULL or a pointer to a valid struct for extending VkDeviceCreateInfo

Which basically means "I don't know what VkStructureType (55) is because it's not defined on this API version."

I eventually figured out that we were chaining feature structs up to the highest available version the users machine could run instead of what we intend to target. I fixed this by choosing the minimum version between what we intend to target and what the driver supports and then using this as a check before chaining feature structs. The code snippet for this is below:

1. We get the device properties so we can see which API version our graphics drivers support

2. We check this against our requested API version and take the minimum

3. We use this as a gate for feature chaining so we don't chain anything over the requested API version

That's all of my updates for now!

Month 1 - Part 1 - Implementing IntersectOBBToOBB

 Over the past few weeks I've gotten introduced to the Gateware project and started contributing. I've discovered a new Vulkan bug (by accident), implemented a function that outputs an OBB given a transform, and began implementation on another function that finds the minimum penetration distance between two intersecting oriented bounding boxes and returns it to the user. I've learned a lot during this time which I will share in the following post. 

During my first week I focused on getting my environment set up, reading through the code base and trying to compile. Given that this was my first time compiling and running the project I expected all tests to pass. Much to my surprise I was greeted with my first test failure. The issue was related to Vulkan and wasn't reproducible on others machines. After some debugging with Lari we found that it was related to the newer versions of Vulkan. We decided to table this for now as I was just getting started with the project and it worked fine on recent versions of the Vulkan SDK. 

The next week consisted of implementing a function that would output to the user an oriented bounding box given a transform. An oriented bounding box consists of a center, extents (width, height and depth), and rotation. I was familiar with OBBs on a high level, but never worked with them directly before thus began my research. I learned about the difference between Axis-Aligned Bounding Boxes and Oriented Bounding Boxes and their different use cases. I learned what defined an Oriented Bounding Box and how, given a transform, one can generate the center, extent and rotation of a bounding box. This turned out to be pretty simple and my solution is as follows:

Luckily, Gateware's math library contains functions that already compute the data we need given a transform. This ended up being as easy as plugging the transform into these functions to get the center and rotation of the desired bounding box. The one extra step I needed to do was take the limits vector given by the user and scale it by half since the extents of the bounding box are half width, half height and half depth. After calculating these, I set the appropriate properties of the output bounding box and return SUCCESS to let the user know it was created. 

The great thing about Gateware is that we are required to practice test driven development which means my work was not finished there. I then proceeded to create test cases for this function which consisted of:

1. Given an identity transform, we output a centered OBB
2. Translating an identity transform and then creating the OBB correctly sets the OBB center
3. Given a transform with rotation correctly sets the OBBs rotation
4. Given non-uniform extents still produces the correct extents for the OBB
5. Given zero limits will produce an OBB with zero extents

I then did the same for a double variant, added to the interface and added a fallback function. After this my work was complete! I submitted a merge request, received some feedback and finally merged into the main branch. It was an awesome feeling knowing my code, however small, was live and possibly being used by people building games.

After this I decided to move onto another OBB function. This time the function was supposed to return the minimum penetration distance between two intersecting OBBs. Again, I started with research. I found that in order to detect two intersecting OBBs I would need to implement something called Separate Axis Theorem which luckily is well documented but also much more complex than the method you would use to detect collision between two AABBs. I read Real-Time Collision Detection by Christer Ericson and started doing other research on topics that were fuzzy to me such as vector projection, cross products and how to use these to derive the information needed to complete this function. 

After a talk with Lari I decided to see if there was anything in the codebase that could help me finish this function. Low and behold, I found a little (not so little) piece of code that was commented out with the function name IntersectOBBToOBB. Well that's awesome, someone had already started this and not only that but the function was practically finished! This was a relief but at the same time I felt like I had wasted so much time looking elsewhere and holding up development which would turn into a learning for me. I decided to start dissecting this function and figure out why it wasn't working the way the author intended. 

Through debugging I found that we were initializing our out distance to 0.0f and then using this out distance in future comparisons when finding a new penetration depth in the form of (x < out distance) which meant that we were simply never updating out distance because the new penetration depth would never be less than the initialized zero! To solve this issue I created a variable local to the function that initialized to FLT_MAX. I then used this in our comparison so that the first time we found a new penetration distance it would be guaranteed to be less and on subsequent comparisons we would then be comparing against the current minimum penetration distance. There were a couple other issues that popped up after this but this was the root issue. We finally were returning the true minimum distance between these two objects.

I added some tests, clapped my hands together, but wait.. there was a failure. In one of my tests I simulate a separation of the two objects by moving the object by the minimum penetration distance found and then recheck for a collision, expecting no collision. The test was returning collision? After further inspection I realized it's because we add on an epsilon value to each axis. This epsilon value exists to prevent numerical instability when the boxes are near parallel. The problem with this is that we get an inflated projected radii so on the next call to the function (after we've separated the objects) our function will detect another collision even though the objects are geometrically separate. This is the issue I've been stuck on for the past week and I'm not sure what the best way to solve it is just yet. I don't want to just return an estimate as this could cause jitter for the user. I am currently researching open source physics libraries like box2d and qu3e to see how they handle similar situations as well as reading and watching material from other collision experts such as Dirk Gregorius, Erin Catto, and Randy Gaul in hopes to find an elegant solution to this problem. 

If I can't resolve this soon, I plan on tabling it while I tackle the original Vulkan issue I encountered during my first week of set up as this is a more pressing issue that is currently affecting users on newer Vulkan SDKs. On to the next week of development!

Gateware : The End of the Second Month

Another two weeks have passed since my last post, so it has now been two months since I started working. In the past two weeks, I have fully implemented a raytracing flag to the graphics options. Also in this blog, I will be reflecting on my overall time working on Gateware.

The main focus, as I previously mentioned, was implementing the raytracing flag as an option to make it easier for users to use raytracing, as it will set up all the extensions for them. This meant that the first thing I did was simply add the raytracing option to the graphics options enum in graphics define.

After that, I went into the GVulkanSurface_core.hpp and started by adding the if check for the flag and added the extensions if the if check was successful.

The next part I implemented was in the device creation. Again, we checked for the flag, and if it was true, then the needed extensions were implemented. These included:  

• VkPhysicalDeviceRayTracingPipelineFeaturesKHR
• VkPhysicalDeviceAccelerationStructureFeaturesKHR
• VkPhysicalDeviceVulkan12Features
• VkPhysicalDeviceFeatures2

After they were implemented, I also modified the get best gpu function to make it so that a gpu with raytracing support would be higher scored.

The last part was adding the unit test, which was a bit tricky as I thought it would be pretty much the same as the bindless one, but it actually required an extra check for extensions, otherwise it would fail on the CI/CD pipeline on all the runners except for one.

That leads us to the code review, which left me with a bit more to do before the feature was implemented. The problem now was that the additional extensions added to the device extension count were hardcoded, so I changed it to be based on the flags instead of hardcoding it to the max amount of possible extensions.

That finished off the raytracing flag addition, and it was finally ready to be implemented!

That now brings me to the end of my open portfolio classes and the end of my time here, so reflecting on the time I worked on Gateware, I can say that I learned a lot. It was my first time really working on such a large codebase. So, there were lots of things that I encountered for the first time and that were challenging to me, so it was a great learning experience, and it feels like I learned a lot that will be helpful moving forward, and when working in the industry. Overall, I am really grateful for this opportunity and am excited to see where these skills will take me next.

Gateware : The Start of the Second Month

Another two weeks have passed since my last post, so it has now been a month and a half since I started working. In the past two weeks, I have completed adding compute queue support to GVulkanSurface so you can now use compute shaders and also started to work on adding a raytracing flag to the graphics options.

The main focus of the last two weeks has been on fully implementing the compute queue support and the matching unit test, so let's start by going over that. The first thing I did was create a new function for getting the compute queue. This was quite simple as it followed the same pattern as getting the graphics or present queue. 

Then I also updated the function that got all of the queue family indexes (more on that later). The last thing to update before working on the unit testing side was to update the get gpu function to add more weight to a gpu that had compute queue support.

To test the compute queue support I had to write a glsl shader, for the compute queue test I was told to test it away from graphics, so I focused on testing with actual computing/math by writing my shader to take an input buffer square the values and return them in an output buffer. I chose to use two seperate buffers instead of just one for better readability.

The test was a success! Which meant I had successfully written compute queue support for Vulkan, and a successful unit test as well. After a code review, I just needed to add back a deprecated version of a function using Doxygen's deprecated feature to prevent any problems on the back end. And then the merge was successful! Meaning that Gateware now supports compute queues and shaders in Vulkan.

Now That leaves us with what I am currently working on which is adding a raytracing flag to the graphics options so that it will be easier to work with raytracing using Gateware. I just started this so I haven't got much to say yet, but will be posting about that in the next post!

Gateware: the first month

Two weeks have passed since my last post, making the total time working in Gateware one month!! In the past two weeks I have worked on various things, including finishing up the derive a plane from a given point + normal, and also fixed an issue where the GVulkanSurface test for checking for compute queue support was failing.

Starting with the math function implementation, I was able to finish of the function and unit testing in the first few days following my last post. The main thing I had left to write and clean up was the unit testing itself. Following that the merge request went into code review and after getting some feedback I cleaned up the function itself by removing a redundant magnitude check and changing the formatting a bit for clarity. I also added more fail tests to cover more cases to ensure a full test suite for the function.

After that, another code review left me with some changes I forgot to make to the Doxygen documentation, which was luckily really quick, and then the merge request was approved!

After that was successfully merged, I started work on a new issue, which was GVulkanSurface not allowing compute queue support. The test was already written to check, but it was crashing. There were two main problems in the code, firstly there was a cleanup call at the end of the create function that was deallocating everything, which was causing GetDevice to fail. After commenting that out, the GetDevice succeeded, I was able to identify the next problem. The queueFamilyIndex was being used without being initialized. This was causing problems because it was being used to get the index of an array. Luckily, the fix for this was simple! I just needed to add a for loop to loop through the queueFamilyCount and then set the queueFamilyIndex equal to the current iteration and then it was able to be successfully used. The test was now running with no crashes and succeeding.

And that has been the past two weeks! I am now going to be working on actually adding the compute queues and so look out for that in another two weeks!

Gateware: The first two weeks

 It has been two weeks since I started working with Gatware, and time definitely moves fast it feels like it has been only a few days. But I have been very productive over these two weeks, with my introduction being just a simple change by adding the OUT_OF_BOUNDS failure code. My first real task was reviewing the bmp addition to the GBlitter. The issue said there was a memory leak, however there were none when I looked through the code. Although this did lead to the discovery of a problem with the UWP runner, as the pipeline kept failing. So I helped try and look into that issue with the other members of the team. It was successfully resolved towards the end of the first week. I then looked through the remaining comments/threads left on the merge request and fixed up some messy commits and then added macros to replace the magic number if checks for the decompression in GBlitter to finish off my work with the bmp addition to the GBlitter.

Then for the remaining time in the second week I started working on an issue from four years ago to derive a plane from given point and a normal, I have gotten it pretty much finished and will be going through and cleaning it up a bit before sending it to be reviewed. The logic was relatively simple, though I almost forgot the magnitude 0 check, lol! But besides that it went pretty smoothly. I have written the unit tests out and tested them a couple times and they are all passing, yay!

Month 5 - Week 4 - Fixing Arch Linux X11 Bug

 Last week we touched upon a bug that was occurring with X11 for Arch Linux. The bug was specifically when creating a window for a test. The window would fail to create and now show up at all, failing the test. I was only modifying the unit tests at the time but took time this week to look more deeply into the actual code to see what could be going wrong.

One of the first things I decided to do was enable the macros inside of GWindow to see the X11 debug output. This would allow me to watch in real time what was failing, and better help towards finding the problem child in the code.

As you can see, the errors being reported are talking about a bad window. This normally means that the pixel map for the internal window is bad, or the window itself failed to create and can’t have anything done to it. I knew immediately the first place to check was the create function for GWindow.

Here we see two functions are called. The first function is just initializing some basic fields, the second function, however, is the one we’re interested in. `OpenWindow()` is the function responsible for creating the window and initializing all the information that GWindow needs to properly manage it.

Remember, we can’t debug, or the error won’t occur. So, instead, I decided to inspect some documentation and play around with the code, probing it to see which function was giving problems. Upon inspection, I noticed that anytime I messed with the `XInitThreads()` function, it had a change in the functionality itself. It seems this might be the problem child I mentioned earlier. I decided to put a `std::this_thread::sleep_for()` call right after calling for the initialization of threads. The sleep time was small, only 10ms, however, doing just this resulted in all of my tests passing consistently.

I found the bug that was occurring. So, how do I fix this then? Well, the moment I discovered this, I made a commit with the sleep call change as the fix. However, using this kind of call to fix the bug isn’t exactly good. Some issues can occur. For example:

• How would this code react on faster or slower hardware? We can’t be certain that this sleep fix will work for all systems, it might even only work on mine.
• Is this truly a threading issue, or a mutex issue? The display for X11 is multithreading capable, so if I don’t lock it, it could be other X11 functions are modifying or trying to modify the variable I’m reading immediately.
• Lastly, what if this fix is temporary like the event system bug months prior? This is a real concern; it would not be the first time I “fixed” a bug only to have it come back spontaneously because I wasn’t doing the idiomatic approach expected.

So, what I have to do is test for different solutions and see what I can find. This was, unfortunately, all I could do for now. I wanted to do more but was constrained on time with due dates and graduation fast approaching. While this is going to be my last blog post, it will not be the last of the changes I make to Gateware. I have full intention of continuing work on Gateware to not only fix this bug, but also to see what I can do to fix, implement, and update features for the project. This is in the hope that I learn more from it over time, that and I simply enjoy working at a low level. So, with that, this is my last blog post.

Month 5 - Week 3 - X11 Fullscreen Bug

 After the final touches to audio, my focus has now shifted to a new and rather interesting bug involving X11 window events and, arguably more importantly, Arch Linux. This bug has shown itself to me for a little while but was largely inconsistent in how it appeared. It would show in the unit tests ~80% of the time, and the source of it seems to be unknown currently. This bug will be my last bug to fix for the project as part of FSU, however, if I can't fix this bug by the deadline, I will be continuing work to fix it after. This is largely because this is the only bug left to fix to finally add Arch Linux as a fully supported platform. This will be brief as I didn't have a lot of time to fully inspect the bug like I wanted to, but there will be enough here to mark my current progress. So, with that, let's get started.

The first thing to note is the description of the bug.

Here are the things to note:

• The bug has a visual aspect (the window will sometimes show, sometimes stay hidden, or show completely but be transparent).
• It fails specifically when the check is run to see if the window is fullscreen.
• Previous calls in the said test don't do anything.

Here is the test.

We can see on the overview that the window is initially created with a basic position (0, 0) and dimension (800, 500). It then reconfigures itself to a different position with different dimensions. Something else you might have taken notice of is the usage of `GiveWindowTimeBeforeRequiringPass`. From what I've gathered, this is here because sometimes X11 requires a bit of time to process events before it can check for what we want. So this is here to effectively wait out the processing of events to check if a function has passed properly. From here, this is where I started testing.

First off, breakpoint debugging is useless for this bug. Because if I placed a breakpoint and walked the function line by line, nothing would go wrong. Debugging would mainly be useful here if I could catch the bug failing in action, however, I'm not able to do that here. So, this leads me to test a little more conventionally. The thing I checked for immediately was timing. It could be that the code is going too quick and preventing X11 from processing everything in time. So I added small delays to check for this, however, it did not work. I also tried using the exact delays used in other tests, but the results were the same.

Interestingly enough, if I duplicated the exact same test and ran the full unit test suite, only the first test would fail, with the duplicate passing. Odd behavior.

From the discord messages above (which are posted in the Gateware Development channel), I described my difficulties with this bug as well as what I feel the bug is.

I feel this bug is a timing bug. Basically, one thread is doing things so quickly that the other thread can't keep up in time. This would explain the odd behavior, and why the test doesn't always fail. Of course, there is more testing and inspecting of the code that will be needed, but in total, this seems to be the main issue. So now, what has to happen, is we have to specialize the probes for this bug to target the timing over the functionality. If I can pinpoint that it is timing-related, then I focus my efforts on fixing that. However, if this isn't the case, I will have to inspect further to see what is exactly happening.

Currently, this is all that I have done for this bug. Next week should hopefully go into a deeper discussion on how I found the fix for it. But for now, this is all.

Month 5 - Week 2 - Final Touches

 Last week, we discussed finishing the WAV file reader and getting the overall status for it marked as completed. There were some final changes. Moving over to GFile for UWP was a big step, but there was still some branching in the code, primarily due to Linux and Windows differences. I decided to focus my energy on getting these rounded out and fixed. This will be a light post, and step through everything fairly quickly.

The thing to really focus on and correct is how Linux and Windows structure the WAV header. Internally, Linux structures a WAV fairly simple, and this is due to it not requiring the extra data that was later attached when Microsoft extended the WAV format. Windows contains many things, however. It contains the primary information which is the same as Linux, but a union that holds additional information. This is also paired with a GUID. The purpose of the union and GUID is unknown to me, but to bridge compatibility with platforms, I had to move away from a platform-specific definition and move to an internal definition, which is exactly what I did.

If you have a keen eye with the Windows API, you will notice immediately that this new structure setup for WavReader is the exact same as Windows's definition. I did this for a simple reason: to support Windows and allow support for Linux. If I used the Windows definition, it wouldn't be able to compile for Linux due to obvious reasons (Windows API not being available for other OSes). After moving to the new structure, it was now the process of hooking everything up and removing old branching.

After making these changes, now WavReader no longer branches and works on every platform universally. Additionally, it acts as a wonderful introduction to what is possible with the library, as all of the heavy lifting for data handling is done by library code (GFile). So a lot of beginners just getting into Gateware could learn a fair bit by reading through the source code of WavReader.

This marks this merge request as complete. I marked my name in the list of contributors, and let the merge get reviewed again. It was merged. The last thing I did for audio was make two merge requests, one focusing on some simple Windows refactors, and the other focusing on UWP refactors. Effectively syncing both to the same modern standard as Linux. In total, everything is now completed.

What now? Now I find a new issue I can tackle, a bug or possible feature that will take only two weeks to implement. With my time almost finished with Gateware, I'm no longer able to tackle such large issues as I have in the past, so that will be left up to others. However, I will try to provide supporting changes here and there, and see what I can do with the time I have left for finals.