Thursday, July 16, 2026

Month 4 - Blog 7 - Gateware Development

Time is coming to an end with GNodeFactory and I have had an immense pleasure being a part of it. As I am writing this a fair few additions have been made to the project. In order to organize this into a succinct post, listing out each item will help:


GNodeFactoryF Allocation Strategy being updated (again..)

    The update to the allocation strategy this time went from having pools of data that could theoretically be endless to having a fixed maximum size pool. This would mean that when the user allocates nodes, they are either filling up existing pools, or if needed having more pools being made at a fixed size and having the handles returned. Handles still work with cross-pool relationships which was exciting to not have to go track and fix more bugs.

    Currently the node count per allocated pool is 4096 which should be modified as testing happens finding out the best "bang for buck" allocation size for the system. While users typically shouldn't need to allocate a "large" number of nodes (i.e. one million nodes), GNodeFactory should still be able to maintain performance at those large node counts, within reason.


FindSphereNodes being implemented, then multithreaded

    Lari and I discussed that at least one spatial query method would be implemented before my time with Gateware was up. Looking at the workload amongst my program requirements I initially wanted to leave spatial query methods to a future developer. Since Lari wouldn't consider GnodeFactory "completed" until at least one was implemented, FindSphereNodes is now here.

    Initially I had waited to do multithreading until the entire library was created, then I would go back and multithread where reasonable. In this case since we need to look down all the pools we can tell GConcurrent to run some work for each pool to find the node in parallel. The idea is that if multiple threads are working to find the node within their pool, the time spent overall would be less if it was linear. Functionality was tested using a separate solar systems simulation. Below is that simulation where I would press `1` to toggle FindSphereNodes within 100m of the camera.



ResolveHierarchy being multithreaded

    One of the largest features within GNodeFactory is the ability to resolve the hierarchies transforms. In order to do this, every pool will be tested to find out which nodes are dirty. Then based on those nodes, run work in parallel to recompute them. Since node roots don't overlap, every node has exactly one parent. This means you can follow a parent from anywhere resulting in a single root. Two roots can never share a node, letting each root get its own thread with no locking between them.

    The write lock is taken once at the top and the workers run under it without ever locking again, which also fixed an earlier version that deadlocked itself by re-locking on every level of the recursion. ResolveSubHierarchy stays single threaded since the handles a user passes in can overlap, and two threads resolving overlapping subtrees would trip over each other.


FindAll Child, Sibling, and Parent Nodes being multithreaded

    These three take a list of nodes to search from, so instead of splitting the work up by pool, I split it up by search. Each search is on its own anyway, so each one gets its own task and its own list to write results into.

    Once they all finish, those lists get copied into the real output array in order. That order matters because the output is one flat array with a count for each search, and a thread has no way to know where its results go until the searches before it are done. Copying them in afterwards means everything ends up exactly where it did before.

    That said, this one probably wasn't worth it. It only helps if you're searching from a lot of nodes at once, and most of the time you're only searching from one, so you get a single task and pay for the threading without getting anything back. It does keep the Find methods consistent with each other, at least.


Find Capsule, Frustrum, AABB, and OBB Nodes 

    Once FindSphereNodes was done, the rest were easy. Every node already has a sphere around it, which is just the collider radius sitting wherever the node resolved to. So all four of these are really asking the same thing, does this sphere touch that shape? GCollision already had those tests written, so each method ended up being the same scan as the sphere one with a single line swapped out in the middle.

    AABB needed its min and max corners turned into a center and extent first, since that's the form the test wants. The frustum is the odd one out, because it isn't one shape, it's six planes. A node is outside if its sphere sits fully behind any one of them, so it checks all six and stops at the first miss. That assumes the planes point inward, which is normal, but worth knowing if a frustum ever comes from somewhere unexpected.


As for everything else, I don't really have any major updates towards GNodeFactoryF. GNodeFactoryD however will be getting made soon since it will be mostly a copy and paste from the float implementation. Since that covers most of the past two weeks, I hope everyone who gets the chance to use GNodeFactory can create some awesome projects with it!

Friday, June 26, 2026

Month 3 - Blog 6 - Gateware Development

GNodeFactory... a factory to manage large amounts of nodes.

An initial task that was started throughout these previous weeks was to implement most of the Get and Find methods regarding GNodeFactory. Without these methods being implemented the developer or user will not be able to actually retrieve information in the system. Say they lost a child node's handle, without the methods getting the handle back easily would become a lot more difficult. That being said the following methods were implemented to help provide this functionality: 

  • GetImmediateParentNodes
  • GetImmediateChildNodes
  • GetImmediateSiblingNodes
  • GetFactoryInfo
  • GetFactorySize
  • FindAllParentNodes
  • FindAllChildNodes
  • FindAllSiblingNodes
This grouping of functionality was fairly straightforward with its approach. Simply access the pool, then access the state of the given node. Repeat for the number of requested nodes. This takes into account that all the data has been validated beforehand. Since swapping to using a GNodeHandle it makes determining the pool and node a bit easier since they are in a union as an unsigned long long. 



Finding the nodes however was a little bit more involved since it required using a DFS Traversal to gather the nodes. Since the user might not know the amount of parents or children that a node could have, the number of nodes gathered is also given as an out parameter. Traversal can get a bit cost heavy, to avoid this there is a thread_local vector maximized to the number of possible nodes we have. This way we will never run out of space to allocate nodes and there is always enough space to fit all existing nodes in it. For FindAllChildNodes, siblings are pushed onto the stack first, then children. Since it's a stack, children come out first when we popped, which helps with traversal since they are still in order.

Aside from the Finding and Getting, in a previous meeting Lari pointed out that AllocateNodes didn't properly allocate to the right pools. Embarrassing as that was, he suggested to use a GPool Structure which was similar. Instead of having 4 pools (states, metas, internals, and nodes) there would be a single factory of GPool. So instead of trying to manage both data oriented and parallelization. We shifted to 100% parallelization and attempting to maximize the efficiency of that. 


Allocation was adjusted to create a pool of an ideal size (currently 4096) and push nodes into those pools if there is room. If the user asks for a larger count than 4096 then a pool of that size is made. Otherwise if there is room found in an existing pool to use that, or make a new pool of ideal if none is found.


GHandle also was introduced to help identify nodes and pools properly. Instead of the node and pool being individual memebers, they are now a union as a "handle" which can be passed as an unsigned long long. This makes it so the user can store handles quicker and easier for the end user/developer.

The next milestone will be nearing the end of GNodeFactory's development time with me as I hope to implement the Resolving of the Hierarchy and a visual demonstration of it working!

Sunday, June 14, 2026

Month 3 - Blog 5 - Gateware Development

 These last two weeks have been quite hectic to say the least with Gateware. As development continues with GNodeFactory (GNF) a couple of hurdles came along. First was how to actually store the data within GTL's Factory architecture since I was still getting the hang of it. GNF is supposed to be structured as a factory containing multiple root factories. These root factories will help manage threads and computations making operations faster than single threaded executions. I initially was not able to fully grasp how the implementation would look, but after some time with a single root factory I slowly was able to understand and created the below implementation so far. Previously there was going to be a pool for the State and Meta information in their own factories as well, but after discussing some storage and accessing viewpoints with Lari we decided it was optimal to keep the information within the Internal Node structure. This makes looking up the information faster since there is no need to perform additional access operations. 


This led into the next task for implementing allocate and deallocate for GNF. Whenever Allocate is called, it will attempt to recycle an existing pool identifier before creating a new one. With this design I choose to create a pool of root nodes wherever is available first. Deallocating performs the inverse by validating the data is available to remove then cleans up any associated data pools and recycling their identifiers. Since this task was done in the previous week portion for this blog post, I have now code reviewed and merge requested the Allocation/Deallocation of GNF with Lari.

Moving forwards to the bulk of this previous weeks work: Parenting and Unparenting. There were some challenges at first to Parenting and Unparenting, first was making sure the calculation was one with the proper inverse performed on the parent nodes rotation. One issue that I found while writing the unit tests was that if I wanted a duplicate index to unparent then the operation should technically fail since I shouldn't be able to unparent an already unparented node. Provided that most of the implementation still stays relatively the same, I believe Parenting and Unparenting are nearly complete as of this post being written.


This milestone also covered reading and writing to the nodes, matrices, and meta data while keeping reading of the states non-writeable. State information regarding the nodes would be purely manipulated internally since there should be no reason to modify a parent/child/sibling/depth/flags associated within the nodes. Unit tests to validate that these operations are performing as suspected are also being written since validation and unit tests are required for very part of Gateware.

Moving forward into the next two weeks I will be focusing more so on implementing the resolving of the hierarchy and ensuring that all data within GNF uses the flags properly. There will be many.. many more unit tests to cover with more implementations and samples near the end of that.

Saturday, May 30, 2026

Month 2 - Blog 4 - Gateware Development

Over the past two weeks, I have worked on getting the new GNodeFactory library set up and having a basic skeleton structure with boilerplate implemented. An initial work item for adding the initial GNodeFactory boilerplate files was made. Within it, the Universal/Dummy Implementation, Class Header, and Unit Test files were all incorporated. Since most of the functionality within GNodeFactory is shared between the floating and double precision types they can exist within a shared implementation class. A few structures needed to be added within GMathDefines being the GNode, GNodeState, and GNodeMeta. Each of these structures will help manage the basic information required within the factory. GNode will contain the transformation/rotation/collision information. GNodeState would incorporate how to traverse the hierarchy with access to the parent, child, and siblings. Lastly, GNodeMeta data will be utilized for future libraries within Gateware were the data can be managed by downstream systems.



Since GNodeFactory is part of Gateware it also requires Unit Tests to be made using the Catch2 API as all previous libraries have. For the Unit Tests I wanted to make them clear as to what functionality I would be testing with descriptors of the precision type. Some prior libraries use the SECTION macro to differentiate tests, so I opted to also use that with distinguish precision.


Lastly, I have been working on a spike solution for GNodeFactory as I wait for Merge Requests and Code Reviews to take place. I have made some significant progress with the spike solution as I was able to get some users to test out the work in progress library and give some feedback. The spike solution is a significantly condensed version of GNodeFactory but aims to stick as true as it can to the goal of the library. Within the spike solution I am opting to use return types instead of pass by references/pointers which Gateware utilizes. This is for simplicity on what the library will do in the meantime. Below are some implementations of the spike solution:






As we get into the next couple of weeks I will aim to implement the spike solutions research into the GNodeFactory library itself. Currently awaiting code reviews and feedback from maintains, advisors, and instructors. 

Saturday, May 16, 2026

Month 2 - Blog 3 - Gateware Development

Another two weeks have past and that means another blog update. This time with progress not only on the GSample repository but also the GNodeFactory implementation. Some of the samples I have worked on in the past two weeks were:

  • GFile
  • GLogic
  • GBlitter
  • GController
  • GInterface

Most of these samples will help when it comes to unit testing for GNodeFactory since stress testing the library will be conducted over a period of a week. This will need to involve automated data analysis which GFile and GLogic can help assist with. GInterface will be utilized (Specifically GInterfaceInterface) within GNodeFactory to inherit functionality helping manage the libraries lifetime and tasks.

GBlitter, while unable to really assist with GNodeFactory itself, has reinforced the concept of descriptive code. That will help me make GNodeFactory as understandable as possible for users. The GBlitter sample features a slime sprite spinning around on some grass tiles. By using TGA files, GBlitter can import and have the tiles defined. 


I have also made progress with drafting more of the official proposal out. A section stating the allocation and management of nodes is still to be written but will be accomplished before the next milestone. I have started to implement the interface for GNodeFactory within the source. Since it is a math library it will utilize both Float and Double operation types. Also since this library is in progress it will be marked with the "In Development Flag".



Tuesday, May 5, 2026

Month 1 - Blog 2 - Gateware Development

 The past two weeks have been hectic to say the least. Being filled with development and preparing for my presentation, my nerves were definitely on the line. Since my ongoing task was to create samples for the new GSamples repository, I added some more tasks for samples expanding on the following Gateware modules:
- GMatrix
- GVector
- GQuaternion
- GDaemon
- GThreadShared
- GConcurrent


So far most of theses samples have been merged into the main GSamples repository which is very exciting. They didn't come with the fun hiccups of learning more about CMake, like with GAudio and GBufferedInput, but did offer more learning into how threading and concurrency work. 

Developing GDaemon's sample created an interesting point about the documentation provided in the Gateware Doxygen resource. My assumption was to make a daemon, then add event responders to the daemon. This ended up not being the intended functionality and needed to be swapped out for something more focused on GDaemon itself. Swapping to demonstrate the Fibonacci Sequence using a daemon seemed more suitable, so one daemon was used to run iterations of the sequence while another toggled the sequence daemon on and off. Both of these daemons had event responders to print out the status of the daemons, while also running at specific intervals.


The rest of the samples went smoothly, with little to no issues arising. This helped the fact that I had to work on my GNodeFactory Proposal Presentation. This was a requirement of the CSMS degree in which I had to get the project approved by the Committee members. The past week I worked with Lari to present my materials to the Gateware Maintainers, which was insightful. They provided a ton of feedback which I tried to incorporate as best I could into the upcoming presentation. After the meeting with Lari and the maintainers, I had my meeting with the Committee. They really liked the project and were interested in a ton of the unit testing and research ideology. Overall I think the past two weeks have been really fun, not only for my confidence, but for learning more with modern systems.

Thursday, April 23, 2026

Month 1 - Blog 1 - Gateware Development

So being onboarded into Gateware was fun to say the least. My tasks for onboarding into Gateware were simple at first, recursively clone the repositories and verify they build. Since I have a dual-boot machine I can test Linux (Ubuntu) and Windows implementations fairly quickly. Mac on the other hand wanted to cause a bit of trouble by not being updateable. After eventually getting the required software installed and validated everything was cloned properly, I started working on the GSamples. 

GSamples is a new repository that aims to help developers who are unfamiliar with how Gateware operates into showing how to utilize the library. There were some samples already provided such as the GVulkanSurface and GOpenGLSurface which helped give me a jumpstart into what was expected. The first sample I was to work on was for the audio interfaces GSound and GMusic. As I started to make the samples, I found that isPlaying wasn't implemented into the main API which was what I used to identify if the sample should stop or continue playing. Since the Audio Interfaces needed a running application to continue playing the sound, this created a bit of a snag. Though thankfully Alex Cusaac mentioned opting to use sleep_for and yield instead. 

So safe to say I was out of the clear for the time being. Turns out testing all available platforms is really important. The audio I happened to use for the sample had a formatting issue which exposed an underlying offset bit read in the GMusic implementation. With that fix pushed, I was able to continue validating the GMusic and GSound samples. These samples helped me understand how the Gateware Developers use a Continuous Integration / Continuous Development (CI/CD) Pipeline to always test and push updated releases.

While I wasn't familiar with CI/CD I have grown to like this format of development. The next couple of samples I worked on were GInput and GBuffered Input. Since I wanted to print out the ASCII equivalent of the key pressed, CI/CD allowed me to pull the latest version of Gateware and implement that fix. 

My future work with Gateware will be to inevitably incorporate the GNodeFactory module. This module will help manage large 3D Transformation Hierarchies like scene graphs. For now I am researching into how I can parallelize and optimize operations in the factory before implementing them. To accomplish this task, I was requested to look into GConcurrent, GThreadShared, and GDaemon. These modules in Gateware are also on the coding chopping block for samples to create. Additionally GVector, GMatrix, GQuaternion are also going to be implemented over the next two weeks.