Month 3 - Blog 6 - Gateware Development

GNodeFactory... a factory to manage large amounts of nodes.

An initial task that was started throughout these previous weeks was to implement most of the Get and Find methods regarding GNodeFactory. Without these methods being implemented the developer or user will not be able to actually retrieve information in the system. Say they lost a child node's handle, without the methods getting the handle back easily would become a lot more difficult. That being said the following methods were implemented to help provide this functionality: 

• GetImmediateParentNodes
• GetImmediateChildNodes
• GetImmediateSiblingNodes
• GetFactoryInfo
• GetFactorySize
• FindAllParentNodes
• FindAllChildNodes
• FindAllSiblingNodes

This grouping of functionality was fairly straightforward with its approach. Simply access the pool, then access the state of the given node. Repeat for the number of requested nodes. This takes into account that all the data has been validated beforehand. Since swapping to using a GNodeHandle it makes determining the pool and node a bit easier since they are in a union as an unsigned long long. 

Finding the nodes however was a little bit more involved since it required using a DFS Traversal to gather the nodes. Since the user might not know the amount of parents or children that a node could have, the number of nodes gathered is also given as an out parameter. Traversal can get a bit cost heavy, to avoid this there is a thread_local vector maximized to the number of possible nodes we have. This way we will never run out of space to allocate nodes and there is always enough space to fit all existing nodes in it. For FindAllChildNodes, siblings are pushed onto the stack first, then children. Since it's a stack, children come out first when we popped, which helps with traversal since they are still in order.

Aside from the Finding and Getting, in a previous meeting Lari pointed out that AllocateNodes didn't properly allocate to the right pools. Embarrassing as that was, he suggested to use a GPool Structure which was similar. Instead of having 4 pools (states, metas, internals, and nodes) there would be a single factory of GPool. So instead of trying to manage both data oriented and parallelization. We shifted to 100% parallelization and attempting to maximize the efficiency of that. 

Allocation was adjusted to create a pool of an ideal size (currently 4096) and push nodes into those pools if there is room. If the user asks for a larger count than 4096 then a pool of that size is made. Otherwise if there is room found in an existing pool to use that, or make a new pool of ideal if none is found.

GHandle also was introduced to help identify nodes and pools properly. Instead of the node and pool being individual memebers, they are now a union as a "handle" which can be passed as an unsigned long long. This makes it so the user can store handles quicker and easier for the end user/developer.

The next milestone will be nearing the end of GNodeFactory's development time with me as I hope to implement the Resolving of the Hierarchy and a visual demonstration of it working!

Month 3 - Blog 5 - Gateware Development

 These last two weeks have been quite hectic to say the least with Gateware. As development continues with GNodeFactory (GNF) a couple of hurdles came along. First was how to actually store the data within GTL's Factory architecture since I was still getting the hang of it. GNF is supposed to be structured as a factory containing multiple root factories. These root factories will help manage threads and computations making operations faster than single threaded executions. I initially was not able to fully grasp how the implementation would look, but after some time with a single root factory I slowly was able to understand and created the below implementation so far. Previously there was going to be a pool for the State and Meta information in their own factories as well, but after discussing some storage and accessing viewpoints with Lari we decided it was optimal to keep the information within the Internal Node structure. This makes looking up the information faster since there is no need to perform additional access operations. 

This led into the next task for implementing allocate and deallocate for GNF. Whenever Allocate is called, it will attempt to recycle an existing pool identifier before creating a new one. With this design I choose to create a pool of root nodes wherever is available first. Deallocating performs the inverse by validating the data is available to remove then cleans up any associated data pools and recycling their identifiers. Since this task was done in the previous week portion for this blog post, I have now code reviewed and merge requested the Allocation/Deallocation of GNF with Lari.

Moving forwards to the bulk of this previous weeks work: Parenting and Unparenting. There were some challenges at first to Parenting and Unparenting, first was making sure the calculation was one with the proper inverse performed on the parent nodes rotation. One issue that I found while writing the unit tests was that if I wanted a duplicate index to unparent then the operation should technically fail since I shouldn't be able to unparent an already unparented node. Provided that most of the implementation still stays relatively the same, I believe Parenting and Unparenting are nearly complete as of this post being written.

This milestone also covered reading and writing to the nodes, matrices, and meta data while keeping reading of the states non-writeable. State information regarding the nodes would be purely manipulated internally since there should be no reason to modify a parent/child/sibling/depth/flags associated within the nodes. Unit tests to validate that these operations are performing as suspected are also being written since validation and unit tests are required for very part of Gateware.

Moving forward into the next two weeks I will be focusing more so on implementing the resolving of the hierarchy and ensuring that all data within GNF uses the flags properly. There will be many.. many more unit tests to cover with more implementations and samples near the end of that.