Gateware Bible

I've started work on the paper that will explain Gateware and act as a source of all our collected knowledge. I am attempting to pool all of our tribal knowledge together in the written word for future developers. My technical writing's a bit rusty so I'm reading a few books and taking some online classes to help freshen up. All seems to be going well so far so here's hoping in stays that way.

Triple Scalar Product

The cross product between two vectors dotted with another vector happens so frequently in linear algebra that it has its own special name: scalar triple product. To truly realize why this expression is so common and important I recommend looking at the visual illustration I drew in ms paint below. The triple scalar product provides the signed volume of the parallelepiped in 3d space. It's important to note the signed part because often in linear algebra a signed area or volume can tell you information about the vectors and their orientation to one another (clockwise or counter-clockwise). In addition, it may also be able to cheaply check if a line is within a triangle by checking which side the line lies on compared to the vertices of a triangle (figure 2).

(u × v) • w = (v × w) • u

 Figure 1. Triple Scalar Product visually.

Figure 2. Comparing a line to check if it's within a triangle.

GInput is a simple port or is it? And what's next?

I've been doing the GInput port to the new platform for a week. It's been a struggle but it is almost complete. I had issues with the unit testing, had to introduce GWindow into GInput (since it didn't exist back then), added error checking, rewrote unit tests. Problems came across not only from  GInput itself but other libraries or improper internal implementations, that I had to resolve quickly with 0 knowledge of the library at a time.

Most of the issues were fixed and GInput is working on all platforms, but currently GDaemon implementation on Linux requires some notice (quick destruction of GDaemon consequently in unit tests causes some weird behavior). I also fixed mouse input not getting registered on Linux platform.

After GInput issues are completely resolved, I will be moving to porting GAudio... This could be a 2 day job or a month job depending on the issues that occur and potential issues with incorporating other libraries. It already has a separate implementation for each platform which should make my life easier but it is 6 different classes in 1 package of libraries (GAudio/GAudio3D) which might be a fun experience with proxy logic/optimizations.

Triple your framerate with this one weird trick! Dermatologists hate her!

Last week saw GRasterSurface go from an uncompileable heap of probably functional but untested code to passing all its unit tests and drawing output at acceptable framerates in most cases on Windows. Lari spent a lot of time working with me on friday last week to get the surface running as well as possible, and his suggestion to switch from single to double bufferring took less than fifteen minutes to implement and tripled the framerate of the surface. There are still some bugs left to fix, but I should be moving on to Linux implementation no later than tomorrow.

GFile in the cabinet, Time to be a woodsman

Over teh course of last week I worked on getting GFile through the porting process. For the most part it wasn't bad but I retroactively learned that I did a pretty large amount wrong on it and that it would have to be changed or fixed in various ways. I actually still have to go back into it and take away a few "using" calls because as things stand it's technically not complete. For the most part though, the new architecture is there and I'm starting work on GLog, the successor to GFile.

Thus far GLog has been more of the same, save for the fact that the create functions are a little harder to port, but things are progressing at a steady pace regardless. Not really any major hindrances or obstacles have been overcome or interacted with, just work as usual.

GWindow ported!

Surprise surprise surprise...GWindow got ported, I was able to find a way around Mac's objective-c issue. The solution for that is making the main.cpp into a mainOBJC.mm, this is because XCode needs at least 1 .mm file so that it can compile the file with objective-c and c++, mainOBJC.mm only contains 1 line:
#import main.cpp

doing this enables gateware developers to port those gateware libraries that require objective-c code to put the code in the _mac.hpp file itself. Example:
GWindow_mac.hpp:
#ifdef __OBJC__
@import Foundation;
@import Cocoa;
#endif

namespace GW
{
namespace I
{
class GWindowImplementation;
}
}

// The GWDelegate will be the delegate of the main window which will receive window events
@interface GWDelegate : NSObject <NSWindowDelegate>
{
@public GW::I::GWindowImplementation* pWindow;
    @public GW::I::GWindowInterface::EVENT_DATA eventData;
@public GW::GEvent gevent;
}
+(void)doNothing : (id)threadID;
-(NSSize)windowWillResize:(NSWindow*)sender
toSize : (NSSize)frameSize;
-(void)windowDidResize:(NSNotification*)notification;
-(void)windowDidMove:(NSNotification*)notification;
-(void)windowDidMiniaturize:(NSNotification*)notification;
-(void)windowDidDeminiaturize:(NSNotification*)notification;
-(void)windowDidEnterFullScreen:(NSNotification*)notification;
-(void)windowWillClose:(NSNotification*)notification;
@end

// The GWResponder is our interpretation of the NSResponder that will propagate window messages to other responders
@interface GWResponder : NSResponder
-(bool)acceptFirstResponder;
-(bool)acceptsFirstMouse:(NSEvent*)event;
@end

@interface GWAppDelegate : NSObject <NSApplicationDelegate>
-(void)applicationDidFinishLaunching : (NSNotification*)notification;
@end

Above code is valid syntax and is able to be compiled in XCode, XCode compiles this hpp as both c++ code and objective-c code.

Now I am working on porting GOpenGLSurface while keeping an eye on my tool making sure nothing breaks when the tool generates the single header

Same same but different but still the same.

Month 3, week 2 started with more setting up of the GitLab CI environment. This has been somewhat of a daunting task; not because of the metaphorical documentation that is abundant in DevOps, nor is it due to how complicated the system is, no all that is quite fine. What makes this process painstaking is that the commands don't perform the same way, or at all sometimes, as they would in a regular terminal. So the conclusion is that the scripts need to be written in a way that assumes a perfect run each time. If there is a failure at some point it needs to fail before it has the opportunity to mess up future jobs in the pipeline, and that wouldn't be so bad... if conditional statements would work as intended. Another part of this process has obviously been debugging, and good practice for debugging states to make a few changes as possible in-between tests because if you go and change to much and the problem is fixed then what part actually fixed it or is the problem hiding? So, in order to follow good practice I make few changes, and then I push, it runs the pipeline and I make the necessary fixes and repeat. Pushing up to GitLab does take a bit longer than hitting build in Visual Studio so this can sometimes cause some frustration.
Overall, GitLab has been a thrilling learning experience and my command line knowledge has increased by a substantial amount, this, I am thankful for because it was a skill I was sorely lacking up until this point. Though it can be frustrating at times, what part of development isn't? Being able to stay calm and know what to do next in order to get the job done is a skill that is being exercised aggressively in Gateware and that too, I am thankful for.

GRasterSurface underway

Implementation for GRasterSurface has begun properly. I have written implementations for most functions on Windows by this point, although there are obstacles in the way of testing my implementations that I have not solved yet. There are some compile errors that I have not identified the source of yet, and I am not able to run the unit test solution because of permissions issues. Despite this, though, I am making progress toward GRasterSurface working on at least Windows, and once that's done it shouldn't take a terribly long time to have it working on Linux as well. Mac still doesn't have a working proof of concept because nobody has useful information about problems with programming in Cocoa.

From audio to porting

Month: 3, Week: 3

Our priorities got changed, and currently, I am no longer working on improving GAudio but rather helping to port the rest of the libraries to the new architecture. My initial task was to port GBufferedInput but now I am transitioned to porting GInput instead. Supposedly, GInput is an easier library to port but it has many internal issues I would need to resolve to proceed with the complete port. My initial idea of multi-threading our unit testing has failed, since Catch2 only supports serial testing, so I would need to isolate the GInput testing with the window creation within the GInput unit tests, instead of running the input and Catch on separate threads.

I had a lot of questions about the new architecture and GInput refactoring, but now I feel pretty confident in my what I am supposed to do now. And since GWindow already got ported (at the moment, Windows and Linux only), I would be able to use it as a baseline for some of the input simulations and create functions.

In the meantime, I am also waiting on complete GDaemon changes, so I can finish up the testing I did on it that I started last week. Also, after I am done with the input libraries, I will move to GAudio porting which due its complexity might take a while to do.

GWindow port status and what comes next of it (if mac wants to work)

GWindow port has been rough, fix a lot of the known issues ever since the old architecture on Linux platform. Some of the issues include race conditions because the window thread on Linux is launched as detached, so whenever the main thread finishes working, the window thread will still reference the memory that has already been cleaned up. Another issue is that X11 library doesn't have good documentation, at one point the window thread was deadlocking, preventing main thread from accessing any resource. That issue would've been fixed early on if X11 had better documentation on its API functionalities, but nonetheless with the asssistance of Alex, the bug has been squashed. Currently working on porting mac, because the new architecture hasn't been completely ported, so I have to also learn CMake and Objective-C to port GWindow on Mac. I also ported GDirectX11Surface, it was an straight forward porting because is in Windows OS. In summary: GWindow in new archietecture is fully operational on Windows and Linux OS, still working on Mac (I'm not sure if I'll be able to finish it to be honest) Afterwards I'll port GOpenGLSurface and update the executable for my tool (If I finish GWindow on Mac)


Here are some code snippets

This code is what caused race condition issue on Linux:

runLock = true;
linuxLoop = new std::thread(GWindowProcedure, std::ref(*this));
linxLoop->detach();

This is code that fixed the race condition:

runLock = true;
linuxLoop = new std::thread(GWindowProcedure, std::ref(*this));

~GWindowImplementation()
{
if (linuxLoop)
{
runLock = false;
linuxLoop->join();
delete linuxLoop;
linuxLoop = nullptr;
}
}

I'm a goon

Over the last week I finished porting GFile to windows, and began porting to Linux.

Most of the time was spent finishing GFile for windows, but on the last couple of days I made the hpp for linux. Once I was finished getting that through I went to try and test my alterations to make sure they worked or rather didn't work, but found I couldn't compile at all. After coming to terms with having to trash my VM and make a new one due to its acting up, I committed my changes and nuked the VM. I've spent my monday today getting it back after a few failed attempts and now that I've got everything ready to go I opened up the repo in gitkraken and learned that I never pushed that commit I made. So now it's back to re-doing those linux changes before I can begin properly porting again.

First Post of 2020 in the Gateware

To start off the new year we got most of the team into the new arch which is super exciting. Tone and Alex started helping us with the port job which will be a big boost in speed. On the downside our studio got hit with the plague so we didn't get alot done sadly but its a new week and hopefully a healthier week.

Seeing 20/20 with the New Architecture (Here comes GCollisionX)

I got the green light to port the currently incomplete GCollision library to the new architecture and it went smooth. It took about a day and I can say that both users and their device's memory that run the GMath library will be happy. Previously, the GMath libraries were all virtual functions and required you to instantiate the respected class in order to call its functions to operate on data. Now, you can just call the namespace to access these functions. In addition, this reduces the amount of overhead for the virtual function table. Another great change in this new architecture is users no longer required to free any G objects they had created. If the object falls out of scope then it goes, "bye-bye". Those are just a couple benefits that I've enjoyed in this new architecture, to begin with, and I'm sure there will be more.

Couldn't outrun the Gateware Plague

The first week back after winter break was uneventful for the most part. There were a few team meetings to get caught up, but mostly everyone was just continuing their previous work. I started on actual programming work rather than just research, and I wrote prototype interfaces for both GRasterSurface and GBlitter, as well as drafting unit tests for GRasterSurface. At that point though, the flu which had been picking people off one by one got to me and put me down for the count for friday and the entire weekend.

The delayed old-architecture Gateware Release.

Month: 3, Week: 2

Starting this month, my main priority (with Anthony) was to make a final release of Gateware on the old-architecture. As of right now, I am the person who is holding back with it as new release requires an implementation of all the new libraries into the demo (in this case it is GAudio3D). The whole process is done: the documentation updated, all the library files updated and uploaded. The biggest hindrance of last week was me being hit by the flu, so the progress was postponed by half a week.

Currently, I am working on the demo, but I ran into linking issues which Lari helps me to resolve. I completed a changelog, so as soon as I am done with the demo and upload the independent version to the google drive, we can call this a completed, tested release. The other final project teams are heavily dependent on it, so I am trying my best to be done with this process asap (some of my GAudio fixes are absolutely crucial).

Changelog for 7.0.A:

Known Issues in update 7.0.A
a. GWindow events ping several times instead of one per event on Linux.
b. GWindow threads are potentially unsafe and rarely prone to race condition crashes on Linux.
c. GAudio volume controls on Linux are currently not working as expected.
d. GAudio3D surround sound with full 5.1 support works only on Windows right now.
e. Minor popping sound with GMusic on Linux.

Fixed in update 7.0.A:
a. GWindow event system now responding and sending events on Linux
b. Large scale fix for memory leaks in GWindow on Linux
c. All of the memory leaks resolved for GAudio/GSound/GMusic/GAudio3D/GSound3D/GMusic3D on Windows and Mac, and majority of memory leaks resolved on Linux.
d. GSound and GMusic are getting instantly cleaned up when the count drops to 1 (which stops the audio from playing).
e. GSound and GMusic Stop() is now setting isPaused flag to false.
f. Interface of GAudio libraries got changed, some of the functions got renamed (check the interface headers).
g. Popping sound on repeated play of GSound should be resolved.
h. Majority of potential deadlocks resolved in GAudio libraries.
i. GAudio libraries are significantly more stable on all platforms.
j. Improved the performance of Stop() for GMusic.
k. GFile improved and interface updated with new functions.
l. A lot of refactoring done on all platforms (optimizations and/or compiler warnings resolved).

Plague Doctor's Journal - January 6th - 10th

January 6th:
      I arrived at 11:30 A.M. to this diseased place, returning from my winter tide's retreat.
      Signs of the disease began with our Vulkan engineer, who was feeling unwell and coughing in            fits. The beginning of the plague for certain, though I did not notice at the time.
      On this day I was given task to aid Alex in readying for a release of Gateware on the olden                  architecture, and began such before the day was out. Our time to do so was consumed primarily          by the meeting through which we were assigned the task, this day was primarily uneventful.

January 7th:
      After stopping for a morning meal I arrived at the office only to learn that Derrick had been taken        by the plague. An honorable sacrifice indeed, as he attempted to save our kind from the                        pestilence that had struck him down. It wasn't enough however, as Ryan was showing similar              signs of suffering by the end of the day. I spent this time oblivious, aiding Alex in clearing out            errors before we were ready to release gateware. I found him a step ahead of me throughout, and        was not as impactful as I might have hoped. By the end of the day we had near readied ourselves.

January 8th:
      Another casualty to the plague. Striking fast, it took Ryan with it on this day. Unforeseen it took          another! Chris had also fallen victim on this day, two of our own stricken with plague anew. On          this day Alex had also begun to show symptoms of the dreaded disease, but he was able to power        through his raising discomfort and aid me in releasing gateware to our users. A valiant effort                indeed.

January 9th:
      By now my empathy was waning. Three more victims stricken down. Alex had fallen after his            glorious stand on the previous day, Kai's dedication to his work proved weaker than the might of        fate, and Johnny was left without his teammate as Marc was overwhelmed from a distance, the            first non-gateware to be felled by our curse. Only Caio, Jadon and I remaining, this day was a              complete wash. Morale low, I found myself trying, failing, and trying yet again to install a new            program that simply refused to cooperate. A day without progress is a sad day indeed.

January 10th:
      Jadon fell as well on this day, but the dead returned! Chris found his way to the land of the living        on this day, as well as Ryan, who had been haunting the office until this point. Beaten by the                rogue program, I changed priorities to engage myself with the renewed member of the porting            team, and began work on porting GFile to the new architecture. Our team may be recovering, but        I fear that the spread of pestilence has left us and joined the world outside, to strike anew those            spared from our graces. I only thank the fates that I was not stricken, and pray that mine and                Caio's safety may continue.

Docker and Thread Pools

Coming back from break and jumping back on to Gitlab CI was a bit of an uphill struggle. I started my week trying to run Docker through my Ubuntu VM but I kept running into errors that lead me to empty stackoverflow threads and the day after that Docker just decided to completely fail me on Ubuntu. So, I blew up Ubuntu and loaded up Docker on my host OS; I'm not longer getting the strange errors like on Ubuntu, now I just have to fight with writing a good Docker image.

Meanwhile, I got a secondary assignment which was to write unit(stress) tests for GDaemon. GDaemon was a hot pile of intertwined confusion at first, but after I understood it, I began to break it... rather easily. So, when a 'std::cout' "fixed" an issue, that was about the time Lari decided to revisit GDaemon and make a revision. GDaemon is now so nice, I'm gonna feel bad when someone breaks it.

Working on porting GWindow for all platforms

Begun porting GWindow, Windows OS is complete, currently working on Linux OS, not so fun because X11 API is spitting out random error messages. Didn't really get much done this week because I was sick