Colby Peck - Gateware Week 6: A New Pipeline

 My sixth week here on the Gateware team started out with finalizing a refactor of our float-comparison macros, as well as cleaning up some rather opaquely-worded doxygen documentation. When I first started on the task of patching a small hole in the GMatrix library, I was not expecting to get a task that would balloon out into a full-week ordeal, but such is the way of things I suppose. 

With all of that finally finished, I was ready to move on to my next big task: a pipeline refactor. Lari and I went over the desired final state of the pipeline on Monday. We made several tasks on gitlab defining incremental deployable changes that would ultimately take us to that desired final pipeline (#214-#219). TLDR, we want to set up an automated release process that will: 

• Contain a new ‘release’ branch
• Generate a new single header and run all unit tests on it whenever changes are pushed, then push that single header to the repo if all tests pass
• Generate and push new doxygen documentation any time a new valid single header is generated
• Update the release branch any time a new valid single header is generated

I got started on these tasks on Tuesday. I managed to make a project that tests the current single header and partially integrate it into the pipeline much faster than I had anticipated; it only took about a day. Of course, as soon as I did, I ran headfirst into a bug. It turns out that the mac and linux DUMMY_MAIN projects weren’t testing the single header like they were supposed to, and the single-header compiles into a broken state on Linux. As in, projects that include a single-header generated on Linux will not compile. So, Gateware’s Linux release is broken. It has been for some time, but we didn’t know until now. 

Even if this bug wasn’t more important than refactoring the pipeline, it prevents me from knowing if the pipeline is working on Linux. So I’m diving into the compiler once again. The issue is that the files in the Core directory are ordered differently on Linux than they are on Windows/Mac, and the compiler currently expects them to be ordered the way they are on Windows/Mac. My current plan is to add a patch to the current compiler (hard-coding a specified order for Core), and to create a task for a more comprehensive compiler refactor (to be picked up by myself or someone else in the future). The compiler is currently very brittle; it’s ordering the directories manually and the files within directories are ordered by the number of dependencies they have. It is assumed on good faith that writing the files with more dependencies first will result in a good single header (this is already not the case with Core). A more robust compiler would actually build and traverse a dependency hierarchy, but that’s a much more comprehensive refactor. I don’t like kicking the can down the road, but I can only work on one thing at a time. So for now, patching and putting off the refactor is probably the way to go.