tldr: Read ALL the documentation.
I'm writing this at the end of my 3rd week of my 4th month on Gateware. Everything was on fire for a little bit, but it's all okay now. There's a few scorch marks and some things might have some embers still, but overall it's all coming together. It's crazy to see all our work come together in the end. What started as a huge daunting task list, is now. . . still a huge daunting task list. At least with DevOps that's the case. Though the porting is basically done, and new libraries are at the point of making demos, the CI/CD systems are still in their infancy, which is somewhat painful to type considering how many hours I poured into it, but I digress.
First off, what is DevOps and CI/CD? Going into Gateware, I was completely incapable of defining or even talking about these topics, however, now I feel much more confident that I know very little. Though, these terms have formal definitions that I should probably be providing here, I would rather describe them from my perspective. DevOps for me means, making the lives of developer, producers, and customers easier by streamlining the integration and deployment process. This is where the CI/CD comes into play. CI stands for continuous integration, and CD is continuous deployment. Usually, DevOps is a whole team and not just one man, so at the point of writing this post I was not able to get to the CD part implemented due to fighting so many issues on the way through setting up CI. See, I believe I had a particularly difficult time with my tasks due to the fact that I had to setup a stable system in an unstable environment. Having to decipher whether the build or tests failed because of my work, or because of the developer's work was a task on its own. Though this was a hindrance just about everyday, it really put not only my own, but everyone's communication skills through a lot of practice. Having to understand everyone's work and everyone's problems to the point of being able to build an automated system around all these moving parts was an interesting way to learn a lot of random things all at once all the time.
Now what does one do when faced with a task that they have no idea of where or how to start? The answer is actually surprisingly easy, you google it. On a serious note though, you google it. I did so much reading for the first two months that I would rack up about 30 unique tabs everyday just trying to understand what CI/CD was and how I could begin learning it. The next two months got better with the tabs, as in I would only pull up about 15 unique tabs a day. At the end of each week, I would take all the tabs I racked up and throw it into a bookmark folder. This made it very helpful to find things, "I knew I saw somewhere" and things, "I thought I read". All that reading didn't write my scripts, build my pipelines, or debug code that wasn't mine. No, that would've been too easy. All that reading simply told me how to do "things". Finding out what "things" I needed, to do the things I thought I needed to do was more complicated than I thought it'd be. Probably, because it was difficult enough finding out what I needed to do in the first place. Thankfully, I wasn't completely alone. I had a good friend, Greg W., who has had experience in DevOps, which made him a very helpful resource and a life-line when I found myself very lost, confused, and frustrated. He also helped me get into programming in the very beginning, so a special thanks to him.
My biggest hindrances were finding Docker Images, finding out what dependencies I actually needed to install, and learning the limits of the permissions in GitLab's runners. Mac was surprisingly easy, Linux was as difficult as usual, and Windows was a huge pain. First off, Windows doesn't come with a package manager so that was a headache until I was able to find a Docker Image with Chocolatey pre-installed. That same image also claimed to have MSVC and CMake installed but I quickly found that to be false. Mac was easiest because I was able to manually install everything on the Mac Mini so that the Mac jobs only had to run the build and compile scripts. Linux just had a long dependency list that required constant updating due to the porting process and new libraries being introduced (thanks, Vulkan). Then there were some issues with not being able to compile some libraries and therefor not being to run their tests. We initially thought we could just disable the library so that the dummy implementation would run, but that still caused all the Unit Tests to fail. So the solution was to create a #define system that would enable all libraries and all tests by default, and through my ruby script, which was honestly a joy to write, we could skip specified unit tests. On top of that fix, Lari began coming up with a fix that would test the test to see if it could run in the first place, if it couldn't then it would return a failure code and skip that unit test. This was the desirable solution as this allowed us to at least test the compilation of interfaces and their implementations.
At the end of the day, I believe I have laid a considerable amount of foundation, and I do desire to continue my work until I either get the pipeline into a self-sustaining state or until another DevOps engineer comes rolling in that can pick it up. Overall, this has been an enlightening experience filled with self-discovery and overcoming challenges.
No comments:
Post a Comment