I finished a lot of functions to make a long lasting robust library that will hopefully be very timeless. The operations provide a great base for anyone's game engine to give adequate information on detecting collisions. With over 80 functions, I feel most users will be satisfied with the library. The demo was a great way to test the functionality of the collision detection library and see it in action. The unit tests cover a lot of cases. I learned a lot in breaking down math problems such as knowing when to use the perpendicular vector of a cross product or its individual components to give me the largest area the vectors span in a specific plane and much mch more.
The team was also great and helped with any issues in the architecture or cross platform that I was unfamiliar with.
Friday, February 28, 2020
GCollision: What Went Wrong?
Working the past few months on the Collision Detection library came with intersting problems which could only be solved by putting in the time. Floating point precision is a lasting battle as we're trying to represent numbers bigger than the data types that hold them. The macros FLT_EPSILON and DBL_EPSILON are good values for comparisons -2 to -1 and 1 to 2. For near zero comparisons it would be better to use a constant different than those. For values less than -2 and greater than 2, a solution exists in the form a function. Another problem was covering all the unit tests for a query. Even with ~1,200 asserts for just one data type implementation, it doesn't feel enough.
Another thing was severely underestimating the time it takes to create a quality product with test driven development. I set out with a gargantuan library and trimmed a lot of by the end of these months of the project. I would have definitely chosen to make a semi-realistic to high goals instead of a somewhat impossible goal for the sake of my mental health and sleep.
Another thing was severely underestimating the time it takes to create a quality product with test driven development. I set out with a gargantuan library and trimmed a lot of by the end of these months of the project. I would have definitely chosen to make a semi-realistic to high goals instead of a somewhat impossible goal for the sake of my mental health and sleep.
GCollision: What Did I Do?
I researched and created a collision detection library for low-level collision detection. I created it using test-driven development using previously created math libraries: GVector and GMatrix. I did closest point to a shape for points, lines, rays, triangles, plane, spheres, capsules, AABBs, and OBBs. I implemented testing whether or not two shapes collide between those previously mentioned for the closest point queries. I also provided extended collision detection information such as minimal translation distance for shapes I deemed would be the most sought.
GAudio adventures during the last week
Month: 4 Week: 4 (Final Week)
Several bugs were found during the last week in preparation for presentation and the release. Luckily enough, with a little bit of crunch time I managed to resolve some of the bugs that were connected with Audio systems (my main concern) and attempted to help my teammates with issues I had some knowledge about.
Biggest problem of the week was GAudio3D failing in release on Windows problem due to compiler AND having some issues with stack memory on Linux. While the stack memory allocation issue was fairly straight forward (prevented the allocation falling out of stack and properly transfered data from 1 array to another); the other issue was not so simple. The bug only persisted on Windows and only in release which was impossible to track without some rewriting which brought the issue in the debug - then I found out the crashes were caused by base GAudio class accessing GAudio3D resources in destructor, and due to order of destruction - derived class (GAudio3D) was destroyed before base (GAudio).
I implemented a "hacky" solution to explicitly separate base class event generator from derived class event generator. The functions to access base generator were overridden in derived class for direct access (since the exposure of the base class is restricted to Protected) and the actual generator of GAudio3D was created separately. This allowed GMusic3D and GSound3D to explicitly connect to correct event generators during creation and this resolved the crashing issue. (All GAudio and GAudio3D classes are now operational and stable on Windows, Mac and Linux.)
Several bugs were found during the last week in preparation for presentation and the release. Luckily enough, with a little bit of crunch time I managed to resolve some of the bugs that were connected with Audio systems (my main concern) and attempted to help my teammates with issues I had some knowledge about.
Biggest problem of the week was GAudio3D failing in release on Windows problem due to compiler AND having some issues with stack memory on Linux. While the stack memory allocation issue was fairly straight forward (prevented the allocation falling out of stack and properly transfered data from 1 array to another); the other issue was not so simple. The bug only persisted on Windows and only in release which was impossible to track without some rewriting which brought the issue in the debug - then I found out the crashes were caused by base GAudio class accessing GAudio3D resources in destructor, and due to order of destruction - derived class (GAudio3D) was destroyed before base (GAudio).
I implemented a "hacky" solution to explicitly separate base class event generator from derived class event generator. The functions to access base generator were overridden in derived class for direct access (since the exposure of the base class is restricted to Protected) and the actual generator of GAudio3D was created separately. This allowed GMusic3D and GSound3D to explicitly connect to correct event generators during creation and this resolved the crashing issue. (All GAudio and GAudio3D classes are now operational and stable on Windows, Mac and Linux.)
Monday, February 24, 2020
What to do with a job you don't know how to do.
tldr: Read ALL the documentation.
I'm writing this at the end of my 3rd week of my 4th month on Gateware. Everything was on fire for a little bit, but it's all okay now. There's a few scorch marks and some things might have some embers still, but overall it's all coming together. It's crazy to see all our work come together in the end. What started as a huge daunting task list, is now. . . still a huge daunting task list. At least with DevOps that's the case. Though the porting is basically done, and new libraries are at the point of making demos, the CI/CD systems are still in their infancy, which is somewhat painful to type considering how many hours I poured into it, but I digress.
First off, what is DevOps and CI/CD? Going into Gateware, I was completely incapable of defining or even talking about these topics, however, now I feel much more confident that I know very little. Though, these terms have formal definitions that I should probably be providing here, I would rather describe them from my perspective. DevOps for me means, making the lives of developer, producers, and customers easier by streamlining the integration and deployment process. This is where the CI/CD comes into play. CI stands for continuous integration, and CD is continuous deployment. Usually, DevOps is a whole team and not just one man, so at the point of writing this post I was not able to get to the CD part implemented due to fighting so many issues on the way through setting up CI. See, I believe I had a particularly difficult time with my tasks due to the fact that I had to setup a stable system in an unstable environment. Having to decipher whether the build or tests failed because of my work, or because of the developer's work was a task on its own. Though this was a hindrance just about everyday, it really put not only my own, but everyone's communication skills through a lot of practice. Having to understand everyone's work and everyone's problems to the point of being able to build an automated system around all these moving parts was an interesting way to learn a lot of random things all at once all the time.
Now what does one do when faced with a task that they have no idea of where or how to start? The answer is actually surprisingly easy, you google it. On a serious note though, you google it. I did so much reading for the first two months that I would rack up about 30 unique tabs everyday just trying to understand what CI/CD was and how I could begin learning it. The next two months got better with the tabs, as in I would only pull up about 15 unique tabs a day. At the end of each week, I would take all the tabs I racked up and throw it into a bookmark folder. This made it very helpful to find things, "I knew I saw somewhere" and things, "I thought I read". All that reading didn't write my scripts, build my pipelines, or debug code that wasn't mine. No, that would've been too easy. All that reading simply told me how to do "things". Finding out what "things" I needed, to do the things I thought I needed to do was more complicated than I thought it'd be. Probably, because it was difficult enough finding out what I needed to do in the first place. Thankfully, I wasn't completely alone. I had a good friend, Greg W., who has had experience in DevOps, which made him a very helpful resource and a life-line when I found myself very lost, confused, and frustrated. He also helped me get into programming in the very beginning, so a special thanks to him.
My biggest hindrances were finding Docker Images, finding out what dependencies I actually needed to install, and learning the limits of the permissions in GitLab's runners. Mac was surprisingly easy, Linux was as difficult as usual, and Windows was a huge pain. First off, Windows doesn't come with a package manager so that was a headache until I was able to find a Docker Image with Chocolatey pre-installed. That same image also claimed to have MSVC and CMake installed but I quickly found that to be false. Mac was easiest because I was able to manually install everything on the Mac Mini so that the Mac jobs only had to run the build and compile scripts. Linux just had a long dependency list that required constant updating due to the porting process and new libraries being introduced (thanks, Vulkan). Then there were some issues with not being able to compile some libraries and therefor not being to run their tests. We initially thought we could just disable the library so that the dummy implementation would run, but that still caused all the Unit Tests to fail. So the solution was to create a #define system that would enable all libraries and all tests by default, and through my ruby script, which was honestly a joy to write, we could skip specified unit tests. On top of that fix, Lari began coming up with a fix that would test the test to see if it could run in the first place, if it couldn't then it would return a failure code and skip that unit test. This was the desirable solution as this allowed us to at least test the compilation of interfaces and their implementations.
At the end of the day, I believe I have laid a considerable amount of foundation, and I do desire to continue my work until I either get the pipeline into a self-sustaining state or until another DevOps engineer comes rolling in that can pick it up. Overall, this has been an enlightening experience filled with self-discovery and overcoming challenges.
I'm writing this at the end of my 3rd week of my 4th month on Gateware. Everything was on fire for a little bit, but it's all okay now. There's a few scorch marks and some things might have some embers still, but overall it's all coming together. It's crazy to see all our work come together in the end. What started as a huge daunting task list, is now. . . still a huge daunting task list. At least with DevOps that's the case. Though the porting is basically done, and new libraries are at the point of making demos, the CI/CD systems are still in their infancy, which is somewhat painful to type considering how many hours I poured into it, but I digress.
First off, what is DevOps and CI/CD? Going into Gateware, I was completely incapable of defining or even talking about these topics, however, now I feel much more confident that I know very little. Though, these terms have formal definitions that I should probably be providing here, I would rather describe them from my perspective. DevOps for me means, making the lives of developer, producers, and customers easier by streamlining the integration and deployment process. This is where the CI/CD comes into play. CI stands for continuous integration, and CD is continuous deployment. Usually, DevOps is a whole team and not just one man, so at the point of writing this post I was not able to get to the CD part implemented due to fighting so many issues on the way through setting up CI. See, I believe I had a particularly difficult time with my tasks due to the fact that I had to setup a stable system in an unstable environment. Having to decipher whether the build or tests failed because of my work, or because of the developer's work was a task on its own. Though this was a hindrance just about everyday, it really put not only my own, but everyone's communication skills through a lot of practice. Having to understand everyone's work and everyone's problems to the point of being able to build an automated system around all these moving parts was an interesting way to learn a lot of random things all at once all the time.
Now what does one do when faced with a task that they have no idea of where or how to start? The answer is actually surprisingly easy, you google it. On a serious note though, you google it. I did so much reading for the first two months that I would rack up about 30 unique tabs everyday just trying to understand what CI/CD was and how I could begin learning it. The next two months got better with the tabs, as in I would only pull up about 15 unique tabs a day. At the end of each week, I would take all the tabs I racked up and throw it into a bookmark folder. This made it very helpful to find things, "I knew I saw somewhere" and things, "I thought I read". All that reading didn't write my scripts, build my pipelines, or debug code that wasn't mine. No, that would've been too easy. All that reading simply told me how to do "things". Finding out what "things" I needed, to do the things I thought I needed to do was more complicated than I thought it'd be. Probably, because it was difficult enough finding out what I needed to do in the first place. Thankfully, I wasn't completely alone. I had a good friend, Greg W., who has had experience in DevOps, which made him a very helpful resource and a life-line when I found myself very lost, confused, and frustrated. He also helped me get into programming in the very beginning, so a special thanks to him.
My biggest hindrances were finding Docker Images, finding out what dependencies I actually needed to install, and learning the limits of the permissions in GitLab's runners. Mac was surprisingly easy, Linux was as difficult as usual, and Windows was a huge pain. First off, Windows doesn't come with a package manager so that was a headache until I was able to find a Docker Image with Chocolatey pre-installed. That same image also claimed to have MSVC and CMake installed but I quickly found that to be false. Mac was easiest because I was able to manually install everything on the Mac Mini so that the Mac jobs only had to run the build and compile scripts. Linux just had a long dependency list that required constant updating due to the porting process and new libraries being introduced (thanks, Vulkan). Then there were some issues with not being able to compile some libraries and therefor not being to run their tests. We initially thought we could just disable the library so that the dummy implementation would run, but that still caused all the Unit Tests to fail. So the solution was to create a #define system that would enable all libraries and all tests by default, and through my ruby script, which was honestly a joy to write, we could skip specified unit tests. On top of that fix, Lari began coming up with a fix that would test the test to see if it could run in the first place, if it couldn't then it would return a failure code and skip that unit test. This was the desirable solution as this allowed us to at least test the compilation of interfaces and their implementations.
At the end of the day, I believe I have laid a considerable amount of foundation, and I do desire to continue my work until I either get the pipeline into a self-sustaining state or until another DevOps engineer comes rolling in that can pick it up. Overall, this has been an enlightening experience filled with self-discovery and overcoming challenges.
When you tried so hard but you have to rewrite your library before you even write it the first time
I wrote some temporary test code as a proof-of-concept for GBlitter's functionality, but on explaining what I've been doing to Lari, it turns out I misunderstood how he wants the library to be structured and now I have to scrap and rewrite my test code. At least I hadn't started writing actual functions yet. It's always better to identify problems as early as possible. I'm confident now that I've properly understood what Lari wants from the library, and I believe that Lari is confident that I've understood as well. I spent an entire day just writing diagrams and notes about the structure and functionality, now I just have to write code to match.
Post-mortem
What I did:
1. Porting old libraries of Gateware into a new architecture
2. Writing a file concatenation tool that concatenates all implementation includes into one gigantic file
3. Assiting teammates when needed
What went wrong:
1. Porting the first ever library to the new architectue. This is because during the time we were still learning about the new architcture, and that library was GWindow and it contained cross-platform code. Since use of static stuff is limited I have come with a work around to solve the issue for all platforms.
2. Setting up new architecture project in Mac. This is because I did not know about objective files can be compiled with C++ code in it. Spent a couple days on setting up the project because of that.
3. Porting Linux libraries. X11's documentation:
What went right:
1. Begun the tool the first month. The tool was a really fun task, I enjoyed it. I'm glad I started early because in the final month I figured out my recursion algorithm caused stack overflow and was able to implement a fix quickly.
2. Porting on Windows. I LOVE WINDOWS, Win32 API is really well documented and we've been coding on Windows for so long. So porting windows library wasn't diffcult. Though there are some hinderances along the way. STATICS :(
1. Porting old libraries of Gateware into a new architecture
2. Writing a file concatenation tool that concatenates all implementation includes into one gigantic file
3. Assiting teammates when needed
What went wrong:
1. Porting the first ever library to the new architectue. This is because during the time we were still learning about the new architcture, and that library was GWindow and it contained cross-platform code. Since use of static stuff is limited I have come with a work around to solve the issue for all platforms.
2. Setting up new architecture project in Mac. This is because I did not know about objective files can be compiled with C++ code in it. Spent a couple days on setting up the project because of that.
3. Porting Linux libraries. X11's documentation:
What went right:
1. Begun the tool the first month. The tool was a really fun task, I enjoyed it. I'm glad I started early because in the final month I figured out my recursion algorithm caused stack overflow and was able to implement a fix quickly.
2. Porting on Windows. I LOVE WINDOWS, Win32 API is really well documented and we've been coding on Windows for so long. So porting windows library wasn't diffcult. Though there are some hinderances along the way. STATICS :(
Subscribe to:
Posts (Atom)