I finished a lot of functions to make a long lasting robust library that will hopefully be very timeless. The operations provide a great base for anyone's game engine to give adequate information on detecting collisions. With over 80 functions, I feel most users will be satisfied with the library. The demo was a great way to test the functionality of the collision detection library and see it in action. The unit tests cover a lot of cases. I learned a lot in breaking down math problems such as knowing when to use the perpendicular vector of a cross product or its individual components to give me the largest area the vectors span in a specific plane and much mch more.
The team was also great and helped with any issues in the architecture or cross platform that I was unfamiliar with.
Friday, February 28, 2020
GCollision: What Went Wrong?
Working the past few months on the Collision Detection library came with intersting problems which could only be solved by putting in the time. Floating point precision is a lasting battle as we're trying to represent numbers bigger than the data types that hold them. The macros FLT_EPSILON and DBL_EPSILON are good values for comparisons -2 to -1 and 1 to 2. For near zero comparisons it would be better to use a constant different than those. For values less than -2 and greater than 2, a solution exists in the form a function. Another problem was covering all the unit tests for a query. Even with ~1,200 asserts for just one data type implementation, it doesn't feel enough.
Another thing was severely underestimating the time it takes to create a quality product with test driven development. I set out with a gargantuan library and trimmed a lot of by the end of these months of the project. I would have definitely chosen to make a semi-realistic to high goals instead of a somewhat impossible goal for the sake of my mental health and sleep.
Another thing was severely underestimating the time it takes to create a quality product with test driven development. I set out with a gargantuan library and trimmed a lot of by the end of these months of the project. I would have definitely chosen to make a semi-realistic to high goals instead of a somewhat impossible goal for the sake of my mental health and sleep.
GCollision: What Did I Do?
I researched and created a collision detection library for low-level collision detection. I created it using test-driven development using previously created math libraries: GVector and GMatrix. I did closest point to a shape for points, lines, rays, triangles, plane, spheres, capsules, AABBs, and OBBs. I implemented testing whether or not two shapes collide between those previously mentioned for the closest point queries. I also provided extended collision detection information such as minimal translation distance for shapes I deemed would be the most sought.
GAudio adventures during the last week
Month: 4 Week: 4 (Final Week)
Several bugs were found during the last week in preparation for presentation and the release. Luckily enough, with a little bit of crunch time I managed to resolve some of the bugs that were connected with Audio systems (my main concern) and attempted to help my teammates with issues I had some knowledge about.
Biggest problem of the week was GAudio3D failing in release on Windows problem due to compiler AND having some issues with stack memory on Linux. While the stack memory allocation issue was fairly straight forward (prevented the allocation falling out of stack and properly transfered data from 1 array to another); the other issue was not so simple. The bug only persisted on Windows and only in release which was impossible to track without some rewriting which brought the issue in the debug - then I found out the crashes were caused by base GAudio class accessing GAudio3D resources in destructor, and due to order of destruction - derived class (GAudio3D) was destroyed before base (GAudio).
I implemented a "hacky" solution to explicitly separate base class event generator from derived class event generator. The functions to access base generator were overridden in derived class for direct access (since the exposure of the base class is restricted to Protected) and the actual generator of GAudio3D was created separately. This allowed GMusic3D and GSound3D to explicitly connect to correct event generators during creation and this resolved the crashing issue. (All GAudio and GAudio3D classes are now operational and stable on Windows, Mac and Linux.)
Several bugs were found during the last week in preparation for presentation and the release. Luckily enough, with a little bit of crunch time I managed to resolve some of the bugs that were connected with Audio systems (my main concern) and attempted to help my teammates with issues I had some knowledge about.
Biggest problem of the week was GAudio3D failing in release on Windows problem due to compiler AND having some issues with stack memory on Linux. While the stack memory allocation issue was fairly straight forward (prevented the allocation falling out of stack and properly transfered data from 1 array to another); the other issue was not so simple. The bug only persisted on Windows and only in release which was impossible to track without some rewriting which brought the issue in the debug - then I found out the crashes were caused by base GAudio class accessing GAudio3D resources in destructor, and due to order of destruction - derived class (GAudio3D) was destroyed before base (GAudio).
I implemented a "hacky" solution to explicitly separate base class event generator from derived class event generator. The functions to access base generator were overridden in derived class for direct access (since the exposure of the base class is restricted to Protected) and the actual generator of GAudio3D was created separately. This allowed GMusic3D and GSound3D to explicitly connect to correct event generators during creation and this resolved the crashing issue. (All GAudio and GAudio3D classes are now operational and stable on Windows, Mac and Linux.)
Monday, February 24, 2020
What to do with a job you don't know how to do.
tldr: Read ALL the documentation.
I'm writing this at the end of my 3rd week of my 4th month on Gateware. Everything was on fire for a little bit, but it's all okay now. There's a few scorch marks and some things might have some embers still, but overall it's all coming together. It's crazy to see all our work come together in the end. What started as a huge daunting task list, is now. . . still a huge daunting task list. At least with DevOps that's the case. Though the porting is basically done, and new libraries are at the point of making demos, the CI/CD systems are still in their infancy, which is somewhat painful to type considering how many hours I poured into it, but I digress.
First off, what is DevOps and CI/CD? Going into Gateware, I was completely incapable of defining or even talking about these topics, however, now I feel much more confident that I know very little. Though, these terms have formal definitions that I should probably be providing here, I would rather describe them from my perspective. DevOps for me means, making the lives of developer, producers, and customers easier by streamlining the integration and deployment process. This is where the CI/CD comes into play. CI stands for continuous integration, and CD is continuous deployment. Usually, DevOps is a whole team and not just one man, so at the point of writing this post I was not able to get to the CD part implemented due to fighting so many issues on the way through setting up CI. See, I believe I had a particularly difficult time with my tasks due to the fact that I had to setup a stable system in an unstable environment. Having to decipher whether the build or tests failed because of my work, or because of the developer's work was a task on its own. Though this was a hindrance just about everyday, it really put not only my own, but everyone's communication skills through a lot of practice. Having to understand everyone's work and everyone's problems to the point of being able to build an automated system around all these moving parts was an interesting way to learn a lot of random things all at once all the time.
Now what does one do when faced with a task that they have no idea of where or how to start? The answer is actually surprisingly easy, you google it. On a serious note though, you google it. I did so much reading for the first two months that I would rack up about 30 unique tabs everyday just trying to understand what CI/CD was and how I could begin learning it. The next two months got better with the tabs, as in I would only pull up about 15 unique tabs a day. At the end of each week, I would take all the tabs I racked up and throw it into a bookmark folder. This made it very helpful to find things, "I knew I saw somewhere" and things, "I thought I read". All that reading didn't write my scripts, build my pipelines, or debug code that wasn't mine. No, that would've been too easy. All that reading simply told me how to do "things". Finding out what "things" I needed, to do the things I thought I needed to do was more complicated than I thought it'd be. Probably, because it was difficult enough finding out what I needed to do in the first place. Thankfully, I wasn't completely alone. I had a good friend, Greg W., who has had experience in DevOps, which made him a very helpful resource and a life-line when I found myself very lost, confused, and frustrated. He also helped me get into programming in the very beginning, so a special thanks to him.
My biggest hindrances were finding Docker Images, finding out what dependencies I actually needed to install, and learning the limits of the permissions in GitLab's runners. Mac was surprisingly easy, Linux was as difficult as usual, and Windows was a huge pain. First off, Windows doesn't come with a package manager so that was a headache until I was able to find a Docker Image with Chocolatey pre-installed. That same image also claimed to have MSVC and CMake installed but I quickly found that to be false. Mac was easiest because I was able to manually install everything on the Mac Mini so that the Mac jobs only had to run the build and compile scripts. Linux just had a long dependency list that required constant updating due to the porting process and new libraries being introduced (thanks, Vulkan). Then there were some issues with not being able to compile some libraries and therefor not being to run their tests. We initially thought we could just disable the library so that the dummy implementation would run, but that still caused all the Unit Tests to fail. So the solution was to create a #define system that would enable all libraries and all tests by default, and through my ruby script, which was honestly a joy to write, we could skip specified unit tests. On top of that fix, Lari began coming up with a fix that would test the test to see if it could run in the first place, if it couldn't then it would return a failure code and skip that unit test. This was the desirable solution as this allowed us to at least test the compilation of interfaces and their implementations.
At the end of the day, I believe I have laid a considerable amount of foundation, and I do desire to continue my work until I either get the pipeline into a self-sustaining state or until another DevOps engineer comes rolling in that can pick it up. Overall, this has been an enlightening experience filled with self-discovery and overcoming challenges.
I'm writing this at the end of my 3rd week of my 4th month on Gateware. Everything was on fire for a little bit, but it's all okay now. There's a few scorch marks and some things might have some embers still, but overall it's all coming together. It's crazy to see all our work come together in the end. What started as a huge daunting task list, is now. . . still a huge daunting task list. At least with DevOps that's the case. Though the porting is basically done, and new libraries are at the point of making demos, the CI/CD systems are still in their infancy, which is somewhat painful to type considering how many hours I poured into it, but I digress.
First off, what is DevOps and CI/CD? Going into Gateware, I was completely incapable of defining or even talking about these topics, however, now I feel much more confident that I know very little. Though, these terms have formal definitions that I should probably be providing here, I would rather describe them from my perspective. DevOps for me means, making the lives of developer, producers, and customers easier by streamlining the integration and deployment process. This is where the CI/CD comes into play. CI stands for continuous integration, and CD is continuous deployment. Usually, DevOps is a whole team and not just one man, so at the point of writing this post I was not able to get to the CD part implemented due to fighting so many issues on the way through setting up CI. See, I believe I had a particularly difficult time with my tasks due to the fact that I had to setup a stable system in an unstable environment. Having to decipher whether the build or tests failed because of my work, or because of the developer's work was a task on its own. Though this was a hindrance just about everyday, it really put not only my own, but everyone's communication skills through a lot of practice. Having to understand everyone's work and everyone's problems to the point of being able to build an automated system around all these moving parts was an interesting way to learn a lot of random things all at once all the time.
Now what does one do when faced with a task that they have no idea of where or how to start? The answer is actually surprisingly easy, you google it. On a serious note though, you google it. I did so much reading for the first two months that I would rack up about 30 unique tabs everyday just trying to understand what CI/CD was and how I could begin learning it. The next two months got better with the tabs, as in I would only pull up about 15 unique tabs a day. At the end of each week, I would take all the tabs I racked up and throw it into a bookmark folder. This made it very helpful to find things, "I knew I saw somewhere" and things, "I thought I read". All that reading didn't write my scripts, build my pipelines, or debug code that wasn't mine. No, that would've been too easy. All that reading simply told me how to do "things". Finding out what "things" I needed, to do the things I thought I needed to do was more complicated than I thought it'd be. Probably, because it was difficult enough finding out what I needed to do in the first place. Thankfully, I wasn't completely alone. I had a good friend, Greg W., who has had experience in DevOps, which made him a very helpful resource and a life-line when I found myself very lost, confused, and frustrated. He also helped me get into programming in the very beginning, so a special thanks to him.
My biggest hindrances were finding Docker Images, finding out what dependencies I actually needed to install, and learning the limits of the permissions in GitLab's runners. Mac was surprisingly easy, Linux was as difficult as usual, and Windows was a huge pain. First off, Windows doesn't come with a package manager so that was a headache until I was able to find a Docker Image with Chocolatey pre-installed. That same image also claimed to have MSVC and CMake installed but I quickly found that to be false. Mac was easiest because I was able to manually install everything on the Mac Mini so that the Mac jobs only had to run the build and compile scripts. Linux just had a long dependency list that required constant updating due to the porting process and new libraries being introduced (thanks, Vulkan). Then there were some issues with not being able to compile some libraries and therefor not being to run their tests. We initially thought we could just disable the library so that the dummy implementation would run, but that still caused all the Unit Tests to fail. So the solution was to create a #define system that would enable all libraries and all tests by default, and through my ruby script, which was honestly a joy to write, we could skip specified unit tests. On top of that fix, Lari began coming up with a fix that would test the test to see if it could run in the first place, if it couldn't then it would return a failure code and skip that unit test. This was the desirable solution as this allowed us to at least test the compilation of interfaces and their implementations.
At the end of the day, I believe I have laid a considerable amount of foundation, and I do desire to continue my work until I either get the pipeline into a self-sustaining state or until another DevOps engineer comes rolling in that can pick it up. Overall, this has been an enlightening experience filled with self-discovery and overcoming challenges.
When you tried so hard but you have to rewrite your library before you even write it the first time
I wrote some temporary test code as a proof-of-concept for GBlitter's functionality, but on explaining what I've been doing to Lari, it turns out I misunderstood how he wants the library to be structured and now I have to scrap and rewrite my test code. At least I hadn't started writing actual functions yet. It's always better to identify problems as early as possible. I'm confident now that I've properly understood what Lari wants from the library, and I believe that Lari is confident that I've understood as well. I spent an entire day just writing diagrams and notes about the structure and functionality, now I just have to write code to match.
Post-mortem
What I did:
1. Porting old libraries of Gateware into a new architecture
2. Writing a file concatenation tool that concatenates all implementation includes into one gigantic file
3. Assiting teammates when needed
What went wrong:
1. Porting the first ever library to the new architectue. This is because during the time we were still learning about the new architcture, and that library was GWindow and it contained cross-platform code. Since use of static stuff is limited I have come with a work around to solve the issue for all platforms.
2. Setting up new architecture project in Mac. This is because I did not know about objective files can be compiled with C++ code in it. Spent a couple days on setting up the project because of that.
3. Porting Linux libraries. X11's documentation:
What went right:
1. Begun the tool the first month. The tool was a really fun task, I enjoyed it. I'm glad I started early because in the final month I figured out my recursion algorithm caused stack overflow and was able to implement a fix quickly.
2. Porting on Windows. I LOVE WINDOWS, Win32 API is really well documented and we've been coding on Windows for so long. So porting windows library wasn't diffcult. Though there are some hinderances along the way. STATICS :(
1. Porting old libraries of Gateware into a new architecture
2. Writing a file concatenation tool that concatenates all implementation includes into one gigantic file
3. Assiting teammates when needed
What went wrong:
1. Porting the first ever library to the new architectue. This is because during the time we were still learning about the new architcture, and that library was GWindow and it contained cross-platform code. Since use of static stuff is limited I have come with a work around to solve the issue for all platforms.
2. Setting up new architecture project in Mac. This is because I did not know about objective files can be compiled with C++ code in it. Spent a couple days on setting up the project because of that.
3. Porting Linux libraries. X11's documentation:
What went right:
1. Begun the tool the first month. The tool was a really fun task, I enjoyed it. I'm glad I started early because in the final month I figured out my recursion algorithm caused stack overflow and was able to implement a fix quickly.
2. Porting on Windows. I LOVE WINDOWS, Win32 API is really well documented and we've been coding on Windows for so long. So porting windows library wasn't diffcult. Though there are some hinderances along the way. STATICS :(
Post-mortem Chris Kennedy
Christopher Kennedy
What did I do?
What did I do?
- Gateware Bible
- Started ports
- Rewrote tutorial documents
- Loaded Linux on Gateware macs
- Wrote a gtl class
- I relearned technical communication
- I have a great portfolio peace in the bible and supplementary documents
- Multiplatform development
- Learning new arch was interesting
- Multiplatform development
- porting from old arch to new arch was interesting
- time crunch
Friday, February 21, 2020
GAudio and new architecture
Month:4 Week:2-3
I had to delay with this post due to being mid-progress on an issue I've been stuck for some time which I could not explain it without a proper research/debugging. GAudio is now completely ported to new architecture and at this point it is completely rewritten. GAudio is now event-based and completely asynchronous unlike on old architecture.
GAudio had significant issues with volume controls on Mac and Linux. While Mac solution took me a day at most (had to write and algorithm to pan 6 channels to 1 float value), linux was causing significantly more problems. I couldn't obtain a proper audio sink (where it writes a volume to) without affecting other sounds/music objects in the future. I attempted to create virtual sinks and connect using several pulseaudio callbacks without any success.
Only about now I figured out the solution with obtaining a proper sink index and even then it still doesn't work as expected (per-channel volume controls make everything play in mono and not stereo). This is due to our system internally using 6 channels (5.1 support) and while there is a fallback on Windows and Mac (custom-made), Linux doesn't have a proper fallback. I could make the system work with 2 channels for stereo support and this is a huge breakthrough as now I know how to deal with it. My next task is to implement a proper fallback from 5.1 to stereo to mono, so the setchannels controls would properly set the channel volumes AND our GAudio3D system would be able to work on stereo mode. (5.1 hardware support was not tested on Mac and Linux as of right now but it does work on software level.)
I had to delay with this post due to being mid-progress on an issue I've been stuck for some time which I could not explain it without a proper research/debugging. GAudio is now completely ported to new architecture and at this point it is completely rewritten. GAudio is now event-based and completely asynchronous unlike on old architecture.
GAudio had significant issues with volume controls on Mac and Linux. While Mac solution took me a day at most (had to write and algorithm to pan 6 channels to 1 float value), linux was causing significantly more problems. I couldn't obtain a proper audio sink (where it writes a volume to) without affecting other sounds/music objects in the future. I attempted to create virtual sinks and connect using several pulseaudio callbacks without any success.
Only about now I figured out the solution with obtaining a proper sink index and even then it still doesn't work as expected (per-channel volume controls make everything play in mono and not stereo). This is due to our system internally using 6 channels (5.1 support) and while there is a fallback on Windows and Mac (custom-made), Linux doesn't have a proper fallback. I could make the system work with 2 channels for stereo support and this is a huge breakthrough as now I know how to deal with it. My next task is to implement a proper fallback from 5.1 to stereo to mono, so the setchannels controls would properly set the channel volumes AND our GAudio3D system would be able to work on stereo mode. (5.1 hardware support was not tested on Mac and Linux as of right now but it does work on software level.)
Post-mortem thoughts
From a generalist position to almost completely being audio programmer:
Achieved tasks:
-Memory leaks found and resolved on all platforms in Audio (was leaking everything on each platform)
-Fixed reference counting logic on each platform in Audio which caused crashes and memory leaks
-Fixed crashes due to unsafe pointer usage in Audio libraries
-Memory leaks found and resolved on all platforms in Audio (was leaking everything on each platform)
-Fixed reference counting logic on each platform in Audio which caused crashes and memory leaks
-Fixed crashes due to unsafe pointer usage in Audio libraries
-Fixed crashes due to multi-threading communications with audio libraries
-Fixed a lot of issued with Music streaming loop (dirty buffers, main thread delays and slowdowns)
-Ported GAudio, GSound, GMusic and GInput libraries to new architecture
-Wrote or completely refactored unit tests for GDaemon, GInput and GAudio classes
-Significantly improved memory usage of Audio libraries
-Wrote or completely refactored unit tests for GDaemon, GInput and GAudio classes
-Significantly improved memory usage of Audio libraries
-Significantly improved the perfomance of Audio libraries
-Audio classes were completely rewritten for a completely asynchronous and event-based system
Issues and incomplete tasks:
-Despite being my one of the main tasks, I never managed to implement full 5.1 support on Linux and Mac due to more critical issues mentioned above being a higher priority.
-Volume controls on Linux side is a huge pain and currently not implemented (NOTE: This is most likely going to be fixed before post-mortem)
-Despite being my one of the main tasks, I never managed to implement full 5.1 support on Linux and Mac due to more critical issues mentioned above being a higher priority.
-Volume controls on Linux side is a huge pain and currently not implemented (NOTE: This is most likely going to be fixed before post-mortem)
-A lot of time was spent on efficient reference counting system which is no longer used on the new architecture (Despite that, it is currently in use by 2 final project teams)
Goonery Concluded / post mortem
Things that I did over the course of gateware:
*Purged memory leaks from Linux and mac
*Fixed the event system for GWindow on Linux
*Did a special release of Gateware for a team
*Failed to cleanse the last leak in holy fire
*Released Gateware / experienced plague
*Ported GFile to the new Architecture
*Ported GLog to the new Architecture
*Learned that our HP laptops have strange raw scan codes on their keys
*Enabled Mousewheel support for mac and windows
*Sent the input classes to a new namespace / folder / the shadow realm
*Worked on porting the demo
Things that were good:
*Doing the memory leak purge both taught me a new operating system and tools and helped improve my debugging skills, forcing me to figure out several race condition errors.
*Fixing the event system was easier than I could have hoped
*Doing the special release was a very relaxing side duty, I got to feel helpful
*Porting GFile was good experience with sequencing changes to best reflect a working environment and active debug while making changes
Things that weren't:
*X11 has documentation except when it doesn't
*The Plague
*Getting bamboozled by scanning key codes only to learn our code was fine
*Mousewheel was harder than it had any right to be on mac
*Purged memory leaks from Linux and mac
*Fixed the event system for GWindow on Linux
*Did a special release of Gateware for a team
*Failed to cleanse the last leak in holy fire
*Released Gateware / experienced plague
*Ported GFile to the new Architecture
*Ported GLog to the new Architecture
*Learned that our HP laptops have strange raw scan codes on their keys
*Enabled Mousewheel support for mac and windows
*Sent the input classes to a new namespace / folder / the shadow realm
*Worked on porting the demo
Things that were good:
*Doing the memory leak purge both taught me a new operating system and tools and helped improve my debugging skills, forcing me to figure out several race condition errors.
*Fixing the event system was easier than I could have hoped
*Doing the special release was a very relaxing side duty, I got to feel helpful
*Porting GFile was good experience with sequencing changes to best reflect a working environment and active debug while making changes
Things that weren't:
*X11 has documentation except when it doesn't
*The Plague
*Getting bamboozled by scanning key codes only to learn our code was fine
*Mousewheel was harder than it had any right to be on mac
Wednesday, February 19, 2020
More tweaks on the tool
The tool now is much more organized than the one in the previous post. This is the new updated code for Preprocess method. The code is much more cleaner, but theres additional functionalities, one of which is that if any files that are included using "" and the tool isn't able to find it. It will output an error message and write a comment that says the file might be an external file that is not part of Gateware.
void File::Preprocess(FileHeap& headerHeap, FileHeap& sourceHeap)
{
GW::SYSTEM::GFile gFile;
gFile.Create();
// iterate through the raw data and read line by line until we found an include
for (auto iter = std::sregex_iterator(rawData.begin(), rawData.end(), REGEX::PoundInclude); iter != std::sregex_iterator(); ++iter)
{
std::smatch match = *iter;
std::string lineString = match.str(0);
std::string includeDirectory, includeFileName;
size_t index = lineString.find_last_of("/");
// there is a file directory
if (index != std::string::npos)
{
#if defined (_WIN32)
includeDirectory = path + '\\' + lineString.substr(lineString.find_first_of("\"") + 1, index - lineString.find_first_of("\"") - 1);
#elif defined(__APPLE__) || defined(__linux__)
includeDirectory = path + '/' + lineString.substr(lineString.find_first_of("\"") + 1, index - lineString.find_first_of("\"") - 1);
#endif
includeFileName = lineString.substr(index + 1, lineString.find_last_of("\"") - index - 1);
gFile.SetCurrentWorkingDirectory(includeDirectory.data());
std::string temp; temp.resize(260);
gFile.GetCurrentWorkingDirectory(const_cast<char*>(temp.data()), 260);
includeDirectory.clear();
#if defined (_WIN32)
includeDirectory = temp.substr(0, temp.find_last_of('\\'));
#elif defined(__APPLE__) || defined(__linux__)
includeDirectory = temp.substr(0, temp.find_last_of('/'));
#endif
}
// the file directory is in the same directory as this file
else
{
index = lineString.find_first_of("\"");
includeDirectory = path;
includeFileName = lineString.substr(index + 1, lineString.find_last_of("\"") - index - 1);
}
// reset the working directory back to this files path
gFile.SetCurrentWorkingDirectory(path.data());
std::string ext = GetFileExtension(includeFileName);
if (ext == "h")
{
auto headerFile = headerHeap.Find(includeFileName);
if (!headerFile)
includeFiles.push_back(headerHeap.Create(includeDirectory, includeFileName));
else
includeFiles.push_back(headerFile);
}
else if (ext == "hpp")
{
auto sourceFile = sourceHeap.Find(includeFileName);
if (!sourceFile)
includeFiles.push_back(sourceHeap.Create(includeDirectory, includeFileName));
else
includeFiles.push_back(sourceFile);
}
}
for (size_t i = 0; i < includeFiles.size(); ++i)
{
includeFiles[i]->Preprocess(headerHeap, sourceHeap);
}
auto RetriveFileName = [](const std::string& fileName) -> std::string
{
std::string truncatedName;
size_t index = fileName.find_last_of("/");
// there is a file directory, if thats the case, we just want the name of the file and remove the last "
if (index != std::string::npos)
{
truncatedName = fileName.substr(fileName.find_last_of("/") + 1);
truncatedName.erase(truncatedName.size() - 1);
}
else // there is no directory for this file, remove the #include and ""
{
truncatedName = fileName.substr(fileName.find_first_of("\"") + 1);
truncatedName.erase(truncatedName.size() - 1);
}
return truncatedName;
};
std::string newRawData = rawData; // i need a place holder for rawData is because regex_iter will become invalid if original content were modified
for (auto iter = std::sregex_iterator(rawData.begin(), rawData.end(), REGEX::PoundInclude); iter != std::sregex_iterator(); ++iter)
{
std::smatch match = *iter;
std::string lineString = match.str(0);
std::string truncatedString = RetriveFileName(lineString);
std::regex regex(lineString);
auto headerFile = headerHeap.Find(truncatedString);
if (headerFile)
{
if (headerFile->isExternalFile)
{
std::string substitution = lineString + "\t// This file might be an external library file that is not part of Gateware!";
newRawData = std::regex_replace(newRawData, regex, substitution);
continue;
}
else
{
++headerFile->dependencyCount;
std::string substitution = "";
//std::string substitution = "\b"; // Produce weird characters...idk why
newRawData = std::regex_replace(newRawData, regex, substitution);
continue;
}
}
auto sourceFile = sourceHeap.Find(truncatedString);
if (sourceFile)
newRawData = std::regex_replace(newRawData, regex, sourceFile->GetRawData());
}
rawData = std::move(newRawData);
}
Monday, February 17, 2020
GRasterSurface isn't perfect but it's time to move on
GRasterSurface still has some improvements that could be made, but it works, mostly. It doesn't draw on Mac, but my proof of concept in the first month also didn't, and after two and a half weeks of searches, I wasn't able to get anything remotely close to useful answers about the problem, so this was expected. It has to work in a non-intuitive way on Linux because of X11 limitations, but that's beyond my control. The only major feature that didn't make the cut in the time I was given is parallelized processing in SmartUpdateSurface, and Lari planned for that to be done if there was time anyway.
Overall, GRasterSurface is in the state it was expected to reach. Lari has now officially moved me onto GBlitter and I've spent most of this week working on the GBlitter interface. It's a good thing this library doesn't need platform-specific implementations, because I only have about a month left to get it finished in.
Overall, GRasterSurface is in the state it was expected to reach. Lari has now officially moved me onto GBlitter and I've spent most of this week working on the GBlitter interface. It's a good thing this library doesn't need platform-specific implementations, because I only have about a month left to get it finished in.
Tuesday, February 11, 2020
C++11 Architecture ported
All of the old library is ported and there are new libraries coming in, Going back to work on the single header generateter, found out that my recursive algorithm doesnt work anymore because stack memory is overflowed by it. I fixed this by implementing a technique called dynamic programming or memoization. Here is some code snippets
void File::Preprocess(const std::map<std::string, File*>& headerFiles, const std::map<std::string, const File*>& implFiles)
{
auto RetriveFileName = [](const std::string& fileName) -> std::string
{
std::string truncatedName;
size_t index = fileName.find_last_of("/");
// there is a file directory, if thats the case, we just want the name of the file and remove the last "
if (index != std::string::npos)
{
truncatedName = fileName.substr(fileName.find_last_of("/") + 1);
truncatedName.erase(truncatedName.size() - 1);
}
else // there is no directory for this file, remove the #include and ""
{
truncatedName = fileName.substr(fileName.find_first_of("\"") + 1);
truncatedName.erase(truncatedName.size() - 1);
}
return truncatedName;
};
// INCREMENT FILTER PASS IN CASE OF EXTRA DIRECTORY DEPTH
#define FILTER_PASS 6
// 3 passes
// First pass is replace #include library.hpp
// Second pass is replace #include library_platformOS.hpp
// Third pass is replace any #include in library_platformOS.hpp
// rest of the filter is for safety check
for (int i = 0; i < FILTER_PASS; ++i)
{
std::string newRawData = rawData; // i need a place holder for rawData is because regex_iter will become invalid if original content were modified
for (auto iter = std::sregex_iterator(rawData.begin(), rawData.end(), REGEX::PoundInclude); iter != std::sregex_iterator(); ++iter)
{
std::smatch match = *iter;
std::string lineString = match.str(0);
std::string truncatedString = RetriveFileName(lineString);
std::regex regex(lineString);
auto implFilesIter = implFiles.find(truncatedString);
if (implFilesIter != implFiles.end())
newRawData = std::regex_replace(newRawData, regex, implFilesIter->second->GetRawData());
else
newRawData = std::regex_replace(newRawData, regex, "");
auto headerFilesIter = headerFiles.find(truncatedString);
if (headerFilesIter != headerFiles.end())
++headerFilesIter->second->dependencyCount;
}
for (size_t i = 0; i < includeFiles.size(); ++i)
{
auto headerFilesIter = headerFiles.find(includeFiles[i]);
if (headerFilesIter != headerFiles.end())
++headerFilesIter->second->dependencyCount;
}
rawData = std::move(newRawData);
}
#undef FILTER_PASS
}
void File::Preprocess(const std::map<std::string, File*>& headerFiles, const std::map<std::string, const File*>& implFiles)
{
auto RetriveFileName = [](const std::string& fileName) -> std::string
{
std::string truncatedName;
size_t index = fileName.find_last_of("/");
// there is a file directory, if thats the case, we just want the name of the file and remove the last "
if (index != std::string::npos)
{
truncatedName = fileName.substr(fileName.find_last_of("/") + 1);
truncatedName.erase(truncatedName.size() - 1);
}
else // there is no directory for this file, remove the #include and ""
{
truncatedName = fileName.substr(fileName.find_first_of("\"") + 1);
truncatedName.erase(truncatedName.size() - 1);
}
return truncatedName;
};
// INCREMENT FILTER PASS IN CASE OF EXTRA DIRECTORY DEPTH
#define FILTER_PASS 6
// 3 passes
// First pass is replace #include library.hpp
// Second pass is replace #include library_platformOS.hpp
// Third pass is replace any #include in library_platformOS.hpp
// rest of the filter is for safety check
for (int i = 0; i < FILTER_PASS; ++i)
{
std::string newRawData = rawData; // i need a place holder for rawData is because regex_iter will become invalid if original content were modified
for (auto iter = std::sregex_iterator(rawData.begin(), rawData.end(), REGEX::PoundInclude); iter != std::sregex_iterator(); ++iter)
{
std::smatch match = *iter;
std::string lineString = match.str(0);
std::string truncatedString = RetriveFileName(lineString);
std::regex regex(lineString);
auto implFilesIter = implFiles.find(truncatedString);
if (implFilesIter != implFiles.end())
newRawData = std::regex_replace(newRawData, regex, implFilesIter->second->GetRawData());
else
newRawData = std::regex_replace(newRawData, regex, "");
auto headerFilesIter = headerFiles.find(truncatedString);
if (headerFilesIter != headerFiles.end())
++headerFilesIter->second->dependencyCount;
}
for (size_t i = 0; i < includeFiles.size(); ++i)
{
auto headerFilesIter = headerFiles.find(includeFiles[i]);
if (headerFilesIter != headerFiles.end())
++headerFilesIter->second->dependencyCount;
}
rawData = std::move(newRawData);
}
#undef FILTER_PASS
}
Monday, February 10, 2020
X11 is haunted
After trying off and on for a week now to get GRasterSurface working on Linux, I can only conclude that X11 is haunted by a malicious poltergeist. I can't find any other reason that the XPutImage function should freeze the test program every time it's called from another thread than main, except for the times where it doesn't and closes immediately instead. Yet the code all works fine if stepped through in the debugger instead of run normally, and every piece of logic works fine with XPutImage commented out.
I spent eight hours yesterday trying everything I could think of to fix the problem, then when I ran out of ideas, everything Lari and three classmates could think of, and the only deviation in behavior was a handful of things causing crashes instead of freezes. Ultimately, I simply don't know enough about how GThreadShared, GConcurrent, or X11 work at a low level to be able to identify the problem, and after three or four hours of suggestions that didn't fix the problem, Lari told me to just make Linux be single-threaded and document every attempted fix for future reference.
Sometimes the bug wins, and you just have to cut your losses and move on.
I spent eight hours yesterday trying everything I could think of to fix the problem, then when I ran out of ideas, everything Lari and three classmates could think of, and the only deviation in behavior was a handful of things causing crashes instead of freezes. Ultimately, I simply don't know enough about how GThreadShared, GConcurrent, or X11 work at a low level to be able to identify the problem, and after three or four hours of suggestions that didn't fix the problem, Lari told me to just make Linux be single-threaded and document every attempted fix for future reference.
Sometimes the bug wins, and you just have to cut your losses and move on.
Bible Progress
With the first draft of the Gateware bible done I have started to process of revising and editing it. I have some ideas for some figures and code snippets that boost the readability of it and strengthen the sections they are in. The biggest pain at the moment would be changing my passive voice to active voice but other than that all in progressing smoothly.
Monday, February 3, 2020
The Power of Jamming Out
I attended Global Game Jam this weekend and it truly reminded me how many people are driven to make games. Not just make games, but possibly work with strangers whose ideas may clash with their own. Global Game Jam is a crazy experience especially for those who haven't worked at a studio such as myself. Everyone has to sync together and help out. Your artist doesn't know GIT? Well, here's a chance to explain it to them and give them a leg up on other artists when they graduate school. You also never know who will show up or what opportunities appear. At the end of the session, there was a chance to submit resumes to a game studio interested in the people who joined this year's jam. Some people are tired and don't want to attend such an event. It's understandable. But, the gaming industry is a luxury industry and not a requirement for a living. The result is that it is very competitive because it's not a requirement. In my experience in luxury industries, you're up against the people living and breathing on improving that skill which makes them stand out who are not going to go down levels to work on mundane everyday tasks. The people at these jams who show up every year are the ones who have endurance and will probably help the industry grow. So, if you love what you do then do it. Breathe it. Live it.
Subscribe to:
Posts (Atom)