Getting Gateware Unit Tests to Run on Xbox One

    Since last Wednesday when I wrote my last blog, I have got quite a bit done. The GFile port is officially done with all new GFile functionality working on all platforms. The issue that I found on Mac last week turned out to be a new bug that only seems to appear on MacBooks, very strange.  After that, I finally got the chance to merge with GApp, the class that Lari wrote for all future mobile ports of Gateware. There was only a few conflicts to resolve and a tweak to get it working perfectly. I finished all that before the end of the day on Friday and figured it was about time I got around to actually putting Gateware on an Xbox, the main reason of bringing Gateware to UWP.

The Process:

    The steps required to get an Xbox One or any other variant of the Xbox lineup into developer mode is quite simple. The Xbox One that I have currently was lent to me to work on and it already had dev mode enabled, but for the sake of knowing how it is done, I started from scratch. I deactivated it and went from there.

    There are very comprehensive guides that can be found on Microsoft's website. I followed them as long as I could until I ran into an issue. As of writing this, there seems to be a bug with the dev mode app that can be downloaded from the Windows Store. It seems to allows hang on this screen, which is a big problem if you want to start developing on the console.

    After looking around for a solution, I found an alternate solution as to how to get into dev mode. This solution requires hitting all four shoulder buttons on the controller at the same time in the System Console Info section in the settings app.

    Once all 4 buttons have been pressed the option shown above appears, pressing that will bring up the dev mode conformation screen.

    Pressing Continue, the console will appear to reboot and after about 5 or so minutes, the Xbox will boot up into dev mode.

    The next step in the process is getting Visual Studios setup properly to connect with the Xbox. Instead of me just rehashing what is already explained really well in a Microsoft document, I will just link it here. In short though, it is as simple as typing in the correct IP address in the properties tab of the project.

The Result:

WARNING: fixing warnings...

Week 1

    So I'm new but I've been here for a couple of weeks already. I just forgot to do the first weeks blog post. Super professional, I know. 

    In my defense, there really wasn't anything too interesting going on. I was just trying to get dev environments set up on the three major platforms (Windows, Mac, Linux). The only real problem I had was during the installation of Linux.

    I was trying to set up a dual boot situation on my main Windows machine so I would be able to boot into either. The Gateware repo has a tutorial on the subject which I followed. Everything started off just fine. I was able to boot into the Linux Mint installation environment from a USB drive and then create a new partition on one of my SSD's. The problem arose when I went to perform the actual install of Linux. 

    The installer is supposed to recognize your Windows installation and then give you an option to "install alongside Windows" but for whatever reason, mine didn't. I had to do everything manually. This meant I had to designate parts of the new partition as /root and /home and pick their sizes/formats by hand. Thankfully there's plenty of online materials and tutorials that tell you how big and what format these things should be. The only problem was a suspicious dropdown box at the bottom of the installer that I didn't pay attention to. This box is for telling the installer where to put the boot sector.

    It defaulted to my Windows drive...

    So, as you can probably surmise, I ended up overwriting my Windows boot sector and loosing access to my Windows install. All of my files and data were still there, I just couldn't boot into Windows to access them. 

    I didn't realize exactly what happened at first so I thought I could just boot into Linux, delete the boot sector, and then have my Windows back, good as new. So I did that and now I had no access to any operating systems at all. 

    The solution to this, was to go on another computer, create a Windows installation media USB drive, boot off of that, access the command prompt, and tell it to reinstall the Windows boot sector. Thankfully it worked, and I was able to begin the process again and properly select a different drive for the Linux boot sector. 

    And that's pretty much all the excitement for the first week. Everything else was just, "does this compile and run? Ok, cool. Moving on."

Week 2

    Now that I think about it, week one was way more interesting than week two. The first half of the week was just experimenting with the end-user side of Gateware to get a better understanding. The second half was just searching through warnings on Windows and fixing the ones that affect the end-user.

    That's pretty much it. 

Porting GFile to UWP

The Task:

    About two weeks ago, I ported the audio libraries over to UWP. Soon after I got the unit tests all passing I realized that the auditory output did not match what was heard in the win32 version of the Gateware unit tests. This was cause for concern but I could not continue to spend too much time on one library, so it was decided that I was to move on to another and mark the audio difference as a bug to come back to later. I decided to begin the porting of GFile as I had to do a little bit of file I/O work in the audio unit tests and it was fresh in my mind.

The Problem:

    My initial thoughts on the GFile port were ones of ease. While this did turn out to be the case, it did not seem like that at first. Everything I read up to that point in my research suggested that the code that was already written for the Win32 implementation of GFile should work just fine for UWP. But for the life of me, I could not get it to work. This led me to the conclusion that I should rewrite the entire GFile UWP implementation using WinRT API calls. This caused many problems in the end, mostly with functionality differences with the other platforms.

The Solution:

    It was this difference in functionality that would drive me back to looking at the code that was already there from Win32. It was at that point that I discovered why GFile was failing at writing files out to the directory that I had chosen, the install location. I found out that the install location of a UWP app is read-only, which is fine for assets and resources that are only meant to be read, so I made the change to writing to the app folders instead which are both read and write capable. That was the change that got GFile working perfectly using the Win32 code.

    That wasn't the end for GFile though. Severe code branching was happening in the unit tests, specifically for audio to help differentiate between the many platforms Gateware can run on. Imagine what is pictured to the right but for every sound requested for the audio unit tests, it was happening almost 90 times.  It wasn't that this code didn't work, it was just messy and not really sustainable as more and more platforms are eventually added to Gateware down the road, such as iOS and Android.

    This brought up the proposal of adding additional functionality to GFile to get the different accessible file locations for the different platforms. This is mostly for the addition of the mobile platforms as this has little application to the ones on desktop. Lari sat down with Jacob, the iOS dev on the project, and I last Friday and we came up with a solution. These new functions will get the correct directory for each platform you are on as each platform implementation will have their own version of this function. I finished the UWP implementation before the end of the weekend but have been finishing up the rest of the platforms such as Windows, Linux, and Mac through the beginning of the week. So far GFile is up and running on UWP, Windows, and Linux with all unit tests passing. Mac is still a issue but that may be because I am running an older build of Gateware that has since had some bugs fixed which I am experiencing, but that is a separate issue in of itself.

    With GFile done and working, GLog kind of just fell into place and it worked perfectly without any issues or tweaking needed. Next, is fixing up the Mac issue and finally merging with Lari's GApp branch.

    
 

XCode and iOS with CMake: A Challenge

    I was tasked to come up with a way to develop Gateware on iOS. At the moment Gateware is using CMake to create the projects to develop on. This way the Gateware team can make multiple projects for every platform that they develop for. At first I took this task as making another project for iOS specfically, and made a simple CMake script that made an XCode project that would build Gateware for iOS. Making a XCode project purely for building on iOS is pretty simple, but I learned that it is possible for a single XCode project to have multiple build targets, and set out to make the Gateware MacOS project also build for iOS. 

    It turns out that adding an iOS target to an XCode project through CMake is not very intuitive as making an XCode project only with iOS. At first I tried to change the Project settings when generating the iOS target. The issue with this approach is that the other targets in the project were using the project defaults, and changing the project default to iOS would effect the other targets to also target iOS. I would have to change the targets already setup in the project for this strategy to work, as that would require a large rewrite of the CMakeLists file, I set out to find another way to go about this issue.

    After searching the internet for a way to have a Mac and iOS target in a single project for XCode, I ended up going back to the CMake documentation. I found that in CMake you can change specific target settings in XCode using the set_target_properies command and XCODE_ATTRIBUTE property. I ended up creating a dummy target in CMake and overwriting the necessary build settings to essentially convert a Mac executable to an iOS app. The next step would be to get Vulkan installing on an iPhone using CMake.

References:

https://cmake.org/

https://gitlab.com/gateware-development/gateware

    

Designing a Multiplatform Touch Interface Part 1

     With more people using mobile devices and touchscreen laptops, developers need to interface with the devices' touchscreen API. Gateware should be doing the same thing if it hopes to support mobile devices. One of the big issues with designing a touch interface is the different ways that devices store the touch information. A good rule is that an interface should provide as much functionality to the user that they should need to use.

    When designing this interface one should consider what major platforms support touch and which are most popular. iOS and Android devices are likely the most common touch devices with touch as a primary method of input. Windows and macOS also support touch input, but they are secondary input methods, as most Windows and macOS devices do not have touch input. Other notable devices are Nintendo Switch, Smart Watches, and embedded devices. As such, I propose this touch interface mainly for iOS and Android, but with expansion options for those other devices. 

    All of these different platforms have different information sent to the developer when the screen is touched. I have created a struct that supports the main information that most developers would need.

enum class GTouchState : unsigned int {
	TOUCH_BEGIN			= 0,	// Finger just hit the screen.
	TOUCH_MOVED			= 1,	// Finger moved from last position.
	TOUCH_END			= 2,	// Finger lifted off the screen.
	TOUCH_STATIONARY	= 3,	// Finger is on screen but hasn't moved.
	TOUCH_CANCELLED		= 4,	// System stopped tracking the touch.
	TOUCH_COUNT,
};
 
struct GTouchData {
	// Current location of the touch in screen coordinates
	float x, y;					
	// Change in location in screen coordinates
	float deltaX, deltaY;		
	// First recorded location of the touch in screen coordinates
	float initialX, inititalY;	
	// Change in time between the current position and delta
	float deltaTime;			
	// Pressure of the touch
	float pressure;			
	// Number of consecutive touches in the same general area
	unsigned int tapCount;		
	// What touch is this? Always less than the number of total touches on the screen
	unsigned int index;			
	// What state is this touch in?
	GTouchState state;			
};

iOS and Android support more information, but most of the information not included here is for stylus support, with information on how the stylus is aligned with specific axes. Most Gateware users won't need this information, as such I have omitted them to give some simplicity to this interface.

    Another part of the API is common support for touch gestures. Many mobile users are used to common touch controls to get around their mobile device. Swipe up to go down on a page, pinch to zoom out, and two finger spin for rotate, along with others are just some of the gestures that this API should support for a better experience for the user. As such here are the common gestures that most platforms recognize.

Single Tap

Double Tap

Rotation (Two finger)

Swipe

Pan (Two Finger)

Long Press

Pinch

    Here is the end for now. Expect another part to this blog once I have figured out what interface would be best for Gateware.

References:

    Touch data: 

        https://developer.apple.com/documentation/uikit/uitouch?language=objc 

        https://developer.android.com/reference/android/view/MotionEvent.PointerCoords

    Gesture recognizers:

        https://developer.apple.com/documentation/uikit/uigesturerecognizer?language=objc

        https://developer.android.com/reference/android/view/GestureDetector

 

     

Porting the Audio Libraries to UWP

The Task:

    Getting CMake to properly work with Gateware Win32 and UWP projects was no simple task but since it was finally done, I was able to start porting Gateware to UWP in earnest. I was given the choice of what to start with, I chose the audio libraries as it seemed like an (almost) one to one port was in the cards. I was right, but it didn't come without its issues.

The Problem:

    My initial evaluation of the audio libraries had me changing just a few functions here and there that were not available for UWP apps. This assumption did turn out to be correct but with some caveats. I based all of the UWP implementation of the GAudio libraries off of the Win32 ones as they are all build using XAudio2 which is compatible with UWP. After changing some key functions that were not compatible, ex. CreateFile() to CreateFile2(), I got everything compiling. Success? No, not really. When I would then run the Unit Tests, the program would call abort() from somewhere.

The Solution:

    The first thing to do was to find where abort() was being called from. Using the debugger I was able to track it down to the CreateFile2(). I was very surprised to find this as the documentation for the function says it is compatible with Windows Store apps. After reading through it again more thoroughly, I found this "When called from a Windows Store app, CreateFile2 is simplified. You can open only files or directories inside the ApplicationData.LocalFolder or Package.InstalledLocation directories." Knowing that I have not done anything with file I/O yet with any of the libraries, I knew pretty quickly this was the issue.

    Getting to the installed location of the app is done pretty easily with WinRT calls so that was not the hard part. Getting the Unit Test Resources, such as the audio files, in the installed location is what stumped me for the longest time. This is easily done in Visual Studios by just including it in the project and setting it to "content" in the properties but this needs to be done before a dev even opens up the project. Back to CMake I went as I already had the Asset folder being put in as content for the UWP app, so I figured it would not be any different for another folder. I was right in that thinking, it just took a lot of trial and error to get it just right.

    With that out of the way, I was able to get the sound files from the installed location and everything compiled and ran perfectly.

   As of right now, all audio unit tests pass.

The Extents of Controller Support

During testing last week, I had one of the manual Unit Tests fail for GController on the Mac. It was a test of the down button input on a controller, and no matter how much I pressed the button or reran the test, the input wasn't being detected. I found this very strange because I had tested it not too long before, and the test passed without issue. The difference between the time before and last week is that I was using a different controller.

Different controllers, different codes

The problem turned out to be a difference in input codes for the D-Pad between the two controllers. The generic controller I initially tested with used codes of 0 - 360 for the D-Pad. The PS4 controller used codes of 0 - 8. This discrepancy was easy to fix by adding support for both ranges of input codes. However, it got me wondering about the rest of the controller input.

Unit Test upgrade

The current Unit Test sampled a small portion of controller input, only conducting 11 input checks. On Monday, I made modifications to support the testing of each input supported by GController, totaling 40 input checks. I also changed the way the test behaved to speed up the ability to get through the test. The changes to the test were necessary to find out if there were more issues with controller inputs.

How a Unit Test is written matters

The first thing I discovered with the new Unit Test is a range problem with the trigger inputs. On Windows, the trigger inputs gave a value of 0 - 1. On Mac and Linux, the trigger inputs gave a value of 0 - 5. This difference was never caught during the original Unit Test because it looked for a value greater than 0 to satisfy the test's trigger is pressed state check. In the new test, the value must be exactly 1. The actual values both my controllers reported was 0 - 255. Initially, I thought dividing by 255 to normalize the value would be sufficient. I eventually discovered that I had one other thing to consider, how the device is connected to the computer.

Different connections, different values

When connected wirelessly, the trigger input of the PS4 controller gave values of 0 - 255. When connected via USB, the trigger input of the PS4 controller gave values of 0 - 315.

 

Thankfully, IOKit has a function for getting the maximum value of the trigger. Using that, we can futureproof the normalizing of the trigger input values so we will always have a range 0 -1 regardless of the controller or connection type. A solution like this is needed for the Mac with the controllers I have tested so far. It also may be necessary on Windows and Linux implementations as different controllers are tested. At this time, a solution like that isn't implemented on those platforms.

Different connections, different identities

Linux is sensitive to connectivity as well. The PS4 controller input mappings are different from the generic controller. When the PS4 controller is connected via Bluetooth, Linux identifies it as a Wireless Controller. Before, GController would use generic mappings for a controller with this identification. With the wrong mappings, the square and triangle buttons on the PS4 controller are swapped. I changed the code so that mappings for PS4 controllers are used when a Wireless Controller is identified. This may be an issue if other brands of wireless controllers use different input mappings, but I don't have any other controller to test that theory.

It's not over yet

Next week, I'll be able to test an official Xbox controller with GController. The library was designed with this type of controller, so I don't expect any issues. However, I wonder if there will be a difference depending on the connection type used. We'll see.

Sidenote

I tried using the steam controller on Linux to test GController. When connected via USB, some of the inputs worked, some were unresponsive, and some caused the Unit Tests to crash. I tried installing the driver for it by installing Steam and connecting it via Bluetooth. Unfortunately, that ended up crashing Linux and causing issues with X Server. I managed to get X Server working again, but it ended up changing my entire Linux GUI environment. If anyone tries this, beware.