Using Objective-C Runtime to Make Header-Only Files

The original problem
Duplicate symbol errors occur when multiple definitions for a class appear in different translation units. The Gateware isolation tests are written to produce these errors. The goal is that the tests will fail to cause the errors, which is the case on each supported platform with the exception of the Mac. Gateware creates Objective-C classes on the Mac which cannot be inlined like in C++. Because of this limitation, Gateware fails the isolation tests on the Mac, and the linker throws duplicate symbol errors.

The solution
With inlining the class definitions impossible, the popular alternative is to separate the interface and implementation of classes into separate files. Gateware is released as a header-only file, therefor any solution involving adding additional files is not to be considered. Utilizing the Objective-C Runtime Library is the only suitable solution.

Why this works
The Objective-C Runtime Library can be used to create Objective-C classes using C. These classes are created at runtime, therefore avoiding the linker, and thus never causing duplicate symbol errors. To make sure the runtime library would work for Gateware, I ran some tests on a spike solution. With those tests successful, I converted one of the Gateware libraries to use it. Not only did the library pass the isolation tests, it also passed all the Unit Tests with no discernible difference in behavior and execution speed.

Turning it into macros
The first library converted to use the Objective-C Runtime Library to make Objective-C classes took a considerable amount of time while adding a significant number of uneasy to read function calls. To speed up the process and make the code more legible, I wrote macros that took care of most of the work to define a class and use it. The subsequent libraries took a quarter of the time to convert than the first.

Case closed
At the time of this writing, I still have a few more libraries to convert. However, the tough ones are out of the way and all tests are passing. The only other thing left to do now is to update the developer forums I posted to, and inform them of the solution. Ultimately, this solution means Gateware users can expect a more seamless experience developing across platforms.

Stack Overflow and Avoiding the Linker

The question

Long time reader, first-time asker. My question is, "Can an Objective-C implementation be defined in a header file and also be imported by multiple source files?" I posted that question on the Apple Developer Forums and Stack Overflow. On Stack Overflow, many of the replies that said to use a .m file or they wanted to know why I would even try to do that. I didn't expect these responses considering my full question included that "[adding a source file] would break the single-header architecture" and that "[I'm looking for] some way to implement an Objective-C class in a header file." Thankfully, there were people who tried to help me with a workaround.

The Engineer

My experience with the Apple Developer Forums was different. An Apple Engineer gave it to me straight; there is no way to inline a class in Objective-C. So how do we get around the duplicate symbols error without inlining or creating a separate file for implementation? 

A new solution

Lari found a new solution. Using the Objective-C Runtime Library, we can get passed the linking error by creating and defining the Objective-C class at runtime. After modifying my spike solution to use the runtime library, it worked surprisingly well with no issues.

The catch

Around the same time, I got my spike solution working, I also was informed by the engineer at Apple that the runtime library is per process, and as a result, I could get errors with the way I intended to use it. After they clarified about what per process meant, I was able to revise my spike solution and see the issues they were talking about. I wasn't getting any errors, but I was getting unexpected behavior that could lead to them.

Observing the behavior

One of the things I learned with my tests is that any runtime object allocated in one translation unit (TU) could not be allocated in any other translation unit. That is also dependent on which TU is compiled first. For instance, Gateware.h defines an Objective-C runtime object named Foo. Main.mm includes Gateware.h and creates an instance of Foo. Test.mm does the same. If Main.mm is compiled first, it will be able to allocate instances of Foo. Test.mm will have all allocations return nil. The reverse is true if Test.mm is compiled first instead.

Expecting the unexpected

That doesn't mean subsequent TU's compiled can't use runtime objects that already have instances allocated. If Main.mm creates an instance of a runtime object, it can pass it to Test.mm without any issue. Test.mm can then call the runtime object's members and use it normally.

No need to be concerned?

This limitation can be a problem in other projects, but thankfully it's not a limitation in Gateware. If we prevent the end-user from allocating Gateware Objective-C runtime objects, we can ensure there are never any attempts made to allocate them in different TU's. By restricting runtime objects for internal usage in Gateware, we can control how they are used and stay away from unexpected behavior.

Continuing to move forward

The tests I've conducted so far suggest that this is possible. The Objective-C Runtime Library brings some new considerations to keep in mind. There might be other unexpected behavior, but I'm ready to start moving on from my spike solution and start implementing it in Gateware. I'm starting with GWindow_mac because I believe it will be the most challenging to convert to the runtime library than the rest of the libraries that use Objective-C. It uses inheritance, instance variable and methods, overrided methods, static methods, pointers, and threading. If the Objective-C classes in this library can be converted to the Objective-C Runtime Library, then the rest of the libraries shouldn't be an issue. I am optimistic that is the case.

Duplicate Symbols

The problem
There are several tests for the Gateware project that attempt to create duplicate symbols. Passing these isolation tests mean that the project can be built without producing any duplicate symbol errors. On Windows and Linux, the isolation tests pass, but on Mac, they do not. Instead, duplicate symbol errors appear from several modules.

What are duplicate symbols?

Translation units are created for every source file in a project. Header guards are used to prevent redefining a class in a translation unit (TU) during compile time. However, it does not prevent against redefinition across TU's. Each TU becomes an object file with a .o or .obj extension. In the linking phase, those objects files are linked together into an executable. 

During this process, if the linker finds multiple object files to have the same definition for a class, a duplicate symbol error is produced.

A solution
In the case of a class, duplicate symbol errors can be avoided by separating the interface from the implementation.

The interface goes into a header file (.h or .hpp) and the  implementation goes into a source file (.cpp, .m, or .mm). Done this way, the implementation is defined once in its own object file. Using this solution fixes the issue with the project. However, releases of Gateware are flattened into a single file. So this solution will not do.

Another solution
In order to still be able to flatten the project into a single header, we need to combine the implementation with the interface. We can do this by implicitly inlining the implementation with interface in the header.

By writing the body with each function declaration, C++ permits us to have more than one definition in different TU's. Therefore, we don't receive duplicate symbols errors in the linking phase.

However Obj-C...
On closer inspection, the duplicate symbol errors are all coming from Objective-C classes. Right now in Gateware, the interface for Objective-C classes is placed in the same header file as the implementation, the same way we do with C++ classes. This solution works fine when a single TU is created; however, it fails during the isolation tests when multiple TU's are created. My current research into this problem suggests that the only way to fix this issue in Objective-C is by separating the implementation and the interface into separate files. Again, that would break ability to flatten Gateware into a single header.

A partial solution
Some of the Objective-C classes aren't necessary and can be rewritten as C++ classes. However, this doesn't work for every Objective-C class that relies on inheritance to receive events. Such is the case with GWindow_mac, which currently defines an Objective-C class that overrides functions from NSWindowDelegate to receive and handle window events.

To be continued...
A week into the problem, and this is where I am so far. The issue is more involved than I understood from the start. However, I haven't exhausted all of my resources yet to solve this problem. I'm currently exploring some leads that Lari found. I also posted a question on the Apple Developer Forums. I'll keep probing, and with any luck, my next post will be about how the issue was solved, instead of just what I tried.