Bringing Sourcery to Linux

Hello, dear colleagues :wave:

For the last number of weeks I am trying to make Sourcery work under Linux.
Apart from all the NSObject-related overrides in the Sourcery codebase that are not supported, what I am facing is the error related to some "missing input files":

While trying to run tests with swift test for both release and debug configurations, I hit an error:

error: couldn't build /home/tart/Documents/projects/Sourcery/.build/aarch64-unknown-linux-gnu/release/SourceryPackageTests.xctest because of missing inputs: /home/tart/Documents/projects/Sourcery/.build/aarch64-unknown-linux-gnu/release/SourceryPackageTests.build/runner.swift.o

on CI the error is similar, but not exactly the same (albeit Swift compiler version is 5.8 on CI against 5.8.1 locally):

 error: couldn't build /home/runner/work/Sourcery/Sourcery/.build/x86_64-unknown-linux-gnu/debug/SourceryPackageDiscoveredTests.derived/CodableContextTests.swift because of missing inputs: /home/runner/work/Sourcery/Sourcery/.build/x86_64-unknown-linux-gnu/debug/SourceryPackageTests.build/GeneratorSpec.swift.o,
, /home/runner/work/Sourcery/Sourcery/.build/x86_64-unknown-linux-gnu/debug/SourceryPackageTests.build/Models/TypealiasSpec.swift.o, /home/runner/work/Sourcery/Sourcery/.build/x86_64-unknown-linux-gnu/debug/SourceryPackageTests.build/Parsing/Helpers/TemplateAnnotationsParserSpec.swift.o, /home/runner/work/Sourcery/Sourcery/.build/x86_64-unknown-linux-gnu/debug/SourceryPackageTests.build/Stub/Stubs.swift.o
error: fatalError

Any pointers would be very much appreciated, thank you :pray:t2:

Notes

  • Changes related to Sourcery under Linux are available here in my MR.
  • I am using swift-5.8.1-RELEASE-ubuntu22.04-aarch64 under Ubuntu 22.04. Though, GitHub Actions are running Swift version 5.8 (swift-5.8-RELEASE) which also shows the same error.
  • GitHub Actions throw a similar, but not exactly the same error visible here.
  • I was not able to make test discovery work without the flag -enable-testing mentioned here on my local environment for debug configuration.

Edit: I have the same error on the latest trunk development snapshot of Swift 5.9 from here.

4 Likes

I have figured the reason of the initial error with the missing input file:

  • the name of the .testTarget in Package.swift was set to exactly match SourceryPackageTests during development (to try to fix the issue with resources in tests under Linux, to be more specific), and it seems like <#module_name#>PackageTests.xctests is a system-reserved bundle name.

However, even though having swift test successful, tests are not discovered still. :person_shrugging:
I'll update this thread if I find the reason why, or solution how to make tests discoverable under Linux. For information, swift test command on macOS runs tests w/o any issue).

2 Likes

Are you sure swift test itself is running the tests on macOS? Most Swift packages add XCTestCases which SwiftPM finds, but I see none in Sourcery. I suspect that's why your new linux build finds no tests: the macOS build is probably doing something different to run the tests that you need to replicate on linux.

Hi :wave:

Thanks for the heads-up!

Sourcery tests indeed do not subclass XCTestCase directly, but through QuickSpec which is a subclass of XCTestCase, thus the discovery logic should still work.

Apart from that, I have just checked and added SampleTestCase: XCTestCase with a simple failing test function - still tests was not discovered on Linux.

Are you sure swift test itself is running the tests on macOS?

Yes, indeed, here's the pipeline confirming that.

Currently I am trying to debug swift-package-manager on Linux to see what is going on, why tests are not found. But of course, it is not that easy for a "newcomer" to just clone and launch the debugger along with swift-package-manager... I'll update on my progress if I'll find anything.

OK, probably related to this known issue with Quick and SwiftPM on non-Darwin platforms then.

1 Like

Thank you very much for the information and response! Seems like this is the case. I am trying to make it work, and will update this thread along with my progress. So far I am unable to reference symbols in that executable proposed in the linked comment, where I hit an error from the linker, stating "undefined reference to <#TestNameSpec.self#>".

:wave: A little update:

  1. Following suggestion mentioned by @Finagolfin, I was able to run all unit tests under Linux.
  2. I have reported my findings on the related SPM issue

All in all, once I will make all UTs green, this task will be done. I'll update this thread once I have any updates. :wave:

2 Likes

While updating on the topic - I was able to achieve most unit tests to pass on both macOS and Linux Ubuntu 22.04,

I'd like to write down that I have faced a crash in Swift, specifically upon decoding of an encoded object (with NSCoder). Here's my issue with the details about this crash.

For now I'll omit the tests not to be blocked.

Cheers :watermelon:

2 Likes

:wave: Hi all,

great news, Sourcery Linux support will be soon released as Experimental!

Working on updating documentation, most of the unit tests are green, it actually works :+1:

3 Likes

Hello, dear colleagues :wave:

I have just merged my MR, and so, Sourcery is now working under Linux!

The support is experimental, there are a couple of missing pieces, but overall, CI is green and so, it is possible now to do codegeneration, mocking and much more on Linux!

More info is available in README.

Thanks @Finagolfin for helping in my journey, it was an essential piece of information :heart:

4 Likes

No problem. I had never heard of that Quick package before you mentioned it, so I found that linux issue by simply searching for "xctest" in github code search for that repo, as I wondered how it worked on macOS.

1 Like

:wave: Hello, dear colleagues,

I am very proud to tell you that Sourcery 2.1.0 release is published and contains compiled version of Sourcery under Linux Ubuntu for aarch64!

Happy coding :keyboard:

4 Likes

I'd recommend the Ubuntu archive to specify in its name the exact Ubuntu version it was compiled for. A binary compiled for Jammy is not likely to work on Bionic.

1 Like

Thank you @Max_Desiatov , I did just that for 2.1.1 release now. Appreciate your concern :handshake:

1 Like