as i’m working on improving the test coverage for the [Pitch] Replaceable Library Plugins feature, i’m running into a conundrum where we need to be able to test that SwiftPM can use a precompiled binary dependency, but none of the test artifacts we have on hand are compatible with the Swift CI runners.
Option 1: Test prebuilt Linux libraries
this will probably Never Work, because “Linux” is too broad a term and provides too few environmental guarantees for a generic “Linux” CI runner to robustly support testing Environment-Dependent Shared Libraries (EDSLs).
Option 2: Test prebuilt macOS libraries
this is less than ideal, as EDSLs are a principally Linux-targeting feature, but the specific problem we have is that the macOS CI runners are x86_64 runners, and any test artifacts we could possibly supply are aarch64 binaries, since that’s the hardware we have.
Option 3: Build EDSLs locally, in the CI runner
this is unattractive, as EDSLs are a bit complex to create, tag, persist, and distribute, and i’m not sure if adding infrastructure to the swift-package-manager repo specifically for distributing test EDSLs to itself is a Good Idea.
plus, i’m not sure if the SwiftPM test infrastructure supports running bash scripts from within a test case.
Option 4: Don’t test EDSLs at all
listed for completeness.
posting this here in case anyone has advice for how to tackle this type of situation. cc @hassila
In my opinion, the EDSL should be built as part of testing on the CI runner to guarantee compatibility. Using .binaryTarget(name:path:) eliminates the need for tagging, persisting, or creating checksums. The existing integration tests already use this approach, but they appear outdated, and I'm unable to run these via Utilities/test-toolchain.
EDSL tests should probably be under Tests directory. I found checkParseArtifactsPlatformCompatibility() which creates similar binary tool definitions. There's also PR #8214 which adds tests for pre-built swift-syntax tests.