Approaches for testing environment-dependent SwiftPM behavior?

as i’m working on improving the test coverage for the [Pitch] Replaceable Library Plugins feature, i’m running into a conundrum where we need to be able to test that SwiftPM can use a precompiled binary dependency, but none of the test artifacts we have on hand are compatible with the Swift CI runners.

Option 1: Test prebuilt Linux libraries

this will probably Never Work, because “Linux” is too broad a term and provides too few environmental guarantees for a generic “Linux” CI runner to robustly support testing Environment-Dependent Shared Libraries (EDSLs).

Option 2: Test prebuilt macOS libraries

this is less than ideal, as EDSLs are a principally Linux-targeting feature, but the specific problem we have is that the macOS CI runners are x86_64 runners, and any test artifacts we could possibly supply are aarch64 binaries, since that’s the hardware we have.

Option 3: Build EDSLs locally, in the CI runner

this is unattractive, as EDSLs are a bit complex to create, tag, persist, and distribute, and i’m not sure if adding infrastructure to the swift-package-manager repo specifically for distributing test EDSLs to itself is a Good Idea.

plus, i’m not sure if the SwiftPM test infrastructure supports running bash scripts from within a test case.

Option 4: Don’t test EDSLs at all

listed for completeness.

posting this here in case anyone has advice for how to tackle this type of situation. cc @hassila

4 Likes

In my opinion, the EDSL should be built as part of testing on the CI runner to guarantee compatibility. Using .binaryTarget(name:path:) eliminates the need for tagging, persisting, or creating checksums. The existing integration tests already use this approach, but they appear outdated, and I'm unable to run these via Utilities/test-toolchain.

EDSL tests should probably be under Tests directory. I found checkParseArtifactsPlatformCompatibility() which creates similar binary tool definitions. There's also PR #8214 which adds tests for pre-built swift-syntax tests.

Regarding CI, the smoketest uses a private GitHub repo (https://github.com/apple/swift-continuous-integration). It's unclear what's being tested and how since it spends over 4 hours building the full toolchain, and the only test output with Test Case is for SwiftFormatTests:

--- Running tests for swiftpm ---
+ /Users/ec2-user/jenkins/workspace/swift-package-manager-PR-macos-smoke-test/branch-main/swiftpm/Utilities/bootstrap test --release --swiftc-path /Users/ec2-user/jenkins/workspace/swift-package-manager-PR-macos-smoke-test/branch-main/build/buildbot_incremental/intermediate-install/macosx-x86_64/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swiftc --clang-path /Users/ec2-user/jenkins/workspace/swift-package-manager-PR-macos-smoke-test/branch-main/build/buildbot_incremental/intermediate-install/macosx-x86_64/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang --cmake-path /usr/local/bin/cmake --ninja-path /Users/ec2-user/jenkins/workspace/swift-package-manager-PR-macos-smoke-test/branch-main/build/buildbot_incremental/ninja-build/ninja --build-dir /Users/ec2-user/jenkins/workspace/swift-package-manager-PR-macos-smoke-test/branch-main/build/buildbot_incremental/swiftpm-macosx-x86_64 --llbuild-build-dir /Users/ec2-user/jenkins/workspace/swift-package-manager-PR-macos-smoke-test/branch-main/build/buildbot_incremental/llbuild-macosx-x86_64 --verbose


Build Percentage         Build Duration (sec)    Build Phase
================         ====================    ===========
12.7%                    1967.03                 Running tests for swiftpm