1

The search function yields lots of results for "source compat suite" -- but those are all CI Notifications ;-)
Is there any detailed information on how to use the scripts?

[Like choosing which platforms to inculde or exclude (or just not fail for the majority of projects ;-)]

2

Good question (although possibly the wrong topic; maybe something under Development since it's not relevant to most developers). @Rostepher, who's the best person to answer this?

1 Like
3

I guess this is a hard nut to crack... to add some details:
The CI logs contain the right call to trigger the tests — but it needs some sandbox configuration files, which are not available (and there is not documentation for .sb files either :slightly_frowning_face:)

4

@Tino have you looked at GitHub - apple/swift-source-compat-suite: The infrastructure and project index comprising the Swift source compatibility suite.? This might be able to give you some answers about how to build the source compatibility suite.

To build all projects against a specified Swift compiler locally, use the runner.py utility as shown below.

./runner.py --swift-branch master --projects projects.json --include-actions 'action.startswith("Build")' --swiftc path/to/swiftc
Use the --include-repos flag to build a specific project.

./runner.py --swift-branch master --projects projects.json --include-actions 'action.startswith("Build")' --include-repos 'path == "Alamofire"' --swiftc path/to/swiftc
By default, build output is redirected to per-action .log files in the current working directory. To change this behavior to output build results to standard out, use the --verbose flag.
5

I did - but that results in failures for most of the builds (afaics, non-macos targets never work).
The "right" call is

./runner.py --swiftc /Users/buildnode/jenkins/workspace-private/swift-master-source-compat-suite/build/compat_macos/install/toolchain/usr/bin/swiftc --projects projects.json --include-actions 'action.startswith("Build")' --swift-branch master --sandbox-profile-xcodebuild ../../../workspace-private/swift-source-compat-suite-sandbox/sandbox_xcodebuild.sb --sandbox-profile-package ../../../workspace-private/swift-source-compat-suite-sandbox/sandbox_package.sb

but sandbox_xcodebuild.sb doesn't seem to exist on public servers :-(

6

sandbox profile is not open sourced, however you should be able to run it without the sandbox flag

For example:

./runner.py --swiftc <swiftc_path> --projects projects.json --include-actions 'action.startswith("Build")' --swift-branch master
1 Like
7

The script ran on the first try -- but only a handful of targets passed the tests.
As many macos builds have been successful, I looked into the build of the toolchain and noticed that you have to add some flags to build for multiple platforms.
So, I rebuilt swiftc, but as that didn't help, so I started looking for documentation (actually, I'd even prefer to just build for macOS anyways, because I just want to check if the code itself is compatible with a modified toolchain...)

But as you wrote that runner.py doesn't needs to be started in the exact same way as Jenkins does, I just triggered it with the regular swiftc -- and that did produce only a single failure so far, so I'm sure it's still an issue with the build of the toolchain...

I started looking into the compat suite sources to check wether there is an easy way to simply skip ios and the other ARM-platforms, but afaics, it might be easier for me write a small script that parses and filters projects.json on its own.

Thanks for your help!