Configure utils/build-script for building header files only in CI

Development Compiler
1

I have a mixed C++/Swift package that I want to build in CI.

The C++ portion needs to include headers from the swiftlang/swift repo, specifically the C++ target has the similar configuration as in SwiftCompilerSources:

private extension Target {
  static func compilerModuleTarget(
    name: String,
    dependencies: [Dependency],
    path: String? = nil,
    sources: [String]? = nil,
    swiftSettings: [SwiftSetting] = []) -> Target {
      .target(
        name: name,
        dependencies: dependencies,
        path: path ?? "Sources/\(name)",
        exclude: ["CMakeLists.txt"],
        sources: sources,
        swiftSettings: [
          .interoperabilityMode(.Cxx),
          .unsafeFlags([
            "-static",
            "-Xcc", "-DCOMPILED_WITH_SWIFT", "-Xcc", "-DPURE_BRIDGING_MODE",
            "-Xcc", "-UIBOutlet", "-Xcc", "-UIBAction", "-Xcc", "-UIBInspectable",
            "-Xcc", "-I../include",
            "-Xcc", "-I../../llvm-project/llvm/include",
            "-Xcc", "-I../../llvm-project/clang/include",
            "-Xcc", "-I../../build/Default/swift/include",
            "-Xcc", "-I../../build/Default/llvm/include",
            "-Xcc", "-I../../build/Default/llvm/tools/clang/include",
            "-cross-module-optimization",
          ]),
        ] + swiftSettings)
    }
}

This works perfectly well locally. I simply follow the Getting Started guide in the swiftlang/swift repo, calling utils/build-script with the same configuration as in the guide:

utils/build-script --skip-build-benchmarks \
  --swift-darwin-supported-archs "$(uname -m)" \
  --release-debuginfo --swift-disable-dead-stripping \
  --bootstrapping=hosttools

The problem is this takes over 3 hours, and I would rather not build the whole thing from scratch every CI trigger when I just need some header files.

I’ve searched these forums and found two possible approaches:

  1. Use some build preset instead of the Getting Started configuration.
  2. Pass –-skip-build to utils/build-script and then call Ninja directly.

So my questions are:

Is there a suggested build preset that would just build the header files. Or at least a preset that would skip as much as possible? Maybe [preset: buildbot,tools=RA,stdlib=RD,test=no]?

What Ninja targets would be appropriate? The Ninja swift target suggested elsewhere on these forums doesn’t seem to exist anymore.

Many thanks!

2

+1 for this. Still have not found a satisfactory solution yet.

3

The most slimmed down set of commands I could figure out was:

utils/build-script –-release –-skip-build ...

ninja -C ../build/Ninja-ReleaseAssert/llvm-macosx-arm64
ninja -C ../build/Ninja-ReleaseAssert/cmark-macosx-arm64
ninja -C ../build/Ninja-ReleaseAssert/swift-macosx-arm64 swift-frontend

Note the Ninja target in the last line is swift-frontend not bin/swift-frontend and reported elsewhere.

But this still takes over 3 hours.

I think the path forward is to enable sccache and configure the GitHub - Mozilla-Actions/sccache-action: sccache github action action, seeing as I am using GitHub Workflows. Will report back.

4

I got the build down to 56 minutes in GitHub, with sccache reporting a 99% cache hit rate.

About 10 minutes is calling utils/update-checkout, so I might be able to optimise that further.

1 Like
5

Another approach I'd like to take is use a prebuilt header only git package (Failed due to the bad support of swift repo header location + SPM integrate issue) or include a zip library or artifactbundle. (including swift_6.2_macosx-arm64 / swift_6.2_macosx-x86_64 / swift_6.2_linux_aarch64 etc.)

So we will only need to build it once and use the downloaded content for later use.

1 Like