Testing swift-format as part of Swift’s CI infrastructure

[ahoppen (Alex Hoppen)]

In the last couple of weeks, we have made changes to the Swift testing infrastructure that allow testing dependent projects of SwiftSyntax without needing to rebuild SwiftSyntax. As a result, it would now be possible to test swift-format as part of the Swift CI testing infrastructure.

Integrating swift-format with the Swift CI infrastructure would allow PR testing on the apple/swift-format repository. Additionally, swift-format would be tested as part of Swift’s continuous integration jobs, ensuring that no change to the master branch of SwiftSyntax or the Swift compiler will break the master branch of swift-format.

The integration comes with the expectation that developers of swift-format take active part in fixing CI issues in a timely manner. Because of this, we leave it up to you to decide when and if the CI integration should be pushed forward. I have composed a list of the items below that need to be done in order to get the CI integration working:

• Update swift-format to depend on SwiftSyntax’s master branch
• Make all @testable imports in swift-format’s test normal imports, making all necessary types and methods public [1]
• Depend on local dependencies if the SWIFTCI_USE_LOCAL_DEPS environment flag is specified
  • See Allow depending on local dependencies for CI by ahoppen · Pull Request #101 · apple/swift-stress-tester · GitHub for reference
• Add a build-script-helper.py script to the package which can be invoked by the build-script in the main Swift repository
  • See swift-stress-tester/build-script-helper.py at main · apple/swift-stress-tester · GitHub as an example
  • Only the build and test actions need to be supported
  • Particularly the SWIFTCI_USE_LOCAL_DEPS environment variable needs to be specified when invoking swift build . Furthermore, the --multiroot-data-file option must be passed to swift build
• Add swift-format as a product to Swift’s swift_build_support
  • See https://github.com/apple/swift/blob/master/utils/swift_build_support/swift_build_support/products/skstresstester.pyas an example. The install phase should not be supported for now, installing swift-format in the toolchain can be discussed after CI integration has been implemented
  • Additionally, the correct command line flags need to be added to build-script . See Move stress tester and swiftevolve to new swift_build_support build infrastructure by ahoppen · Pull Request #27580 · apple/swift · GitHub for an example of which flags to add
• Create a preset for PR testing in the swift-format repository
  • See Move stress tester and swiftevolve to new swift_build_support build infrastructure by ahoppen · Pull Request #27580 · apple/swift · GitHub for an example of a macOS preset and [build-presets] Add preset for testing SwiftSyntax on Linux by ahoppen · Pull Request #28094 · apple/swift · GitHub for a example of a Linux preset
• Let us know that you have completed the above steps, so swift-format can be added to the checkout on the CI bots and CI testing for the swift-format repository can be enabled
• After verifying that the CI works on the swift-format repository using a dummy PR, update the presets to test swift-format as a downstream dependency of SwiftSyntax
  • As a starting point, it would be a good idea to test it everywhere swift-evolve is being tested
  • If swift-format should also be tested on Linux, it needs to be added to the corresponding Linux bots as well

[1] We are only building SwiftSyntax once without testability enabled, because that’s the version we are installing in the toolchain. SwiftPM does not support building targets with different build configurations, so we cannot build swift-format with testability enabled.

CC @allevato

[allevato (Tony Allevato)]

Thanks for the detailed steps, Alex! Having swift-format integrated into the rest of the CI pipeline and getting PR testing will be a great improvement to our workflows.

Since we'll need some time to migrate to the latest SwiftSyntax API changes that were recently pushed, I think we'll tentatively aim to work on this after Thanksgiving.

[ahoppen (Alex Hoppen)]

Sounds like a good plan to me. Please let me know if you run into any questions during the integrations and I should be able to help you out.

[dylansturg (Dylan Sturgeon)]

My PR to create a testing preset for macOS and Linux was just submitted. It looks like we're ready for this step:

Let us know that you have completed the above steps, so swift-format can be added to the checkout on the CI bots and CI testing for the swift-format repository can be enabled

@ahoppen @akyrtzi Can someone add swift-format to the checkout on the CI bots?

[blangmuir (Ben Langmuir)]

We can break this down into:

1. Update the update-checkout configuration. This is something you can do in a PR.
2. Update bots to checkout swift-format (some of our bots don't use update-checkout for security reasons).
3. Add PR test bots for swift-format using the new presets added in Add a build product and presets for swift-format. by dylansturg · Pull Request #32546 · apple/swift · GitHub

@mishal_shah can you help us with (2) and (3)?

[dylansturg (Dylan Sturgeon)]

Update the update-checkout configuration. This is something you can do in a PR.

It looks like swift-format is already included in the update-checkout workflow. What else needs to be done for this?

[mishal_shah]

I will update the CI bots next week to checkout swift-format.

[blangmuir (Ben Langmuir)]

@dylansturg

It looks like swift-format is already included in the update-checkout workflow.

My mistake, I didn't realize this had already been done.

[dylansturg (Dylan Sturgeon)]

Are there any updates regarding the CI bots?

[mishal_shah]

Swift CI development (master) jobs are configured to checkout swift-format with master branch.

[allevato (Tony Allevato)]

Thank you!

[allevato (Tony Allevato)]

@mishal_shah I noticed in recent CI logs that swift-format was being checked out, but should @swift-ci please test also be working now on the swift-format repo? I just tried it on this PR but I haven't seen any activity from it.