Factoring out a new repo for shared code between llbuild & SwiftPM

We are starting to code more of the build system interaction with llbuild in Swift, which uses bindings in the llbuild repository. As part of that, we would like to use the infrastructure we have built up for SwiftPM. SwiftPM depends on llbuild, so this would introduce a cycle to the existing process.

To solve this, we are creating a new repository “github.com/apple/swift-tools-support-core” which will be a new Swift package which contains the infrastructural code that both SwiftPM and llbuild need. We are still working out the details around the build process and CI integration, and we plan to use the JIRA component “llbuild” to track work related to this repository.

NOTE: This code is meant to support building Swift open source projects, and not intended as general purpose, Apple-supported APIs.

3 Likes

The repository has been created.

Would this include the code within the Utility library of SPM, (things like the ArgumentParser) or would it be just llbuild integration? In any case, it would great to have the Utility code in a smaller repo than SPM with proper semver releases :+1:

3 Likes

Indeed, it would be a great idea to extract Utility (and Basic) as separate packages.

It’s basically what is in the Utility product.

And indeed, this would be a great chance for us to start using semver so we can also live on the same experience we are trying to encourage other packages to use!

Here is the initial implementation plan:

Bootstrapping and CI infrastructure:

We want tools-support-core to be a Swift package but we also want to leverage Swift CI infrastructure to add CI support. We need a way to bootstrap this library on the CI without using SwiftPM as we need a cross platform solution to easily bootstrap for all platforms that are currently supported and the new platforms that might be added in the future. To solve this problem, we will add a new backend in SwiftPM to emit a Makefile. The Makefile support will be only accessible using an hidden flag and will be intended to be used only for Swift related projects. We will check-in the Makefile to tools-support-code and execute it on the CI using Swift’s build-script to build and test the project. Most of the time, a contributor is expected to only require a recent trunk toolchain to build the repository using the SwiftPM that is present in the toolchain.

Makefile location and layout:

TBD.

Module prefix:

Currently, Swift requires module names to be unique across the package graph. To avoid conflict with other modules in the package graph, we will use the prefix “TSC” in all of the modules in this repository.

Git history:

Git history is usually a very useful feature but re-writing it could be annoying. Given that we don’t really have huge amount of code that we want to factor out, we will do a “best effort” of preserving the git history.

ToolsSupportCore product:

We will create a “ToolsSupportCore” library product in the Swift package. This product will re-export all library targets that the package wants to make “public”.

I like the idea of moving core modules from SwiftPM (Basic, Utilities, etc) to this new ToolsSupportCore repository. It would make it easier to do proper semver releases that way.

I'll be happy to help with this migration, if there is any way possible.

Thank you,
Eneko

Any plans to actually commit the sources? :slight_smile:

This is something that we really want to do but are lacking time and bandwidth to make progress.

If you have a general idea of which components should get pulled out I'm happy to start working on this.

I can preserve the git history as well, I've broken apart and combined different repos several times.

The major issue here is figuring out the bootstrapping process on the CI as we would need to build this new package before building SwiftPM itself. I am now less convinced of the Makefile approach described above and I think we should look at different options like CMake or using an older version of SwiftPM to bootstrap.

CMake is powerful but last I checked (and just know with a quick Google) it didn't support Swift projects so it will add some pain there but it should be very doable.

If we can just use SwiftPM it has the advantage of being just 1 technology people need to know work on TSC components. Would it be sufficient to use the most recent official release of SwiftPM for this so it is stable but hopefully new enough to meet the needs of the project?

Terms of Service

Privacy Policy

Cookie Policy