Making SwiftPM use development swift build (macos monterey, swift 5.8)

Development Package Manager
1

Hi, this might just be my naivity about SPM usage. I have my own custom built compiler with patches that I need. The swift source and dependencies are checked out in the usual way, using utils/update-checkout.sh with the release/5.8 branch. So I've got the directories next to each other cmark, llvm-project, swift, llbuild, swift-driver, swift-syntax, swift-system, swiftpm, yams... etc. I successfully did a build, which made build and applied my patches into the swift source folder and rebuilt binaries, which are now in build/Ninja-ReleaseAssert/swift-macosx-x86_64/bin as you'd expect. swift-frontend and the usual symlinks. The compiler works, all is good.

I then built swiftpm in its own directory just using swift build and have a working swiftpm in swiftpm/.build/debug as normal. I added both the swift and swiftpm directories to the front of my PATH and did some basic tests to make sure that from the command line swift and swift package are now calling the versions that I built (not the ones bundled in Xcode on my mac).

I used swift package init to create a basic project, added some code and tried to compile it with swift build -v. The problem is that for some reason, it then falls back to using swiftc from Xcode, presumably picking it up from a global mac setting...

/Applications/Xcode-14.2.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swiftc -module-name blink -incremental -emit-dependencies -emit-module ...

It's a simple enough question! ... How do I get swiftpm to use my custom swift compiler to do its build? It should at least be respecting the PATH variable? (I've confirmed that in the shell, PATH works fine and my custom swiftc is always run, not one from Xcode.)

Thanks for any help or pointers you can give!

Regards,
Carl

The bonus/follow on question... I realised that part of how SPM works is it compiles my Package.swift first to make a blink-manifest MachO executable... and that compilation step might need swiftc from a system location such as Xcode (I'm not sure)... but to be honest, if it tries to run my custom compiler for both that's fine. It will just mean I might need to fix bits on my customer compiler, which is no bad thing!

1 Like
2

You can use the SWIFT_EXEC environment variable to point SwiftPM to a custom compiler.

1 Like
3

Thanks. I ended up asking ChatGPT and it told me the same!

Can I ask a more complex question?

I’d like to use the normal compiler to build Package.swift, but then when those instructions are parsed, have the actual build process use my custom compilers and stdlib. Is that possible without altering the code of SPM?

I read somewhere that you can specify a —destination parameter to SPM that takes a json file with a list of specified tools. Do you think that would work? I’m trying to use a custom swiftc and a custom clang!

Thank you! Carl

4

Yes, that should work (except that I wouldn't recommend asking ChatGPT about anything). Although if SE-0387 is accepted we'd consider deprecating the --destination option and the old JSON schema that it consumes.

3 Likes
5

ChatGPT is clearly very good at BS is the conclusion I came to! It was able to tell me about SWIFT_EXEC, after that it told me that the --destination flag on SPM specified the destination the products were built to!

Back in the real world, do you know where I can find a link that documents the current version of the destination JSON file format, so I can start experimenting locally with using my own custom compiler builds?

I couldn't see anything really here PackageDescription API — Swift Package Manager or here swift-package-manager/Documentation at main · apple/swift-package-manager · GitHub, unless I'm being dumb (highly likely!)

Regards,
Carl

6

It's documented in SE-0387 itself, or is there anything missing from the proposal text that you need for your use case?

7

I think there’s probably quite a bit that could be added. Say I want to add swift support to a random linux distribution like raspbian or Suse. Or something more exotic like OpenBSD.

I would probably want to know things like

“what is in the SDK? What files does it contain? Just a swiftc? Where do libSwift.a and libSwiftOnone.a go? The swiftshims folder? What about libc? Standard include headers for libc? Etc.”

In my current case I put al of those in various folders and I use a Makefile to organise the swiftc and clang commands to build bitcode files, object files and the final ELF.

I’m really trying to translate this simple makefile into SPM. Each command is pretty simple

swiftc <various flags> —emit-bc <include paths for the standard library, libc and any third party modules> main.swift otherfile1.swift otherfile2.swift

Then llc to make an object

Then link them with lld

That’s all I need to do!

8

Wherever you want within a bundle, the new schema makes it customisable. Then you should specify paths to these libraries in the JSON configuration file, as shown in the proposal.

9

Cool. Looks straightforward! I've rebased my code on top of 5.8, can I just go right ahead and use the new implementation? Or do I need to compile SPM with special flags?

10

This new SDK layout is unlikely to fully work with 5.8. There's an experimental implementation in 5.9 development snapshots, but some things may change depending on the proposal review results. I'd recommend to hold off until there's an announcement available clarifying its status.

11

OK, thanks Max. I'll try to piece together how to run SPM using the legacy --destination flag with my tools for now. I appreciate all your help, thank you.