It seems that Package.resolved is not the final source of truth.
SwiftPM adds a dependency on swift-llbuild master and this seems to cause Package.resolved to be ignored. Even when the contents of the resolved file are the following and --skip-update is specified:
As you can deduce, this behavior of SwiftPM is problematic or at lease surprising, especially since once would expect builds to be reproducible. I would be grateful if anyone could shed some light.
I donât know what is happening here for you. With Swift 5.0.1, I cloned and built and it respected Package.resolved. Maybe there is was a bug in 4.x that has since been fixed?
You have to use a release of SwiftPM that corresponds to the compiler you want it to work with. You should not be linking against a snapshot unless that snapshot is what you are using to build with and that snapshot is what SwiftPM should communicate with at runtime. Mixing and matching isnât supported; where it happens to work, you are just lucky.
SwiftPM 0.4.0 corresponds to Swift 5.0.x.
SwiftPM 0.3.0 correspondes to Swift 4.2.x.
I donât remember the older pairs off the top of my head.
I don't think we're mixing and matching like you suggest, at least to my understanding. Please help me understand what you mean if this is not the case.
Then it is using Swift 4.2.1, so depend on .exact("0.3.0") instead of .revision("swift-DEVELOPMENT-SNAPSHOT-2019-03-04-a") (here).
But the result will not be able to parse newer manifests, so it would be wise to upgrade to Swift 5.0 (Xcode 10.2) so you can also upgrade to SwiftPM 0.4.0.
That would be why the Package.resolved is found to be invalid/stale by Swift 4.2.1. Having Package.resolved checked in only means it does not have to check out multiple Package.swift files per repository to find the best compatible revision. It still must parse the resolved manifest to know what the targets are and where their sources reside.
This it totally unexpected, at least for me. It is de facto preventing build from being reproducible.
It's a strange design decision if I may. You see here how this is clearly creating issues and I can see this being a problem in other settings too.
A build with a given (valid) .resolved file and a given version of Swift (and on a particular platform) is reproducible. Two different versions of Swift can result (logically, I think) in two different builds, just as the build on Linux can differ from the build on macOS.
Imagine you had an #if os(Linux) dependency in your manifest at 1.0, and the same dependency at 2.0 for #if os(macOS). If you check in your .resolved on one platform, it will appear stale on the other, since it conflicts with the package manifest. The package manager will discard the invalid .resolved and try again from scratch. Its behaviour in this situation is the exact same as when you add a new dependency to the manifest and then rebuild.
What happened in your case is that someone updated the pins with Swift 5.0, which is comparable to being under an #if compiler(>=5.0) statement. (Probably this commit.) From then on, whenever someone (such as CI) checks it out to build with Swift 4.2, the new build is comparable to being under an #if complier(<5.0) statement. The .resolved pins are inconsistent with the manifest, and so the package manager starts resolution over from scratch. Basically for the last while everything in your CI has been behaving the same as if .resolved were not checked in at all.
If you want your CI to guard your repository against entering this state, you can validate that git diff Package.resolved reports no difference after the CI build. Then you know the pins were valid and it used them.
However, this only actually grew into a problem because you depend on branches (albeit indirectly), which by definition are everâchanging. With a (valid) .resolved file, branches are pinned in place, but without one, they change with every fresh checkout. That is the point of specifying a branch. Pins are ignored by client packages and for this reason branch dependencies are only allowed during development. Your real problem here is that your branch dependency outran you and evolved too far.
To avoid runaway branches, only declare dependencies on semantic versions of packages you donât control. A semantic version of one package cannot depend on a branch of anotherâSwiftPM will object. So you can rest assured that there are no branches sneaking in lower in your dependency graph.
Swift 4.2 ought to select it. (LLBuild 0.1.1âused by SwiftPM 0.4.0âhas a @swift-4.2 too.) The documentation for such files is here.
The problem is that I donât think LLBuild was ever completely usable as package dependency before the extra linker flags enabled by the Swift 5 manifest format. You can try feeding the extra flags to each $ swift build invocation, but your own clients may not like having to deal with them themselves too.
It looks like the llbuild folks goofed and their 0.1.1 @swift-4.2 manifest is way out of date. Your error is because it declares an llvmDemangle target which does not actually exist in the file system.
The @swift-4.2 manifest is gone on master altogether, so I suspect the real mistake was not so much that the manifest fell behind, but that it wasnât deleted when Swift 4.2 was no longer supported.