About reproducible builds

Development Package Manager
1

Hello,

over at Carthage we're experiencing some issues with builds being not reproducible.

It seems that Package.resolved is not the final source of truth.

SwiftPM adds a dependency on swift-llbuild master and this seems to cause Package.resolved to be ignored. Even when the contents of the resolved file are the following and --skip-update is specified:

      {
    "package": "llbuild",
    "repositoryURL": "https://github.com/apple/swift-llbuild.git",
    "state": {
      "branch": "master",
      "revision": "3aeecb428d202afe15633266dc862de27feab723",
      "version": null
    }
  },

The current Package.swift at swift-llbuild master is for version 5 and it breaks usage with previous versions (which carthage is using).

https://github.com/apple/swift-llbuild.git @ master: error: manifest parse error(s):
/var/folders/b4/qz7m1r9x3mj1bm3nfsjjf64m0000gp/T/TemporaryFile.uoaa3X.swift:55:16: error: incorrect argument label in call (have 'name:dependencies:path:linkerSettings:', expected 'name:dependencies:path:exclude:')
    .target(
           ^

As you can deduce, this behavior of SwiftPM is problematic or at lease surprising, especially since once would expect builds to be reproducible. I would be grateful if anyone could shed some light.

1 Like
2

I don’t know what is happening here for you. With Swift 5.0.1, I cloned and built and it respected Package.resolved. Maybe there is was a bug in 4.x that has since been fixed?

You have to use a release of SwiftPM that corresponds to the compiler you want it to work with. You should not be linking against a snapshot unless that snapshot is what you are using to build with and that snapshot is what SwiftPM should communicate with at runtime. Mixing and matching isn’t supported; where it happens to work, you are just lucky.

  • SwiftPM 0.4.0 corresponds to Swift 5.0.x.
  • SwiftPM 0.3.0 correspondes to Swift 4.2.x.

I don’t remember the older pairs off the top of my head.

3

Thanks for your answer.

Carthage is building on Xcode10.1 using the bundled SwiftPM.

Carthage also depends on the swift-pm library which it links statically Swift Package Manager: Accommodate any swift toolchain without a `swi… · Carthage/Carthage@427ed02 · GitHub, specifically snapshot swift-DEVELOPMENT-SNAPSHOT-2019-03-04-a

I don't think we're mixing and matching like you suggest, at least to my understanding. Please help me understand what you mean if this is not the case.

With Swift 5.0.1 you won't see the problem because the Package file will parse. The issue reproduces reliably on CI Travis CI - Test and Deploy Your Code with Confidence

4

Then it is using Swift 4.2.1, so depend on .exact("0.3.0") instead of .revision("swift-DEVELOPMENT-SNAPSHOT-2019-03-04-a") (here).

But the result will not be able to parse newer manifests, so it would be wise to upgrade to Swift 5.0 (Xcode 10.2) so you can also upgrade to SwiftPM 0.4.0.

That would be why the Package.resolved is found to be invalid/stale by Swift 4.2.1. Having Package.resolved checked in only means it does not have to check out multiple Package.swift files per repository to find the best compatible revision. It still must parse the resolved manifest to know what the targets are and where their sources reside.

5

Thanks, it's more clear now.

This it totally unexpected, at least for me. It is de facto preventing build from being reproducible.
It's a strange design decision if I may. You see here how this is clearly creating issues and I can see this being a problem in other settings too.

6

A build with a given (valid) .resolved file and a given version of Swift (and on a particular platform) is reproducible. Two different versions of Swift can result (logically, I think) in two different builds, just as the build on Linux can differ from the build on macOS.

Imagine you had an #if os(Linux) dependency in your manifest at 1.0, and the same dependency at 2.0 for #if os(macOS). If you check in your .resolved on one platform, it will appear stale on the other, since it conflicts with the package manifest. The package manager will discard the invalid .resolved and try again from scratch. Its behaviour in this situation is the exact same as when you add a new dependency to the manifest and then rebuild.

What happened in your case is that someone updated the pins with Swift 5.0, which is comparable to being under an #if compiler(>=5.0) statement. (Probably this commit.) From then on, whenever someone (such as CI) checks it out to build with Swift 4.2, the new build is comparable to being under an #if complier(<5.0) statement. The .resolved pins are inconsistent with the manifest, and so the package manager starts resolution over from scratch. Basically for the last while everything in your CI has been behaving the same as if .resolved were not checked in at all.

If you want your CI to guard your repository against entering this state, you can validate that git diff Package.resolved reports no difference after the CI build. Then you know the pins were valid and it used them.

However, this only actually grew into a problem because you depend on branches (albeit indirectly), which by definition are ever‐changing. With a (valid) .resolved file, branches are pinned in place, but without one, they change with every fresh checkout. That is the point of specifying a branch. Pins are ignored by client packages and for this reason branch dependencies are only allowed during development. Your real problem here is that your branch dependency outran you and evolved too far.

To avoid runaway branches, only declare dependencies on semantic versions of packages you don’t control. A semantic version of one package cannot depend on a branch of another—SwiftPM will object. So you can rest assured that there are no branches sneaking in lower in your dependency graph.

7

Thanks for the detailed explanation. This all makes sense now.

I see that at swift-llbuild-3aeecb428d202afe15633266dc862de27feab723 there is a @4.2 package.

It is possible to use that instead of Package.swift?

8

I think the answer is “sort of”.

Swift 4.2 ought to select it. (LLBuild 0.1.1—used by SwiftPM 0.4.0—has a @swift-4.2 too.) The documentation for such files is here.

The problem is that I don’t think LLBuild was ever completely usable as package dependency before the extra linker flags enabled by the Swift 5 manifest format. You can try feeding the extra flags to each $ swift build invocation, but your own clients may not like having to deal with them themselves too.

9

I think you're running into an old bug where SwiftPM doesn't respect the SHA in Package.resolved file during package resolution for branch-based dependencies if the resolution is started from scratch. I have a patch in works to fix this issue: Respect resolved file for branch-based dependencies by aciidb0mb3r ¡ Pull Request #2197 ¡ apple/swift-package-manager ¡ GitHub

10

Hello again!

We tried with a stable tag 0.4.0 but unfortunately it exhibits a whole other set of issues:

error: could not find source files for target(s): llvmDemangle; use the 'path' property in the Swift 4 manifest to set a custom target path

11

Well, 0.4.0 works fine when built with Swift 5.0.

It looks like the llbuild folks goofed and their 0.1.1 @swift-4.2 manifest is way out of date. Your error is because it declares an llvmDemangle target which does not actually exist in the file system.

12

The @swift-4.2 manifest is gone on master altogether, so I suspect the real mistake was not so much that the manifest fell behind, but that it wasn’t deleted when Swift 4.2 was no longer supported.

13

I created another thread over in the LLBuild category:

Hopefully they will have an answer for you.

14

Thank you very much! Very kind