1

I’ve got a swift package and I’m running into an issue when switching branches from main to a branch that has updated a dependency. When I swift build it causes the Package.resolved to be changed back to the version from main . I expect it’s getting the versions based on the checked out dependencies in .build since running swift package reset before swift build will make it use the versions in Package.resolved . I’ve only been able to reproduce it using Apollo but nothing looks weird about their package. Any other ideas for what the cause could be?

I've filed a bug for it with a package/git repo that reproduces it.
https://bugs.swift.org/browse/SR-14647?filter=-2

1 Like
2

So I also posted about this on the Apollo repo here

https://github.com/apollographql/apollo-ios/issues/1799

I was shared a link to a twitter conversion from @NeoNacho that said by default the Package.resolved can be changed. I'm still unsure why SPM thinks the package couldn't build since it was just successfully built before switching branches.

I guess it's not a bug if it's the intended behaviour. But it's certainly not what I expected the default behaviour to be. I figured Package.resolved would be followed be default unless you were doing something like swift package update.

1 Like
3

What he means by the intended behaviour is that if Package.resolved does not match Package.swift, then it will simply be resolved anew. That is what normally happens the first time you build after making changes to the manifest. But if the pins already satisfy the manifest, then those pins should be used exactly as they are—otherwise it would defeat the purpose of having pins at all.

Over time there have occasionally been individual bugs where the pins would appear invalid when loaded from .resolved, but end up actually valid after full graph resolution. Such bugs have caused the sort of behaviour you observe, where the package manager ends up discarding the very pins it just wrote.

One way you can force this sort of thing without it being a bug is if you #if os(macOS) and #if os(Linux) in the manifest to create two mutually exclusive dependency constraints. Then every time you check out the package on the opposite platform it will discard the invalid pins—legitimately so—and then resolve afresh, quite possibly settling on different versions than last time it was resolved on that platform. Such a package undermines the point of the Package.resolved file and will never be able to use pins in any meaningful way.

The last bug I saw that made the package manager stumble into such behaviour erroneously was a mishandling of the .git suffix. The same dependency was declared in one place with .git on the end, and in another without it. The pin loading code thought they were different, but the resolution code recognized them as the same, and thus the package was functional, but the pins were always discarded.

I would inspect the dependency graph for some sort of variation in the way the dependencies are declared. If necessary, you can trim them from the graph one by one until the problem goes away in order to zero in on the particular dependency declaration that confuses the package manager. If you can narrow it down to a particular cause, a bug report would be appreciated at bugs.swift.org. You may also be able to dodge the issue by adjusting the declaration in some way, so that you can get your pins working immediately without waiting for an update to Swift.

4

We do add Package.resolved to .gitignore by default.

This way we dont come across any git issues when eg switching between branches.
Sometimes when switching the git branches - I dont have repro steps atm - the Package.resolved is regenerated even when the package dependencies used by Xcode project are not changed - the list of dependencies in the Package.resolved file end up reordered - a change that gets tracked by git in the end. The
above mentioned issue is related to the nature of unordered dictionary.
This was the primary issue that lead us to just add the Package.resolved file to gitignore.

5

This is currently the simplest way I've found to reproduce it.

  1. Start on a branch with the following dependency graph in the Package.resolved.

    MyPackage
  - Starscream@4.0.3
    - swift-nio-zlib-support@1.0.0

  2. Run swift build

  3. Switch to another branch with following dependency graph in the Package.resolved.

    MyPackage
- Starscream@4.0.4

  4. Run swift build again.

It will cause Package.resolved to be reset back to the initial value. My current theory is it has something to with a sub dependency being removed but it's really difficult to test. Starscream did move adding swift-nio-zlib-support into a #if os(Linux) but I'm only ever running on macOS so it should just appear like the subdependecy was removed.

I have already created a bug for it (see the first post). I'll update it with an even simpler package reproducing the issue, but the bug tracker seems to be down at the moment (returning 503).

6

I assume you are speaking of the same project as @CraigSiemens?

With Package.resolved registered in .gitignore, the expected behaviour would be that any new clone would resolve afresh, because no pins are travelling with the clone. When switching branches, Git would not touch the pins file, and so SwiftPM should use the pins as long as they still satisfy the constraints in the manifest. If the manifest changed with the branch switch then the pins may no longer satisfy it, in which case SwiftPM will discard them, resolve anew and write out those pins. Anything besides this behaviour would indicate a bug.

Xcode is a whole different can of worms because it seems to reimplement parts of SwiftPM separately and it has a lot of bugs that SwiftPM itself does not.

I noticed in at least Xcode 12.4 that Xcode seemed to erroneously load its checkouts before looking at the pins, and since the last loading step is registering the pins, it would consistently revert the pins to match any existing checkout. That meant that any change to the pins by SwiftPM, by Git or by hand would be erased upon opening Xcode. The only remedy was to delete the derived data directory each time you make such a change. I do not know if 12.5 still suffers from that problem.

Other relevant Xcode bugs include differences in the way it handles percent encoding of dependency URLs, and other URL normalizations such as capitalization, .git, etc. That can in turn lead to Xcode and SwiftPM rejecting each other’s pins.

So if Xcode is breaking your pins, I am entirely unsurprised, but you are asking in the wrong place to get a fix. You will have to try your luck with the Feedback Assistant.

I have made a habit of ensuring that I choose either SwiftPM or Xcode as “standard” for each particular project and watch to make sure I erase any changes the opposite tool makes to the pins before I check them in.

This is not the case. The pins file is supposed to be stable. Unless one of the corner case bugs above is triggered, it should come out identical every time.

So if it is changing, then it is tripping something unusual. The same URL with differing capitalization is an example of the sort of thing to look for, because that used to be able to throw off the string sort position or hash depending on which declaration the resolver encountered first and decided was canonical. This destroyed the deterministic order, but it had nothing to do with an unordered dictionary. I do not know if that particular bug still persists, but it gives you an idea of the sort of thing that might be going on.

I will try to take a closer look and get back to you.

7

Very strange. To be more specific, I can confirm that if...

  • the dependency declaration is verbatim .package(url: "https://github.com/daltoniam/Starscream", from: "4.0.3") in both branches,
  • both branches have Package.resolved checked in, and
  • one branch has 4.0.3 pinned and the other 4.0.4...

...then building the 4.0.3 branch, switching to the 4.0.4 branch, and building again will revert the 4.0.4 branch’s pins to match those of the 4.0.3 branch still checked out.

It is a very narrow bug, because everything must be just so.

  • Deleting the .build folder in between fixes the problem.
  • If the 4.0.4 branch’s manifest matches the pin at from: "4.0.4" (usually a good idea anyway) the problem does not occur.
  • With Package.resolved ignored instead, everything behaves as it should.
  • Doing the same thing with the https://github.com/apple/swift-numerics package instead works as expected.

I was using Swift 5.4 on macOS.

It is unrelated to this bug, but it would wise to use conditional dependencies for this sort of thing instead.

8

Jira is finally working again so I've updated the bug I filed to contain the simpler example that just contains Starscream as a dependency.

https://bugs.swift.org/browse/SR-14647?filter=-2

9

This worked for me
https://stackoverflow.com/a/69793517/9440709

1 Like