Error during package resolution on CI machine whenever updating package dependencies

On a CI machine I'm using xcodebuild with automatic package resolution disabled, like this:

    /Applications/ -resolvePackageDependencies -scheme "SJ Debug" -project SJ.xcodeproj -derivedDataPath "~/Library/Developer/Xcode/DerivedData" -scmProvider system -disableAutomaticPackageResolution

This works, however, whenever I update a package and the Resolved.packages file is changed I get errors of the following kind:

xcodebuild: error: Could not resolve package dependencies:
  Couldn’t check out revision ‘8354a50fe01a7e54e196d3b5493b5ab53dd5866a’:
    fatal: reference is not a tree: 8354a50fe01a7e54e196d3b5493b5ab53dd5866a
  an out-of-date resolved file was detected at /repo.git/SJ.xcodeproj/project.xcworkspace/xcshareddata/swiftpm/Package.resolved, which is not allowed when automatic dependency resolution is disabled; please make sure to update the file to reflect the changes in dependencies

The real error is that git checkout fails, the rest of the error message is not correct. The question then is why does it fail?

If I wipe the package cache things will work again.

Also, if I add the flag -clonedSourcePackagesDirPath SourcePackages then package resolution also works (at least if SourcePackages is an empty directory).

On the machine xcodebuild is invoked by fastlane which invokes xcodebuild which uses swift package manager (some built in version of it?). So it is hard to know where to actually report this error but I feel like it must be affecting lots of people.

Does anyone here have any insight into this?

The actual error output looks like it is coming from here: swift-package-manager/Workspace.swift at 7a75eb8cfdba5a8c73914db3ad08e83b8051e68d · apple/swift-package-manager · GitHub

I believe the real reason behind the "not a tree" error may be that something is wrong with the git invocation perform by swift package manager (wrong working dir or something like that).

This is known issue with -disableAutomaticPackageResolution in versions prior to 5.7 -- it essentially behaves like --skip-update so any checkout state that's present will not be updated. It was fixed here: Ensure checkouts get updated when enforcing resolved versions by neonichu · Pull Request #4114 · apple/swift-package-manager · GitHub

Thank you! I have been struggling with this issue for months and haven't been able to pinpoint the problem. Great to know that it's been fixed. I guess now all I can do is to use the workaround that checks things out from scratch until the fix has made it into Xcode.

Strangely this is also happening for our CI builds but its only started failing recently. I'm sure we were able to add new packages or update existing ones without this happening. We had a pre-build step that explicitly updated dependencies and cached the cloned sources directory and then passed disableAutomaticPackageResolution to xcodebuild in the build step. We've had to remove this switch for now.