If any of the fine Apple folks are listening, I have today filed a feedback FB13428167.
In short, what I have found is that when Xcode initially resolves package dependencies, and clones everything into DerivedData it will include refs/remotes for each git@github.com dependency. Not sure why it does that and how, but refs/remotes is never present for any public https:// backed dependency. Only for git@ ones.
This refs/remotes directory will then somehow confuse Xcode and "Update to Latest Package Versions" will always fail with the error mentioned above.
Removing the refs/remotes directory will make Xcode happy indefinitely, until you delete the DerivedData folder, when the mess starts again.
I am currently using the following script to make the Xcode happy even while it is running (Note that I am also cleaning the ~/Library/org.swift.swiftpm repository clones, but this proved to not be necessary for Xcode to work. The DerivedData bit is what is important).
€ cat ~/fix_spm_cache.sh
#!/bin/zsh
echo "Looking for problematic dirs..."
find ~/Library/Caches/org.swift.swiftpm/ -name remotes -type d
find ~/Library/Developer/Xcode/DerivedData/ -name remotes -type d | grep -vw checkouts
echo "Getting rid of them..."
find ~/Library/Caches/org.swift.swiftpm/ -name remotes -type d | xargs rm -r
find ~/Library/Developer/Xcode/DerivedData/ -name remotes -type d | grep -vw checkouts | xargs rm -r
This issue has been present in Xcode since initial SPM support was introduced and according to my experience it strictly affects only git@github.com based dependencies. The git@ dependencies work great in Swift Package Manager itself nicely, only Xcode is affected.