Swift package update fails if run in NSTask "Couldn’t fetch updates from remote repositories"

Using Swift
1

I'm getting a fairly strange error. I have a Makefile that runs swift package update to get down all the transitive dependencies required by a project and specified in the local Package.swift file (and then on to their dependencies).

This works fine if I run make all in a terminal but if I run make all from an NSTask in an XPC Service run by my mac app, the package update doesn't work and I get errors like...

swift package update
Fetching https://github.com/microswift-packages/Adafruit_BluefruitLE_nRF51 from cache
warning: skipping cache due to an error: Couldn’t fetch updates from remote repositories:
Fetching https://github.com/microswift-packages/Adafruit_BluefruitLE_nRF51 from cache
warning: skipping cache due to an error: Couldn’t fetch updates from remote repositories:
error: Failed to clone repository https://github.com/microswift-packages/Adafruit_BluefruitLE_nRF51:

I'm pretty stumped what's going on here? Is SPM needing some kind of permissions? This is not a sandboxed mac app (it's DeveloperID) and not a sandboxed XPC Service. I thought it might be some kind of permission issue, but I can run a cURL command in the same makefile and it works fine, retrieving arbitrary data from my own webserver.

Can anyone suggest why SPM might be failing here? I see that it creates the usual .build subfolder and the usual subdirectories inside that (artifacts, checkouts, repositories) so SPM is starting. Looking at SPM source code, internally it seems to fail trying git remote -v update -p at a guess... but running the same command in Terminal in the same current directory the NSTask uses works just fine! I'm not even using SSH or any security, these are public repositories.

Any help anyone can give would be greatly appreciated!

1 Like
2

GitHub has a very low rate limit for unauthenticated checkouts, even for public repos. My first guess is when you run Git in Terminal it’s using your user configured ssh keys to authenticate, but when it’s through NSTask, it has nothing.

3

I'm not doing a git command from the command line. I'm running make which in turn is running a load of commands (that all work fine in both contexts) and also it's running swift package update which seems to be the bit of the makefile that fails when make is being run in an NSTask by the IDE but works fine when make is just run normally in a Terminal (same working directory, same user). The SPM package dependencies are declared with https so they are using the identical same transport in both cases. The only difference is in one case make is run from the command line interactively and in the not working case make is run in an NSTask in an XPC service. It's 100% consistent and reliable, SPM works on the command line, always fails in NSTask. And the failure is instant, there's no obvious timeout. So I don't think it can be the git transport mechanism.

4

Everything I can find online points to Github authentication. One suggestion:

Use git ssh based authentication to avoid authentication issue. e.g. change from https:// to git@.

Another fixed it where the Git URLs were pointed to the repositories' http pages with anchor links, rather than the .git: Why can my Swift app not be built with xcodebuild, but it can be built with XCode? - Stack Overflow

I did try

let taskProcess = Process()
taskProcess.launchPath = "/usr/bin/ssh"
taskProcess.arguments = [
    "-vT", "git@github.com",
]
try taskProcess.run()

and it picked up my SSH keys correctly inside the task run.

1 Like
5

That's helpful! I think your approach is good, basically start from running a Process with a playground or similar and slowly build up one step at a time until we get it working or get an informative failure.

Axel Roest (@axello) gave me the pointer that I think solved it! He said check the environment variables I'm passing in. Sure enough, in my NSTask I pass no environment. If I add PATH it seems to fix it in my local tests!

What a silly small thing I got stuck on.

Thank you so much everyone! I hope this thread helps someone else as lost as I was!

2 Likes
6

Wow, esoteric old unix knowledge is actually helpful!
:smiling_face:

7

Everything is unix. Unix ate the world. :laughing:

p.s. For anyone reading this thread, when I run make from the command line normally, the SPM command it triggers works fine.
If I run it with env -i make (in other words I clean the environment before running make) then I get the exact same weird error messages on the command line that I saw from NSTask!
Just adding PATH=/usr/local/bin:/usr/bin:/bin in the environment is enough to fix SPM inside make!

1 Like