SPM support basic auth for non-git binary dependency hosts

My company provides limited public access to our iOS SDK binary dependencies in a Nexus repository. Developers are required to supply a login and generated token to authenticate.

This has not been an issue for Cocoapods support, since cURL supports the gnu .netrc file; developers can install a file at ~/.netrc containing the credentials as follows, to have them utilized for download.

machine repositories.sap.ondemand.com 
    login sap-xxxxxx 
    password xxxxxxxxxxxxxxxxxx

For SPM, is there a similar technique that can be used? I've seen recommendations to use Keychain--not sure yet if that is correct. Ultimately, I'm looking for something that is CI compatible, though in the short term something which can be used by individual developers would be useful also.

2 Likes

Unfortunately, there is no way to do this today, but this seems like a good potential enhancement for the binary dependencies feature.

Boris, does the .netrc approach sounds legit or we have to consider something else like keychain?

That's a good question. I think we should not have a Keychain-only solution, because that would be problematic once we add support for binary dependencies to non-Apple platforms in the future. So maybe the way to go would be supporting .netrc but also more secure, platform-specific credential stores such as Keychain on macOS.

FYI, this feature was [finally] added to Carthage last year. I'll investigate incorporating some of the implementation.

1 Like

I have a working version by copying the Netrc.swift implementation from Carthage to TSCUtility, and making the following change to Downloader.swift:

    public func downloadFile(
        at url: Foundation.URL,
        to destination: AbsolutePath,
        progress: @escaping Downloader.Progress,
        completion: @escaping Downloader.Completion
    ) {
        queue.addOperation {
            
            var request = URLRequest(url: url)
                        
            switch Netrc.load() {
            case let .success(netrc):
                if let authorization = netrc.authorization(for: url) {
                    request.addValue(authorization, forHTTPHeaderField: "Authorization")
                }
            case .failure(_): break // Do nothing
            }
            
            let task = self.session.downloadTask(with: request)
            
            let download = Download(
                task: task,
                destination: destination,
                progress: progress,
                completion: completion)
            self.downloads[task.taskIdentifier] = download
            task.resume()
        }
    }

This doesn't yet require the user to opt-in to use netrc via a command line option.

In Carthage implementation, this flag is named use-netrc, and defaults to false. This is functionally equivalent to --netrc-optional below.

Curl provides three (3) options:

  • -n, --netrc: Makes curl scan the .netrc (_netrc on Windows) file in the user's home directory for login name and password.
  • --netrc-optional: Very similar to -n, --netrc, but this option makes the .netrc usage optional and not mandatory as the -n, --netrc option does. // ie, if no netrc file is located at the user's home directory, or no matching machine is found, an error will not be thrown.
  • --netrc-file <filename>: This option is similar to -n, --netrc, except that you provide the path (absolute or relative) to the netrc file that curl should use. You can only specify one netrc file per invocation. If several --netrc-file options are provided, the last one will be used. It will abide by --netrc-optional if specified.

@NeoNacho what are your thoughts on adding some variants of the --use-netrc or curl netrc options to the swift package API?

Next steps for me are to copy the tests from Carthage over (needs stripping out the Quick/Nimble dependencies), and also, I'm seeing the unzip procedure take an unexpectedly long time, need to debug.

2 Likes

Opened PR with parsing & loading implementation: swift-tools-support-core#88.

Features supported:

  • Parse Netrc string text: single and multi-line
  • Match machine, login, password key values
  • Ignore comments
  • Ignore account and macdef entries (relevant only to ftp remotes)
  • Accept any order of login, password, account key values
  • Match default connection settings.
  • Validates single instance of default connection setting, and that default connection setting is last entry

TODO:

  • Add command line options for activating & configuring Netrc loading behavior
    • use netrc file if available
    • specify location of netrc file
  • Add netrc loading tests to DownloaderTests
1 Like

Great! A CLI option for opt-in doesn't seem necessary to me -- presumably if you have a netrc, you want it to be used. The --netrc-file <filename> option does seem useful to use an alternative file.

2 Likes

@NeoNacho what is the guiding strategy for platform support? Is it reasonable to fence the feature to >=OSX 10.13? Or should all >= 10.10 be supported?

Reason for question is the regex implementation relies on 'named capture groups', which is supported in NSTextCheckingResult from 10.13. I could supply a custom 'named capture group' implementation which is used for <10.13, but tbh it seems less robust than the native NSTextCheckingResult.range(withName:).

I think it's fair to increase the deployment target of SwiftPM to 10.13 as part of your PR.

1 Like

@NeoNacho this bumps swift-driver min platform to 10.13 also... acceptable?

Oh because this is in TSC? Good question, @Douglas_Gregor what do you think?

That’s fine by me!

1 Like

@NeoNacho swift-tools-support-core/pull/88 is ready for review. I see merges happening to the codebase in both spm and the tools-support-core repos... should this be submitted directly to spm as directed in the readme?

Thanks!

Right now, we would need the change in both, we will be able to get rid of the vendored copy of TSC in SwiftPM's repo soon, but are not quite there, yet.

SPM mirror of the PR here: Add support for Netrc for Downloader by sstadelman · Pull Request #2833 · apple/swift-package-manager · GitHub

Thank you! Sorry that I haven't gotten to review your PR in detail, yet, I plan on getting to it soon.

np thanks!

Minor update: since corelibs-foundation does not yet have an implementation of the required NSTextCheckerResultmethod, the implementation is now fenced with #if os(macOS) & #available(macOS 10.13). This has the nice side effect of not obligating the swift-driver min-platform bump.

2 Likes

Hi Stan. We are also interested in this feature. I see a thread on Github where your changes are being reviewed. Do you know when this could become available as part of an official xCode release? Thanks!