Package URL Conflicts

I tried searching around for this but can't find a topic or bug ticket.

The tl;dr is that the URL used for a package within the Package dependencies value causes a conflict based on case or the inclusion of an extension, rather than validating against the repository itself.

For example, say I have iOS Project A which imports Swift Package B. if iOS Project A uses Alamofire but imports it via "https://github.com/alamofire/alamofire" and Package B uses Alamofire but imports it via "https://github.com/Alamofire/Alamofire", SPM is unable to resolve dependencies because it sees these as two individual packages. In Xcode, this then spins indefinitely with no proper error output.

This is annoying but resolvable within a project I own, where I now know of the issue and can simply update the packages, but if iOS Project A imports Swift Package B, which imports Alamofire using either of the URLs above, but then iOS Project A imports the AlamofireImage project, which is outside of my organization and imports Alamofire via another different URL "https://github.com/Alamofire/Alamofire.git", there's now a conflict and I have to update my packages to match this dependency to resolve them.

Which version of Xcode are you seeing this with?

Yes. The question of “To .git or not to .git?” annoyed me for quite some time. SwiftPM would succeed, but always believe it’s own pins were invalid and re‐resolve for every operation. Xcode would just crash trying to load it. Eventually I weeded all the differences out, so I don’t know where it is and isn’t still a problem.

But I’m not sure what the fix is. Technically .../X, .../x are legitimately distinct URLs and only refer to the same package if it happens to be hosted from a case‐insensitive file system. Git might require .../X.git and .../x.git to be the same as their base names, but I don’t really know. Someone would have to check its specifications.

It might be possible to add a manifest entry where alternate URLs could be specified, which SwiftPM could then treat as equal, similar to its mirror handling. But I’m not sure if that would be useful enough to invest the effort, because it wouldn’t have any effect unless the vendor foresaw clients using the alternate URL and prepared for it.

Either way, as a matter of practice, it is best if each package provides an example dependency declaration in its documentation, which displays the URL it intends as its canonical location. That would be the single source of truth, and any client who spells it differently would be the one at fault for any incompatibilities it causes.

1 Like

Sorry I missed this message. It's an underlying SPM issue, not necessarily an Xcode issue, but I've had it occurring in all versions of Xcode 11.

All good points. How do other package managers do it? I've never run into the issue anywhere else, granted I think in almost all cases I'm using a centralized package manager which reduces the complexity by having a single source, but there has to be a way to resolve this from a decentralized standpoint.

Even if we didn't want to restrict on case-insensitive keys, there's clearly a moment when the packages are cloned that incurs a conflict. At this point, couldn't we check the latest commit hash for both packages at that time and, if they match, safely assume they are the same repository?

I'm just spitballing.

1 Like

Xcode 11.4 introduced a new dependency resolver which should help with these kind of issues. That's why I was wondering if you're still seeing this with a recent Xcode.

It's unfortunately still occurring as of 11.4.1 and 11.5 beta in Xcode.

Would be great if you could file a bug with a reproducer, thanks!

1 Like

I've hit this a few times. Luckily in repos I'm an owner of so I can fix, but always more painful to debug and track down than I expect given the error messaging. The bug does appear to be non-deterministic, but here's an open source project that exhibits the issue on CI, and I was able to reproduce it:

The error is "Missing package product 'XCTestDynamicOverlay'", which comes from a package that is depended on both upstream in several libraries, where ".git" is omitted from the repo name:

And then downstream, where the URL included ".git": isowords/Package.swift at 2db490517a7768cd8d82807eabb33bf7ae1418ce · pointfreeco/isowords · GitHub

Removing the ".git" has things reliably building again, but it'd definitely be nicer if SPM could "canonicalize" URL differences like these or at the very least provide error messaging that points directly to these mismatches.

Edit: filed https://bugs.swift.org/browse/SR-14393

The most robust way to canonicalise URLs would be if we defined our own scheme (e.g. swift-package://github.com/user/package). Then we could make up whichever rules we like about whether a .git extension is meaningful, or if the path is case-insensitive.

As long as we use http(s) URLs, the only reasonable thing to do is to follow the HTTP standard, which says:

The scheme and host are case-insensitive and normally provided in lowercase; all other
components are compared in a case-sensitive manner.

EDIT: If you want to go further down the rabbit hole, consider this:

import Foundation

let url1 = URL(string: "https://example.com/%66%6f%6f")!
let url2 = URL(string: "https://example.com/%66%6F%6F")!
let url3 = URL(string: "https://example.com/foo")!

print(url1 == url2) // false
print(url2 == url3) // false
print(url1 == url3) // false

RFC 3986 says that all of these URLs should be considered equivalent. Apparently hardly any implementations bother to do that (it is by no means just Foundation), which IMO is part of the reason why, every so often, people who thought they understood URLs suddenly realise that they don't understand URLs.

2 Likes

Yeah, I definitely agree, though short of canonicalizing the URL, if SPM fetches both remotes shallowly and finds they resolve to the same Git SHA/fingerprint, it seems that SPM could "safely" proceed and assume both packages are the same.

But at the very least I'd welcome an error message that makes it clear that there's a conflict between two packages that have the same name.

This will likely be fixed in the near future if it hasn’t been already. The current iteration of SE‐0292 includes these canonicalization rules.

1 Like

Our original proposal used URLs as package identifiers. However, our latest revision of SE-0292 proposes a new package identifier.

I'd be interested to see whether Swift Package Manager adopts the URL canonicalization we implemented originally, independent of the package registry.

1 Like

Interesting. Regardless, a different identifier would also eliminate the issue.

Many PRs have been merged in the last few months related to package identity. All the relevant code has been recently overhauled to solve a host of similar issues. I haven’t exhaustively read all the PRs, so I don’t know for sure, but my impression is that what was done should have already eliminated the particular problem discussed here.

@SDGGiesbrecht yes, the goal of those fixes was to eliminate the systematic identity issue we had in the code-base which led to many negative side effects, possibly including issues in target based dependencies resolution. i'd love to see the team re-engage in completing this feature.

cc @NeoNacho

1 Like
Terms of Service

Privacy Policy

Cookie Policy