SwiftPM vs. Xcode build behavior

Using Swift
,
1

I have some swift packages that since several months exhibit strange behavior: When building with swift on the command line using swift build, I get errors about missing or duplicated symbols, e.g.:

Building for debugging...
error: link command failed with exit code 1 (use -v to see invocation)
Undefined symbols for architecture arm64:
  "(extension in Swift_Automotive_Core):Swift.UInt8.vin.unsafeMutableAddressor : Swift.UInt8", referenced from:
      ECUmulation.KWPDiagnosticSession.configure(property: Swift.String, value: Swift.String) -> () in KWP.swift.o
      ECUmulation.KWPDiagnosticSession.configure(property: Swift.String, value: Swift.String) -> () in KWP.swift.o
  "(extension in Swift_Automotive_Core):Swift.UInt8.scaling.unsafeMutableAddressor : Swift.UInt8", referenced from:
      ECUmulation.KWPDiagnosticSession.performSessionSetup() -> () in KWP.swift.o
  "(extension in Swift_Automotive_Core):Swift.UInt8.dataTable.unsafeMutableAddressor : Swift.UInt8", referenced from:
      ECUmulation.KWPDiagnosticSession.performSessionSetup() -> () in KWP.swift.o
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

When opening this package with Xcode, the build succeeds. This is building on the same machine.

I can't disclose the proprietary package and I did not have a chance to build a minimal example, but perhaps this behavioral difference rings a bell for someone, so I figured, I'd ask anyways.

Thanks.

2

Haven't seen anything like that. Please raise an issue on swift-package-manager and if you can provide a minimal example to help us reproduce it for debugging.

3

I've seen these sorts of issues very frequently, most of the time it's because a dependency is missing. In your .target(name: "ECUmulation") do you have "Swift_Automotive_Core" listed in dependencies?

And did you try swift build --explicit-target-dependency-import-check error?

Regardless, this sounds like a Swift/SwiftPM bug. Swift/SwiftPM shouldn't let you import Foo unless you declared a dependency on Foo. And currently this still often works coincidentally if Foo happens to have been compiled ahead of the module in question.

1 Like
4

Thanks. Indeed I missed Swift_Automotive_Core in the list of dependencies, I had a package that internally includes it in the dependencies. That said, adding it (and doing a full rebuild) did not change the situation.

swift build --explicit-target-dependency-import-check error did not lead to anything different, neither before nor after fixing the dependency.

What's even more strange: I have two machines, on the 20 core machine it even builds with swift build, on the 11 core machine it doesn't. That would fit your theory that build parallelisation also has something to do with it.

1 Like
5

Yes, missing dependencies often show behaviour similar to what you describe and parallelism is pretty much key. Without parallelism, it usually is deterministic at least for the same build system. Not guaranteed, but more likely to be deterministically succeeding or deterministically failing. But of course SwiftPM vs. Xcode is sadly a completely different build system today.

If you don't mind filing a bug that the explicit dependency check thing didn't notice, that'd be fantastic. I thought it's supposed to catch these issues :/

6

BTW, being able to import dependencies you haven't declared but are built before you has me troubled. I can definitely see why that is happening. Everything is being deposited in the same directory so as long as the swiftmodule is there, you end up able to import it.

Problem is, have developers now assumed that should work and if we "fix" it would we end up breaking things. We'll need to tread carefully here.

7

No, no, no, pretty please :-)

This was previously fixed for an even worse use case: Move .swiftmodule output directory by neonichu · Pull Request #7103 · swiftlang/swift-package-manager · GitHub

But just providing a good error message (you need to declare a dependency on x to be able to import it) would be great.

Much better to be strict - it doesn’t seem like a reasonable thing to support really, if this works, why have dependency declarations at all…

8

Hmm, I'm not sure how that PR actually fixed it. It created a Modules directory but it still deposits all the swiftmodules there. I'll need to dig more.

9

Yup, exactly. That's always been the issue.

I feel that this could be solved with warnings and a transition period where the old behaviour keeps working.

As a strawman, we could create a folder for each module that's being built which is getting only the swiftmodules symlinked in that it explicitly depends on. But during the transition phase, we would also create another "global" folder which works like the one today -- getting symlinks to all the modules everywhere. From the compiler output we should then be able to see what if anything has been loaded from that special global modules folder and warn accordingly.

1 Like