Command line applications crashes with Xcode 10.2

Yes.

https://forums.swift.org/t/xcode-10-2-beta-command-line-tools/19998

Swift 5 Release Notes for Xcode 10.2

  • Starting with Xcode 10.2, Swift command line tools require the Swift libraries in macOS. They’re included by default starting with macOS Mojave 10.14.4. In macOS Mojave 10.14.3 and earlier, there’s an optional package to provide these runtime support libraries for Swift command line tools that you can download from More Downloads for Apple Developers. If you installed the beta version of this package, replace it with the release version. This package is only needed for Swift command line tools, not for apps with graphical user interfaces.

  • Swift command line projects won’t run on macOS 10.14.3 and earlier unless you install the Swift 5 Runtime Support for Command Line Tools package. Without that package, Swift command line projects crash on launch with “dyld: Library not loaded” errors. (46824656)

5 Likes

Thanks, worked after updating to macOS 10.14.4.

1 Like

This has been causing a lot of grief for folks that make command line tools with Swift. Particularly in that this happens no matter what the build settings are.

  • If you target an older macOS: No embedded libs
  • If you use an older Swift: No embedded libs
  • If you explicitly set the build to "Always Embed" the libraries: No embedded libs

I know that embedding libs made the executables big like a clown shoe, but simply ignoring the settings to embed them isn't any better. In particular I think this will be an issue for folks that embed a CLI tool inside an app. Now they will need to have customers install their app AND a runtime to use it.

1 Like

There's no way to "embed" a shared library (dylib) inside a command line tool, since it's just a single file. If you want to handle the embedding-in-an-app case, you can manually set your tool's "Runpath Search Paths" to be able to find the dylibs that'll get embedded in the app. Make sure that the ones in /usr/lib/swift are still preferred, though, so that it'll run correctly on newer versions of macOS.

1 Like

I think the point was that previous versions of Xcode compiled command-line Swift executables with all of the Swift libraries included in the binary and there's no way to replicate that behavior anymore. Meaning that such tools can't be deployed on anything older than 10.14.4. Your solution has nothing to do with that, except for the case where the tool is embedded in an app which includes the dylibs. Won't those be stripped if shipped through the App Store on systems with the system libs?

2 Likes

Previous versions of Swift allowed static linking of the standard library and runtime on macOS, but that's not going to work anymore because parts of the OS also link against the standard library and runtime, and the two wouldn't be able to interoperate. (We had to jump through hoops not to break Swift 4-and-below apps, but we know that those aren't trying to talk to any of the Swift parts of the system, because there were no Swift parts of the system when those were deployed.)

We're in an inconvenient spot now, but in the long run this won't be a problem.

Yes, which is why you also need to have a Runpath Search Path for /usr/lib/swift/.

2 Likes

Hopefully the Swift 5 Runtime Support package can be offered through brew so tools published that way can use it as a dependency.

My understanding is that brew packages are built from source on the machine you intend to use them on, so they shouldn't ever need this shouldn't need this once everyone running Mojave has updated to 10.4.4.

brew packages are mostly built from source, but the brew cask option allows the posting of binaries. There are even a lot of Cocoa apps that are distributed that way.

2 Likes

This is exactly the issue. During the beta you were able to continue including the libraries in the binary if you wanted to via a user defined compiler flag. That flag no longer seems to work with the GM version.

I've not had a chance to test it yet, but will this also remove the ability to use the -static-stdlib flag with SPM when building executables?

It just is incongruous that bundle based apps continue to function and build as they always have but CLI apps do not. I guess my main beef with it is that it's not just a behavior change, but the option to build as you could in Xcode 10.1 has been removed.

I've not had a chance to test it yet, but will this also remove the ability to use the -static-stdlib flag with SPM when building executables?

Yes this has been removed:

% swift build --static-swift-stdlib
warning: Swift compiler no longer supports statically linking the Swift libraries. They're included in the OS by default starting with macOS Mojave 10.14.4 beta 3. For macOS Mojave 10.14.3 and earlier, there's an optional Swift library package that can be downloaded from "More Downloads" for Apple Developers at https://developer.apple.com/download/more/

As noted, it is not possible (at a technical level) to support statically linking the standard library and runtime and to have a standard library and runtime in the OS. I know it's a regression but there weren't any other options. (We Apple folks looked for them.)

2 Likes

This claim seems rather specious.

If I, as a developer, need to write a program that will run on an OS which does not include the Swift standard library and runtime, I am perfectly capable of doing so using an older version of Xcode running on an older version of macOS. And when I do so, the Swift standard library and runtime are embedded in my app.

I am having a lot of difficulty imagining what could possibly present such a large impediment to making available a separate download of the Swift standard library and runtime, whose only purpose is to be embedded in programs which will run on an OS which does not include those things.

:-/ I suppose "there weren't any other options" is an exaggeration. "The other options were considered to have worse tradeoffs" would be more accurate. A command-line tool with a statically-linked Swift 5 stdlib would not even launch on macOS 10.4.4 (the one with Swift in the OS), and so rather than make it possible to build executables that would stop working when you updated your OS, we instead went the route of the "runtime support libraries for command line tools" package described above.

The difference between doing it with Swift 5 and doing it with Swift 4.2 is that in Swift 5 we had to rename all the symbols in the standard library so that they wouldn't conflict with Swift 4.2 (or any earlier version of Swift). Building two versions of the runtime just to work around this would have meant more variants of the standard library running around and more chance for bugs to creep in. It's also unclear how this would extend to later versions of the runtime; we did not want to get into a space where someone could use Swift 7 features on macOS 10.20 but not macOS 10.19, but if you go back all the way to macOS 10.13 then they would work again.

3 Likes

Unless you force them to be built from source, most are distributed as bottles, which are precompiled.

jshier$ brew install cowsay
==> Downloading https://homebrew.bintray.com/bottles/cowsay-3.04.mojave.bottle.t
######################################################################## 100.0%
==> Pouring cowsay-3.04.mojave.bottle.tar.gz
🍺  /usr/local/Cellar/cowsay/3.04: 65 files, 82.9KB

Swift tools would need to have the Swift 5 package installed for this to work, so hopefully brew will allow that.

1 Like

If I, as a developer, keep a machine running an older version of Xcode on an older version of macOS, will I be able to install and use newer versions of Swift, and thereby create programs which embed the newer standard library and runtime?

If so, doesn’t that create an incentive for developers to stay on older versions?

And if not, why are we—the Swift open source project—allowing an OS vendor to control what versions of the Swift programming language can be used?

I bundle swift command line tools with my apps, and I was able to work around this by pointing them to the Frameworks folder of my app bundle. If the command line tools are in your resources folder, you can point "Runpath Search Paths" to @executable_path/../Frameworks/.

I also needed to add an additional swift framework to my main bundle's frameworks. Dragging in /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/macosx/libswiftSwiftOnoneSupport.dylib to the frameworks list did the trick. Running a test swift 5 command line tool in macOS 10.13 gave me a Hello, World!

If you want to build them yourself, I guess so? But they won't work on newer OSs.

It matters what OS the users are on, not the developer, no?

This is a fair question, but I think it's addressed in Joe's blog post: "Evolving Swift On Apple Platforms After ABI Stability". It's definitely a tradeoff, but it's one that this project was going to make from the start. If we—Apple or the open source community—weren't going to do this, then we probably wouldn't have bothered with ABI stability at all, like Go.

(That's not a slight against Go! It's an indication of different project goals. One of Swift's project goals has always been to be a language that Apple can use in its own OSs, which means supporting Apple's notion of binary compatibility across software updates.)

EDIT: I may need to stop answering soon to avoid crossing a line in my dual roles as Apple engineer and Swift open source contributor, sorry. I do think this will be a drastically smaller issue in a year, and that therefore it's worth the inconvenience now.

Right, and if the users are on an OS without embedded Swift, then I as a developer must build for what the users are running.

Thus, it should be possible to embed the Swift standard library and runtime in a program, so that I can distribute it to users on OSes without those things.

I am perfectly fine compiling two entirely separate versions of my program, one for OSes with embedded Swift and one for those without. That’s not particularly onerous. The problem arises when I am prevented from compiling one of those versions.

1 Like

If the command line tools are in your resources folder, you can point "Runpath Search Paths" to @executable_path/../Frameworks/.

I tried this but it didn't work for me. My command line tool resides in the MacOS directory rather than Resources - could this be the reason it's failing for me?

1 Like