Bundle stdlib with swift binary?

I have a swift binary I am packaging in an electron app. The binary simply connects to AVFoundation and serves as a wrapper to enable recording screen and microphone (https://github.com/getdebrief/aperture-node forked from https://github.com/wulkano/aperture-node). I then bundle this binary into an electron app and distribute it. As such, I attempt to compile and link the swift stdlib to the binary so that I do not require the consumer to have the stdlib installed. This appears to not work with a warning:

warning: Swift compiler no longer supports statically linking the Swift libraries. They're included in the OS by default starting with macOS Mojave 10.14.4 beta 3. For macOS Mojave 10.14.3 and earlier, there's an optional Swift library package that can be downloaded from "More Downloads" for Apple Developers at https://developer.apple.com/download/more/

When distributing the app, I am blocked from running the code. Specifically, when the app starts I receive:

XprotectService: [com.apple.xprotect:xprotect] File /Applications/Debrief.app/Contents/Resources/app.asar.unpacked/node_modules/@getdebrief/aperture/aperture failed on rPathCmd /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/macosx/libswiftAVFoundation.dylib (rpath resolved to: (path not found), bundleURL: /Applications/Debrief.app)

This made me infer that the binary is actually dynamically linked, and using otool, I found it sure is:

otool -L aperture
aperture:
	/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.60.1)
	/System/Library/Frameworks/AVFoundation.framework/Versions/A/AVFoundation (compatibility version 1.0.0, current version 2.0.0)
	/System/Library/Frameworks/AppKit.framework/Versions/C/AppKit (compatibility version 45.0.0, current version 2022.20.117)
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1770.255.0)
	/System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics (compatibility version 64.0.0, current version 1463.2.1)
	/System/Library/Frameworks/CoreMedia.framework/Versions/A/CoreMedia (compatibility version 1.0.0, current version 1.0.0)
	/System/Library/Frameworks/CoreMediaIO.framework/Versions/A/CoreMediaIO (compatibility version 1.0.0, current version 1.0.0)
	/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 1770.255.0)
	/System/Library/Frameworks/IOKit.framework/Versions/A/IOKit (compatibility version 1.0.0, current version 275.0.0)
	/usr/lib/swift/libswiftCoreMIDI.dylib (compatibility version 1.0.0, current version 2.0.0, weak)
	/usr/lib/swift/libswiftUniformTypeIdentifiers.dylib (compatibility version 1.0.0, current version 633.0.2, weak)
	@rpath/libswiftAVFoundation.dylib (compatibility version 1.0.0, current version 2000.5.4, weak)
	@rpath/libswiftAppKit.dylib (compatibility version 1.0.0, current version 103.10.0, weak)
	@rpath/libswiftCloudKit.dylib (compatibility version 1.0.0, current version 962.0.0, weak)
	@rpath/libswiftCore.dylib (compatibility version 1.0.0, current version 1200.2.41)
	@rpath/libswiftCoreAudio.dylib (compatibility version 1.0.0, current version 1.1.0, weak)
	@rpath/libswiftCoreData.dylib (compatibility version 1.0.0, current version 3.0.0, weak)
	@rpath/libswiftCoreFoundation.dylib (compatibility version 1.0.0, current version 1.6.0, weak)
	@rpath/libswiftCoreGraphics.dylib (compatibility version 1.0.0, current version 2.0.0)
	@rpath/libswiftCoreImage.dylib (compatibility version 1.0.0, current version 1.0.0, weak)
	@rpath/libswiftCoreLocation.dylib (compatibility version 1.0.0, current version 5.0.0, weak)
	@rpath/libswiftCoreMedia.dylib (compatibility version 1.0.0, current version 1.0.0)
	@rpath/libswiftDarwin.dylib (compatibility version 1.0.0, current version 0.0.0, weak)
	@rpath/libswiftDispatch.dylib (compatibility version 1.0.0, current version 4.40.2, weak)
	@rpath/libswiftFoundation.dylib (compatibility version 1.0.0, current version 20.0.0)
	@rpath/libswiftIOKit.dylib (compatibility version 1.0.0, current version 1.0.0, weak)
	@rpath/libswiftMetal.dylib (compatibility version 1.0.0, current version 1.3.1, weak)
	@rpath/libswiftObjectiveC.dylib (compatibility version 1.0.0, current version 1.0.0, weak)
	@rpath/libswiftQuartzCore.dylib (compatibility version 1.0.0, current version 1.0.0, weak)
	@rpath/libswiftXPC.dylib (compatibility version 1.0.0, current version 1.1.0, weak)
	@rpath/libswiftsimd.dylib (compatibility version 1.0.0, current version 1.3.0, weak)

My question is: Does a way exist to bundle the stdlib with the app for distribution? If not, can anyone suggest another path to create a native binary for interacting with AVFoundation, please? Failing that, any other threads you could suggest for me to pull on?

1 Like

On Apple platforms, statically linking the standard library is not supported and may lead to runtime failures in the future even if you can get it working on a currently-shipping OS.

If you're willing to accept the binary-size impact of statically linking the standard library, why is the supported approach of dynamically linking it unacceptable? What's the overhead you're measuring?

2 Likes

If i understand correctly, the last few MacOS and iOS operating system upgrades install the Swift runtime support as part of the system, starting with Swift 5.1, which I think is Mojave on MacOS. Unless you're trying to support an older operating system version, the stdlib should be loaded as part of the system. That was the point, I think, of ABI stability. Your customers should have it already installed as a dynamic library.

1 Like

Ah very interesting!

The binary size is something I'm not concerned with right now (although I'll need to address it down the road.)

So if I'm understanding you correctly you're recommending to package the swift stdlib in my Frameworks deployed with the app? This sounds like it might work. I'm going to give it a shot!

That makes complete sense. For whatever reason though compiling from the commandline leaves the @rpath directive pointing do the stdlib resources rather than going to the system available ones. This feels strange to me and there may be a way around it but I haven't found it yet. Any thoughts on this one?

Just wanted to say I found a fix here thanks to some of the thoughts in this thread (sorry I can't mention you because I'm a new user and it blocks me from doing so for spam reasons, presumably)

The issue was in the rpath not being updated to refer to a relative path (in my case based on @executable_path). I did some reverse engineering of other electron apps to see how they handled this and they indeed package swift dylib with their app. Their LC_RPATH is then updated to refer to a relative path from the executable.

In order to accomplish the latter part, I used install_name_tool with the -rparth command to replace the reference to my system's dylib path to the relative path:

install_name_tool -rpath /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/macosx @executable_path/../../../swift-libs <binary_name>

Once this was complete and the package setup properly, everything operates as expected.

1 Like

There's a standard way this is done when building Swift executables so that it uses the system library when available and otherwise falls back on the library from your bundle. Essentially, the libraries are linked with @rpath-relative paths, and the executable's rpath is configured to search first in /usr/lib/swift and then relative to the executable. Please try building a trial Swift executable with an older deployment target (one prior to Swift being included in the OS) and verify that your executable seems to match. If you accidentally set up your executable to always use the standard library from your bundle, it may stop working on future operating systems.

There is an option in Xcode build settings: "Always embed swift standard libraries"
But it should still run fine on macOS 10.14.4+ without that.

Maybe I'm missing something on my system (or I installed something incorrectly) but my /usr/lib/swift contains the following:

% ls -al /usr/lib/swift
total 7440
drwxr-xr-x   7 root  wheel      224 Jan  1  2020 .
drwxr-xr-x  37 root  wheel     1184 Jan  1  2020 ..
drwxr-xr-x   7 root  wheel      224 Jan  1  2020 CreateMLInternal
-rwxr-xr-x   1 root  wheel  8176144 Jan  1  2020 libswiftCreateML.dylib
-rwxr-xr-x   1 root  wheel   648384 Jan  1  2020 libswiftDemangle.dylib
-rwxr-xr-x   1 root  wheel  1219312 Jan  1  2020 libswiftRemoteMirror.dylib
-rwxr-xr-x   1 root  wheel   342848 Jan  1  2020 libswiftXCTest.dylib

Perhaps am I missing something here? My fear is if my system can be in this state, any of my users could end up in this state too!

The Swift standard library itself is actually in the shared cache. I don't know exactly what's going with libswiftAVFoundation, but it probably just has a different deployment rule.

Again, you need to set this up so that it picks up libraries from the system when present, which is mostly a matter of matching Swift's normal linking behavior, assuming you can't actually just use Swift to link.

Terms of Service

Privacy Policy

Cookie Policy