Confused by unsafe flags being disallowed in dependencies

Weak linking seems like a good point, we should consider adding settings for that. What is the use case for enabling library evolution? Swift packages are currently always build from source, so there should be no need to enable library evolution for them.

What is the use case for enabling library evolution?

You may be using swiftpm as a way to build a dynamic library that will be distributed in binary form, or statically linking your package module into a larger binary framework. These aren't convenient to do today, but they are possible if you can pass the library evolution flags when building. I think having a target setting to turn on library evolution would be genuinely useful, even if it's a bit niche for a package currently.

3 Likes

Since most frameworks are auto-linked simply by importing them and you specify the minimum deployment version to the compiler, it would be nice if the auto-link information that gets emitted could decide whether the framework should be weakly linked or not based on whether the OS that the framework was first introduced in was later than the minimum deployment target. Then users wouldn't have to audit the use of -weak_framework flags. But, I don't believe the information about "when this framework first appeared" is recorded anywhere that the compiler could access it, so I don't think it's possible to do that today without manually providing that mapping?

The min deployment target version has Side effect, not only effect the weak linking, but also effect the Swift Standard library (For example, when you target iOS 11, you will have to copy libSwift.dylib into binary, your linker flags will use that instead of /usr/lib/libSwift.dylib). So it's not always a best solution to automatically resolve whice frameworks should be weak-linking, which are not.

And you're right, there are no "when this framework first appeared" information in compiler. It's in the Apple's release page about Public framework (note a framework can become runtime available in old firmware, until mark it public). So it's a solution in theroy, but not practice.

The simplest solution for this, it's that SwiftPM add a new function like

public static func linkedLibrary(_ library: String, _ condition: PackageDescription.BuildSettingCondition? = nil, type: LinkerSetting.LinkType = .weak ) -> PackageDescription.LinkerSetting

// or?
public static func weakLinkedLibrary(_ library: String, _ condition: PackageDescription.BuildSettingCondition? = nil) -> PackageDescription.LinkerSetting

What is the status on this? Including packages with unsafe flags from a branch or local does not work. Please fix!

You really shouldn't have to weak-link an Apple framework explicitly; availability information on the symbols you use will take care of that. Tony's "it would be nice" already works. However, Combine (and possibly also SwiftUI?) missed some availability markers in its first release, and the standard workaround for that would be -weak_framework. And if you had a third-party library installed that you found with pkg-config, you might want to weak-link that too.

This is good to know, thanks!

I'm curious now—where does the linker get the information about which frameworks need to be weakly linked? If I compile a code snippet that imports a framework to an object file, the LC_LINKER_OPTIONs are identical regardless of which deployment target version I set (they both contain -framework, not -weak_framework):

echo "import Combine; if #available(macOS 10.15, *) { print(Publishers.self) }" | \
  swiftc -target x86_64-apple-macos10.10 -emit-object -o foo.o - && \
  otool -l foo.o

...
Load command 4
     cmd LC_LINKER_OPTION
 cmdsize 32
   count 2
  string #1 -framework
  string #2 Combine
...
echo "import Combine; if #available(macOS 10.15, *) { print(Publishers.self) }" | \
  swiftc -target x86_64-apple-macos10.15 -emit-object -o foo.o - && \
  otool -l foo.o

...
Load command 4
     cmd LC_LINKER_OPTION
 cmdsize 32
   count 2
  string #1 -framework
  string #2 Combine
...

But if I emit an executable, the framework is strongly or weakly linked correctly based on deployment target:

echo "import Combine; if #available(macOS 10.15, *) { print(Publishers.self) }" | \
  swiftc -target x86_64-apple-macos10.10 -emit-executable -o foo - && \
  otool -l foo

...
Load command 15
          cmd LC_LOAD_WEAK_DYLIB
      cmdsize 88
         name /System/Library/Frameworks/Combine.framework/Versions/A/Combine (offset 24)
   time stamp 2 Wed Dec 31 16:00:02 1969
      current version 132.10.0
compatibility version 0.0.0
...
echo "import Combine; if #available(macOS 10.15, *) { print(Publishers.self) }" | \
  swiftc -target x86_64-apple-macos10.15 -emit-executable -o foo - && \
  otool -l foo

...
Load command 15
          cmd LC_LOAD_DYLIB
      cmdsize 88
         name /System/Library/Frameworks/Combine.framework/Versions/A/Combine (offset 24)
   time stamp 2 Wed Dec 31 16:00:02 1969
      current version 132.10.0
compatibility version 0.0.0
...

I didn't see anything immediately obvious in the framework's .tbd file either that would act as a hint to the linker. What am I missing?

The compiler decides whether to emit strong or weak references based on availability, and then if autolinking sees that all references found in a framework are weak, it uses weak linking. I don't think it applies this heuristic to an explicit -framework on the command line, though, on the grounds that you need to be able to override it somehow.

3 Likes

Question? Related to the original topic. I have 2 packages. One for Linux that uses unsafe flags to compile c++ code with Vulkan and OpenCV. Then a MacOS package using Metal and OpenCV with unsafe flags to provide the same API feature set.

Then I have a general package that runs on Linux/MacOS and fetches the required OS package. If I build the general package on Linux everything works. If I build on MacOS is complains about unsafe flags. Why is this. How do I get around it??

Also to clarify – both Linux/MacOS are using swift 5.1

2 Likes

Any ideas on a quick fix? I need a solution for MacOS without having to build a separate code base that needs to be maintained. It's kinda crappy that Swift works better on Linux than it does on MacOS. But maybe that's just because Apple likes to make it harder on developers and lock everything down.

2 Likes

Really need a workaround here!!!

3 Likes

Same issue. I don’t think this behavior is correct for a Package Manager should do, which limit the extension ability for framework author and framework consumer.

1 Like

I'm curious if some recent change now prevents unsafe flags from being used in local SPM dependencies. I see the pull request to allow unsafe flags in local SPM dependencies. I've created a sample project here: GitHub - yuzuquats/TestSPMLocalUnsafe so I'm wondering if there's something obvious I'm missing. Specifically, I'm doing the following:

  1. Create a local package with an unsafe flag
  2. Create a new xcode project
  3. Drag the local package folder into the left "file explorer" sidebar
  4. In the xcode project's "Framework, Libraries, and Embeded Content" -> Add the local package framework from step 1

This was working for me during the xcode 12 beta (12A8158a) but no longer with the newer releases (currently on 12A70209). Any help would be much appreciated!

1 Like

The detection of unsafe flags was incorrect in prior builds of Xcode and was missing them in cases where they're used by a target that's a transitive dependency of a product.

AFAIK, exceptions aren't supported at all by Xcode, but you might be able to work around it by putting a package in between and depending on the other one using .package(path:).

3 Likes

Thank you! Fixed this with the proposed workaround (.package(path:)). In case anyone is interested, you may have to reexport with @_exported import

1 Like

Was this ever addressed?

The issue is still happening for me.

I've filed a bug about this: [SR-15398] Xcode blocks local-branch SPM packages with unsafe flags in an app targets · Issue #4379 · apple/swift-package-manager · GitHub

Couldn't reply in jira so I'll copy my comment here

I'd also like to add that it could be nice to have an -allow-unsafe-flags flag for xcodeproj target and allow unsafe flags for local packages by default, for example, if I need -Xfrontend -warn-long-expression-type-checking for optimization debugging purposes I'd like to make it work out of the box, without need to wrap my local package into another local package... I'm not even sure it's actually unsafe :new_moon_with_face:

Project structure without a wrapper package is semantically correct and simple

- App.xcworkspace
- App.xcodeproj
- AppPackage

And the wrapper package seems pretty much redundant (actually it does nothing but exposing AppPackage.AppTarget to the App.xcodeproj via _AppPackage._AppTarget (we cant even keep naming for the target))

- App.xcworkspace
- App.xcodeproj
- _AppPackage
- AppPackage

My use case is for linking with private frameworks elegantly by exporting headers via a C target and linking against the relevant framework by passing an unsafe linker flag. By disallowing versioned packages from using unsafe flags, wrappers like this are barred from the semver world and the dependency resolution that comes with it. I understand that a version bump could introduce malicious flags, but I control all ends of the pipeline and would appreciate an option to override this safeguard.

In the meantime I'll be adding the package as a branch-item dependency, like a madman, because there is no alternative that offers the same ease of use for depending packages. GitHub - EricRabil/Paris: SPM-native private framework bindings

Update 22h later: I did not test out adding Paris as a revision-item, which allows me to specify a commit hash. I am now able to depend on specific snapshots of Paris across projects, without worrying about breaking changes.

I still think that being able to use semantic versioning with a package that uses unsafe flags should not be arbitrarily gatekeeped by the package manager. Until SPM is able to cover more advanced targets, there should be a mechanism to at least whitelist specific unsafe flags. Disallowing this outright makes it difficult to push the boundaries of what you can do with Swift.