Confused by unsafe flags being disallowed in dependencies

Development Package Manager
1

I'm continuing a discussion from [PackageGraph] Disallow unsafe build flags in dependencies by aciidb0mb3r · Pull Request #1884 · apple/swift-package-manager · GitHub with @SDGGiesbrecht in the forums to stop polluting the PR.

Background: I'm using some experimental or advanced compiler flags in a multi-package setting, and was surprised that that PR seemed to block support for that.

I was using: .package(url: ..., .branch("experimental_branch")) when I ran into this issue, so perhaps this is just a bug? If so, I'm happy to contribute a fix that allows .branch to work.

2 Likes
2

This was part of the target-specific build settings proposal, see: swift-evolution/0238-package-manager-build-settings.md at master · apple/swift-evolution · GitHub

The initial proposal did have an escape hatch which would bypass this limitation by using a command-line flag (something like --allow-unsafe-flags-in-dependencies). However, we decided to revisit this topic at a later point. I am not sure if using a command-line flag is the best idea. Maybe we should provide a API in the manifest file that allows you explicitly opt-in into using a package with unsafe flags. It'd be great if you wanted to explore this and come up with a proposal.

That said, I do think that this limitation doesn't make sense for local/branch-based package dependencies and I am planning to work on a patch to lift this limitation for those cases. Filed: [SR-11225] Allow 'unsafe' flags in local/branch-based dependencies · Issue #4681 · apple/swift-package-manager · GitHub

5 Likes
3

Sounds good. Once you have the mechanism to allow unsafe, if necessary, we just need a bool attached to the Package.Dependency.Requirement for .rangeItem and .exact telling if it allows unsafe or not.

Seems like .exact("v1.0.0").allowUnsafeFlags() would be best, but that doesn't quite work.
However, .exactWithUnsafeFlags("v1.0.0") could work?

4

The pull request lifting the (accidental?) restriction from local and branch dependencies is here if you want to track its progress, @pschuh.

3 Likes
5

@SDGGiesbrecht That pull request is already merged and I am trying with this toolchain swift-DEVELOPMENT-SNAPSHOT-2019-08-22-a and still getting the error about .unsafeFlags in local dependencies. I am referring to local dependency with .package(path: "../first-dep").

1 Like
6

CC: @Aciid

7

@Aciid Can you help me about this? Is this change is available in DEVELOPMENT or 5.1 snapshots now?

8

It should be in the swift-DEVELOPMENT-SNAPSHOT-2019-08-22-a toolchain. Are you trying this in Xcode or using swift build? Note that SwiftPM changes are not picked up via snapshots in Xcode.

9

Thanks for the hint, I am trying with Xcode, because I am doing some tests with an iOS project. Will this changes be in the next Xcode beta?

10

I tried with Xcode 11 beta 7 today and the problem exists yet.

11

@Aciid, I tried again with Xcode 11 GM and it is the same. It does not allow local packages to have dependencies with unsafe flags. Do you know if it will be in the final version of Xcode 11 or not?

1 Like
12

Was this ever clarified? I'm getting conflicting information, or maybe just not looking in the right place, but SR-11225 and a few other forum posts gave me the impression that if a branch is specified (and/or revision?) that unsafeFlags would be allowed.

With Xcode 11.2, specifying a branch or revision with unsafeFlags doesn't allow compilation, but I was unsure if this was a bug or my own ineptitude.

13

What's the news of this ?

I talk about my use case. I'm using SwiftPM with Xcode (not open-source swift packge manager imple mentation)

I found that Swift Package Manager does not allows me to make a framework with -enable-library-evolution (Called BUILD_LIBRARY_FOR_DISTRIBUTION in Xcode)

So, I decide to add the compiler flags by my own, using the .swiftSettings. However, it does not works because of this. Showing:

The package product 'SDWebImageSwiftUI' cannot be used as a dependency of this target because it uses unsafe build flags.

So I'm sucked. And stop to support SwiftPM to let our user to choose CocoaPods && Carthage instead.

Another issue, I want to weak linking SwiftUI and Combine system framework. In Xcode, I can simply use -weak_framework SwiftUI -weak_framework Combine, or the Linked library to Optional (In Build Phase).

However, when using SwiftPM, again, it does not have a linkerSettings for weak linking (only non-weak linking, good). When using unsafeFlags, the error appears the same.

So, as a framework author, I found it hard to implements the same easy thing compared to Xcode project itself. And can anybody here tell me: Why Swift Package Manager using Swift 5.1, does not support -enable-library-evolution ?

14

Weak linking seems like a good point, we should consider adding settings for that. What is the use case for enabling library evolution? Swift packages are currently always build from source, so there should be no need to enable library evolution for them.

15

What is the use case for enabling library evolution?

You may be using swiftpm as a way to build a dynamic library that will be distributed in binary form, or statically linking your package module into a larger binary framework. These aren't convenient to do today, but they are possible if you can pass the library evolution flags when building. I think having a target setting to turn on library evolution would be genuinely useful, even if it's a bit niche for a package currently.

4 Likes
16

Since most frameworks are auto-linked simply by importing them and you specify the minimum deployment version to the compiler, it would be nice if the auto-link information that gets emitted could decide whether the framework should be weakly linked or not based on whether the OS that the framework was first introduced in was later than the minimum deployment target. Then users wouldn't have to audit the use of -weak_framework flags. But, I don't believe the information about "when this framework first appeared" is recorded anywhere that the compiler could access it, so I don't think it's possible to do that today without manually providing that mapping?

17

The min deployment target version has Side effect, not only effect the weak linking, but also effect the Swift Standard library (For example, when you target iOS 11, you will have to copy libSwift.dylib into binary, your linker flags will use that instead of /usr/lib/libSwift.dylib). So it's not always a best solution to automatically resolve whice frameworks should be weak-linking, which are not.

And you're right, there are no "when this framework first appeared" information in compiler. It's in the Apple's release page about Public framework (note a framework can become runtime available in old firmware, until mark it public). So it's a solution in theroy, but not practice.

The simplest solution for this, it's that SwiftPM add a new function like

public static func linkedLibrary(_ library: String, _ condition: PackageDescription.BuildSettingCondition? = nil, type: LinkerSetting.LinkType = .weak ) -> PackageDescription.LinkerSetting

// or?
public static func weakLinkedLibrary(_ library: String, _ condition: PackageDescription.BuildSettingCondition? = nil) -> PackageDescription.LinkerSetting
18

What is the status on this? Including packages with unsafe flags from a branch or local does not work. Please fix!

19

You really shouldn't have to weak-link an Apple framework explicitly; availability information on the symbols you use will take care of that. Tony's "it would be nice" already works. However, Combine (and possibly also SwiftUI?) missed some availability markers in its first release, and the standard workaround for that would be -weak_framework. And if you had a third-party library installed that you found with pkg-config, you might want to weak-link that too.

20

This is good to know, thanks!

I'm curious now—where does the linker get the information about which frameworks need to be weakly linked? If I compile a code snippet that imports a framework to an object file, the LC_LINKER_OPTIONs are identical regardless of which deployment target version I set (they both contain -framework, not -weak_framework):

echo "import Combine; if #available(macOS 10.15, *) { print(Publishers.self) }" | \
  swiftc -target x86_64-apple-macos10.10 -emit-object -o foo.o - && \
  otool -l foo.o

...
Load command 4
     cmd LC_LINKER_OPTION
 cmdsize 32
   count 2
  string #1 -framework
  string #2 Combine
...

echo "import Combine; if #available(macOS 10.15, *) { print(Publishers.self) }" | \
  swiftc -target x86_64-apple-macos10.15 -emit-object -o foo.o - && \
  otool -l foo.o

...
Load command 4
     cmd LC_LINKER_OPTION
 cmdsize 32
   count 2
  string #1 -framework
  string #2 Combine
...

But if I emit an executable, the framework is strongly or weakly linked correctly based on deployment target:

echo "import Combine; if #available(macOS 10.15, *) { print(Publishers.self) }" | \
  swiftc -target x86_64-apple-macos10.10 -emit-executable -o foo - && \
  otool -l foo

...
Load command 15
          cmd LC_LOAD_WEAK_DYLIB
      cmdsize 88
         name /System/Library/Frameworks/Combine.framework/Versions/A/Combine (offset 24)
   time stamp 2 Wed Dec 31 16:00:02 1969
      current version 132.10.0
compatibility version 0.0.0
...

echo "import Combine; if #available(macOS 10.15, *) { print(Publishers.self) }" | \
  swiftc -target x86_64-apple-macos10.15 -emit-executable -o foo - && \
  otool -l foo

...
Load command 15
          cmd LC_LOAD_DYLIB
      cmdsize 88
         name /System/Library/Frameworks/Combine.framework/Versions/A/Combine (offset 24)
   time stamp 2 Wed Dec 31 16:00:02 1969
      current version 132.10.0
compatibility version 0.0.0
...

I didn't see anything immediately obvious in the framework's .tbd file either that would act as a hint to the linker. What am I missing?