Dependency only for Linux?

Is there a way to express the fact that a dependency will only be used on Linux?

In my case, I am using Network.framework in my Library for Apple builds and I would like not to have to download and build SwiftNIO. SwiftNIO is only used on Linux.

That would help keep my build and dependency minimal when using the lib to create an iOS application for example.

I’m not sure if there is a better or more official way to do this, but i typically use compiler flags in my Package.swift to generate the list of dependencies depending on the OS which is compiling the package.

I’m just typing this up on my phone so I can’t give an explicit example right now but I’ll get on in an hour or so and add an example if no one else has answered your question any better.

Please try to avoid #if in Package.swift files as that will break cross compilation and will also cause trouble with Package.resolved.

Regarding Network.framework and SwiftNIO, there's a pretty straightforward solution: Just depend on swift-nio-transport-services on all platforms.

In the respective Swift files you can then do

#if canImport(Network)
// Network.framework specific code
#else
// Linux and older macOS/iOS code for POSIX sockets
#endif

So the key is to push all the #ifs into the .swift files under Sources/ and Tests/ as opposed to Package.swift.

swift-nio-transport-services is set up to compile just fine on Linux, it won't really export any API but it should definitely compile on Linux and the modules should be available. We did this by basically guarding all files by #if canImport(Network).

As an additional benefit, that will allow you to use SwiftNIO on Sockets on Linux but also on older macOS/iOS systems that don't support Network.framework yet.

If you have any questions, please reach out.

3 Likes

Actually, we do have an example for what I propose: NIOSMTP does support iOS 10+ (maybe even iOS 9?) despite the fact that that iOS 10 doesn't have Network.framework.

Obviously NIOSMTP is an iOS app so it won't work on Linux but it does the same kind of dance where (at compile-time) it decides whether it should use Network.framework or not. Here's the PR that added the switching.

Thanks for your feedback :slight_smile:
The idea was originally to avoid depends on the whole SwiftNIO library on iOS to have a more lightweight dependency chain. I guess if SwiftPM does not allow adding or ignoring a dependency at build time, it defeat the purpose and then, yes, depending directly on SwiftNIO with transport is probably the best approach.

Actually there is another case where I would need dependencies only on Linux.

My library use libxml2. When building for Apple platforms, I need the build system to use standard lib from the SDK (so that it can link the code with the right lib for the target platform).

On Linux, I need to depend on a module that will add the modulemap for the system libxml2.

For now, the only way I see to do it is to uncomment the right line when building on Linux.

I would agree with that.

Also if you write your client with SwiftNIO (for Linux) anyway, why not use SwiftNIO (on Sockets) for Linux and older iOS versions and SwiftNIOTransportServices (on Network.framework) on newer iOS/macOS versions?

That way you can just re-use all your SwiftNIO code and need to implement the whole thing only once for both platforms. As you can see in NIOSMTP, the "bootstrapping" of connections is ever so slightly different between NIOTS (NIOTransportServices) and NIO especially around SSL setup but that's usually only around 20 to 30 lines that are platform specific. All other code, ie. the complicated bits for your protocol implementations can be 100% re-used between NIOTS and NIO because NIOTS is NIO, the only difference is that the Channels are backed by Network.framework rather than BSD sockets, that's all.

By the way, the grpc-swift (the 1.0.0 series) is an implementation of gRPC on top of SwiftNIO also uses the exact same setup. It uses NIOTS for newer iOS versions and NIO on sockets for everything else. It also always depends on swift-nio as well as swift-nio-transport-services, see the Package.swift.

I think that can also be solved. swift-nio-zlib-support is a package that SwiftNIO 1.x used to depend zlib for Linux and uses the SDK's zlib on macOS/iOS. The Package.swift specifies

let package = Package(
    name: "swift-nio-zlib-support",
    pkgConfig: "zlib")

so on Linux, it'll find everything it needs using pkgConfig: "zlib" and on iOS/macOS it will just find the relevant stuff in the SDK. The important thing here is that the module.modulemap is actually empty! You can see a similar setup in the other swift-nio-*-support repositories. All of those were only used for SwiftNIO 1.x so today they're less relevant for us but everything still works.

These days (NIO 2.x) we do it slightly differently and as you can see in swift-nio-extras we no longer depend on swift-nio-zlib-support but specify this in the Package.swift:

    .target(name: "CNIOExtrasZlib",
            dependencies: [],
            linkerSettings: [
                .linkedLibrary("z")
            ]),

I'm pretty sure that with a setup either like swift-nio-zlib-support or the other way in swift-nio-extras you could get libxml2 to work for both platforms too.

2 Likes

Thanks, I will give it a try a let you know :grinning:

1 Like

I tried this approach and works when I build it with XCode 11 beta (either targeting MacOS or iOS). however, strangely, it does not work when using swift build from command-line (on MacOS or Linux). It can't find the libxml2 includes when run from the command-line:

$ swift build 
/Users/mremond/devel/swift/XMPP/Sources/CXML/include/CXML.h:4:10: note: while building module 'libxml2' imported from /Users/mremond/devel/swift/XMPP/Sources/CXML/include/CXML.h:4:
#include <libxml2/libxml/tree.h>
         ^
<module-includes>:1:9: note: in file included from <module-includes>:1:
#import "libxml/HTMLparser.h"
        ^
/Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/libxml2/libxml/HTMLparser.h:15:10: error: 'libxml/xmlversion.h' file not found
#include <libxml/xmlversion.h>
         ^
<module-includes>:1:9: note: in file included from <module-includes>:1:
#import "/Users/mremond/devel/swift/XMPP/Sources/CXML/include/CXML.h"
        ^
...

I commited my attempt here: https://github.com/FluuxIO/XMPP/blob/master/Package.swift

Any idea?

Hi,

More test reporting: The build work from command-line if I force path to SDK libxml includes.

For example, on MacOS:

swift build -Xcc -I/Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/libxml2

or on Linux:

swift build -Xcc -I/usr/include/libxml2

I have no idea why the swift build command does not have the same complete path to system libraries includes, like XCode has.

Update: I tried building that package from an XCode 11 iOS project. and it does not work and fails because it cannot find the libxml2 headers.

I think the build is somewhat a bug as the Swift package build fine on XCode beta 11, but not an application depending on that same module.
I would also expect swift build to have the same behaviour as XCode build and build successfully the project as well.

I think I need to create a bug report.

Update: Submitted Feedback FB7213576

Thanks for filing a bug report on this! If you believe there is also a bug with swift build here, please also file a JIRA on SwiftPM (https://github.com/apple/swift-package-manager/blob/master/Documentation/Resources.md#reporting-a-swiftpm-bug). Thanks!

1 Like

Sure, I will fill the report on SwiftPM tracker tomorrow.

Thanks !

Here is the SwiftPM bug report: https://bugs.swift.org/browse/SR-11427

I updated my bug reports with the latest Xcode 11 GM.
At least now, we have a consistent behaviour: Both Xcode or SwiftPM build cannot properly find the libxml2 headers.
Here is the Package.swift file: https://github.com/FluuxIO/XMPP/blob/master/Package.swift

So, it is unclear now how to properly set this up. Was the fact that Xcode was building fine the project a bug and what would be the preferred way to build a code depending on libxml2 ?

I just tried getting libxml2 from the macOS SDK to work and failed. It's possible that's due to this comment in MacOSX10.15.sdk/usr/include/libxml2/module.modulemap:

module libxml2 [system] [extern_c] {
    // Add "-Xcc -I$(SDKROOT)/usr/include/libxml2" to OTHER_SWIFT_FLAGS in Xcode project.

@Aciid / @NeoNacho could you help us getting libxml2 from the SDK to work? If we get that to work, then I'm sure we can get Linux to work too.

I tried .systemLibrary as well as .target with linkerSettings/cSettings etc.

Is it even possible to add something like $(SDKROOT) to Package.swift?

1 Like

There is no solution for this at the moment, we're tracking this issue for packages in Xcode already as rdar://problem/52406575

Thanks for the feedback.
With the release "train" that is schedule to arrive soon, I bet you have other things on your plate, but if you need a hand to test this type of build at a later point in time, I would be glad to help.

I tried the exact same steps with the same results over time: Referring to libxml2 in Swift Package Description
I'm very interested in the resolution the Swift team might bring in time. Why not broaden the scope of .systemLibrary to something like .systemLibrary(name: "libxml2", constraints: ...) to target SDKs for platforms in Xcode while still maintaining a possibility to specify a different "source" for Linux.

Terms of Service

Privacy Policy

Cookie Policy