What's the right way to support both package and SDK versions of swift-system?

Related Projects System
1

I'd like for my cross-platform project to use functionality from swift-system, but I'm not sure of the best way to do it.

Right now, my idea is to have a target with a conditional dependency on the swift-system package, then for code within that target to use #canImport to decide which library they actually want to use:

let package = Package(
  name: "mylib",
  products: [
    // Includes everything.
    .library(name: "MyLib", targets: ["MyLib"]),

    // Base functionality which doesn't require swift-system.
    .library(name: "MyLibCore", targets: ["MyLibCore"]),
  ],
  dependencies: [
    .package(url: "https://github.com/apple/swift-system", .upToNextMinor(from: "0.0.2")),
  ],
  targets: [
    // Everything - Core + System extras.
    .target(
      name: "MyLib",
      dependencies: ["MyLibCore", "MyLibSystemExtras"]
    ),
    // Base functionality.
    .target(
      name: "MyLibCore"
    ),
    // Swift-system additions.
    .target(
      name: "MyLibSystemExtras",
      dependencies: [
        "MyLibCore",
        .product(
          name: "SystemPackage",
          package: "swift-system",
          condition: .when(platforms: [.linux, .windows, .wasi, .android, ...every other non-Apple platform... ])
        )
      ]
    ),
  ]
)

Then, within MyLibSystemExtras, I'd write:

#if canImport(System)
  import System
#else
  import SystemPackage
#endif

extension FilePath { ... }

Are there any drawbacks to this approach? I think it should do the right thing, but I can't find any "official" guidance in the swift-system repository.

2

I personally would recommend SystemPackage if you're targeting apple and non-apple platforms.

API in System.framework is annotated with availability which means you either:

  • can only use System API <= to your oldest supported macOS/iOS version on all of your platforms
  • or you pull in the SystemPackage on macOS/iOS and can use all System API across all platforms at the cost of a small binary increase.

cc @Michael_Ilseman for his thoughts.

1 Like
3

Yes, that is a concern.

The cost of not using the SDK version of System is not just binary size, though - it is another package to download and build, and will present as an entirely different library. It won't interoperate with any existing code the user may have which does use the SDK version.

Ideally SwiftPM would also allow conditional dependencies based on deployment target: before macOS 11, there was no System framework in the SDK so there's no interoperability concern (anybody using swift-system code and targeting that version of the platform must be using the package), or perhaps some APIs I really, really need are only available from macOS 12, so I'd rather provide those on older OSes via the package than not provide it at all.

4

This is a big issue with Swift and SwiftPM's current approach (or lack of one) for libraries that can be distributed with multiple versions, and especially for libraries distributed in either binary or source form.

  1. Module versioning and/or module aliasing

There's some on-going work in this area. This would allow you to depend on either SDK System or package system, and just call it "System" in your own source code as well as your dependents. This would leave you in an either/or world, though, but is still a big improvement.

  1. Type conversions between module versions

This is just speculation at this point, but I think it's important for libraries to be able to define conversion functions for their types between module versions. This allows for interoperability, which could be essential for dual-distribution style libraries like System (and perhaps, eventually, for other "Swift Standard Packages" it they can be distributed with a toolchain, SDK, etc).

A type from a source dependency is a different type at run time than the "same" type from a binary dependency (binary dependencies have API and ABI, source just has API). So you could imagine something similar to a cross-import overlay that defines conversion functions.

For cases where the layout is known to be compatible (e.g. via ABI annotation today, but hopefully via other means in the future), it's essentially a bit-cast. You could imagine the compiler even synthesizing this for at least a subset of types. User code doing the conversion might start off being explicit, but you could similarly imagine the compiler inserting conversion calls for you.

  1. Ability to back-deploy new functionality based on top of old binaries as source

Using points #1 and #2, you could imagine many new features as source that is built on top of existing functionality and interfaces. For example, everything in System is source code built on top of long-established OS interfaces, and this is why a package build has no real SDK version requirements. Something like SwiftNIO would be the same, though they might have newer/better APIs when newer/better interfaces are available (e.g. io_uring), and System might also find itself in this situation. Either way, the vast majority of functionality can be auto-back-deployable given tooling support.

You could back-deploy the new FilePath syntactic APIs on to the old version of System by just distributing the relevant source code. This might not be a case where the layout is equivalent, so there might be an explicit conversion/copy, but that's still dramatically preferable to it not being possible. If System were to acquire file system APIs in the future, then those too should be back-deployable to the extent they use pre-existing OS interfaces.

4 Likes
5

That's very interesting, thanks!

Another package which would seem to benefit from these efforts is swift-crypto. It's a bit different because the SDK and source versions of the library have very different implementations (based on CryptoKit/BoringSSL respectively), but broadly-speaking the issues are similar:

  • For developers, you'd want to depend on one library and get the best implementation for the platform you deploy to (module aliasing)
  • Potentially, some features might be back-deployable based on the source implementation.

But yeah, I'm looking forward to improved tooling for this! Hopefully lots of folks can benefit from it.

After experimenting a little bit, I think the best option is to use the source package, adding some shims to ferry people to/from SDK-land (we only have withCString rather than withPlatformString, but luckily on Darwin platforms there shouldn't be any functional difference). Hopefully users won't actually need to do this very often, since the source package can do basically everything anyway.

import SystemPackage
import System

extension SystemPackage.FilePath {

  @available(macOS 11, iOS 14, tvOS 14, watchOS 7, *)
  public var filePath: System.FilePath {
    withCString { .init(cString: $0) }
  }

  @available(macOS 11, iOS 14, tvOS 14, watchOS 7, *)
  public init(_ filePath: System.FilePath) {
    self = filePath.withCString { .init(cString: $0) }
  }
}