Introduce `#if stdlib` or clarify `#if swift`

Evolution Pitches
1

SE-0020 introduced the #if swift compile-time conditional for Swift language versions.

However, Apple have repeatedly shipped newer compilers with old SDKs for macOS (including an older version of the standard library). That means that a condition such as #if swift(>=5.7) can pass, even though the standard library is actually from 5.6 and may not have adopted language features introduced in 5.7.

Most recently, this occurred with primary associated types. The following code fails to compile with Xcode 14.0.1 on macOS, despite the #if swift guard:

#if swift(>=5.7)
  // Fails when building for macOS: 
  // error: cannot specialize protocol type 'Collection'
  func foo() -> some Collection<Int> { ... }
#endif

This is technically allowed by SE-0020 (as it refers only to "language version"), but Swift developers will likely consider this behaviour undesirable. I would suggest that there are two ways to rectify it:

  1. Introduce an #if stdlib compile-time conditional so we can distinguish between the availability of language features and standard library features (if we accept that they may not match in all toolchains), or

  2. Amend SE-0020 and require that the Swift language version match the standard library version. This would require a change in Apple's deployment practice and preclude macOS users from taking advantage of new language features until a matching SDK is ready, but that may also be an improvement. See:

While it is true that other parts of the Apple's SDKs have similar issues, and some sort of #if SDK may also be warranted for compatibility with multiple versions of the SDK, I think the idea that #if swift does not match the standard library version is a particularly severe situation and needs an urgent remedy.

10 Likes
2

To elaborate on this a little bit. Currently, we have:

  • #if swift(>=5.7) to guard code by the language mode, and
  • #if compiler(>=5.7) to guard code by the compiler version,

But there is a third critical versioned component in the toolchain which we cannot check the version of: the standard library.

Until now, we have generally gone with the idea that language mode == standard library version. In other words, if I wanted to conditionally use Clock, Instant, or Duration (added to the standard library in Swift 5.7), or to use primary associated types with standard library protocols, I'd use #if swift(>=5.7).

However, reading SE-0020 again after all these years, it appears that is actually a misuse of #if swift. We don't actually have a way to check the standard library version specifically, and to conditionally include/exclude library features based on that. That's the argument for adding #if stdlib.

1 Like
3

To me, that sounds pretty similar to the previously requested #if sdk(...) meant to know which APIs are available at compile time. It's pretty clear that #if stdlib(...) will be used as a proxy for checking the SDK version for the compile-time availability of platform APIs if it becomes available in this form. I'm wondering if there's a way to unify both.

1 Like
4

As far as I know, there's no definition of what makes a valid Swift distribution. So, Apple has, on occasion, shipped toolchains in their SDKs where the compiler version and language mode do not match the standard library version. But if that's still a valid toolchain configuration (and I have to assume it is), then technically any toolchain distribution, from anybody, distributed in any way (even outside of a platform SDK) could do the same.

e.g. Somebody could distribute an installable toolchain with a Swift 5.7 compiler, running in language mode 5.7, but with a 5.4 (pre-concurrency) stdlib, for whatever reason. I can't rule out such a toolchain nor gracefully adapt my code to its existence.

#if sdk(...) is only relevant to Apple platforms, and would likely use some versioning schema relevant to Apple's SDKs (maybe the ... would be an Xcode version, rather than a Swift version?). The idea with #if stdlib(...) is that it is more precise, and does not imply that the stdlib is distributed as part of a versioned platform SDK.

1 Like
5

Does using the appropriate @available(SwiftStdlib …) marking inside the #if swift check here not work? (I would expect it to, but I agree that the “new compiler with old SDK” is an unusual configuration that often behaves in unexpected ways.)

6

@available [EDIT: with versions, as opposed to unconditional unavailability] is compile-time enforcement of a run-time check; it doesn't actually allow you to have alternate implementations depending on which stdlib is available, or to skip compiling something if the stdlib you're compiling against is older than what you specify.

4 Likes
7

I get: warning: unrecognized platform name 'SwiftStdlib' :man_shrugging:

@available still requires the declarations to be present, I think. In this case I'm looking for a build condition which tells the compiler not to even bother looking for the declarations (it will not find a declaration for Clock).

1 Like
8

Ah, right, of course—the issue in these cases isn’t that the appropriate definition of Collection is “unavailable” in the Swift sense, it’s that as far as the compiler knows the definition literally doesn’t exist.

9

I'm not sure if it often works in unexpected ways. The introduction of primary associated types certainly caused confusion and it would be good to figure out a solution that avoids that confusion if something like this happens again in future. But as far as I remember, this is the one instance when this has been a noticeable problem* (though maybe I'm forgetting something from previous years, apologies if so).

* edit: problem directly entangled with the language I mean. Of course it's annoying when a new standard library function appears in one SDK before another – but that's a fairly "conventional" problem in the same sense that any API might be present on iOS but not (yet) on macOS.

However, this proposal is not an appropriate solution. The standard library is not really part of the toolchain. On ABI-stable platforms, it can ship in the OS and checking for things like types and functions needs to be guarded by platform version. Introducing a "stdlib version" that referred not to the runtime version (the thing that needs to be checked 99.99% of the time) but rather the Swift interface files (the thing that needs to be checked in unusual circumstances – AFAICT that have occurred once) would make the situation more confusing.

Not just similar – it's identical. The issue with primary associated types was entirely about what version of the SDK you were using. It happened to affect the standard library, but it would also affect other cross-platform libraries if they had adopted that feature.

1 Like
10

I think the introduction of primary associated types was the most visible situation where this problem arose, but I have a vague memory that it wasn’t the first time where people put in effort to get their libraries compiling both with old Xcode versions and the new betas, only to then run into further issues when the GM seed was released with the new compiler version. I don’t remember that this was due to Swift specifically as opposed to other Apple APIs, though.

11

It's been a problem every year. The .0 releases of Xcode basically haven't been usable for macOS ever since the macOS release was pushed out of sync with the iOS release. I've never seen a reason to bother reporting bugs because it's not like it'd be fixed when it becomes irrelevant in .1 anyway.

1 Like
12

I understand – but that's not a language concern and not a topic for an evolution pitch to address – other than perhaps to introduce language affordances Apple could use to ease that process. But such affordances wouldn't be specific to the standard library as suggested here. The std lib is no different in this regard to, for example, SwiftUI introducing a new function which would be available in the new iOS SDK but not on macOS because that new SDK has not yet been released.

2 Likes
13

The runtime version checking that you mention already exists - it's @available. The issue is precisely that we need to selectively include code based on the compile-time availability of declarations in the standard library's interface files.

It's a compile-time conditional, and is no different to how #if swift does not consider the runtime version.

To illustrate, the declaration for Clock is:

@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
public protocol Clock<Duration> : Sendable

If I try to write a function using Clock:

func doSomething(_: some Clock)

The compiler first needs to see the declaration of Clock (which requires an appropriately-recent Swift interface file for the stdlib) before it can diagnose runtime availability. Absent that declaration, the whole thing fails to build, because the compiler has no idea what Clock refers to.

To allow this code to build with older toolchains, we must guard this function declaration based on the whether the compiler can see the declaration of Clock. There is simply no other way.

14

Yes, I understand the difference. However, as I mention above, the Swift standard library isn't really different in this regard to other parts of the SDK. So introducing a specific feature for it, rather than some more general #if sdk(...), wouldn't be the right solution.

In previous discussions of #if sdk(...) it was pointed out that merely #if available(macOS 11.0, *) would be very misleading, because its similarity with if #available would be too easy for people to mix up the runtime and compile time checks and use the wrong one – hence sdk would be a better spelling. The same applies here – #if stdlib would be a tempting but incorrect thing to reach for when what you actually need is a runtime check against the OS version. This is a secondary concern – "this is a general problem not a specific problem" is the main one. But it's still a concern.

2 Likes
15

This is an availability macro defined by the standard library. You can see the list of defined macros here: https://github.com/apple/swift/blob/main/utils/availability-macros.def

1 Like
16

In Objective-C land, we have the MAC_OS_X_VERSION_MAX_ALLOWED constant that comes from the SDK for these checks. Maybe we could just re-bundle this as:

#if maxAvailable(macOS 13, *)
   ...
#endif

or:

#if maxAvailable(SwiftStdLib 5.7, *)
   ...
#endif
17

Are you saying the name #if stdlib is confusing? If so, how is it more confusing than the current #if swift, and its misuse as a proxy for standard library versions?

But even so, I find this argument itself confusing - even if somebody were to add superfluous #if stdlib checks out of misunderstanding, it would be a relatively harmless mistake. The compiler would still ensure that runtime @available checks were present wherever necessary.

And in the common case of developers who only care about compatibility with Apple's latest SDK, nothing would change, and, as today, they would only be directed to add @available checks. In fact, it would kind of be impossible for the compiler to suggest adding a #if stdlib condition - it requires prior knowledge that a symbol exists in other toolchain configurations, so I see little chance that developers would be tempted to use it accidentally.

May I suggest that this is perhaps a bit of an Apple-centric view of things. You're considering the Swift standard library as only one part of Apple's SDK (so you consider the general problem to be Apple SDK versioning), whereas I'm considering Apple's SDKs as being only one kind of Swift toolchain distribution, so I consider the general problem to be that we have no way to test the standard library version that is included with a particular toolchain.

As I explained above, since we have no concrete definition of what toolchain configurations are "valid", or what even constitutes a "version of Swift", it is entirely possible that other distributions could choose to ship mismatched compilers and standard library versions, for whatever reason. As developers, we need to be able to test for these specific features so that our code can gracefully adapt to different toolchain configurations.

For example, SwiftWasm also distributes installable Swift toolchains (just to give an example). Perhaps, for some reason, it will be necessary one day for them to ship a Swift y compiler, supporting Swift y language features, but with a Swift x standard library. In my mind, that's the general case. It has nothing to do with Apple's SDK versioning. The general problem is that the standard library version is decoupled from the other versions we are able to test for.

Now, Apple's SDK versions may imply a particular standard library version, so there may be some overlap with #if sdk if or when it becomes a thing. That's fine, but I think it is a different problem.

If you still think the spelling is confusing, do you have any suggestions for ways to make it clearer?

1 Like
18

Toolchain is a fuzzy term, but I think it would be a mistake to consider by definition all libraries you might use with the compiler "part of the toolchain". The standard library is often bundled with the toolchain, as are other libraries like dispatch, as a convenience to avoid having to download them as well as the tools, but this isn't always the case, and not just on Apple platforms.

Your example of Wasm is an appropriate, because there it is very important to distinguish between libraries, even fundamental ones like the standard library, and the build tools. Suppose you wanted to create multiple standard libraries – say a stripped down one without stuff like Unicode support, and a "full fat" version that had everything the other platforms have but takes longer to download. That would not need a separate compiler, just a separate library. You would use the same "toolchain" on both. Or you want to use a toolchain that cross compiled for multiple platforms. There again the separation between the tools and the "SDK/frameworks" or "platform interfaces/runtimes" or whatever you want to call them become more important.

2 Likes
19

Might I suggest that is is a general need but doesn't need to be generally implemented initially in order to be useful? For instance, we could:

  1. Introduce module(<Name>, <version>) (or target or sdk) as the general syntax this feature will use.
  2. Tie the standard library's existing versioning information into the feature so it can be used like #if module(Swift, >=5.7) or #if module(_Concurrency, <5.8).
  3. Allow other Apple SDKs to adopt the feature at some point in the near future, given they already have applicable version information.
  4. Expand the feature in another proposal to add the way third party Swift modules can tie their version information into it. This can come at any point in the future since there's not a lot of demand for it now.

This would give us the immediate win of fixing the issues that cause crashing binaries in .0 Xcode releases, establish the shape of the general feature, and allow the evolution process to drive the rest of the solution later, removing the need to handle every case now.

1 Like
20

This is already most of the way there today using an underscored syntax:

#if canImport(Swift, _version: 5.7)
  print("primary associated types for everyone")
#else
  print("new compiler who dis")
#endif

I don't think it supports >= syntax but that could probably be added, @tshortli has been looking at adding features in this area recently and might have some thoughts.

11 Likes