[Pitch] Function Back Deployment

Evolution Pitches
30

Having a number of conversations about the design has helped me clarify the intuitive sense that led to proposed design. Leaving aside brevity and diagnostics, I think there are more fundamental arguments to consider about the role of the attributes.

There are two pieces of information that need to be conveyed when back deploying an API:

  • The version of the OS the client can first use the API in
  • The version of the OS when the API became ABI in the OS

While it is useful to talk about these two versions as a package when describing the proposal and its mechanics, these two versions represent very different concepts.

The versions of the OS an API may be used from is directly relevant to API clients. This information is already expressed with the existing @available attribute. Introducing a secondary label or attribute that also describes the first OS that a back deployed API can be used from doesn't give the API client new information that is actionable. I'm thinking of this in terms of progressive disclosure; you shouldn't need to understand what "back deployment" is in order to use a back deployed API. A back deployed API has availability that works like every other API.

The version of the OS the API became ABI in is a detail that the library author needs to provide to the compiler so that the generated code can handle the absence of the ABI version. This version is not availability information; it does not by itself alter which contexts an API can be used from. The primary audiences of this information are the library maintainers and the compiler.

To sum it up, we should not introduce a second version label (or separate attribute) that Swift users need to understand as having the same effect as the introduced: label in @available. The version that the API became ABI in is also not the same type of information as the existing fields of @available since it does not describe a restriction on the contexts in which the API can be used. Therefore it does not belong inside the @available attribute. I think the design as proposed takes these considerations into account best. It introduces a single new attribute with the one new type of version necessary to achieve back deployment while leaving the existing semantics of availability unchanged.

7 Likes
31

In a world where back deployment doesn’t exit, both of the above are the same datapoint in available. The proposal wants to split them up so that available only means usable when using back deployment. Moving the ABI piece to the back deployment part is problematic since I believe that’s one of the most important parts that’s should stay with available.

In a perfect world we would’ve had usability and ABI separated.

1 Like
32

To be clear, existing usage of @available only focuses on API level, which stands for the availability at call site. I don’t think it’s a good idea to confuse it with ABI stuffs. We’re already on the right path of separating ABI and usability.

2 Likes
33

After some initial hesitation I also think separate declarations are clearer

@available()
@backDeploy()

It would be great if this feature allowed developers to “fix” buggy platform implementations on earlier versions.

34

Pardon my ignorance. If this pitch went through would this theoretically enable apple to backport the swiftui fixes and features from iOS 14-15 back to iOS 13?

35

I’m apple platforms availability means the following (unless something was back-deployed like concurrency):

36

There are many reasons why this isn't enough to port all APIs or updates back to a prior OS. The most important, as called out in the proposal, is that this is only for functions not types. Many new API rely on the introduction of a new type as well. Many APIs also aren't just trivially implemented on top of pre-existing code, but instead are implemented in terms of new features in other frameworks only present in the new OS.

Another reason is binary bloat. Unlike a back deployment library (like Swift's pre-ABI stability standard library) the thunks and inlined versions cannot be stripped out by the app store when deploying to a new enough OS, because they are entangled into the binary itself.

Finally, this technique relies on inlining, and inlining is inherently risky, because it exposes internal invariants and ABI in ways that can be very subtle and easy to miss. This proposal goes a long way to mitigate that, by calling into the platform-supplied version whenever it is available. But there are still cases where using this feature could expose behaviors inside a framework in unexpected ways that end up manifesting as binary compat issues or regretted ABI legacy in later releases.

6 Likes
37

It looks like a great proposal but my head spins after realizing that one has to adjust the version in @available when combining it with @backDeploy. I find that very confusing and somewhat inconvenient.

The proposal mentions the introduction of this extension in toasterOS 2, which is already well understood.

extension Toaster {
  @available(toasterOS 2.0, *)
  public func makeBatchOfToast(_ slices: [BreadSlice]) -> [Toast] {}
}

Later on the version is changed because of the addition of the @backDeploy attribute. However this seems to conditionally shift the meaning of @available, which up until now meant that a certain API was introduced with the given OS version.

In the proposed example, this is no longer true:

extension Toaster {
  @available(toasterOS 1.0, *) // this is not when the method was introduced anymore
  @backDeploy(before: toasterOS 2.0)
  public func makeBatchOfToast(_ breadSlices: [BreadSlice]) -> [Toast] { ... }
}

Therefore I somewhat second the to label which was previously mentioned by @Jon_Shier but in combination with the @available attribute.

extension Toaster {
  // introduced in 2.0 (upper bound)
  @available(toasterOS 2.0, *) 
  // describes the lower bound of the deployment range
  @backDeploy(to: toasterOS 1.0) 
  public func makeBatchOfToast(_ breadSlices: [BreadSlice]) -> [Toast] { ... }
}

Yet another alternative naming suggestion: @deployable(from: ...)

5 Likes
38

I don't believe that's true. @available has always meant "you can call this code on this operating system or later". Usually that coincides with when the feature was first added to the OS, but not always. For example, the new Swift Concurrency stuff wasn't added until iOS 15, but it is available back to iOS 13. And thus, the declaration of Task looks like this (with other platforms elided for clarity):

@available(iOS 13.0, *)
@frozen public struct Task<Success, Failure> : Sendable 
    where Success : Sendable, Failure : Error { }

Note that the @available declaration here indicates when the API can be used, not when it was introduced.

So this proposal isn't changing anything in that regard.

1 Like
39

Well it's a bit of an unfair comparison example as the Task type is already back ported and not present in the OS on older OSs as I understood it. I would assume that if @backDeploy was already a thing when Task was introduced, it would probably have a higher OS version parameter in @available.

Also the "upper bound" in my previous message is not meant to be the final OS version, it's as you said, that OS version and later.

To clarify, to me backDeploy lowers the entry OS version from available and implies its availability, but it does not need to change how available marks an API when it got introduced into an OS version.

    iOS 13     iOS 14     iOS 15
------|----------|-----------|------->
      ^                      ^
      |                      |
      |                      @available(iOS 15, *) --*
      |                                              |
      *--------------------- @backPort(to: iOS 13) <-*
40

But why would you assume that? You're asserting that @available has always meant "the OS when this API was first introduced", but when given a counterexample, you say that one doesn't count because @backDeploy didn't exist yet? If that's the case, then no matter how much prior art we demonstrate where @available meant "the OS where you can first call this API", you could still argue that it's invalid because @backDeploy didn't exist. In doing so, you've invalidated the entire appeal to prior art, so your assertion that @available has meant any particular thing is irrelevant.

The proposal itself says that currently, when an API is to be back-deployed, they set the @available version to the earliest OS where the code can run, and then force it to always be inlined. So the proposal itself states that @available currently means "where you can call this code", regardless of when it was added to the OS binaries.

Let me take another approach. Let's assume that in iOS 20, Apple introduces UnicornKit. They want to make it deployable back to iOS 19, though. If the declaration looks like this, as you propose:

@available(iOS 20, *)
@backDeploy(to: iOS 19)
func generateUnicornName() -> String

Then in my code, I might do this:

if #available(iOS 19, *) {
  let name = generateUnicornName()
}

But that seems really odd, because now we've introduced a dissimilarity between @available and if #available. The word "available" now means two different things there, and that will lead to confusion.

More generally, why should code care about what version actually introduced the code to the OS binaries? Why would it be more valuable for @available to point to the version where it's part of the OS binaries instead of the one where the SDK introduces it via inline code? I don't understand the value there.

41

I don't get why you seem so triggered that I found your first counter example unfair. If you have presented something else then I may have argued differently. You cannot imply from a single message that I will always push back on any argument you make. Task is back ported, that's a fact and since there was no other explicit way to mark its availability while it's being back ported, @available was used. That's fairly simple to explain and understand.

As I previously said in my message, I view @backPort as an implicit @available but only which overrides and lowers the explicit introduction OS version. I don't think that this goes against #available either as both @available and @backPort act on the API availability. Unlike in my pitched change, the proposed solution is simply the other way around. Nothing more nothing less, but I find it very confusing to reason about, because the proposed solution shifts the first availability marker away from @available into @backPort. Again, I only meant to say that it's very confusing to me, so I hope we're finally on the same page now. I didn't meant to anger you or something.

42

My apologies—I wasn't trying to express anger. Just confusion. I may have worded things poorly.

Perhaps the problem is simply in the proposed naming. If the proposal were written like this, would you find it less confusing? As I understand it, the proposed semantics would be the same:

@available(iOS 13, *)
@inline(before: iOS 15) // instead of @backDeploy(before: iOS 15)
func whatever() {}
3 Likes
43

Glad you weren't actually triggered. :slightly_smiling_face:

This could possibly do it, but it requires the reader to understand the semantics of an API being inlined. It's not a deal breaker, but I find that version more pleasant to reason about than the proposed form of the attribute.

Edit: Honestly, the more I think about it, the more do I like it. :+1: The general API user probably does not care about the internal shenanigans which the @inline(before:) attribute brings with it and the whole OS introduction thing fades away nicely. As another bonus, this attribute clarify why not everything can actually be back ported as inlining has its limits.

44

FWIW I've been trying to avoid use the term "inlining" for this because I think that there is an important distinction between the existing meaning of inlining in Swift and what @backDeploy would entail. I think of inlining as specifically when the optimizer removes the overhead of a function call, replacing the function call with a copy of its body. Both @backDeploy and @_alwaysEmitIntoClient are different in that they create a copy of the function in your module, but that copy of the function is still called as a function rather than inlined. I've been trying to write "emission into the client" when referring to the behavior of @backDeploy.

4 Likes
45

Right, I think this is the key point: @backDeploy is purely about the implementation of the API, whereas everything in @available is a meaningful constraint on the use of the API. If you were printing the API for documentation or something similar, you would not print @backDeploy, the same way that you would presumably not print something like @inlinable, or any of the other (currently unofficial) optimization-directing attributes.

7 Likes
46

One potential interesting consideration is making a type (like a structure or class, or heck even protocols) be back deployable. I know there are limited facilities for doing this manually; but it might be an interesting future direction at the bare minimum.

1 Like
47

Having done a lot of web development, I am more familiar with the terms shim and polyfill than backDeploy (in fact, this is the first time I ever heard of back deployment).

Would this be clearer ?

@available(macOS 11.0, iOS 14.0, tvOS 14.0, watchOS 7.0, *)
@polyfilled(before: macOS 12.0, iOS 15.0, tvOS 15.0, watchOS 8.0)
func example()
48

...which is why it's in this proposal's future directions section.

1 Like
49

One thing which I haven't covered in the Future Directions section of the proposal yet but seems like it would be a logical next step would be back deployment of protocol conformances, e.g.

@available(toasterOS 1.0, *)
@backDeploy(toasterOS 2.0, *)
extension Toast: Hashable {
  // ...
}

There are plenty of backend details to work out about how this would be accomplished, but there are also some design questions we could ask now about how the proposed attribute would be extended to cover this case. Allowing the attribute to be applied to the extension declaration seems like the natural way to do this, but that raises some questions like whether you should be able to back deploy extensions in general. We'd also need to think about how to handle extension members with heterogenous availability or back deployment. The conservative approach would be to require annotations on every member for maximum clarity, but it would be verbose.

EDIT: I found a thread from a few years ago where @jrose discussed back deployment for protocol conformances.

I think "back deploy" is a fairly well established term of art in the Swift community (there are 35 hits when I search this forum for the phrase "back deploy"). This is the first time I've heard the term polyfill myself, but perhaps it feels more familiar to others (EDIT: I searched for it here on the forums as well and it also comes up plenty, though not as frequently; I guess this is just my ignorance!) As for shim, I think it could work when combined with "compatibility", but to me "back deploy" feels like a more precise description of what the attribute accomplishes.

3 Likes