[Pitch] Function Back Deployment

Evolution Pitches
21

I'm not a design patterns expert, but does using two separate APIs in close proximity actually count as composition? I've never heard of, say, drawing APIs which require context.allocate(); context.fill() to fill an already set color as being a composition of allocation and filling.

Mainly I'm worried that the feature, as pitched, is easy to get wrong since it requires both sets of targets to be separate. It also relies on two attributes to get one feature.

But you're right that the alternative @available syntax isn't great, especially requiring separate declarations for each platform. I do wonder if there's room for something like @available(macOS (12 backDeployedTo: 10.15), iOS (15, backDeployedTo: 13), *). Verbose, but allows us to immediately align both stable and back deployed targets, and could easily be replaced with year macros when they're available. @available(apple (2021, backDeployedTo: 2019), *).

I'm not really satisfied with that version either, but perhaps others have ideas.

22 
//current 

extension Toaster {
  @available(toasterOS 2.0, *)
  public func makeBatchOfToast(_ slices: [BreadSlice]) -> [Toast] { … }
}

// proposed 
extension Toaster {
  @available(toasterOS 1.0, *)
  @backDeploy(before: toasterOS 2.0)
  public func makeBatchOfToast(_ breadSlices: [BreadSlice]) -> [Toast] { ... }
}

// suggestion 
extension Toaster {
  @available(toasterOS 2.0, *)
  @available(toasterOS, introduced: 1.0, backDeployBefore: 2.0)
  public func makeBatchOfToast(_ breadSlices: [BreadSlice]) -> [Toast]
}

The divergence of available starting at min and backDeploy starting max is throwing me off. I see why the before label is needed but actually the directionality is confusing to me. Please see my suggestion above.

23

I like this alternative.

2 Likes
24

As a reader, these seem vastly superior to me. I can immediately understand what this means for each OS I’m interested in, whereas with the original version I have to jump back and forth between the two attributes and piece together an understanding.

1 Like
25

As I was reading down the thread this came to mind, and then it was suggested!

In my opinion this solution makes much more sense.

  • You can see the availability of a function in a single place and reduces the cognitive load of trying to work out when something is available by combining two attributes in your head (I know it's not much load, but clarity should be key). It might make one line longer, but it keeps all availability information on a single line rather than multiple (imagine a case where other @ attributes are used between @available and @backDeploy, its going to get very confusing/messy and easy to miss).
  • @availability is already widely used, people are used to it and will automatically look there to see when something was made available, they won't be expecting to look for another entirely separate attribute.
  • Regarding your point about diagnostics; I think as long as the diagnostic message is clear this will be fine.
1 Like
26

@available(macOS, introduced: 12.0, backDeployedTo: 11.0)

This form does look nicer in isolation. But let's be clear about what it will mean in practice. Currently, an API declaration uses a shorthand declaration availability form like this:

@available(macOS 11.0, iOS 14.0, tvOS 14.0, watchOS 7.0, *)
func example()

That multi-platform syntax is not available if you're adding additional parameters, so it would have to look like this instead:

@available(macOS, introduced: 12.0, backDeployedTo: 11.0)
@available(iOS, introduced: 15.0, backDeployedTo: 14.0)
@available(tvOS, introduced: 15.0, backDeployedTo: 14.0)
@available(watchOS, introduced: 8.0, backDeployedTo: 7.0)
func example()

On the other hand, the original proposal would give us this spelling:

@available(macOS 11.0, iOS 14.0, tvOS 14.0, watchOS 7.0, *)
@backDeploy(before: macOS 12.0, iOS 15.0, tvOS 15.0, watchOS 8.0)
func example()

Putting it directly into the @available attribute is easy to understand in isolation, but causes an explosion of attributes in common cases. I'd be interested in exploring ways we can make the spelling clear without multiplying the number of attributes we need.

7 Likes
27

My main issue with this approach is that the first available is not longer the introduction version which has now to be encoded in the backDeploy section.

@available(macOS 12.0, iOS 15.0, tvOS 15.0, watchOS 8.0, *) // this should stay the same as today
@availableBackDeploy(macOS 11.0, iOS 14.0, tvOS 14.0, watchOS 7.0) // this is the diff. 
func example()
28

Why does it matter when it was actually introduced, though? If I can call the API on macOS 11.0 or later, then it is available from macOS 11 or later. It doesn't matter too much to me as a caller whether that's calling through to a native OS library or whether it's being emitted as inline code into my app's binary.

I think it would be more confusing if there were a function annotated @available(macOS 12), but I could call it on macOS 11.

3 Likes
29

In my view func back deployment is a really great feature to support 1-3 versions OS versions back when the user base long tail still has not updated. Let’s say if I introduced an API today with this feature ( let’s call it AppleOS 21) and decide to back deploy 3 versions back ( AppleOS 18 ) then my hope is that I can start removing older than 3 version back deployment every year.

// today 
@available(macOS 12.0, iOS 15.0, tvOS 15.0, watchOS 8.0, *) // this should stay the same as today
@availableBackDeploy(macOS 11.0, iOS 14.0, tvOS 14.0, watchOS 7.0) // this is the diff. 
func example()

// 1-3 years future back deployment gone. 
@available(macOS 12.0, iOS 15.0, tvOS 15.0, watchOS 8.0, *) 
func example()
1 Like
30

Having a number of conversations about the design has helped me clarify the intuitive sense that led to proposed design. Leaving aside brevity and diagnostics, I think there are more fundamental arguments to consider about the role of the attributes.

There are two pieces of information that need to be conveyed when back deploying an API:

  • The version of the OS the client can first use the API in
  • The version of the OS when the API became ABI in the OS

While it is useful to talk about these two versions as a package when describing the proposal and its mechanics, these two versions represent very different concepts.

The versions of the OS an API may be used from is directly relevant to API clients. This information is already expressed with the existing @available attribute. Introducing a secondary label or attribute that also describes the first OS that a back deployed API can be used from doesn't give the API client new information that is actionable. I'm thinking of this in terms of progressive disclosure; you shouldn't need to understand what "back deployment" is in order to use a back deployed API. A back deployed API has availability that works like every other API.

The version of the OS the API became ABI in is a detail that the library author needs to provide to the compiler so that the generated code can handle the absence of the ABI version. This version is not availability information; it does not by itself alter which contexts an API can be used from. The primary audiences of this information are the library maintainers and the compiler.

To sum it up, we should not introduce a second version label (or separate attribute) that Swift users need to understand as having the same effect as the introduced: label in @available. The version that the API became ABI in is also not the same type of information as the existing fields of @available since it does not describe a restriction on the contexts in which the API can be used. Therefore it does not belong inside the @available attribute. I think the design as proposed takes these considerations into account best. It introduces a single new attribute with the one new type of version necessary to achieve back deployment while leaving the existing semantics of availability unchanged.

7 Likes
31

In a world where back deployment doesn’t exit, both of the above are the same datapoint in available. The proposal wants to split them up so that available only means usable when using back deployment. Moving the ABI piece to the back deployment part is problematic since I believe that’s one of the most important parts that’s should stay with available.

In a perfect world we would’ve had usability and ABI separated.

1 Like
32

To be clear, existing usage of @available only focuses on API level, which stands for the availability at call site. I don’t think it’s a good idea to confuse it with ABI stuffs. We’re already on the right path of separating ABI and usability.

2 Likes
33

After some initial hesitation I also think separate declarations are clearer

@available()
@backDeploy()

It would be great if this feature allowed developers to “fix” buggy platform implementations on earlier versions.

34

Pardon my ignorance. If this pitch went through would this theoretically enable apple to backport the swiftui fixes and features from iOS 14-15 back to iOS 13?

35

I’m apple platforms availability means the following (unless something was back-deployed like concurrency):

36

There are many reasons why this isn't enough to port all APIs or updates back to a prior OS. The most important, as called out in the proposal, is that this is only for functions not types. Many new API rely on the introduction of a new type as well. Many APIs also aren't just trivially implemented on top of pre-existing code, but instead are implemented in terms of new features in other frameworks only present in the new OS.

Another reason is binary bloat. Unlike a back deployment library (like Swift's pre-ABI stability standard library) the thunks and inlined versions cannot be stripped out by the app store when deploying to a new enough OS, because they are entangled into the binary itself.

Finally, this technique relies on inlining, and inlining is inherently risky, because it exposes internal invariants and ABI in ways that can be very subtle and easy to miss. This proposal goes a long way to mitigate that, by calling into the platform-supplied version whenever it is available. But there are still cases where using this feature could expose behaviors inside a framework in unexpected ways that end up manifesting as binary compat issues or regretted ABI legacy in later releases.

6 Likes
37

It looks like a great proposal but my head spins after realizing that one has to adjust the version in @available when combining it with @backDeploy. I find that very confusing and somewhat inconvenient.

The proposal mentions the introduction of this extension in toasterOS 2, which is already well understood.

extension Toaster {
  @available(toasterOS 2.0, *)
  public func makeBatchOfToast(_ slices: [BreadSlice]) -> [Toast] {}
}

Later on the version is changed because of the addition of the @backDeploy attribute. However this seems to conditionally shift the meaning of @available, which up until now meant that a certain API was introduced with the given OS version.

In the proposed example, this is no longer true:

extension Toaster {
  @available(toasterOS 1.0, *) // this is not when the method was introduced anymore
  @backDeploy(before: toasterOS 2.0)
  public func makeBatchOfToast(_ breadSlices: [BreadSlice]) -> [Toast] { ... }
}

Therefore I somewhat second the to label which was previously mentioned by @Jon_Shier but in combination with the @available attribute.

extension Toaster {
  // introduced in 2.0 (upper bound)
  @available(toasterOS 2.0, *) 
  // describes the lower bound of the deployment range
  @backDeploy(to: toasterOS 1.0) 
  public func makeBatchOfToast(_ breadSlices: [BreadSlice]) -> [Toast] { ... }
}

Yet another alternative naming suggestion: @deployable(from: ...)

5 Likes
38

I don't believe that's true. @available has always meant "you can call this code on this operating system or later". Usually that coincides with when the feature was first added to the OS, but not always. For example, the new Swift Concurrency stuff wasn't added until iOS 15, but it is available back to iOS 13. And thus, the declaration of Task looks like this (with other platforms elided for clarity):

@available(iOS 13.0, *)
@frozen public struct Task<Success, Failure> : Sendable 
    where Success : Sendable, Failure : Error { }

Note that the @available declaration here indicates when the API can be used, not when it was introduced.

So this proposal isn't changing anything in that regard.

1 Like
39

Well it's a bit of an unfair comparison example as the Task type is already back ported and not present in the OS on older OSs as I understood it. I would assume that if @backDeploy was already a thing when Task was introduced, it would probably have a higher OS version parameter in @available.

Also the "upper bound" in my previous message is not meant to be the final OS version, it's as you said, that OS version and later.

To clarify, to me backDeploy lowers the entry OS version from available and implies its availability, but it does not need to change how available marks an API when it got introduced into an OS version.

    iOS 13     iOS 14     iOS 15
------|----------|-----------|------->
      ^                      ^
      |                      |
      |                      @available(iOS 15, *) --*
      |                                              |
      *--------------------- @backPort(to: iOS 13) <-*
40

But why would you assume that? You're asserting that @available has always meant "the OS when this API was first introduced", but when given a counterexample, you say that one doesn't count because @backDeploy didn't exist yet? If that's the case, then no matter how much prior art we demonstrate where @available meant "the OS where you can first call this API", you could still argue that it's invalid because @backDeploy didn't exist. In doing so, you've invalidated the entire appeal to prior art, so your assertion that @available has meant any particular thing is irrelevant.

The proposal itself says that currently, when an API is to be back-deployed, they set the @available version to the earliest OS where the code can run, and then force it to always be inlined. So the proposal itself states that @available currently means "where you can call this code", regardless of when it was added to the OS binaries.

Let me take another approach. Let's assume that in iOS 20, Apple introduces UnicornKit. They want to make it deployable back to iOS 19, though. If the declaration looks like this, as you propose:

@available(iOS 20, *)
@backDeploy(to: iOS 19)
func generateUnicornName() -> String

Then in my code, I might do this:

if #available(iOS 19, *) {
  let name = generateUnicornName()
}

But that seems really odd, because now we've introduced a dissimilarity between @available and if #available. The word "available" now means two different things there, and that will lead to confusion.

More generally, why should code care about what version actually introduced the code to the OS binaries? Why would it be more valuable for @available to point to the version where it's part of the OS binaries instead of the one where the SDK introduces it via inline code? I don't understand the value there.