SE-0376 (second review): Function back deployment

It seems that this proposal very subtly changes the meaning of @available from “is provided at runtime by the OS since version X” to “can be used from your code when targeting version Y, potentially provided by code emitted into your binary”.

Has there been discussion to maintain the semantic of @available, and annotating back deployment more explicitly?

i.e. instead of

extension Toaster {
  @available(toasterOS 1.0, *)
  @backDeployed(upTo: toasterOS 2.0)
  public func makeBatchOfToast(_ breadSlices: [BreadSlice]) -> [Toast] { ... }
}

consider

extension Toaster {
  @available(toasterOS 2.0, *)
  @backDeployed(to: toasterOS 1.0)
  public func makeBatchOfToast(_ breadSlices: [BreadSlice]) -> [Toast] { ... }
}

Yes, see "Alternatives considered" for a summary of considerations, or refer to preceding threads linked in the document.

2 Likes

As @xwu mentioned, this was a common category of feedback in the previous threads. I don't think this proposal introduces a new meaning for the attribute. The primary function of @available in the language is to aid the type checker in ensuring that each lexical region of code in your program will execute safely at runtime with respect to OSes it may run on. The semantics of @available are unchanged; you may make use of a @backDeployed function in any contexts where it is "available" according to the context's availability. What's new is how the compiler generates code for calls to functions that are @backDeployed. Note that if you consider the interaction of @available and the existing @_alwaysEmitIntoClient attribute (which is widely used in the standard library and some modules in Apple's SDKs) then @available already isn't a direct indication of the presence of an ABI symbol in a binary.

11 Likes

That makes a lot of sense. Thanks!

1 Like

Not to mention @available in app code, or third-party library code. :-)

1 Like

I'm sorry for the relatively late feedback here; I had meant to post this earlier.

Here's Allan's argument for why @backDeployed requires @available from the previous thread:

This is a classic case of proving too much. Nothing in this is specific to back deployment or even mentions it. It is, very plainly, an argument that there's a pervasive problem where the developers of libraries with stable ABIs are forgetting to add @available to all of their ABI-exposed APIs.

I completely sympathize with this problem, and we should take action to address it. There are two obvious ways to do so. One would be to add ABI checking as part of the standard build process, which is building on the tooling that Allan is talking about; there are a lot of good reasons to pursue that, but it's a longer-term thing. The second is to simply require all ABI-exposed APIs in ABI-stable libraries to have @available attributes, maybe with some new spelling (@available(*)?) to explicitly say that an API has no availability restrictions.

Neither of these solutions is specific to @backDeployed functions, and the restriction on @backDeployed functions does not make any significant impact on the problem. In my opinion, Swift should not address this in such a piecemeal way. This restriction should be removed from this proposal, and then we should solve the problem for real.

11 Likes

I think this is a reasonable point of view, and I'm happy to remove the restriction if the language workgroup concludes that it's unwarranted. I think I differ on the more meta point of whether taking a piecemeal action to resolve a part of the common problem is worthwhile but like I said, there are other tooling approaches available to address the problem. More importantly, when I originally pitched this feature the simultaneous attribute requirement would have served as the only reminder to think about availability because the compiler was not able to otherwise help. Now, however, the compiler does actually diagnose availability inconsistencies in the declarations and bodies of APIs that are available before the module's deployment target. So the tooling to address possible mistakes in @backDeployed functions is already in place, really.

I did have another reason for wanting to require the attributes simultaneously which I didn't express directly anywhere. As evidenced by the review and pitch threads, folks understandably have a bit of trouble grokking what it means for an API to be back-deployed "up to" a release, instead of "back to". Many expressed that they wanted to see the role of @available and the new attribute reversed, where @available would indicate the release of ABI introduction and the new attribute would indicate how far back the the API can be used. For reasons I explained earlier in this thread I think the roles of the attributes as proposed are correct, but the reviewer sentiment still indicates something about how people think about back deploying an API. Requiring @available may help an author understand as they are adding @backDeployed to an API that there are two different OS versions that are relevant and they may need to think explicitly about which release they are back deploying to. However, the type checker improvement I described above serves largely the same role in a more direct way.

3 Likes

I agree it's hard to reason about. I think the problem is backDeployed is expressing a counter measure for the absence of an ABI rather than just declaring its presence. It highlights what is not there rather than what is there. I think if the attribute was describing things going forward rather than backward, it'd be easier to read correctly:

@available(macOS 11, *)
@abi(macOS 13, *)
func doSomething() {...}
1 Like

I'm just dropping by to say that I've had the opportunity to fix a few library bugs in the stdlib using this feature already and it's been an absolute lifesaver, really looking forward to it -- thanks @tshortli!

8 Likes

Sorry for the extremely late breaking review, but I happened to be working with this attribute today and wanted to register the feedback that only really occurred to me whilst I was doing so, which is that I think the original choice of @backdeploy(before:) was pretty clearly better than the newly proposed upTo:.

I understand the analogy with PartialRangeUpTo, but it seems irrelevant to the naming of this property. PartialRangeUpTo never appears to users and is really an implementation detail of the range expression infrastructure – a mechanic of how we make the ..< prefix operator work. It is not a type to name other more visible parts of the language after.

The decision should instead be made based on what will be most obvious to readers wanting to understand the availability of a function, and in this case before: seems the better choice as it represents more natural English language usage for when the back-deployment functionality applies.

8 Likes

We already have visible parts of the language that use this terminology:

2 Likes

This helped solidify my feeling that upTo: is a poor choice.

  • upTo: is not a single consistent term used throughout the language, as demonstrated by stride
  • it’s striking how much uglier the compound camel-cased word is when comparing it to stride’s to:
  • there is a clear driver here that matters specifically for these two examples: the need to distinguish between “to” and “through”, that requires a matching pair of argument labels That does not apply to the availability usage.

But more important than all of these is that “the function is back deployed up to iOS 13” is an unnatural phrasing. You just wouldn’t say it in normal use. Whereas “the function is back deployed before iOS 13” is entirely natural. This isn’t just hypothetical – someone I spoke to, looking at this for the first time yesterday, was confused by what it mean until I explained it. I believe this wouldn’t have been as likely with before:.

So it seems that we are sacrificing readability for consistency (and, given the comparison with stride, mixed consistency at best), which is the wrong way around.

10 Likes

As someone who totally missed the name is being proposed to be changed in this review, and used the before: spelling in practice already... Yeah before: was very easy to understand what it'll do -- I wasn't confused what version to put in there, especially since it paired well with availability "available from N and backDeploy it before M". I've not been following this review closely, other than "I had to use it, it worked great" so I came over to share some enthusiasm.

I was not that "someone" Ben is mentioning here, so I can be counted as another datapoint in favor if that helps making the case. :slight_smile:

Just as a counterpoint, I was confused as to the exact meaning of it with before, and with upTo I'm less confused. :sweat_smile:

2 Likes

Honestly I wouldn’t use either of these phrasings. The function is back deployed by shipping it with the client specifically so that the same binary can run on many different OS versions. It’s used on certain older versions.

For me, the confusion between upTo and before is that one is an open interval and the other is closed, and it’s not clear which is the correct interpretation.

To be pedantic (the best kind of antic) it should only back deploy when your minimum deployment target requires it. So to expand the phrase: “the function is back deployed <when your minimum deployment target is set to> before iOS 13”. There is not a similarly smooth expansion in the "up to" case (let me try... "the function is back deployed <when your minimum deployment target is set to a value> up to iOS 13"... not great).

Note that this kind of expansion is pretty normal e.g. "Up to" is really short for "up to but not including" but that meaning is supposed to be given as a term of art (though just now, someone in our office chat just said "IMO it isn't immediately clear whether upTo is inclusive or not" :) We should not try to stuff more words into it, either way, just try and find the best short word (or two if we must) that sounds right and most people get the right inference from.

1 Like

The term until is used in the SDKs, for "up to" a condition, event, or point in time.

@available(SwiftStdlib 5.1, *)
@backDeployed(until: SwiftStdlib 5.8)
3 Likes

If this is something folks would like to explore further, I'd like to hear if there are any other suggestions for what the name could be. I personally find that @abi is not specific enough to the behavior in question because the term ABI is used to emphasize different concepts in various contexts. I also think that for many developers it may not provide any kind of intuitive sense of the effect the attribute would have. But the general idea of reversing the polarity is one that I think is worth exploring.

I like this suggestion a lot. Do others also feel like until: helps clarify the exclusivity of the OS version bounds?

I feel like @backDeploy(before:) would be clear as well.

2 Likes