SE-0376: Function back deployment

Hello Swift community,

The review of SE-0376: Function back deployment begins now and runs through November 7, 2022.

Reviews are an important part of the Swift evolution process. All review feedback should be either on this forum thread or, if you would like to keep your feedback private, directly to the review manager. When emailing the review manager directly, please keep the proposal link at the top of the message.

What goes into a review?

The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of Swift. When writing your review, here are some questions you might want to answer in your review:

  • What is your evaluation of the proposal?
  • Is the problem being addressed significant enough to warrant a change to Swift?
  • Does this proposal fit well with the feel and direction of Swift?
  • If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
  • How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

More information about the Swift evolution process is available at

https://github.com/apple/swift-evolution/blob/main/process.md

Thank you,

Freddy Kellison-Linn
Review Manager

24 Likes

Wow this is excellent! I'm gald to see that back-deployment is getting some love!

I read the whole post, but I still have a pretty basic question: Does it matter where the @backDeploy functions get defined?

E.g. if a certain fruit company releases a new convenience API, but not a @backDeploy for it, would it be possible for the community to implement it themselves?

A recent simple example that comes to mind is xpc_array_create_empty(), which is simply a wrapper for xpc_array_create(nil, 0).

Would it be possible for me to define it myself?

@available(macOS 10.7) // Needed for `xpc_array_create`
@backDeploy(before: macOS 11.0)
func xpc_array_create_empty() -> xpc_object_t {
    xpc_array_create(nil, 0)
}
3 Likes

@backDeploy only adds a new capability to APIs defined in libraries that are distributed as dylibs. If you were to implement this convenience in your own project or as an API in an SPM package, then the xpc_array_create_empty() convenience would already be built from source at the desired deployment target already and the attribute would not be enabling anything that was not already previously possible. On the other hand, if you implemented this convenience in a library that is distributed as a library evolution enabled binary, then yes the example would have an effect. However, I'm still not sure it would be useful because in order for clients of your library to ship the library with their app the minimum deployment target of the app would be limited by the minimum deployment target of your dylib. Therefore clients would not really benefit from the use of @backDeploy in your convenience API.

In summary, I wouldn't expect a polyfill library built by the community to use this tool; it would be simplest to just make it an SPM package like you would today.

1 Like

Props to you, Allan et al! I had forgotten all about this proposal, but this will definitely improve the situation for Apple frameworks. its_happening.avi

A few comments:

  • I don’t think this is incompatible with @inlinable; if you use both, you’re saying the body must be used on older OSs and may be used on later ones. That’s a pretty subtle place to be, though, so I can understand leaving it out for now, especially since it would require more implementation work. I suspect it will be interesting in the future, though; many things currently marked @_alwaysEmitIntoClient are trivial helpers that are often completely inlined away.

  • The bodies of @inlinable functions must always be valid for all deployment targets, which has gotten the stdlib into trouble before! A (non-@inlinable) @backDeploy function has a similar concern: the parts not guarded by availability must always be valid for all deployment targets. Since that’s statically optimizable, though, that shouldn’t be a problem; I just predict a fair amount of this idiom:

@available(toasterOS 1, *)
@backDeploy(before: toasterOS 2)
func batchToast() {
  if #available(toasterOS 3, *) {
    // Depends on new API.
    // The check would match whatever API is used,
    // not the back-deploy floor.
    self.slots.forEach { $0.startToasting() }
  } else {
    // Only this part will be back-deployed
    self.startToasting()
    // etc
  }
}
7 Likes

Thanks for bringing this up! Looking back at the proposal I think I failed to clarify the main motivation for making the two attributes incompatible. One of the primary goals of designing the @backDeploy attribute was to have an alternative to @_alwaysEmitIntoClient that guarantees that the system's implementation of the API is used when it is available. This allows you to be confident that any fixes made to the system API will take effect when they are available. So while it is possible to describe a sensible interaction model for @inlinable and @backDeploy, this goal of preferring the system implementation would be compromised. That said, it seems like it could be a reasonable exception given that you'd have to explicitly opt-in to this behavior by writing both attributes together. If reviewers think it would be useful then lifting the restriction is definitely an option.

6 Likes

Just want to lay out a, er, very theoretical situation to make sure I fully understand what this means.

Let's say there's a big company called Orange that has an smartphone operating system (orangeOS). The Swift runtime is included on this OS, along with some UI and other frameworks. In version 16 of orangeOS, they include some new functions in the SDK, but they're only usable on orangeOS 16. On version 17 they add more new functions, but they use this new backDeploy feature.

Assuming a developer needs to recompile their apps with the latest version of the SDKs...

  • Are the backDeployed functions added in version 17 now theoretically usable in all previous orangeOS versions, from orangeOS 1.0?
  • Could they retroactively mark functions added in version 16 with backDeploy and make them usable in previous OS versions?
6 Likes

The availability of the APIs used in the implementation of the function will dictate the back deployed function's availability. If all the APIs called by the function were available in orangeOS 1.0 then the function could be back deployed all the way back to that release, but that won't necessarily be the case.

Yes, nothing inherently prevents @backDeploy from being added to functions that were introduced in a previous SDK without the @backDeploy attribute.

3 Likes

Great feature. One naming feedback (sorry): I find "back" in backDeploy and "before" convey the same information. Maybe we can apply the API guidelines here, and spell it as either @deploy(before: ...) or @backDeploy(to: ...).

4 Likes

“back-deploy” or “backwards-deploy” may be the wrong name, but I don’t think there’s a redundancy here. It’s not a normal deployment (so just “deploy” wouldn’t cover it) and the range to treat specially is a ..< range (so “before” or “upTo” could work, but “to” gets a little ambiguous).

4 Likes
  • Explicit availability must be specified with @available on the same declaration for each of the platforms that the declaration is back deployed on.

Why this restriction? Is it not possible to back-deploy an API infinitely in the past? For instance, if my function body is return 1 there should be no limit to back-deployment because there's no dependency on anything.


Wouldn't it be better to call it @emitIntoClient(before: macOS 11)? This describes what is actually happening, whereas @backDeploy(before: macOS 11) describes an intention without clarity as to what it's doing and what the compromises are (code size mostly).


I remark this attribute is useful only for libraries versioned in lockstep with OS versions; meaning Apple system frameworks.

But it does not help people who want to write their own polyfills for system frameworks outside of Apple; those still need to be written as separate functions with a different (qualified) name. It'd be nice to have something to solve this, but I suppose it'd have to be a separate mechanism. I don't think this is a big problem, mostly an annoyance to have to use a different name.

5 Likes

I think the feature that would (eventually) solve this would be based on disambiguating extension methods by module. Once you have that, you have an answer to the “what if two people did this” problem that already exists today for new names. But that is indeed a separate mechanism from this, because you wouldn’t need an external polyfill-alike to be emitted into the client’s code; being @inlinable would be sufficient.

3 Likes

Agree that “deploy” is not going to cover if. But backDeploy is exactly the right word. It is precisely what we thought it meant.

1 Like

+1 This looks good to me.

I think that's not necessarily true, though I doubt the following use-case would be common: Imagine a company sells a product that allows plug-ins (maybe it's an app controlling a hardware gadget). For third-party plug-in devs they provide an SDK but naturally they don't want to publish their entire source code. Yet, some features only may be available starting with specific newer versions of the app, for that the SDK could @backDeploy at least parts those. In other words: The installed app is/provides the dylib that the SDK accesses but the SDK can also contain @backDeployed methods if the app version is behind the SDK version.

Admittedly, it'd probably be easier to just ensure the app receives an update along with each new SDK version, but perhaps that's not always possible (business users might not renew a license, maybe the connected hardware gadget doesn't support it and you don't want to go through the additional work to still release updates for those, or something else).
I think in such cases it would be useful, no?

Of course the Apple platforms (atm) would benefit the most (or rather we would if that's done and Apple uses it frequently!).

I read the entire thing and it gets a +1 from me. I am also indifferent about the name (I think it's reasonably short and gets the meaning across).

A good use case, but @backDeploy is tied to availability and there's no way (that I'm aware of) to have the app SDK provide an availability target such that you can write if #available(TheApp 2.0) or @backDeploy(before: TheApp 2.0). Meaning you can only back deploy relative to the OS version, which has no relation to the app SDK version. Or should the app SDK include its own compiler toolchain?

Defining library-relative availability independent of the OS would be a good feature to have. Until then this attribute is only useful for frameworks that ship with the OS. This is what I meant.

5 Likes

Oh, of course! I hadn't considered that, my bad!

I was focusing so much about the back deploy logic itself that I forgot @availability relates to the OS version (which is tied to the toolchain, I assume fro your comment?).

I still think this is worthwhile, so my +1 stays, but I guess for the scenario I described we would need more. I guess this could build on top of this proposals implementation then, though.

P.S.: This is why I love it here! I've learned so much about language design and what all goes into it, it's really an invaluable experience. :smiley:

2 Likes

Hopefully we can have that in the future - I opened Availability annotations for third party libraries when using Library evolution/ resilience · Issue #60458 · apple/swift · GitHub for it a while ago. I could see us using @backDeploy in conjunction with this for our frameworks, most definitely.

3 Likes

This restriction is not strictly necessary but I do think it can help with clarity of intent. We have found over time that allowing API owners to omit availability entirely from public declarations in library modules is a nuisance because it's too easy to forget to add availability to new APIs when it is necessary to do so for correctness. We have other tooling approaches that can address that, though.

That's right, although it's not that useful to add @backDeploy to functions in non-SDK libraries today, it could become useful in more contexts in the future if the scope of Swift's availability model expands.

1 Like

This is great! I think it's been mentioned at the start, but being able to provide back deployments within the app itself would be a huge benefit. Some of the examples in this post would be much simpler without having to do the Backport stuff and/or availability checks (which for modifiers in SwiftUI is a pain). Maybe it could be spelt like: @available(iOS 13.0, iOS 15.0) instead of the backDeploy attribute for that case.

The author has updated this proposal to adopt the suggestion that @backDeploy and @inlinable not be mutually exclusive. Review remains active and still runs through Monday, November 7.

6 Likes