[Pitch] Function Back Deployment

Evolution Pitches
1

Hi folks,

I'd like your feedback on an attribute I've been designing that allows functions to back deploy to older OSes:

/// This function is available to clients targeting macOS 11
/// or later, but it wasn't present in the OS until macOS 12.
@available(macOS 11, *)
@backDeploy(before: macOS 12)
public func foo() { ... }

When running on an older OS, a client calling this function would use a local copy of the function made by copying the body exposed in the library's swiftinterface file (similarly to how @inlinable works). If the API is available from the library at runtime, though, then the copy of the function in the library will be used instead. This gives authors of resilient libraries the option to make convenient APIs more available while also ensuring that users with up-to-date OSes get bug fixes without a recompilation of the client. It also reduces code size bloat in comparison to @_alwaysEmitIntoClient (the existing solution to this problem) when the client's deployment target is high enough that the back deployed API will always be present.

The full text of the proposal is on GitHub. Thanks for reading!

26 Likes
2

It seems like you could remove the need to have both @available and @backDeploy by just using a @backDeploy(to:) form rather than before, since before requires the lowest point to be provided separately.

1 Like
3

Could you give a concrete example of what you're imagining?

4

From your example, I'd think

would be equivalent to

@backDeploy(to: macOS 11)
public func foo() { ... }

Since the back deployment implies the lowest availability, why use two attributes?

5

The compiler needs to know which version the function became ABI in so that it can generate this code in the intermediary thunk:

  if #available(macOS 12, *) {
    // call original library function
  } else {
    // call fallback function
  }
6

So that's implied by the before value? Still seems like there should be a way to express that more directly. :thinking:

7

It seems to me like it's being specified rather explicitly by the before value. That's the entire purpose of the whole clause, isn't it? It's saying "before macOS 12, use the special 'back-deployment' option"

2 Likes
8

Given its label doesn't actually what it does, no, it's not explicit. You have to already know what the combination of @available and @backDeploy does to get the full picture.

@backDeploy(fromStable: macOS 12, to: macOS 11)

would be more clear to me.

1 Like
9

My mental model is that @available and @backDeploy are doing different things here, but those things are composable. The availability attribute is necessary if things referenced in the implementation are only available from a specific deployment target. That's the existing functionality of @available today, so I don't think it makes sense to invent a new way to spell that. @backDeploy tells the compiler to emit the implementation of the API into the client code when the deployment target is below the given bound.

7 Likes
10

I think the proposal 'cheats' slightly by showing most of the examples with only one or two OS' listed in the @available block. The reality is, the SDKs are littered with long declarations like:

@available(iOS 15.0, macOS 12.0, tvOS 15.0, watchOS 8.0, *)

So @available(toasterOS 1.0) @backDeploy(before: toasterOS 2.0) is more realistically:

@available(iOS 12.0, macOS 10.8, tvOS 12.0, watchOS 5.0, *)
@backDeploy(before: iOS 15.0, macOS 12.0, tvOS 15.0, watchOS 8.0)

That is a pretty unwieldy long spelling, and quite hard to mentally parse. Unfortunately, I don't really have a solution to suggest; it might be an issue that is bigger in scope like some kind of macro to specify all Apple OS's of a particular year or something.

1 Like
11

I think this pattern is definitely one worth streamlining. One interesting wrinkle is that, in our experience with the Swift standard library and Apple frameworks, the "back deploy" and "in-the-OS" implementations often end up diverging for various reasons, often because the version for emitting into clients has to implement by piecing together existing bits of ABI from the older OS, whereas the "real" implementation can take direct advantage of new OS functionality. It'd be good to explore how that can look with an official back deploy feature.

13 Likes
12

Would it be reasonable to infer the availability attribute based on the versions provided in the @backDeploy attribute? For example, using the major/minor/patch version before the one specified. So for example, @backDeploy(before: macOS 12) implies @available(macOS 11, *), but if one needs more control they can explicitly add @available with specific version numbers.

We can come up with some rules for the inference, but this might help simplify the common case. We could also enhance Xcode to show the inferred availability attribute when one long clicks the backDeploy attribute.

Although, I guess some people might not prefer this kind of inference as it may make things less explicit.

The macro idea from @bzamayo is also interesting. I think we do have the ability in the compiler to specify a “typealias” for availability attributes, IIRC. It might be worth providing a way to use it here if it makes sense.

13

From a purely naming perspective, it seems to me that “back deploy before” is logically equivalent to “ABI stable since”.

If we spell the attribute more like @abiStable(...) then it becomes a statement about the function itself.

This would also open the door to the possibility of requiring explicit annotation for ABI stability. We already say, “Nothing is public unless it is explicitly marked as such”, and it seems reasonable to do the same for ABI stability.

That would allow, for example, new additions to the standard library to have an “introductory” period where they are always emitted into clients, and then when they are well and truly ready, their ABI can be finalized and the @abiStable annotations added.

3 Likes
14

It would depend on the exact design, but inference seems a little risky to me because the first version you can call the function from becomes a source compatibility requirement and therefore needs to be maintained with care.

There is an "availability macro" feature (swift-frontend argument -define-availability) and this attribute would support it. I'm actually in the midst of adding support for it to the prototype right now. Note, however, that availability macros are currently designed mainly as tools to help the API author with brevity and consistency. When a swiftinterface is generated from source using availability macros, the macros are exploded into their individual components (which is necessary since the macro definitions are not part of the source).

2 Likes
15

Something like this would work when the in-the-OS version needs to be implemented differently than the back deployed version:

@available(macOS 12, *)
@usableFromInline
internal fooImplementation() {
  // ...
}

@available(macOS 11, *)
@backDeploy(before: macOS 12)
public func foo() {
  if #available(macOS 12, *) {
    fooImplementation()
  } else {
    // reimplement foo using other ABI stable interfaces available in macOS 11+
  }
}
16

I thought about this, but in addition to rigidity you need as @tshortli noted, I also wonder whether that's the right default in the first place. We have no data yet, but I feel like most people would want to go back as far as Swift supports and only work forward if that's impossible. That intuition might be wrong, but I don't know that we want to limit the feature so severely right away.

They are doing different things, but since they're related, it would make the feature much easier to use if they were part of the same interface, not two separate interfaces you need to know to compose together. Now, good diagnostics could help with that pain, but general clean design seems to indicate, since we can't explicitly compose @available and @backDeploy together, as offered in one of the alternatives, that a system where we can would be better. We can do that in @available or @backDeploy. I like @backDeploy here since using that feature is really two things in one, the actual generation of the back deployment API and the availability markup.

In the end though, the feature really just needs two things: the stable availability and the back deployed availability. I just think those should both be expressed the same way.

17

That might not be as efficient as it could be, though, since the availability check is a runtime check. It'll likely get constant folded out of the in-the-OS version where the deployment target makes it always true, but that check would remain in the back-deployed emit-into-client version, on top of the implied check that already has to be there to trampoline into the in-the-OS version.

18

This isn't beautiful but it is something that we could potentially make work using the existing machinery for how inlinable bodies are emitted into swiftinterfaces and would elide the version check from the client copy:

@available(macOS 11, *)
@backDeploy(before: macOS 12)
public func foo() {
#if SOME_FLAG_NOT_DEFINED_WHEN_EMITTING_SWIFTINTERFACE
  if #available(macOS 12, *) {
    fooImplementation()
    return
  }
#endif
  // reimplement foo using other ABI stable interfaces available in macOS 11+
}

We could give framework owners a way to specify flags specifically for this purpose.

Another way to approach this entirely would be to somehow allow the library author to implement two entirely separate copies of the body and then link them with attributes that indicate they implement the same logical function, but one gets emitted into the swiftinterface and one gets compiled into the library. That feels pretty heavy-weight to me, though.

1 Like
19

Something like that is definitely needed to cut down on boilerplate. @xymus has been doing some early prototyping in this area. But I think it's orthogonal to this proposal. Also, even when we have the ability to emit "packs" of platform versions that all go together most of the time, you'll still need the ability to lay out the full fine-grained versions individually because occasionally an API will be introduced to different OSs in different releases/times rather than all in the same time-aligned release.

7 Likes
20

I do think that a single attribute that carries both versions is a viable option for this, but if we go that route I would advocate that the one attribute be the existing @available attribute (as described in the Alternatives Considered). I chose not to propose that design because, as @hborla put it, I see these attributes as ones that compose together.

Specifying the "introduced" version to @backDeploy would effectively be creating a parallel way to spell the basic capability of @available, but without all the related features. Then you would either need to also be able to still use @available when additional behaviors are needed (deprecation, etc) or @backDeploy would need to duplicate @available's capabilities. Explaining to an API client how one determines which contexts you can call a given API from becomes a bit more complicated.

I would prefer a combined attribute design like one of these:

@available(macOS, introduced: 11.0, backDeployedBefore: 12.0)
// or
@available(macOS, introduced: 12.0, backDeployedTo: 11.0)

The drawbacks in my mind are:

  • the long spelling of @available (one line per platform) is required when it would not otherwise be
  • less attention is drawn to the special behavior this function will have
  • it's harder to provide clear diagnostics explaining issues raised by opting in to back deployment (there is not an independent attribute to point to directly)

None of these are showstoppers, of course. But I also still find the composition approach a bit cleaner.

5 Likes