[Pitch #2] Called-once function types

Evolution Pitches
21

The existing rules for class method override variance are already a small subset of the function conversion rules allowed in expressions (and I believe the feature was originally motivated by Objective-C compatibility), so extending this behavior to allow more forms of variance does not help consistency, I think.

22

Why not have the override rules match the protocol conformance rules? I’m thinking of a class as a bit like a protocol where all the requirements have a default implementation.

23

The rules for matching witnesses against protocol requirements are even stricter than what we allow for class method overrides, basically you can’t have any variance at all except for a few special cases like Any vs any Sendable in preconcurrency contexts. This is because of associated type inference; you can’t infer an associated type from the parameter or return type of a witness if there’s also variance involved because then the problem becomes ambiguous.

1 Like
24

This is true, although I have to say that I think we could be strict for associated type inference and then forgiving when matching witnesses against known signatures.

2 Likes
25

Unlike with other NC types, you know the only way to use the value is to consume it... I feel like it's fine to make it an error to write a function signature that cannot be used at all; @once is exactly once, so you must call a function you cannot call. If we don't diagnose the signature you get an error about not calling it, attempt to call it, get an error when you attempt to call it, then scratch your head about why you cannot call it (unless we add a special case diagnostic wrt once + borrowing when you attempt to call it). This lets us have an error with a diagnostic group explaining how the need to call and consuming call semantics conflict with borrowing access.

26

In theory, on an @isolated(any) closure, you could access .isolation without consuming it.

2 Likes
27

Lovely and extremely thorough proposal, I'm excited about the expressivity this will provide and the performance it can unlock.

While I don't know why one does this, point taken, we should have some specialized diagnostics for attempting to call borrowing @callable(once) if we don't forbid this. borrowing @once would still be an error.

Since a borrowed function value cannot be called, explicitly declaring these parameters borrowing is an error

inout is acceptable for @callable(once), and requires the body to either not call it, or replace it if called? what would inout + @once do, is the obligation given back to the caller? or is the body of the callee unable to satisfy ~Deinitable?

Since Optional requires its wrapped type to be Deinitable (see Storage), optional @once function types cannot be formed, so called_once parameters that are _Nullable or _Null_unspecified are imported as ordinary optional function types:

Hmm. Should this be imported as an optional @callable(once)? That would increase what the Swift side caller is allowed to do, although it seems unfortunate to use the wrong weakening due to lack of generalization... Or is the thinking that we can change the importer behavior under these conditions for the first time if/when we generalize Optional to Deinitable?

Clang only diagnoses violations of called_once with warnings. To preserve memory safety for Swift code using @once functions, the compiler generates wrapper closures for values passed as arguments to called_once parameters that dynamically enforce that the closure is called exactly once. If the wrapper closure is called more than once or is destroyed without being called, the program traps.

How does this compose? If the function is passed back and forth between C++ and Swift, could it accumulate N wrappers? Can this be disabled for programs where clangs enforcement is configured to error, to avoid losing performance? We generally treat C++ annotations as unsafe and trusted, I'm very sympathetic to not trusting them (so many are wrong) but for a mixed project who has made all relevant checks errors is it necessary to pessimize their interop performance? I see the future direction, but it seems a little strange to draw a line in the sand for this kind of unsafe annotation assumption.

Switching a parameter or result between @once and @callable(once) is not ABI-compatible either, in any position. The compiler may assume that a @once closure is never destroyed without being called, so it would be unsafe for an existing client to pass a @once closure to a library that changed the parameter to @callable(once). The two attributes are distinguished in the mangling of function types, so changing one to the other changes the symbol names of the declarations that use the type.

It should be ABI compatible to go from @escapable @callable(once) to @escapable @once if you preserve mangling, right? Not that people should do that... but it seems ABI compatible to me given both will be heaped, neither needs refcounting, and @once (presumably?) features a program shared trapping deinit to make conversions cheaper.

Is there a conversation to be had about how necessary withoutActuallyDeiniting is at all? I suspect we will add it, but unlike withoutActuallyEscaping we are introducing both types of ~Copyable function types at the same time, so there should not exist APIs which choose the "wrong" annotation. In what case does the programmer know more than the api author about if a function will be called that should not have been encoded into a different api instead?

Is the composition of withoutActuallyEscaping and withoutActuallyDeiniting correct? It would allow you to take a non-escapable @once and get an @escaping @callable(once), but the program will reliably trap if it escapes the body or is not called before control flow proceeds, right?

An example would be an exactly once function which is used to restore memory to an initialized / defined state before the programmer can access it again. Consider an alternative Mutex implementation that returns a @once function instead of using withValue. Fortunately, we do make that promise with a non-escapable @once function, which depends on a mutating access on the value it wants to clean up. This is fine so long as composing withoutFoo utilities doesn't allow another mutating access before calling that function.

28

In this case any sound static analysis would prevent you from ever accessing the value again, because there would be no reachable point in the program where the value is subsequently known to be initialized.

29

I don't believe this is the case. Consider a span which takes a borrowing access of an array; there are many reachable points where we can prove the span is dead and the array may be mutated again.

30

There are still a large number of APIs that use a regular closure type. You can do interop with one of those using a wrapper like this:

var onceClosure: (@callable(once) () -> Void)? = { ... }
legacyAPI {
  onceClosure.take()!()
}

So there’s definitely a need to at least be able to convert @once functions to regular functions. withoutActuallyDeiniting is one way to fill that need.

31

This would have to be a non-escaping function to express this safety property, though. Even putting the array invalidation aside, it's capturing a non-escapable value (the span).

I think my original statement holds up. There are no safety properties that rely on rules like "X must happen, but I don't care when", which is good, because we can't prove that without throwing out Turing completeness. The safety properties we do have all rely on rules like "X must happen before Y", and the language tool we have for that[1] is tying X to a value that can't escape some scope S and then doing Y at the end of S.

  1. if you need to be more flexible than just having one piece of code that does both X and Y in order ↩︎

4 Likes
32

This is a good analogy, but not in the way I think you mean. withoutActuallyEscaping is not necessary because there are APIs that annotate their parameters incorrectly — I'm sure there are, but that's not the primary motivation. It is necessary because there are APIs that correctly do not escape their parameters, but have implementations that can't prove that to the compiler. withoutActuallyEscaping downgrades the static check to a dynamic one so that the API can state its external behavior precisely instead of being forced to say that the function escapes because of the limits of the compiler's analysis.

That is exactly why withoutActuallyDeiniting is also important. There are APIs that call their parameter function exactly once for which we have no reasonable hope of proving that automatically. By downgrading the static check to a dynamic one, withoutActuallyDeiniting allows the API to state its external contract more precisely.

An obvious example would be adapting a "split" completion handler into a unified one, like so:

func transmogrify(completion: @once (Result<String, Error>) -> ()) {
  // this must be rejected by the compiler
  transmogrifyImpl(onSuccess: { string in completion(.success(string)) },
                   onFailure: { error in completion(.failure(error)) })
}

completion will in fact be called exactly once, but proving this relies knowing that transmogrifyImpl calls exactly one of its argument functions. Maybe someday that's something that we'll have the ability to express, but it's not going to come out of the box in the first release — and no matter how complex the language support gets, you can always find another example that the language support won't be good enough for. Without a dynamic downgrade, transmogrify would have to declare completion not just as @callable(once) but actually as an ordinary, called-arbitrary-times function. But with a dynamic downgrade, it can internally do something like this:

func transmogrify(completion: @once (Result<String, Error>) -> ()) {
  withoutActuallyDeiniting(completion) { downgraded in
    var temp = Optional(downgraded)
    transmogrifyImpl(onSuccess: { string in temp.take()(.success(string)) },
                     onFailure: { error in temp.take()(.failure(error)) })
    // temp will always be nil here, so the assertion won't fire
  }
}
5 Likes
33

As mentioned, exactly once is the more common case, so how about @once and @once(atMost). Would @once(atLeast) make any sense?

Edit: @once can be considered shorthand for common case @once(exactly)