[Pitch] Formally defining consuming and nonconsuming argument type modifiers

Evolution Pitches
1

Hey everyone!

This is the consuming/nonconsuming argument type modifier pitch I was speaking about. The permanent URL is: swift-evolution/000b-consuming-nonconsuming.md at consuming-nonconsuming-pitch-v1 · gottesmm/swift-evolution · GitHub.

For convenience I also attached it inline below.

Formally defining consuming and nonconsuming argument type modifiers

Introduction

By default the Swift compiler uses simple heuristics to determine whether a
function takes ownership of its arguments. In some cases, these heuristics
result in compiled code that forces the caller or callee to insert unnecessary
copies and destroys. We propose new consuming and nonconsuming argument type
modifiers to allow developers to override said compiler heuristics and
explicitly chose the convention used by the compiler when writing performance
sensitive code.

Swift-evolution thread: Discussion thread topic for that proposal

Motivation

In Swift, all non-trivial function arguments use one of two conventions that
specify if a caller or callee function is responsible for managing the
argument's lifetime. The two conventions are:

  • consuming. The caller function is transferring ownership of a value to
    the callee function. The callee then becomes responsible for managing the
    lifetime of the value. Semantically, this is implemented by requiring the
    caller to emit an unbalanced retain upon the value that then must be balanced
    by a consuming operation in the callee. This unbalanced retain causes such an
    operation to be called "passing the argument at +1".

  • nonconsuming. The caller function is lending ownership of a value to the
    callee. The callee does not own the value and must retain the value to consume
    it (e.x.: passing as a consuming argument). A reference counted nonconsuming
    argument is called a "+0 argument" since it is passed without emitting an
    unbalanced retain (in contrast to a "+1 argument") and all retain/release pairs
    are properly balanced locally within the caller/callee rather than over the
    call boundary.

By default Swift chooses which convention to use based upon the function type of
the callee as well as the position of the argument in the callee's argument
list. Specifically:

  1. If a callee is an initializer, then an argument is always passed as
    consuming.

  2. If a callee is a setter, then an argument is passed as consuming if the
    argument is a non-self argument.

  3. Otherwise regardless of the callee type, an argument is always passed as
    nonconsuming.

Over all, these defaults been found to work well, but in performance sensitive
situations an API designer may need to customize these defaults to eliminate
unnecessary copies and destroys. Despite that need, today there does not exist
source stable Swift syntax for customizing those defaults.

Motivating Examples

Despite the lack of such source stable syntax, to support the stdlib, the
compiler has for some time provided underscored, source unstable keywords that
allowed stdlib authors to override the default conventions:

  1. __shared. This is equivalent to nonconsuming.
  2. __owned. This is equivalent to consuming.
  3. __consuming. This is used to have methods take self as a consuming argument.

Here are some examples of situations where developers have found it necessary to
use these underscored attributes to eliminate overhead caused by using the
default conventions:

  • Passing a non-consuming argument to an initializer or setter if one is going
    to consume a value derived from the argument instead of the argument itself.

    1. String initializer for Substring. This API uses the underscored API __shared since semantically the author's want to create a String that is a copy of the substring. Since the Substring itself is not being consumed, without __shared we would have additional ref count traffic.
    extension String {
  /// Creates a new string from the given substring.
  ///
  /// - Parameter substring: A substring to convert to a standalone `String`
  ///   instance.
  ///
  /// - Complexity: O(*n*), where *n* is the length of `substring`.
  @inlinable
  public init(_ substring: __shared Substring) {
    self = String._fromSubstring(substring)
  }
}
    1. Initializing a cryptographic algorithm state by accumulating over a collection. Example: ChaCha. In this case, the ChaCha state is initialized using the contents of the collection "key" rather than "key" itself. NOTE: One thing to keep in mind with this example is that the optimizer completely inlines away the iterator so even though we use an iterator here. This results in the optimizer eliminating all of the ARC traffic from the usage of the CollectionOf32BitLittleEndianIntegers.makeIterator() causing the only remaining ARC traffic to be related to key being passed as an argument. Hence if we did not use shared, we would have an unnecessary release in init.
    init<Key: Collection, Nonce: Collection>(key: __shared Key, nonce: Nonce, counter: UInt32) where Key.Element == UInt8, Nonce.Element == UInt8 {
    /* snip */
    var keyIterator = CollectionOf32BitLittleEndianIntegers(key).makeIterator()
    self._state.4 = keyIterator.next()!
    self._state.5 = keyIterator.next()!
    self._state.6 = keyIterator.next()!
    /* snip */
}

  • Passing a consuming argument to a normal function or method that isn't a
    setter but acts like a setter.

    1. Implementing append on a collection. Example: Array.append(_:). In this example, we want to forward the element directly into memory without inserting a retain, so we must use the underscored attribute __owned to change the default convention to be consuming.
    public mutating func append(_ newElement: __owned Element) {
  // Separating uniqueness check and capacity check allows hoisting the
  // uniqueness check out of a loop.
  _makeUniqueAndReserveCapacityIfNotUnique()
  let oldCount = _buffer.mutableCount
  _reserveCapacityAssumingUniqueBuffer(oldCount: oldCount)
  _appendElementAssumeUniqueAndCapacity(oldCount, newElement: newElement)
  _endMutation()
}
    1. Bridging APIs. Example: _bridgeAnythingNonVerbatimToObjectiveC(). In this case, we want to consume the object into its bridged representation so we do not have to copy when bridging.
    func _bridgeAnythingNonVerbatimToObjectiveC<T>(_ x: __owned T) -> AnyObject

  • Consuming self when calling a method that is not an initializer.

    1. Creating an iterator for a collection. Example: Collection.makeIterator(). The iterator needs to have a reference to self so to reduce ARC traffic, we pass self into makeIterator at +1.
    extension Collection where Iterator == IndexingIterator<Self> {
  /// Returns an iterator over the elements of the collection.
  @inlinable
  public __consuming func makeIterator() -> IndexingIterator<Self> {
    return IndexingIterator(_elements: self)
  }
}
    1. Sequence based algorithms that use iterators. Example: Sequence.filter(). In this case since we are using makeIterator, we need self to be __consuming.
      @inlinable
  public __consuming func filter(
    _ isIncluded: (Element) throws -> Bool
  ) rethrows -> [Element] {
    var result = ContiguousArray<Element>()

    var iterator = self.makeIterator()

    while let element = iterator.next() {
      if try isIncluded(element) {
        result.append(element)
      }
    }

    return Array(result)
  }

In all of the above cases, by using underscored attributes, authors changed the
default convention since it introduced extra copy/destroys.

Proposed solution

As mentioned in the previous section, the compiler already internally supports
these semantics in the guise of underscored, source unstable keywords __owned,
__shared and for self the keyword __consuming. We propose that we:

  1. Add two new keywords to the language: consuming and nonconsuming.

  2. Make consuming a synonym for __consuming when using __consuming to make
    self a +1 argument.

  3. On non-self arguments, make consuming a synonym for __owned and
    nonconsuming a synonym for __shared.

Detailed design

We propose formally modifying the Swift grammar as follows:

// consuming, nonconsuming for parameters
- type-annotation → : attributes? inout? type
+ type-annotation → : attributes? type-modifiers? type
+ type-modifiers → : type-modifier type-modifier*
+ type-modifier → : inout
+               → : consuming
+               → : nonconsuming
+

// consuming for self
+ declaration-modifier → : consuming

The only work that is required is to add support to the compiler for accepting
the new spellings mentioned (consuming and nonconsuming) for the underscored
variants of those keywords.

Source compatibility

Since we are just adding new spellings for things that already exist in the
compiler, this is additive and there isn't any source compatibility impact.

Effect on ABI stability

This will not effect the ABI of any existing language features since all uses
that already use __owned, __shared, and __consuming will work just as
before. Applying consuming, nonconsuming to function arguments will result
in ABI break to existing functions if the specified convention does not match
the default convention.

Effect on API resilience

Changing a argument from consuming to nonconsuming or vice versa is an
ABI-breaking change. Adding an annotation that matches the default convention
does not change the ABI.

Alternatives considered

We could reuse owned and shared and just remove the underscores. This was
viewed as confusing since shared is used in other contexts since shared can
mean a rust like "shared borrow" which is a much stronger condition than
nonconsuming is. Additionally, since we already will be using consuming to
handle +1 for self, for consistency it makes sense to also rename owned to
consuming.

Acknowledgments

Thanks to Robert Widmann for the original underscored implementation of
__owned and __shared: https://forums.swift.org/t/ownership-annotations/11276.

23 Likes
2

I believe __shared, __owned, and __consuming should be removed from the language in Swift 6, or whenever the stable equivalents are implemented.

1 Like
4

These are interesting.

Personally, I view them somewhat like branch hints - really, there is no way for the writer of Array.append to know what the calling code is going to do with the element after it has been added to the array; so we guess and talk about what is "likely" or "unlikely". If correct, maybe we can save a retain or release somewhere -- but if incorrect, we actually add ARC traffic.

FWIW, almost every time I've tried to use branch hints, it has gone badly. Maybe I'm just really bad at it, or maybe it's just a bad idea for library authors to guess what their callers are going to do. A poor-man's PGO.

I'd be interested to see data about the current heuristics, and whether they really amount to a significant win. Maybe they do. But maybe they do more harm than good.

It took me a little while to see what was going on here - in these cases, it seems like they are working around the heuristics. Now I'm looking at my own initializers (parsers, etc - which don't directly store the arguments they are given; they read from them and create their own data from it), and I finally understand why there are all kinds of unwanted retains around them :frowning:

I decide to write a function as an initializer as a matter of API design. I didn't know they had this hidden meaning attached to them. It's very surprising how subtle this is.

So since these are __owned (or consuming), then, assuming the caller has the value at +0, doesn't it have to perform an unbalanced retain? What's the benefit over just passing the value in at +0 and letting methods like append perform the retain, rather than their callers? And does it happen that often to make a significant difference?

Same sort of thing here. makeIterator is inlinable, and IndexingIterator is frozen and inlinable as well. In code which iterates a collection and does not escape the iterator, why can't the compiler see that, whilst IndexingIterator stores a copy of self (and hence needs a retain), the iterator gets destroyed while the original reference to self is still live and hence there is no lifetime extension here.

This is even more surprising, to be honest. So you're saying that simple functions in Collection extensions which iterate self need to add keywords to avoid unnecessary ARC? Even if they do not escape self or iterator? I'm rather disappointed to learn that. But it seems to be true:

func doSomething(_ dict: [Int: String]) -> Int {
    // callq   swift_bridgeObjectRetain@PLT <- inserted by compiler
    dict.keys.sum()
    // callq   swift_bridgeObjectRelease@PLT <- inserted by compiler
}

extension Collection where Element == Int {
    func sum() -> Int {
        var total = 0
        for value in self {
            total += value
        }
        return total
    }
}

I can move the release in to sum() by marking the function as __consuming, but how do I get rid of both the retain and release entirely, and use the value at +0 as it was given to me?

I don't know... the benefits just don't seem very obvious. I mean, they are probably necessary in light of the heuristics, but I'm sceptical whether the heuristics themselves are worth the complexity and secret ARC traffic.

As for spelling, I like @dabrahams comment in the manifesto thread about how consuming/nonconsuming are very similar to escaping/nonescaping. The difference between them seems to be that escaping/nonescaping is necessary for semantics (it is impossible to extend the value's lifetime, perhaps because it points to stack memory), while consuming/nonconsuming is just a hint.

I wonder if we could call consuming escaping(hint) and nonconsuming nonescaping(hint). Or something less ugly which also communicates the relationship with escaping/nonescaping arguments.

5

I think the proposal would be easier to understand if it could describe the exact semantics of consuming and nonconsuming without reference to the old underscored spellings. As its written now, someone who doesn’t know anything about the underscored spellings (i.e. most people who don’t work on the standard library) have to learn five keywords to grok the proposal instead of just two.

25 Likes
6

I concur: officially, unstable language features don’t exist.

7

Let me make sure I’m getting this right:

  • consuming call:
    1. Caller retain
    2. Call
    3. Callee release
  • nonconsuming call:
    1. Caller retain
    2. Call
    3. Callee retain x n
    4. Callee release x n
    5. Caller release

I’m a little unclear on why the compiler can’t identify consuming calls automatically.

I’m also unclear on how to determine which to use. It sounds like consuming is inherently more efficient, but that obviously can’t be right: if it was, it wouldn’t need to be specified.

8

This proposal is on purpose not taking a position on removing these. They at least will have to live some time into the future since they are printed into interface files. Regardless, such source unstable keywords are not part of the language so their existence/removal isn't relevant to the evolution of Swift the language (and thus Swift evolution). I also think there isn't much benefit to removing them in the short term since it is just some extra textual keywords in the stdlib and a small amount of code in the swift frontend to recognize those additional keywords.

9

A few things:

  1. The default heuristics can not be changed without a very significant ABI break across the language.
  2. Many years ago, Swift always passed arguments as +1. This was done since we found that in Objective-C ARC that often times we had extra retain/release in setters/inits compared to situations in manual retain, release. These cases came up if one was allocating a new object and then storing it into an ivar via a setter or init. Ideally one would just allocate the object, not retain the object and funnel it into memory. Before Swift's ABI was finalized, we realized that this was actively harmful for many simple function cases that were not setters/inits (e.x.: often times they could not be readonly due to the release if we didn't inline). Thus, we decided to change the ABI so that in the cases where we were not semantically initializing a field of a class (e.x.: a function vs a setter), we would change the signature to +0. We found that this reduced retains/releases the most and also fit with our experience from comparing Objective-C ARC with manual RR code.

So in that case, these type modifiers are exactly what you need to express your API design = p.

That being said, can we avoid using words like "shocking"? It adds a bit of pathos to the discussion that I would like to avoid (I appreciate your feelings though).

If the caller has the value at +0, then it has to retain the value. The callee does not retain/release the value and just forwards into memory. So there isn't a benefit in this case vs having the retain in the callee. That being said in the case where one already has a +1 value (e.x.: function return value, constructor), the benefit is significant since we avoid an additional release in the caller and a retain in the callee. So no benefit/no harm in the +0 case, but large benefit in the +1 case.

A few things:

  1. This is not just about the specific implementation in the stdlib. It is also about the protocol itself which is marked with these keywords.
  2. So with that in mind, we also have to consider the future possibility of having move only types and that we can not see the body of said routine since it isn't inlinable. So the compiler has nothing to reason about.

This is incorrect. A few things:

  1. When I look at this on ToT, I see that sum is not inlined into doSomething() and that the retains that you are talking about are on the dictionary, not in sum due to the iterator.
  2. This example to be optimized as aggressively as possible requires us to run the later Ownership SSA optimizations which are not enabled by default outside of the stdlib yet. These are enabled with the flag -Xfrontend -enable-ossa-modules.
  3. With that enables, there is also Ownership SSA mem2reg optimization that isn't coming into play here.

So this has nothing to do with the specific conventions here.

As I mentioned above, the heuristics are part of Swift's ABI so they aren't something that we can discuss without talking about a large ABI break that is so fundamental that I doubt it would be an acceptable proposition.

4 Likes
10

This is incorrect. I would view it as the following situations:

Consuming call:

  1. Caller retain (or forward an already retained value from the caller [e.x.: a just constructed object]).
  2. Call
  3. Callee releases, forwards into memory, or passes off the value as an additional consuming parameter.

Non consuming call:

  1. Call
  2. Callee retain if the object needs to be consumed due to forwarding into memory or passing off the value as a consuming parameter.

Consuming is not inherently more efficient. Let me give a specific example: consider a situation where one is passing a class to an function that will do something that is read only. Despite that, if the function takes its argument at +1 the function can not be read only since it has to call release on the argument. A release can invoke /any/ code due to potential associated objects. Thus, we are forced to mark the function as always having unknown side-effects. So it inherently causes many more functions to be forced to have side-effects when it isn't necessary.

7 Likes
11

Yes, today I learned that I'm going to have to change how I design APIs or scrutinise them very carefully for the heuristics.

I apologise for using the word "shocking" - I meant to say I'm very surprised (I don't mean anything mean-spirited by it). It's just one more very subtle thing, and I'm sure I won't always remember it (and others won't always remember it).

Okay - well, they are actually due to the iterator (@_assemblyVision shows that a Builtin.BridgeObject is retained and a _NativeDictionary<String, Int>.Iterator is released).

I tried out 3 versions of sum() using the OSSA modules flag - Godbolt.

Using a for loop, or the default makeIterator(), seems to cause a retain/release pair, whilst a custom makeIterator() (or using indexes) does not.

Adding __consuming to the custom makeIterator() doesn't add a retain/release pair though, so I guess it's off the hook. Maybe the flag won't apply OSSA optimisations to code which is inlined across modules or something.

Not even with thunks? :pleading_face:

12

So functions with consuming arguments can never be marked as free of side effects (not that that is a stable attribute anyway)? That is a serious downside, I agree.

Would it be possible to establish a (human-interpreted) heuristic for when to use these attributes? Is it as simple as never using them until you actually see undesired outcomes?

1 Like
13

If everyone will excuse a stupid question: can ABI breaks be smoothed over long-term by virtue of language modes? And why would the size of the ABI break actually matter?

14

To me it seems like whether something should be consuming/nonconsuming is pretty much correlated to whether it is escaping/nonescaping. Because if the parameter escapes, the function needs to retain it so it might as well tell the caller to pass it +1 (already retained), and if it does not escape it just needs to be alive for duration of the call so the caller passing it +0 is ideal.

So my thought about this is that perhaps escaping/nonescaping should imply consuming/nonconsuming. And if for extreme situations we really need to bypass this default we could have escaping(nonconsumng) and nonescaping(consuming) as overrides. Spelling it that way forces you to think about escaping first, and consumption becomes subordinate to that.

But the issue might be the migration path. It does not correlate right now, as everything is implicitly escaping today (except certain closures) but not everything is consuming.

5 Likes
15

What if we just called these owned and unowned? Seems to me this would most directly demonstrate that they're talking about the concept of ownership. Would that conflict at all with the existing usage of unowned captures? If so, how about retained and nonretained?

At least for me, I find the terminology about "consuming" to be actively working against intuition. In common usage, that which is consumed no longer exists, since to consume something is to eat it up or destroy it. But, as in the example where an element is appended to an array, it's precisely because the array wants to keep the value around that we want it to be "consuming" the element—this is the exact opposite of the natural meaning of the word.

By contrast, to "own" a thing suggests the correct reason why one might want to use this feature here, which is to keep the thing around.

I suspect one reason that some folks are surprised or alarmed at the already existing heuristics is how unnatural it sounds that an initializer or setter, for instance, "consumes" its arguments by default when clearly initializing or setting a property means you'd want to hang on to its value.

18 Likes
16

+1 to exposing this functionality as keywords in user-facing Swift.

As to the choice of keywords; I agree that consuming and nonconsuming are non-obvious, so I wonder about borrowed (for nonconsuming) and maybe kept (for consuming)? They’re asymmetrical but I think clearly identify e.g. that we’re keeping this value after the function returns vs. that we’re only temporarily borrowing a reference. (There’s also the option of lent as an alternative to borrowed but I think that’s probably brevity over clarity.)

17

if there is inout maybe in instead of nonconsuming ?
But in is used in for loop: for x in 0...10 or for name in names

Would it be possible to write something like this:
for name in nonconsuming names
or like this:
for nonconsuming name in names ?

Rust allows something like this:
fn immut_borrow(x: &i32){...}
and
for x in &values{...}
the & character can be used in fn and in for loop.

2 Likes
18

+1: This pitch's retain count has been increased.
This is a very welcome feature to optimize a program further.

The new keywords consuming and nonconsuming sounded fair at first sight, but:

I think I love this suggestion more, it's obvious what it does.
Or some other I came up:

  • copying, noncopying
  • retaining, releasing
  • releasing, nonreleasing
1 Like
19

They are related ideas but different. escaping/nonescaping is far more limiting and not a substitute for consuming/nonconsuming.

--

It isn't, and that alarm won't disappear with a name change. Essentially, if you have an initializer which takes a string:

struct Foo {
  init?(_ string: String) { ... }
}

The compiler just assumes you are going to be assigning that string to a stored property and will even insert a retain to give you a +1 string. That differs from how strings are passed as regular function parameters - the following gets a +0 string with no hidden inserted retain:

struct Foo {
  static func make(_ string: String) -> Foo? { ... }
}

Initializers instead need to opt-in to what is the default behaviour almost everywhere else in the language, meaning you need to:

(a) Know and remember that this is even happening. Is it obvious from looking at the init signature?
(b) Add a special keyword to opt-out of the heuristic which inserts the extra retain.

That's what's alarming. The name of keyword isn't the significant thing.

It's in the ABI so can't easily be changed. But I wonder if we can change the default in a language mode, so for example, in Swift 6, all parameters to initializers would be implicitly nonconsuming/__shared (+0, just like regular function arguments) and you'd need to opt-in to special behaviour.

2 Likes
20

Just to be clear, I'm not suggesting getting rid of consuming/nonconsuming, only that it mirroring escaping/nonescaping seems to be a better default (when left unspecified) instead of using ad-hoc rules about initializers and setters. Of course, if the escaping/nonescaping annotations are often wrong (if everything is always escaping like things are now) then using it as a default for consuming probably won't help.

  • escaping consuming for when the parameter always escapes, or does so most of the time
  • nonescaping nonconsuming for when the parameter does not escape and is likely to be used again by the caller
  • escaping nonconsuming for when the parameter might escape but is unlikely to do so
  • nonescaping consuming for when the parameter will not escape but is unlikely to be used again afterwards (allowing for early release in the middle of the function)

I think it's better if people think about escaping and consuming together instead of as two separate things, hence why I suggested this:

  • escaping — short for escaping(consuming)
  • nonescaping — short for nonescaping(nonconsuming)
  • escaping(nonconsuming)
  • nonescaping(consuming)

There's no technical reason for linking them together, but I think this link helps understand when to use consuming and nonconsuming. I think of it as some sort of progressive disclosure.

Perhaps we could make it even clearer about when to use consuming/nonconsuming with names expressing intent such as escaping(unlikely) — unlikely to escape would make it nonconsuming — but unfortunately I'm not sure what name would work with nonescaping.

21

How does the position of the argument in the argument list matter? The three heuristics sound like they only take into account the type of the function.

Does this apply to case constructors too? Or are arguments to case constructors passed as nonconsuming, per the third heuristic?