Noncopyable types and implicit conversions

Evolution Pitches
1

Swift performs implicit type conversions in a few situations for convenience. While this normally interacts fine with noncopyable types, ever since the introduction of noncopyable generics, there are a few places where it can be confusing:

protocol Fooable: ~Copyable {}
struct Foo: ~Copyable, Fooable {}

func test(_ v: borrowing Foo?) {}

func conversions() -> any Fooable & ~Copyable {
  let foo = Foo()
  test(foo) // implicit conversion `foo as Foo?`
  return foo // implicit conversion `foo as any Fooable & ~Copyable`
}

But, these conversions are consuming operations. So the program above has an error:

example.swift:7:7: error: 'foo' consumed more than once
 5 | 
 6 | func conversions() -> any Fooable & ~Copyable {
 7 |   let foo = Foo()
   |       `- error: 'foo' consumed more than once
 8 |   test(foo)
   |        `- note: consumed here
 9 |   return foo
   |          `- note: consumed again here
10 | }

I think most people would look at test and see that it's taking a borrowed argument, and become confused at how it's being consumed.

But, it's the caller conversions that is doing an implicit conversion of foo in order to pass it as an Optional, and conversions are consuming.

I'd like to hear what people think about this. Here are a few ideas to make this situation less confusing:

  1. Emit a warning when doing an implicit conversion, that is silenced by writing test(consume foo). For example:
example.swift:8:8: warning: implicit conversion to 'Foo?' is consuming
 6 | func conversions() -> any Fooable & ~Copyable {
 7 |   let foo = Foo()
 8 |   test(foo)
   |        |- warning: implicit conversion to 'Foo?' is consuming
   |        `- note: add 'consume' to silence this warning
 9 |   return foo
10 | }

The explicit consume would only be required in situations where the conversion is happening on a variable.

It's easy to avoid warning in situations like return foo or test(Foo()), as they are clearly last-use or not converting a variable. But, identifying last-uses in general requires control-flow analysis to avoid superfluous warnings like this:

func returnNothing() {
  let foo = Foo()
  // ...
  test(foo) // warning: implicit conversion to 'Foo?' is consuming
  return
}

I'll note that there is already a precedent for warnings about ambiguous things / common mistakes that can be silenced.

  1. The second approach is to make the implicit conversions, explicit. So, similar to (1), but instead of warning to add a consume, we can make it an warning (or perhaps an error) that you need to write the cast explicitly, like test(foo as Foo?), test(.some(foo)).
6 Likes
2

I’m glad this is being brought up, but also I wonder how many of these conversions can be done as borrowing through some kind of trickery. :-/ borrowing Foo? not being able to borrow from Foo is a real loss. borrowing Superclass not being able to borrow from Subclass would be ridiculous (fortunately that one doesn’t require a format change). borrowing any Bar not being able to borrow from BarImpl is a bit more obvious than the Optional case, but still not ideal.

I do think requiring consume is reasonable for the current period while these will actually do a consume, but this was not an area I would have anticipated needing to change how I design function signatures to accommodate both presence and absence.

(Rust can represent this explicitly with Option<&Foo>, but Swift doesn’t do that.)

7 Likes
3
Trickery

Optional is special. It has a fragile layout involving exactly one “extra inhabitant”, and we take advantage of that elsewhere. For any address-only type, that means borrowing Optional<Foo> will have the same representation as borrowing Foo, and similarly for any fully-fragile type with a statically-known extra inhabitant. So the worrisome case is a type that doesn’t have extra inhabitants. In that case, borrowing Optional<Foo> might be passed directly, in which case this is easy enough—we know where the flag for validity is being passed, and therefore the compiler can conjure it. The indirect case, however, is a problem, since the whole point is that there is no Optional<Foo> in memory anywhere.

EDIT: I'm wrong about address-only, Foo being address-only means the entire Optional<Foo> also becomes address-only, so it's the same as the "decided to pass indirectly" case.

In this last case, if we declared that an indirect value of type Optional<Foo> always pushed the indirection inside the Optional, we’d be fine. However, this would be ABI-breaking (unfortunately! that might have been superior to what we actually do!). It also doesn’t cover multiple levels of Optional injection, but that’s quite rare in practice, so maybe that’s okay.

(Also, if we can get this promotion to work at call boundaries but not necessarily for locals, I think that’s still a win.)

At this point I’m stuck, and I’m fairly sure I missed something in this analysis, but I hope it’s enough to be worth pursuing…

1 Like
4

Could we instead convert these implicit conversions for ~Copyable types into something like this:

func conversions() -> any Fooable & ~Copyable {
  var foo = Foo()
  var optionalFoo = Optional<Foo>.some(consume foo)
  test(optionalFoo) 
  foo = consume optionalFoo!
  return foo // implicit conversion `foo as any Fooable & ~Copyable`
}

But even this feels a bit to restrictive. Can conversions not be made borrowing? The resulting value need to depend on the value and no consuming operations can happen in the meantime but that sounds like what ~Escapable could do for us.

@jrose what does Option<&Foo> mean in Rust?

1 Like
5

Roughly Optional<borrowing Foo>, a thing not expressible in Swift-the-surface-language.

(And the consume/unconsume only works if the Foo you have initially is owned; if you’ve borrowed it from somewhere else, you’re still in trouble.)

5 Likes
6

I think this is a very reasonable & pragmatic first step, at least. Is there any reason that couldn't be added relatively easily & promptly?

Of course, ideally you'd simply be able to do the obvious thing: be able to wrap any borrowed thing into an Optional and pass that around, subject to normal rules about lifetimes / escaping.

That you can write this manually is what makes the compiler warning viable. But one can intuit that it'll get old fast if you have to write that out often.

1 Like
7

Are the escapability proposals introducing the @dependsOn(…) attribute useful here by any chance?

1 Like
8

I don’t know what you’re specifically thinking of, but probably not: the problem isn’t the current borrower; it’s whoever else may also be borrowing the value at the moment higher up the call stack, or possibly even on other threads.

2 Likes
9

Can you spell out why test(borrow foo) can’t be made to work? I’m not following the implication. I would expect this to create a borrow of foo, and then the conversion to create a new anonymous value of type Foo? whose lifetime depends on foo, and then the call to test would borrow from the anonymous value. In fact, couldn’t I do this manually with a thunk?

10

If your non-copyable value is a mutex, that would move the mutex (because Optional<Mutex<Whatever>> is a perfectly valid type that stores a mutex within itself).

11

Is the problem is that this might change the location in memory of the value? I thought non-copyable types were insufficient to address this, requiring the use of the @_staticExclusiveOnly attribute. Otherwise a non-copyable Int couldn’t be stored in a register.

12

All of this is about implementation rather than interface, and yeah, borrowing in Swift doesn’t necessarily mean “in memory”. The trouble is, it doesn’t necessarily come out to “bitwise copy, but treat as derived from the original value” either. That’s sufficient for trivial types, and it’s sufficient for object references, but it doesn’t cover “address-only types”. This affects generic types, and resilient types, but the concrete way this can go wrong even without the newfangled “Mutex” is ObjC weak references, which have to be registered on every copy. So now we’re no longer talking about passing something in registers, or copying with memcpy; we’re running arbitrary code to do a “borrow” operation, which kind of defeats the point.

But you make a good point that this part shouldn’t be a problem for anything that isn’t address-only. And constraining the problem might be enough to solve it piecemeal in the compiler!

4 Likes
13

Yeah, that transformation does work for simple cases, but in general it won't work if the original value is borrowed as two different types, since it can't exist as both as the same time:

extension Foo { borrowing func getAnswer() -> Bool { ... } }
func query(_ foo: borrowing Foo?, _ generate: () -> Bool) -> Bool { ... } 

var foo = Foo()
query(foo) {
  return foo.getAnswer()
}

This example makes it less obvious, but foo is borrowed as a Foo? in the argument, and as a Foo as captured in the closure. It's suppose to valid to have multiple read accesses at the same time. For Copyable types, this is implemented by copying foo to do the cast.

A general borrowed-cast mechanism, or borrowed types, would be needed to fix this. I think we one day could implement a borrow x as Foo cast expression that yields a ~Escapable result whose lifetime depends on x.

Like borrowing switches, it'd be nice if the casts were borrowing like that by default, if the original type of the cast is noncopyable. It would only be consuming if you wrote consume x as Foo. So for the more immediate timeframe, it seems like it makes sense to say that you need to write consume explicitly until then.

I'm working on a PR that so far takes care of implicit optional casts; seems to work ok. My hope is to get at least a warning into Swift 6.

2 Likes
14

@_staticExclusiveOnly isn't meant for any of this, it's meant to literally prevent users from declaring vars with a type that is static exclusive. var potentially brings baggage (depending where you declare it) that we do not want for Atomic or Mutex.

No, this is not the case today. Noncopyable types today don't really change the convention of how a type is passed. One storing an integer for example can still be put in a register.

1 Like
15

let isn’t static, so that name makes no sense.

If @_staticExclusiveOnly isn’t responsible for the address-only behavior of Atomic you described, what surface-level element of the language is?

16

Well it is an underscored attribute, so names don't really matter here :smile:

It is yet another underscored attribute @_rawLayout

1 Like
17

I don’t see how we can claim to have a complete story for move-only types if the things necessary to implement useful move-only types are underscored and poorly-named.

1 Like