~Copyable and functions

Development Compiler
1

I've been experimenting with what the compiler allows with regards to ~Copyable and I think I may be beginning to understand. The below is Swift 5.9.2 in Xcode on an MBP.

I got interested in the isomorphism between (T) -> T and (inout T) -> Void after a recent forum discussion and as an exercise decided to explore consuming and borrowing. The following code compiles cleanly:

func transform<T>(_ f: @escaping (T) -> T ) -> (consuming T) -> T {
    { t in f(t) }
}

Shouldn't the compiler be operating under the restrictive assumption that t is ~Copyable there (even though at the call site it may not be) and refuse to allow a call to f(t) in transform?

It's possible to get this to eventually generate errors in a couple of different ways, the following still compiles cleanly:

func identity(value: Any) -> Any { value }

struct Ref<T>: ~Copyable { var value: T }

var iRef = Ref(value: 13)
let consumingIdentity = transform(identity)

But the next steps one might take generate errors that don't lead you back to the root problem.

type(of: consumingIdentity)

causes: error: couldn't IRGen expression. Please enable the expression log by running "log enable lldb expr", then run the failing expression again, and file a bug report with the log output.

and:

let iRef2 = consumingIdentity(iRef)

causes: Noncopyable type 'Ref' cannot be erased to copyable existential type 'Any'

Am I getting this right conceptually?

1 Like
2

No, consuming is still a valid operation on copyable types so there's no need for it to try and infer ~Copyable here. The real bug is the fact that it even allowed you to pass a Ref to transform(identity) when Ref isn't copyable.

1 Like
3

True, and maybe (probably) I'm missing something, but it seems to me that in this situation t may be noncopyable and f demands it be copyable which I would have thought should be disallowed at that call site.

Playing with it some more, if I try to implement this directly without the higher order func:

func nominalConsumingIdentity(t: consuming Ref<Int>) -> Any {
    identity(value: t)
}

It produces the error above: Noncopyable type 'Ref' cannot be erased to copyable existential type 'Any' which I read to be saying that the init on Any requires a copyable type which Ref<Int> is not. This makes sense but it seems to me that the error could (should) have been clear in the generic.

4

The issue here is that T was already implicitly constrained to be copyable by transform. So, f can never take a non-copyable type. Nor can t ever be noncopyable since it's a T.

2 Likes
5

Ah.... but at this point of course I can't mark T on transform to be ~Copyable. So the compiler is letting me write checks it can't cash. And its only discovering this after it has specialized transform in this case.

6

Hmmm... ok so what does this bug look like when fixed? My thought was that it shouldn't compile transform because the returned function is not well-typed (internally calls a copyable-requiring function while marking the returned function as safe for ~copyable types), but IIUC you are saying that it shouldn't compile consumingIdentity(iRef) even though the signature of that function reads as being consuming

7

Here, transform expects T to be copyable. You have to declare it explicitly as ~Copyable for T to accept a non-copyable type:

func transform<T: ~Copyable>(_ f: @escaping (T) -> T ) -> (consuming T) -> T {
    { t in f(t) }
}

This feature is not ready and not in any release yet however. There is a pitch, but no evolution proposal yet. Try a nightly build with flags -enable-experimental-feature NoncopyableGenerics if you want to play with it. You can also try with godbolt.

3 Likes
8

yes. The confusion (from me) stems from whether this should be a compile error and if so what should it say. In this case it still seems (again to me, apparently not to others) that the compiler should tell me that either:

  1. T must be marked ~Copyable or
  2. the signature of transform's returned function must change to accept an implicitly Copyable T or
  3. transform must not invoke f with t or I suppose
  4. the signature of f must change to consuming or borrowing.

for this code to be valid. Given that 1 is not in the language yet, I would have expected it to tell me 2 or 3 and my expectation when writing this code was 3. I was a little surprised that it did not.

9
  1. T doesn't need to be marked ~Copyable because consuming a copyable value is a perfectly legitimate operation.
  2. transform already accepts an implicitly copyable T, because generic type parameters are always implicitly copyable unless they can be inferred to be ~Copyable due to a protocol conformance or associated type requirement (although as @michelf said, this isn't implemented yet).
  3. transform absolutely can invoke f with t. I'm not sure why you thought it shouldn't.
  4. consuming is fine here, you can consume a copyable type.

The only error is that T can't be a Ref because T is implicitly constrained to be copyable.

1 Like
10

The call transform(identity) should definitely be ill-formed, because transform requires T to be a copyable type. When we allow non-copyable generics, you will be able to opt transform in to supporting non-copyable type arguments, and then that call would become well-formed.

2 Likes
11

I don't understand, identity also requires T to be copyable, and I don't know why (consuming T) -> T woudn't be valid for a copyable type. AIUI this:

print(transform(identity)("Hello, world!"))

Should pass the string into the closure as a consuming parameter, copy it into identity, copy it out as the result (return values optimization notwithstanding) then print it.

What am I missing here?

1 Like
12

Yes, you're right. I'm sorry, I'm not sure what I was thinking when I posted that. There's no reason this shouldn't work, at least until you try to call either identity or consumingIdentity with a Ref, which correctly fails.

I don't think we're tracking that failure about type(of:) with functions with consuming parameters (which I assume is the problem); please file a bug.

3 Likes
13

So thinking about it more, here's my confusion:

func copy(_ anInt: Int) -> Int { anInt }

func f(a: Int, b: inout Int, c: consuming Int, d: borrowing Int) -> Int {
    b = a
//    a = b  // Cannot assign to value: 'a' is a 'let' constant
//    b = d  // 'd' is borrowed and cannot be consumed
    b = copy(c)
//    b = c    // 'c' consumed more than once
    return b
}

The call to copy consumed c without having its argument marked as consuming. Intuitively, I would have thought that (in the typing context of f) c is of "type" consuming Int and that any function invocation passing such a noncopyable argument (c in this case) should therefore be required to be marked consuming in order to assure the safety of the invocation.

what had me confused through this entire thread is that using generics (as in the OP) I'm able to defer what seemed to me to be the error (calling a function allowing copying arguments with a non-copyable argument) until much further along in the compilation.

Am I right in deducing from the above that in this case the compiler promoted copy to be consuming in the same way that non-throwing, non-async functions can be promoted into throwing or async when passed as arguments?

14

My previous assumption was that all functions that consume their arguments must mark the consumed args as consuming, but here copy clearly consumes c and its argument is not marked as consuming, so that assumption was wrong. The problem is that I'm not sure what the rule about consuming arguments is now.

15

The issue here is that marking a parameter as explicitly consuming or borrowing turns off implicit copying for that parameter (which I had forgotten about). The reason it works in the previous case is that we only use the paramter once, so the lack of copying doesn't matter.

// anInt is passed by value, so as long as it lives until the function ends, it doesn't care if the caller `consumed` or copied it in.
func copy(_ anInt: Int) -> Int { anInt }

func f(a: Int, b: inout Int, c: consuming Int, d: borrowing Int) -> Int {
    b = a       // a = b  // Cannot assign to value: 'a' is a 'let' constant
    b = d       // 'd' is borrowed and cannot be consumed
    b = copy(c) // c isn't copied, so it's lifetime ends after we pass it to `copy`.
    b = c       // 'c' consumed more than once
    return b
}

There's two ways to fix this.

  1. Explicitly copy everything.
    func f(a: Int, b: inout Int, c: consuming Int, d: borrowing Int) -> Int {
    b = a
    b = copy d 
    b = copy(copy c)
    b = c 
    return b
}
  2. Make copy(_:) guarantee that it doesn't consume it's input (you'll still need an explicit copy on d though):
    func copy(_ anInt: borrowing Int) -> Int { copy anInt }
16

or:

  1. func copy(_ anInt: consuming Int) -> Int { anInt }

which was what my intuition told me to do. In:

func f(a: Int, b: inout Int, c: consuming Int, d: borrowing Int) -> Int {
    b = copy(c)
//    b = c    // 'c' consumed more than once
    return b
}

I'm still surprised that the compiler doesn't complain that I'm invoking a func (copy) that requires its argumen to be copyable with an argument that is explicitly non-copyable at that call site.

I see how the various solutions here get things to compile, and I see that the compiler allows it... but if the rule is that:

func copy(_ anInt: Int) -> Int { anInt }

requires that any argument passed to it be copyable, then in f we're breaking that rule bc c has had copyability explicitly turned off in the function signature.

So clearly I need to internalize some other rule. I just don't have a clean statement of what the rule is.

17

i.e. That is still what bugs me and leads to my surprise at the behavior.

18

This function isn't a copy though, it's the functional equivilant of the consume operator.

copy isn't what consumed c, the consumption happened in f. And c is copyable, it's just doesn't happen implicitly.

Let me break down what happens when you pass a consuming parameter to a normal by value function parameter. This:

func bar(_ n: Int) {
    print(n)
}

func foo(arg: consuming Int) {
    arg += 1
    bar(arg)
}

Is the same as this:

func foo(arg: consuming Int) {
    arg += 1
    bar(consume arg)
}

The consume doesn't happen inside bar, it happens inside foo because it's not allowed to copy arg unless you use the copy operator, but bar isn't borrowing the parameter. But arg is still copyable. If it weren't, then you couldn't even use the copy operator.

For instance:

struct Unique: ~Copyable { var value: Int }

do {
    let x = Unique(value: 42)
    let y = copy x // error: 'copy' cannot be applied to noncopyable types
}

consuming and borrowing parameters are still copyable, they don't change the type of the parameter at all. All they do is specifiy who owns the parameter. Disabling implicit copying just means that the copys are no longer automatic, it doesn't mean they aren't allowed.

1 Like
19

Ok, so this seems sort of important: Transitive No Implicit Copy Constraint was excluded in SE-377. Unfortunately for me I suppose, that is actually the approach that seems most intuitive. Not sure that I agree with this rationale:

we think those additional restrictions would only make the borrowing and consuming modifiers harder to adopt, since developers would only be able to use them in cases where they can introduce them bottom-up from leaf functions.

But it is what it is..

20

It's the implicit insertion of consume in bar(consume arg) that I did not understand. Still finding it somewhat, well, magical and inconsistent with how, for example inout works.