How is __shared keyword supposed to work in Swift 5.0?

Hi all! Super excited about Swift 5.0 in the new Xcode 10.2 Beta and very happy to discover that __shared keyword no longer triggers asserts in the compiler, like it did with 5.0 snapshots previously. But I'm still trying to understand its intended behavior, I'm assuming that the Ownership Manifesto is still the best description available for this?

I've previously worked with Rust and this seemed to make perfect sense to me:

(Readers familiar with Rust will see many similarities between shared values and Rust's concept of an immutable borrow.)

But now I'm trying to wrap my head around this code in Swift 5.0 and I'm not sure why does it even compile, shouldn't it trigger the Law of Exclusivity violation at compile time?

func f(sh: __shared Int, io: inout Int) {
  io += 1
  print(sh)
}

var x = 5

f(sh: x, io: &x)

But even at run time this prints 5, not 6 as I'd expect if __shared prevents it from creating a copy. Does it mean that I misunderstand the intended use of __shared? Or another possibility I'm thinking, is __shared keyword currently only "decorative" to annotate the code for ABI stability purposes and there's no real borrow checker available for Swift 5.0?

Hope this can be clarified, thanks!

2 Likes

The basic intuition to go with is that __shared acts like a borrow, but that every type is still copyable (where "copying" a reference means retaining the referent). What that means is that __shared vs. __owned is mostly distinguishing who destroys a value at this point (the caller or the callee), rather than where copies occur. You can observe the different destruction behavior (messily) with the following code:

import Foundation

class BacktraceMe {
  var name: String

  init(name: String) {
    self.name = name
  }

  deinit {
    print("Destroying \(name)")
    Thread.callStackSymbols.forEach { print($0) }
    print()
  }
}

func testOwnership(owned: __owned BacktraceMe, shared: __shared BacktraceMe) {
  print("in test")
}

print("start")
testOwnership(
  owned: BacktraceMe(name: "owned"),
  shared: BacktraceMe(name: "shared"))
print("end")
2 Likes

Note that __shared is the default for most parameters anyway. Most of the benefit one would see is switching __shared to __owned when the parameter is going to be stored somewhere past the end of the function. In that case, taking the value as __owned avoids having to copy it into the storage; it can just be moved there instead.

8 Likes

Great, thanks. From the Ownership Manifest I got an impression that __owned is actually the default:

  • Pass-by-value, owned. This is the rule for ordinary arguments. There is no way to spell this explicitly.

Is that a wrong context for this then or should the Ownership Manifesto itself be updated to reflect the actual state of things?

Whoops, it should be updated. It was true at the time it was written, but that changed back in Swift 4.2. Maybe earlier, even.

(@John_McCall confirm/deny?)

1 Like

How do you compile that sample on your machine? Playgrounds in Xcode 10.2 is broken for me. I tried it in a simple command line tool, but that errors out when I try to run the app with dyld: Library not loaded: @rpath/libswiftCore.dylib. I set the project to always embed swift stdlib into the project, still no luck.

Yeah, Playgrounds and REPL are broken for me too, I had to create an empty macOS CLI project to try this out :disappointed:

Do you configure it somehow specific? I'm on the latest macOS version (non-beta) but I'm out of luck making it run.

No, nothing specific, just a default plain empty macOS command-line project from a template in Xcode 10.2 Beta on latest stable Mojave (apparently Xcode 10.2 isn't ever going to support High Sierra)

That's strange, I seem unable to compile even a newly created and empty macOS command line app with no settings modified in Xcode 10.2. :face_with_raised_eyebrow: I could try the above example in a Cocoa project thought, but still this is a little painful to be honest.

I'll try later if I can use a Swift 5 snapshot with Xcode 10.1.

@jrose Confirm. I haven't been updating the ownership manifesto as changes go in, but that's probably a good idea.

4 Likes

The __shared keyword is simply not the same thing as "Shared values" explained in the Ownership Manifesto. It does not change the argument passing semantics and is not analagous to Rust's immutable borrow. Your argument is still passed by value. That's the reason you don't see any exclusivity violation.

@John_McCall , if the Ownership Manifesto is updated it should really say that __shared does not implement shared values.

__shared is really an ABI convention that tells the compiler to destroy references on the caller vs. callee side. It's nothing more than that.

The terminology will need to be straightened out before we expose either __shared or move only types as public features.

5 Likes

You’re encountering this known issue:

Swift command line projects crash on launch with “dyld: Library not loaded” errors. (46824656) Workaround: Add a user-defined build setting SWIFT_FORCE_STATIC_LINK_STDLIB=YES .

2 Likes

The implicit copying behavior seems to me like it's independent of the argument passing semantics. We copy when we have to because, unlike Rust, our model introduces implicit copies where necessary to satisfy the conventions of the operations being performed instead of raising borrow check errors. When the argument to a shared parameter is sharably borrowable without copying, then we can and in some situations do share the value. The fact that it's passed by value is an optimization allowed because most of our types are address-independent and can be moved around freely in memory; a shared class reference still represents at the semantic level shared ownership of a single strong reference to the underlying object, for instance. If we added move-only types, or the ability to opt in to requiring explicit copies, then it seems like the only thing that changes is that the implicit copies become errors if they can't be avoided. The convention would otherwise be identical.

2 Likes

@Joe_Groff, @jrose

The implicit copying behavior seems to me like it's independent of the argument passing semantics.

No, the copying behavior changes semantics as the example above makes obvious.

fact that it's passed by value is an optimization allowed

No, no. The argument absolutely must be passed by value per the language semantics. Passing by reference in the above code would print the wrong value!

The OP's question is why doesn't Swift __shared have the semantics of Rust's immutable borrow. The answer is that was not designed to. Literally the only thing it does is force a +0 calling convention at the ABI level. The source of confusion is that Swift's terminology and documentation are currently misleading.

The __shared keyword does not today and cannot ever guarantee that the argument won't be copied. Nor will __shared ever be sufficient to enforce borrowed value semantics (We aren't going to break the code above in some future release).

The only way we will ever get the semantics that Max expects for copyable types is by introducing a new keyword.

I am strongly opposed to publicly documenting the __shared keyword. It will cause unending confusion with the new keyword that we introduce to actually do whatever people now think __shared does. We can call it unowned and document that it is the required convention for borrowed values, but otherwise has absolutely no semantic effect and certainly does not provide pass-by-reference semantics.

2 Likes

Which example? If you're referring to @Max_Desiatov's OP, then it indicates that the caller must copy x in order to satisfy the constraints of f's interface.

But you can't pass by reference in this case, because it would violate the borrow model. The code would be illegal if x were move-only. The fact that x is copyable is what allows the caller to insert the copies necessary to form the call. That's independent of whether the callee takes its arguments shared.

Please do!

I'm a normal Swift user trying to understand the semantics of this code--does the code compile, and what value is printed? What happens on the caller or callee side at run time is neither part of the language semantics nor relevant to me.

func f(sh: shared Int, io: inout Int) {
  io += 1
  print(sh)
}

var x = 5
f(sh: x, io: &x)

The possiblities are:

  1. The shared argument is an immutable borrow and the code is illegal.

  2. The shared argument has pass-by-reference semantics.

  3. shared does not affect program semantics for any code that I can write today.

First I read Swift's documentation. That leads me to the first possiblity. __shared -> "shared values" -> "immutable borrow". But the code does compile. What could this mean?

The only reasonable assumption now is that __shared provides pass-by-reference semantics, what else could it possibly mean? After all, language designers just told me that:

  • __shared does affect semantics by suppressing copies!
  • "__shared acts like a borrow"!
  • "passed by value is [just] an optimization"!

How could this be more misleading?

What language designers should do about this:

Surface __shared as unowned. Document that it is the inverse of owned which allows user control over caller vs. callee ownership of references, that it has no effect on program semantics, that it does not enforce exclusivity (which would be clearly implied by "shared"), and that value types are passed by value either way.

I did try to nip this in the bud last year:

Back to the OP:

But even at run time this prints 5, not 6 as I'd expect if __shared prevents it from creating a copy. Does it mean that I misunderstand the intended use of __shared?

Swift language semantics require that the argument be copied. You were being mislead.

Or another possibility I'm thinking, is __shared keyword currently only "decorative" to annotate the code for ABI stability purposes and there's no real borrow checker available for Swift 5.0?

@Max_Desiatov You are absolutely correct. Congratulations for seeing throught the nonsense even though you had to reverse engineer the true behavior of the language.

3 Likes

Official releases from Apple are built with assertions off to speed up compile times. Also the beta 1 release was branched from the 5.0 branch a few weeks ago. Since no work was done on __shared in a while, do you mind re-testing with the latest 5.0 snapshot and filing bugs?

I think we're in violent agreement about what the documentation ought to say about the language feature today. The issues I have are with the future impact onthe model once we have move-only types. I think __shared can and should be both the calling convention feature we have today and the semantic borrow control we have for move-only values in the future, and that these aren't separate contradictory features. Nonetheless a lot of the confusion and contradictory statements we’re making are probably arising because we aren’t being precise in separating the behavior that applies to Swift today from what we think will apply to future language features.

I don't think that's true. What happens on the caller side is all about language semantics. Part of our goal here AIUI is that, by default, users don't have to think about borrowing or copies in Swift unless they use language features that require these be made explicit.

#2 is an implementation detail. #1 is separable into two claims—"the shared argument is an immutable borrow" is true today; "the code is illegal" is not true, because the language semantics allow for implicit copies to be introduced to make borrows legal.

None of this is an "official" finalized language feature yet, so there isn't really an finalized statement one way or another. We don't have move-only types yet, and we don't have any language mechanisms for suppressing implicit copies, so nobody should be saying that "__shared does affect semantics by suppressing copies", because there's no way to suppress copies today. If we say that somewhere, it's wrong. If there is a means to suppress copies in the future, it ought to be an orthogonal concern to the calling convention.

I think it's important we avoid using the terms "pass by value" and "pass by reference", since we've tried to make sure that for most types in our model addresses are semantically irrelevant. A shared borrow of most Swift values can be passed either by value or reference at the machine level. Other than that, I totally agree with your statement. (We may still want a parenthetical note for clients such as the standard library that are planning for forward compatibility with future move only types about the effect the convention will have on values that don't admit implicit copying.)

1 Like