I’m sympathetic to @KeithBauerANZ’s views here about this not quite fitting right or feeling like a cohesive story. My take is that it’s purely a syntax issue and a result of this proposal being introduced before we have local reference bindings as described in the future directions.
I think from a user perspective local reference bindings are the more fundamental building block and the types described here would be rarely-used convenience wrappers. What I’d naturally/naively expect would be having a borrowing accessor return a borrowed T, and that being a distinct type from a mutable T returned from a mutating accessor, with the accessor called depending on the type of the variable (as if borrowed let x: T were equivalent to and followed the same overload resolution as let x: Borrowed<T> in this proposal). Then, if I want to be able to rebind the reference as an advanced feature, the e.g. Borrowed<T> type could be used as a wrapper. (I’d also expect implicit conversion of mutable T to borrowed T, and then borrowed T to just plain T when T: Copyable, though I admit I haven’t given much though as to how that would interact with generics). That’s pretty much the same semantics as what’s in this proposal; it’s just the types look different, you wouldn’t need to use .value, and Borrowed<T> would build on borrowed T rather than the other way around. (Also, as is probably obvious, I agree that Borrowed and Mutable are better names for the explicit types than Borrow and Inout.)
I've spent a lot of time reading everything that goes on in this forum and for a few years was even following every individual PR being merged into the Swift compiler.
I have the same opinion as @KeithBauerANZ , I don't feel the design space has been explored, I don't feel the evolution process is working as intended (rushed, feedback is often ignored or brushed away, design space is mostly unexplored and disregarded when people ask questions).
I've also started using Rust and I do find it much simpler, whether low-level or high-level. I still mainly use Swift and do prefer Swift overall, but not as much anymore. Everything feels half-baked, I encounter compiler bugs and unfinished features regularly, diagnostics are unhelpful, etc. I'm the onboard Swift expert (and do make heavy use of these "advanced" concepts) at my company and the number of times I've had to tell coworkers that this is just a Swift issue is approaching the daily.
I'd like a thorough exploration of the design space and justification of why we're going this way. The fact Hylo has sprung up should be indication that there's clearly frustration in the community, but also love for the language. We don't want to leave Swift, we want to see it fixed.
Apologies, I haven't read the pitch thread, but I feel like this proposal creates a confusing disconnect between these new types and inout function parameters, which is a language feature. The proposal shows this example:
func updateTotal(in dictionary: inout [String: Int], for key: String,
with values: [Int]) {
// Project a key out of a dictionary once...
var entry = Inout(&dictionary[key, default: 0])
// ...and then repeatedly modify it, without repeatedly looking into the
// hash table
for value in values {
entry.value += value
}
}
Without the existence of reference vars, currently the only way get references is by using a function with an inout parameter. You might write the above example as follows:
func updateTotal(in dictionary: inout [String: Int], for key: String,
with values: [Int]) {
// Project a key out of a dictionary once...
{(_ entry: inout Int, _ values: [Int]) in
// ...and then repeatedly modify it, without repeatedly looking into the
// hash table
for value in values {
entry += value
}
}(&dictionary[key, default: 0], values)
}
But this is obviously rather unwieldy. Once a user has seen inout parameters, the natural expectation is to just do it like this:
func updateTotal(in dictionary: inout [String: Int], for key: String,
with values: [Int]) {
// Project a key out of a dictionary once...
var entry = &dictionary[key, default: 0]
// ...and then repeatedly modify it, without repeatedly looking into the
// hash table
for value in values {
entry += value
}
}
This doesn't work of course, but I feel like that's what we need it to be. The language already works this way with inout parameters, why can't we do it with vars? Or are inout parameters now one of those "regrets"?
That would be a natural future step once we have these types. I discuss that a bit as a future direction in the proposal. @Torust's comment aligns with my own feelings about this feature:
It's reasonable to ask why we wouldn't start first with local reference bindings if we think that's ultimately the feature we expect developers to reach for once it's available. As discussed in the proposal, I don't think local reference bindings, or further generalization of reference bindings into other places in the language, would ever be able to fully supplant the usefulness of having explicit types for references; sometimes you do want the reference to be a variable in and of itself so you can update it in a traversal loop. The types also "just work" as generic arguments without requiring deeper tampering with the type system. With these types established, we can also explain how local bindings work in the future in terms of sugar over these types. Providing these types now gives our developers as much expressivity as they will ever get from future binding features, without making the community wait on us to provide those features; and even with fully-developed binding features, these types will have their uses.
I think Borrow is good. I also agree that Borrowed is a better spelling.
But Inout makes a critical mistake (and so does MutableSpan). They go against Swift's established conventions for reference and pointer types.
Existing convention is that for types with reference semantics, the value is mutable if the type allows it. But they can only be reassigned if they stored in a var or passed inout.
UnsafeMutable[...]Pointer types simulate reference semantics using nonmutating accessors.
Inout and MutatingSpan are effectively references like UnsafeMutablePointer, but they still behave as if they have value semantics. If you want to mutate the value they reference, you must also be able to reassign their value, even if it makes no sense.
It boils down to mutation and reassignment being equivalent operations for types with value semantics, but distinct operations for types with reference semantics.
This is a known limitation. The fundamental requirement is not that you be able to reassign the reference value, but that you have exclusive ownership of the reference while mutating through it. Unfortunately, inout/mutating is the only tool the language currently provides to assert exclusive ownership. We have been exploring features to allow exclusivity to be asserted without requiring mutability, and Inout (or whatever we decide to name it) would benefit from that as much as MutableSpan would.
UnsafeMutablePointer and class-based reference types do not have this problem, since they defer exclusivity to either programmer responsibility (in the case of UnsafeMutablePointer) or dynamic exclusivity checks (in the case of classes). By contrast, Inout and MutableSpan are intended to be zero-overhead and fully statically checked.
Ah, I see. In that case I think it'd be better to propose Inout separately, since it will need to be updated to use ExclusivelyCopyable or whatever other solution is proposed. I don't really see the point in adding it now if we know its' semantics will have to be fundamentally altered at some point in the future.
IMO, a lot of the contention goes back to the fact that even though these types have binding-adjacent names, they are actually safe counterparts to UnsafePointer. Even when considering swift-evolution’s storied tapestry of bikeshedding, I think that the names are creating an unusual amount of confusion and frustration. Safe pointers are a great idea, but giving them names that are virtually identical to bindings creates expectations that this proposal does not meet. I think reception would be very different if the types were called BorrowingPointer or MutatingPointer or something like that.
When we get one of the features discussed in that thread, then Inout and MutableSpan should be able to adopt them without fundamentally breaking their existing interface. The language will become more permissive in allowing projections through immutable exclusively-owned values, but won't break existing code that works within the limitation of only being able to project through mutable values.
In the "Proposed solution" section, this code is given as an example:
@_lifetime(&array)
func element(of array: inout [Int], at: Int) -> Inout<Int>? {
if at >= 0 && at < array.count {
return &array[at]
} else {
return nil
}
}
Should &array[at] be Inout(&array[at])? Or is implicit conversion to Borrow/Inout part of this proposal?
The argument made for the inline array type sugar review was: if sugar is warranted, it's important to add at the start so people dont need to re-learn.
I do not agree with the sugar being warranted for that feature, however the guidance there feels relevant here. These are complex features and asking users to relearn bits and pieces as the model comes together is likely to lead to frustration and confusion. (see concurrency today)
Thanks, yeah I did read that part. And I do understand the value in having these types, I guess I still felt confused about how it comes together with local reference bindings.
I think this is part of my confusion. If inout parameters are just a language feature and don't need a special type to work, why will the types be needed for local references?
A couple further questions that I didn't manage to answer by reading the proposal:
Is Inout compatible with inout parameters? I.e. if I create an Inout<Int>, can I pass it to a function taking inout Int?
If I mutate an Inout's value, am I actually mutating the Inout itself? I.e. does the Inout need to be var to mutate the referenced value, or is it considered similar to weak let?
I understand that this is a personal statement and not an official declaration of intent, but perhaps you can understand that the revelation that there may be a third evolution proposal in this set, which we cannot even see a first draft of, and might again be mis-ordered with respect to the first two, fills me with neither excitement nor confidence, and does not improve my opinion of the pieces we can see already.
Regarding the "magic" of looking through references, the "accessors" proposal already provides this; if I have var x: T { mutate { &_whatever } } then I can write val.x.member = 3 and val.x.method() and f(val.x) and so forth.
Swift supports computed properties in some strange places; could we "already" write something like
func whatever() {
let _x = Inout(...)
var x { mutate { &_x.value } }
}
in order to get the magic for x?
(and if so, why couldn't we just make all bindings with Inout/Borrow type automatically work that way?)
In fact, if Inout/Borrow were marked as @propertyWrapper and used wrappedValue instead of value, could this "just work" already? I guess you'd need a weird initializer too...
This makes a lot of sense to me and I agree 99%.
With the exception that I think naming fundamental types of a language is not "bikeshedding" at all, but rather 80% of the job. The implementation basically writes itself once everything is clear, and it can evolve as needed - names require thinking and are forever.
In general I agree with the various voices saying that it is really hard to gage whether this proposal will end up a regret or a solid building block. We are missing a good understanding of how it will all click together with future bindings, auto-deref, possible sugar, ...
Currently, to me, it feels a bit like shotgunning a handful of types in hoping that they'll fit. And, imho, anything called Inout should raise eyebrows.
I agree with others upthread that the syntax hinders readability and doesn't feel idiomatic. Core parts of the language do receive special syntax, such as some/any and async/throws. We could have chosen Result<_, Error> instead of throws, and Task<_, Error> instead of async. However, giving these special syntax makes the type signature easier to read and allows for special handling of things like unhandled async. Similarly, Borrow and Inout are closely tied to the type system. I agree that expressing them as non-escapable types is a very elegant solution that allows them to fit into the current type system without further modification. Nevertheless, I see three main downsides to introducing the proposed types in their current form.
Decreased Readability
Since information about ownership is now expressed in both the binding and the type, it becomes harder to reason about and quickly parse a property's ownership semantics. For instance, consider the following type:
struct Person: ~Copyable, ~Escapable {
var name: Inout<String>
var age: Inout<Int>
}
Here, ownership information lies in both the var keyword (we're saying it's mutable) and in the type Inout<String>. I understand that in this example, the Inout type tells the compiler that Person will only store a reference and not actually own a String; however, I don't see why we can't do the same by allowing inout bindings in non-copyable, non-escapable structs:
struct Person: ~Copyable, ~Escapable {
inout name: String
inout age: Int
}
Of course, specifying the ownership of some value isn't unique to types; functions already use ownership modifiers like borrowing, consuming, and inout. I am simply not convinced that types are special enough to warrant their own spelling of these ownership modifiers as types instead of extending them to the type system, e.g., Optional<inout Value>.
Types as "Bags of Functions"
I intuitively think of types in Swift as a bunch of related functions that operate on the same core data. This view is admittedly very simplistic, but I think it elucidates my point that functions shouldn't get special treatment over types. For instance, we can think of the following computed property as a function that takes the struct's fields and returns a String:
struct NormalPerson {
let name: String
let age: Int
var description: String { "\(name), \(age)" }
}
// Similar to:
func NormalPerson_description(name: String, age: Int) -> String { ... }
So, then, why not extend this logic to non-copyable, non-escapable types?
struct Person: ~Copyable, ~Escapable {
inout name: String
inout age: Int
mutating func incrementAge() { age += 1 }
}
// Similar to:
func Person_incrementAge(name: inout String, age: inout Int) -> String { ... }
The only difference from the previous case is that we're accepting inout arguments in this example, instead of borrowing as we did for NormalPerson. An astute reader might notice that treating Borrow and Inout as modifiers is more restrictive than what the proposal allows. Namely, we can't currently return a borrowing String from a function. Or could we?
Returning borrowing and inout
You'll notice that subscripts and properties already support a sort of borrowing and mutating access through the aptly named read and modify coroutines:
struct Person: ~Copyable, ~Escapable {
inout name: String
inout age: Int
// Current syntax
var nameGetter: String {
modify { yield &name }
}
So why not allow regular functions and getters to return a borrowing String, like so?
extension Person {
// Proposed syntax (getter)
var ageGetter: Int {
inout get { return &age }
}
// Proposed syntax (functions)
func getAgeReference() -> inout Int {
return &age
}
}
My counterproposal is to just treat inout Int the same way the proposal handles Inout<Int> and the same for borrowing.
However, you might rightfully argue that my proposal of supporting both inout get and modify is simply a new spelling for the existing coroutine feature. However, the main difference is that coroutines always return to the accessor for cleanup, whereas a function returning a (mutable) reference can simply return and rest assured that the type checker will take care of the rest. Returning borrowing/inout references is a useful addition for cases when reentering a coroutine for cleanup adds unnecessary overhead. (I believe there was a discussion about this distinction, which languages such as Hylo make.)
I think the underlying feature is incredibly useful and I'm very grateful we now have the compiler infrastructure to implement it. Yet, I strongly believe that we should choose a syntax that aligns with existing Swift features and idioms.
I think that Alias<T> and MutableAlias<T> are much better names for these types. They don’t collide with language keywords and also avoid using “pointer”.
Alias<T> and Span<T> would be the safe alternatives to UnsafePointer<T> and UnsafeBufferPointer<T>.
I don’t think that “alias” is a great word for this because of baggage from C-family languages (“aliasing”).
MutableAlias<T> would (to me) imply that you can have more than one of them to the same object. Similarly, I would expect Alias<T> to update when the aliased object does, rather than ending its lifetime.
This is a helpful reminder. I think I recall someone made the same point in the pitch thread too.
I agree that these types seem likely to be useful. While more capable borrow accessors would also be nice sooner than later, having these still feels worthwhile.
As for the names, I think the (Mutable)Reference would be better:
- The pitch thread and this proposal thread both describe these new types out as “references”. It’s right there in the name of the proposal. I have seen no posts disagreeing with that characterization, though I didn’t read everything.
- The existence of
Unsafe(Mutable)Pointerlightly argues for the reference-ish counterparts to be namedReferences Referencehas the same feel IMO asResult, which we’ve recognized as a common enough name to make the type added to the language (standard library?) as-isReferencehas the same feel IMO asBox, which while we don’t have an official(Modifier)Boxtype currently, we do have theUniqueBoxreview currently openInoutis not a name used in any other languages that I know of, nor does it IMO really describe whatInoutas pitched does. Whereinoutfor function parameters has been described as copy-in to start and copy-out when complete,Inoutas I read it essentially carries around a pointer. I agree with the other folks who’ve called this out as a major difference.