Pitch: Introduce `for borrow` and `for inout` to provide non-copying collection iteration

Sure, I'm just saying that inout doesn't seem to make sense here, unless I'm missing something.

My “instinct” wants to write things like this for the mutating version:

for &n in numbers {
  n += 1
}

I don’t know enough about borrow semantics to weigh in on whether the collection and/or its elements should be implicitly borrowed. From a language-user perspective, it would be nice if the compiler did The Right Thing™ automatically, but again I don’t know what the right thing is.

2 Likes

Here's my suggestion:

var numbers = [1, 2, 3]

for alias x in numbers {
    // immutable borrow
}

for alias y in &numbers {
    y += 1
    // mutable borrow
}

The types/bindings (e.g. when inspected in Xcode) would be:

var numbers: [Int]
alias x: Int
mutating alias y: Int

I agree that this feels natural, but it doesn’t necessarily make sense if you think about it too hard.

I feel like every discussion there’s been about borrowing has produced some subtle nudge toward Rust-like syntax. It all just seems to hang together more naturally, and it’s easier to predict the correct spelling of combinations of language features. For example, @JuneBash and I independently arrived at & being the universal signal for borrow expressions, which is very similar to Rust.

But as @jrose pointed out, borrowing is so common in Rust that the pervasive use of & feels like burdensome ceremony. That’s a good thing to avoid.

All that said, what if Swift 6 used & as a sigil rather than an operator? It could be optional when implicit copies are permissible.

var lines: [String] = …
for var &line in lines {
  // `&line` is the name of a mutable borrow
  // this very clearly operates through the reference
  &line += "\n"

  // you can use it without `&` to implicitly copy (except for move-only types)
  print(line)
}

Per the above discussion, lines can be optimized to a borrow during the iteration if it’s statically proven not to escape the enclosing scope. To force a borrow of the whole collection, prefix it with &:

for var &line in &lines { … }

Types are the immediate question—specifically anonymous function types. But it’s not uncommon for things to be “backwards” in that position (e.g. @Sendable):

// currently proposed as
// func forEach<S>(_: borrow S, do: (borrow S.Element) -> Void)
func forEach<S: Sequence>(_ &sequence: S, do operation: (& S.Element) throws -> Void) rethrows {
  for &element in sequence {
    // including the & sigil is necessary
    // to indicate we are passing the borrow
    operation(&element)
  }
}:
let strings: [String] = …
forEach(strings) {
  // semantic copy without the &
  // (possibly optimized to implicit borrow)
  print($0)

  // explicit borrow
  doSomethingElse(with: &$0)
}

This doesn’t mean & String is a true type—you can’t create an Optional<& String>. Borrowing is an intrinsic property of bindings, not an aspect of their types.

This avoids the awkwardness of inout in declaration position. I’m personally a fan of deprecating inout entirely, because I think it’s dangerous that mutations which appear to be local are in fact silently happening through a borrow:

// In Swift 5.x, this is currently spelled
// func reverse<C>(_ collection: inout C)
func reverse<C: MutableCollection>(_ &collection: var C) {
  // let’s imagine  MutableCollection.reversed() returned one of these
  if collection is ReversedCollection<C> {
    // note & on LHS of assignment
    // (also demonstrates lack of & in cast)
    &collection = (collection as ReversedCollection<C>)._base
    return
  }
    
  let indices = collection.indices
  let reverseIndices = indices.reversed()
  for indexPair in zip(indices, reverseIndices) {
    swap(&collection[$0], &collection[$1])
  }
}

If it helps Swift keep its safe-by-default behavior without additional ceremony, and as long as it’s possible to easily wrap a value in a move only type, I don’t mind typing &.

I really like how this proposal avoids needing to make a new type of iterator, seems very cool that existing types can get this behavior for free, even if they come from binaries built with an older swift version.

Can the approach of producing a reference from a subscript be used outside of a for loop? For example:

if let inout ref = someObject[index] {
   ref.someField = foo
}

Apologies if this has already been answered elsewhere. I skimmed the originally borrowing proposal but didn’t see any mention of how this is supported, if at all.

Rather than thinking of the inout as being attached to for, think of it as being attached to x. The for loop will bind the elements to inout x instead of just x.

3 Likes

OK but still, in other contexts inout is never attached to the variable, and not even present at the call site. It's an annotation used in function declarations, on the type.

I suppose one might see a for loop as something in between a declaration and a call site, but I personally prefer to see it as a call site, and for as a special kind of (meta) function.

By the way, what does the $ in "$n" mean in the desugared examples?

This pitch is explicitly built on top of the one about local borrow and inout bindings.

The borrowing/inout proposal does deal with this case. Remember that collections can implement their subscript operations with explicit accessors. For example, here is one of the subscript operations on the standard library Array implementation. The section on "Visibility of mutations" details how this works with get/set accessors and there's a brief mention later in that document (under "Future Directions") of how this might extend to work with (not-yet-standardized) read/modify accessors.

So in your example, if someObject[index] were implemented with get/set accessors, then there would be an implied set at the closing brace to write back the modified value. If that subscript made use of read/modify coroutine-based accessors, then it can be implemented more efficiently.

1 Like

Out of curiosity: Any progress on this pitch?

3 Likes

I find the & with the inout thoroughly confusing to read. It reminds me of battling to understand C pointer notation.

Is there a way that this functionality could be provided with a function?

something like...

//Square numbers
numbers.mutatingMap { n in return n*n }

or
numbers.mutatingMap { $0 * $0 }

to me, that's a much easier read.

obviously, the name could be different. inPlaceMap, inOutMap (or whatever).
In Ruby, it would be map!

That would not be compatible with other control flow like continue, would make return have a different meaning, and would introduce some complexity around effects (eg we don’t currently have reasync so there would either have to be two versions or you couldn’t call async functions in it).

Sigils like & do have some disadvantages (hard to search the web for, for example), but at least this is one that’s already used in Swift in other places for essentially the same meaning.

8 Likes

Non-copying iteration is also desirable for C++ interop. See this issue on Github for an example.