[Proposal] Scoped resources (like C# using statement)

# Introduction

Add a new `Scoped` protocol and enhance the do statement to automatically
call enter/exit actions on resources.

# Motivation

Resources (e.g., locks, files, sockets, etc.) often need to be scoped to a
block, where some action is taken at the start of the block and another is
required at the end. Examples include locking and unlocking a lock in a
critical section or closing a file at the end of a block.

Doing this manually is possible using `defer` statements among other
options, but this is error prone as a `defer` can be forgotten,
`lock`/`unlock` calls for two locks can be switched due to a typo, etc.
Having a dedicated language construct for this common case makes it easier
to read and write while making code shorter and clearer.

# Language Survey

At least three major languages have widely used statements for this use
case.

## C#

C# has the `using` statement and the associated `IDisposable` interface.

using (StreamReader sr = new StreamReader(filename)) {
    txt = sr.ReadToEnd();
}

C#'s solution only handles close/exit actions via the
`IDisposable.Dispose()` method and so cannot easily be used with items such
as locks; however, C# has the additional `lock` statement for that use case.

## Java

Java has try-with-resources and the associated `AutoCloseable` interface.

try (BufferedReader br = new BufferedReader(new FileReader(path))) {
    return br.readLine();
}

Java's solution only handles close/exit actions via the
`AutoCloseable.close()` method and so cannot easily be used with items such
as locks; however, Java has the additional `synchronized` statement for
that use case.

## Python

Python has with `with` statement and the associated `__enter__` and
`__exit__` special methods that classes may implement to become a "context
manager".

with lock, open(path) as my_file:
    contents = my_file.read()
    # use contents

Python's solution handles both enter and exit actions and so this one
construct is usable for locks as well as resources like sockets and files.

# Proposed Solution

We add a new protocol called `Scoped` to the standard library. Types for
resources that have enter/exit actions will be extended to add conformance
to this protocol.

The `do` statement will be extended to allow a new `using <resources>`
"suffix".

# Detailed Design

The `Scoped` protocol shall be as follows:

public protocol Scoped {
    func enterScope()
    func exitScope()
}

The compiler statement will accept a new form for resources. For example,

do using lock, let file = try getFileHandle() {
    // statements
}

As can be seen in the example above, the resources can be bindings that
already exist (like `lock`) or can be new bindings. Bindings created with
`do using` are not available outside of the scope of the `do using`. Only
types conforming to `Scoped` may be using with `do using`. Use of
non-conforming types will result in a compiler error.

The above example would be syntactic sugar for the following:

do {
    lock.enterScope()
    defer { lock.exitScope() }

    let file = try getFileHandle()
    file.enterScope()
    defer { file.exitScope() }

    // statements
}

# Framework Changes / Examples

As an example of some real-world classes that would be useful with
`Scoped`, from Foundation:

// Would be nice to extend the NSLocking protocol instead, but that's not
supported yet.
extension NSLock: Scoped {
    func enterScope() {
        self.lock()
    }

    func exitScope() {
        self.unlock()
    }
}

extension NSFileHandle: Scoped {
    func enterScope() {}

    func exitScope() {
        self.closeFile()
    }
}

# Questions and Concerns
* Bikeshedding protocol naming and scoping syntax
* Should the enter and exit actions be allowed to throw errors?

···

--
Trent Nadeau

We have another pattern that types can use, which is:

lock.withThisLockHeld {
  … stuff ...
}

This can be done today with trailing closures. Other examples of this are “autoreleasepool” and withUnsafePointer (for other reasons).

-Chris

···

On Dec 29, 2015, at 8:02 PM, Trent Nadeau via swift-evolution <swift-evolution@swift.org> wrote:

Doing this manually is possible using `defer` statements among other options, but this is error prone as a `defer` can be forgotten, `lock`/`unlock` calls for two locks can be switched due to a typo, etc. Having a dedicated language construct for this common case makes it easier to read and write while making code shorter and clearer.

```swift
do {
    lock.enterScope()
    defer { lock.exitScope() }

    let file = try getFileHandle()
    file.enterScope()
    defer { file.exitScope() }

    // statements
}

An alternative solution is to do what Rust and C++ do, which is to use
RAII. Which is to say, instead of introducing a new language construct
that's explicitly tied to a scope, you just use a struct to represent
the resource that you hold (e.g. a File that represents an open file).
Of course, this does require some changes to structs, notably the
addition of a deinit. And if structs have a deinit, then they also need
to have a way to restrict copies. This is precisely what Rust does; any
struct in Rust that implements Drop (the equivalent to deinit) loses the
ability to be implicitly copied (a second trait called Clone provides a
.clone() method that is the normal way to copy such non-implicitly-
copyable structs).

This solution is elegant for a few reasons:

1. Once you allow deinit on structs (which is useful) and deal with the
   fact that structs are no longer implicitly copyable (because almost
   all deinit functions on structs won't work right if they're called
   twice, such as on two copies), then RAII just sort of falls out of
   all of this and doesn't require any specific language features.
2. It's more flexible, because you can actually return the RAII value
   from a scope in order to extend its lifetime.
3. The RAII value itself provides the means to access the resource it's
   protecting; e.g. a Lock might return a LockGuard RAII value from the
   .lock() method, and LockGuard provides the means to access the
   protected value (as opposed to just having the lock sitting next to
   the value, which makes it trivially easy to accidentally access the
   value without holding the lock).
4. In Rust, this pattern also integrates extremely well with Rust's
   lifetime system (the system that prevents data races / memory
   corruption at compile time), because e.g. a LockGuard contains the
   lifetime of the Lock, which prevents you at compile-time from
   attempting to lock the Lock while you already have it locked (though
   it doesn't prevent deadlocks where you and another thread try and
   lock two locks in opposite orders, but there is plenty of stuff it
   does catch).

The biggest problem with adding deinit to structs in Swift right now is
the fact that we don't have references, which means you can't take a
RAII struct and pass it to another function without losing it. Heck, you
can't even call a method on it, because `self` on non-mutating methods
in Swift is a value and not a reference (although we could hand-wave
that away and make `self` in non-mutating RAII methods actually be an
"in" reference internally, but this kind of hand-waving doesn't work
when passing the struct as an argument to another function). So we'd
actually have to introduce a special "in" reference, which would be like
"inout" except it doesn't actually copy it out (and is guaranteed to
actually pass a pointer, although this pointer may be a pointer to a
temporary). Except even that fails if you want to have a computed
property that returns an existing RAII value (for example, having an
Array of these things; barring optimizations, the subscript getter
returns a computed value). And you can't generalize such "in" references
beyond function arguments without basically providing raw C pointers.
Rust's lifetime system lets them do it safely, but barring such a
system, Swift can't really do this (I assume it's obvious why we don't
want to start using raw C pointers everywhere).

All that said, I think this is a problem Swift needs to solve, because
having non-copyable structs would be very useful. In particular, I
_really_ want some way to do atomics in Swift, but the only safe way I
can think of to do it requires non-copyable structs (because it's not
correct to do a nonatomic read (such as a memcpy) of an atomic that's
visible to other threads). I suppose you could provide a way to override
how struct copies work (e.g. so you could do an atomic read of the old
value), but it's rather problematic to break the assumption that struct
copies are cheap.

Of course, you could model RAII with classes instead of structs, that
just has the overhead of heap allocation (+ atomic reference counting)
for every RAII value.

-Kevin Ballard

···

On Tue, Dec 29, 2015, at 08:02 PM, Trent Nadeau via swift-evolution wrote:

# Introduction

Add a new `Scoped` protocol and enhance the do statement to
automatically call enter/exit actions on resources.

# Motivation

Resources (e.g., locks, files, sockets, etc.) often need to be scoped
to a block, where some action is taken at the start of the block and
another is required at the end. Examples include locking and unlocking
a lock in a critical section or closing a file at the end of a block.

Doing this manually is possible using `defer` statements among other
options, but this is error prone as a `defer` can be forgotten,
`lock`/`unlock` calls for two locks can be switched due to a typo,
etc. Having a dedicated language construct for this common case makes
it easier to read and write while making code shorter and clearer.

# Language Survey

At least three major languages have widely used statements for this
use case.

## C#

C# has the `using` statement and the associated `IDisposable`
interface.

txt = sr.ReadToEnd(); } ```

C#'s solution only handles close/exit actions via the
`IDisposable.Dispose()` method and so cannot easily be used with items
such as locks; however, C# has the additional `lock` statement for
that use case.

## Java

Java has try-with-resources and the associated `AutoCloseable`
interface.

```java try (BufferedReader br = new BufferedReader(new
FileReader\(path\)\)\) \{    return br\.readLine\(\); \} \`\`\`

Java&#39;s solution only handles close/exit actions via the
\`AutoCloseable\.close\(\)\` method and so cannot easily be used with items
such as locks; however, Java has the additional \`synchronized\`
statement for that use case\.

\#\# Python

Python has with \`with\` statement and the associated \`\_\_enter\_\_\` and
\`\_\_exit\_\_\` special methods that classes may implement to become a
&quot;context manager&quot;\.

```python with lock, open(path) as my_file:    contents =
my_file.read()    # use contents ```

Python's solution handles both enter and exit actions and so this
one construct is usable for locks as well as resources like sockets
and files.

# Proposed Solution

We add a new protocol called `Scoped` to the standard library. Types
for resources that have enter/exit actions will be extended to add
conformance to this protocol.

The `do` statement will be extended to allow a new `using <resources>`
"suffix".

# Detailed Design

The `Scoped` protocol shall be as follows:

```swift public protocol Scoped {    func enterScope()    func
exitScope\(\) \} \`\`\`

The compiler statement will accept a new form for resources\. For
example,

```swift do using lock, let file = try getFileHandle() {    //
statements } ```

As can be seen in the example above, the resources can be bindings
that already exist (like `lock`) or can be new bindings. Bindings
created with `do using` are not available outside of the scope of the
`do using`. Only types conforming to `Scoped` may be using with `do
using`. Use of non-conforming types will result in a compiler error.

The above example would be syntactic sugar for the following:

```swift do {    lock.enterScope()    defer { lock.exitScope() }

let file = try getFileHandle\(\)    file\.enterScope\(\)    defer \{
file\.exitScope\(\) \}

// statements \} \`\`\`

\# Framework Changes / Examples

As an example of some real\-world classes that would be useful with
\`Scoped\`, from Foundation:

\`\`\`swift // Would be nice to extend the NSLocking protocol instead,
but that&#39;s not supported yet\. extension NSLock: Scoped \{    func
enterScope\(\) \{        self\.lock\(\)    \}

func exitScope\(\) \{        self\.unlock\(\)    \} \}

extension NSFileHandle: Scoped \{    func enterScope\(\) \{\}

func exitScope\(\) \{        self\.closeFile\(\)    \} \} \`\`\`

\# Questions and Concerns
\* Bikeshedding protocol naming and scoping syntax \* Should the enter
and exit actions be allowed to throw errors?

\-\-
Trent Nadeau

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
swift\-evolution mailing list swift\-evolution@swift\.org
https://lists.swift.org/mailman/listinfo/swift-evolution

While useful, that pattern doesn't seem to compose well. What if you need
two locks? Would that be:

lock1.withThisLockHeld {
    lock2.withThisLockHeld {
        // statements
    }
}

If so, it seems like it has the "pyramid of doom" issue that prompted
allowing `if let` to have multiple bindings.

In addition to the possible indentation and vertical space issue, you need
to look up if and how each resource type does this. I believe this is a
general enough pattern that it deserves language support. I think an
analogy to the current situation would be if each collection type had its
own way to iterate (Array.forEach, Set.withEachElement, etc.) instead of
having for-in.

···

On Tue, Dec 29, 2015 at 11:15 PM, Chris Lattner <clattner@apple.com> wrote:

On Dec 29, 2015, at 8:02 PM, Trent Nadeau via swift-evolution < > swift-evolution@swift.org> wrote:
> Doing this manually is possible using `defer` statements among other
options, but this is error prone as a `defer` can be forgotten,
`lock`/`unlock` calls for two locks can be switched due to a typo, etc.
Having a dedicated language construct for this common case makes it easier
to read and write while making code shorter and clearer.
>
> ```swift
> do {
> lock.enterScope()
> defer { lock.exitScope() }
>
> let file = try getFileHandle()
> file.enterScope()
> defer { file.exitScope() }
>
> // statements
> }

We have another pattern that types can use, which is:

lock.withThisLockHeld {
  … stuff ...
}

This can be done today with trailing closures. Other examples of this are
“autoreleasepool” and withUnsafePointer (for other reasons).

-Chris

--
Trent Nadeau

I'm really standing with Kevin on this one — no non-orthogonal features and special cases please. If we allow RAII- (DIRR-)-style constructors and destructors, we get a solution that is more general (needs no new syntax nor a separate language construct) and doesn't introduce syntactic noise (via the triangle of death).

···

Sent from my iPhone

On 30 Dec 2015, at 05:55, Kevin Ballard via swift-evolution <swift-evolution@swift.org> wrote:

An alternative solution is to do what Rust and C++ do, which is to use RAII. Which is to say, instead of introducing a new language construct that's explicitly tied to a scope, you just use a struct to represent the resource that you hold (e.g. a File that represents an open file). Of course, this does require some changes to structs, notably the addition of a deinit. And if structs have a deinit, then they also need to have a way to restrict copies. This is precisely what Rust does; any struct in Rust that implements Drop (the equivalent to deinit) loses the ability to be implicitly copied (a second trait called Clone provides a .clone() method that is the normal way to copy such non-implicitly-copyable structs).

This solution is elegant for a few reasons:

1. Once you allow deinit on structs (which is useful) and deal with the fact that structs are no longer implicitly copyable (because almost all deinit functions on structs won't work right if they're called twice, such as on two copies), then RAII just sort of falls out of all of this and doesn't require any specific language features.
2. It's more flexible, because you can actually return the RAII value from a scope in order to extend its lifetime.
3. The RAII value itself provides the means to access the resource it's protecting; e.g. a Lock might return a LockGuard RAII value from the .lock() method, and LockGuard provides the means to access the protected value (as opposed to just having the lock sitting next to the value, which makes it trivially easy to accidentally access the value without holding the lock).
4. In Rust, this pattern also integrates extremely well with Rust's lifetime system (the system that prevents data races / memory corruption at compile time), because e.g. a LockGuard contains the lifetime of the Lock, which prevents you at compile-time from attempting to lock the Lock while you already have it locked (though it doesn't prevent deadlocks where you and another thread try and lock two locks in opposite orders, but there is plenty of stuff it does catch).

The biggest problem with adding deinit to structs in Swift right now is the fact that we don't have references, which means you can't take a RAII struct and pass it to another function without losing it. Heck, you can't even call a method on it, because `self` on non-mutating methods in Swift is a value and not a reference (although we could hand-wave that away and make `self` in non-mutating RAII methods actually be an "in" reference internally, but this kind of hand-waving doesn't work when passing the struct as an argument to another function). So we'd actually have to introduce a special "in" reference, which would be like "inout" except it doesn't actually copy it out (and is guaranteed to actually pass a pointer, although this pointer may be a pointer to a temporary). Except even that fails if you want to have a computed property that returns an existing RAII value (for example, having an Array of these things; barring optimizations, the subscript getter returns a computed value). And you can't generalize such "in" references beyond function arguments without basically providing raw C pointers. Rust's lifetime system lets them do it safely, but barring such a system, Swift can't really do this (I assume it's obvious why we don't want to start using raw C pointers everywhere).

All that said, I think this is a problem Swift needs to solve, because having non-copyable structs would be very useful. In particular, I _really_ want some way to do atomics in Swift, but the only safe way I can think of to do it requires non-copyable structs (because it's not correct to do a nonatomic read (such as a memcpy) of an atomic that's visible to other threads). I suppose you could provide a way to override how struct copies work (e.g. so you could do an atomic read of the old value), but it's rather problematic to break the assumption that struct copies are cheap.

Of course, you could model RAII with classes instead of structs, that just has the overhead of heap allocation (+ atomic reference counting) for every RAII value.

-Kevin Ballard

On Tue, Dec 29, 2015, at 08:02 PM, Trent Nadeau via swift-evolution wrote:
# Introduction

Add a new `Scoped` protocol and enhance the do statement to automatically call enter/exit actions on resources.

# Motivation

Resources (e.g., locks, files, sockets, etc.) often need to be scoped to a block, where some action is taken at the start of the block and another is required at the end. Examples include locking and unlocking a lock in a critical section or closing a file at the end of a block.

Doing this manually is possible using `defer` statements among other options, but this is error prone as a `defer` can be forgotten, `lock`/`unlock` calls for two locks can be switched due to a typo, etc. Having a dedicated language construct for this common case makes it easier to read and write while making code shorter and clearer.

# Language Survey

At least three major languages have widely used statements for this use case.

## C#

C# has the `using` statement and the associated `IDisposable` interface.

using (StreamReader sr = new StreamReader(filename)) {
    txt = sr.ReadToEnd();
}

C#'s solution only handles close/exit actions via the `IDisposable.Dispose()` method and so cannot easily be used with items such as locks; however, C# has the additional `lock` statement for that use case.

## Java

Java has try-with-resources and the associated `AutoCloseable` interface.

try (BufferedReader br = new BufferedReader(new FileReader(path))) {
    return br.readLine();
}

Java's solution only handles close/exit actions via the `AutoCloseable.close()` method and so cannot easily be used with items such as locks; however, Java has the additional `synchronized` statement for that use case.

## Python

Python has with `with` statement and the associated `__enter__` and `__exit__` special methods that classes may implement to become a "context manager".

with lock, open(path) as my_file:
    contents = my_file.read()
    # use contents

Python's solution handles both enter and exit actions and so this one construct is usable for locks as well as resources like sockets and files.

# Proposed Solution

We add a new protocol called `Scoped` to the standard library. Types for resources that have enter/exit actions will be extended to add conformance to this protocol.

The `do` statement will be extended to allow a new `using <resources>` "suffix".

# Detailed Design

The `Scoped` protocol shall be as follows:

public protocol Scoped {
    func enterScope()
    func exitScope()
}

The compiler statement will accept a new form for resources. For example,

do using lock, let file = try getFileHandle() {
    // statements
}

As can be seen in the example above, the resources can be bindings that already exist (like `lock`) or can be new bindings. Bindings created with `do using` are not available outside of the scope of the `do using`. Only types conforming to `Scoped` may be using with `do using`. Use of non-conforming types will result in a compiler error.

The above example would be syntactic sugar for the following:

do {
    lock.enterScope()
    defer { lock.exitScope() }

    let file = try getFileHandle()
    file.enterScope()
    defer { file.exitScope() }

    // statements
}

# Framework Changes / Examples

As an example of some real-world classes that would be useful with `Scoped`, from Foundation:

// Would be nice to extend the NSLocking protocol instead, but that's not supported yet.
extension NSLock: Scoped {
    func enterScope() {
        self.lock()
    }

    func exitScope() {
        self.unlock()
    }
}

extension NSFileHandle: Scoped {
    func enterScope() {}

    func exitScope() {
        self.closeFile()
    }
}

# Questions and Concerns
* Bikeshedding protocol naming and scoping syntax
* Should the enter and exit actions be allowed to throw errors?

--
Trent Nadeau

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

An alternative solution is to do what Rust and C++ do, which is to use RAII. Which is to say, instead of introducing a new language construct that's explicitly tied to a scope, you just use a struct to represent the resource that you hold (e.g. a File that represents an open file). Of course, this does require some changes to structs, notably the addition of a deinit. And if structs have a deinit, then they also need to have a way to restrict copies. This is precisely what Rust does; any struct in Rust that implements Drop (the equivalent to deinit) loses the ability to be implicitly copied (a second trait called Clone provides a .clone() method that is the normal way to copy such non-implicitly-copyable structs).

This solution is elegant for a few reasons:

1. Once you allow deinit on structs (which is useful) and deal with the fact that structs are no longer implicitly copyable (because almost all deinit functions on structs won't work right if they're called twice, such as on two copies), then RAII just sort of falls out of all of this and doesn't require any specific language features.
2. It's more flexible, because you can actually return the RAII value from a scope in order to extend its lifetime.
3. The RAII value itself provides the means to access the resource it's protecting; e.g. a Lock might return a LockGuard RAII value from the .lock() method, and LockGuard provides the means to access the protected value (as opposed to just having the lock sitting next to the value, which makes it trivially easy to accidentally access the value without holding the lock).
4. In Rust, this pattern also integrates extremely well with Rust's lifetime system (the system that prevents data races / memory corruption at compile time), because e.g. a LockGuard contains the lifetime of the Lock, which prevents you at compile-time from attempting to lock the Lock while you already have it locked (though it doesn't prevent deadlocks where you and another thread try and lock two locks in opposite orders, but there is plenty of stuff it does catch).

The biggest problem with adding deinit to structs in Swift right now is the fact that we don't have references, which means you can't take a RAII struct and pass it to another function without losing it. Heck, you can't even call a method on it, because `self` on non-mutating methods in Swift is a value and not a reference (although we could hand-wave that away and make `self` in non-mutating RAII methods actually be an "in" reference internally, but this kind of hand-waving doesn't work when passing the struct as an argument to another function). So we'd actually have to introduce a special "in" reference, which would be like "inout" except it doesn't actually copy it out (and is guaranteed to actually pass a pointer, although this pointer may be a pointer to a temporary). Except even that fails if you want to have a computed property that returns an existing RAII value (for example, having an Array of these things; barring optimizations, the subscript getter returns a computed value). And you can't generalize such "in" references beyond function arguments without basically providing raw C pointers. Rust's lifetime system lets them do it safely, but barring such a system, Swift can't really do this (I assume it's obvious why we don't want to start using raw C pointers everywhere).

All that said, I think this is a problem Swift needs to solve, because having non-copyable structs would be very useful. In particular, I _really_ want some way to do atomics in Swift, but the only safe way I can think of to do it requires non-copyable structs (because it's not correct to do a nonatomic read (such as a memcpy) of an atomic that's visible to other threads). I suppose you could provide a way to override how struct copies work (e.g. so you could do an atomic read of the old value), but it's rather problematic to break the assumption that struct copies are cheap.

Of course, you could model RAII with classes instead of structs, that just has the overhead of heap allocation (+ atomic reference counting) for every RAII value.

+1 to using RAII for this.

I want to see all of the enhancements you describe that would enable structs to be used.

The situation for classes isn't necessarily quite as dire you make it sound. In some cases the optimizer will perform stack promotion. There was a brief discussion about this a few days ago. I don't recall which thread.

There is one challenge with using classes however. ARC does not guarantee an instance will stay alive until the end of the scope. Depending on the use case you may need this guarantee.

So in the end, improved language support for RAII one way or another is very desirable. I don't expect to see major improvements in this area sin Swift 3, but maybe there are some smaller improvements that can be made.

···

Sent from my iPad

On Dec 29, 2015, at 10:55 PM, Kevin Ballard via swift-evolution <swift-evolution@swift.org> wrote:

-Kevin Ballard

On Tue, Dec 29, 2015, at 08:02 PM, Trent Nadeau via swift-evolution wrote:
# Introduction

Add a new `Scoped` protocol and enhance the do statement to automatically call enter/exit actions on resources.

# Motivation

Resources (e.g., locks, files, sockets, etc.) often need to be scoped to a block, where some action is taken at the start of the block and another is required at the end. Examples include locking and unlocking a lock in a critical section or closing a file at the end of a block.

Doing this manually is possible using `defer` statements among other options, but this is error prone as a `defer` can be forgotten, `lock`/`unlock` calls for two locks can be switched due to a typo, etc. Having a dedicated language construct for this common case makes it easier to read and write while making code shorter and clearer.

# Language Survey

At least three major languages have widely used statements for this use case.

## C#

C# has the `using` statement and the associated `IDisposable` interface.

using (StreamReader sr = new StreamReader(filename)) {
    txt = sr.ReadToEnd();
}

C#'s solution only handles close/exit actions via the `IDisposable.Dispose()` method and so cannot easily be used with items such as locks; however, C# has the additional `lock` statement for that use case.

## Java

Java has try-with-resources and the associated `AutoCloseable` interface.

try (BufferedReader br = new BufferedReader(new FileReader(path))) {
    return br.readLine();
}

Java's solution only handles close/exit actions via the `AutoCloseable.close()` method and so cannot easily be used with items such as locks; however, Java has the additional `synchronized` statement for that use case.

## Python

Python has with `with` statement and the associated `__enter__` and `__exit__` special methods that classes may implement to become a "context manager".

with lock, open(path) as my_file:
    contents = my_file.read()
    # use contents

Python's solution handles both enter and exit actions and so this one construct is usable for locks as well as resources like sockets and files.

# Proposed Solution

We add a new protocol called `Scoped` to the standard library. Types for resources that have enter/exit actions will be extended to add conformance to this protocol.

The `do` statement will be extended to allow a new `using <resources>` "suffix".

# Detailed Design

The `Scoped` protocol shall be as follows:

public protocol Scoped {
    func enterScope()
    func exitScope()
}

The compiler statement will accept a new form for resources. For example,

do using lock, let file = try getFileHandle() {
    // statements
}

As can be seen in the example above, the resources can be bindings that already exist (like `lock`) or can be new bindings. Bindings created with `do using` are not available outside of the scope of the `do using`. Only types conforming to `Scoped` may be using with `do using`. Use of non-conforming types will result in a compiler error.

The above example would be syntactic sugar for the following:

do {
    lock.enterScope()
    defer { lock.exitScope() }

    let file = try getFileHandle()
    file.enterScope()
    defer { file.exitScope() }

    // statements
}

# Framework Changes / Examples

As an example of some real-world classes that would be useful with `Scoped`, from Foundation:

// Would be nice to extend the NSLocking protocol instead, but that's not supported yet.
extension NSLock: Scoped {
    func enterScope() {
        self.lock()
    }

    func exitScope() {
        self.unlock()
    }
}

extension NSFileHandle: Scoped {
    func enterScope() {}

    func exitScope() {
        self.closeFile()
    }
}

# Questions and Concerns
* Bikeshedding protocol naming and scoping syntax
* Should the enter and exit actions be allowed to throw errors?

--
Trent Nadeau

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

An alternative solution is to do what Rust and C++ do, which is to use RAII. Which is to say, instead of introducing a new language construct that's explicitly tied to a scope, you just use a struct to represent the resource that you hold (e.g. a File that represents an open file). Of course, this does require some changes to structs, notably the addition of a deinit. And if structs have a deinit, then they also need to have a way to restrict copies. This is precisely what Rust does; any struct in Rust that implements Drop (the equivalent to deinit) loses the ability to be implicitly copied (a second trait called Clone provides a .clone() method that is the normal way to copy such non-implicitly-copyable structs).

deinit doesn't make sense for value types. Classes already support deinit, and you can use withExtendedLifetime to bound the lifetime of a resource-holding class. It would be reasonable to have a scoped lifetime marker similar to ObjC ARC too.

-Joe

···

On Dec 29, 2015, at 8:55 PM, Kevin Ballard via swift-evolution <swift-evolution@swift.org> wrote:

This solution is elegant for a few reasons:

1. Once you allow deinit on structs (which is useful) and deal with the fact that structs are no longer implicitly copyable (because almost all deinit functions on structs won't work right if they're called twice, such as on two copies), then RAII just sort of falls out of all of this and doesn't require any specific language features.
2. It's more flexible, because you can actually return the RAII value from a scope in order to extend its lifetime.
3. The RAII value itself provides the means to access the resource it's protecting; e.g. a Lock might return a LockGuard RAII value from the .lock() method, and LockGuard provides the means to access the protected value (as opposed to just having the lock sitting next to the value, which makes it trivially easy to accidentally access the value without holding the lock).
4. In Rust, this pattern also integrates extremely well with Rust's lifetime system (the system that prevents data races / memory corruption at compile time), because e.g. a LockGuard contains the lifetime of the Lock, which prevents you at compile-time from attempting to lock the Lock while you already have it locked (though it doesn't prevent deadlocks where you and another thread try and lock two locks in opposite orders, but there is plenty of stuff it does catch).

The biggest problem with adding deinit to structs in Swift right now is the fact that we don't have references, which means you can't take a RAII struct and pass it to another function without losing it. Heck, you can't even call a method on it, because `self` on non-mutating methods in Swift is a value and not a reference (although we could hand-wave that away and make `self` in non-mutating RAII methods actually be an "in" reference internally, but this kind of hand-waving doesn't work when passing the struct as an argument to another function). So we'd actually have to introduce a special "in" reference, which would be like "inout" except it doesn't actually copy it out (and is guaranteed to actually pass a pointer, although this pointer may be a pointer to a temporary). Except even that fails if you want to have a computed property that returns an existing RAII value (for example, having an Array of these things; barring optimizations, the subscript getter returns a computed value). And you can't generalize such "in" references beyond function arguments without basically providing raw C pointers. Rust's lifetime system lets them do it safely, but barring such a system, Swift can't really do this (I assume it's obvious why we don't want to start using raw C pointers everywhere).

All that said, I think this is a problem Swift needs to solve, because having non-copyable structs would be very useful. In particular, I _really_ want some way to do atomics in Swift, but the only safe way I can think of to do it requires non-copyable structs (because it's not correct to do a nonatomic read (such as a memcpy) of an atomic that's visible to other threads). I suppose you could provide a way to override how struct copies work (e.g. so you could do an atomic read of the old value), but it's rather problematic to break the assumption that struct copies are cheap.

Of course, you could model RAII with classes instead of structs, that just has the overhead of heap allocation (+ atomic reference counting) for every RAII value.

-Kevin Ballard

On Tue, Dec 29, 2015, at 08:02 PM, Trent Nadeau via swift-evolution wrote:

# Introduction

Add a new `Scoped` protocol and enhance the do statement to automatically call enter/exit actions on resources.

# Motivation

Resources (e.g., locks, files, sockets, etc.) often need to be scoped to a block, where some action is taken at the start of the block and another is required at the end. Examples include locking and unlocking a lock in a critical section or closing a file at the end of a block.

Doing this manually is possible using `defer` statements among other options, but this is error prone as a `defer` can be forgotten, `lock`/`unlock` calls for two locks can be switched due to a typo, etc. Having a dedicated language construct for this common case makes it easier to read and write while making code shorter and clearer.

# Language Survey

At least three major languages have widely used statements for this use case.

## C#

C# has the `using` statement and the associated `IDisposable` interface.

using (StreamReader sr = new StreamReader(filename)) {
    txt = sr.ReadToEnd();
}

C#'s solution only handles close/exit actions via the `IDisposable.Dispose()` method and so cannot easily be used with items such as locks; however, C# has the additional `lock` statement for that use case.

## Java

Java has try-with-resources and the associated `AutoCloseable` interface.

try (BufferedReader br = new BufferedReader(new FileReader(path))) {
    return br.readLine();
}

Java's solution only handles close/exit actions via the `AutoCloseable.close()` method and so cannot easily be used with items such as locks; however, Java has the additional `synchronized` statement for that use case.

## Python

Python has with `with` statement and the associated `__enter__` and `__exit__` special methods that classes may implement to become a "context manager".

with lock, open(path) as my_file:
    contents = my_file.read()
    # use contents

Python's solution handles both enter and exit actions and so this one construct is usable for locks as well as resources like sockets and files.

# Proposed Solution

We add a new protocol called `Scoped` to the standard library. Types for resources that have enter/exit actions will be extended to add conformance to this protocol.

The `do` statement will be extended to allow a new `using <resources>` "suffix".

# Detailed Design

The `Scoped` protocol shall be as follows:

public protocol Scoped {
    func enterScope()
    func exitScope()
}

The compiler statement will accept a new form for resources. For example,

do using lock, let file = try getFileHandle() {
    // statements
}

As can be seen in the example above, the resources can be bindings that already exist (like `lock`) or can be new bindings. Bindings created with `do using` are not available outside of the scope of the `do using`. Only types conforming to `Scoped` may be using with `do using`. Use of non-conforming types will result in a compiler error.

The above example would be syntactic sugar for the following:

do {
    lock.enterScope()
    defer { lock.exitScope() }

    let file = try getFileHandle()
    file.enterScope()
    defer { file.exitScope() }

    // statements
}

# Framework Changes / Examples

As an example of some real-world classes that would be useful with `Scoped`, from Foundation:

// Would be nice to extend the NSLocking protocol instead, but that's not supported yet.
extension NSLock: Scoped {
    func enterScope() {
        self.lock()
    }

    func exitScope() {
        self.unlock()
    }
}

extension NSFileHandle: Scoped {
    func enterScope() {}

    func exitScope() {
        self.closeFile()
    }
}

# Questions and Concerns
* Bikeshedding protocol naming and scoping syntax
* Should the enter and exit actions be allowed to throw errors?

--
Trent Nadeau

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

It is an appealing idea on first sight, but it makes the language more complex, and I don't think scoped resources are versatile enough to back justify their inclusion.

The defer feature which is already mentioned in the proposal is very similar — and I expect that will not only be the case for the effect, but also for the usefulness:
I rarely use defer… I still like the concept, but I guess our ecosystem just has little need for it.

Tino

IMO the biggest problem with this approach is that it leads to a much more complicated programming model. The vast majority of structs should be copiable, which gets you into rule-of-five programming (where nearly every nontrivial struct needs to implement init, deinit, copy, assign, move) pretty quickly. I don’t think Swift should go down this road.

We do need a way to handle atomics, but I think there are other ways to address some of these concerns, including optimizing away class heap allocations and refcounting in special cases.

-Dave

···

On Dec 29, 2015, at 8:55 PM, Kevin Ballard via swift-evolution <swift-evolution@swift.org> wrote:

An alternative solution is to do what Rust and C++ do, which is to use RAII. Which is to say, instead of introducing a new language construct that's explicitly tied to a scope, you just use a struct to represent the resource that you hold (e.g. a File that represents an open file). Of course, this does require some changes to structs, notably the addition of a deinit. And if structs have a deinit, then they also need to have a way to restrict copies. This is precisely what Rust does; any struct in Rust that implements Drop (the equivalent to deinit) loses the ability to be implicitly copied (a second trait called Clone provides a .clone() method that is the normal way to copy such non-implicitly-copyable structs).

This solution is elegant for a few reasons:

1. Once you allow deinit on structs (which is useful) and deal with the fact that structs are no longer implicitly copyable (because almost all deinit functions on structs won't work right if they're called twice, such as on two copies), then RAII just sort of falls out of all of this and doesn't require any specific language features.
2. It's more flexible, because you can actually return the RAII value from a scope in order to extend its lifetime.
3. The RAII value itself provides the means to access the resource it's protecting; e.g. a Lock might return a LockGuard RAII value from the .lock() method, and LockGuard provides the means to access the protected value (as opposed to just having the lock sitting next to the value, which makes it trivially easy to accidentally access the value without holding the lock).
4. In Rust, this pattern also integrates extremely well with Rust's lifetime system (the system that prevents data races / memory corruption at compile time), because e.g. a LockGuard contains the lifetime of the Lock, which prevents you at compile-time from attempting to lock the Lock while you already have it locked (though it doesn't prevent deadlocks where you and another thread try and lock two locks in opposite orders, but there is plenty of stuff it does catch).

The biggest problem with adding deinit to structs in Swift right now is the fact that we don't have references, which means you can't take a RAII struct and pass it to another function without losing it. Heck, you can't even call a method on it, because `self` on non-mutating methods in Swift is a value and not a reference (although we could hand-wave that away and make `self` in non-mutating RAII methods actually be an "in" reference internally, but this kind of hand-waving doesn't work when passing the struct as an argument to another function). So we'd actually have to introduce a special "in" reference, which would be like "inout" except it doesn't actually copy it out (and is guaranteed to actually pass a pointer, although this pointer may be a pointer to a temporary). Except even that fails if you want to have a computed property that returns an existing RAII value (for example, having an Array of these things; barring optimizations, the subscript getter returns a computed value). And you can't generalize such "in" references beyond function arguments without basically providing raw C pointers. Rust's lifetime system lets them do it safely, but barring such a system, Swift can't really do this (I assume it's obvious why we don't want to start using raw C pointers everywhere).

All that said, I think this is a problem Swift needs to solve, because having non-copyable structs would be very useful. In particular, I _really_ want some way to do atomics in Swift, but the only safe way I can think of to do it requires non-copyable structs (because it's not correct to do a nonatomic read (such as a memcpy) of an atomic that's visible to other threads). I suppose you could provide a way to override how struct copies work (e.g. so you could do an atomic read of the old value), but it's rather problematic to break the assumption that struct copies are cheap.

Of course, you could model RAII with classes instead of structs, that just has the overhead of heap allocation (+ atomic reference counting) for every RAII value.

-Kevin Ballard

On Tue, Dec 29, 2015, at 08:02 PM, Trent Nadeau via swift-evolution wrote:

# Introduction

Add a new `Scoped` protocol and enhance the do statement to automatically call enter/exit actions on resources.

# Motivation

Resources (e.g., locks, files, sockets, etc.) often need to be scoped to a block, where some action is taken at the start of the block and another is required at the end. Examples include locking and unlocking a lock in a critical section or closing a file at the end of a block.

Doing this manually is possible using `defer` statements among other options, but this is error prone as a `defer` can be forgotten, `lock`/`unlock` calls for two locks can be switched due to a typo, etc. Having a dedicated language construct for this common case makes it easier to read and write while making code shorter and clearer.

# Language Survey

At least three major languages have widely used statements for this use case.

## C#

C# has the `using` statement and the associated `IDisposable` interface.

using (StreamReader sr = new StreamReader(filename)) {
    txt = sr.ReadToEnd();
}

C#'s solution only handles close/exit actions via the `IDisposable.Dispose()` method and so cannot easily be used with items such as locks; however, C# has the additional `lock` statement for that use case.

## Java

Java has try-with-resources and the associated `AutoCloseable` interface.

try (BufferedReader br = new BufferedReader(new FileReader(path))) {
    return br.readLine();
}

Java's solution only handles close/exit actions via the `AutoCloseable.close()` method and so cannot easily be used with items such as locks; however, Java has the additional `synchronized` statement for that use case.

## Python

Python has with `with` statement and the associated `__enter__` and `__exit__` special methods that classes may implement to become a "context manager".

with lock, open(path) as my_file:
    contents = my_file.read()
    # use contents

Python's solution handles both enter and exit actions and so this one construct is usable for locks as well as resources like sockets and files.

# Proposed Solution

We add a new protocol called `Scoped` to the standard library. Types for resources that have enter/exit actions will be extended to add conformance to this protocol.

The `do` statement will be extended to allow a new `using <resources>` "suffix".

# Detailed Design

The `Scoped` protocol shall be as follows:

public protocol Scoped {
    func enterScope()
    func exitScope()
}

The compiler statement will accept a new form for resources. For example,

do using lock, let file = try getFileHandle() {
    // statements
}

As can be seen in the example above, the resources can be bindings that already exist (like `lock`) or can be new bindings. Bindings created with `do using` are not available outside of the scope of the `do using`. Only types conforming to `Scoped` may be using with `do using`. Use of non-conforming types will result in a compiler error.

The above example would be syntactic sugar for the following:

do {
    lock.enterScope()
    defer { lock.exitScope() }

    let file = try getFileHandle()
    file.enterScope()
    defer { file.exitScope() }

    // statements
}

# Framework Changes / Examples

As an example of some real-world classes that would be useful with `Scoped`, from Foundation:

// Would be nice to extend the NSLocking protocol instead, but that's not supported yet.
extension NSLock: Scoped {
    func enterScope() {
        self.lock()
    }

    func exitScope() {
        self.unlock()
    }
}

extension NSFileHandle: Scoped {
    func enterScope() {}

    func exitScope() {
        self.closeFile()
    }
}

# Questions and Concerns
* Bikeshedding protocol naming and scoping syntax
* Should the enter and exit actions be allowed to throw errors?

--
Trent Nadeau

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

My understanding is that you suggest using the "do using" statement for two purposes:

to deterministically free resources (files, sockets);
to create a scope with a guarantee about something (locks).

The first purpose is very relevant for garbage-collected languages because the GC generally only monitors memory pressure to decide when to run, and so the GC won't budge if you're running out of file descriptors even if some could be reclaimed. However, Swift is not garbage-collected and resources are already reclaimed deterministically. If you create a Swift object that represents a file descriptor and don't allow references to escape the function, the object will be destroyed (and its resources reclaimed) at the latest when the function returns. In my opinion, this makes a "do using" statement useless for resource management.

For scopes with guarantees, as Chris said, the most common pattern is to have a function that accepts a closure. I've seen some pretty serious nesting with `if let` (which are one very frequent case of scopes with guarantees), but other than that, I don't see lots of nesting and I've been pretty happy with what the language can do so far. The only thing I can complain about is that you can't use break/continue with the current setup.

I see the discrepancy between objects, but I would say that any scope-based solution will often be just as hard to discover as the current solutions. `do using AutoreleasePool() { ... }` isn't an improvement over `autoreleasepool { ... }`.

It's better for the lock case, but only if you agree that `do using` is useless for resource management. Otherwise, if your mutex itself benefits from being scoped, `do using lock` is probably creating/deleting a mutex, and you'd need to use `do using Lock(mutex) { ... }` to actually lock it (like with C++ mutex/lock objects), which is as discoverable as `mutex.lock { ... }`.

Félix

···

Le 29 déc. 2015 à 23:24:53, Trent Nadeau via swift-evolution <swift-evolution@swift.org> a écrit :

While useful, that pattern doesn't seem to compose well. What if you need two locks? Would that be:

lock1.withThisLockHeld {
    lock2.withThisLockHeld {
        // statements
    }
}

If so, it seems like it has the "pyramid of doom" issue that prompted allowing `if let` to have multiple bindings.

In addition to the possible indentation and vertical space issue, you need to look up if and how each resource type does this. I believe this is a general enough pattern that it deserves language support. I think an analogy to the current situation would be if each collection type had its own way to iterate (Array.forEach, Set.withEachElement, etc.) instead of having for-in.

On Tue, Dec 29, 2015 at 11:15 PM, Chris Lattner <clattner@apple.com <mailto:clattner@apple.com>> wrote:
On Dec 29, 2015, at 8:02 PM, Trent Nadeau via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
> Doing this manually is possible using `defer` statements among other options, but this is error prone as a `defer` can be forgotten, `lock`/`unlock` calls for two locks can be switched due to a typo, etc. Having a dedicated language construct for this common case makes it easier to read and write while making code shorter and clearer.
>
> ```swift
> do {
> lock.enterScope()
> defer { lock.exitScope() }
>
> let file = try getFileHandle()
> file.enterScope()
> defer { file.exitScope() }
>
> // statements
> }

We have another pattern that types can use, which is:

lock.withThisLockHeld {
  … stuff ...
}

This can be done today with trailing closures. Other examples of this are “autoreleasepool” and withUnsafePointer (for other reasons).

-Chris

--
Trent Nadeau
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

It would if we extended the model for value types to be richer, e.g. to introduce the notion of "move only” structs.

-Chris

···

On Dec 30, 2015, at 9:53 AM, Joe Groff via swift-evolution <swift-evolution@swift.org> wrote:

On Dec 29, 2015, at 8:55 PM, Kevin Ballard via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

An alternative solution is to do what Rust and C++ do, which is to use RAII. Which is to say, instead of introducing a new language construct that's explicitly tied to a scope, you just use a struct to represent the resource that you hold (e.g. a File that represents an open file). Of course, this does require some changes to structs, notably the addition of a deinit. And if structs have a deinit, then they also need to have a way to restrict copies. This is precisely what Rust does; any struct in Rust that implements Drop (the equivalent to deinit) loses the ability to be implicitly copied (a second trait called Clone provides a .clone() method that is the normal way to copy such non-implicitly-copyable structs).

deinit doesn't make sense for value types.

Based on the discussion here, I'm withdrawing this proposal.

I created it because of a section in the Error Handling Rationale and
Proposal document (
https://github.com/apple/swift/blob/master/docs/ErrorHandlingRationale.rst\)
about `using`:

Swift should consider providing a using statement which acquires a
resource, holds it for a fixed period of time, optionally binds it to a
name, and then releases it whenever the controlled statement exits.

using has many similarities to defer. It does not subsume defer, which is
useful for many ad-hoc and tokenless clean-ups. But it is convenient for
the common pattern of a type-directed clean-up.

We do not expect this feature to be necessary in the first release.

However, it looks like there isn't enough need to warrant a language
change, at least for now. Is the Rationale doc a living document? If so, it
should perhaps be updated to mention that `using` needs a strong use case
given the current language features.

Thanks to everyone involved in this discussion.
<https://github.com/apple/swift/blob/master/docs/ErrorHandlingRationale.rst#c-and-objective-c-interoperation&gt;

···

On Wed, Dec 30, 2015 at 6:12 AM, Tino Heth <2th@gmx.de> wrote:

It is an appealing idea on first sight, but it makes the language more
complex, and I don't think scoped resources are versatile enough to back
justify their inclusion.

The defer feature which is already mentioned in the proposal is very
similar — and I expect that will not only be the case for the effect, but
also for the usefulness:
I rarely use defer… I still like the concept, but I guess our ecosystem
just has little need for it.

Tino

--
Trent Nadeau

If you run with the idea that any resource-holding class should also be
the mechanism by which you access the resource (e.g. a LockGuard that
represents holding the lock and also provides access to the guarded
value) then there's no need for extended lifetimes, because as long as
you're accessing the resource, you're keeping the resource-holding class
alive. I suppose there might be rare cases where you need to extend the
lifetime of a resource-holding class even when you're not accessing the
resource, just to guarantee e.g. order of resource releasing, but you
can always just say something like `withExtendedLifeetime(&val) {}` at
the end of the scope to ensure the value is still alive at that point.
Although I'd really like to define `_ = val` as guaranteeing that the
value is alive at that point (the expression doesn't actually do
anything, but because it references `val` it expresses the programmer's
intent that `val` should still be alive at that point in time).
Alternatively, if we end up with move-only structs (or uniquely-owned
classes), we could even define the expression `_ = val` as "dropping"
the value , i.e. forcing it to deinit at that spot (because it's moving
the value out of the `val` variable). This would be analogous to Rust's
`std::mem::drop()` function (which is literally defined as `pub fn
drop<T>(_x: T) { }` because all it does is move the value into the
function and then forget about it).

-Kevin Ballard

···

On Wed, Dec 30, 2015, at 09:53 AM, Joe Groff wrote:

On Dec 29, 2015, at 8:55 PM, Kevin Ballard via swift-evolution <swift- >> evolution@swift.org> wrote:

An alternative solution is to do what Rust and C++ do, which is to
use RAII. Which is to say, instead of introducing a new language
construct that's explicitly tied to a scope, you just use a struct to
represent the resource that you hold (e.g. a File that represents an
open file). Of course, this does require some changes to structs,
notably the addition of a deinit. And if structs have a deinit, then
they also need to have a way to restrict copies. This is precisely
what Rust does; any struct in Rust that implements Drop (the
equivalent to deinit) loses the ability to be implicitly copied (a
second trait called Clone provides a .clone() method that is the
normal way to copy such non-implicitly-copyable structs).

deinit doesn't make sense for value types. Classes already support
deinit, and you can use withExtendedLifetime to bound the lifetime of
a resource-holding class. It would be reasonable to have a scoped
lifetime marker similar to ObjC ARC too.

There's no rule-of-five if you explicitly define (as Rust does) that you
cannot override assign/copy/move. In Rust, a copyable struct is always
memcpy()'d. Anything that's not memcpy'd but still wants to support
explicit copying conforms to the Clone trait, and the compiler lets such
structs automatically derive Clone (the Clone trait provides a single
method .clone() that returns a copy of the struct). So it's basically
just init, which we already require (though we infer an init if you
don't otherwise define one), deinit which is totally optional, and then
just one line to derive a Clone implementation if appropriate (and of
course you can always manually implement Clone if you need special logic
to do so, e.g. the Arc type (Atomic Reference Counted) manually
implements Clone in order to bump the retain count, but that's actually
pretty rare, usually structs that need to actually do things (like bump
retain counts) on clone are composed of smaller pieces (like Arc) that
do that automatically).

Overall this system works pretty well. Because you can't override
assign/copy/move, you can move and copy structs around without worry
(just as we do in Swift). And if a struct is not copyable, you can
invoke .clone() and it's obvious from the code that you're doing
something more expensive than just a memcpy(). This system also
encourages the use of references when possible because it lets you avoid
the potentially-expensive clones (versus overriding copy, where you end
up invoking potentially-expensive copies without even realizing it). Of
course, this does require actually having references in the language to
begin with.

Ultimately, given Swift's current direction, I'm leaning right now
towards the idea of having uniquely-owned stack-allocated classes, and
that's basically just because classes are already a reference type,
which makes it a bit easier to explain adding magic that lets you say
things like non-mutating methods are acting on a reference (even though
it's stack-allocated) and provides an avenue for doing things like
saying things like `func foo(x: ref LockGuard<Bar>)` to represent the
idea of a pass-by-reference that's treated as @noescape (or the
alternative, `func foo(x: move LockGuard<Bar>)` that explicitly moves
the value, except we do need the @noescape part of passing a reference
to a uniquely-owned value so I think annotating the parameter with `ref`
or `in` makes more sense).

-Kevin Ballard

···

On Wed, Dec 30, 2015, at 01:40 PM, Dave Abrahams wrote:

IMO the biggest problem with this approach is that it leads to a much
more complicated programming model. The vast majority of structs
*should* be copiable, which gets you into rule-of-five programming
(where nearly every nontrivial struct needs to implement init, deinit,
copy, assign, move) pretty quickly. I don’t think Swift should go
down this road.

We do need a way to handle atomics, but I think there are other ways
to address some of these concerns, including optimizing away class
heap allocations and refcounting in special cases.

-Dave

On Dec 29, 2015, at 8:55 PM, Kevin Ballard via swift-evolution <swift- >> evolution@swift.org> wrote:

An alternative solution is to do what Rust and C++ do, which is to
use RAII. Which is to say, instead of introducing a new language
construct that's explicitly tied to a scope, you just use a struct to
represent the resource that you hold (e.g. a File that represents an
open file). Of course, this does require some changes to structs,
notably the addition of a deinit. And if structs have a deinit, then
they also need to have a way to restrict copies. This is precisely
what Rust does; any struct in Rust that implements Drop (the
equivalent to deinit) loses the ability to be implicitly copied (a
second trait called Clone provides a .clone() method that is the
normal way to copy such non-implicitly-copyable structs).

This solution is elegant for a few reasons:

1. Once you allow deinit on structs (which is useful) and deal with
   the fact that structs are no longer implicitly copyable (because
   almost all deinit functions on structs won't work right if they're
   called twice, such as on two copies), then RAII just sort of falls
   out of all of this and doesn't require any specific language
   features.
2. It's more flexible, because you can actually return the RAII value
   from a scope in order to extend its lifetime.
3. The RAII value itself provides the means to access the resource
   it's protecting; e.g. a Lock might return a LockGuard RAII value
   from the .lock() method, and LockGuard provides the means to
   access the protected value (as opposed to just having the lock
   sitting next to the value, which makes it trivially easy to
   accidentally access the value without holding the lock).
4. In Rust, this pattern also integrates extremely well with Rust's
   lifetime system (the system that prevents data races / memory
   corruption at compile time), because e.g. a LockGuard contains the
   lifetime of the Lock, which prevents you at compile-time from
   attempting to lock the Lock while you already have it locked
   (though it doesn't prevent deadlocks where you and another thread
   try and lock two locks in opposite orders, but there is plenty of
   stuff it does catch).

The biggest problem with adding deinit to structs in Swift right now
is the fact that we don't have references, which means you can't take
a RAII struct and pass it to another function without losing it.
Heck, you can't even call a method on it, because `self` on non-
mutating methods in Swift is a value and not a reference (although we
could hand-wave that away and make `self` in non-mutating RAII
methods actually be an "in" reference internally, but this kind of
hand-waving doesn't work when passing the struct as an argument to
another function). So we'd actually have to introduce a special "in"
reference, which would be like "inout" except it doesn't actually
copy it out (and is guaranteed to actually pass a pointer, although
this pointer may be a pointer to a temporary). Except even that fails
if you want to have a computed property that returns an existing RAII
value (for example, having an Array of these things; barring
optimizations, the subscript getter returns a computed value). And
you can't generalize such "in" references beyond function arguments
without basically providing raw C pointers. Rust's lifetime system
lets them do it safely, but barring such a system, Swift can't really
do this (I assume it's obvious why we don't want to start using raw C
pointers everywhere).

All that said, I think this is a problem Swift needs to solve,
because having non-copyable structs would be very useful. In
particular, I _really_ want some way to do atomics in Swift, but the
only safe way I can think of to do it requires non-copyable structs
(because it's not correct to do a nonatomic read (such as a memcpy)
of an atomic that's visible to other threads). I suppose you could
provide a way to override how struct copies work (e.g. so you could
do an atomic read of the old value), but it's rather problematic to
break the assumption that struct copies are cheap.

Of course, you could model RAII with classes instead of structs, that
just has the overhead of heap allocation (+ atomic reference
counting) for every RAII value.

-Kevin Ballard

On Tue, Dec 29, 2015, at 08:02 PM, Trent Nadeau via swift- >> evolution wrote:

# Introduction

Add a new `Scoped` protocol and enhance the do statement to
automatically call enter/exit actions on resources.

# Motivation

Resources (e.g., locks, files, sockets, etc.) often need to be
scoped to a block, where some action is taken at the start of the
block and another is required at the end. Examples include locking
and unlocking a lock in a critical section or closing a file at the
end of a block.

Doing this manually is possible using `defer` statements among other
options, but this is error prone as a `defer` can be forgotten,
`lock`/`unlock` calls for two locks can be switched due to a typo,
etc. Having a dedicated language construct for this common case
makes it easier to read and write while making code shorter and
clearer.

# Language Survey

At least three major languages have widely used statements for this
use case.

## C#

C# has the `using` statement and the associated `IDisposable`
interface.

txt = sr.ReadToEnd(); } ```

C#'s solution only handles close/exit actions via the
`IDisposable.Dispose()` method and so cannot easily be used with
items such as locks; however, C# has the additional `lock` statement
for that use case.

## Java

Java has try-with-resources and the associated `AutoCloseable`
interface.

```java try (BufferedReader br = new BufferedReader(new
FileReader\(path\)\)\) \{    return br\.readLine\(\); \} \`\`\`

Java&#39;s solution only handles close/exit actions via the
\`AutoCloseable\.close\(\)\` method and so cannot easily be used with
items such as locks; however, Java has the additional \`synchronized\`
statement for that use case\.

\#\# Python

Python has with \`with\` statement and the associated \`\_\_enter\_\_\` and
\`\_\_exit\_\_\` special methods that classes may implement to become a
&quot;context manager&quot;\.

```python with lock, open(path) as my_file:    contents =
my_file.read()    # use contents ```

Python's solution handles both enter and exit actions and so this
one construct is usable for locks as well as resources like sockets
and files.

# Proposed Solution

We add a new protocol called `Scoped` to the standard library. Types
for resources that have enter/exit actions will be extended to add
conformance to this protocol.

The `do` statement will be extended to allow a new `using
<resources>` "suffix".

# Detailed Design

The `Scoped` protocol shall be as follows:

```swift public protocol Scoped {    func enterScope()    func
exitScope\(\) \} \`\`\`

The compiler statement will accept a new form for resources\. For
example,

```swift do using lock, let file = try getFileHandle() {    //
statements } ```

As can be seen in the example above, the resources can be bindings
that already exist (like `lock`) or can be new bindings. Bindings
created with `do using` are not available outside of the scope of
the `do using`. Only types conforming to `Scoped` may be using with
`do using`. Use of non-conforming types will result in a compiler
error.

The above example would be syntactic sugar for the following:

```swift do {    lock.enterScope()    defer { lock.exitScope() }

let file = try getFileHandle\(\)    file\.enterScope\(\)    defer \{
file\.exitScope\(\) \}

// statements \} \`\`\`

\# Framework Changes / Examples

As an example of some real\-world classes that would be useful with
\`Scoped\`, from Foundation:

\`\`\`swift // Would be nice to extend the NSLocking protocol instead,
but that&#39;s not supported yet\. extension NSLock: Scoped \{    func
enterScope\(\) \{        self\.lock\(\)    \}

func exitScope\(\) \{        self\.unlock\(\)    \} \}

extension NSFileHandle: Scoped \{    func enterScope\(\) \{\}

func exitScope\(\) \{        self\.closeFile\(\)    \} \} \`\`\`

\# Questions and Concerns
\* Bikeshedding protocol naming and scoping syntax \* Should the enter
and exit actions be allowed to throw errors?

\-\-
Trent Nadeau

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
swift\-evolution mailing list swift\-evolution@swift\.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

I disagree that `do using` would be useless for resources. Essentially no
Cocoa resource classes have a `deinit`. NSFileHandle, NSStream, etc.
require a close-like call to free the underlying resource. Also, per the
Swift documentation about deinitialization:

    "Typically you don’t need to perform manual clean-up when your
instances are deallocated. However, when you are working with your own
resources, you might need to perform some additional clean-up yourself. For
example, if you create a custom class to open a file and write some data to
it, you might need to close the file before the class instance is
deallocated."

···

On Wed, Dec 30, 2015 at 1:19 AM, Félix Cloutier <felixcca@yahoo.ca> wrote:

My understanding is that you suggest using the "do using" statement for
two purposes:

   1. to deterministically free resources (files, sockets);
   2. to create a scope with a guarantee about something (locks).

The first purpose is very relevant for garbage-collected languages because
the GC generally only monitors memory pressure to decide when to run, and
so the GC won't budge if you're running out of file descriptors even if
some could be reclaimed. However, Swift is not garbage-collected and
resources are already reclaimed deterministically. If you create a Swift
object that represents a file descriptor and don't allow references to
escape the function, the object will be destroyed (and its resources
reclaimed) at the latest when the function returns. In my opinion, this
makes a "do using" statement useless for resource management.

For scopes with guarantees, as Chris said, the most common pattern is to
have a function that accepts a closure. I've seen some pretty serious
nesting with `if let` (which are one very frequent case of scopes with
guarantees), but other than that, I don't see lots of nesting and I've been
pretty happy with what the language can do so far. The only thing I can
complain about is that you can't use break/continue with the current setup.

I see the discrepancy between objects, but I would say that any
scope-based solution will often be just as hard to discover as the current
solutions. `do using AutoreleasePool() { ... }` isn't an improvement over
`autoreleasepool { ... }`.

It's better for the lock case, but only if you agree that `do using` is
useless for resource management. Otherwise, if your mutex itself benefits
from being scoped, `do using lock` is probably creating/deleting a mutex,
and you'd need to use `do using Lock(mutex) { ... }` to actually lock it
(like with C++ mutex/lock objects), which is as discoverable as `mutex.lock
{ ... }`.

Félix

Le 29 déc. 2015 à 23:24:53, Trent Nadeau via swift-evolution < > swift-evolution@swift.org> a écrit :

While useful, that pattern doesn't seem to compose well. What if you need
two locks? Would that be:

lock1.withThisLockHeld {
    lock2.withThisLockHeld {
        // statements
    }
}

If so, it seems like it has the "pyramid of doom" issue that prompted
allowing `if let` to have multiple bindings.

In addition to the possible indentation and vertical space issue, you need
to look up if and how each resource type does this. I believe this is a
general enough pattern that it deserves language support. I think an
analogy to the current situation would be if each collection type had its
own way to iterate (Array.forEach, Set.withEachElement, etc.) instead of
having for-in.

On Tue, Dec 29, 2015 at 11:15 PM, Chris Lattner <clattner@apple.com> > wrote:

On Dec 29, 2015, at 8:02 PM, Trent Nadeau via swift-evolution < >> swift-evolution@swift.org> wrote:
> Doing this manually is possible using `defer` statements among other
options, but this is error prone as a `defer` can be forgotten,
`lock`/`unlock` calls for two locks can be switched due to a typo, etc.
Having a dedicated language construct for this common case makes it easier
to read and write while making code shorter and clearer.
>
> ```swift
> do {
> lock.enterScope()
> defer { lock.exitScope() }
>
> let file = try getFileHandle()
> file.enterScope()
> defer { file.exitScope() }
>
> // statements
> }

We have another pattern that types can use, which is:

lock.withThisLockHeld {
  … stuff ...
}

This can be done today with trailing closures. Other examples of this
are “autoreleasepool” and withUnsafePointer (for other reasons).

-Chris

--
Trent Nadeau
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
Trent Nadeau

Perhaps, but I feel like it's a more natural extension of our existing model to support uniquely-owned classes though, which would give you all the same benefits.

-Joe

···

On Dec 30, 2015, at 10:26 AM, Chris Lattner <clattner@apple.com> wrote:

On Dec 30, 2015, at 9:53 AM, Joe Groff via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Dec 29, 2015, at 8:55 PM, Kevin Ballard via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

An alternative solution is to do what Rust and C++ do, which is to use RAII. Which is to say, instead of introducing a new language construct that's explicitly tied to a scope, you just use a struct to represent the resource that you hold (e.g. a File that represents an open file). Of course, this does require some changes to structs, notably the addition of a deinit. And if structs have a deinit, then they also need to have a way to restrict copies. This is precisely what Rust does; any struct in Rust that implements Drop (the equivalent to deinit) loses the ability to be implicitly copied (a second trait called Clone provides a .clone() method that is the normal way to copy such non-implicitly-copyable structs).

deinit doesn't make sense for value types.

It would if we extended the model for value types to be richer, e.g. to introduce the notion of "move only” structs.

Another possibility I've thought of is defining `defer { val }` to guarantee that val remains alive until the defer fires on scope exit. That might let us leave `defer` as the one "guarantee something happens exactly at scope exit" language construct.

-Joe

···

On Dec 30, 2015, at 1:27 PM, Kevin Ballard <kevin@sb.org> wrote:

On Wed, Dec 30, 2015, at 09:53 AM, Joe Groff wrote:

On Dec 29, 2015, at 8:55 PM, Kevin Ballard via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

An alternative solution is to do what Rust and C++ do, which is to use RAII. Which is to say, instead of introducing a new language construct that's explicitly tied to a scope, you just use a struct to represent the resource that you hold (e.g. a File that represents an open file). Of course, this does require some changes to structs, notably the addition of a deinit. And if structs have a deinit, then they also need to have a way to restrict copies. This is precisely what Rust does; any struct in Rust that implements Drop (the equivalent to deinit) loses the ability to be implicitly copied (a second trait called Clone provides a .clone() method that is the normal way to copy such non-implicitly-copyable structs).

deinit doesn't make sense for value types. Classes already support deinit, and you can use withExtendedLifetime to bound the lifetime of a resource-holding class. It would be reasonable to have a scoped lifetime marker similar to ObjC ARC too.

If you run with the idea that any resource-holding class should also be the mechanism by which you access the resource (e.g. a LockGuard that represents holding the lock and also provides access to the guarded value) then there's no need for extended lifetimes, because as long as you're accessing the resource, you're keeping the resource-holding class alive. I suppose there might be rare cases where you need to extend the lifetime of a resource-holding class even when you're not accessing the resource, just to guarantee e.g. order of resource releasing, but you can always just say something like `withExtendedLifeetime(&val) {}` at the end of the scope to ensure the value is still alive at that point. Although I'd really like to define `_ = val` as guaranteeing that the value is alive at that point (the expression doesn't actually do anything, but because it references `val` it expresses the programmer's intent that `val` should still be alive at that point in time). Alternatively, if we end up with move-only structs (or uniquely-owned classes), we could even define the expression `_ = val` as "dropping" the value , i.e. forcing it to deinit at that spot (because it's moving the value out of the `val` variable). This would be analogous to Rust's `std::mem::drop()` function (which is literally defined as `pub fn drop<T>(_x: T) { }` because all it does is move the value into the function and then forget about it).

That's completely false. All of the Cocoa classes that manage resources absolutely clean them up in deinit. The explicit call to closeFile() on NSFileHandle exists for two reasons:

1. To let you close the file before throwing away the handle (e.g. if
   you know the handle is being kept alive by something else but still
   want to close the file right now), and
2. Because you can create instances of NSFileHandle that explicitly
   don't close the handle in dealloc (typically because they don't "own"
   the file handle), the closeFile() call lets you change your mind and
   close it anyway.

NSStream doesn't document its deinit behavior, but it definitely closes
itself automatically if it's still open when it deinits.

Also, the quoted passage from the Swift documentation is talking about
why you might need to implement deinit. In your class's deinit you can
close/free any resources that your class manages.

So none of this is at all relevant to any kind of scoped operation.

-Kevin Ballard

···

On Tue, Dec 29, 2015, at 10:45 PM, Trent Nadeau via swift-evolution wrote:

I disagree that `do using` would be useless for resources. Essentially
no Cocoa resource classes have a `deinit`. NSFileHandle, NSStream,
etc. require a close-like call to free the underlying resource. Also,
per the Swift documentation about deinitialization:

"Typically you don’t need to perform manual clean-up when your
instances are deallocated. However, when you are working with your own
resources, you might need to perform some additional clean-up
yourself. For example, if you create a custom class to open a file and
write some data to it, you might need to close the file before the
class instance is deallocated."

On Wed, Dec 30, 2015 at 1:19 AM, Félix Cloutier > <felixcca@yahoo.ca> wrote:

My understanding is that you suggest using the "do using" statement
for two purposes:

1. to deterministically free resources (files, sockets);
2. to create a scope with a guarantee about something (locks).

The first purpose is very relevant for garbage-collected languages
because the GC generally only monitors memory pressure to decide when
to run, and so the GC won't budge if you're running out of file
descriptors even if some could be reclaimed. However, Swift is not
garbage-collected and resources are already reclaimed
deterministically. If you create a Swift object that represents a
file descriptor and don't allow references to escape the function,
the object will be destroyed (and its resources reclaimed) at the
latest when the function returns. In my opinion, this makes a "do
using" statement useless for resource management.

For scopes with guarantees, as Chris said, the most common pattern is
to have a function that accepts a closure. I've seen some pretty
serious nesting with `if let` (which are one very frequent case of
scopes with guarantees), but other than that, I don't see lots of
nesting and I've been pretty happy with what the language can do so
far. The only thing I can complain about is that you can't use
break/continue with the current setup.

I see the discrepancy between objects, but I would say that any scope-
based solution will often be just as hard to discover as the current
solutions. `do using AutoreleasePool() { ... }` isn't an improvement
over `autoreleasepool { ... }`.

It's better for the lock case, but only if you agree that `do using`
is useless for resource management. Otherwise, if your mutex itself
benefits from being scoped, `do using lock` is probably
creating/deleting a mutex, and you'd need to use `do using
Lock(mutex) { ... }` to actually lock it (like with C++ mutex/lock
objects), which is as discoverable as `mutex.lock { ... }`.

Félix

Le 29 déc. 2015 à 23:24:53, Trent Nadeau via swift-evolution <swift- >>> evolution@swift.org> a écrit :

While useful, that pattern doesn't seem to compose well. What if you
need two locks? Would that be:

lock1.withThisLockHeld { lock2.withThisLockHeld { //
statements } }

If so, it seems like it has the "pyramid of doom" issue that
prompted allowing `if let` to have multiple bindings.

In addition to the possible indentation and vertical space issue,
you need to look up if and how each resource type does this. I
believe this is a general enough pattern that it deserves language
support. I think an analogy to the current situation would be if
each collection type had its own way to iterate (Array.forEach,
Set.withEachElement, etc.) instead of having for-in.

On Tue, Dec 29, 2015 at 11:15 PM, Chris >>> Lattner<clattner@apple.com>wrote:

On Dec 29, 2015, at 8:02 PM, Trent Nadeau via swift-evolution <swift- >>>> evolution@swift.org> wrote:
> Doing this manually is possible using `defer` statements among
> other options, but this is error prone as a `defer` can be
> forgotten, `lock`/`unlock` calls for two locks can be switched
> due to a typo, etc. Having a dedicated language construct for
> this common case makes it easier to read and write while making
> code shorter and clearer.
>
> ```swift do { lock.enterScope() defer {
> lock.exitScope() }
>
>let file = try getFileHandle() file.enterScope() defer {
>file.exitScope() }
>
>// statements }

We have another pattern that types can use, which is:

lock.withThisLockHeld { … stuff ... }

This can be done today with trailing closures. Other examples of
this are “autoreleasepool” and withUnsafePointer (for other
reasons).

-Chris

--
Trent Nadeau
_______________________________________________
swift-evolution mailing list swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
Trent Nadeau

_________________________________________________
swift-evolution mailing list swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

+1 to Joe’s comment.

···

On Dec 30, 2015, at 10:31 AM, Joe Groff via swift-evolution <swift-evolution@swift.org> wrote:

On Dec 30, 2015, at 10:26 AM, Chris Lattner <clattner@apple.com <mailto:clattner@apple.com>> wrote:

On Dec 30, 2015, at 9:53 AM, Joe Groff via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Dec 29, 2015, at 8:55 PM, Kevin Ballard via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

An alternative solution is to do what Rust and C++ do, which is to use RAII. Which is to say, instead of introducing a new language construct that's explicitly tied to a scope, you just use a struct to represent the resource that you hold (e.g. a File that represents an open file). Of course, this does require some changes to structs, notably the addition of a deinit. And if structs have a deinit, then they also need to have a way to restrict copies. This is precisely what Rust does; any struct in Rust that implements Drop (the equivalent to deinit) loses the ability to be implicitly copied (a second trait called Clone provides a .clone() method that is the normal way to copy such non-implicitly-copyable structs).

deinit doesn't make sense for value types.

It would if we extended the model for value types to be richer, e.g. to introduce the notion of "move only” structs.

Perhaps, but I feel like it's a more natural extension of our existing model to support uniquely-owned classes though, which would give you all the same benefits.

-Joe

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

So long as it guarantees no heap allocation for the class instance, ok.

-Chris

···

On Dec 30, 2015, at 10:31 AM, Joe Groff <jgroff@apple.com> wrote:

On Dec 30, 2015, at 10:26 AM, Chris Lattner <clattner@apple.com <mailto:clattner@apple.com>> wrote:

On Dec 30, 2015, at 9:53 AM, Joe Groff via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Dec 29, 2015, at 8:55 PM, Kevin Ballard via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

An alternative solution is to do what Rust and C++ do, which is to use RAII. Which is to say, instead of introducing a new language construct that's explicitly tied to a scope, you just use a struct to represent the resource that you hold (e.g. a File that represents an open file). Of course, this does require some changes to structs, notably the addition of a deinit. And if structs have a deinit, then they also need to have a way to restrict copies. This is precisely what Rust does; any struct in Rust that implements Drop (the equivalent to deinit) loses the ability to be implicitly copied (a second trait called Clone provides a .clone() method that is the normal way to copy such non-implicitly-copyable structs).

deinit doesn't make sense for value types.

It would if we extended the model for value types to be richer, e.g. to introduce the notion of "move only” structs.

Perhaps, but I feel like it's a more natural extension of our existing model to support uniquely-owned classes though, which would give you all the same benefits.