Why doesn't the example violate SE-0176 (Exclusive Access to Memory)?

Using Swift
1

Hi, I find this example compiles and works. I wonder why it doesn't violate SE-0176?

struct Foo {
    var x: Int = 0
    var y: Int = 0

    mutating func modifyX() {
        x = 1
        modifyY()
    }

    mutating func modifyY() {
        y = 1
    }
}

var f = Foo()
f.modifyX()
print("\(f.x), \(f.y)")  
// Output: 1, 1

The following is from SE-0176

Calling a method on a value type is an access to the entire value: a write if it's a mutating method, a read otherwise. This is because we have to assume that a method might read or write an arbitrary part of the value. Trying to formalize rules like "this method only uses these properties" would massively complicate the language.

So both modifyX() and modifyY() in above example have write access to the entire value. If so, how is it possible to call modifyY() from within modifyX()?

Am I misunderstanding something here? Thanks.

My environment: XCode 12.5.1, Swift 5.4.2

2 Likes
2

modifyX cannot access self while it calls modifyY, so exclusive access to memory is enforced.

5 Likes
3

@1-877-547-7272 Thanks, but I doubt it. If what you said is true, it should be unlikely to run into exclusive memory access error in single thread code. But I'm quite sure that's not the case.

SE-176 described non-instantaneous access as below:

For example, when you call a mutating method on a stored property, it's really one long access to the property: self just becomes another way of referring to the property's storage. This access isn't instantaneous because all of the code in the method executes during it, ...

Then it continued to describe overlap:

... so if that code manages to access the same property again, the accesses will overlap.so if that code manages to access the same property again, the accesses will overlap.

I think my example code is similar. The only difference is that in the above description it's different code modifying the same property, but in my example it's different code modifying self, which is the entire value. So I think my example has the same issue and shouldn't compile.

4

The first answer is effectively correct, but if you want something more technical, remember that mutating is equivalent to inout on self, and inout is formally “copy-in, copy-out”. So the Law of Exclusivity says that f is being accessed for the entire duration of the call to modifyX. But within modifyX, self is an independent value that was initialized from f. Then that value is accessed for the duration of the call to modifyY, being copied in to another self.

The way you get an exclusivity violation on one thread is by referring back to the original storage, which can only happen if it’s a global or if there’s a reference type involved (class or closure).

15 Likes
5

Thanks! That's the exactly the answer I'm looking for. I knew how inout works, but I used to just think it as a reference, which doesn't work in this case. That's an important takeaway for me from this discussion.

Also, after reading your explanation, I realize I completely misunderstood SE-176. I used to think its purpose is to prevent logical error by disallowing conflict access on the same value on concept level. Now I realize it doesn't do that at all. It works on a lower level (memory address). For example, I used to think the following code doesn't compile (I got the wrong impression from the document), but now I realize it actually works.

struct Foo {
    var x: Int = 0
    
    mutating func modifyX() {
        x = 1
        modifyXAgain()
    }
    
    mutating func modifyXAgain() {
        x = 2
    }
}

var f = Foo()
f.modifyX()
print("\(f.x)")

So SE-176 is not as restrictive as I had thought.

That all makes sense now. Thanks again!

1 Like
6

Hi @jrose May I ask how the fact that closure is reference type could cause the issue? The code example below does prove what you said, but I can't figure out why.

In the example the closure captures the mutable self implicitly. My guess is that's implemented by using "copy-in, copy-out" also. That is, when the closure is called, the compiler passes a copy of self to the closure; when the closure exits, the compiler copies the modified self in the closure back to the self in modifyX(). Is this understanding correct? If so, then modifyX() and the cloure work on different copies of self and shouldn't run into memory exclusive access issue. What am I misunderstanding here? Thanks.

struct Foo {
    var x: Int = 0
    var y: Int = 0

    mutating func modifyX(_ modifyOtherProperty: () -> Void) {
        x = 1
        modifyOtherProperty()
    }
    
    mutating func modifyXAndY() {
        modifyX {
            // Compile Error: Overlapping accesses to 'self', but modification requires exclusive access
            self.y = 1
        }
    }
}

var f = Foo()
f.modifyXAndY()
print("\(f.x), \(f.y)")
7

That is not correct.

Closures capture variables by reference (unless the variable is included in a capture list).

2 Likes
8

That explains it. Thanks.

9

Formally, the storage you pass to an inout parameter is considered accessed during the call. Within the callee, the inout parameter is introduced as separate storage, which happens to share its value with the original storage.

You can think of that as “copy-in, copy-out” if you like, pretending that the inout parameter is actually a variable that gets initialized. It gives a very intuitive explanation of what happens with computed get/set storage. On the other hand, it’s misleading when the storage is actually stored, because it suggests that there are real copies that the compiler has to work to eliminate. And in some ways it’s especially misleading with exclusivity, because you might think that it ought to turn into one instantaneous read later followed by an instantaneous write, as opposed to a prolonged access.

Ultimately, there’s no conflict in your example because the self parameter in modifyX is formally different from the original variable and because within modifyX the accesses to self do not overlap. The call to modifyY performs a non-instantaneous access to the self in modifyX; as before, the self in modifyY is formally different storage from both the other self and the original variable.

This pattern of nested accesses ensuring that you can locally reason about local variables and parameters is the key principle of why exclusivity works.

10 Likes
10

Thanks, that clarifies a lot!

I wasn't sure about what you meant at first, but then I found more details in Swift reference manual. So if I understand it correctly, as a result of optimization, the original self, the self in modifyX(), and the self in modifyY() may be at the same memory address. But that's just an implementation detail. Formally they are considered separate storages when the compiler checks exclusivity.

Yes, it's indeed simpler to consider the entire mutating method as a single prolonged write access to the copy of the self owned by that method by its caller when reasoning about exclusivity. On the other hand, however, since each mutating method is considered to have its own storage of self, I suppose there needs a way to think about how the changes to each copy of self accumulate. For that purpose, I find the "copy in, copy out" works well for me.

I suppose modifyX is a typo and should be modifyY?

Got it. Thanks!

11

No. modifyX contains the call modifyY(), which is really self.modifyY(). self refers to the implicit parameter of modifyX, which is accessed for the duration of this call. Within modifyY, of course, there is another implicit parameter named self, which is accessed by the assignment statement there.

1 Like
12

Hi John, I find an example where the "copy-in, copy-out" approach can explain why it works but the "prolonged access" approach you recommended seems to suggest it shouldn't.

struct Foo {
    var x: Int = 0

    func modifyX(_ foo: inout Foo) {
        foo.x = 1
        print("In modifyX(): \(self.x)")
        print("In modifyX(): \(foo.x)")
    }
}

var foo = Foo()
foo.modifyX(&foo)
print("After modifyX(): \(foo.x)")

// Output:
// In modifyX(): 0
// In modifyX(): 1
// After modifyX(): 1

The modifyX() has read access to the globle variable foo through the implicit immutable self. It also has write access to the global variable foo through the inout foo parameter. If I understand it correctly, that's an overlap and the issue should be caught at runtime, right? But the code actually runs well.

13

I don't think this is a problem because the function is not mutating and thus self just stands for the value before any modifications.

1 Like
14

The issue is that the code in modifyX() gets inconsistent view of the value of global variable foo, through the immutable self and inout parameter foo, respectively. My understanding is that the issue in this case is similar to the swap(&x, &x) example in SE-176. The only difference is that one is read/write overlap and another is write/write overlap.

15

It is not inconsistent. As far as modifyX() is concerned, there is no relation between self and its argument value foo. So, there is no inconsistency. If it behaved otherwise, it would be a compiler bug. This is how a value type is supposed to behave: Each time you pass a value type, it is logically copied. In the case of inout, it is first copied into foo argument, then copied back to foo global upon function return and not before it returns. (Also note: What physically happens is an implementation detail and subject to various optimizations and may change in a future compiler version, but the logical behavior stays the same.)

Here is how to trigger exclusivity violation that compiler can't statically detect yet:

struct Foo {
    var x: Int = 0

    func modifyX(_ foo: inout Foo, action: ()->Void ) {
        foo.x = 1
        action()
    }
}

var foo = Foo()
foo.modifyX(&foo) { foo.x = 2 } // Runtime crash: Fatal access conflict detected

EDIT: Also, note that the following code is not exclusivity violation and behaves the same as your example:

    func modifyX(action: ()->Void ) {
        action()
        print("Inside: self.x=\(x)")
    }

Because self is logically copied into the method. Remember that self is a normal argument to unapplied method Foo.modifyX(_ self: Foo)(action: ()->Void).

2 Likes
16

Thanks, I understand all what you said, but I still doubt if it's the right behavior. I think my confusion can be boiled down to a simple question: is it OK to modify a value in a non-mutating method in Swift? I thought it shouldn't, based on the following statement in SE-0176:

We should add a rule to Swift that two accesses to the same variable are not allowed to overlap unless both accesses are reads. By "variable", we mean any kind of mutable memory: global variables, local variables, class and struct properties, and so on.

but the examples show that it's OK and hence my confusion.

The above statement contains an important detail about the definition of the "variable" in the document. If I understand it correctly, it basically means memory location. I think that's the part that makes SE-1076 a bit hard to understand for app developers, because we don't deal with memory directly in Swift. Instead we deal with value copy when doing read-only access or "copy in, copy out" when doing write access. And it's a bit hard to determine which is considered as "accessing same memory location" and which isn't.

For example, I just need to modify my example a bit (not using closure) and it violates SE-0176 immediately:

struct Foo {
    var x: Int = 0

-   func modifyX(_ foo: inout Foo) {
+   mutating func modifyX(_ foo: inout Foo) {
        foo.x = 1
        print("In modifyX(): \(self.x)")
        print("In modifyX(): \(foo.x)")
    }
}

var foo = Foo()
foo.modifyX(&foo)
print("After modifyX(): \(foo.x)")

While I like SE-0176 (it's a good constraint helping to write robust code), it's an artificial constraint. I mean, if we wrote the same code in c it could work. Also, I guess the same code might work in earlier versions of Swift before SE-0176 was added. So, just because the example works doesn't necessarily mean it should. That's the reason I raised the question.

Background: why I try to understnad SE-0176

At first I tried to understand SE-0176 because I ran into compilation or runtime errors caused by it. I passed that stage soon, but I found another issue: I often misunderstood it. I thought some code were invalid but they weren't. Confusion like this makes it hard for me to program with value types because it's not clear what's allowed and what isn't. That's the reason why I started the thread.

17

The key difference between these two lines is that in the former, self stands for the immutable value of foo as of the time before entering the function for the entire body of it, whereas in the latter, self is mutable just like the inout argument—which is not allowed (when they refer to the same variable).

2 Likes
18

I'm afraid I don't agree your interpretation. Since it's non-mutating method, why it has to stand for the value of the global variable foo before entering the function? Shouldn't it be the value of the global variable foo throughout the execution of the modifyX()?

19

Yes.

The nonmutating version of modifyX is roughly equivalent to the global function

func modifyX(_ self: Foo, _ foo: inout Foo)

while the mutating version of modifyX is roughly equivalent to

func modifyX(_ self: inout Foo, _ foo: inout Foo)

. There are no exclusivity violations if self and foo are the same in the first function since only foo is passed by reference. However, the second function does violate exclusivity if self and foo are the same because both self and foo are passed by reference.

The point of the law of exclusivity is to make it such that each variable has its own contents. If two variables stored the same contents, things could get confusing. If you wrote the following code:

func bar(_ x: inout Int, _ y: inout Int) {
    x += 1
    y += 1
}

you’d probably expect to be mutating two different variables. But if you passed the same variable for both parameters (i.e. bar(&baz, &baz)), then x and y would refer to the same variable and you’d mutate the same instance twice. The law of exclusivity prevents this unexpected behavior.

Allowing variables to alias each other can also affect the compiler’s ability to optimize your code (see the “Aliasing” section of this blog post if you want more details).

3 Likes
20

Thanks for the clear answer. I was about to make a note of this behavior and just consider a non-mutating method as an instantaneous read to the original self, but then I find an example in SE-0176:

// CONFLICT.  Calling a non-mutating method on 'array[0]' performs a
// read access to it, and thus to 'array', for the duration of the method.
// Calling a mutating method on 'array[1]' performs a write access to it,
// and thus to 'array', for the duration of the method.  These accesses
// therefore conflict.
array[0].forEach { array[1].append($0) }

The document says this example violates the law of exclusivity, but it works well with Swift 5.5.

I wonder what the right behavior should be?

The purpose of accessing array[0] and array[1] in the document's example is to demonstrate reading/writing an item is equivalent to reading/writing the entire array. If we remove that factor, I think @John_McCall proposed the following code should be invalid too:

array[0].forEach { array[0].append($0) }

That is very similar to the the example in my question: invoking a non-mutating function on a value and changing the original value somehow inside the function.