[Review] SE-0176: Enforce Exclusive Access to Memory

Evolution Announcements
1

Hello Swift community,

The review of SE-0176: "Enforce Exclusive Access to Memory" begins now and runs through May 8, 2017.

The proposal is available here:

Reviews are an important part of the Swift evolution process. All reviews should be sent to the swift-evolution mailing list at:

https://lists.swift.org/mailman/listinfo/swift-evolution
or, if you would like to keep your feedback private, directly to the review manager.

When replying, please try to keep the proposal link at the top of the message:

Proposal link:

Reply text

Other replies

<GitHub - apple/swift-evolution: This maintains proposals for changes and user-visible enhancements to the Swift Programming Language.;
What goes into a review?

The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of Swift. When writing your review, here are some questions you might want to answer in your review:

What is your evaluation of the proposal?
Is the problem being addressed significant enough to warrant a change to Swift?
Does this proposal fit well with the feel and direction of Swift?
If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
How much effort did you put into your review? A glance, a quick reading, or an in-depth study?
More information about the Swift evolution process is available at:

Thanks,
Ben Cohen
Review Manager

2

+1. Yes please.

Austin

···

On Tue, May 2, 2017 at 1:07 PM, Ben Cohen <ben_cohen@apple.com> wrote:

Hello Swift community,

The review of SE-0176: "Enforce Exclusive Access to Memory" begins now
and runs through May 8, 2017.

The proposal is available here:
GitHub - apple/swift-evolution: This maintains proposals for changes and user-visible enhancements to the Swift Programming Language.
proposals/0176-enforce-exclusive-access-to-memory.md

Reviews are an important part of the Swift evolution process. All reviews
should be sent to the swift-evolution mailing list at:

https://lists.swift.org/mailman/listinfo/swift-evolution

or, if you would like to keep your feedback private, directly to the
review manager.

When replying, please try to keep the proposal link at the top of the
message:

Proposal link:
GitHub - apple/swift-evolution: This maintains proposals for changes and user-visible enhancements to the Swift Programming Language.
proposals/0176-enforce-exclusive-access-to-memory.md
Reply text

Other replies

<GitHub - apple/swift-evolution: This maintains proposals for changes and user-visible enhancements to the Swift Programming Language.;

What goes into a review?

The goal of the review process is to improve the proposal under review
through constructive criticism and, eventually, determine the direction of
Swift. When writing your review, here are some questions you might want to
answer in your review:

   - What is your evaluation of the proposal?
   - Is the problem being addressed significant enough to warrant a
   change to Swift?
   - Does this proposal fit well with the feel and direction of Swift?
   - If you have used other languages or libraries with a similar
   feature, how do you feel that this proposal compares to those?
   - How much effort did you put into your review? A glance, a quick
   reading, or an in-depth study?

More information about the Swift evolution process is available at:

https://github.com/apple/swift-evolution/blob/master/process.md

Thanks,
Ben Cohen
Review Manager

_______________________________________________
swift-evolution-announce mailing list
swift-evolution-announce@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution-announce

3

I’m not a language or memory expert, so I think the proposal could use a few more examples of the code it would disallow. Perhaps examples from real world projects? Without being an expert I’m having a hard time seeing the full impact of the proposed rules.
  Also, the @exclusivity syntax doesn’t seem fully defined. Is there a @exclusivity(checked) counterpart? Would subclasses be allowed to mark inherited properties more or less strictly? Also, is the logic for @ attributes and keyword attributes documented anywhere? Otherwise it seems like exclusive / nonexclusive would be an option.

Jon

···

On May 2, 2017, at 4:07 PM, Ben Cohen via swift-evolution <swift-evolution@swift.org> wrote:

Hello Swift community,

The review of SE-0176: "Enforce Exclusive Access to Memory" begins now and runs through May 8, 2017.

The proposal is available here:

https://github.com/apple/swift-evolution/blob/master/proposals/0176-enforce-exclusive-access-to-memory.md
Reviews are an important part of the Swift evolution process. All reviews should be sent to the swift-evolution mailing list at:

https://lists.swift.org/mailman/listinfo/swift-evolution
or, if you would like to keep your feedback private, directly to the review manager.

When replying, please try to keep the proposal link at the top of the message:

Proposal link:

https://github.com/apple/swift-evolution/blob/master/proposals/0176-enforce-exclusive-access-to-memory.md
Reply text

Other replies

<GitHub - apple/swift-evolution: This maintains proposals for changes and user-visible enhancements to the Swift Programming Language.;
What goes into a review?

The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of Swift. When writing your review, here are some questions you might want to answer in your review:

What is your evaluation of the proposal?
Is the problem being addressed significant enough to warrant a change to Swift?
Does this proposal fit well with the feel and direction of Swift?
If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
How much effort did you put into your review? A glance, a quick reading, or an in-depth study?
More information about the Swift evolution process is available at:

https://github.com/apple/swift-evolution/blob/master/process.md

Thanks,
Ben Cohen
Review Manager

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

4

I like the spirit of the proposal. It leaves me a bit uneasy because the practical ramifications are difficult to fully anticipate. Here follow some questions / concerns for the proposal authors.

(Apologies if these were answered somewhere deep in a discussion thread; it’s the end of the semester and I’ve not kept up with the list.)

• • •

First, have you done any systematic exploration of how impactful this rule is? The proposal states in several places that it hopes/expects violations will be rare, but is that empirical or speculative?

I tried playing around with the implications, thought I’d found a problematic example by writing an in-place Array.shuffle() method — and then was pleased to realize that the proposal already covered my example with swapAt(). In half an hour, I couldn’t come up with something the proposal disallows that it clearly should allow.

Still, the implications of a rule like this are difficult to anticipate. It would be interesting to know how many errors it flags in the Swift source compatibility suite, for example. Is that something we could do before going all-in on a fully robust implementation?

• • •

Second, what are the performance implications of dynamic exclusivity enforcement? Introducing overhead to something as basic as variable access seems dicey. Will it kill “C-like” performance for object properties? For closure vars? Will “accesses underway” flags add to the memory footprint of every object & every closure? You write that your “hope is that…dynamic access-tracking [will be] cheap enough to enable by default;” have you proved this out with a prototype implementation yet?

Or maybe this check is disabled in optimized builds, and I’m fussing over nothing?

Cheers,

Paul

···

On May 2, 2017, at 3:07 PM, Ben Cohen <ben_cohen@apple.com> wrote:

Hello Swift community,

The review of SE-0176: "Enforce Exclusive Access to Memory" begins now and runs through May 8, 2017.

The proposal is available here:

https://github.com/apple/swift-evolution/blob/master/proposals/0176-enforce-exclusive-access-to-memory.md
Reviews are an important part of the Swift evolution process. All reviews should be sent to the swift-evolution mailing list at:

https://lists.swift.org/mailman/listinfo/swift-evolution
or, if you would like to keep your feedback private, directly to the review manager.

When replying, please try to keep the proposal link at the top of the message:

Proposal link:

https://github.com/apple/swift-evolution/blob/master/proposals/0176-enforce-exclusive-access-to-memory.md
Reply text

Other replies

<GitHub - apple/swift-evolution: This maintains proposals for changes and user-visible enhancements to the Swift Programming Language.;
What goes into a review?

The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of Swift. When writing your review, here are some questions you might want to answer in your review:

What is your evaluation of the proposal?
Is the problem being addressed significant enough to warrant a change to Swift?
Does this proposal fit well with the feel and direction of Swift?
If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
How much effort did you put into your review? A glance, a quick reading, or an in-depth study?
More information about the Swift evolution process is available at:

https://github.com/apple/swift-evolution/blob/master/process.md

Thanks,
Ben Cohen
Review Manager

_______________________________________________
swift-evolution-announce mailing list
swift-evolution-announce@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution-announce

5

review of SE-0176: "Enforce Exclusive Access to Memory"

What is your evaluation of the proposal?

+1, not only will the code be better optimised but also the code will be easier to understand. It is also nice that there is formal clarity about what the compiler should do. Proposal should probably be expanded to require an annotation to mark when part of a value is aliased, e.g. a method that returns a slice of an array should be annotated to mark that an access to the returned slice will add exclusivity to the original array as well as the slice itself.

Is the problem being addressed significant enough to warrant a change to Swift?

Yes, Swift is meant to be an easy to use and safe language with good performance. These changes improve all these aspects.

Does this proposal fit well with the feel and direction of Swift?

Yes, brings clarity and sets Swift up well for the future.

If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?

Not exactly and not as strong as these guarantees at the compiler level. Other languages with a runtime VM do offer such guarantees dynamically, e.g. Java. The java compiler codes conservative code and then the JVM will optimise that code once running and it can be proved to be safe. Obviously in Swift with optimisation happening at compile time this isn't possible.

How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

Have had problems with accessing sub-sections of arrays, which whilst safe was very slow, presumable because of conservative coding by the compiler.

-- Howard.

···

On 3 May 2017, at 6:07 am, Ben Cohen <ben_cohen@apple.com> wrote:

Hello Swift community,

The review of SE-0176: "Enforce Exclusive Access to Memory" begins now and runs through May 8, 2017.

The proposal is available here:

https://github.com/apple/swift-evolution/blob/master/proposals/0176-enforce-exclusive-access-to-memory.md
Reviews are an important part of the Swift evolution process. All reviews should be sent to the swift-evolution mailing list at:

https://lists.swift.org/mailman/listinfo/swift-evolution

or, if you would like to keep your feedback private, directly to the review manager.

When replying, please try to keep the proposal link at the top of the message:

Proposal link:

https://github.com/apple/swift-evolution/blob/master/proposals/0176-enforce-exclusive-access-to-memory.md
Reply text

Other replies

What goes into a review?

The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of Swift. When writing your review, here are some questions you might want to answer in your review:

What is your evaluation of the proposal?
Is the problem being addressed significant enough to warrant a change to Swift?
Does this proposal fit well with the feel and direction of Swift?
If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
How much effort did you put into your review? A glance, a quick reading, or an in-depth study?
More information about the Swift evolution process is available at:

https://github.com/apple/swift-evolution/blob/master/process.md

Thanks,
Ben Cohen
Review Manager

_______________________________________________
swift-evolution-announce mailing list
swift-evolution-announce@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution-announce

6

• What is your evaluation of the proposal?

+1. However, it's unclear to me what the dynamic enforcement will look like: will it trap, emit a warning, be catchable in a debugger, ... More details on the developer-facing interface of the dynamic enforcement would be nice.

• Is the problem being addressed significant enough to warrant a change to Swift?

The problem is significant, however I can see two significant downsides. The first is the source-breaking nature of the proposal. Breaking source is a problem by itself, but here I'm afraid the errors reporting won't be easily understandable, because "exclusivity" is kind of an advanced feature that won't be easily grasped by developers.

My second concern is the performance of the dynamic enforcement. How confident are you that the performance hit of the enforcement will not nullify the gain made by the enabling of more compile-time optimisations?

• Does this proposal fit well with the feel and direction of Swift?

Yes.

• If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
• How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

Full read. BTW, there is a typo in the "Eliminating non-instantaneous accesses?" section, _Int_appendABunchOfStuff => _Array_appendABunchOfStuff

···

More information about the Swift evolution process is available at:

https://github.com/apple/swift-evolution/blob/master/process.md

Thanks,
Ben Cohen
Review Manager

_______________________________________________
swift-evolution-announce mailing list
swift-evolution-announce@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution-announce

--
Florent Bruneau

7

https://github.com/apple/swift-evolution/blob/master/proposals/0176-enforce-exclusive-access-to-memory.md

What is your evaluation of the proposal?

It's a bit hard to evaluate without experience. But I would summarize the benefits like this:

1. Clearer semantics inside functions. Variables that can alias each other makes things hard to reason about. Generally, this we just disregard the possibility of this happening. Having the language guard against this is going to make it easier to reason about the effects of the code.

2. Better optimizations. Variable that can alias each other force the optimizer to be pessimistic. All that to support cases that probably weren't well though out when the code was written because the semantics are hard to follow.

I find the first point about clearer semantics most interesting.

Is the problem being addressed significant enough to warrant a change to Swift?

Looks like something that should be done. Swift already disallows that I think, but it only checks for the obvious cases.

Providing guaranties about exclusivity opens the road to a lot of useful things for future evolution of the language.

Does this proposal fit well with the feel and direction of Swift?

Given that the alternative option to dynamic checking is to have a complex set of annotations to track everything statically, I think the proposed approach makes sense.

If I understand well however, allowing the optimizer to be more optimistic about exclusivity is going to be make memory corruption more likely in cases the exclusivity rule is violated for types that contain pointers. I'm thus a bit weary of the idea of disabling the runtime checks in production by default while at the same time enabling the optimizations that depends on this unverified exclusivity.

If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?

This reminds me of `restrict` in C, which can be dangerous if used incorrectly. Having a checking mechanism for exclusivity removes those concerns.

How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

Read the ownership manifesto, the proposal, the discussion about the proposal.

···

--
Michel Fortin
https://michelf.ca

8

The proposal looks good in general. Like some others, my chief concerns are
around dynamic enforcement. If it's not certain to be cheap enough to be
enabled without per-member opt-out and possibly build settings to turn it
off, perhaps the way to go is to add static enforcement now and prototype
what the cost will be for dynamic enforcement before designing all of these
knobs and switches.

···

On Tue, May 2, 2017 at 15:08 Ben Cohen via swift-evolution < swift-evolution@swift.org> wrote:

Hello Swift community,

The review of SE-0176: "Enforce Exclusive Access to Memory" begins now
and runs through May 8, 2017.

The proposal is available here:

https://github.com/apple/swift-evolution/blob/master/proposals/0176-enforce-exclusive-access-to-memory.md

Reviews are an important part of the Swift evolution process. All reviews
should be sent to the swift-evolution mailing list at:

https://lists.swift.org/mailman/listinfo/swift-evolution

or, if you would like to keep your feedback private, directly to the
review manager.

When replying, please try to keep the proposal link at the top of the
message:

Proposal link:

https://github.com/apple/swift-evolution/blob/master/proposals/0176-enforce-exclusive-access-to-memory.md
Reply text

Other replies

<GitHub - apple/swift-evolution: This maintains proposals for changes and user-visible enhancements to the Swift Programming Language.;

What goes into a review?

The goal of the review process is to improve the proposal under review
through constructive criticism and, eventually, determine the direction of
Swift. When writing your review, here are some questions you might want to
answer in your review:

   - What is your evaluation of the proposal?
   - Is the problem being addressed significant enough to warrant a
   change to Swift?
   - Does this proposal fit well with the feel and direction of Swift?
   - If you have used other languages or libraries with a similar
   feature, how do you feel that this proposal compares to those?
   - How much effort did you put into your review? A glance, a quick
   reading, or an in-depth study?

More information about the Swift evolution process is available at:

https://github.com/apple/swift-evolution/blob/master/process.md

Thanks,
Ben Cohen
Review Manager

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

9

I have one concern and one general comment.

General comment: there's a switch to turn off overflow checks for performance reasons. It seems to me that this check is going to be much more expensive than an overflow check, and it's certainly not much more likely to trigger in a release build after some amount of testing, so my opinion would be that there should be a way to turn it off globally. To me, the checking is UBSan behavior more than a language feature.

Concern: `swap` is quoted a lot for a method that would break under this rule, but as it happens, `swap` with the same value on both sides is (should be) a no-op. Is there a way to not trip the static or dynamic checkers in well-defined cases like that one? Is there any way to check that two inout parameters refer to the same value?

···

Le 2 mai 2017 à 13:07, Ben Cohen via swift-evolution <swift-evolution@swift.org> a écrit :

Hello Swift community,

The review of SE-0176: "Enforce Exclusive Access to Memory" begins now and runs through May 8, 2017.

The proposal is available here:

https://github.com/apple/swift-evolution/blob/master/proposals/0176-enforce-exclusive-access-to-memory.md
Reviews are an important part of the Swift evolution process. All reviews should be sent to the swift-evolution mailing list at:

https://lists.swift.org/mailman/listinfo/swift-evolution
or, if you would like to keep your feedback private, directly to the review manager.

When replying, please try to keep the proposal link at the top of the message:

Proposal link:

https://github.com/apple/swift-evolution/blob/master/proposals/0176-enforce-exclusive-access-to-memory.md
Reply text

Other replies

<GitHub - apple/swift-evolution: This maintains proposals for changes and user-visible enhancements to the Swift Programming Language.;
What goes into a review?

The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of Swift. When writing your review, here are some questions you might want to answer in your review:

What is your evaluation of the proposal?
Is the problem being addressed significant enough to warrant a change to Swift?
Does this proposal fit well with the feel and direction of Swift?
If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
How much effort did you put into your review? A glance, a quick reading, or an in-depth study?
More information about the Swift evolution process is available at:

https://github.com/apple/swift-evolution/blob/master/process.md

Thanks,
Ben Cohen
Review Manager

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

10

- What is your evaluation of the proposal? +1
- Is the problem being addressed significant enough to warrant a change to
Swift?Yes
- Does this proposal fit well with the feel and direction of Swift?I think
so.
- If you have used other languages or libraries with a similar feature, how
do you feel that this proposal compares to those?I have only looked into
rust's ability to track a similar attribute and I prefer Swift's approach
on the face of it.
- How much effort did you put into your review? A glance, a quick reading,
or an in-depth study?

···

On Tue, May 2, 2017 at 4:07 PM, Ben Cohen via swift-evolution < swift-evolution@swift.org> wrote:

Hello Swift community,

The review of SE-0176: "Enforce Exclusive Access to Memory" begins now
and runs through May 8, 2017.

The proposal is available here:
GitHub - apple/swift-evolution: This maintains proposals for changes and user-visible enhancements to the Swift Programming Language.
proposals/0176-enforce-exclusive-access-to-memory.md

Reviews are an important part of the Swift evolution process. All reviews
should be sent to the swift-evolution mailing list at:

https://lists.swift.org/mailman/listinfo/swift-evolution

or, if you would like to keep your feedback private, directly to the
review manager.

When replying, please try to keep the proposal link at the top of the
message:

Proposal link:
GitHub - apple/swift-evolution: This maintains proposals for changes and user-visible enhancements to the Swift Programming Language.
proposals/0176-enforce-exclusive-access-to-memory.md
Reply text

Other replies

<GitHub - apple/swift-evolution: This maintains proposals for changes and user-visible enhancements to the Swift Programming Language.;

What goes into a review?

The goal of the review process is to improve the proposal under review
through constructive criticism and, eventually, determine the direction of
Swift. When writing your review, here are some questions you might want to
answer in your review:

   - What is your evaluation of the proposal?
   - Is the problem being addressed significant enough to warrant a
   change to Swift?
   - Does this proposal fit well with the feel and direction of Swift?
   - If you have used other languages or libraries with a similar
   feature, how do you feel that this proposal compares to those?
   - How much effort did you put into your review? A glance, a quick
   reading, or an in-depth study?

More information about the Swift evolution process is available at:

https://github.com/apple/swift-evolution/blob/master/process.md

Thanks,
Ben Cohen
Review Manager

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

11

  I’m not a language or memory expert, so I think the proposal could use a few more examples of the code it would disallow. Perhaps examples from real world projects? Without being an expert I’m having a hard time seeing the full impact of the proposed rules.

We hope that there will not be any widespread examples from real-world projects that run afoul of the rule. If the examples seem a little contrived, that's good; we believe the rule is almost always already followed.

  Also, the @exclusivity syntax doesn’t seem fully defined. Is there a @exclusivity(checked) counterpart?

Yes, which would be the default (on class properties and global variables).

Would subclasses be allowed to mark inherited properties more or less strictly?

No. The rule has to be established by the place which creates the storage.

Also, is the logic for @ attributes and keyword attributes documented anywhere? Otherwise it seems like exclusive / nonexclusive would be an option.

The guidelines about when to use an @ attribute and when to use a keyword attribute are always a bit uncertain, but in this case I think they clearly point us towards an @ attribute. This is an attribute which opts in to unsafe behavior for performance reasons; it is not expected to be important enough to warrant a keyword.

John.

···

On May 2, 2017, at 6:44 PM, Jon Shier via swift-evolution <swift-evolution@swift.org> wrote:

Jon

On May 2, 2017, at 4:07 PM, Ben Cohen via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Hello Swift community,

The review of SE-0176: "Enforce Exclusive Access to Memory" begins now and runs through May 8, 2017.

The proposal is available here:

https://github.com/apple/swift-evolution/blob/master/proposals/0176-enforce-exclusive-access-to-memory.md
Reviews are an important part of the Swift evolution process. All reviews should be sent to the swift-evolution mailing list at:

https://lists.swift.org/mailman/listinfo/swift-evolution
or, if you would like to keep your feedback private, directly to the review manager.

When replying, please try to keep the proposal link at the top of the message:

Proposal link:

https://github.com/apple/swift-evolution/blob/master/proposals/0176-enforce-exclusive-access-to-memory.md
Reply text

Other replies

<GitHub - apple/swift-evolution: This maintains proposals for changes and user-visible enhancements to the Swift Programming Language.;
What goes into a review?

The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of Swift. When writing your review, here are some questions you might want to answer in your review:

What is your evaluation of the proposal?
Is the problem being addressed significant enough to warrant a change to Swift?
Does this proposal fit well with the feel and direction of Swift?
If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
How much effort did you put into your review? A glance, a quick reading, or an in-depth study?
More information about the Swift evolution process is available at:

https://github.com/apple/swift-evolution/blob/master/process.md

Thanks,
Ben Cohen
Review Manager

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

12

• What is your evaluation of the proposal?

+1. However, it's unclear to me what the dynamic enforcement will look like: will it trap, emit a warning, be catchable in a debugger, ... More details on the developer-facing interface of the dynamic enforcement would be nice.

It will trap. That trap will of course be caught in the debugger, and hopefully the debugger will be able to deliver a nice debugging experience here.

• Is the problem being addressed significant enough to warrant a change to Swift?

The problem is significant, however I can see two significant downsides. The first is the source-breaking nature of the proposal. Breaking source is a problem by itself, but here I'm afraid the errors reporting won't be easily understandable, because "exclusivity" is kind of an advanced feature that won't be easily grasped by developers.

Our hope is that this will trigger very rarely and that we can make sure that there's suitable documentation for what it means.

My second concern is the performance of the dynamic enforcement. How confident are you that the performance hit of the enforcement will not nullify the gain made by the enabling of more compile-time optimisations?

It's a performance trade-off where, unfortunately, the upsides and downsides will be seen in very different code. Dynamic enforcement will kick in mostly for code that makes heavy use of mutable class properties, e.g. traditional Cocoa programming. We expect that the optimization advantages will mostly be seen in code that makes heavy use of value types, especially copy-on-write value types like the standard library's data structures. Of course, traditional Cocoa programming also does use a lot of dictionaries and arrays, so it's possible that the advantages will offset each other.

One direction we're exploring for dynamic enforcement is enabling it only in certain builds by default (e.g. debug configurations). Unlike array bounds checks and integer overflows, exclusivity violations are generally not data-dependent: it's usually true that just executing the code in any configuration will adequately test for dynamic exclusivity violations. You can get them with races, of course, but the dynamic enforcement mechanisms we're looking at will probably not detect cross-thread violations anyway. Of course, you can make a reasonable argument that not enabling dynamic enforcement checks in production builds would contradict Swift's general policy of "safe by default", so this is not a certain thing. We would very much like to hear swift-evolution's thoughts about this.

• Does this proposal fit well with the feel and direction of Swift?

Yes.

• If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
• How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

Full read. BTW, there is a typo in the "Eliminating non-instantaneous accesses?" section, _Int_appendABunchOfStuff => _Array_appendABunchOfStuff

Thank you, this has been fixed.

John.

···

On May 3, 2017, at 3:42 AM, Florent Bruneau via swift-evolution <swift-evolution@swift.org> wrote:

More information about the Swift evolution process is available at:

https://github.com/apple/swift-evolution/blob/master/process.md

Thanks,
Ben Cohen
Review Manager

_______________________________________________
swift-evolution-announce mailing list
swift-evolution-announce@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution-announce

--
Florent Bruneau

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

13

I like the spirit of the proposal. It leaves me a bit uneasy because the practical ramifications are difficult to fully anticipate. Here follow some questions / concerns for the proposal authors.

(Apologies if these were answered somewhere deep in a discussion thread; it’s the end of the semester and I’ve not kept up with the list.)

• • •

First, have you done any systematic exploration of how impactful this rule is? The proposal states in several places that it hopes/expects violations will be rare, but is that empirical or speculative?

Initially, it was just well-informed speculation, but we're gathering more and more evidence that it's borne out in practice. Even large code bases have at most a couple of static violations, and that's before we've implemented some of the refinements around struct properties.

I tried playing around with the implications, thought I’d found a problematic example by writing an in-place Array.shuffle() method — and then was pleased to realize that the proposal already covered my example with swapAt(). In half an hour, I couldn’t come up with something the proposal disallows that it clearly should allow.

Still, the implications of a rule like this are difficult to anticipate. It would be interesting to know how many errors it flags in the Swift source compatibility suite, for example. Is that something we could do before going all-in on a fully robust implementation?

That is definitely an experiment we're running. Unfortunately, we can only test for static violations with this, because checking for dynamic violations requires execution tests.

Second, what are the performance implications of dynamic exclusivity enforcement? Introducing overhead to something as basic as variable access seems dicey. Will it kill “C-like” performance for object properties? For closure vars?

It will definitely add overhead to class member access, global variable access, and closure access. Optimization may be able to reduce those overheads; for example, we can look at every use of a private property and often quickly prove that it is impossible to have an access violation on it. (That can be extended to internal properties in WMO builds; it is impossible for open properties.)

Will “accesses underway” flags add to the memory footprint of every object & every closure?

We were initially considering the use of "access underway" flags, but eventually decided that it made more sense to use a separate global tracking system. Currently this table is thread-local, and we expect it to be small enough at any given time that a linear scan will perform acceptably. (In your experiments, you've no doubt observed that it takes some effort to actually get an access to be non-instantaneous.)

You write that your “hope is that…dynamic access-tracking [will be] cheap enough to enable by default;” have you proved this out with a prototype implementation yet?

This is underway.

Or maybe this check is disabled in optimized builds, and I’m fussing over nothing?

That is something we're considering. To quote from a different reply (that I just made, you haven't missed anything):

One direction we're exploring for dynamic enforcement is enabling it only in certain builds by default (e.g. debug configurations). Unlike array bounds checks and integer overflows, exclusivity violations are generally not data-dependent: it's usually true that just executing the code in any configuration will adequately test for dynamic exclusivity violations. You can get them with races, of course, but the dynamic enforcement mechanisms we're looking at will probably not detect cross-thread violations anyway. Of course, you can make a reasonable argument that not enabling dynamic enforcement checks in production builds would contradict Swift's general policy of "safe by default", so this is not a certain thing. We would very much like to hear swift-evolution's thoughts about this.

John.

···

On May 3, 2017, at 12:05 AM, Paul Cantrell via swift-evolution <swift-evolution@swift.org> wrote:

Cheers,

Paul

On May 2, 2017, at 3:07 PM, Ben Cohen <ben_cohen@apple.com <mailto:ben_cohen@apple.com>> wrote:

Hello Swift community,

The review of SE-0176: "Enforce Exclusive Access to Memory" begins now and runs through May 8, 2017.

The proposal is available here:

https://github.com/apple/swift-evolution/blob/master/proposals/0176-enforce-exclusive-access-to-memory.md
Reviews are an important part of the Swift evolution process. All reviews should be sent to the swift-evolution mailing list at:

https://lists.swift.org/mailman/listinfo/swift-evolution
or, if you would like to keep your feedback private, directly to the review manager.

When replying, please try to keep the proposal link at the top of the message:

Proposal link:

https://github.com/apple/swift-evolution/blob/master/proposals/0176-enforce-exclusive-access-to-memory.md
Reply text

Other replies

<GitHub - apple/swift-evolution: This maintains proposals for changes and user-visible enhancements to the Swift Programming Language.;
What goes into a review?

The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of Swift. When writing your review, here are some questions you might want to answer in your review:

What is your evaluation of the proposal?
Is the problem being addressed significant enough to warrant a change to Swift?
Does this proposal fit well with the feel and direction of Swift?
If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
How much effort did you put into your review? A glance, a quick reading, or an in-depth study?
More information about the Swift evolution process is available at:

https://github.com/apple/swift-evolution/blob/master/process.md

Thanks,
Ben Cohen
Review Manager

_______________________________________________
swift-evolution-announce mailing list
swift-evolution-announce@swift.org <mailto:swift-evolution-announce@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution-announce

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

14

I have one concern and one general comment.

General comment: there's a switch to turn off overflow checks for performance reasons. It seems to me that this check is going to be much more expensive than an overflow check, and it's certainly not much more likely to trigger in a release build after some amount of testing, so my opinion would be that there should be a way to turn it off globally. To me, the checking is UBSan behavior more than a language feature.

-Ounchecked should disable this check, yes. As for "UBSan" vs. language, the general policy in Swift is to enforce its language rules by default, except in the cases of race conditions (where it would be prohibitive) and explicitly unsafe features (which are intended in part to allow things that are not possible under the normal language rules).

Concern: `swap` is quoted a lot for a method that would break under this rule, but as it happens, `swap` with the same value on both sides is (should be) a no-op. Is there a way to not trip the static or dynamic checkers in well-defined cases like that one? Is there any way to check that two inout parameters refer to the same value?

The only reasonable way to do this is statically, and why would you call 'swap' statically with the same l-value for both arguments?

John.

···

On May 6, 2017, at 1:11 PM, Félix Cloutier via swift-evolution <swift-evolution@swift.org> wrote:

Le 2 mai 2017 à 13:07, Ben Cohen via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> a écrit :

Hello Swift community,

The review of SE-0176: "Enforce Exclusive Access to Memory" begins now and runs through May 8, 2017.

The proposal is available here:

https://github.com/apple/swift-evolution/blob/master/proposals/0176-enforce-exclusive-access-to-memory.md
Reviews are an important part of the Swift evolution process. All reviews should be sent to the swift-evolution mailing list at:

https://lists.swift.org/mailman/listinfo/swift-evolution
or, if you would like to keep your feedback private, directly to the review manager.

When replying, please try to keep the proposal link at the top of the message:

Proposal link:

https://github.com/apple/swift-evolution/blob/master/proposals/0176-enforce-exclusive-access-to-memory.md
Reply text

Other replies

<GitHub - apple/swift-evolution: This maintains proposals for changes and user-visible enhancements to the Swift Programming Language.;
What goes into a review?

The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of Swift. When writing your review, here are some questions you might want to answer in your review:

What is your evaluation of the proposal?
Is the problem being addressed significant enough to warrant a change to Swift?
Does this proposal fit well with the feel and direction of Swift?
If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
How much effort did you put into your review? A glance, a quick reading, or an in-depth study?
More information about the Swift evolution process is available at:

https://github.com/apple/swift-evolution/blob/master/process.md

Thanks,
Ben Cohen
Review Manager

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

15

• What is your evaluation of the proposal?

+1. However, it's unclear to me what the dynamic enforcement will look like: will it trap, emit a warning, be catchable in a debugger, ... More details on the developer-facing interface of the dynamic enforcement would be nice.

It will trap. That trap will of course be caught in the debugger, and hopefully the debugger will be able to deliver a nice debugging experience here.

• Is the problem being addressed significant enough to warrant a change to Swift?

The problem is significant, however I can see two significant downsides. The first is the source-breaking nature of the proposal. Breaking source is a problem by itself, but here I'm afraid the errors reporting won't be easily understandable, because "exclusivity" is kind of an advanced feature that won't be easily grasped by developers.

Our hope is that this will trigger very rarely and that we can make sure that there's suitable documentation for what it means.

I'm worrying this may trigger more often on beginners' code because beginners will experiment and write incorrect code more often than experimented dev who have learned idiomatic code patterns.

My second concern is the performance of the dynamic enforcement. How confident are you that the performance hit of the enforcement will not nullify the gain made by the enabling of more compile-time optimisations?

It's a performance trade-off where, unfortunately, the upsides and downsides will be seen in very different code. Dynamic enforcement will kick in mostly for code that makes heavy use of mutable class properties, e.g. traditional Cocoa programming. We expect that the optimization advantages will mostly be seen in code that makes heavy use of value types, especially copy-on-write value types like the standard library's data structures. Of course, traditional Cocoa programming also does use a lot of dictionaries and arrays, so it's possible that the advantages will offset each other.

One direction we're exploring for dynamic enforcement is enabling it only in certain builds by default (e.g. debug configurations). Unlike array bounds checks and integer overflows, exclusivity violations are generally not data-dependent: it's usually true that just executing the code in any configuration will adequately test for dynamic exclusivity violations. You can get them with races, of course, but the dynamic enforcement mechanisms we're looking at will probably not detect cross-thread violations anyway. Of course, you can make a reasonable argument that not enabling dynamic enforcement checks in production builds would contradict Swift's general policy of "safe by default", so this is not a certain thing. We would very much like to hear swift-evolution's thoughts about this.

I would go for disabling the dynamic checks in productions. The rational is mostly the little gain compared to the performance overhead.

···

Le 3 mai 2017 à 17:23, John McCall <rjmccall@apple.com> a écrit :

On May 3, 2017, at 3:42 AM, Florent Bruneau via swift-evolution <swift-evolution@swift.org> wrote:

• Does this proposal fit well with the feel and direction of Swift?

Yes.

• If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
• How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

Full read. BTW, there is a typo in the "Eliminating non-instantaneous accesses?" section, _Int_appendABunchOfStuff => _Array_appendABunchOfStuff

Thank you, this has been fixed.

John.

More information about the Swift evolution process is available at:

https://github.com/apple/swift-evolution/blob/master/process.md

Thanks,
Ben Cohen
Review Manager

_______________________________________________
swift-evolution-announce mailing list
swift-evolution-announce@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution-announce

--
Florent Bruneau

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
Florent Bruneau

16

Thanks for these reassurances, John. This is the sort of feature that calls for an iterative design-implement-design sort of process, so it’s good to know you’ve already done the static analysis explorations you describe. Given what you wrote, and given the apparent importance of this proposal to the large memory vision, I’m now +1.

This feature calls to mind two things I remember Chris Lattner wrote in the early days of the list. The first is that he preferred that Swift stick with design ideas that have been proved out in other languages. The second is that he’s concerned with the “programmer model” that features create — the whole ecosystem of habits, best practices, tools, and mental models that form around a language’s approach.

This whole constellation of memory seems to me to be fairly novel. (Maybe a little Rust-like? But very much its own beast. I don’t know of clear precedent for all of this in other languages.) It also seems this will all have a strong bearing on the programmer model, but it’s hard to fully anticipate how.

I’m nervous about how the evolution process will handle a design problem of this kind. My trust in the core team’s good judgement tempers that nervousness.

We were initially considering the use of "access underway" flags, but eventually decided that it made more sense to use a separate global tracking system. Currently this table is thread-local

Fascinating! I’m surprised that a table you have to scan beats flags with known offsets. Maybe the small size of that table has less cache impact than larger objects? Or is it just a space vs. time tradeoff, and you chose space? Then again, maybe it’s better that I don’t know how the sausage is made!

In your experiments, you've no doubt observed that it takes some effort to actually get an access to be non-instantaneous.

*smirk*

Cheers, P

···

On May 3, 2017, at 10:43 AM, John McCall <rjmccall@apple.com> wrote:

On May 3, 2017, at 12:05 AM, Paul Cantrell via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
I like the spirit of the proposal. It leaves me a bit uneasy because the practical ramifications are difficult to fully anticipate. Here follow some questions / concerns for the proposal authors.

(Apologies if these were answered somewhere deep in a discussion thread; it’s the end of the semester and I’ve not kept up with the list.)

• • •

First, have you done any systematic exploration of how impactful this rule is? The proposal states in several places that it hopes/expects violations will be rare, but is that empirical or speculative?

Initially, it was just well-informed speculation, but we're gathering more and more evidence that it's borne out in practice. Even large code bases have at most a couple of static violations, and that's before we've implemented some of the refinements around struct properties.

I tried playing around with the implications, thought I’d found a problematic example by writing an in-place Array.shuffle() method — and then was pleased to realize that the proposal already covered my example with swapAt(). In half an hour, I couldn’t come up with something the proposal disallows that it clearly should allow.

Still, the implications of a rule like this are difficult to anticipate. It would be interesting to know how many errors it flags in the Swift source compatibility suite, for example. Is that something we could do before going all-in on a fully robust implementation?

That is definitely an experiment we're running. Unfortunately, we can only test for static violations with this, because checking for dynamic violations requires execution tests.

Second, what are the performance implications of dynamic exclusivity enforcement? Introducing overhead to something as basic as variable access seems dicey. Will it kill “C-like” performance for object properties? For closure vars?

It will definitely add overhead to class member access, global variable access, and closure access. Optimization may be able to reduce those overheads; for example, we can look at every use of a private property and often quickly prove that it is impossible to have an access violation on it. (That can be extended to internal properties in WMO builds; it is impossible for open properties.)

Will “accesses underway” flags add to the memory footprint of every object & every closure?

We were initially considering the use of "access underway" flags, but eventually decided that it made more sense to use a separate global tracking system. Currently this table is thread-local, and we expect it to be small enough at any given time that a linear scan will perform acceptably. (In your experiments, you've no doubt observed that it takes some effort to actually get an access to be non-instantaneous.)

You write that your “hope is that…dynamic access-tracking [will be] cheap enough to enable by default;” have you proved this out with a prototype implementation yet?

This is underway.

Or maybe this check is disabled in optimized builds, and I’m fussing over nothing?

That is something we're considering. To quote from a different reply (that I just made, you haven't missed anything):

One direction we're exploring for dynamic enforcement is enabling it only in certain builds by default (e.g. debug configurations). Unlike array bounds checks and integer overflows, exclusivity violations are generally not data-dependent: it's usually true that just executing the code in any configuration will adequately test for dynamic exclusivity violations. You can get them with races, of course, but the dynamic enforcement mechanisms we're looking at will probably not detect cross-thread violations anyway. Of course, you can make a reasonable argument that not enabling dynamic enforcement checks in production builds would contradict Swift's general policy of "safe by default", so this is not a certain thing. We would very much like to hear swift-evolution's thoughts about this.

John.

Cheers,

Paul

On May 2, 2017, at 3:07 PM, Ben Cohen <ben_cohen@apple.com <mailto:ben_cohen@apple.com>> wrote:

Hello Swift community,

The review of SE-0176: "Enforce Exclusive Access to Memory" begins now and runs through May 8, 2017.

The proposal is available here:

https://github.com/apple/swift-evolution/blob/master/proposals/0176-enforce-exclusive-access-to-memory.md
Reviews are an important part of the Swift evolution process. All reviews should be sent to the swift-evolution mailing list at:

https://lists.swift.org/mailman/listinfo/swift-evolution
or, if you would like to keep your feedback private, directly to the review manager.

When replying, please try to keep the proposal link at the top of the message:

Proposal link:

https://github.com/apple/swift-evolution/blob/master/proposals/0176-enforce-exclusive-access-to-memory.md
Reply text

Other replies

<GitHub - apple/swift-evolution: This maintains proposals for changes and user-visible enhancements to the Swift Programming Language.;
What goes into a review?

The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of Swift. When writing your review, here are some questions you might want to answer in your review:

What is your evaluation of the proposal?
Is the problem being addressed significant enough to warrant a change to Swift?
Does this proposal fit well with the feel and direction of Swift?
If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
How much effort did you put into your review? A glance, a quick reading, or an in-depth study?
More information about the Swift evolution process is available at:

https://github.com/apple/swift-evolution/blob/master/process.md

Thanks,
Ben Cohen
Review Manager

_______________________________________________
swift-evolution-announce mailing list
swift-evolution-announce@swift.org <mailto:swift-evolution-announce@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution-announce

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

17

I guess that the example doesn't make a lot of sense outside of collections, and the proposal already says that swap needs to be implemented on collections, but consider the Fisher-Yates shuffle:
-- To shuffle an array a of n elements (indices 0..n-1):
for i from n−1 downto 1 do
     j ← random integer such that 0 ≤ j ≤ i
     exchange a[j] and a[i]
I can come up with other contrived ways to force this into a dynamic analysis situation (but I am not convinced that I would ever see them in real code).

Also, I'm not too solid on how inout works, especially for properties without "physical storage". My understanding is that in some cases, Swift will get a direct pointer to what needs to be modified, and in others, it'll get a pointer to a temporary value and then call a setter with that temporary as the value argument. Does that sound right?

···

Le 6 mai 2017 à 10:36, John McCall <rjmccall@apple.com> a écrit :

On May 6, 2017, at 1:11 PM, Félix Cloutier via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

I have one concern and one general comment.

General comment: there's a switch to turn off overflow checks for performance reasons. It seems to me that this check is going to be much more expensive than an overflow check, and it's certainly not much more likely to trigger in a release build after some amount of testing, so my opinion would be that there should be a way to turn it off globally. To me, the checking is UBSan behavior more than a language feature.

-Ounchecked should disable this check, yes. As for "UBSan" vs. language, the general policy in Swift is to enforce its language rules by default, except in the cases of race conditions (where it would be prohibitive) and explicitly unsafe features (which are intended in part to allow things that are not possible under the normal language rules).

Concern: `swap` is quoted a lot for a method that would break under this rule, but as it happens, `swap` with the same value on both sides is (should be) a no-op. Is there a way to not trip the static or dynamic checkers in well-defined cases like that one? Is there any way to check that two inout parameters refer to the same value?

The only reasonable way to do this is statically, and why would you call 'swap' statically with the same l-value for both arguments?

John.

Le 2 mai 2017 à 13:07, Ben Cohen via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> a écrit :

Hello Swift community,

The review of SE-0176: "Enforce Exclusive Access to Memory" begins now and runs through May 8, 2017.

The proposal is available here:

https://github.com/apple/swift-evolution/blob/master/proposals/0176-enforce-exclusive-access-to-memory.md
Reviews are an important part of the Swift evolution process. All reviews should be sent to the swift-evolution mailing list at:

https://lists.swift.org/mailman/listinfo/swift-evolution
or, if you would like to keep your feedback private, directly to the review manager.

When replying, please try to keep the proposal link at the top of the message:

Proposal link:

https://github.com/apple/swift-evolution/blob/master/proposals/0176-enforce-exclusive-access-to-memory.md
Reply text

Other replies

<GitHub - apple/swift-evolution: This maintains proposals for changes and user-visible enhancements to the Swift Programming Language.;
What goes into a review?

The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of Swift. When writing your review, here are some questions you might want to answer in your review:

What is your evaluation of the proposal?
Is the problem being addressed significant enough to warrant a change to Swift?
Does this proposal fit well with the feel and direction of Swift?
If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
How much effort did you put into your review? A glance, a quick reading, or an in-depth study?
More information about the Swift evolution process is available at:

https://github.com/apple/swift-evolution/blob/master/process.md

Thanks,
Ben Cohen
Review Manager

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

18

Concern: `swap` is quoted a lot for a method that would break under this rule, but as it happens, `swap` with the same value on both sides is (should be) a no-op. Is there a way to not trip the static or dynamic checkers in well-defined cases like that one? Is there any way to check that two inout parameters refer to the same value?

The only reasonable way to do this is statically, and why would you call 'swap' statically with the same l-value for both arguments?

When static analysis can determine that a swap is always a noop, I can’t see any reason not to flag it as an error.

But Félix’s question was also about the runtime case. And he has a good point.

Here’s a compelling example where allowing the noop swap would make sense:

    extension Array {
      mutating func shuffle() {
        for i in indices {
          let j = i + Int(arc4random_uniform(UInt32(count - i)))
          swap(&self[i], &self[j])
        }
      }
    }

In this code:

It’s necessary to allow j to sometimes equal i to get a uniform distribution of random permutations.
Static analysis can’t determine that the noop occurs sometimes but not always.
Adding a check for i == j adds a branch condition to a tight loop. More importantly, it’s easy to forget this check since static analysis won’t flag it. It’s just a hidden land mine, which seems contrary to Swift’s “safety first” aesthetic.

Putting all that together, it would probably be more Swift-like to have swap() be safe, and if necessary add an unsafeSwap() func that fails on the noop case and thus avoids the cost of the check in optimized code.

Note that the code above fails even under Swift 3, which explicitly disallows swapping with a runtime check (“fatal error: swapping a location with itself is not supported”). So nothing here to undermine the SE-0176; the proposal isn’t introducing a problem that doesn’t already exist.

Cheers, P

···

On May 6, 2017, at 12:36 PM, John McCall via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On May 6, 2017, at 1:11 PM, Félix Cloutier via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

John.

Le 2 mai 2017 à 13:07, Ben Cohen via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> a écrit :

Hello Swift community,

The review of SE-0176: "Enforce Exclusive Access to Memory" begins now and runs through May 8, 2017.

The proposal is available here:

https://github.com/apple/swift-evolution/blob/master/proposals/0176-enforce-exclusive-access-to-memory.md
Reviews are an important part of the Swift evolution process. All reviews should be sent to the swift-evolution mailing list at:

https://lists.swift.org/mailman/listinfo/swift-evolution
or, if you would like to keep your feedback private, directly to the review manager.

When replying, please try to keep the proposal link at the top of the message:

Proposal link:

https://github.com/apple/swift-evolution/blob/master/proposals/0176-enforce-exclusive-access-to-memory.md
Reply text

Other replies

<GitHub - apple/swift-evolution: This maintains proposals for changes and user-visible enhancements to the Swift Programming Language.;
What goes into a review?

The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of Swift. When writing your review, here are some questions you might want to answer in your review:

What is your evaluation of the proposal?
Is the problem being addressed significant enough to warrant a change to Swift?
Does this proposal fit well with the feel and direction of Swift?
If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
How much effort did you put into your review? A glance, a quick reading, or an in-depth study?
More information about the Swift evolution process is available at:

https://github.com/apple/swift-evolution/blob/master/process.md

Thanks,
Ben Cohen
Review Manager

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

19

• What is your evaluation of the proposal?

+1. However, it's unclear to me what the dynamic enforcement will look like: will it trap, emit a warning, be catchable in a debugger, ... More details on the developer-facing interface of the dynamic enforcement would be nice.

It will trap. That trap will of course be caught in the debugger, and hopefully the debugger will be able to deliver a nice debugging experience here.

• Is the problem being addressed significant enough to warrant a change to Swift?

The problem is significant, however I can see two significant downsides. The first is the source-breaking nature of the proposal. Breaking source is a problem by itself, but here I'm afraid the errors reporting won't be easily understandable, because "exclusivity" is kind of an advanced feature that won't be easily grasped by developers.

Our hope is that this will trigger very rarely and that we can make sure that there's suitable documentation for what it means.

I'm worrying this may trigger more often on beginners' code because beginners will experiment and write incorrect code more often than experimented dev who have learned idiomatic code patterns.

That's generally a very important concern. I do think it's less likely to apply here because beginners are quite a bit less likely to be experimenting with oddly-nested inout arguments, or inout arguments at all.

John.

···

On May 3, 2017, at 3:56 PM, Florent Bruneau <florent.bruneau@intersec.com> wrote:

Le 3 mai 2017 à 17:23, John McCall <rjmccall@apple.com> a écrit :

On May 3, 2017, at 3:42 AM, Florent Bruneau via swift-evolution <swift-evolution@swift.org> wrote:

My second concern is the performance of the dynamic enforcement. How confident are you that the performance hit of the enforcement will not nullify the gain made by the enabling of more compile-time optimisations?

It's a performance trade-off where, unfortunately, the upsides and downsides will be seen in very different code. Dynamic enforcement will kick in mostly for code that makes heavy use of mutable class properties, e.g. traditional Cocoa programming. We expect that the optimization advantages will mostly be seen in code that makes heavy use of value types, especially copy-on-write value types like the standard library's data structures. Of course, traditional Cocoa programming also does use a lot of dictionaries and arrays, so it's possible that the advantages will offset each other.

One direction we're exploring for dynamic enforcement is enabling it only in certain builds by default (e.g. debug configurations). Unlike array bounds checks and integer overflows, exclusivity violations are generally not data-dependent: it's usually true that just executing the code in any configuration will adequately test for dynamic exclusivity violations. You can get them with races, of course, but the dynamic enforcement mechanisms we're looking at will probably not detect cross-thread violations anyway. Of course, you can make a reasonable argument that not enabling dynamic enforcement checks in production builds would contradict Swift's general policy of "safe by default", so this is not a certain thing. We would very much like to hear swift-evolution's thoughts about this.

I would go for disabling the dynamic checks in productions. The rational is mostly the little gain compared to the performance overhead.

• Does this proposal fit well with the feel and direction of Swift?

Yes.

• If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
• How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

Full read. BTW, there is a typo in the "Eliminating non-instantaneous accesses?" section, _Int_appendABunchOfStuff => _Array_appendABunchOfStuff

Thank you, this has been fixed.

John.

More information about the Swift evolution process is available at:

https://github.com/apple/swift-evolution/blob/master/process.md

Thanks,
Ben Cohen
Review Manager

_______________________________________________
swift-evolution-announce mailing list
swift-evolution-announce@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution-announce

--
Florent Bruneau

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
Florent Bruneau

20

• What is your evaluation of the proposal?

+1. However, it's unclear to me what the dynamic enforcement will look like: will it trap, emit a warning, be catchable in a debugger, ... More details on the developer-facing interface of the dynamic enforcement would be nice.

It will trap. That trap will of course be caught in the debugger, and hopefully the debugger will be able to deliver a nice debugging experience here.

• Is the problem being addressed significant enough to warrant a change to Swift?

The problem is significant, however I can see two significant downsides. The first is the source-breaking nature of the proposal. Breaking source is a problem by itself, but here I'm afraid the errors reporting won't be easily understandable, because "exclusivity" is kind of an advanced feature that won't be easily grasped by developers.

Our hope is that this will trigger very rarely and that we can make sure that there's suitable documentation for what it means.

I'm worrying this may trigger more often on beginners' code because beginners will experiment and write incorrect code more often than experimented dev who have learned idiomatic code patterns.

That's generally a very important concern. I do think it's less likely to apply here because beginners are quite a bit less likely to be experimenting with oddly-nested inout arguments, or inout arguments at all.

John.

I think it would be fairly important to get a nice and helpful error message if the exclusivity is violated, for developers of all skill levels to understand what’s happening and to know what to do about it. I don’t know if it’s something that’s worth pointing out specifically or not.

/ David

···

On 3 May 2017, at 22:06, John McCall via swift-evolution <swift-evolution@swift.org> wrote:

On May 3, 2017, at 3:56 PM, Florent Bruneau <florent.bruneau@intersec.com> wrote:

Le 3 mai 2017 à 17:23, John McCall <rjmccall@apple.com> a écrit :

On May 3, 2017, at 3:42 AM, Florent Bruneau via swift-evolution <swift-evolution@swift.org> wrote:

My second concern is the performance of the dynamic enforcement. How confident are you that the performance hit of the enforcement will not nullify the gain made by the enabling of more compile-time optimisations?

It's a performance trade-off where, unfortunately, the upsides and downsides will be seen in very different code. Dynamic enforcement will kick in mostly for code that makes heavy use of mutable class properties, e.g. traditional Cocoa programming. We expect that the optimization advantages will mostly be seen in code that makes heavy use of value types, especially copy-on-write value types like the standard library's data structures. Of course, traditional Cocoa programming also does use a lot of dictionaries and arrays, so it's possible that the advantages will offset each other.

One direction we're exploring for dynamic enforcement is enabling it only in certain builds by default (e.g. debug configurations). Unlike array bounds checks and integer overflows, exclusivity violations are generally not data-dependent: it's usually true that just executing the code in any configuration will adequately test for dynamic exclusivity violations. You can get them with races, of course, but the dynamic enforcement mechanisms we're looking at will probably not detect cross-thread violations anyway. Of course, you can make a reasonable argument that not enabling dynamic enforcement checks in production builds would contradict Swift's general policy of "safe by default", so this is not a certain thing. We would very much like to hear swift-evolution's thoughts about this.

I would go for disabling the dynamic checks in productions. The rational is mostly the little gain compared to the performance overhead.

• Does this proposal fit well with the feel and direction of Swift?

Yes.

• If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
• How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

Full read. BTW, there is a typo in the "Eliminating non-instantaneous accesses?" section, _Int_appendABunchOfStuff => _Array_appendABunchOfStuff

Thank you, this has been fixed.

John.

More information about the Swift evolution process is available at:

https://github.com/apple/swift-evolution/blob/master/process.md

Thanks,
Ben Cohen
Review Manager

_______________________________________________
swift-evolution-announce mailing list
swift-evolution-announce@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution-announce

--
Florent Bruneau

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
Florent Bruneau

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution