[Review] SE-0159: Fix Private Access Levels

Evolution Announcements
121

A core team member (I'm blanking on who) has pointed out that, in the end,
the only necessary access modifiers are public and not public (spelled
"internal" in Swift).

It is not clear to me how this squares with the decision in SE-0025 that
other access modifiers were necessary. Can you clarify?

I believe the reference is to this comment by Slava Pestov on 2017-02-16:

While we’re bikeshedding, I’m going to add my two cents. Hold on to your

hat
because this might be controversial here.

I think both ‘private’ and ‘fileprivate’ are unnecessary complications
that
only serve to clutter the language.

It would make a lot more sense to just have internal and public only. No

private, no fileprivate, no lineprivate, no protected. It’s all silly.

https://www.mail-archive.com/swift-evolution@swift.org/msg21766.html

···

On Tue, Mar 21, 2017 at 6:44 PM, Drew Crawford via swift-evolution < swift-evolution@swift.org> wrote:

122

These two points are where I have to jump in. As someone who actually
teaches Swift, and not just lectures others about it on the mailing
lists, both of these arguments are terribly fallacious. Core Swift
features like optionals are fundamental to the idea of the language,
that have semantic meaning and impact on compiled code.

People are thrilled to learn how to use enums because they make for a
better time writing code. These access modifier are can only charitably
be called code flavor. I have spent entire weeks of class trying to
extoll the benefits, so breathlessly shared on these mailing lists, of
how beautiful it is to have a scoped access level. I have yet to
succeed. They frustrate readers and they are a constant source of
roadblocks for the inexperienced.

Segregating private and fileprivate are, at best, a distraction, and at
worst are actively imposing on all of us the personal code style
preferences of those that were most active on the mailing lists while
Swift 2 was open.

It doesn't matter if desire for using extensions is the root of the
problem. Swift should only impose a preference when it's important to
the speed, functionality, and safety of the language. I have yet to be
convinced that there's a benefit to a scoped access level that fits
anywhere in there.

Best,

  Zach Waldowski

  zach@waldowski.me

···

On Tue, Mar 21, 2017, at 07:44 PM, Drew Crawford via swift-evolution wrote:

When new `private` and `fileprivate` were shipped, there were
numerous questions asked by users on forums such as Stack Overflow

And before they were shipped, there were people asking for a scoped
modifier. And when we change them again, what questions will that
generate? So I don’t see what we are accomplishing with this line of
inquiry.

You see the new access modifiers described, for example, as something
that "take[s] some getting used to." Rather damning as a euphemism,
don't you think?

Not at all. We could describe strong typing, or optionals, enums,
pattern matching, etc., in this way. ARC in particular is “difficult
to teach” but we should not get rid of it, it solves a safety problem.
A scoped access modifier is similar.

123

Sent from my iPhone

> I’m not arguing that it is less or more than a majority. I’m just saying that we’ve seen a lot of talk against the original change.

This proposal asks us to balance the convenience of one group (extension-writers) against the existence of another (scoped-access users). To do that, we need a clear idea of the composition of both groups.

“A lot of talk” is not the evidentiary standard to remove a feature. It was not good enough when we introduced the feature, that required argument and clear use-cases.

"A lot of talk" is not the evidence supporting the proposal: it's just a warning that something may be very controversial among the community. The arguments for the revert are in the proposal and in the discussions in this thread.

> By default, I did not mean the syntactic default of the language but the access modifier users will use “by default” when trying to restrict visibility. In most languages, that keyword is “private” so its valid to say that newcomers to the language will “default” to using that one.

Apologies, but I do not understand the argument:

A user wants to restrict visibility (e.g. they are dissatisfied with “internal”)
The user *chooses* private because of familiarity from another language
The user is then surprised that their choice of private indeed restricted the visibility, thus achieving their goal?
What language does the user come from in which “private” is file-visible? It isn’t Java, C++, or PHP. C#’s “partial” is the closest I can think of, and it isn’t at all close.

It has pointed quite a few times by core team members that comparison to languages is not a very strong arguments, especially when Swift does things differently for a good reason. I can't stop from quoting Xiaodi from a month back:

«The beauty of Swift 2's access modifiers was that they were based around files and modules, explicitly rejecting types and scopes as units for determining visibility.» -- Xiaodi

I do not see how Swift explicitly rejected both types and scope for determining visibility, we have private and fileprivate for a reason and they may not help everybody to feel Swifty or whatever that means in practice, but it allows others to do actual work. Having additional flexibility does not hurt people that want to stick with file based visibility.

None of us here is paid to write haiku/poetry code, so while beauty has a place it is not the necessary and sufficient condition for language features: code is meant to be read, refactored, and most of all shipped and debugged quickly.

···

Sent from my iPhone

On 21 Mar 2017, at 18:41, David Hart via swift-evolution <swift-evolution@swift.org> wrote:
On 21 Mar 2017, at 16:57, Drew Crawford <drew@sealedabstract.com> wrote:

A user who wants a middle-ground visibility would “default” to “protected”, “friend”, “partial”, or similar. After that does not compile, they will use google to find a middle-road visibility keyword, for which the only candidate is “fileprivate”. But they will not choose “private”, it’s just not a reasonable expectation of what the keyword means to a new Swift developer.

The popularity of private “as a default” is simply because many users prefer to hide their implementation details as a matter of routine code hygiene. Redefining private in order to thwart their code hygiene goal seems extreme.

The point is that keeping both private and fileprivate feels like an un-necessary complication:

• either a programmer falls on your side of the fence and will use private as often as possible and relegate to fileprivate when the design leaves no other choice. At that point it feels like a language wart.
• or a programmer will fall on my side of the fence and use fileprivate all the time and the language feels like it has an unnecessary access modifier.

I'd argue that the cases when a programmer will use both meaningfully is very rare. As a consequence, we should try to only keep one. Removing fileprivate is a no-go with extensions so that leaves us with removing private.

I agree with several here (as I did in SE-0025) that our access modifiers are not well-named. However, that’s not the proposal in front of us.

> My own statistics in my projects show the contrary. At best, this shows how divisive this feature is.

This *may* show that, if contrary statistics were presented, but that hasn’t occurred.

I can generate statistics from my projects if you want. But it's unnecessary: I haven't used private once since it's introduction in Swift 3. I don't see the advantages it brings worth the trouble.

In old code, statistics could be biased by the migrator having replaced all previous instances of private by fileprivate.

If the migrator migrated code to private, and it *worked* (e.g. did not introduce visibility errors) this is not bias, this is a correct use of the feature.

The migrator migrated to fileprivate everywhere, not private, disagreeing with your use of fileprivate.

> I'm just arguing that the additional scope-based access modifier does not provide enough differentiation to be worth that complexity.

The only argument I have seen so far around “complexity” boils down to: “some people do not use it”. But some people *do* use it, and anyway if we are going to remove all the features “not enough people” use then we are in for a ride.

Some people used the for(;;) loop, the ++ operator, var parameters. Many other features were removed from Swift to simplify he language, make it more consistent. Those are worthwhile goals. Yes, we are past Swift 3 now, but that doesn't mean we shouldn't be able to propose a few rare breaking proposals. The implementation of access modifiers came so late in the Swift 3 timeframe that we had little time to play around with them before Swift 3 was released. Now that we have, we have a short window of time to fix mistakes that were made. I'm just arguing that the proposal was one of those mistakes. But you have a right to disagree.

Swift 3 shipped, so what we are discussing now is yanking a keyword without replacement. There is code written that uses private to enforce its threading or security invariants. There is code written that uses private in order to shadow another declaration. There is code that will not compile after migration. We need more than a vague fear of complexity generally to throw a brick through all those windows. That brick will introduce quite a bit of complexity itself.

Concerning the one-class-per-file argument, I would suggest this counter-argument: when working in large projects, I believe it's a good thing if the language encourages (forces is too strong a word for my taste) a one class per file structure, it's good practice.

The form of the argument is invalid. Suppose I argued: "it’s a good thing for the language to encourage one definition per class (no extensions), it’s good practice. So we do not need fileprivate.” That would be very silly (although it has already been advanced as a straw-man position elsewhere in this thread). The argument that we do not need private because nobody should put multiple classes in a file is equally silly. There are reasons to do so, in fact one motivation was given in SE-0025:

> Putting related APIs and/or related implementations in the same file helps ensure consistency and reduces the time to find a particular API or implementation.

These concerns are not resolved by arguments of the form “just don’t do that”.

I empathize with the Swift2 programmer who got through two releases without a scoped access modifier and is annoyed by change. However, removing the feature now is more change, not less, so it makes their problem worse, not better.

On March 21, 2017 at 2:17:40 AM, David Hart (david@hartbit.com) wrote:

Perhaps it was a mistake, but I purposefully did not go into too much detail in the proposal because I think this debate is purely a question of philosophy on Swift and its language features. I did not want to add un-necessary bloat that would have added little rationalisation. Let me try to explain the holes in the proposal by answering your review:

On 21 Mar 2017, at 02:26, Drew Crawford via swift-evolution <swift-evolution@swift.org> wrote:

I disagree quite strongly with the proposal.

First, the document draws conclusions without apparent supporting evidence, e.g.

> Since the release of Swift 3, the access level change of SE–0025 was met with dissatisfaction by a substantial proportion of the general Swift community. Those changes can be viewed as actively harmful, the new requirement for syntax/API changes.
What is “dissatisfaction by a substantial proportion of the general Swift community”? How was this measured/determined?

It’s not feasible to measure precisely the feeling of a whole community. But we get a feeling for it by following the mailing-list, by talking to colleagues, by reading twitter, etc… And it think we all agree that the debate is highly divisive and that a “substantial proportion” of the community was dissatisfied: I’m not arguing that it is less or more than a majority. I’m just saying that we’ve seen a lot of talk against the original change.

What was done to control for the population happy with SE-0025 who would e.g. not be likely to take up pitchforks?

That’s why its important we have this debate now.

Who argues these changes are “actively harmful” and where were they during SE-0025?

The proposal makes the argument that the changes are actively harmful. It’s now up to debate. By the way, even if several people (including me) were already against this proposal during the review, I don’t see why anybody would not have the right to change his mind, especially after several months of production usage and argue differently now.

> subtly encourages overuse of scoped access control and discourages the more reasonable default
Who claims that scoped access is “overused” and what is their argument for doing so?
Why is “fileprivate” the “more reasonable default”? In fact neither fileprivate *nor* private are default (reasonable or not!). Internal is the default. Nor does this proposal suggest we change that. So this seems a very strange statement.

By default, I did not mean the syntactic default of the language but the access modifier users will use “by default” when trying to restrict visibility. In most languages, that keyword is “private” so its valid to say that newcomers to the language will “default” to using that one. If the proposal is accepted, file-scoped private will regain that status.

> But is that distinction between private and fileprivate actively used by the larger community of Swift developers?
Yes. To cite some evidence, here are codebases I actively maintain:

> codebase | private # | fileprivate # | ratio |

>--------------------------------------------------------|-----------|---------------|-------|

> "M" (proprietary) | 486 | 249 | 2x |

> "N"(proprietary) | 179 | 59 | 3x |

> NaOH https://code.sealedabstract.com/drewcrawford/NaOH | 15 | 1 | 15x |

> atbuild GitHub - AnarchyTools/atbuild: The Anarchy Tools build tool | 54 | 5 | 11x |

So from my chair, not only is the distinction useful, but scoped access control (private) is overwhelmingly (2-15x) more useful than fileprivate.

My own statistics in my projects show the contrary. At best, this shows how divisive this feature is. During the discussion of this proposal, it was argued that making decisions based upon project statistics would be dangerous:

In old code, statistics could be biased by the migrator having replaced all previous instances of private by fileprivate.
In new code, satistics could be biased by people using private because of it being the “soft-default”, regardless of proper semantics.

> And if it were used pervasively, would it be worth the cognitive load and complexity of keeping two very similar access levels in the language? This proposal argues that answer to both questions is no

This proposal does not make any later argument about “cognitive load” or “complexity” I can identify. Did the proposal get truncated?

Sorry if I did not state it explicitly, but I see any feature/keyword added to the language as “additional complexity”. And that complexity is completely worth it when the feature adds significant expressivity. I'm just arguing that the additional scope-based access modifier does not provide enough differentiation to be worth that complexity.

What is stated (without evidence) is that "it is extremely common to use several extensions within a file” and that use of “private” is annoying in that case. I now extend the above table

> codebase | private # | fileprivate # | ratio | # of extensions (>=3 extensions in file) |

>--------------------------------------------------------|-----------|---------------|-------|------------------------------------------|

> "M" (proprietary) | 486 | 249 | 2x | 48 |

> "N"(proprietary) | 179 | 59 | 3x | 84 |

> NaOH https://code.sealedabstract.com/drewcrawford/NaOH | 15 | 1 | 15x | 3 |

> atbuild GitHub - AnarchyTools/atbuild: The Anarchy Tools build tool | 54 | 5 | 11x | 6 |

in order to demonstrate in my corner of Swift this is not “extremely common”, and is actually less popular than language features the proposal alleges aren’t used.

My point here is that **different people in different corners of the community program Swift differently and use different styles**. I can definitely empathize with folks like the author who use extensions to group functions and are annoyed that their favorite visibility modifier grew four extra characters. Perhaps we can come up with a keyword that is more succint.

I agree that different people in different corners use different styles. But you could use that argument to validate many features which would make a group of users happy; but all those feature together would just add bloat to the language. Swift has been known to be a very opinionated language, to keep the language simple yet expressive.

However, that is no reason to take away features from working codebases. A scoped access modifier is perhaps my favorite feature in Swift 3. Let’s not throw stuff away because it adds extra characters to one programming style.

Finally, SE-0025 establishes clear motivation for the scoped access modifier:

> Currently, the only reliable way to hide implementation details of a class is to put the code in a separate file and mark it as private. This is not ideal for the following reasons:

> It is not clear whether the implementation details are meant to be completely hidden or can be shared with some related code without the danger of misusing the APIs marked as private. If a file already has multiple classes, it is not clear if a particular API is meant to be hidden completely or can be shared with the other classes.

> It forces a one class per file structure, which is very limiting. Putting related APIs and/or related implementations in the same file helps ensure consistency and reduces the time to find a particular API or implementation. This does not mean that the classes in the same file need to share otherwise hidden APIs, but there is no way to express such sharability with the current access levels.

As far as I can see, the proposal does not actually address or acknowledge these problems at all, but cheerfully returns us to them. It would be a mistake to deprecate this feature without examining at all why we introduced it. And realistically we need new solutions to those problems before removing the existing one.

Drew

On March 20, 2017 at 6:54:55 PM, Douglas Gregor (dgregor@apple.com) wrote:

Hello Swift community,

The review of SE–0159 “Fix Private Access Levels” begins now and runs through March 27, 2017. The proposal is available here:

https://github.com/apple/swift-evolution/blob/master/proposals/0159-fix-private-access-levels.md Reviews are an important part of the Swift evolution process. All reviews should be sent to the swift-evolution mailing list at

https://lists.swift.org/mailman/listinfo/swift-evolution or, if you would like to keep your feedback private, directly to the review manager. When replying, please try to keep the proposal link at the top of the message:

Proposal link:

https://github.com/apple/swift-evolution/blob/master/proposals/0159-fix-private-access-levels.md Reply text Other replies What goes into a review?

The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of Swift. When writing your review, here are some questions you might want to answer in your review:

What is your evaluation of the proposal? Is the problem being addressed significant enough to warrant a change to Swift? Does this proposal fit well with the feel and direction of Swift? If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those? How much effort did you put into your review? A glance, a quick reading, or an in-depth study? More information about the Swift evolution process is available at

https://github.com/apple/swift-evolution/blob/master/process.md Thank you,

-Doug

Review Manager

swift-evolution-announce mailing list swift-evolution-announce@swift.org https://lists.swift.org/mailman/listinfo/swift-evolution-announce

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

124

The arguments for the revert are in the proposal and in the discussions in this thread.

What is in the proposal and this thread are a set of opinions, e.g.

* "the access level change of SE-0025 was met with dissatisfaction by a substantial proportion of the general Swift community."
* "Those changes can be viewed as actively harmful,”
* "it is extremely common to use several extensions within a file.”
* “[existing regime] encourages overuse of scoped access control”
* “[fileprivate is the] more reasonable default”

I am asking for the arguments or evidence that led you or others to these opinions. I understand that you and others believe them, but you understand that they are now a matter of debate. We are not going to reach any mutual understanding just asserting things we believe. The underlying evidence is what would be necessary to convince others to our point of view. I am trying to ascertain what that evidence is.

> It has pointed quite a few times by core team members that comparison to languages is not a very strong arguments,

The context here is that you presented an argument that Swift’s “private” was confusing to users of other languages:

> In most languages, that keyword is “private” so its valid to say that newcomers to the language will “default” to using that one.

If you no longer think that argument is strong because it relies on a comparison to other languages, then we agree :-) But in that case we lose an argument for this proposal. The reason we are talking about other languages is because that was an argument advanced to support the proposal.

Excuse me, I did not express my point well. What I meant is that wanting a feature in Swift because it exists in another language has not been a very strong argument. But contrarily, making Swift familiar to users of other languages because it uses similar concepts or keywords is a different story. That’s why protected (a feature) is on the commonly rejected proposals list, because it doesn’t fit with Swift even though it exists in other languages. But, on the other hand, keywords like try/throws (syntax) have been used despite the fact that Swift doesn’t have exceptions specifically to feel familiar to newcomers.

In that regard, wanted the scoped-private private (feature) in Swift because other languages have it, seems to me like a weaker argument than the argument that the private keyword (syntax) brings familiarity.

···

On 21 Mar 2017, at 21:59, Drew Crawford <drew@sealedabstract.com> wrote:

> Some people used the for(;;) loop, the ++ operator, var parameters.

In those cases, there was extensive discussion of how to achieve the things people were achieving with those features with alternative patterns. var parameters for example have a 1LOC workaround to achieve the previous semantics, ++ had two characters, for(;;) was slightly trickier but we had a clear concept of the scope of the impact. So far as I’m aware, the only suggestion to people who currently use a scoped access modifier is to stop having their problems. So the circumstances of earlier removals and this one are very different.

> • either a programmer ... will use private as often as possible
> • or a programmer will … use fileprivate all the time

There is also the possibility that a programmer considers which modifier is appropriate for the code they are writing. You "argue that... is very rare” but I still am not sure what this argument is or what led you or others to this conclusion.

On March 21, 2017 at 1:41:48 PM, David Hart (david@hartbit.com <mailto:david@hartbit.com>) wrote:

Sent from my iPhone
On 21 Mar 2017, at 16:57, Drew Crawford <drew@sealedabstract.com <mailto:drew@sealedabstract.com>> wrote:

> I’m not arguing that it is less or more than a majority. I’m just saying that we’ve seen a lot of talk against the original change.

This proposal asks us to balance the convenience of one group (extension-writers) against the existence of another (scoped-access users). To do that, we need a clear idea of the composition of both groups.

“A lot of talk” is not the evidentiary standard to remove a feature. It was not good enough when we introduced the feature, that required argument and clear use-cases.

"A lot of talk" is not the evidence supporting the proposal: it's just a warning that something may be very controversial among the community. The arguments for the revert are in the proposal and in the discussions in this thread.

> By default, I did not mean the syntactic default of the language but the access modifier users will use “by default” when trying to restrict visibility. In most languages, that keyword is “private” so its valid to say that newcomers to the language will “default” to using that one.

Apologies, but I do not understand the argument:

A user wants to restrict visibility (e.g. they are dissatisfied with “internal”)
The user *chooses* private because of familiarity from another language
The user is then surprised that their choice of private indeed restricted the visibility, thus achieving their goal?
What language does the user come from in which “private” is file-visible? It isn’t Java, C++, or PHP. C#’s “partial” is the closest I can think of, and it isn’t at all close.

It has pointed quite a few times by core team members that comparison to languages is not a very strong arguments, especially when Swift does things differently for a good reason. I can't stop from quoting Xiaodi from a month back:

«The beauty of Swift 2's access modifiers was that they were based around files and modules, explicitly rejecting types and scopes as units for determining visibility.» -- Xiaodi

A user who wants a middle-ground visibility would “default” to “protected”, “friend”, “partial”, or similar. After that does not compile, they will use google to find a middle-road visibility keyword, for which the only candidate is “fileprivate”. But they will not choose “private”, it’s just not a reasonable expectation of what the keyword means to a new Swift developer.

The popularity of private “as a default” is simply because many users prefer to hide their implementation details as a matter of routine code hygiene. Redefining private in order to thwart their code hygiene goal seems extreme.

The point is that keeping both private and fileprivate feels like an un-necessary complication:

• either a programmer falls on your side of the fence and will use private as often as possible and relegate to fileprivate when the design leaves no other choice. At that point it feels like a language wart.
• or a programmer will fall on my side of the fence and use fileprivate all the time and the language feels like it has an unnecessary access modifier.

I'd argue that the cases when a programmer will use both meaningfully is very rare. As a consequence, we should try to only keep one. Removing fileprivate is a no-go with extensions so that leaves us with removing private.

I agree with several here (as I did in SE-0025) that our access modifiers are not well-named. However, that’s not the proposal in front of us.

> My own statistics in my projects show the contrary. At best, this shows how divisive this feature is.

This *may* show that, if contrary statistics were presented, but that hasn’t occurred.

I can generate statistics from my projects if you want. But it's unnecessary: I haven't used private once since it's introduction in Swift 3. I don't see the advantages it brings worth the trouble.

In old code, statistics could be biased by the migrator having replaced all previous instances of private by fileprivate.

If the migrator migrated code to private, and it *worked* (e.g. did not introduce visibility errors) this is not bias, this is a correct use of the feature.

The migrator migrated to fileprivate everywhere, not private, disagreeing with your use of fileprivate.

> I'm just arguing that the additional scope-based access modifier does not provide enough differentiation to be worth that complexity.

The only argument I have seen so far around “complexity” boils down to: “some people do not use it”. But some people *do* use it, and anyway if we are going to remove all the features “not enough people” use then we are in for a ride.

Some people used the for(;;) loop, the ++ operator, var parameters. Many other features were removed from Swift to simplify he language, make it more consistent. Those are worthwhile goals. Yes, we are past Swift 3 now, but that doesn't mean we shouldn't be able to propose a few rare breaking proposals. The implementation of access modifiers came so late in the Swift 3 timeframe that we had little time to play around with them before Swift 3 was released. Now that we have, we have a short window of time to fix mistakes that were made. I'm just arguing that the proposal was one of those mistakes. But you have a right to disagree.

Swift 3 shipped, so what we are discussing now is yanking a keyword without replacement. There is code written that uses private to enforce its threading or security invariants. There is code written that uses private in order to shadow another declaration. There is code that will not compile after migration. We need more than a vague fear of complexity generally to throw a brick through all those windows. That brick will introduce quite a bit of complexity itself.

Concerning the one-class-per-file argument, I would suggest this counter-argument: when working in large projects, I believe it's a good thing if the language encourages (forces is too strong a word for my taste) a one class per file structure, it's good practice.

The form of the argument is invalid. Suppose I argued: "it’s a good thing for the language to encourage one definition per class (no extensions), it’s good practice. So we do not need fileprivate.” That would be very silly (although it has already been advanced as a straw-man position elsewhere in this thread). The argument that we do not need private because nobody should put multiple classes in a file is equally silly. There are reasons to do so, in fact one motivation was given in SE-0025:

> Putting related APIs and/or related implementations in the same file helps ensure consistency and reduces the time to find a particular API or implementation.

These concerns are not resolved by arguments of the form “just don’t do that”.

I empathize with the Swift2 programmer who got through two releases without a scoped access modifier and is annoyed by change. However, removing the feature now is more change, not less, so it makes their problem worse, not better.

On March 21, 2017 at 2:17:40 AM, David Hart (david@hartbit.com <mailto:david@hartbit.com>) wrote:

Perhaps it was a mistake, but I purposefully did not go into too much detail in the proposal because I think this debate is purely a question of philosophy on Swift and its language features. I did not want to add un-necessary bloat that would have added little rationalisation. Let me try to explain the holes in the proposal by answering your review:

On 21 Mar 2017, at 02:26, Drew Crawford via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

I disagree quite strongly with the proposal.

First, the document draws conclusions without apparent supporting evidence, e.g.

> Since the release of Swift 3, the access level change of SE–0025 was met with dissatisfaction by a substantial proportion of the general Swift community. Those changes can be viewed as actively harmful, the new requirement for syntax/API changes.
What is “dissatisfaction by a substantial proportion of the general Swift community”? How was this measured/determined?

It’s not feasible to measure precisely the feeling of a whole community. But we get a feeling for it by following the mailing-list, by talking to colleagues, by reading twitter, etc… And it think we all agree that the debate is highly divisive and that a “substantial proportion” of the community was dissatisfied: I’m not arguing that it is less or more than a majority. I’m just saying that we’ve seen a lot of talk against the original change.

What was done to control for the population happy with SE-0025 who would e.g. not be likely to take up pitchforks?

That’s why its important we have this debate now.

Who argues these changes are “actively harmful” and where were they during SE-0025?

The proposal makes the argument that the changes are actively harmful. It’s now up to debate. By the way, even if several people (including me) were already against this proposal during the review, I don’t see why anybody would not have the right to change his mind, especially after several months of production usage and argue differently now.

> subtly encourages overuse of scoped access control and discourages the more reasonable default
Who claims that scoped access is “overused” and what is their argument for doing so?
Why is “fileprivate” the “more reasonable default”? In fact neither fileprivate *nor* private are default (reasonable or not!). Internal is the default. Nor does this proposal suggest we change that. So this seems a very strange statement.

By default, I did not mean the syntactic default of the language but the access modifier users will use “by default” when trying to restrict visibility. In most languages, that keyword is “private” so its valid to say that newcomers to the language will “default” to using that one. If the proposal is accepted, file-scoped private will regain that status.

> But is that distinction between private and fileprivate actively used by the larger community of Swift developers?
Yes. To cite some evidence, here are codebases I actively maintain:

> codebase | private # | fileprivate # | ratio |

>--------------------------------------------------------|-----------|---------------|-------|

> "M" (proprietary) | 486 | 249 | 2x |

> "N"(proprietary) | 179 | 59 | 3x |

> NaOH https://code.sealedabstract.com/drewcrawford/NaOH | 15 | 1 | 15x |

> atbuild GitHub - AnarchyTools/atbuild: The Anarchy Tools build tool | 54 | 5 | 11x |

So from my chair, not only is the distinction useful, but scoped access control (private) is overwhelmingly (2-15x) more useful than fileprivate.

My own statistics in my projects show the contrary. At best, this shows how divisive this feature is. During the discussion of this proposal, it was argued that making decisions based upon project statistics would be dangerous:

In old code, statistics could be biased by the migrator having replaced all previous instances of private by fileprivate.
In new code, satistics could be biased by people using private because of it being the “soft-default”, regardless of proper semantics.

> And if it were used pervasively, would it be worth the cognitive load and complexity of keeping two very similar access levels in the language? This proposal argues that answer to both questions is no

This proposal does not make any later argument about “cognitive load” or “complexity” I can identify. Did the proposal get truncated?

Sorry if I did not state it explicitly, but I see any feature/keyword added to the language as “additional complexity”. And that complexity is completely worth it when the feature adds significant expressivity. I'm just arguing that the additional scope-based access modifier does not provide enough differentiation to be worth that complexity.

What is stated (without evidence) is that "it is extremely common to use several extensions within a file” and that use of “private” is annoying in that case. I now extend the above table

> codebase | private # | fileprivate # | ratio | # of extensions (>=3 extensions in file) |

>--------------------------------------------------------|-----------|---------------|-------|------------------------------------------|

> "M" (proprietary) | 486 | 249 | 2x | 48 |

> "N"(proprietary) | 179 | 59 | 3x | 84 |

> NaOH https://code.sealedabstract.com/drewcrawford/NaOH | 15 | 1 | 15x | 3 |

> atbuild GitHub - AnarchyTools/atbuild: The Anarchy Tools build tool | 54 | 5 | 11x | 6 |

in order to demonstrate in my corner of Swift this is not “extremely common”, and is actually less popular than language features the proposal alleges aren’t used.

My point here is that **different people in different corners of the community program Swift differently and use different styles**. I can definitely empathize with folks like the author who use extensions to group functions and are annoyed that their favorite visibility modifier grew four extra characters. Perhaps we can come up with a keyword that is more succint.

I agree that different people in different corners use different styles. But you could use that argument to validate many features which would make a group of users happy; but all those feature together would just add bloat to the language. Swift has been known to be a very opinionated language, to keep the language simple yet expressive.

However, that is no reason to take away features from working codebases. A scoped access modifier is perhaps my favorite feature in Swift 3. Let’s not throw stuff away because it adds extra characters to one programming style.

Finally, SE-0025 establishes clear motivation for the scoped access modifier:

> Currently, the only reliable way to hide implementation details of a class is to put the code in a separate file and mark it as private. This is not ideal for the following reasons:

> It is not clear whether the implementation details are meant to be completely hidden or can be shared with some related code without the danger of misusing the APIs marked as private. If a file already has multiple classes, it is not clear if a particular API is meant to be hidden completely or can be shared with the other classes.

> It forces a one class per file structure, which is very limiting. Putting related APIs and/or related implementations in the same file helps ensure consistency and reduces the time to find a particular API or implementation. This does not mean that the classes in the same file need to share otherwise hidden APIs, but there is no way to express such sharability with the current access levels.

As far as I can see, the proposal does not actually address or acknowledge these problems at all, but cheerfully returns us to them. It would be a mistake to deprecate this feature without examining at all why we introduced it. And realistically we need new solutions to those problems before removing the existing one.

Drew

On March 20, 2017 at 6:54:55 PM, Douglas Gregor (dgregor@apple.com <mailto:dgregor@apple.com>) wrote:

Hello Swift community,

The review of SE–0159 “Fix Private Access Levels” begins now and runs through March 27, 2017. The proposal is available here:

https://github.com/apple/swift-evolution/blob/master/proposals/0159-fix-private-access-levels.md Reviews are an important part of the Swift evolution process. All reviews should be sent to the swift-evolution mailing list at

https://lists.swift.org/mailman/listinfo/swift-evolution or, if you would like to keep your feedback private, directly to the review manager. When replying, please try to keep the proposal link at the top of the message:

Proposal link:

https://github.com/apple/swift-evolution/blob/master/proposals/0159-fix-private-access-levels.md Reply text Other replies What goes into a review?

The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of Swift. When writing your review, here are some questions you might want to answer in your review:

What is your evaluation of the proposal? Is the problem being addressed significant enough to warrant a change to Swift? Does this proposal fit well with the feel and direction of Swift? If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those? How much effort did you put into your review? A glance, a quick reading, or an in-depth study? More information about the Swift evolution process is available at

https://github.com/apple/swift-evolution/blob/master/process.md Thank you,

-Doug

Review Manager

swift-evolution-announce mailing list swift-evolution-announce@swift.org <mailto:swift-evolution-announce@swift.org> https://lists.swift.org/mailman/listinfo/swift-evolution-announce_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

125

Imo there is no absolute and irrevocable proof needed to accept or reject a change proposal.

very true — because for many proposals, there won't ever be any sort of proof for their effect (positive or negative).
Sometimes, there is general agreement that a change is a real improvement, but personal preference varies in dramatic ways.
In such situations, imho discussion doesn't make much sense, as the result will mostly depend on the preference of the core team (which, as it seems, isn't uniform as well ;-)

126

I'm sorry, I used an idiomatic phrase which may not be obvious to non-native speakers.

When you say someone has the "burden of proof", that does not mean that they must objectively prove their position beyond any doubt. It means that it's their job to convince you, and absent a convincing argument from them, you will choose against them.

What I mean is that, when we evaluate a proposal, we should assume it should be rejected; it's the proposal's job to convince us to accept it instead. This is true for all proposals, but especially for this one, because it reverts a previous proposal and removes an existing language feature.

Personally, I *am* convinced that we should accept this proposal. In my own personal experience, I found scoped `private` to be occasionally useful, but more often a hinderance. And this review thread has turned up some good, novel arguments on the "drop it" side. Meanwhile, I think that the "keep it" side's arguments are generally pretty weak, and some of them seem to stem from beliefs about Swift that are at odds with my own.

But this is not a clear-cut technical issue and there are larger process issues at stake, so even though I personally endorse the change, I want to encourage the core team to evaluate this proposal conservatively. That's all I'm saying.

···

On Mar 22, 2017, at 3:30 AM, Rien <Rien@balancingrock.nl> wrote:

I am less sanguine about the burden of proof.

Imo there is no absolute and irrevocable proof needed to accept or reject a change proposal.

--
Brent Royal-Gordon
Architechies

127

Without commenting on anything else, it is considered a bug that this does not work under SE-0025, and it has been complained about <Issues · apple/swift-issues · GitHub; (though not vigorously, since the workaround is trivial), and fixing it would not be an ABI-breaking change (because private functions are not part of a module's ABI).

Jordan

···

On Mar 21, 2017, at 15:26, Xiaodi Wu via swift-evolution <swift-evolution@swift.org> wrote:

The kicker is, _it also doesn't work_. Try, for instance:

struct Foo {
  private var bar: Int { return 42 }
}

extension Foo {
  private var bar: Int { return 43 }
}

The code above should compile and does not. If I understood correctly the explanation from a core team member on this list, it's unclear if it can be made to work without changing how mangling works, which I believe impacts ABI and is not trivial at all. Thus, (a) even proponents of new `private` disagree on one of two key goals stated for new `private`; (b) that goal was never accomplished, and making it work is not trivial; (c) no one even complained about it, suggesting that it was a low-yield goal in the first place.

128

> We will get static linking at some point in the near future.

Static linking does not fix this issue. Just change "framework" to ".a".

I'm curious to hear what issue your client had with you using many frameworks that static linking doesn't solve.

> If we wait until we get submodules, we won't be able to revisit. This is probably our last chance to "remove" a feature. Submodules can always add features down the way.

Maybe submodules will solve this issue, maybe not. But submodules are *much* more complex than scoped access:

* Performance. This is hot code we compile with WMO. Moving it into a submodule could reduce visibility for optimization in a way that causes a performance regression. In particular, we know that specialization of T is a performance requirement, it isn't clear whether that would be preserved. Does WMO provide the same visibility across submodules? Nobody knows.

I don't see why submodules could not profit from WMO: the module is still compiled all together. Submodules are simply a scoping/hiding mechanism.

* Namespacing. It's possible that one program may ship 3-4 versions of this code because each dependency has a slightly different version under our current samizdat process. It is not clear whether submodules would avoid the "duplicate symbols" issue from C/ObjC. Xiaodi seems quite concerned about a related "duplicate functions" problem involved with private today, doubling down on that is not a good idea.

That looks like a very corner case. I haven't yet found myself in the case where I needed multiple versions of a code base in a same product (binary, framework, application)

* It is not clear whether submodules are from an objectcode point of view merged into the parent library or kept as individual libraries

It would be very strange to me if they were independent libraries: what would different them from modules then? No other language I've used works that way.

* It is not clear from a .swiftmodule point of view whether submodules are merged into the parent module or distributed as .swiftmodules / .swiftdocs

* Not clear how much ABI impact there is from submodules at a time when we are supposed to be trying to stabilize it

I would love to believe that a proposal on submodules will come through having solutions to all these issues and many more, then we will implement it and all sing kumbayah. But we are a long distance from that, and it may never happen at all, certainly we cannot evaluate proposals that haven't been written. Meanwhile we have a solution in the hand.

But at the same time, we can't write and review proposals with no regard for future proposals coming down the road or we end up with a clunky language.

···

On 23 Mar 2017, at 16:49, Drew Crawford <drew@sealedabstract.com> wrote:

On March 23, 2017 at 2:22:20 AM, David Hart (david@hartbit.com) wrote:

129

I would argue that supporting whatever the programmer's chosen mental model is actually Swift's greatest strength. We could have a language with only reference types for example, it would be far, far simpler and easier to teach. I am glad that we don't have that language.

We kinda do, though, or did, in Objective-C (well, the “Objective” parts of it, anyway).

Charles

Exactly. I mean the same people who brought us ObjC decided we needed a language with multiple mental models. They could have shipped reference types only, they shipped value types on day 1. To me there is no clearer plank in the platform than that.

If you like your ObjC, you can keep it. Swift exists exactly because we reject some (not all) of the ObjC way, and it's not just syntax. One of the things rejected is There Should Be One Way To Do It™. Swift embraces many ways to write your program and has from 1.0.

+1. I love this aspect of Swift. It lets me choose the tool and model best suited to each specific task.

···

Sent from my iPad

On Mar 24, 2017, at 11:53 AM, Drew Crawford via swift-evolution <swift-evolution@swift.org> wrote:

On March 24, 2017 at 11:46:00 AM, Charles Srstka (cocoadev@charlessoft.com) wrote:

On Mar 24, 2017, at 11:41 AM, Drew Crawford via swift-evolution <swift-evolution@swift.org> wrote:

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

130

It isn’t the fact that there are multiple models, but they way those models interact with each other in the brain. You can actually have lots of different models without causing confusion, as long as the models fit well together. It also isn’t the number of access levels that is a problem. For example, you can have a ton of shirt sizes (XS, S, M, L, XL, XXL, 3XL) and it doesn’t cause more confusion as you add more sizes because they don’t conflict with one another.

A big part of the issue with our current access scheme is how similar the concepts are without being the same. That is then made worse by the fact that they have been given similar names.

It is unclear what distinction you intend to draw here. For example, value types and reference types are similar without being the same. The names of both concepts also contain the substring "type". So the difference you draw between that situation and the private/fileprivate situation is not immediately clear.

The distinction between value and reference types is one of the most confusing aspects of learning to program. We are helped a bit by the fact that we use the concrete Class, Struct, and Enum. Many programmers just think of Classes as behaving differently than Structs/Enums and don’t really think about it as value vs reference. We never write valueType or referenceType in swift (only when having abstract discussions here on Evolution). Notice also that swift doesn’t use the term “pass-by-reference”. It has explicitly avoided using those terms in actual swift code.

I agree with you about statically vs dynamically dispatched being an issue. For the most part, Swift tries to make it so we don’t ever have to think about it, but when the distinction does poke through, it can be confusing. I think we will need to do some redesign work here when more dynamic features are added. I had a proposal a while back to allow power users to force static dispatch by disambiguating the implementation which gets used. I will probably re-propose that when it seems in scope.

It may help if I clarify that a mental model is not how something actually works, but rather our model of how we *think* it works. Mental models are almost always incorrect, but they are good enough to get by... for the most part. (https://www.nngroup.com/articles/mental-models/\)

In your examples above, Swift is projecting a system image which is much simpler than the underlying concepts (e.g. your statement about static vs dynamic typing that: "I would guess the vast majority of Swift developers are not aware there's a difference at all”). This helps to avoid confusion (except where rough spots poke through the facade).

For access controls, the user is being presented with the full complexity of that choice directly.

Here we have two superficially similar tools with slightly different features and performance characteristics, and for most problems it does not even matter which one you choose.

This is exactly the problem. Both for access controls and dispatch.

IMO shipping a full toolbox with plenty of overlap is one of the core values of Swift.

Is it? Can you point to an instance where a member of the core team said they are aiming for “plenty of overlap”?

Do you feel that allowing both scoped private & file-based private is actually important enough to warrant the significant design work it would take to bring it up to par with these other features?

It is not clear to me what design work is actually outstanding. I would love to see submodules, but that seems only indirectly related to visibility keywords, and I'm not aware of what you seem to believe we need.

I do use scoped access extensively, and I've left some examples of that upthread.

Honestly, most of your examples could just be split into multiple files. They might also benefit from submodules.

It is not impossible, but it really doesn’t feel worth the effort to me (say compared to using that time/energy/thought to check off features from the generics manifesto).

Rather, as Carl Brown argued, it would take a lot of time/energy to migrate even the official projects off of private/fileprivate. Keeping what we have is free, change is what is expensive, and this proposal is change.

You are conflating effort by the swift design and implementation community with your personal effort around migration.

In general, Swift is an opinionated language

Not clear what you mean here either. Swift is clearly less opinionated than ObjC for example. I am having trouble connecting this claim to a practical application.

Oh, that is a quote from Lattner, when he was describing the core philosophy behind Swift. Others can probably dig up the actual quote. It has been repeated over and over.

···

On Mar 24, 2017, at 6:45 PM, Drew Crawford <drew@sealedabstract.com> wrote:
On March 24, 2017 at 7:53:09 PM, Jonathan Hull (jhull@gbis.com <mailto:jhull@gbis.com>) wrote:

131

Sorry, listing should read

protocol Foo {

func foo\(\)

}

extension Foo {

func foo\(\) \{\}

func bar\(\) \{\}

}

···

On March 24, 2017 at 8:46:02 PM, Drew Crawford (drew@sealedabstract.com) wrote:

protocol Foo {

func foo\(\) \{ \}

}

extension Foo {

func bar\(\) \{\}

}

132

It isn’t the fact that there are multiple models, but they way those
models interact with each other in the brain. You can actually have lots
of different models without causing confusion, as long as the models fit
well together. It also isn’t the number of access levels that is a
problem. For example, you can have a ton of shirt sizes (XS, S, M, L, XL,
XXL, 3XL) and it doesn’t cause more confusion as you add more sizes because
they don’t conflict with one another.

A big part of the issue with our current access scheme is how similar the
concepts are without being the same. That is then made worse by the fact
that they have been given similar names.

It is unclear what distinction you intend to draw here. For example,
value types and reference types are similar without being the same. The
names of both concepts also contain the substring "type". So the
difference you draw between that situation and the private/fileprivate
situation is not immediately clear.

Note also that your list above is a list of typical stumbling blocks for
new programmers in most languages. Swift has actually done a remarkable job
with most of these through careful design of the way that they are framed /
exposed to the programmer. In general, these things are well separated
from each other.

I do not think Swift has done even a *good* job framing these concepts.
For example

protocol Foo {

    func foo() { }

}

extension Foo {

    func bar() {}

}

One of these is statically dispatched and the other one is dynamically
dispatched. I would guess the vast majority of Swift developers are not
aware there's a difference at all, let alone can explain why one is slower
in a benchmark.

Swift is good at giving programmers flexibility in their tools. Here we
have two superficially similar tools with slightly different features and
performance characteristics, and for most problems it does not even matter
which one you choose. IMO shipping a full toolbox with plenty of overlap
is one of the core values of Swift.

I could hardly disagree more with this opinion. Figuring out what's
statically vs. dynamically dispatched can be very difficult with Swift.
Although each rule is defensible when dissected individually, the overall
scheme once `final`, `dynamic`, `@objc`, etc. are included is supremely
inelegant. Far from being an example of a core value, I would say it's an
example of a major weakness. I see one of the key goals of the Swift
evolution process as reducing and, where possible, eliminating such designs
from the language.

I would suggest that the practicable reason we are talking about removing

···

On Fri, Mar 24, 2017 at 8:45 PM, Drew Crawford via swift-evolution < swift-evolution@swift.org> wrote:

On March 24, 2017 at 7:53:09 PM, Jonathan Hull (jhull@gbis.com) wrote:
private/fileprivate instead of protocol/extension or reference/value is
that while `extension` etc. makes your code more powerful, `private` makes
it less powerful. It is not obvious to everyone why less power is
desirable and so here we are.

Do you feel that allowing both scoped private & file-based private is
actually important enough to warrant the significant design work it would
take to bring it up to par with these other features?

It is not clear to me what design work is actually outstanding. I would
love to see submodules, but that seems only indirectly related to
visibility keywords, and I'm not aware of what you seem to believe we need.

I do use scoped access extensively, and I've left some examples of that
upthread.

It is not impossible, but it really doesn’t feel worth the effort to me
(say compared to using that time/energy/thought to check off features from
the generics manifesto).

Rather, as Carl Brown argued, it would take a lot of time/energy to
migrate even the official projects off of private/fileprivate. Keeping
what we have is free, change is what is expensive, and this proposal is
change.

In general, Swift is an opinionated language

Not clear what you mean here either. Swift is clearly less opinionated
than ObjC for example. I am having trouble connecting this claim to a
practical application.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

133

This is exactly the problem. Both for access controls and dispatch.

How would you respond to clattner's position piece on this? He disputes this point directly:

Swift is another case of a hybrid model: its semantics provide predictability between obviously static (structs, enums, and global funcs) and obviously dynamic (classes, protocols, and closures) constructs. A focus of Swift (like Java and Javascript) is to provide an apparently simple programming model. However, Swift also intentionally "cheats" in its global design by mixing in a few tricks to make the dynamic parts of the language optimizable by a static compiler in many common cases...
The upshot of this is that Swift isn’t squarely in either of the static or dynamic camps: it aims to provide a very predictable performance model (someone writing a bootloader or firmware can stick to using Swift structs and have a simple guarantee of no dynamic overhead or runtime dependence) while also providing an expressive and clean high level programming model - simplifying learning and the common case where programmers don’t care to count cycles.
Is it? Can you point to an instance where a member of the core team said they are aiming for “plenty of overlap”?

See above

Honestly, most of your examples could just be split into multiple files.
Specific arguments were advanced in those examples that they cannot. Can you refute them?

You are conflating effort by the swift design and implementation community with your personal effort around migration.
No, I am referencing a Swift@IBM developer who reported that

the open-source version of Foundation still has a long way to go to get the level of quality of the existing Objective-C frameworks, and we already have enough work to do without having to go make a bunch of arbitrary changes and risk a bunch of regressions because someone doesn't like a keyword... Accepting this proposal would waste hundreds of person-hours of work...

···

On March 24, 2017 at 10:21:17 PM, Jonathan Hull (jhull@gbis.com) wrote:

134

Chris's actual quote was:

"We intentionally want Swift to have a common 'center of gravity' and be an
'opinionated' language, rather than fall to the 'design by committee'
approach that leads to a watered-down design."

This is diametrically opposite to "shipping a full toolbox with plenty of
overlap."

···

On Fri, Mar 24, 2017 at 22:21 Jonathan Hull via swift-evolution < swift-evolution@swift.org> wrote:

On Mar 24, 2017, at 6:45 PM, Drew Crawford <drew@sealedabstract.com> > wrote:

On March 24, 2017 at 7:53:09 PM, Jonathan Hull (jhull@gbis.com) wrote:

It isn’t the fact that there are multiple models, but they way those
models interact with each other in the brain. You can actually have lots
of different models without causing confusion, as long as the models fit
well together. It also isn’t the number of access levels that is a
problem. For example, you can have a ton of shirt sizes (XS, S, M, L, XL,
XXL, 3XL) and it doesn’t cause more confusion as you add more sizes because
they don’t conflict with one another.

A big part of the issue with our current access scheme is how similar the
concepts are without being the same. That is then made worse by the fact
that they have been given similar names.

It is unclear what distinction you intend to draw here. For example,
value types and reference types are similar without being the same. The
names of both concepts also contain the substring "type". So the
difference you draw between that situation and the private/fileprivate
situation is not immediately clear.

The distinction between value and reference types is one of the most
confusing aspects of learning to program. We are helped a bit by the fact
that we use the concrete Class, Struct, and Enum. Many programmers just
think of Classes as behaving differently than Structs/Enums and don’t
really think about it as value vs reference. We never write valueType or
referenceType in swift (only when having abstract discussions here on
Evolution). Notice also that swift doesn’t use the term
“pass-by-reference”. It has explicitly avoided using those terms in actual
swift code.

I agree with you about statically vs dynamically dispatched being an
issue. For the most part, Swift tries to make it so we don’t ever have to
think about it, but when the distinction does poke through, it can be
confusing. I think we will need to do some redesign work here when more
dynamic features are added. I had a proposal a while back to allow power
users to force static dispatch by disambiguating the implementation which
gets used. I will probably re-propose that when it seems in scope.

It may help if I clarify that a mental model is not how something actually
works, but rather our model of how we *think* it works. Mental models are
almost always incorrect, but they are good enough to get by... for the most
part. (https://www.nngroup.com/articles/mental-models/\)

In your examples above, Swift is projecting a system image which is much
simpler than the underlying concepts (e.g. your statement about static vs
dynamic typing that: "I would guess the vast majority of Swift developers
are not aware there's a difference at all”). This helps to avoid confusion
(except where rough spots poke through the facade).

For access controls, the user is being presented with the full complexity
of that choice directly.

Here we have two superficially similar tools with slightly different
features and performance characteristics, and for most problems it does not
even matter which one you choose.

This is exactly the problem. Both for access controls and dispatch.

IMO shipping a full toolbox with plenty of overlap is one of the core
values of Swift.

Is it? Can you point to an instance where a member of the core team said
they are aiming for “plenty of overlap”?

Do you feel that allowing both scoped private & file-based private is
actually important enough to warrant the significant design work it would
take to bring it up to par with these other features?

It is not clear to me what design work is actually outstanding. I would
love to see submodules, but that seems only indirectly related to
visibility keywords, and I'm not aware of what you seem to believe we need.

I do use scoped access extensively, and I've left some examples of that
upthread.

Honestly, most of your examples could just be split into multiple files.
They might also benefit from submodules.

It is not impossible, but it really doesn’t feel worth the effort to me
(say compared to using that time/energy/thought to check off features from
the generics manifesto).

Rather, as Carl Brown argued, it would take a lot of time/energy to
migrate even the official projects off of private/fileprivate. Keeping
what we have is free, change is what is expensive, and this proposal is
change.

You are conflating effort by the swift design and implementation community
with your personal effort around migration.

In general, Swift is an opinionated language

Not clear what you mean here either. Swift is clearly less opinionated
than ObjC for example. I am having trouble connecting this claim to a
practical application.

Oh, that is a quote from Lattner, when he was describing the core
philosophy behind Swift. Others can probably dig up the actual quote. It
has been repeated over and over.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

135

+1. Creates a mechanism that actually makes sense for what people are using ‘extension’ for, solves the scoping problems, eliminates the need for fileprivate, solves all the problems that people are complaining about.

You could also enforce it by disallowing a ‘conformance’ block from containing any non-private methods that don’t work towards implementing the protocol.

Charles

···

On Mar 27, 2017, at 12:00 PM, Ross O'Brien via swift-evolution <swift-evolution@swift.org> wrote:

I'm considering this from a different angle.

When we declare a type, we declare properties and functions which that type has.
When we extend a type, we add functions to the type. (I'm including computed properties in this.)
It's become an idiom of Swift to declare an extension to a type for each protocol we want it to conform to, for reasons of code organisation and readability. This may be true even if conformance to the protocol was a primary intent of creating the type in the first place.

The intent of the scoped access level is to allow programmers to create properties and functions which are limited to the scope of their declaration. A protocol conformance can be written, with the aid of helper functions, in the confidence that the helper functions are not visible outside the extension, minimising their impact on other components of the module.
However, some protocol conformances require the type to have a specific property, which the extension cannot facilitate. Some protocol conformances don't require a property, but it would be really useful to have one, and again an extension can't facilitate.

Example: we want to be able to write this, but we can't:
private protocol Bar
{
  var integer : Int { get }
  func increment()
}

struct Foo
{
}

extension Foo : Bar
{
  var integer : Int

  private var counter : Int
  func increment()
  {
    counter += 1
  }
}

This leads to a workaround: that properties are added to the original type, and declared as fileprivate. They're not intended to be visible to any scope other than the conforming extension - not even, really, to the type's original scope.

Continuing the example: we've compromised and written this:
private protocol Bar
{
  var integer : Int { get }
  func increment()
}

struct Foo
{
  fileprivate var integer : Int
  fileprivate var counter : Int
}

extension Foo : Bar
{
  func increment()
  {
    counter += 1
  }
}

This is not a fault of fileprivate (though it's a clunky name), or private. Renaming these levels does not solve the problem. Removing private, such that everything becomes fileprivate, does not solve the problem. The problem is in the extension system.
(At this point I realise I'm focusing on one problem as if it's the only one.)

Supposing we approached extensions differently. I think around SE-0025 we were considering a 'nested' access level.

Supposing we created a 'conformance region' inside a type declaration - a scope nested within the type declaration scope - and that this conformance region had its own access level. It's inside the type declaration, not separate from it like an extension, so we can declare properties inside it. But literally the only properties and functions declared inside the region but visible anywhere outside of it, would be properties and functions declared in the named protocol being conformed to.

So, visually it might look like this:

private protocol Bar
{
  var integer : Int { get }
  func increment()
}

struct Foo
{
  conformance Bar // or conformance Foo : Bar, but since the region is inside Foo that's redundant
  {
    var integer : Int // visible because Foo : Bar, at Bar's access level

    var counter : Int = 0 // only visible inside the conformance scope, because not declared in Bar

    func increment() // visible because Foo : Bar, at Bar's access level
    {
      counter += 1
    }
  }
}

I've introduced a new keyword, conformance, though it may be clear enough to keep using extension inside a scope for this. Foo still conforms to Bar, in the same file. We've removed 'extension Foo :' and moved a '}' for this, but that's not a breaking change as this is an addition. Readability is compromised to the extent that this conformance is indented one level.

I've not long had the idea. It's a different approach and may be worth a discussion thread of its own for - or someone might point out some glaringly obvious flaw in it. If it's useful, I don't know the full implications this would have, such as how much this would reduce the use of fileprivate (e.g. to none, to the minimal levels expected in SE-0025, or no effect at all). It's just intended to remove a problem which fileprivate is applied as a bad workaround for.

136

No it does not.

In addition to the protocol-conformance pattern that Ross describes, people
also use extensions to let types declared elsewhere (even in another
module!) work with types declared locally. So in Foo.swift we will have the
implementation of Foo, as well as things like “extension String { /*
Foo-related stuff */ }”.

The file-scope access level, which had been and should be spelled
“private”, enables those extensions of external types to access
implementation details of Foo to do their work. Thus Ross’s scheme does not
eliminate the need for file-level visibility at all. In my view, what we
really need is an access level *broader* than a single file, so that
implementation details can be made visible across a select group of
tightly-related files but not the rest of the module. However that is out
of scope for Swift 4.

Nevin

···

On Mon, Mar 27, 2017 at 1:47 PM, Charles Srstka via swift-evolution < swift-evolution@swift.org> wrote:

On Mar 27, 2017, at 12:00 PM, Ross O'Brien via swift-evolution < > swift-evolution@swift.org> wrote:

<snip>

Creates a mechanism that actually makes sense for what people are using
‘extension’ for, solves the scoping problems, eliminates the need for
fileprivate, solves all the problems that people are complaining about.

137

I'm considering this from a different angle.

When we declare a type, we declare properties and functions which that type
has.
When we extend a type, we add functions to the type. (I'm including
computed properties in this.)
It's become an idiom of Swift to declare an extension to a type for each
protocol we want it to conform to, for reasons of code organisation and
readability. This may be true even if conformance to the protocol was a
primary intent of creating the type in the first place.

The intent of the scoped access level is to allow programmers to create
properties and functions which are limited to the scope of their
declaration. A protocol conformance can be written, with the aid of helper
functions, in the confidence that the helper functions are not visible
outside the extension, minimising their impact on other components of the
module.
However, some protocol conformances require the type to have a specific
property, which the extension cannot facilitate. Some protocol conformances
don't require a property, but it would be really useful to have one, and
again an extension can't facilitate.

Example: we want to be able to write this, but we can't:

privateprotocolBar

{

  varinteger : Int{ get}

  funcincrement()

}

structFoo

{

}

extensionFoo: Bar

{

  varinteger : Int

  privatevarcounter : Int

  funcincrement()

  {

  counter += 1

  }

}

This leads to a workaround: that properties are added to the original type,
and declared as fileprivate. They're not intended to be visible to any
scope other than the conforming extension - not even, really, to the type's
original scope.

Continuing the example: we've compromised and written this:

privateprotocolBar

{

  varinteger : Int{ get}

  funcincrement()

}

structFoo

{

  fileprivatevarinteger : Int

  fileprivatevarcounter : Int

}

extensionFoo: Bar

{

  funcincrement()

  {

  counter += 1

  }

}

This is not a fault of fileprivate (though it's a clunky name), or private.
Renaming these levels does not solve the problem. Removing private, such
that everything becomes fileprivate, does not solve the problem. The
problem is in the extension system.
(At this point I realise I'm focusing on one problem as if it's the only one.)

Supposing we approached extensions differently. I think around SE-0025 we
were considering a 'nested' access level.

Supposing we created a 'conformance region' inside a type declaration - a
scope nested within the type declaration scope - and that this conformance
region had its own access level. It's inside the type declaration, not
separate from it like an extension, so we can declare properties inside it.
But literally the only properties and functions declared inside the region
but visible anywhere outside of it, would be properties and functions
declared in the named protocol being conformed to.

So, visually it might look like this:

privateprotocolBar

{

  varinteger : Int{ get}

  funcincrement()

}

structFoo

{

  conformance Bar // or conformance Foo : Bar, but since the region is
inside Foo that's redundant

  {

  varinteger : Int // visible because Foo : Bar, at Bar's access level

  varcounter : Int = 0// only visible inside the conformance scope, because
not declared in Bar

  funcincrement() // visible because Foo : Bar, at Bar's access level

  {

  counter += 1

  }

  }

}

I've introduced a new keyword, conformance, though it may be clear enough
to keep using extension inside a scope for this. Foo still conforms to Bar,
in the same file. We've removed 'extension Foo :' and moved a '}' for this,
but that's not a breaking change as this is an addition. Readability is
compromised to the extent that this conformance is indented one level.

I've not long had the idea. It's a different approach and may be worth a
discussion thread of its own for - or someone might point out some
glaringly obvious flaw in it. If it's useful, I don't know the full
implications this would have, such as how much this would reduce the use
of fileprivate (e.g. to none, to the minimal levels expected in SE-0025, or
no effect at all). It's just intended to remove a problem
which fileprivate is applied as a bad workaround for.

IMO It seems like really great suggestion - I'd like to discuss it in separate thread, so please start it if you are interested in this.

Currently I think the proposal should allow extensions(not new keyword) inside type declaration, and such extensions should be allowed to to have stored properties and theirs 'scoped' members should be inaccessible outside such extension declaration, but such extensions should be allowed to access 'scoped' members of class (because such extension is defined in the same scope)

I.e.

class MyClass {
   scoped var foo = 10

   extension ProtoA {
     scoped var bar = 20 // visible only in this extension(scope)
     var baz = 30 // will be accessible as 'normal' property of MyClass
     func barFunc() { print(bar); print(foo); } // can access foo
   }

   extension ProtoB {
     scoped var bar = 40 // visible only in this extension(scope)
     //var baz = 50 // can't be re-definied here
     func anotherFunc() { print(baz); print(foo); } // can access baz
   }
}

I see such benefits of this:
* We leave current model of extensions "as is", no changes for extensions declared outside of the type.
* IMO we need stored properties in extensions at least in the same file with type declaration(as was shown by discussion in the list) - and the proposed solution gives them to us
* IMO this solution is better than just allow stored properties in extensions in the same file:
   * in the latter case extension has no access to 'scoped' members of type but this can be very useful to implement protocol conformance with help of internal(scoped) details, that should not be exposed to whole file level.
   * with proposed solution it is clear that any memeber declared in extension block is a native part of type. We see all members in the same type declaration.
   * proposed solution has a simple mental model - "all that is defined inside type declaration is naturally a part of that type, 'scoped' access 'uses' same rules as usually".

Vladimir.

···

On 27.03.2017 20:00, Ross O'Brien via swift-evolution wrote:

Ross

On Mon, Mar 27, 2017 at 4:26 PM, Rien via swift-evolution > <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

    > On 27 Mar 2017, at 16:46, Steven Knodl via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
    >
    > Late to the party here
    >
    > * What is your evaluation of the proposal?
    > I’m -1 on this. The proposal is not a difficult read, but may have been simply more simply named “Remove Scoped Access Level/Revert SE-0025” as that is what is being proposed. “Fix” seems to me to be a unfortunately worded judgmental proposal title.
    >
    > * Is the problem being addressed significant enough to warrant a change?
    > No. I consider myself to be a fairly “new/n00b” Obj-C/Swift developer. Although SE-0025 was a change that required source changes when implemented, the reasoning was not difficult to understand and changes were simple to make once identified. Here they are from SE-0025
    >
    > • public: symbol visible outside the current module
    > • internal: symbol visible within the current module
    > • fileprivate: symbol visible within the current file
    > • private: symbol visible within the current declaration
    >
    > Moving forward these changes are not difficult to comprehend. I tend to make _everything_ “private” up front so I don’t have any API leakage. Then dial back to “fileprivate” as needed. It’s not difficult for me I guess.

    Right. I do that myself more than I would like to admit.
    But when we only loosen up/tighten down during coding then access
    levels are almost useless.
    The point of access level control is in the design, not in the coding.
    If we made a design (including access levels) and then have to dial
    back, that should be a warning that something is wrong.
    To me, this is an argument in favour of the proposal.

    Rien.

    > As such, I don’t believe that this change was “Actively Harmful”,
    especially for new developers who have a clean slate or simply are
    leaving everything unmarked (internal) anyhow until they move up to
    more advanced topics. Unraveling a generic or functional code someone
    else wrote uses way more cognitive power.
    >
    > I’d like to address the suggestion that the migration for SE-0159
    could “simply” be a search and replace without loss of functionality.
    This doesn’t make sense if you consider the entire code lifecycle.
    Sure the code gets migrated and compiles. This is fine if they code
    _never_ has to be read again. But as we know, code is written once and
    _read many times_ as it will need to be maintained. The distinction
    between private and fileprivate contains information, and although it
    may work correctly now, some information meant to help maintain that
    code has been lost if these keywords are merged and the functionality
    of scoped access is removed. So yes if you don’t maintain the code
    where this migration takes place, this would be ok. But Swift strives
    for readability. Moving classes to separate files to address these
    issues, creates a multitude of Bunny classes where again for
    readability some classes belong together in the same file for ease of
    comprehension (again, code is written once , read many times)
    >
    > * Does this proposal fit well with the feel and direction of Swift?
    > The spirit of the proposal to simplify access levels is well taken.
    This proposal however simplifies at the expense of lost functionality
    (Scoped Access levels) with no replacement. The threads talk a about
    submodules and other solutions that could fill this gap that are not on
    the roadmap, planned or possible which makes them non-admissible in
    considering this proposal.
    >
    > * If you have used other languages, libraries, or package managers
    with a similar feature, how do you feel that this proposal compares to
    those?
    > I am more familiar with scoped access so perhaps that feels more
    natural to me. But with the current implementation Swift users can
    choose whether they use File Based or Scope Based tools, so although
    not ideal to either side, acceptable until a suitable replacement could
    be forged.
    >
    > * How much effort did you put into your review? A glance, a quick
    reading, or an in-depth study?
    > Re-read SE-0025/proposal/most of this very long thread
    >
    > From: <swift-evolution-bounces@swift.org
    <mailto:swift-evolution-bounces@swift.org>> on behalf of Tino Heth via
    swift-evolution <swift-evolution@swift.org
    <mailto:swift-evolution@swift.org>>
    > Reply-To: Tino Heth <2th@gmx.de <mailto:2th@gmx.de>>
    > Date: Monday, March 27, 2017 at 6:48 AM
    > To: Zach Waldowski <zach@waldowski.me <mailto:zach@waldowski.me>>
    > Cc: <swift-evolution@swift.org <mailto:swift-evolution@swift.org>>
    > Subject: Re: [swift-evolution] [Review] SE-0159: Fix Private Access
    Levels
    >
    >> I am now absolutely thrilled to create a filter to Mark As Read
    anything else arising from this thread. Good luck.
    >
    > That might be a good idea — after more than 200 messages, and a quite
    circular discussion with an unhealthy amount of ignorance for the
    opposing side ;-).
    >
    > To fight the latter, I just tried to take the position that "new
    private" is really important, and this imho leads to interesting
    consequences...
    > This access modifier really doesn't solve a problem, like "let" does
    (unless the problem you want to solve is having a language with private
    access).
    > Have a look at this:
    >
    > public struct SeperateConcerns {
    > private var foo: Int = 0
    > public mutating func updateFoo(_ value: Int) {
    > print("The only allowed way to change foo was invoked")
    > foo = value
    > }
    >
    > private var bar: Int = 0
    > public mutating func updateBar(_ value: Int) {
    > print("The only allowed way to change bar was invoked")
    > bar = value
    > }
    >
    > private var foobar: Int = 0
    > public mutating func updateFoobar(_ value: Int) {
    > print("The only allowed way to change foobar was invoked")
    > foobar = value
    > }
    > }
    >
    > You can protect foo from being changed by code in other files, and
    from extensions in the same file — and if the latter is a concern,
    there should also be a way to limit access to foo to specific function
    in scope.
    > Afaik, somebody proposed "partial" type declarations, but without
    them, the meaning of private is rather arbitrary, and the feature is
    only useful for a tiny special case.
    > If we had partial types, the situation would be different, and if
    would be possible to declare extensions inside a partial declaration of
    another type, we could even remove fileprivate without an replacement
    (I guess I should write a separate mail for this thought…)
    > _______________________________________________ swift-evolution
    mailing list swift-evolution@swift.org
    <mailto:swift-evolution@swift.org>
    https://lists.swift.org/mailman/listinfo/swift-evolution
    <https://lists.swift.org/mailman/listinfo/swift-evolution&gt;
    > _______________________________________________
    > swift-evolution mailing list
    > swift-evolution@swift.org <mailto:swift-evolution@swift.org>
    > https://lists.swift.org/mailman/listinfo/swift-evolution
    <https://lists.swift.org/mailman/listinfo/swift-evolution&gt;

    _______________________________________________
    swift-evolution mailing list
    swift-evolution@swift.org <mailto:swift-evolution@swift.org>
    https://lists.swift.org/mailman/listinfo/swift-evolution
    <https://lists.swift.org/mailman/listinfo/swift-evolution&gt;

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

138

Unless you’re referring to a different “commonly rejected proposals” list than the one below, this does not appear to be true:

Charles

···

On Mar 21, 2017, at 5:43 PM, David Hart via swift-evolution <swift-evolution@swift.org> wrote:

That’s why protected (a feature) is on the commonly rejected proposals list

139

So, if four/five access modifiers are too many, which one is carrying the least weight? Which one could be removed to simplify the scheme while maintaining the most expressiveness? Which one doesn't fulfill even its own stated goals? Well, one of the key goals of `private` was to allow members to be encapsulated within an extension, hidden even from the type being extended (and vice versa for members defined in the type). It says so in the first sentence of SE-0025. As seen above in my discussion with Charles Srstka, even supporters of `private` disagree with that motivation to begin with. The kicker is, _it also doesn't work_. Try, for instance:

struct Foo {
  private var bar: Int { return 42 }
}

extension Foo {
  private var bar: Int { return 43 }
}

The code above should compile and does not. If I understood correctly the explanation from a core team member on this list, it's unclear if it can be made to work without changing how mangling works, which I believe impacts ABI and is not trivial at all. Thus, (a) even proponents of new `private` disagree on one of two key goals stated for new `private`; (b) that goal was never accomplished, and making it work is not trivial; (c) no one even complained about it, suggesting that it was a low-yield goal in the first place.

Multiple people have already brought up cases in which they are using ‘private’. The repeated mention of another, unrelated use case that was mentioned in the SE-0025 proposal does not invalidate the real-world use cases which have been presented. In fact, it rather makes it appear as if the motivation to remove ‘private’ is based on a strange invocation of the appeal-to-authority fallacy, rather than an actual improvement to the language.

I'm not sure how to respond to this. SE-0025, as designed, is not fully implemented. And as I said above, IIUC, it cannot be fully implemented without ripping out a lot of mangling code that is unlikely to be ripped out before Swift 4. _And there is no evidence that anyone cares about this flaw; in fact, you are saying as much, that you do not care at all!_ If this is not sufficient indication that the design of SE-0025 does not fit with the overall direction of Swift, what would be?

Because there are other uses cases for ‘private', *not* involving extensions, which I *do* care about. The fact that part of the proposal was badly written (and really, that’s all this is—it uses “class or extension” as a synonym for “any type declaration" when really, it makes just as much sense for structs to have private members as classes. Stuff happens!) does not invalidate the other use cases. And yes, I’m aware that my coding style may differ from other people, who may use the language in a different way. We shouldn’t break *their* use cases, either.

Many people used to argue against early returns as being tantamount to littering one’s code with GOTO statements willy-nilly, instead advocating for assigning to a return variable which would then be returned in the last line of the function. Swift explicitly rejects this view, offering constructs such as the guard statement and the error-handling mechanism which specifically encourage early returns. However, I don’t see people using the non-acceptance of this particular argument against GOTOs as an argument for eliminating all structured flow control from the language.

I don't follow. Which people have argued against structured control flow in Swift?

No one has. That’s the point!

Charles

···

On Mar 21, 2017, at 7:49 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
On Tue, Mar 21, 2017 at 6:46 PM, Charles Srstka <cocoadev@charlessoft.com <mailto:cocoadev@charlessoft.com>> wrote:

On Mar 21, 2017, at 5:26 PM, Xiaodi Wu via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

140

Swift should only impose a preference when it's important to the speed, functionality, and safety of the language. I have yet to be convinced that there's a benefit to a scoped access level that fits anywhere in there.
SE-0025 addresses these *specific* points.

speed & safety:

Also, there is a greater danger of using private APIs if they do something similar to public APIs but are somehow more optimized (because they make additional assumptions about the internal state).

functionality:

It forces a one class per file structure, which is very limiting. Putting related APIs and/or related implementations in the same file helps ensure consistency and reduces the time to find a particular API or implementation. This does not mean that the classes in the same file need to share otherwise hidden APIs, but there is no way to express such sharability with the current access levels.

I have spent entire weeks of class trying to extoll the benefits, so breathlessly shared on these mailing lists, of how beautiful it is to have a scoped access level. I have yet to succeed.
Perhaps this suggests scoped access modifiers are more comparable to e.g. “owned” in the Memory Management Manifesto or UnsafeRawPointer/SE-0107. Those features are breathtakingly difficult to teach, with design documents in the dozens of pages that are so dense I do not understand them.

However, they solve hairy problems down in the dungeon somewhere, and most programmers do not actually need to know them to write their code.

I don’t think scoped is quite to that level, but even if it were: if you like your visibility modifiers you can keep them! There is no law that says you must use all the modifiers, and the availability of a feature does not “impose on all of us the personal code style preferences” of anyone. Removing a feature does, and that’s the present proposal.

···

On March 21, 2017 at 7:45:22 PM, Zach Waldowski via swift-evolution (swift-evolution@swift.org) wrote: