[Discussion] A Problem With SE-0025?

I await their opinions, but even with that change we still have a problem with the philosophy here. The motivation behind the fileprivate/private distinction is, as you say in the proposal "[a] reliable way to hide implementation details...". Why would you want any members you're trying to hide to be visible in any scope other than the one you create with the decl (which is, without a doubt, also a lexical scope)? It is analogous with other access control levels: a public outer decl implies public inner members just as a fileprivate one implies fileprivate inner members. Is it not antithetical to being able to hide things if they remain visible despite the outer decl declaring itself supposedly being sealed? In your examples, you declare private members that you do not wish to leak into top level scope because they are, after all, supposedly hidden. Should the same semantics you have given not also apply to private decls no matter where they lie?

Should this compile:

private struct Outer {
    let inner = Inner()
}

Outer().inner

If this shouldn't:

struct Outer {
    private let inner = Inner()
}

extension Outer {
    var inner2 : Inner { return self.inner }
}

Visibility being a function of scope, lexical or otherwise, seems too coarse a metric for access control in general without further clarification of what the definition of "hidden" is.

~Robert Widmann

2016/06/15 12:38、Matthew Johnson <matthew@anandabits.com> のメッセージ:

···

On Jun 15, 2016, at 2:29 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

The meaning of private according to the proposal is not scope-dependent, it is decl-dependent. The mental gymnastics we are both going through right now are not in the proposal. I would like them to be.

I agree that the proposal is too sparse on the details and that is the source of the confusion.

Ilya wanted me to write this proposal and only did it himself because I didn’t have time to do it as quickly as he wanted. I know what the original intent of the proposal was (I can’t speak for the core team on their mental model of it when they accepted it).

That said, the title of the proposal is “scoped access level” and the detailed design says "visible only within that lexical scope”. And the entire discussion was always about lexical scope. I believe the reason the “proposed solution” section says “current declaration” is because *most* lexical scopes are introduced by a declaration. That is sloppy language that should have been cleaned up prior to review.

I appreciate you working on implementing this. I also understand if you want to hold off until the core team can chime in on the semantics they intend.

Matthew

~Robert Widmann

2016/06/15 12:26、Matthew Johnson <matthew@anandabits.com> のメッセージ:

On Jun 15, 2016, at 2:19 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

We have diagnostics specifically to prohibit this case. You cannot raise the access level of members.

private struct Foo {
    internal var x : String = ""
}

warning: declaring an internal var for a private struct.

Hence, the highest allowable level of access for x is private and it becomes invisible.

I would not like the compiler to synthesize this in my code, and if it did I would like the proposal to say it will raise access levels of members as you would like it to.

That diagnostic is a good thing. I am not suggesting that you to disregard it.

What you are missing is that the meaning of `private` is scope-dependent. The following example is perfectly valid:

private struct Foo {
    fileprivate var x : String = “”
}

`fileprivate` inside `Foo` specifies the same visibility as `private` at the top level, thus we are not “raising" the visibility of the member. If no modifier is provided for `x` it receives the same visibility as its containing type (up to `internal`).

Consider another example:

struct Foo {
    private struct Bar {
        var x : String = “”
    }
}

In this example we *cannot* mark `x` as `fileprivate` because that *would* raise the visibility of `x` beyond that of `Bar`. `Bar` is only visible inside `Foo`, not at file scope. This means that `x` also has visibility throughout the lexical scope of `Foo`, but not outside of it.

-Matthew

~Robert Widmann

2016/06/15 12:14、Matthew Johnson <matthew@anandabits.com> のメッセージ:

On Jun 15, 2016, at 2:04 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

2016/06/15 11:47、Matthew Johnson <matthew@anandabits.com> のメッセージ:

On Jun 15, 2016, at 1:37 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

The scope of the *declaration* is not the issue. The scope of its *members* is.

Let’s consider an example:

private struct Foo {
  var bar: Int
}

// elsewhere in the same file:
var foo = Foo(bar: 42)
foo.bar = 44

`Foo` is declared private. Private for this declaration is at the file scope. The `bar` member has no access modifier so it has the same visibility as the struct itself, which is file scope. This will also be true of the implicitly synthesized memberwise initializer.

No, it is also private. It does not inherit its parent scope because, following the letter of the proposal, that symbol will only be visible within the current declaration. We cannot arbitrarily break access control rules because it is convenient in one special case.

This means that it is possible to initialize `foo` with a newly constructed instance of `Foo` and to modify the `bar` member anywhere else in the same file.

bar is not visible here. If it were you could break access control rules.

If `bar` was also declared `private` this would not be possible as its visibility would be restricted to the surrounding scope of the initial declaration of `Foo`. This means `Foo` would need to provide an explicit initializer or factory method with `fileprivate` visibility in order to be usable.

bar is private. Declarations within Foo cannot decide to raise that access level to make themselves more visible. If this should be the case, the proposal must be amended as much.

Members with no explicit access modifier should have the same *visibility* as their containing type (with a maximum implicit visibility of internal), not the same *modifier* as their containing type. The only case where there is a distinction is the new `private` visibility. Maybe that is what is causing the confusion?

That is not what the proposal says. The proposal says it is invisible outside the current decl, which is the containing structure here.

The access modifier is applied to `Foo`, not `bar`. `Foo` is visible in the scope of the “current declaration” (this is badly worded - it should say current scope, which is the file). Because `Foo` has a visibility lower than `internal` the default visibility of its members match the visibility of `Foo`, which again is the current scope: the file. The detailed design section of the proposal is sparse, but it correctly uses the phrase "visible only within that lexical scope” rather than the less precise (in the case of top-level code) “current declaration”.

I didn’t write the proposal but I was very heavily involved in the discussions and IIRC I provided the original suggestion for introducing a scoped access modifier.

If my example was the following:

private struct Foo {
  private var bar: Int
}

Then what you are saying would be correct. However, The example I showed did not provide an access modifier for `bar`.

You cannot just apply the same *access modifier* to members of the type that do not contain an access modifier. You have to apply the same *visibility* (limited to internal). When a type is marked as `private` in the lexical scope of the file, its unmodified members will be visible in the same lexical scope as the type itself (which is the file in the current example).

-Matthew

Does this help?

-Matthew

~Robert Widmann

2016/06/15 11:36、Matthew Johnson <matthew@anandabits.com> のメッセージ:

The scope for a top-level declaration is the file itself. This means that top-level declarations with `private` and `fileprivate` should have the same behavior. They should not be uninstantiable or unusable.

-Matthew

On Jun 15, 2016, at 1:31 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org> wrote:

While implementing SE-0025 (fileprivate), I noticed an interesting bug in the proposal. Under the implementation outlined there, any top-level structure, class, or enum declared private cannot possibly be instantiated and so cannot be used in any way. Because of this, private top-level declarations are more often than not blown away entirely by the compiler for being unused. It seems strange to me to allow a key language feature to act solely as a hint to the optimizer to reduce the size of your binary. Perhaps the restrictions around private needs to be relaxed or the line between fileprivate and private needs to be investigated again by the community before inclusion in the language.

Thoughts?

~Robert Widmann
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Shouldn’t a file act like an individual scope? If so why would A be visible in C? Is it because files act not a lexical scopes?

···

--
Adrian Zubarev
Sent with Airmail

Am 15. Juni 2016 um 21:34:23, Adrian Zubarev (adrian.zubarev@devandartist.com) schrieb:

Your example #2 is just incorrect. `A` is visible inside the scope of `C`.

Now that we have introduced a scope-dependent access modifier it is an incorrect mental model to consider members with no access modifier as having the exact same access modifier as the containing scope. This is no longer correct. They have the same *visibility*, not the same *access modifier*.

I was referencing to the issue Robert discovered in his implementation.

I do understand what the proposal is all about, but thank you for re-clarifying that to me. :)

···

--
Adrian Zubarev
Sent with Airmail

Am 15. Juni 2016 um 21:40:37, Matthew Johnson (matthew@anandabits.com) schrieb:

What seems like a nasty bug missed during review? I don’t follow you there.

This proposal was specifically designed to follow Swift’s design of a scope-based access control mechanism rather than a type-based access control mechanism that is common in other languages.

Shouldn’t a file act like an individual scope? If so why would A be visible in C? Is it because files act not a lexical scopes?

Did you mean that `C` is in a different file? That wasn’t clear to me. Maybe that is what “1” and “2” indicated (different files)? If that is what you meant then `A` would not be visible in `C`, nor would the `a` member of `B`.

···

On Jun 15, 2016, at 2:43 PM, Adrian Zubarev via swift-evolution <swift-evolution@swift.org> wrote:

--
Adrian Zubarev
Sent with Airmail

Am 15. Juni 2016 um 21:34:23, Adrian Zubarev (adrian.zubarev@devandartist.com <mailto:adrian.zubarev@devandartist.com>) schrieb:

Your example #2 is just incorrect. `A` is visible inside the scope of `C`.

Now that we have introduced a scope-dependent access modifier it is an incorrect mental model to consider members with no access modifier as having the exact same access modifier as the containing scope. This is no longer correct. They have the same *visibility*, not the same *access modifier*.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

I was referencing to the issue Robert discovered in his implementation.

I do understand what the proposal is all about, but thank you for re-clarifying that to me. :)

I don’t think it’s a bug, but it is definitely something that isn’t as clear as it should have been.

···

On Jun 15, 2016, at 2:46 PM, Adrian Zubarev via swift-evolution <swift-evolution@swift.org> wrote:

--
Adrian Zubarev
Sent with Airmail

Am 15. Juni 2016 um 21:40:37, Matthew Johnson (matthew@anandabits.com <mailto:matthew@anandabits.com>) schrieb:

What seems like a nasty bug missed during review? I don’t follow you there.

This proposal was specifically designed to follow Swift’s design of a scope-based access control mechanism rather than a type-based access control mechanism that is common in other languages.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

I was referencing to the issue Robert discovered in his implementation.

I do understand what the proposal is all about, but thank you for
re-clarifying that to me. :)

I don’t think it’s a bug, but it is definitely something that isn’t as
clear as it should have been.

Was it intentional on the part of the proposal, then, that there should be
two modifiers meaning the same thing for a top level declaration in a file?
Or was it intended that only one or the other be used in that scenario?

···

On Wed, Jun 15, 2016 at 2:48 PM, Matthew Johnson via swift-evolution < swift-evolution@swift.org> wrote:

On Jun 15, 2016, at 2:46 PM, Adrian Zubarev via swift-evolution < > swift-evolution@swift.org> wrote:

--
Adrian Zubarev
Sent with Airmail

Am 15. Juni 2016 um 21:40:37, Matthew Johnson (matthew@anandabits.com)
schrieb:

What seems like a nasty bug missed during review? I don’t follow you
there.

This proposal was specifically designed to follow Swift’s design of a
scope-based access control mechanism rather than a type-based access
control mechanism that is common in other languages.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

I was referencing to the issue Robert discovered in his implementation.

I do understand what the proposal is all about, but thank you for re-clarifying that to me. :)

I don’t think it’s a bug, but it is definitely something that isn’t as clear as it should have been.

Was it intentional on the part of the proposal, then, that there should be two modifiers meaning the same thing for a top level declaration in a file? Or was it intended that only one or the other be used in that scenario?

I don’t think it was carefully considered, although I think it did come up at some point during discussion in the context of compatibility with existing code (i.e. nothing changes for current top-level `private` declarations).

It is in some sense a “coincidence” that they mean the same thing at file scope. The proposal would have had to introduce a specific prohibition to prevent this situation and it did not do so. That said, I think this kind of issue falls well within the discretion of the core team to make a call without violating the spirit of the proposal.

There are two reasonable options here:

1. Allow both `private` and `fileprivate` at file scope despite the fact that they have the same meaning. This is more consistent in the sense that we are not introducing a special case that arbitrarily prohibits an otherwise valid access modifier. It also means that nothing needs to change for top level `private` declarations in existing code.

2. Prohibit `private` at file scope. Given that it appears as if the behavior of `private` at file scope may not be intuitive and is equivalent to `fileprivate` it might be reasonable to just disallow it. This would result in more consistent *code* (even if there needs to be a special case in the language).

I don’t have a strong opinion on which option we choose. But I do feel strongly that the semantics of `private` need to properly respect the scope in which the keyword is written and into which the associated declaration is introduced (rather than the scope *inside* the declaration it is attached to).

-Matthew

···

On Jun 15, 2016, at 2:55 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
On Wed, Jun 15, 2016 at 2:48 PM, Matthew Johnson via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>>wrote:

On Jun 15, 2016, at 2:46 PM, Adrian Zubarev via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

--
Adrian Zubarev
Sent with Airmail

Am 15. Juni 2016 um 21:40:37, Matthew Johnson (matthew@anandabits.com <mailto:matthew@anandabits.com>) schrieb:

What seems like a nasty bug missed during review? I don’t follow you there.

This proposal was specifically designed to follow Swift’s design of a scope-based access control mechanism rather than a type-based access control mechanism that is common in other languages.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

I was referencing to the issue Robert discovered in his implementation.

I do understand what the proposal is all about, but thank you for
re-clarifying that to me. :)

I don’t think it’s a bug, but it is definitely something that isn’t as
clear as it should have been.

Was it intentional on the part of the proposal, then, that there should be
two modifiers meaning the same thing for a top level declaration in a file?
Or was it intended that only one or the other be used in that scenario?

I don’t think it was carefully considered, although I think it did come up
at some point during discussion in the context of compatibility with
existing code (i.e. nothing changes for current top-level `private`
declarations).

It is in some sense a “coincidence” that they mean the same thing at file
scope. The proposal would have had to introduce a specific prohibition to
prevent this situation and it did not do so. That said, I think this kind
of issue falls well within the discretion of the core team to make a call
without violating the spirit of the proposal.

There are two reasonable options here:

1. Allow both `private` and `fileprivate` at file scope despite the fact
that they have the same meaning. This is more consistent in the sense that
we are not introducing a special case that arbitrarily prohibits an
otherwise valid access modifier. It also means that nothing needs to
change for top level `private` declarations in existing code.

2. Prohibit `private` at file scope. Given that it appears as if the
behavior of `private` at file scope may not be intuitive and is equivalent
to `fileprivate` it might be reasonable to just disallow it. This would
result in more consistent *code* (even if there needs to be a special case
in the language).

I don’t have a strong opinion on which option we choose. But I do feel
strongly that the semantics of `private` need to properly respect the scope
in which the keyword is written and into which the associated declaration
is introduced (rather than the scope *inside* the declaration it is
attached to).

Right, I think both would be OK. More radically, we might want to
re-evaluate the continued utility of a `fileprivate` scope. It seems the
use cases for such a scope not adequately served by either `internal` or
the new `private` would be exceedingly rare.

···

On Wed, Jun 15, 2016 at 3:09 PM, Matthew Johnson <matthew@anandabits.com> wrote:

On Jun 15, 2016, at 2:55 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
On Wed, Jun 15, 2016 at 2:48 PM, Matthew Johnson via swift-evolution < > swift-evolution@swift.org>wrote:

On Jun 15, 2016, at 2:46 PM, Adrian Zubarev via swift-evolution < >> swift-evolution@swift.org> wrote:

-Matthew

--
Adrian Zubarev
Sent with Airmail

Am 15. Juni 2016 um 21:40:37, Matthew Johnson (matthew@anandabits.com)
schrieb:

What seems like a nasty bug missed during review? I don’t follow you
there.

This proposal was specifically designed to follow Swift’s design of a
scope-based access control mechanism rather than a type-based access
control mechanism that is common in other languages.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

I was referencing to the issue Robert discovered in his implementation.

I do understand what the proposal is all about, but thank you for re-clarifying that to me. :)

I don’t think it’s a bug, but it is definitely something that isn’t as clear as it should have been.

Was it intentional on the part of the proposal, then, that there should be two modifiers meaning the same thing for a top level declaration in a file? Or was it intended that only one or the other be used in that scenario?

I don’t think it was carefully considered, although I think it did come up at some point during discussion in the context of compatibility with existing code (i.e. nothing changes for current top-level `private` declarations).

It is in some sense a “coincidence” that they mean the same thing at file scope. The proposal would have had to introduce a specific prohibition to prevent this situation and it did not do so. That said, I think this kind of issue falls well within the discretion of the core team to make a call without violating the spirit of the proposal.

There are two reasonable options here:

1. Allow both `private` and `fileprivate` at file scope despite the fact that they have the same meaning. This is more consistent in the sense that we are not introducing a special case that arbitrarily prohibits an otherwise valid access modifier. It also means that nothing needs to change for top level `private` declarations in existing code.

2. Prohibit `private` at file scope. Given that it appears as if the behavior of `private` at file scope may not be intuitive and is equivalent to `fileprivate` it might be reasonable to just disallow it. This would result in more consistent *code* (even if there needs to be a special case in the language).

I don’t have a strong opinion on which option we choose. But I do feel strongly that the semantics of `private` need to properly respect the scope in which the keyword is written and into which the associated declaration is introduced (rather than the scope *inside* the declaration it is attached to).

Right, I think both would be OK. More radically, we might want to re-evaluate the continued utility of a `fileprivate` scope. It seems the use cases for such a scope not adequately served by either `internal` or the new `private` would be exceedingly rare.

I disagree with that. `fileprivate` is indispensable when you need it. There are times when you want to keep visibility limited to the current file but the new `private` is too restrictive (for example, you need to access a member of one type in a closely related extension of a different type that lives in the same file).

···

On Jun 15, 2016, at 4:08 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
On Wed, Jun 15, 2016 at 3:09 PM, Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:

On Jun 15, 2016, at 2:55 PM, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> wrote:
On Wed, Jun 15, 2016 at 2:48 PM, Matthew Johnson via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>>wrote:

On Jun 15, 2016, at 2:46 PM, Adrian Zubarev via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

-Matthew

--
Adrian Zubarev
Sent with Airmail

Am 15. Juni 2016 um 21:40:37, Matthew Johnson (matthew@anandabits.com <mailto:matthew@anandabits.com>) schrieb:

What seems like a nasty bug missed during review? I don’t follow you there.

This proposal was specifically designed to follow Swift’s design of a scope-based access control mechanism rather than a type-based access control mechanism that is common in other languages.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

I was referencing to the issue Robert discovered in his implementation.

I do understand what the proposal is all about, but thank you for
re-clarifying that to me. :)

I don’t think it’s a bug, but it is definitely something that isn’t as
clear as it should have been.

Was it intentional on the part of the proposal, then, that there should
be two modifiers meaning the same thing for a top level declaration in a
file? Or was it intended that only one or the other be used in that
scenario?

I don’t think it was carefully considered, although I think it did come
up at some point during discussion in the context of compatibility with
existing code (i.e. nothing changes for current top-level `private`
declarations).

It is in some sense a “coincidence” that they mean the same thing at file
scope. The proposal would have had to introduce a specific prohibition to
prevent this situation and it did not do so. That said, I think this kind
of issue falls well within the discretion of the core team to make a call
without violating the spirit of the proposal.

There are two reasonable options here:

1. Allow both `private` and `fileprivate` at file scope despite the fact
that they have the same meaning. This is more consistent in the sense that
we are not introducing a special case that arbitrarily prohibits an
otherwise valid access modifier. It also means that nothing needs to
change for top level `private` declarations in existing code.

2. Prohibit `private` at file scope. Given that it appears as if the
behavior of `private` at file scope may not be intuitive and is equivalent
to `fileprivate` it might be reasonable to just disallow it. This would
result in more consistent *code* (even if there needs to be a special case
in the language).

I don’t have a strong opinion on which option we choose. But I do feel
strongly that the semantics of `private` need to properly respect the scope
in which the keyword is written and into which the associated declaration
is introduced (rather than the scope *inside* the declaration it is
attached to).

Right, I think both would be OK. More radically, we might want to
re-evaluate the continued utility of a `fileprivate` scope. It seems the
use cases for such a scope not adequately served by either `internal` or
the new `private` would be exceedingly rare.

I disagree with that. `fileprivate` is indispensable when you need it.
There are times when you want to keep visibility limited to the current
file but the new `private` is too restrictive (for example, you need to
access a member of one type in a closely related extension of a different
type that lives in the same file).

Sure. This was more of a thought for the future. As we move towards fully
embracing a scope-based model for organizing code, modules will no longer
need to be strictly "single units of code distribution," and a move towards
supporting submodules could serve your use case without `fileprivate`.
That, IMO, would be a logical endpoint of moving from file-based access to
scope-based access.

···

On Wed, Jun 15, 2016 at 4:14 PM, Matthew Johnson <matthew@anandabits.com> wrote:

On Jun 15, 2016, at 4:08 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
On Wed, Jun 15, 2016 at 3:09 PM, Matthew Johnson <matthew@anandabits.com> > wrote:

On Jun 15, 2016, at 2:55 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
On Wed, Jun 15, 2016 at 2:48 PM, Matthew Johnson via swift-evolution < >> swift-evolution@swift.org>wrote:

On Jun 15, 2016, at 2:46 PM, Adrian Zubarev via swift-evolution < >>> swift-evolution@swift.org> wrote:

-Matthew

--
Adrian Zubarev
Sent with Airmail

Am 15. Juni 2016 um 21:40:37, Matthew Johnson (matthew@anandabits.com)
schrieb:

What seems like a nasty bug missed during review? I don’t follow you
there.

This proposal was specifically designed to follow Swift’s design of a
scope-based access control mechanism rather than a type-based access
control mechanism that is common in other languages.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

I was referencing to the issue Robert discovered in his implementation.

I do understand what the proposal is all about, but thank you for re-clarifying that to me. :)

I don’t think it’s a bug, but it is definitely something that isn’t as clear as it should have been.

Was it intentional on the part of the proposal, then, that there should be two modifiers meaning the same thing for a top level declaration in a file? Or was it intended that only one or the other be used in that scenario?

I don’t think it was carefully considered, although I think it did come up at some point during discussion in the context of compatibility with existing code (i.e. nothing changes for current top-level `private` declarations).

It is in some sense a “coincidence” that they mean the same thing at file scope. The proposal would have had to introduce a specific prohibition to prevent this situation and it did not do so. That said, I think this kind of issue falls well within the discretion of the core team to make a call without violating the spirit of the proposal.

There are two reasonable options here:

1. Allow both `private` and `fileprivate` at file scope despite the fact that they have the same meaning. This is more consistent in the sense that we are not introducing a special case that arbitrarily prohibits an otherwise valid access modifier. It also means that nothing needs to change for top level `private` declarations in existing code.

2. Prohibit `private` at file scope. Given that it appears as if the behavior of `private` at file scope may not be intuitive and is equivalent to `fileprivate` it might be reasonable to just disallow it. This would result in more consistent *code* (even if there needs to be a special case in the language).

I don’t have a strong opinion on which option we choose. But I do feel strongly that the semantics of `private` need to properly respect the scope in which the keyword is written and into which the associated declaration is introduced (rather than the scope *inside* the declaration it is attached to).

Right, I think both would be OK. More radically, we might want to re-evaluate the continued utility of a `fileprivate` scope. It seems the use cases for such a scope not adequately served by either `internal` or the new `private` would be exceedingly rare.

I disagree with that. `fileprivate` is indispensable when you need it. There are times when you want to keep visibility limited to the current file but the new `private` is too restrictive (for example, you need to access a member of one type in a closely related extension of a different type that lives in the same file).

Sure. This was more of a thought for the future. As we move towards fully embracing a scope-based model for organizing code, modules will no longer need to be strictly "single units of code distribution," and a move towards supporting submodules could serve your use case without `fileprivate`. That, IMO, would be a logical endpoint of moving from file-based access to scope-based access.

I would really like to see submodules, but I think there would still be valid uses for `fileprivate` even with them. But of course we would need to know the details of submodules to have a good discussion about that so it’s a topic for the future. :)

···

On Jun 15, 2016, at 4:27 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
On Wed, Jun 15, 2016 at 4:14 PM, Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:

On Jun 15, 2016, at 4:08 PM, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> wrote:
On Wed, Jun 15, 2016 at 3:09 PM, Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:

On Jun 15, 2016, at 2:55 PM, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> wrote:
On Wed, Jun 15, 2016 at 2:48 PM, Matthew Johnson via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>>wrote:

On Jun 15, 2016, at 2:46 PM, Adrian Zubarev via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

-Matthew

--
Adrian Zubarev
Sent with Airmail

Am 15. Juni 2016 um 21:40:37, Matthew Johnson (matthew@anandabits.com <mailto:matthew@anandabits.com>) schrieb:

What seems like a nasty bug missed during review? I don’t follow you there.

This proposal was specifically designed to follow Swift’s design of a scope-based access control mechanism rather than a type-based access control mechanism that is common in other languages.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

I was referencing to the issue Robert discovered in his implementation.

I do understand what the proposal is all about, but thank you for
re-clarifying that to me. :)

I don’t think it’s a bug, but it is definitely something that isn’t as
clear as it should have been.

Was it intentional on the part of the proposal, then, that there should
be two modifiers meaning the same thing for a top level declaration in a
file? Or was it intended that only one or the other be used in that
scenario?

I don’t think it was carefully considered, although I think it did come
up at some point during discussion in the context of compatibility with
existing code (i.e. nothing changes for current top-level `private`
declarations).

It is in some sense a “coincidence” that they mean the same thing at
file scope. The proposal would have had to introduce a specific
prohibition to prevent this situation and it did not do so. That said, I
think this kind of issue falls well within the discretion of the core team
to make a call without violating the spirit of the proposal.

There are two reasonable options here:

1. Allow both `private` and `fileprivate` at file scope despite the fact
that they have the same meaning. This is more consistent in the sense that
we are not introducing a special case that arbitrarily prohibits an
otherwise valid access modifier. It also means that nothing needs to
change for top level `private` declarations in existing code.

2. Prohibit `private` at file scope. Given that it appears as if the
behavior of `private` at file scope may not be intuitive and is equivalent
to `fileprivate` it might be reasonable to just disallow it. This would
result in more consistent *code* (even if there needs to be a special case
in the language).

I don’t have a strong opinion on which option we choose. But I do feel
strongly that the semantics of `private` need to properly respect the scope
in which the keyword is written and into which the associated declaration
is introduced (rather than the scope *inside* the declaration it is
attached to).

Right, I think both would be OK. More radically, we might want to
re-evaluate the continued utility of a `fileprivate` scope. It seems the
use cases for such a scope not adequately served by either `internal` or
the new `private` would be exceedingly rare.

I disagree with that. `fileprivate` is indispensable when you need it.
There are times when you want to keep visibility limited to the current
file but the new `private` is too restrictive (for example, you need to
access a member of one type in a closely related extension of a different
type that lives in the same file).

Sure. This was more of a thought for the future. As we move towards fully
embracing a scope-based model for organizing code, modules will no longer
need to be strictly "single units of code distribution," and a move towards
supporting submodules could serve your use case without `fileprivate`.
That, IMO, would be a logical endpoint of moving from file-based access to
scope-based access.

I would really like to see submodules, but I think there would still be
valid uses for `fileprivate` even with them. But of course we would need
to know the details of submodules to have a good discussion about that so
it’s a topic for the future. :)

I wonder what became of this:
https://github.com/apple/swift/blob/master/docs/Modules.rst#id18

···

On Wed, Jun 15, 2016 at 4:30 PM, Matthew Johnson <matthew@anandabits.com> wrote:

On Jun 15, 2016, at 4:27 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
On Wed, Jun 15, 2016 at 4:14 PM, Matthew Johnson <matthew@anandabits.com> > wrote:

On Jun 15, 2016, at 4:08 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
On Wed, Jun 15, 2016 at 3:09 PM, Matthew Johnson <matthew@anandabits.com> >> wrote:

On Jun 15, 2016, at 2:55 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
On Wed, Jun 15, 2016 at 2:48 PM, Matthew Johnson via swift-evolution < >>> swift-evolution@swift.org>wrote:

On Jun 15, 2016, at 2:46 PM, Adrian Zubarev via swift-evolution < >>>> swift-evolution@swift.org> wrote:

-Matthew

--
Adrian Zubarev
Sent with Airmail

Am 15. Juni 2016 um 21:40:37, Matthew Johnson (matthew@anandabits.com)
schrieb:

What seems like a nasty bug missed during review? I don’t follow you
there.

This proposal was specifically designed to follow Swift’s design of a
scope-based access control mechanism rather than a type-based access
control mechanism that is common in other languages.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

I await their opinions

I as well. :)

, but even with that change we still have a problem with the philosophy here. The motivation behind the fileprivate/private distinction is, as you say in the proposal "[a] reliable way to hide implementation details...". Why would you want any members you're trying to hide to be visible in any scope other than the one you create with the decl (which is, without a doubt, also a lexical scope)?

Because you are trying to limit access to the scope that contains the declaration. Restricting access to a declaration that introduces a scope to *inside* the scope it contains doesn't make sense and has no precedent in Swift (or any other language that I know of). *That* is what makes the declaration useless.

It is analogous with other access control levels: a public outer decl implies public inner members

No it doesn't. Nothing is implicitly public in Swift.

just as a fileprivate one implies fileprivate inner members.

This is coincidental because that keyword (and internal) mean the same thing whether they are applied at file scope or inside another scope. The new 'private' modifier *does not* mean the same thing regardless of its containing scope. That difference is the whole point of introducing it. It scales to any scope in which it is used.

Is it not antithetical to being able to hide things if they remain visible despite the outer decl declaring itself supposedly being sealed?

I'm not sure why you mention 'sealed'. That is something else entirely.

In your examples, you declare private members that you do not wish to leak into top level scope because they are, after all, supposedly hidden. Should the same semantics you have given not also apply to private decls no matter where they lie?

The same semantics do apply everywhere. When members themselves are declared 'private' they do not "leak" out of the scope in which they are declared. They are indeed limited to the scope in which the member is declared.

But when a type / extension is declared 'private', 'private' appears within the scope of that declaration (not inside the scope it introduces). This means the type / extension is visible throughout its containing scope. Any members that do not have an access modifier simply receive the same visibility as the containing declaration.

When 'private' is used on a type / extension at the top level scope of a file it just happens to have behavior that coincides with the behavior of 'fileprivate'.

It seems like you expect 'private' to work one way when applied to members (restrict them to the surrounding scope) and another way when applied to types / extensions (restrict the members - and the type / extension itself??? - to the new scope that is introduced). That is inconsistent and confusing semantics IMO.

-Matthew

···

Sent from my iPad

On Jun 15, 2016, at 5:17 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

Should this compile:

private struct Outer {
    let inner = Inner()
}

Outer().inner

If this shouldn't:

struct Outer {
    private let inner = Inner()
}

extension Outer {
    var inner2 : Inner { return self.inner }
}

Visibility being a function of scope, lexical or otherwise, seems too coarse a metric for access control in general without further clarification of what the definition of "hidden" is.

~Robert Widmann

2016/06/15 12:38、Matthew Johnson <matthew@anandabits.com> のメッセージ:

On Jun 15, 2016, at 2:29 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

The meaning of private according to the proposal is not scope-dependent, it is decl-dependent. The mental gymnastics we are both going through right now are not in the proposal. I would like them to be.

I agree that the proposal is too sparse on the details and that is the source of the confusion.

Ilya wanted me to write this proposal and only did it himself because I didn’t have time to do it as quickly as he wanted. I know what the original intent of the proposal was (I can’t speak for the core team on their mental model of it when they accepted it).

That said, the title of the proposal is “scoped access level” and the detailed design says "visible only within that lexical scope”. And the entire discussion was always about lexical scope. I believe the reason the “proposed solution” section says “current declaration” is because *most* lexical scopes are introduced by a declaration. That is sloppy language that should have been cleaned up prior to review.

I appreciate you working on implementing this. I also understand if you want to hold off until the core team can chime in on the semantics they intend.

Matthew

~Robert Widmann

2016/06/15 12:26、Matthew Johnson <matthew@anandabits.com> のメッセージ:

On Jun 15, 2016, at 2:19 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

We have diagnostics specifically to prohibit this case. You cannot raise the access level of members.

private struct Foo {
    internal var x : String = ""
}

warning: declaring an internal var for a private struct.

Hence, the highest allowable level of access for x is private and it becomes invisible.

I would not like the compiler to synthesize this in my code, and if it did I would like the proposal to say it will raise access levels of members as you would like it to.

That diagnostic is a good thing. I am not suggesting that you to disregard it.

What you are missing is that the meaning of `private` is scope-dependent. The following example is perfectly valid:

private struct Foo {
    fileprivate var x : String = “”
}

`fileprivate` inside `Foo` specifies the same visibility as `private` at the top level, thus we are not “raising" the visibility of the member. If no modifier is provided for `x` it receives the same visibility as its containing type (up to `internal`).

Consider another example:

struct Foo {
    private struct Bar {
        var x : String = “”
    }
}

In this example we *cannot* mark `x` as `fileprivate` because that *would* raise the visibility of `x` beyond that of `Bar`. `Bar` is only visible inside `Foo`, not at file scope. This means that `x` also has visibility throughout the lexical scope of `Foo`, but not outside of it.

-Matthew

~Robert Widmann

2016/06/15 12:14、Matthew Johnson <matthew@anandabits.com> のメッセージ:

On Jun 15, 2016, at 2:04 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

2016/06/15 11:47、Matthew Johnson <matthew@anandabits.com> のメッセージ:

On Jun 15, 2016, at 1:37 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

The scope of the *declaration* is not the issue. The scope of its *members* is.

Let’s consider an example:

private struct Foo {
  var bar: Int
}

// elsewhere in the same file:
var foo = Foo(bar: 42)
foo.bar = 44

`Foo` is declared private. Private for this declaration is at the file scope. The `bar` member has no access modifier so it has the same visibility as the struct itself, which is file scope. This will also be true of the implicitly synthesized memberwise initializer.

No, it is also private. It does not inherit its parent scope because, following the letter of the proposal, that symbol will only be visible within the current declaration. We cannot arbitrarily break access control rules because it is convenient in one special case.

This means that it is possible to initialize `foo` with a newly constructed instance of `Foo` and to modify the `bar` member anywhere else in the same file.

bar is not visible here. If it were you could break access control rules.

If `bar` was also declared `private` this would not be possible as its visibility would be restricted to the surrounding scope of the initial declaration of `Foo`. This means `Foo` would need to provide an explicit initializer or factory method with `fileprivate` visibility in order to be usable.

bar is private. Declarations within Foo cannot decide to raise that access level to make themselves more visible. If this should be the case, the proposal must be amended as much.

Members with no explicit access modifier should have the same *visibility* as their containing type (with a maximum implicit visibility of internal), not the same *modifier* as their containing type. The only case where there is a distinction is the new `private` visibility. Maybe that is what is causing the confusion?

That is not what the proposal says. The proposal says it is invisible outside the current decl, which is the containing structure here.

The access modifier is applied to `Foo`, not `bar`. `Foo` is visible in the scope of the “current declaration” (this is badly worded - it should say current scope, which is the file). Because `Foo` has a visibility lower than `internal` the default visibility of its members match the visibility of `Foo`, which again is the current scope: the file. The detailed design section of the proposal is sparse, but it correctly uses the phrase "visible only within that lexical scope” rather than the less precise (in the case of top-level code) “current declaration”.

I didn’t write the proposal but I was very heavily involved in the discussions and IIRC I provided the original suggestion for introducing a scoped access modifier.

If my example was the following:

private struct Foo {
  private var bar: Int
}

Then what you are saying would be correct. However, The example I showed did not provide an access modifier for `bar`.

You cannot just apply the same *access modifier* to members of the type that do not contain an access modifier. You have to apply the same *visibility* (limited to internal). When a type is marked as `private` in the lexical scope of the file, its unmodified members will be visible in the same lexical scope as the type itself (which is the file in the current example).

-Matthew

Does this help?

-Matthew

~Robert Widmann

2016/06/15 11:36、Matthew Johnson <matthew@anandabits.com> のメッセージ:

The scope for a top-level declaration is the file itself. This means that top-level declarations with `private` and `fileprivate` should have the same behavior. They should not be uninstantiable or unusable.

-Matthew

On Jun 15, 2016, at 1:31 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org> wrote:

While implementing SE-0025 (fileprivate), I noticed an interesting bug in the proposal. Under the implementation outlined there, any top-level structure, class, or enum declared private cannot possibly be instantiated and so cannot be used in any way. Because of this, private top-level declarations are more often than not blown away entirely by the compiler for being unused. It seems strange to me to allow a key language feature to act solely as a hint to the optimizer to reduce the size of your binary. Perhaps the restrictions around private needs to be relaxed or the line between fileprivate and private needs to be investigated again by the community before inclusion in the language.

Thoughts?

~Robert Widmann
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

I await their opinions, but even with that change we still have a problem with the philosophy here. The motivation behind the fileprivate/private distinction is, as you say in the proposal "[a] reliable way to hide implementation details...". Why would you want any members you're trying to hide to be visible in any scope other than the one you create with the decl (which is, without a doubt, also a lexical scope)? It is analogous with other access control levels: a public outer decl implies public inner members just as a fileprivate one implies fileprivate inner members. Is it not antithetical to being able to hide things if they remain visible despite the outer decl declaring itself supposedly being sealed? In your examples, you declare private members that you do not wish to leak into top level scope because they are, after all, supposedly hidden. Should the same semantics you have given not also apply to private decls no matter where they lie?

Sorry, missed the example in my last reply.

Should this compile:

private struct Outer {
    let inner = Inner()
}

Outer().inner

Aside from the fact that you can't write a standalone statement at the top level of a file and it isn't clear what 'Inner' is, yes. This would be valid:

private struct Outer {
    struct Inner {}
    let inner = Inner()
}

private let inner = Outer().inner

Note that the top level 'inner' must be 'private' or 'fileprivate' because 'Inner' is not visible outside the file.

If this shouldn't:

struct Outer {
    private let inner = Inner()
}

extension Outer {
    var inner2 : Inner { return self.inner }
}

The extension introduces a new scope so this is not valid.

Visibility being a function of scope, lexical or otherwise, seems too coarse a metric for access control in general without further clarification of what the definition of "hidden" is.

The lexical scope of the declaration itself (not a scope introduced by the declaration) is perfectly clear. You may not like it or agree that it is desirable but it certainly isn't ambiguous.

···

Sent from my iPad

On Jun 15, 2016, at 5:17 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

~Robert Widmann

2016/06/15 12:38、Matthew Johnson <matthew@anandabits.com> のメッセージ:

On Jun 15, 2016, at 2:29 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

The meaning of private according to the proposal is not scope-dependent, it is decl-dependent. The mental gymnastics we are both going through right now are not in the proposal. I would like them to be.

I agree that the proposal is too sparse on the details and that is the source of the confusion.

Ilya wanted me to write this proposal and only did it himself because I didn’t have time to do it as quickly as he wanted. I know what the original intent of the proposal was (I can’t speak for the core team on their mental model of it when they accepted it).

That said, the title of the proposal is “scoped access level” and the detailed design says "visible only within that lexical scope”. And the entire discussion was always about lexical scope. I believe the reason the “proposed solution” section says “current declaration” is because *most* lexical scopes are introduced by a declaration. That is sloppy language that should have been cleaned up prior to review.

I appreciate you working on implementing this. I also understand if you want to hold off until the core team can chime in on the semantics they intend.

Matthew

~Robert Widmann

2016/06/15 12:26、Matthew Johnson <matthew@anandabits.com> のメッセージ:

On Jun 15, 2016, at 2:19 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

We have diagnostics specifically to prohibit this case. You cannot raise the access level of members.

private struct Foo {
    internal var x : String = ""
}

warning: declaring an internal var for a private struct.

Hence, the highest allowable level of access for x is private and it becomes invisible.

I would not like the compiler to synthesize this in my code, and if it did I would like the proposal to say it will raise access levels of members as you would like it to.

That diagnostic is a good thing. I am not suggesting that you to disregard it.

What you are missing is that the meaning of `private` is scope-dependent. The following example is perfectly valid:

private struct Foo {
    fileprivate var x : String = “”
}

`fileprivate` inside `Foo` specifies the same visibility as `private` at the top level, thus we are not “raising" the visibility of the member. If no modifier is provided for `x` it receives the same visibility as its containing type (up to `internal`).

Consider another example:

struct Foo {
    private struct Bar {
        var x : String = “”
    }
}

In this example we *cannot* mark `x` as `fileprivate` because that *would* raise the visibility of `x` beyond that of `Bar`. `Bar` is only visible inside `Foo`, not at file scope. This means that `x` also has visibility throughout the lexical scope of `Foo`, but not outside of it.

-Matthew

~Robert Widmann

2016/06/15 12:14、Matthew Johnson <matthew@anandabits.com> のメッセージ:

On Jun 15, 2016, at 2:04 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

2016/06/15 11:47、Matthew Johnson <matthew@anandabits.com> のメッセージ:

On Jun 15, 2016, at 1:37 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

The scope of the *declaration* is not the issue. The scope of its *members* is.

Let’s consider an example:

private struct Foo {
  var bar: Int
}

// elsewhere in the same file:
var foo = Foo(bar: 42)
foo.bar = 44

`Foo` is declared private. Private for this declaration is at the file scope. The `bar` member has no access modifier so it has the same visibility as the struct itself, which is file scope. This will also be true of the implicitly synthesized memberwise initializer.

No, it is also private. It does not inherit its parent scope because, following the letter of the proposal, that symbol will only be visible within the current declaration. We cannot arbitrarily break access control rules because it is convenient in one special case.

This means that it is possible to initialize `foo` with a newly constructed instance of `Foo` and to modify the `bar` member anywhere else in the same file.

bar is not visible here. If it were you could break access control rules.

If `bar` was also declared `private` this would not be possible as its visibility would be restricted to the surrounding scope of the initial declaration of `Foo`. This means `Foo` would need to provide an explicit initializer or factory method with `fileprivate` visibility in order to be usable.

bar is private. Declarations within Foo cannot decide to raise that access level to make themselves more visible. If this should be the case, the proposal must be amended as much.

Members with no explicit access modifier should have the same *visibility* as their containing type (with a maximum implicit visibility of internal), not the same *modifier* as their containing type. The only case where there is a distinction is the new `private` visibility. Maybe that is what is causing the confusion?

That is not what the proposal says. The proposal says it is invisible outside the current decl, which is the containing structure here.

The access modifier is applied to `Foo`, not `bar`. `Foo` is visible in the scope of the “current declaration” (this is badly worded - it should say current scope, which is the file). Because `Foo` has a visibility lower than `internal` the default visibility of its members match the visibility of `Foo`, which again is the current scope: the file. The detailed design section of the proposal is sparse, but it correctly uses the phrase "visible only within that lexical scope” rather than the less precise (in the case of top-level code) “current declaration”.

I didn’t write the proposal but I was very heavily involved in the discussions and IIRC I provided the original suggestion for introducing a scoped access modifier.

If my example was the following:

private struct Foo {
  private var bar: Int
}

Then what you are saying would be correct. However, The example I showed did not provide an access modifier for `bar`.

You cannot just apply the same *access modifier* to members of the type that do not contain an access modifier. You have to apply the same *visibility* (limited to internal). When a type is marked as `private` in the lexical scope of the file, its unmodified members will be visible in the same lexical scope as the type itself (which is the file in the current example).

-Matthew

Does this help?

-Matthew

~Robert Widmann

2016/06/15 11:36、Matthew Johnson <matthew@anandabits.com> のメッセージ:

The scope for a top-level declaration is the file itself. This means that top-level declarations with `private` and `fileprivate` should have the same behavior. They should not be uninstantiable or unusable.

-Matthew

On Jun 15, 2016, at 1:31 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org> wrote:

While implementing SE-0025 (fileprivate), I noticed an interesting bug in the proposal. Under the implementation outlined there, any top-level structure, class, or enum declared private cannot possibly be instantiated and so cannot be used in any way. Because of this, private top-level declarations are more often than not blown away entirely by the compiler for being unused. It seems strange to me to allow a key language feature to act solely as a hint to the optimizer to reduce the size of your binary. Perhaps the restrictions around private needs to be relaxed or the line between fileprivate and private needs to be investigated again by the community before inclusion in the language.

Thoughts?

~Robert Widmann
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Its my bad habit describing everything as a ‘bug’. Don’t judge me for that. I fully agree with you about the whole issue.

You probably missed // 2. file scope (Level 0), but it’s okay. So my second example was indeed correct. I described ‘levels’ starting from zero for each individual file, because it wouldn’t matter in my (wrong) private vs. fileprivate model.

···

--
Adrian Zubarev
Sent with Airmail

I was referencing to the issue Robert discovered in his implementation.

I do understand what the proposal is all about, but thank you for re-clarifying that to me. :)

I don’t think it’s a bug, but it is definitely something that isn’t as clear as it should have been.

Was it intentional on the part of the proposal, then, that there should be two modifiers meaning the same thing for a top level declaration in a file? Or was it intended that only one or the other be used in that scenario?

I don’t think it was carefully considered, although I think it did come up at some point during discussion in the context of compatibility with existing code (i.e. nothing changes for current top-level `private` declarations).

It is in some sense a “coincidence” that they mean the same thing at file scope. The proposal would have had to introduce a specific prohibition to prevent this situation and it did not do so. That said, I think this kind of issue falls well within the discretion of the core team to make a call without violating the spirit of the proposal.

There are two reasonable options here:

1. Allow both `private` and `fileprivate` at file scope despite the fact that they have the same meaning. This is more consistent in the sense that we are not introducing a special case that arbitrarily prohibits an otherwise valid access modifier. It also means that nothing needs to change for top level `private` declarations in existing code.

2. Prohibit `private` at file scope. Given that it appears as if the behavior of `private` at file scope may not be intuitive and is equivalent to `fileprivate` it might be reasonable to just disallow it. This would result in more consistent *code* (even if there needs to be a special case in the language).

I don’t have a strong opinion on which option we choose. But I do feel strongly that the semantics of `private` need to properly respect the scope in which the keyword is written and into which the associated declaration is introduced (rather than the scope *inside* the declaration it is attached to).

Right, I think both would be OK. More radically, we might want to re-evaluate the continued utility of a `fileprivate` scope. It seems the use cases for such a scope not adequately served by either `internal` or the new `private` would be exceedingly rare.

I disagree with that. `fileprivate` is indispensable when you need it. There are times when you want to keep visibility limited to the current file but the new `private` is too restrictive (for example, you need to access a member of one type in a closely related extension of a different type that lives in the same file).

Sure. This was more of a thought for the future. As we move towards fully embracing a scope-based model for organizing code, modules will no longer need to be strictly "single units of code distribution," and a move towards supporting submodules

Would be interesting to hear the core team on the topic of sub modules, considering how deep the implications would run to open that door (potential compatibility breaking change to the structure of the dylibs). Would be wise for them to say a quick possible/not-in-this-lifetime before people (including me) start to dream too much about them.

···

On Jun 15, 2016, at 11:27 PM, Xiaodi Wu via swift-evolution <swift-evolution@swift.org> wrote:

On Wed, Jun 15, 2016 at 4:14 PM, Matthew Johnson <matthew@anandabits.com> wrote:

On Jun 15, 2016, at 4:08 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Wed, Jun 15, 2016 at 3:09 PM, Matthew Johnson <matthew@anandabits.com> wrote:

On Jun 15, 2016, at 2:55 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Wed, Jun 15, 2016 at 2:48 PM, Matthew Johnson via swift-evolution <swift-evolution@swift.org>wrote:

On Jun 15, 2016, at 2:46 PM, Adrian Zubarev via swift-evolution <swift-evolution@swift.org> wrote:

could serve your use case without `fileprivate`. That, IMO, would be a logical endpoint of moving from file-based access to scope-based access.

-Matthew

--
Adrian Zubarev
Sent with Airmail

Am 15. Juni 2016 um 21:40:37, Matthew Johnson (matthew@anandabits.com) schrieb:

What seems like a nasty bug missed during review? I don’t follow you there.

This proposal was specifically designed to follow Swift’s design of a scope-based access control mechanism rather than a type-based access control mechanism that is common in other languages.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

I would really like to see submodules, but I think there would still be valid uses for `fileprivate` even with them. But of course we would need to know the details of submodules to have a good discussion about that so it’s a topic for the future. :)

I wonder what became of this: https://github.com/apple/swift/blob/master/docs/Modules.rst#id18

As the author of that document, it became clear (or maybe “it became murky”) that everyone wants different things from submodules, both for compiling their own targets and for importing other people’s targets. I’d almost suggest avoiding the word if you want to propose any of myriad features related to them:

- importing a subset of APIs
- having APIs not imported by default with the top-level module
- C++ namespacing within a module
- C++ namespacing within another module
- breaking up compilation units (i.e. not compiling the entire module as one unit)
- adding another access level between internal and fileprivate.
- adding another access level between fileprivate and private.
- something else?

(still catching up on the main thread, but I think Robert and Matthew are both right: we need to explicitly amend the proposal, and the behavior we want is fairly obvious)

Jordan

Its my bad habit describing everything as a ‘bug’. Don’t judge me for that. I fully agree with you about the whole issue.

No problem. :)

You probably missed // 2. file scope (Level 0), but it’s okay. So my second example was indeed correct. I described ‘levels’ starting from zero for each individual file, because it wouldn’t matter in my (wrong) private vs. fileprivate model.

Makes sense.

···

On Jun 15, 2016, at 2:53 PM, Adrian Zubarev via swift-evolution <swift-evolution@swift.org> wrote:

--
Adrian Zubarev
Sent with Airmail

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

I think you guys are making this more complicated than it is. The rules here seem fairly simple:

1. The default access level is “internal”.

2. A type declared “private” at the top level will be visible to the file only.

3. If they are not given an access level, properties declared on the type will get the default “internal” access level, but since their containing type is not visible outside the file, they will not be visible either.

Thus any properties on the type will effectively inherit the access level of that type, unless the type’s access level is broader than “internal”. For the example of a private type at the top level of a file, its properties will effectively be fileprivate.

Charles

···

On Jun 15, 2016, at 5:48 PM, Matthew Johnson via swift-evolution <swift-evolution@swift.org> wrote:

Sent from my iPad

On Jun 15, 2016, at 5:17 PM, Robert Widmann <devteam.codafi@gmail.com <mailto:devteam.codafi@gmail.com>> wrote:

I await their opinions, but even with that change we still have a problem with the philosophy here. The motivation behind the fileprivate/private distinction is, as you say in the proposal "[a] reliable way to hide implementation details...". Why would you want any members you're trying to hide to be visible in any scope other than the one you create with the decl (which is, without a doubt, also a lexical scope)? It is analogous with other access control levels: a public outer decl implies public inner members just as a fileprivate one implies fileprivate inner members. Is it not antithetical to being able to hide things if they remain visible despite the outer decl declaring itself supposedly being sealed? In your examples, you declare private members that you do not wish to leak into top level scope because they are, after all, supposedly hidden. Should the same semantics you have given not also apply to private decls no matter where they lie?

Sorry, missed the example in my last reply.

Should this compile:

private struct Outer {
    let inner = Inner()
}

Outer().inner

Aside from the fact that you can't write a standalone statement at the top level of a file and it isn't clear what 'Inner' is, yes. This would be valid:

private struct Outer {
    struct Inner {}
    let inner = Inner()
}

private let inner = Outer().inner

Note that the top level 'inner' must be 'private' or 'fileprivate' because 'Inner' is not visible outside the file.

If this shouldn't:

struct Outer {
    private let inner = Inner()
}

extension Outer {
    var inner2 : Inner { return self.inner }
}

The extension introduces a new scope so this is not valid.

Visibility being a function of scope, lexical or otherwise, seems too coarse a metric for access control in general without further clarification of what the definition of "hidden" is.

The lexical scope of the declaration itself (not a scope introduced by the declaration) is perfectly clear. You may not like it or agree that it is desirable but it certainly isn't ambiguous.

~Robert Widmann

2016/06/15 12:38、Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> のメッセージ:

On Jun 15, 2016, at 2:29 PM, Robert Widmann <devteam.codafi@gmail.com <mailto:devteam.codafi@gmail.com>> wrote:

The meaning of private according to the proposal is not scope-dependent, it is decl-dependent. The mental gymnastics we are both going through right now are not in the proposal. I would like them to be.

I agree that the proposal is too sparse on the details and that is the source of the confusion.

Ilya wanted me to write this proposal and only did it himself because I didn’t have time to do it as quickly as he wanted. I know what the original intent of the proposal was (I can’t speak for the core team on their mental model of it when they accepted it).

That said, the title of the proposal is “scoped access level” and the detailed design says "visible only within that lexical scope”. And the entire discussion was always about lexical scope. I believe the reason the “proposed solution” section says “current declaration” is because *most* lexical scopes are introduced by a declaration. That is sloppy language that should have been cleaned up prior to review.

I appreciate you working on implementing this. I also understand if you want to hold off until the core team can chime in on the semantics they intend.

Matthew

~Robert Widmann

2016/06/15 12:26、Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> のメッセージ:

On Jun 15, 2016, at 2:19 PM, Robert Widmann <devteam.codafi@gmail.com <mailto:devteam.codafi@gmail.com>> wrote:

We have diagnostics specifically to prohibit this case. You cannot raise the access level of members.

private struct Foo {
    internal var x : String = ""
}

warning: declaring an internal var for a private struct.

Hence, the highest allowable level of access for x is private and it becomes invisible.

I would not like the compiler to synthesize this in my code, and if it did I would like the proposal to say it will raise access levels of members as you would like it to.

That diagnostic is a good thing. I am not suggesting that you to disregard it.

What you are missing is that the meaning of `private` is scope-dependent. The following example is perfectly valid:

private struct Foo {
    fileprivate var x : String = “”
}

`fileprivate` inside `Foo` specifies the same visibility as `private` at the top level, thus we are not “raising" the visibility of the member. If no modifier is provided for `x` it receives the same visibility as its containing type (up to `internal`).

Consider another example:

struct Foo {
    private struct Bar {
        var x : String = “”
    }
}

In this example we *cannot* mark `x` as `fileprivate` because that *would* raise the visibility of `x` beyond that of `Bar`. `Bar` is only visible inside `Foo`, not at file scope. This means that `x` also has visibility throughout the lexical scope of `Foo`, but not outside of it.

-Matthew

~Robert Widmann

2016/06/15 12:14、Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> のメッセージ:

On Jun 15, 2016, at 2:04 PM, Robert Widmann <devteam.codafi@gmail.com <mailto:devteam.codafi@gmail.com>> wrote:

2016/06/15 11:47、Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> のメッセージ:

On Jun 15, 2016, at 1:37 PM, Robert Widmann <devteam.codafi@gmail.com <mailto:devteam.codafi@gmail.com>> wrote:

The scope of the *declaration* is not the issue. The scope of its *members* is.

Let’s consider an example:

private struct Foo {
  var bar: Int
}

// elsewhere in the same file:
var foo = Foo(bar: 42)
foo.bar = 44

`Foo` is declared private. Private for this declaration is at the file scope. The `bar` member has no access modifier so it has the same visibility as the struct itself, which is file scope. This will also be true of the implicitly synthesized memberwise initializer.

No, it is also private. It does not inherit its parent scope because, following the letter of the proposal, that symbol will only be visible within the current declaration. We cannot arbitrarily break access control rules because it is convenient in one special case.

This means that it is possible to initialize `foo` with a newly constructed instance of `Foo` and to modify the `bar` member anywhere else in the same file.

bar is not visible here. If it were you could break access control rules.

If `bar` was also declared `private` this would not be possible as its visibility would be restricted to the surrounding scope of the initial declaration of `Foo`. This means `Foo` would need to provide an explicit initializer or factory method with `fileprivate` visibility in order to be usable.

bar is private. Declarations within Foo cannot decide to raise that access level to make themselves more visible. If this should be the case, the proposal must be amended as much.

Members with no explicit access modifier should have the same *visibility* as their containing type (with a maximum implicit visibility of internal), not the same *modifier* as their containing type. The only case where there is a distinction is the new `private` visibility. Maybe that is what is causing the confusion?

That is not what the proposal says. The proposal says it is invisible outside the current decl, which is the containing structure here.

The access modifier is applied to `Foo`, not `bar`. `Foo` is visible in the scope of the “current declaration” (this is badly worded - it should say current scope, which is the file). Because `Foo` has a visibility lower than `internal` the default visibility of its members match the visibility of `Foo`, which again is the current scope: the file. The detailed design section of the proposal is sparse, but it correctly uses the phrase "visible only within that lexical scope” rather than the less precise (in the case of top-level code) “current declaration”.

I didn’t write the proposal but I was very heavily involved in the discussions and IIRC I provided the original suggestion for introducing a scoped access modifier.

If my example was the following:

private struct Foo {
  private var bar: Int
}

Then what you are saying would be correct. However, The example I showed did not provide an access modifier for `bar`.

You cannot just apply the same *access modifier* to members of the type that do not contain an access modifier. You have to apply the same *visibility* (limited to internal). When a type is marked as `private` in the lexical scope of the file, its unmodified members will be visible in the same lexical scope as the type itself (which is the file in the current example).

-Matthew

Does this help?

-Matthew

~Robert Widmann

2016/06/15 11:36、Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> のメッセージ:

The scope for a top-level declaration is the file itself. This means that top-level declarations with `private` and `fileprivate` should have the same behavior. They should not be uninstantiable or unusable.

-Matthew

On Jun 15, 2016, at 1:31 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

While implementing SE-0025 (fileprivate), I noticed an interesting bug in the proposal. Under the implementation outlined there, any top-level structure, class, or enum declared private cannot possibly be instantiated and so cannot be used in any way. Because of this, private top-level declarations are more often than not blown away entirely by the compiler for being unused. It seems strange to me to allow a key language feature to act solely as a hint to the optimizer to reduce the size of your binary. Perhaps the restrictions around private needs to be relaxed or the line between fileprivate and private needs to be investigated again by the community before inclusion in the language.

Thoughts?

~Robert Widmann
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Aggregation of first and third party frameworks into a single linkable module might be on that pile, if such aggregation was a path decided on to reduce application startup time.

-DW

···

On Jun 22, 2016, at 11:59 AM, Jordan Rose via swift-evolution <swift-evolution@swift.org> wrote:

I would really like to see submodules, but I think there would still be valid uses for `fileprivate` even with them. But of course we would need to know the details of submodules to have a good discussion about that so it’s a topic for the future. :)

I wonder what became of this: https://github.com/apple/swift/blob/master/docs/Modules.rst#id18

As the author of that document, it became clear (or maybe “it became murky”) that everyone wants different things from submodules, both for compiling their own targets and for importing other people’s targets. I’d almost suggest avoiding the word if you want to propose any of myriad features related to them:

- importing a subset of APIs
- having APIs not imported by default with the top-level module
- C++ namespacing within a module
- C++ namespacing within another module
- breaking up compilation units (i.e. not compiling the entire module as one unit)
- adding another access level between internal and fileprivate.
- adding another access level between fileprivate and private.
- something else?