[Discussion] A Problem With SE-0025?


(Erica Sadun) #1

Perhaps the proposal can be amended with this information with the approval of the core team. I imagine that a list of rules and a chart might be valuable. Even if adding to the proposal is the wrong way to go, a publicly accessible and vetted document would be useful to refer people to. Maybe someone can step up and form a working group.

-- E

···

On Jun 15, 2016, at 5:51 PM, Charles Srstka via swift-evolution <swift-evolution@swift.org> wrote:

I think you guys are making this more complicated than it is. The rules here seem fairly simple:

1. The default access level is “internal”.

2. A type declared “private” at the top level will be visible to the file only.

3. If they are not given an access level, properties declared on the type will get the default “internal” access level, but since their containing type is not visible outside the file, they will not be visible either.

Thus any properties on the type will effectively inherit the access level of that type, unless the type’s access level is broader than “internal”. For the example of a private type at the top level of a file, its properties will effectively be fileprivate.

Charles

On Jun 15, 2016, at 5:48 PM, Matthew Johnson via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Sent from my iPad

On Jun 15, 2016, at 5:17 PM, Robert Widmann <devteam.codafi@gmail.com <mailto:devteam.codafi@gmail.com>> wrote:

I await their opinions, but even with that change we still have a problem with the philosophy here. The motivation behind the fileprivate/private distinction is, as you say in the proposal "[a] reliable way to hide implementation details...". Why would you want any members you're trying to hide to be visible in any scope other than the one you create with the decl (which is, without a doubt, also a lexical scope)? It is analogous with other access control levels: a public outer decl implies public inner members just as a fileprivate one implies fileprivate inner members. Is it not antithetical to being able to hide things if they remain visible despite the outer decl declaring itself supposedly being sealed? In your examples, you declare private members that you do not wish to leak into top level scope because they are, after all, supposedly hidden. Should the same semantics you have given not also apply to private decls no matter where they lie?

Sorry, missed the example in my last reply.

Should this compile:

private struct Outer {
    let inner = Inner()
}

Outer().inner

Aside from the fact that you can't write a standalone statement at the top level of a file and it isn't clear what 'Inner' is, yes. This would be valid:

private struct Outer {
    struct Inner {}
    let inner = Inner()
}

private let inner = Outer().inner

Note that the top level 'inner' must be 'private' or 'fileprivate' because 'Inner' is not visible outside the file.

If this shouldn't:

struct Outer {
    private let inner = Inner()
}

extension Outer {
    var inner2 : Inner { return self.inner }
}

The extension introduces a new scope so this is not valid.

Visibility being a function of scope, lexical or otherwise, seems too coarse a metric for access control in general without further clarification of what the definition of "hidden" is.

The lexical scope of the declaration itself (not a scope introduced by the declaration) is perfectly clear. You may not like it or agree that it is desirable but it certainly isn't ambiguous.

~Robert Widmann

2016/06/15 12:38、Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> のメッセージ:

On Jun 15, 2016, at 2:29 PM, Robert Widmann <devteam.codafi@gmail.com <mailto:devteam.codafi@gmail.com>> wrote:

The meaning of private according to the proposal is not scope-dependent, it is decl-dependent. The mental gymnastics we are both going through right now are not in the proposal. I would like them to be.

I agree that the proposal is too sparse on the details and that is the source of the confusion.

Ilya wanted me to write this proposal and only did it himself because I didn’t have time to do it as quickly as he wanted. I know what the original intent of the proposal was (I can’t speak for the core team on their mental model of it when they accepted it).

That said, the title of the proposal is “scoped access level” and the detailed design says "visible only within that lexical scope”. And the entire discussion was always about lexical scope. I believe the reason the “proposed solution” section says “current declaration” is because *most* lexical scopes are introduced by a declaration. That is sloppy language that should have been cleaned up prior to review.

I appreciate you working on implementing this. I also understand if you want to hold off until the core team can chime in on the semantics they intend.

Matthew

~Robert Widmann

2016/06/15 12:26、Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> のメッセージ:

On Jun 15, 2016, at 2:19 PM, Robert Widmann <devteam.codafi@gmail.com <mailto:devteam.codafi@gmail.com>> wrote:

We have diagnostics specifically to prohibit this case. You cannot raise the access level of members.

private struct Foo {
    internal var x : String = ""
}

warning: declaring an internal var for a private struct.

Hence, the highest allowable level of access for x is private and it becomes invisible.

I would not like the compiler to synthesize this in my code, and if it did I would like the proposal to say it will raise access levels of members as you would like it to.

That diagnostic is a good thing. I am not suggesting that you to disregard it.

What you are missing is that the meaning of `private` is scope-dependent. The following example is perfectly valid:

private struct Foo {
    fileprivate var x : String = “”
}

`fileprivate` inside `Foo` specifies the same visibility as `private` at the top level, thus we are not “raising" the visibility of the member. If no modifier is provided for `x` it receives the same visibility as its containing type (up to `internal`).

Consider another example:

struct Foo {
    private struct Bar {
        var x : String = “”
    }
}

In this example we *cannot* mark `x` as `fileprivate` because that *would* raise the visibility of `x` beyond that of `Bar`. `Bar` is only visible inside `Foo`, not at file scope. This means that `x` also has visibility throughout the lexical scope of `Foo`, but not outside of it.

-Matthew

~Robert Widmann

2016/06/15 12:14、Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> のメッセージ:

On Jun 15, 2016, at 2:04 PM, Robert Widmann <devteam.codafi@gmail.com <mailto:devteam.codafi@gmail.com>> wrote:

2016/06/15 11:47、Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> のメッセージ:

On Jun 15, 2016, at 1:37 PM, Robert Widmann <devteam.codafi@gmail.com <mailto:devteam.codafi@gmail.com>> wrote:

The scope of the *declaration* is not the issue. The scope of its *members* is.

Let’s consider an example:

private struct Foo {
  var bar: Int
}

// elsewhere in the same file:
var foo = Foo(bar: 42)
foo.bar = 44

`Foo` is declared private. Private for this declaration is at the file scope. The `bar` member has no access modifier so it has the same visibility as the struct itself, which is file scope. This will also be true of the implicitly synthesized memberwise initializer.

No, it is also private. It does not inherit its parent scope because, following the letter of the proposal, that symbol will only be visible within the current declaration. We cannot arbitrarily break access control rules because it is convenient in one special case.

This means that it is possible to initialize `foo` with a newly constructed instance of `Foo` and to modify the `bar` member anywhere else in the same file.

bar is not visible here. If it were you could break access control rules.

If `bar` was also declared `private` this would not be possible as its visibility would be restricted to the surrounding scope of the initial declaration of `Foo`. This means `Foo` would need to provide an explicit initializer or factory method with `fileprivate` visibility in order to be usable.

bar is private. Declarations within Foo cannot decide to raise that access level to make themselves more visible. If this should be the case, the proposal must be amended as much.

Members with no explicit access modifier should have the same *visibility* as their containing type (with a maximum implicit visibility of internal), not the same *modifier* as their containing type. The only case where there is a distinction is the new `private` visibility. Maybe that is what is causing the confusion?

That is not what the proposal says. The proposal says it is invisible outside the current decl, which is the containing structure here.

The access modifier is applied to `Foo`, not `bar`. `Foo` is visible in the scope of the “current declaration” (this is badly worded - it should say current scope, which is the file). Because `Foo` has a visibility lower than `internal` the default visibility of its members match the visibility of `Foo`, which again is the current scope: the file. The detailed design section of the proposal is sparse, but it correctly uses the phrase "visible only within that lexical scope” rather than the less precise (in the case of top-level code) “current declaration”.

I didn’t write the proposal but I was very heavily involved in the discussions and IIRC I provided the original suggestion for introducing a scoped access modifier.

If my example was the following:

private struct Foo {
  private var bar: Int
}

Then what you are saying would be correct. However, The example I showed did not provide an access modifier for `bar`.

You cannot just apply the same *access modifier* to members of the type that do not contain an access modifier. You have to apply the same *visibility* (limited to internal). When a type is marked as `private` in the lexical scope of the file, its unmodified members will be visible in the same lexical scope as the type itself (which is the file in the current example).

-Matthew

Does this help?

-Matthew

~Robert Widmann

2016/06/15 11:36、Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> のメッセージ:

The scope for a top-level declaration is the file itself. This means that top-level declarations with `private` and `fileprivate` should have the same behavior. They should not be uninstantiable or unusable.

-Matthew

On Jun 15, 2016, at 1:31 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

While implementing SE-0025 (fileprivate), I noticed an interesting bug in the proposal. Under the implementation outlined there, any top-level structure, class, or enum declared private cannot possibly be instantiated and so cannot be used in any way. Because of this, private top-level declarations are more often than not blown away entirely by the compiler for being unused. It seems strange to me to allow a key language feature to act solely as a hint to the optimizer to reduce the size of your binary. Perhaps the restrictions around private needs to be relaxed or the line between fileprivate and private needs to be investigated again by the community before inclusion in the language.

Thoughts?

~Robert Widmann
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution


(Matthew Johnson) #2

Perhaps the proposal can be amended with this information with the approval of the core team. I imagine that a list of rules and a chart might be valuable. Even if adding to the proposal is the wrong way to go, a publicly accessible and vetted document would be useful to refer people to. Maybe someone can step up and form a working group.

I'm sure the core team will chime in when they have a chance (but probably not during WWDC). If Ilya doesn't reply and nobody else wants to do it I would be happy to clarify this in an update to the proposal after we hear from the core team.

···

Sent from my iPad

On Jun 15, 2016, at 8:48 PM, Erica Sadun via swift-evolution <swift-evolution@swift.org> wrote:

-- E

On Jun 15, 2016, at 5:51 PM, Charles Srstka via swift-evolution <swift-evolution@swift.org> wrote:

I think you guys are making this more complicated than it is. The rules here seem fairly simple:

1. The default access level is “internal”.

2. A type declared “private” at the top level will be visible to the file only.

3. If they are not given an access level, properties declared on the type will get the default “internal” access level, but since their containing type is not visible outside the file, they will not be visible either.

Thus any properties on the type will effectively inherit the access level of that type, unless the type’s access level is broader than “internal”. For the example of a private type at the top level of a file, its properties will effectively be fileprivate.

Charles

On Jun 15, 2016, at 5:48 PM, Matthew Johnson via swift-evolution <swift-evolution@swift.org> wrote:

Sent from my iPad

On Jun 15, 2016, at 5:17 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

I await their opinions, but even with that change we still have a problem with the philosophy here. The motivation behind the fileprivate/private distinction is, as you say in the proposal "[a] reliable way to hide implementation details...". Why would you want any members you're trying to hide to be visible in any scope other than the one you create with the decl (which is, without a doubt, also a lexical scope)? It is analogous with other access control levels: a public outer decl implies public inner members just as a fileprivate one implies fileprivate inner members. Is it not antithetical to being able to hide things if they remain visible despite the outer decl declaring itself supposedly being sealed? In your examples, you declare private members that you do not wish to leak into top level scope because they are, after all, supposedly hidden. Should the same semantics you have given not also apply to private decls no matter where they lie?

Sorry, missed the example in my last reply.

Should this compile:

private struct Outer {
    let inner = Inner()
}

Outer().inner

Aside from the fact that you can't write a standalone statement at the top level of a file and it isn't clear what 'Inner' is, yes. This would be valid:

private struct Outer {
    struct Inner {}
    let inner = Inner()
}

private let inner = Outer().inner

Note that the top level 'inner' must be 'private' or 'fileprivate' because 'Inner' is not visible outside the file.

If this shouldn't:

struct Outer {
    private let inner = Inner()
}

extension Outer {
    var inner2 : Inner { return self.inner }
}

The extension introduces a new scope so this is not valid.

Visibility being a function of scope, lexical or otherwise, seems too coarse a metric for access control in general without further clarification of what the definition of "hidden" is.

The lexical scope of the declaration itself (not a scope introduced by the declaration) is perfectly clear. You may not like it or agree that it is desirable but it certainly isn't ambiguous.

~Robert Widmann

2016/06/15 12:38、Matthew Johnson <matthew@anandabits.com> のメッセージ:

On Jun 15, 2016, at 2:29 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

The meaning of private according to the proposal is not scope-dependent, it is decl-dependent. The mental gymnastics we are both going through right now are not in the proposal. I would like them to be.

I agree that the proposal is too sparse on the details and that is the source of the confusion.

Ilya wanted me to write this proposal and only did it himself because I didn’t have time to do it as quickly as he wanted. I know what the original intent of the proposal was (I can’t speak for the core team on their mental model of it when they accepted it).

That said, the title of the proposal is “scoped access level” and the detailed design says "visible only within that lexical scope”. And the entire discussion was always about lexical scope. I believe the reason the “proposed solution” section says “current declaration” is because *most* lexical scopes are introduced by a declaration. That is sloppy language that should have been cleaned up prior to review.

I appreciate you working on implementing this. I also understand if you want to hold off until the core team can chime in on the semantics they intend.

Matthew

~Robert Widmann

2016/06/15 12:26、Matthew Johnson <matthew@anandabits.com> のメッセージ:

On Jun 15, 2016, at 2:19 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

We have diagnostics specifically to prohibit this case. You cannot raise the access level of members.

private struct Foo {
    internal var x : String = ""
}

warning: declaring an internal var for a private struct.

Hence, the highest allowable level of access for x is private and it becomes invisible.

I would not like the compiler to synthesize this in my code, and if it did I would like the proposal to say it will raise access levels of members as you would like it to.

That diagnostic is a good thing. I am not suggesting that you to disregard it.

What you are missing is that the meaning of `private` is scope-dependent. The following example is perfectly valid:

private struct Foo {
    fileprivate var x : String = “”
}

`fileprivate` inside `Foo` specifies the same visibility as `private` at the top level, thus we are not “raising" the visibility of the member. If no modifier is provided for `x` it receives the same visibility as its containing type (up to `internal`).

Consider another example:

struct Foo {
    private struct Bar {
        var x : String = “”
    }
}

In this example we *cannot* mark `x` as `fileprivate` because that *would* raise the visibility of `x` beyond that of `Bar`. `Bar` is only visible inside `Foo`, not at file scope. This means that `x` also has visibility throughout the lexical scope of `Foo`, but not outside of it.

-Matthew

~Robert Widmann

2016/06/15 12:14、Matthew Johnson <matthew@anandabits.com> のメッセージ:

On Jun 15, 2016, at 2:04 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

2016/06/15 11:47、Matthew Johnson <matthew@anandabits.com> のメッセージ:

On Jun 15, 2016, at 1:37 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

The scope of the *declaration* is not the issue. The scope of its *members* is.

Let’s consider an example:

private struct Foo {
  var bar: Int
}

// elsewhere in the same file:
var foo = Foo(bar: 42)
foo.bar = 44

`Foo` is declared private. Private for this declaration is at the file scope. The `bar` member has no access modifier so it has the same visibility as the struct itself, which is file scope. This will also be true of the implicitly synthesized memberwise initializer.

No, it is also private. It does not inherit its parent scope because, following the letter of the proposal, that symbol will only be visible within the current declaration. We cannot arbitrarily break access control rules because it is convenient in one special case.

This means that it is possible to initialize `foo` with a newly constructed instance of `Foo` and to modify the `bar` member anywhere else in the same file.

bar is not visible here. If it were you could break access control rules.

If `bar` was also declared `private` this would not be possible as its visibility would be restricted to the surrounding scope of the initial declaration of `Foo`. This means `Foo` would need to provide an explicit initializer or factory method with `fileprivate` visibility in order to be usable.

bar is private. Declarations within Foo cannot decide to raise that access level to make themselves more visible. If this should be the case, the proposal must be amended as much.

Members with no explicit access modifier should have the same *visibility* as their containing type (with a maximum implicit visibility of internal), not the same *modifier* as their containing type. The only case where there is a distinction is the new `private` visibility. Maybe that is what is causing the confusion?

That is not what the proposal says. The proposal says it is invisible outside the current decl, which is the containing structure here.

The access modifier is applied to `Foo`, not `bar`. `Foo` is visible in the scope of the “current declaration” (this is badly worded - it should say current scope, which is the file). Because `Foo` has a visibility lower than `internal` the default visibility of its members match the visibility of `Foo`, which again is the current scope: the file. The detailed design section of the proposal is sparse, but it correctly uses the phrase "visible only within that lexical scope” rather than the less precise (in the case of top-level code) “current declaration”.

I didn’t write the proposal but I was very heavily involved in the discussions and IIRC I provided the original suggestion for introducing a scoped access modifier.

If my example was the following:

private struct Foo {
  private var bar: Int
}

Then what you are saying would be correct. However, The example I showed did not provide an access modifier for `bar`.

You cannot just apply the same *access modifier* to members of the type that do not contain an access modifier. You have to apply the same *visibility* (limited to internal). When a type is marked as `private` in the lexical scope of the file, its unmodified members will be visible in the same lexical scope as the type itself (which is the file in the current example).

-Matthew

Does this help?

-Matthew

~Robert Widmann

2016/06/15 11:36、Matthew Johnson <matthew@anandabits.com> のメッセージ:

The scope for a top-level declaration is the file itself. This means that top-level declarations with `private` and `fileprivate` should have the same behavior. They should not be uninstantiable or unusable.

-Matthew

On Jun 15, 2016, at 1:31 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org> wrote:

While implementing SE-0025 (fileprivate), I noticed an interesting bug in the proposal. Under the implementation outlined there, any top-level structure, class, or enum declared private cannot possibly be instantiated and so cannot be used in any way. Because of this, private top-level declarations are more often than not blown away entirely by the compiler for being unused. It seems strange to me to allow a key language feature to act solely as a hint to the optimizer to reduce the size of your binary. Perhaps the restrictions around private needs to be relaxed or the line between fileprivate and private needs to be investigated again by the community before inclusion in the language.

Thoughts?

~Robert Widmann
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution