A Comprehensive Rethink of Access Levels in Swift


#1

*Introduction*

There has been a deluge of discussion about access levels lately, all
attempting to simplify the situation. Shortly after Swift 3 was released,
many people realized that the new access modifier story was far more
complex than the old one, and expressed the belief that the changes may
have been a mistake.

In the months that followed, more and more people came to share the same
view, and stage 2 of Swift 4 has seen a profusion of proposals addressing
the issue. These proposals are generally small and focus on changing just
one aspect of access control. However, given the situation we are in now,
it is important to look at the problem in its entirety and arrive at a
cohesive solution.

*Background*

During the Swift 3 timeframe there were lengthy debates about access
control. The end results were to adopt SE-0025
<https://github.com/apple/swift-evolution/blob/master/proposals/0025-scoped-access-level.md>,
which introduced the ‘fileprivate’ keyword and made ‘private’ scope-based,
and SE-0117
<https://github.com/apple/swift-evolution/blob/master/proposals/0117-non-public-subclassable-by-default.md>,
which made ‘public’ classes closed by default and introduced the ‘open’
keyword. At the time, there was broad agreement (and some dissent) that
these were the right changes to make.

That belief, as well as the numerous arguments which led to it, were
well-informed and thoughtfully considered. However, due to the inevitable
linear nature of time, they were not based on first-hand experience with
the new changes. Upon the release of Swift 3, we all gained that first-hand
experience, and it quickly became apparent to many people that the new
access control system was needlessly complicated, and not at all the
improvement it had been heralded as.

Rather than make it easy to encapsulate implementation details of related
types across multiple files, we had instead doubled down on requiring that
many things go in a single file or else reveal their secrets to the entire
module. Even worse, the new scope-based ‘private’ discouraged the preferred
style of using extensions to build up a type. To cap it off, we went from
needing to know two access modifier keywords (‘public’ and ‘private’) to
needing to know four of them (‘private’, ‘fileprivate’, ‘public’, and
‘open’) without even providing a way to share details across a small number
of related files.

*Motivation – Overview*

Many different ideas for access control have been expressed on the Swift
Evolution mailing list. Some people want ‘protected’ or ‘friend’ or
‘extensible’ or various other combinations of type-based visibility. Other
people (*cough* Slava) see no need for any levels finer than ‘internal’ at
all. The common points of agreement, though, are that ‘fileprivate’ is an
awkward spelling, and that the whole system is too complicated.

It is important that we as the Swift community are able to recognize our
mistakes, and even more important that we fix them. We originally thought
that the Swift 3 access control changes would be beneficial. However,
experience with them in practice has shown that not to be the case.
Instead, the language became more complex, and that has real costs. It is
time for a simplification.

The prevailing view from recent discussions is that there should be just
one access level more fine-grained than ‘internal’, and it should be
spelled ‘private’. Let us leave aside for the moment what its exact meaning
should be, and consider the other end of the scale.

*Motivation – Rethinking ‘public’*

Prior to Swift 3 we had just one access level more broad than ‘internal’,
and for simplicity of the model it would be desirable to achieve that
again. However, SE-0117 raised the point that certain library designs
require a class to have subclasses within the defining module, but not
outside. In other words, client code should not be able to create
subclasses, even though they exist in the library. Let us be clear: this is
a niche use-case, but it is important.

The most common situations are that a class either should not be
subclassable at all—in which case it is ‘final’—or that it should be
subclassable anywhere including client code. In order for a library to need
a publicly closed class, it must first of all be using classes rather than
a protocol with conforming structs, it must have a hierarchy with a parent
class that is exposed outside the module, it must have subclasses of that
parent class within the module, and it must also require that no external
subclasses can exist. Putting all those criteria together, we see that
closed classes are a rare thing to use. Nonetheless, they are valuable and
can enable certain compiler optimizations, so we should support them.

Currently, the spelling for a closed class is ‘public’. This makes it very
easy for library authors to create them. However, since they are a niche
feature and most of the time ‘final’ is a better choice, we do not need to
dedicate the entire ‘public’ keyword to them.

Moreover, object-oriented programming is just as much a first-class citizen
in Swift as protocol-oriented programming is, so we should treat it as
such. Classes are inherently inheritable: when one writes “class Foo {}”,
then Foo has a default visibility of ‘internal’, and by default it can have
subclasses. That is a straightforward model, and it is easy to work with.

If subclasses are to be disallowed, then Foo should be marked ‘final’; if
Foo is exported to clients then it should be marked ‘public’; and if both
are true then Foo should be ‘public final’. This covers all the common
cases, and leaves only the narrow corner of closed classes to consider. Per
the motivation of SE-0117, that case is worth handling. Per our collective
experience with Swift 3, however, it is not worth the added complexity of
its own access modifier keyword. We need a better way to spell it.

One of the reasons ‘public’ was previously chosen for closed classes is to
provide a “soft default” for library authors, so they can prevent
subclassing until they decide later whether to allow it in a future
release. This is a misguided decision, as it prioritizes the convenience of
library authors over the productivity of application developers. Library
authors have a responsibility to decide what interfaces they present, and
we should not encourage them to release libraries without making those
decisions.

Moreover, we need to trust client programmers to make reasonable choices.
If a library mistakenly allows subclassing when it shouldn’t, all a client
has to do to work with it correctly is *not make subclasses*. The library
is still usable. Conversely, if a library mistakenly prohibits subclassing,
then there are things a client *should* be able to do but cannot. The harm
to the users of a library is greater in this last case, because the ability
to use the library is compromised, and that diminishes their productivity.

We should not make “soft defaults” that tend to negatively impact the
clients of a library for the dubious benefit of enabling its author to
procrastinate on a basic design decision. If someone truly wants to publish
a library with a closed class, then we should support that. But it should
be an intentional decision, not a default.

*Motivation – Rethinking ‘final’*

The question then comes to spelling. It is evident that preventing
subclasses is closely related to being ‘final’. One possibility, then, is
to allow the ‘final’ keyword to take a parameter. The parameter would be an
access level, to indicate that the type acts like it is final when accessed
from at or above that level.

In particular, ‘final(public)’ would mean “this class cannot be subclassed
from outside the module”, or in other words “this class appears final
publicly, although it is nonfinal internally”. This approach is more
powerful than a ‘closed’ keyword because it also allows ‘final(internal)’,
meaning “this class appears final to the rest of the module, although it
can be subclassed privately”.

*Motivation – Rethinking ‘private’*

Now let us return to ‘private’, which as discussed earlier should be the
only modifier that is tighter than ‘internal’. The purpose of ‘private’ is
to enable encapsulation of related code, without revealing implementation
details to the rest of the module. It should be compatible with using
extensions to build up types, and it should not encourage overly-long files.

The natural definition, therefore, is that ‘private’ should mean “visible
in a small group of files which belong together as a unit”. Of course Swift
does not yet have submodules, and is not likely to gain them this year.
However, if we say that each file is implicitly its own submodule unless
otherwise specified, then the model works. In that view, ‘private’ will
mean “visible in this submodule”, and for the time being that is synonymous
with “visible in this file”.

Although this does not immediately enable lengthy files to be separated
along natural divisions, it does lay the groundwork to allow doing so in
the future when submodules arrive.

*Motivation – Summary*

By looking at access control in its entirety, we can adopt a system that
empowers both library authors and client programmers to organize their code
in a principled way, and to expose the interfaces they want in the places
they need. The complexity of the Swift 3 visibility story, which many
people now regret creating, will be replaced by a far simpler model which
in several respects is even more powerful.

Notably, being able to parameterize ‘final’ lets classes be closed not just
externally, but also in the rest of the module outside the ‘private’ scope
if desired. Furthermore, defining ‘private’ as being scoped to a group of
related files means that, as soon as we get the ability to create such
groups, it will no longer be necessary to write large files just to keep
implementation details hidden.

*Recommendations*

To recap, the ideas presented here focus on simplifying access control
while still supporting important use cases such as closed class
hierarchies. The indicated design uses just three familiar access keywords:

‘private’, to restrict visibility to a group of files, or just one file
until we get that capability.

‘internal’, which is the default and does not have to be written, for
module-wide visibility.

‘public’, to make visible outside the module, including the ability to
subclass.

Additionally, the design allows ‘final’ to take any one of those visibility
levels as a parameter, to indicate that the type should be treated as
‘final’ at and above the specified scope. Thus ‘final(public)’ prevents
subclassing outside the module, while ‘final(internal)’ prevents it outside
the ‘private’ scope. For consistency, ‘final(private)’ is also permitted,
although it means the same thing as ‘final’ by itself.

*Conclusion*

The Swift 3 access situation is harmful—as evidenced by the myriad calls to
fix it—not just because of its excessive complexity, but also because it
prioritizes convenience for library authors over utility for their clients,
and because it has no natural way to accommodate splitting large files into
smaller ones while preserving encapsulation.

We have an opportunity now to correct a mistake we have made, and to set a
precedent that we *will* correct our mistakes, rather than continue down an
undesirable path simply because it seemed like a good idea at the time.
When real-world experience demonstrates that a change has taken us in the
wrong direction, we can and should update our decisions based on that new
experience.

Therefore, in the situation at hand, we should reconsider our access
modifier story and choose a model which is both simple and powerful. I have
presented here my best efforts at describing such a system, and I offer it
as one possible way to move forward.

– Nevin


(Dimitri Racordon) #2

Thanks for that very well written pitch! Not only I think it summarizes most of the opinions I remember reading across the “myriad” of discussion, but it also proposes a quite reasonable reshaping of the system.

There’s just (in my pov) one additional argument in favour of that proposal that I’d like to add: it removes the asymmetry introduced by `open` (wrt access level modifiers), reusing `final` to denote how closed a class should be.

···

On 23 Feb 2017, at 22:56, Nevin Brackett-Rozinsky via swift-evolution <swift-evolution@swift.org<mailto:swift-evolution@swift.org>> wrote:

Introduction

There has been a deluge of discussion about access levels lately, all attempting to simplify the situation. Shortly after Swift 3 was released, many people realized that the new access modifier story was far more complex than the old one, and expressed the belief that the changes may have been a mistake.

In the months that followed, more and more people came to share the same view, and stage 2 of Swift 4 has seen a profusion of proposals addressing the issue. These proposals are generally small and focus on changing just one aspect of access control. However, given the situation we are in now, it is important to look at the problem in its entirety and arrive at a cohesive solution.

Background

During the Swift 3 timeframe there were lengthy debates about access control. The end results were to adopt SE-0025<https://github.com/apple/swift-evolution/blob/master/proposals/0025-scoped-access-level.md>, which introduced the ‘fileprivate’ keyword and made ‘private’ scope-based, and SE-0117<https://github.com/apple/swift-evolution/blob/master/proposals/0117-non-public-subclassable-by-default.md>, which made ‘public’ classes closed by default and introduced the ‘open’ keyword. At the time, there was broad agreement (and some dissent) that these were the right changes to make.

That belief, as well as the numerous arguments which led to it, were well-informed and thoughtfully considered. However, due to the inevitable linear nature of time, they were not based on first-hand experience with the new changes. Upon the release of Swift 3, we all gained that first-hand experience, and it quickly became apparent to many people that the new access control system was needlessly complicated, and not at all the improvement it had been heralded as.

Rather than make it easy to encapsulate implementation details of related types across multiple files, we had instead doubled down on requiring that many things go in a single file or else reveal their secrets to the entire module. Even worse, the new scope-based ‘private’ discouraged the preferred style of using extensions to build up a type. To cap it off, we went from needing to know two access modifier keywords (‘public’ and ‘private’) to needing to know four of them (‘private’, ‘fileprivate’, ‘public’, and ‘open’) without even providing a way to share details across a small number of related files.

Motivation – Overview

Many different ideas for access control have been expressed on the Swift Evolution mailing list. Some people want ‘protected’ or ‘friend’ or ‘extensible’ or various other combinations of type-based visibility. Other people (*cough* Slava) see no need for any levels finer than ‘internal’ at all. The common points of agreement, though, are that ‘fileprivate’ is an awkward spelling, and that the whole system is too complicated.

It is important that we as the Swift community are able to recognize our mistakes, and even more important that we fix them. We originally thought that the Swift 3 access control changes would be beneficial. However, experience with them in practice has shown that not to be the case. Instead, the language became more complex, and that has real costs. It is time for a simplification.

The prevailing view from recent discussions is that there should be just one access level more fine-grained than ‘internal’, and it should be spelled ‘private’. Let us leave aside for the moment what its exact meaning should be, and consider the other end of the scale.

Motivation – Rethinking ‘public’

Prior to Swift 3 we had just one access level more broad than ‘internal’, and for simplicity of the model it would be desirable to achieve that again. However, SE-0117 raised the point that certain library designs require a class to have subclasses within the defining module, but not outside. In other words, client code should not be able to create subclasses, even though they exist in the library. Let us be clear: this is a niche use-case, but it is important.

The most common situations are that a class either should not be subclassable at all—in which case it is ‘final’—or that it should be subclassable anywhere including client code. In order for a library to need a publicly closed class, it must first of all be using classes rather than a protocol with conforming structs, it must have a hierarchy with a parent class that is exposed outside the module, it must have subclasses of that parent class within the module, and it must also require that no external subclasses can exist. Putting all those criteria together, we see that closed classes are a rare thing to use. Nonetheless, they are valuable and can enable certain compiler optimizations, so we should support them.

Currently, the spelling for a closed class is ‘public’. This makes it very easy for library authors to create them. However, since they are a niche feature and most of the time ‘final’ is a better choice, we do not need to dedicate the entire ‘public’ keyword to them.

Moreover, object-oriented programming is just as much a first-class citizen in Swift as protocol-oriented programming is, so we should treat it as such. Classes are inherently inheritable: when one writes “class Foo {}”, then Foo has a default visibility of ‘internal’, and by default it can have subclasses. That is a straightforward model, and it is easy to work with.

If subclasses are to be disallowed, then Foo should be marked ‘final’; if Foo is exported to clients then it should be marked ‘public’; and if both are true then Foo should be ‘public final’. This covers all the common cases, and leaves only the narrow corner of closed classes to consider. Per the motivation of SE-0117, that case is worth handling. Per our collective experience with Swift 3, however, it is not worth the added complexity of its own access modifier keyword. We need a better way to spell it.

One of the reasons ‘public’ was previously chosen for closed classes is to provide a “soft default” for library authors, so they can prevent subclassing until they decide later whether to allow it in a future release. This is a misguided decision, as it prioritizes the convenience of library authors over the productivity of application developers. Library authors have a responsibility to decide what interfaces they present, and we should not encourage them to release libraries without making those decisions.

Moreover, we need to trust client programmers to make reasonable choices. If a library mistakenly allows subclassing when it shouldn’t, all a client has to do to work with it correctly is *not make subclasses*. The library is still usable. Conversely, if a library mistakenly prohibits subclassing, then there are things a client *should* be able to do but cannot. The harm to the users of a library is greater in this last case, because the ability to use the library is compromised, and that diminishes their productivity.

We should not make “soft defaults” that tend to negatively impact the clients of a library for the dubious benefit of enabling its author to procrastinate on a basic design decision. If someone truly wants to publish a library with a closed class, then we should support that. But it should be an intentional decision, not a default.

Motivation – Rethinking ‘final’

The question then comes to spelling. It is evident that preventing subclasses is closely related to being ‘final’. One possibility, then, is to allow the ‘final’ keyword to take a parameter. The parameter would be an access level, to indicate that the type acts like it is final when accessed from at or above that level.

In particular, ‘final(public)’ would mean “this class cannot be subclassed from outside the module”, or in other words “this class appears final publicly, although it is nonfinal internally”. This approach is more powerful than a ‘closed’ keyword because it also allows ‘final(internal)’, meaning “this class appears final to the rest of the module, although it can be subclassed privately”.

Motivation – Rethinking ‘private’

Now let us return to ‘private’, which as discussed earlier should be the only modifier that is tighter than ‘internal’. The purpose of ‘private’ is to enable encapsulation of related code, without revealing implementation details to the rest of the module. It should be compatible with using extensions to build up types, and it should not encourage overly-long files.

The natural definition, therefore, is that ‘private’ should mean “visible in a small group of files which belong together as a unit”. Of course Swift does not yet have submodules, and is not likely to gain them this year. However, if we say that each file is implicitly its own submodule unless otherwise specified, then the model works. In that view, ‘private’ will mean “visible in this submodule”, and for the time being that is synonymous with “visible in this file”.

Although this does not immediately enable lengthy files to be separated along natural divisions, it does lay the groundwork to allow doing so in the future when submodules arrive.

Motivation – Summary

By looking at access control in its entirety, we can adopt a system that empowers both library authors and client programmers to organize their code in a principled way, and to expose the interfaces they want in the places they need. The complexity of the Swift 3 visibility story, which many people now regret creating, will be replaced by a far simpler model which in several respects is even more powerful.

Notably, being able to parameterize ‘final’ lets classes be closed not just externally, but also in the rest of the module outside the ‘private’ scope if desired. Furthermore, defining ‘private’ as being scoped to a group of related files means that, as soon as we get the ability to create such groups, it will no longer be necessary to write large files just to keep implementation details hidden.

Recommendations

To recap, the ideas presented here focus on simplifying access control while still supporting important use cases such as closed class hierarchies. The indicated design uses just three familiar access keywords:

‘private’, to restrict visibility to a group of files, or just one file until we get that capability.

‘internal’, which is the default and does not have to be written, for module-wide visibility.

‘public’, to make visible outside the module, including the ability to subclass.

Additionally, the design allows ‘final’ to take any one of those visibility levels as a parameter, to indicate that the type should be treated as ‘final’ at and above the specified scope. Thus ‘final(public)’ prevents subclassing outside the module, while ‘final(internal)’ prevents it outside the ‘private’ scope. For consistency, ‘final(private)’ is also permitted, although it means the same thing as ‘final’ by itself.

Conclusion

The Swift 3 access situation is harmful—as evidenced by the myriad calls to fix it—not just because of its excessive complexity, but also because it prioritizes convenience for library authors over utility for their clients, and because it has no natural way to accommodate splitting large files into smaller ones while preserving encapsulation.

We have an opportunity now to correct a mistake we have made, and to set a precedent that we *will* correct our mistakes, rather than continue down an undesirable path simply because it seemed like a good idea at the time. When real-world experience demonstrates that a change has taken us in the wrong direction, we can and should update our decisions based on that new experience.

Therefore, in the situation at hand, we should reconsider our access modifier story and choose a model which is both simple and powerful. I have presented here my best efforts at describing such a system, and I offer it as one possible way to move forward.

– Nevin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org<mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution


(David Waite) #3

The prevailing view from recent discussions is that there should be just one access level more fine-grained than ‘internal’, and it should be spelled ‘private’. Let us leave aside for the moment what its exact meaning should be, and consider the other end of the scale.

I think the submodule discussion now goes against this being a prevailing view - submodules are directly tied to be an access control level *above* private/fileprivate.

My expectation is that with submodules, “fileprivate” would go away.

Moreover, object-oriented programming is just as much a first-class citizen in Swift as protocol-oriented programming is, so we should treat it as such. Classes are inherently inheritable: when one writes “class Foo {}”, then Foo has a default visibility of ‘internal’, and by default it can have subclasses. That is a straightforward model, and it is easy to work with.

There is nothing about OOP which requires subclass-based polymorphism. Rust and Go are two examples of modern languages that do not support subclassing at all.

In reality, there is less agreement on what “Object Oriented Programming” means to technologists than there is about the real meaning of “Moore’s Law"

One of the reasons ‘public’ was previously chosen for closed classes is to provide a “soft default” for library authors, so they can prevent subclassing until they decide later whether to allow it in a future release. This is a misguided decision, as it prioritizes the convenience of library authors over the productivity of application developers. Library authors have a responsibility to decide what interfaces they present, and we should not encourage them to release libraries without making those decisions.

I think SE-0117 and the discussion behind it is pretty clear about why this is the default. It is not about convenience, but safety.

Moreover, we need to trust client programmers to make reasonable choices. If a library mistakenly allows subclassing when it shouldn’t, all a client has to do to work with it correctly is *not make subclasses*. The library is still usable. Conversely, if a library mistakenly prohibits subclassing, then there are things a client *should* be able to do but cannot. The harm to the users of a library is greater in this last case, because the ability to use the library is compromised, and that diminishes their productivity.

If a client uses a library type which was never intended to be subclassed and subclasses it, the only options to take the library forward are to:
1. break the client
2. make that usage part of your supported API, and work around any issues that causes

I sympathize with app developers under deadlines. I wouldn’t mind Swift having an ‘escape hatch’ to let an app developer take responsibility and to then abuse a library’s access levels (if that were possible without restricting optimizations in the Swift compiler). But closed-by-default is safer, and lets poorly specified libraries evolve to be higher quality libraries without breaking existing usage that they never intended to support.

We should not make “soft defaults” that tend to negatively impact the clients of a library for the dubious benefit of enabling its author to procrastinate on a basic design decision. If someone truly wants to publish a library with a closed class, then we should support that. But it should be an intentional decision, not a default.

If you were proposing the default for public classes be final rather than closed, I would be on-board with that line of thinking.

If you proposed that final was redundant and should go away, leaving just public and public open, I’d find that defensible (I think even with closed as a default, it is useful to indicate within am module that a class is subclassed or not)

If you proposed that library authors should be forced into a decision, and should get a warning with a default of final until they label public classes as open or final or sealsed, I’d also be on-board with that option.

However, “open by default” is quite different than “final by default”. The default for structs and enums are final by default. Protocols are implementable, but they by definition should have defined semantics for doing so. Open-by-default subclasses have the potential to be modified in ways you were never expected, then handed back to you. Exposing a class the same way you would expose a struct, enum, or protocol and having it directly impact your resiliency because you forgot to also add “final” is just not as safe. I think the SE-0117 discussion covered this *very* well.

Motivation – Rethinking ‘private’

Now let us return to ‘private’, which as discussed earlier should be the only modifier that is tighter than ‘internal’. The purpose of ‘private’ is to enable encapsulation of related code, without revealing implementation details to the rest of the module. It should be compatible with using extensions to build up types, and it should not encourage overly-long files.

As said before, I think this is an incorrect conclusion.

My personal thinking has been swayed over the last few months, and I’ve gone from vehemently opposing private to seeing merit in it. I believe my opposition was not against scoped private, but that it both kept fileprivate as a required access level and a redundant one, because it was not part of a larger change.

Swift need a modifier between private and internal, and “fileprivate” is neither the feature we want, nor the spelling. Fileprivate should have been replaced with a more appropriate access level between private and internal, to represent ‘friend” relationships. I think we may be now looking at doing that, with submodules.

My tact is that keywords should first and foremost document developer intent.

If I were to say which members of a type *should* be private, it would be the members that are not properly designed to uphold class invariance. Allowing access to write a negative integer to a property that should only ever be positive would be a good example. Calling a member without holding a lock or first dispatching onto the right GCD queue would be another.

In that context, a private which restricts even same-file, build-up extensions could be a positive language feature, because the extensions are being built using the safer interface. This is a trade-off, as some interfaces (NSCoding is a prime example) may require access to members otherwise made private.

The natural definition, therefore, is that ‘private’ should mean “visible in a small group of files which belong together as a unit”. Of course Swift does not yet have submodules, and is not likely to gain them this year. However, if we say that each file is implicitly its own submodule unless otherwise specified, then the model works. In that view, ‘private’ will mean “visible in this submodule”, and for the time being that is synonymous with “visible in this file”.

I think internal is a more appropriate visibility within a submodule.

-DW

···

On Feb 23, 2017, at 2:56 PM, Nevin Brackett-Rozinsky via swift-evolution <swift-evolution@swift.org> wrote:


(David Hart) #4

Some very good points. But I’ll concentrate on what I disagree on to see where we can go from there.

Introduction

There has been a deluge of discussion about access levels lately, all attempting to simplify the situation. Shortly after Swift 3 was released, many people realized that the new access modifier story was far more complex than the old one, and expressed the belief that the changes may have been a mistake.

In the months that followed, more and more people came to share the same view, and stage 2 of Swift 4 has seen a profusion of proposals addressing the issue. These proposals are generally small and focus on changing just one aspect of access control. However, given the situation we are in now, it is important to look at the problem in its entirety and arrive at a cohesive solution.

Background

During the Swift 3 timeframe there were lengthy debates about access control. The end results were to adopt SE-0025 <https://github.com/apple/swift-evolution/blob/master/proposals/0025-scoped-access-level.md>, which introduced the ‘fileprivate’ keyword and made ‘private’ scope-based, and SE-0117 <https://github.com/apple/swift-evolution/blob/master/proposals/0117-non-public-subclassable-by-default.md>, which made ‘public’ classes closed by default and introduced the ‘open’ keyword. At the time, there was broad agreement (and some dissent) that these were the right changes to make.

That belief, as well as the numerous arguments which led to it, were well-informed and thoughtfully considered. However, due to the inevitable linear nature of time, they were not based on first-hand experience with the new changes. Upon the release of Swift 3, we all gained that first-hand experience, and it quickly became apparent to many people that the new access control system was needlessly complicated, and not at all the improvement it had been heralded as.

Rather than make it easy to encapsulate implementation details of related types across multiple files, we had instead doubled down on requiring that many things go in a single file or else reveal their secrets to the entire module. Even worse, the new scope-based ‘private’ discouraged the preferred style of using extensions to build up a type. To cap it off, we went from needing to know two access modifier keywords (‘public’ and ‘private’) to needing to know four of them (‘private’, ‘fileprivate’, ‘public’, and ‘open’) without even providing a way to share details across a small number of related files.

Motivation – Overview

Many different ideas for access control have been expressed on the Swift Evolution mailing list. Some people want ‘protected’ or ‘friend’ or ‘extensible’ or various other combinations of type-based visibility. Other people (*cough* Slava) see no need for any levels finer than ‘internal’ at all. The common points of agreement, though, are that ‘fileprivate’ is an awkward spelling, and that the whole system is too complicated.

It is important that we as the Swift community are able to recognize our mistakes, and even more important that we fix them. We originally thought that the Swift 3 access control changes would be beneficial. However, experience with them in practice has shown that not to be the case. Instead, the language became more complex, and that has real costs. It is time for a simplification.

The prevailing view from recent discussions is that there should be just one access level more fine-grained than ‘internal’, and it should be spelled ‘private’. Let us leave aside for the moment what its exact meaning should be, and consider the other end of the scale.

Motivation – Rethinking ‘public’

Prior to Swift 3 we had just one access level more broad than ‘internal’, and for simplicity of the model it would be desirable to achieve that again. However, SE-0117 raised the point that certain library designs require a class to have subclasses within the defining module, but not outside. In other words, client code should not be able to create subclasses, even though they exist in the library. Let us be clear: this is a niche use-case, but it is important.

The most common situations are that a class either should not be subclassable at all—in which case it is ‘final’—or that it should be subclassable anywhere including client code. In order for a library to need a publicly closed class, it must first of all be using classes rather than a protocol with conforming structs, it must have a hierarchy with a parent class that is exposed outside the module, it must have subclasses of that parent class within the module, and it must also require that no external subclasses can exist. Putting all those criteria together, we see that closed classes are a rare thing to use. Nonetheless, they are valuable and can enable certain compiler optimizations, so we should support them.

The whole design and motivation behind SE-0117 is that it is not rare to want classes to be freely subclass-able internally while being closed to subclassing externally. Do you have arguments to contradict the lengthy points mentioned in the SE-0117 motivation section? The reasons for the rarity above do not make much sense to me:

"it must first of all be using classes rather than a protocol with conforming structs”: we are discussing the merits of the different forms of subclassing prevention (public vs final), so I don’t see the reason for mentioning structs.
"it must have a hierarchy with a parent class that is exposed outside the module”: I don’t see why it requires a hierarchy. You might want the soft-default of public to apply to non-subclassed classes which you might subclass in a later version of the library.

Currently, the spelling for a closed class is ‘public’. This makes it very easy for library authors to create them. However, since they are a niche feature and most of the time ‘final’ is a better choice, we do not need to dedicate the entire ‘public’ keyword to them.

Moreover, object-oriented programming is just as much a first-class citizen in Swift as protocol-oriented programming is, so we should treat it as such. Classes are inherently inheritable: when one writes “class Foo {}”, then Foo has a default visibility of ‘internal’, and by default it can have subclasses. That is a straightforward model, and it is easy to work with.

If subclasses are to be disallowed, then Foo should be marked ‘final’; if Foo is exported to clients then it should be marked ‘public’; and if both are true then Foo should be ‘public final’. This covers all the common cases, and leaves only the narrow corner of closed classes to consider. Per the motivation of SE-0117, that case is worth handling. Per our collective experience with Swift 3, however, it is not worth the added complexity of its own access modifier keyword. We need a better way to spell it.

One of the reasons ‘public’ was previously chosen for closed classes is to provide a “soft default” for library authors, so they can prevent subclassing until they decide later whether to allow it in a future release. This is a misguided decision, as it prioritizes the convenience of library authors over the productivity of application developers. Library authors have a responsibility to decide what interfaces they present, and we should not encourage them to release libraries without making those decisions.

Moreover, we need to trust client programmers to make reasonable choices. If a library mistakenly allows subclassing when it shouldn’t, all a client has to do to work with it correctly is *not make subclasses*. The library is still usable. Conversely, if a library mistakenly prohibits subclassing, then there are things a client *should* be able to do but cannot. The harm to the users of a library is greater in this last case, because the ability to use the library is compromised, and that diminishes their productivity.

We should not make “soft defaults” that tend to negatively impact the clients of a library for the dubious benefit of enabling its author to procrastinate on a basic design decision. If someone truly wants to publish a library with a closed class, then we should support that. But it should be an intentional decision, not a default.

That’s a bit harsh. For me, it has nothing to do with procrastination, but with being safe-by-default, which is an important Swift goal. You are arguing for useful-by-default, which is only a different priority.

Motivation – Rethinking ‘final’

The question then comes to spelling. It is evident that preventing subclasses is closely related to being ‘final’. One possibility, then, is to allow the ‘final’ keyword to take a parameter. The parameter would be an access level, to indicate that the type acts like it is final when accessed from at or above that level.

In particular, ‘final(public)’ would mean “this class cannot be subclassed from outside the module”, or in other words “this class appears final publicly, although it is nonfinal internally”. This approach is more powerful than a ‘closed’ keyword because it also allows ‘final(internal)’, meaning “this class appears final to the rest of the module, although it can be subclassed privately”.

As you can guess, I’m not a fan of this change :slight_smile: I know that final has a weird place in the language now, but I’m relatively happy with the public/open design.

Motivation – Rethinking ‘private’

Now let us return to ‘private’, which as discussed earlier should be the only modifier that is tighter than ‘internal’. The purpose of ‘private’ is to enable encapsulation of related code, without revealing implementation details to the rest of the module. It should be compatible with using extensions to build up types, and it should not encourage overly-long files.

The natural definition, therefore, is that ‘private’ should mean “visible in a small group of files which belong together as a unit”. Of course Swift does not yet have submodules, and is not likely to gain them this year. However, if we say that each file is implicitly its own submodule unless otherwise specified, then the model works. In that view, ‘private’ will mean “visible in this submodule”, and for the time being that is synonymous with “visible in this file”.

This is very counterintuitive for me. I’m a proponent for going back to a file-based private but your solution has two big disadvantages for me:

it makes impossible to have a file-based scope in submodules
private would have different semantics depending if its in a submodule or not. As a consequence:
it would be confusing to learn and use
it would make copy-pasting/moving files from a submodule to a top-module potentially breaking:

x.swift in submodule A
class X {
    private init() {}
}

y.swift in submodule A
class Y {
    let x = X()
}

This above code would stop working when moved to a top-module scope because X’s initializer would become un-accessible to Y. That’s just one example, but I’d really like to stress that changing the scope of an access-level depending on the submodule context seems like a really bad idea to me.

···

On 23 Feb 2017, at 22:56, Nevin Brackett-Rozinsky via swift-evolution <swift-evolution@swift.org> wrote:

Although this does not immediately enable lengthy files to be separated along natural divisions, it does lay the groundwork to allow doing so in the future when submodules arrive.

Motivation – Summary

By looking at access control in its entirety, we can adopt a system that empowers both library authors and client programmers to organize their code in a principled way, and to expose the interfaces they want in the places they need. The complexity of the Swift 3 visibility story, which many people now regret creating, will be replaced by a far simpler model which in several respects is even more powerful.

Notably, being able to parameterize ‘final’ lets classes be closed not just externally, but also in the rest of the module outside the ‘private’ scope if desired. Furthermore, defining ‘private’ as being scoped to a group of related files means that, as soon as we get the ability to create such groups, it will no longer be necessary to write large files just to keep implementation details hidden.

Recommendations

To recap, the ideas presented here focus on simplifying access control while still supporting important use cases such as closed class hierarchies. The indicated design uses just three familiar access keywords:

‘private’, to restrict visibility to a group of files, or just one file until we get that capability.

‘internal’, which is the default and does not have to be written, for module-wide visibility.

‘public’, to make visible outside the module, including the ability to subclass.

Additionally, the design allows ‘final’ to take any one of those visibility levels as a parameter, to indicate that the type should be treated as ‘final’ at and above the specified scope. Thus ‘final(public)’ prevents subclassing outside the module, while ‘final(internal)’ prevents it outside the ‘private’ scope. For consistency, ‘final(private)’ is also permitted, although it means the same thing as ‘final’ by itself.

Conclusion

The Swift 3 access situation is harmful—as evidenced by the myriad calls to fix it—not just because of its excessive complexity, but also because it prioritizes convenience for library authors over utility for their clients, and because it has no natural way to accommodate splitting large files into smaller ones while preserving encapsulation.

We have an opportunity now to correct a mistake we have made, and to set a precedent that we *will* correct our mistakes, rather than continue down an undesirable path simply because it seemed like a good idea at the time. When real-world experience demonstrates that a change has taken us in the wrong direction, we can and should update our decisions based on that new experience.

Therefore, in the situation at hand, we should reconsider our access modifier story and choose a model which is both simple and powerful. I have presented here my best efforts at describing such a system, and I offer it as one possible way to move forward.

– Nevin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Rien) #5

Very well written.

Personally I am in favor of very simple systems and to put the responsibility for an application by the application developer (not a library developer). Though I understand that in some cases (Apple!) the developer is the end-customer, and this creates special circumstances warranting concessions on the achievable simplicity.

While it is possible to create an even simpler access level system, I think the system you proposed is probably the best compromise between “simple enough” and “complex enough”, and I could certainly live with it.

The final word (imo) should come after the submodule’s discussion reaches its inevitable peak.

Regards,
Rien

Site: http://balancingrock.nl
Blog: http://swiftrien.blogspot.com
Github: http://github.com/Balancingrock
Project: http://swiftfire.nl

···

On 23 Feb 2017, at 22:56, Nevin Brackett-Rozinsky via swift-evolution <swift-evolution@swift.org> wrote:

Introduction

There has been a deluge of discussion about access levels lately, all attempting to simplify the situation. Shortly after Swift 3 was released, many people realized that the new access modifier story was far more complex than the old one, and expressed the belief that the changes may have been a mistake.

In the months that followed, more and more people came to share the same view, and stage 2 of Swift 4 has seen a profusion of proposals addressing the issue. These proposals are generally small and focus on changing just one aspect of access control. However, given the situation we are in now, it is important to look at the problem in its entirety and arrive at a cohesive solution.

Background

During the Swift 3 timeframe there were lengthy debates about access control. The end results were to adopt SE-0025, which introduced the ‘fileprivate’ keyword and made ‘private’ scope-based, and SE-0117, which made ‘public’ classes closed by default and introduced the ‘open’ keyword. At the time, there was broad agreement (and some dissent) that these were the right changes to make.

That belief, as well as the numerous arguments which led to it, were well-informed and thoughtfully considered. However, due to the inevitable linear nature of time, they were not based on first-hand experience with the new changes. Upon the release of Swift 3, we all gained that first-hand experience, and it quickly became apparent to many people that the new access control system was needlessly complicated, and not at all the improvement it had been heralded as.

Rather than make it easy to encapsulate implementation details of related types across multiple files, we had instead doubled down on requiring that many things go in a single file or else reveal their secrets to the entire module. Even worse, the new scope-based ‘private’ discouraged the preferred style of using extensions to build up a type. To cap it off, we went from needing to know two access modifier keywords (‘public’ and ‘private’) to needing to know four of them (‘private’, ‘fileprivate’, ‘public’, and ‘open’) without even providing a way to share details across a small number of related files.

Motivation – Overview

Many different ideas for access control have been expressed on the Swift Evolution mailing list. Some people want ‘protected’ or ‘friend’ or ‘extensible’ or various other combinations of type-based visibility. Other people (*cough* Slava) see no need for any levels finer than ‘internal’ at all. The common points of agreement, though, are that ‘fileprivate’ is an awkward spelling, and that the whole system is too complicated.

It is important that we as the Swift community are able to recognize our mistakes, and even more important that we fix them. We originally thought that the Swift 3 access control changes would be beneficial. However, experience with them in practice has shown that not to be the case. Instead, the language became more complex, and that has real costs. It is time for a simplification.

The prevailing view from recent discussions is that there should be just one access level more fine-grained than ‘internal’, and it should be spelled ‘private’. Let us leave aside for the moment what its exact meaning should be, and consider the other end of the scale.

Motivation – Rethinking ‘public’

Prior to Swift 3 we had just one access level more broad than ‘internal’, and for simplicity of the model it would be desirable to achieve that again. However, SE-0117 raised the point that certain library designs require a class to have subclasses within the defining module, but not outside. In other words, client code should not be able to create subclasses, even though they exist in the library. Let us be clear: this is a niche use-case, but it is important.

The most common situations are that a class either should not be subclassable at all—in which case it is ‘final’—or that it should be subclassable anywhere including client code. In order for a library to need a publicly closed class, it must first of all be using classes rather than a protocol with conforming structs, it must have a hierarchy with a parent class that is exposed outside the module, it must have subclasses of that parent class within the module, and it must also require that no external subclasses can exist. Putting all those criteria together, we see that closed classes are a rare thing to use. Nonetheless, they are valuable and can enable certain compiler optimizations, so we should support them.

Currently, the spelling for a closed class is ‘public’. This makes it very easy for library authors to create them. However, since they are a niche feature and most of the time ‘final’ is a better choice, we do not need to dedicate the entire ‘public’ keyword to them.

Moreover, object-oriented programming is just as much a first-class citizen in Swift as protocol-oriented programming is, so we should treat it as such. Classes are inherently inheritable: when one writes “class Foo {}”, then Foo has a default visibility of ‘internal’, and by default it can have subclasses. That is a straightforward model, and it is easy to work with.

If subclasses are to be disallowed, then Foo should be marked ‘final’; if Foo is exported to clients then it should be marked ‘public’; and if both are true then Foo should be ‘public final’. This covers all the common cases, and leaves only the narrow corner of closed classes to consider. Per the motivation of SE-0117, that case is worth handling. Per our collective experience with Swift 3, however, it is not worth the added complexity of its own access modifier keyword. We need a better way to spell it.

One of the reasons ‘public’ was previously chosen for closed classes is to provide a “soft default” for library authors, so they can prevent subclassing until they decide later whether to allow it in a future release. This is a misguided decision, as it prioritizes the convenience of library authors over the productivity of application developers. Library authors have a responsibility to decide what interfaces they present, and we should not encourage them to release libraries without making those decisions.

Moreover, we need to trust client programmers to make reasonable choices. If a library mistakenly allows subclassing when it shouldn’t, all a client has to do to work with it correctly is *not make subclasses*. The library is still usable. Conversely, if a library mistakenly prohibits subclassing, then there are things a client *should* be able to do but cannot. The harm to the users of a library is greater in this last case, because the ability to use the library is compromised, and that diminishes their productivity.

We should not make “soft defaults” that tend to negatively impact the clients of a library for the dubious benefit of enabling its author to procrastinate on a basic design decision. If someone truly wants to publish a library with a closed class, then we should support that. But it should be an intentional decision, not a default.

Motivation – Rethinking ‘final’

The question then comes to spelling. It is evident that preventing subclasses is closely related to being ‘final’. One possibility, then, is to allow the ‘final’ keyword to take a parameter. The parameter would be an access level, to indicate that the type acts like it is final when accessed from at or above that level.

In particular, ‘final(public)’ would mean “this class cannot be subclassed from outside the module”, or in other words “this class appears final publicly, although it is nonfinal internally”. This approach is more powerful than a ‘closed’ keyword because it also allows ‘final(internal)’, meaning “this class appears final to the rest of the module, although it can be subclassed privately”.

Motivation – Rethinking ‘private’

Now let us return to ‘private’, which as discussed earlier should be the only modifier that is tighter than ‘internal’. The purpose of ‘private’ is to enable encapsulation of related code, without revealing implementation details to the rest of the module. It should be compatible with using extensions to build up types, and it should not encourage overly-long files.

The natural definition, therefore, is that ‘private’ should mean “visible in a small group of files which belong together as a unit”. Of course Swift does not yet have submodules, and is not likely to gain them this year. However, if we say that each file is implicitly its own submodule unless otherwise specified, then the model works. In that view, ‘private’ will mean “visible in this submodule”, and for the time being that is synonymous with “visible in this file”.

Although this does not immediately enable lengthy files to be separated along natural divisions, it does lay the groundwork to allow doing so in the future when submodules arrive.

Motivation – Summary

By looking at access control in its entirety, we can adopt a system that empowers both library authors and client programmers to organize their code in a principled way, and to expose the interfaces they want in the places they need. The complexity of the Swift 3 visibility story, which many people now regret creating, will be replaced by a far simpler model which in several respects is even more powerful.

Notably, being able to parameterize ‘final’ lets classes be closed not just externally, but also in the rest of the module outside the ‘private’ scope if desired. Furthermore, defining ‘private’ as being scoped to a group of related files means that, as soon as we get the ability to create such groups, it will no longer be necessary to write large files just to keep implementation details hidden.

Recommendations

To recap, the ideas presented here focus on simplifying access control while still supporting important use cases such as closed class hierarchies. The indicated design uses just three familiar access keywords:

‘private’, to restrict visibility to a group of files, or just one file until we get that capability.

‘internal’, which is the default and does not have to be written, for module-wide visibility.

‘public’, to make visible outside the module, including the ability to subclass.

Additionally, the design allows ‘final’ to take any one of those visibility levels as a parameter, to indicate that the type should be treated as ‘final’ at and above the specified scope. Thus ‘final(public)’ prevents subclassing outside the module, while ‘final(internal)’ prevents it outside the ‘private’ scope. For consistency, ‘final(private)’ is also permitted, although it means the same thing as ‘final’ by itself.

Conclusion

The Swift 3 access situation is harmful—as evidenced by the myriad calls to fix it—not just because of its excessive complexity, but also because it prioritizes convenience for library authors over utility for their clients, and because it has no natural way to accommodate splitting large files into smaller ones while preserving encapsulation.

We have an opportunity now to correct a mistake we have made, and to set a precedent that we *will* correct our mistakes, rather than continue down an undesirable path simply because it seemed like a good idea at the time. When real-world experience demonstrates that a change has taken us in the wrong direction, we can and should update our decisions based on that new experience.

Therefore, in the situation at hand, we should reconsider our access modifier story and choose a model which is both simple and powerful. I have presented here my best efforts at describing such a system, and I offer it as one possible way to move forward.

– Nevin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Jon Hull) #6

+1000

This is the best of the proposals I have seen. I think it works by itself as a complete proposal, but since we are talking "comprehensive rethink", there is one use case which most of the proposals seem to miss.

That is, what if I want to make a class which is both Public and Subclassable, but I want to keep some of the implementation details private to the class, while still allowing subclasses and extensions (across the module boundary) to have access to those details. A concrete example of this is UIGestureRecognizer, which is intended to be subclassed outside of its framework, but hides potentially catastrophic things (like being able to set the state) from callers. This isn’t currently possible in Swift AFAICT without importing from ObjC.

My solution to this has been to allow something to be marked ‘public hidden’ or ‘hidden', which means that it is public/internal, but has it’s visibility limited only to files which opt-in to seeing it with ‘import hidden filename’ (instead of just ‘import filename’). I think this is the simplest way to provide this feature, which is sometimes very necessary. It also captures authorial intent very nicely.

Thanks,
Jon

···

On Feb 23, 2017, at 1:56 PM, Nevin Brackett-Rozinsky via swift-evolution <swift-evolution@swift.org> wrote:

Introduction

There has been a deluge of discussion about access levels lately, all attempting to simplify the situation. Shortly after Swift 3 was released, many people realized that the new access modifier story was far more complex than the old one, and expressed the belief that the changes may have been a mistake.

In the months that followed, more and more people came to share the same view, and stage 2 of Swift 4 has seen a profusion of proposals addressing the issue. These proposals are generally small and focus on changing just one aspect of access control. However, given the situation we are in now, it is important to look at the problem in its entirety and arrive at a cohesive solution.

Background

During the Swift 3 timeframe there were lengthy debates about access control. The end results were to adopt SE-0025 <https://github.com/apple/swift-evolution/blob/master/proposals/0025-scoped-access-level.md>, which introduced the ‘fileprivate’ keyword and made ‘private’ scope-based, and SE-0117 <https://github.com/apple/swift-evolution/blob/master/proposals/0117-non-public-subclassable-by-default.md>, which made ‘public’ classes closed by default and introduced the ‘open’ keyword. At the time, there was broad agreement (and some dissent) that these were the right changes to make.

That belief, as well as the numerous arguments which led to it, were well-informed and thoughtfully considered. However, due to the inevitable linear nature of time, they were not based on first-hand experience with the new changes. Upon the release of Swift 3, we all gained that first-hand experience, and it quickly became apparent to many people that the new access control system was needlessly complicated, and not at all the improvement it had been heralded as.

Rather than make it easy to encapsulate implementation details of related types across multiple files, we had instead doubled down on requiring that many things go in a single file or else reveal their secrets to the entire module. Even worse, the new scope-based ‘private’ discouraged the preferred style of using extensions to build up a type. To cap it off, we went from needing to know two access modifier keywords (‘public’ and ‘private’) to needing to know four of them (‘private’, ‘fileprivate’, ‘public’, and ‘open’) without even providing a way to share details across a small number of related files.

Motivation – Overview

Many different ideas for access control have been expressed on the Swift Evolution mailing list. Some people want ‘protected’ or ‘friend’ or ‘extensible’ or various other combinations of type-based visibility. Other people (*cough* Slava) see no need for any levels finer than ‘internal’ at all. The common points of agreement, though, are that ‘fileprivate’ is an awkward spelling, and that the whole system is too complicated.

It is important that we as the Swift community are able to recognize our mistakes, and even more important that we fix them. We originally thought that the Swift 3 access control changes would be beneficial. However, experience with them in practice has shown that not to be the case. Instead, the language became more complex, and that has real costs. It is time for a simplification.

The prevailing view from recent discussions is that there should be just one access level more fine-grained than ‘internal’, and it should be spelled ‘private’. Let us leave aside for the moment what its exact meaning should be, and consider the other end of the scale.

Motivation – Rethinking ‘public’

Prior to Swift 3 we had just one access level more broad than ‘internal’, and for simplicity of the model it would be desirable to achieve that again. However, SE-0117 raised the point that certain library designs require a class to have subclasses within the defining module, but not outside. In other words, client code should not be able to create subclasses, even though they exist in the library. Let us be clear: this is a niche use-case, but it is important.

The most common situations are that a class either should not be subclassable at all—in which case it is ‘final’—or that it should be subclassable anywhere including client code. In order for a library to need a publicly closed class, it must first of all be using classes rather than a protocol with conforming structs, it must have a hierarchy with a parent class that is exposed outside the module, it must have subclasses of that parent class within the module, and it must also require that no external subclasses can exist. Putting all those criteria together, we see that closed classes are a rare thing to use. Nonetheless, they are valuable and can enable certain compiler optimizations, so we should support them.

Currently, the spelling for a closed class is ‘public’. This makes it very easy for library authors to create them. However, since they are a niche feature and most of the time ‘final’ is a better choice, we do not need to dedicate the entire ‘public’ keyword to them.

Moreover, object-oriented programming is just as much a first-class citizen in Swift as protocol-oriented programming is, so we should treat it as such. Classes are inherently inheritable: when one writes “class Foo {}”, then Foo has a default visibility of ‘internal’, and by default it can have subclasses. That is a straightforward model, and it is easy to work with.

If subclasses are to be disallowed, then Foo should be marked ‘final’; if Foo is exported to clients then it should be marked ‘public’; and if both are true then Foo should be ‘public final’. This covers all the common cases, and leaves only the narrow corner of closed classes to consider. Per the motivation of SE-0117, that case is worth handling. Per our collective experience with Swift 3, however, it is not worth the added complexity of its own access modifier keyword. We need a better way to spell it.

One of the reasons ‘public’ was previously chosen for closed classes is to provide a “soft default” for library authors, so they can prevent subclassing until they decide later whether to allow it in a future release. This is a misguided decision, as it prioritizes the convenience of library authors over the productivity of application developers. Library authors have a responsibility to decide what interfaces they present, and we should not encourage them to release libraries without making those decisions.

Moreover, we need to trust client programmers to make reasonable choices. If a library mistakenly allows subclassing when it shouldn’t, all a client has to do to work with it correctly is *not make subclasses*. The library is still usable. Conversely, if a library mistakenly prohibits subclassing, then there are things a client *should* be able to do but cannot. The harm to the users of a library is greater in this last case, because the ability to use the library is compromised, and that diminishes their productivity.

We should not make “soft defaults” that tend to negatively impact the clients of a library for the dubious benefit of enabling its author to procrastinate on a basic design decision. If someone truly wants to publish a library with a closed class, then we should support that. But it should be an intentional decision, not a default.

Motivation – Rethinking ‘final’

The question then comes to spelling. It is evident that preventing subclasses is closely related to being ‘final’. One possibility, then, is to allow the ‘final’ keyword to take a parameter. The parameter would be an access level, to indicate that the type acts like it is final when accessed from at or above that level.

In particular, ‘final(public)’ would mean “this class cannot be subclassed from outside the module”, or in other words “this class appears final publicly, although it is nonfinal internally”. This approach is more powerful than a ‘closed’ keyword because it also allows ‘final(internal)’, meaning “this class appears final to the rest of the module, although it can be subclassed privately”.

Motivation – Rethinking ‘private’

Now let us return to ‘private’, which as discussed earlier should be the only modifier that is tighter than ‘internal’. The purpose of ‘private’ is to enable encapsulation of related code, without revealing implementation details to the rest of the module. It should be compatible with using extensions to build up types, and it should not encourage overly-long files.

The natural definition, therefore, is that ‘private’ should mean “visible in a small group of files which belong together as a unit”. Of course Swift does not yet have submodules, and is not likely to gain them this year. However, if we say that each file is implicitly its own submodule unless otherwise specified, then the model works. In that view, ‘private’ will mean “visible in this submodule”, and for the time being that is synonymous with “visible in this file”.

Although this does not immediately enable lengthy files to be separated along natural divisions, it does lay the groundwork to allow doing so in the future when submodules arrive.

Motivation – Summary

By looking at access control in its entirety, we can adopt a system that empowers both library authors and client programmers to organize their code in a principled way, and to expose the interfaces they want in the places they need. The complexity of the Swift 3 visibility story, which many people now regret creating, will be replaced by a far simpler model which in several respects is even more powerful.

Notably, being able to parameterize ‘final’ lets classes be closed not just externally, but also in the rest of the module outside the ‘private’ scope if desired. Furthermore, defining ‘private’ as being scoped to a group of related files means that, as soon as we get the ability to create such groups, it will no longer be necessary to write large files just to keep implementation details hidden.

Recommendations

To recap, the ideas presented here focus on simplifying access control while still supporting important use cases such as closed class hierarchies. The indicated design uses just three familiar access keywords:

‘private’, to restrict visibility to a group of files, or just one file until we get that capability.

‘internal’, which is the default and does not have to be written, for module-wide visibility.

‘public’, to make visible outside the module, including the ability to subclass.

Additionally, the design allows ‘final’ to take any one of those visibility levels as a parameter, to indicate that the type should be treated as ‘final’ at and above the specified scope. Thus ‘final(public)’ prevents subclassing outside the module, while ‘final(internal)’ prevents it outside the ‘private’ scope. For consistency, ‘final(private)’ is also permitted, although it means the same thing as ‘final’ by itself.

Conclusion

The Swift 3 access situation is harmful—as evidenced by the myriad calls to fix it—not just because of its excessive complexity, but also because it prioritizes convenience for library authors over utility for their clients, and because it has no natural way to accommodate splitting large files into smaller ones while preserving encapsulation.

We have an opportunity now to correct a mistake we have made, and to set a precedent that we *will* correct our mistakes, rather than continue down an undesirable path simply because it seemed like a good idea at the time. When real-world experience demonstrates that a change has taken us in the wrong direction, we can and should update our decisions based on that new experience.

Therefore, in the situation at hand, we should reconsider our access modifier story and choose a model which is both simple and powerful. I have presented here my best efforts at describing such a system, and I offer it as one possible way to move forward.

– Nevin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Kevin Nattinger) #7


Additionally, the design allows ‘final’ to take any one of those visibility levels as a parameter, to indicate that the type should be treated as ‘final’ at and above the specified scope. Thus ‘final(public)’ prevents subclassing outside the module, while ‘final(internal)’ prevents it outside the ‘private’ scope. For consistency, ‘final(private)’ is also permitted, although it means the same thing as ‘final’ by itself.

I feel final(public) could be confusing given how we currently have private(get/set). What about public(final)? That’s at least consistent with current access syntax.


(Christopher Kornher) #8

This is a very elegant improvement.

I think of “fileprivate” as “C legacy”. A submodule system seems to be a much better way to implement grouping a set of tightly coupled code elements. After all, there is no reasonable upper-limit on how large such a set of components might become and it would be unfortunate to force developers and compilers to have to parse very large files just because that is the only way to scope such elements. Getting rid of it ASAP seems like the right way to go.

···

On Feb 23, 2017, at 2:56 PM, Nevin Brackett-Rozinsky via swift-evolution <swift-evolution@swift.org> wrote:

Introduction

There has been a deluge of discussion about access levels lately, all attempting to simplify the situation. Shortly after Swift 3 was released, many people realized that the new access modifier story was far more complex than the old one, and expressed the belief that the changes may have been a mistake.

In the months that followed, more and more people came to share the same view, and stage 2 of Swift 4 has seen a profusion of proposals addressing the issue. These proposals are generally small and focus on changing just one aspect of access control. However, given the situation we are in now, it is important to look at the problem in its entirety and arrive at a cohesive solution.

Background

During the Swift 3 timeframe there were lengthy debates about access control. The end results were to adopt SE-0025 <https://github.com/apple/swift-evolution/blob/master/proposals/0025-scoped-access-level.md>, which introduced the ‘fileprivate’ keyword and made ‘private’ scope-based, and SE-0117 <https://github.com/apple/swift-evolution/blob/master/proposals/0117-non-public-subclassable-by-default.md>, which made ‘public’ classes closed by default and introduced the ‘open’ keyword. At the time, there was broad agreement (and some dissent) that these were the right changes to make.

That belief, as well as the numerous arguments which led to it, were well-informed and thoughtfully considered. However, due to the inevitable linear nature of time, they were not based on first-hand experience with the new changes. Upon the release of Swift 3, we all gained that first-hand experience, and it quickly became apparent to many people that the new access control system was needlessly complicated, and not at all the improvement it had been heralded as.

Rather than make it easy to encapsulate implementation details of related types across multiple files, we had instead doubled down on requiring that many things go in a single file or else reveal their secrets to the entire module. Even worse, the new scope-based ‘private’ discouraged the preferred style of using extensions to build up a type. To cap it off, we went from needing to know two access modifier keywords (‘public’ and ‘private’) to needing to know four of them (‘private’, ‘fileprivate’, ‘public’, and ‘open’) without even providing a way to share details across a small number of related files.

Motivation – Overview

Many different ideas for access control have been expressed on the Swift Evolution mailing list. Some people want ‘protected’ or ‘friend’ or ‘extensible’ or various other combinations of type-based visibility. Other people (*cough* Slava) see no need for any levels finer than ‘internal’ at all. The common points of agreement, though, are that ‘fileprivate’ is an awkward spelling, and that the whole system is too complicated.

It is important that we as the Swift community are able to recognize our mistakes, and even more important that we fix them. We originally thought that the Swift 3 access control changes would be beneficial. However, experience with them in practice has shown that not to be the case. Instead, the language became more complex, and that has real costs. It is time for a simplification.

The prevailing view from recent discussions is that there should be just one access level more fine-grained than ‘internal’, and it should be spelled ‘private’. Let us leave aside for the moment what its exact meaning should be, and consider the other end of the scale.

Motivation – Rethinking ‘public’

Prior to Swift 3 we had just one access level more broad than ‘internal’, and for simplicity of the model it would be desirable to achieve that again. However, SE-0117 raised the point that certain library designs require a class to have subclasses within the defining module, but not outside. In other words, client code should not be able to create subclasses, even though they exist in the library. Let us be clear: this is a niche use-case, but it is important.

The most common situations are that a class either should not be subclassable at all—in which case it is ‘final’—or that it should be subclassable anywhere including client code. In order for a library to need a publicly closed class, it must first of all be using classes rather than a protocol with conforming structs, it must have a hierarchy with a parent class that is exposed outside the module, it must have subclasses of that parent class within the module, and it must also require that no external subclasses can exist. Putting all those criteria together, we see that closed classes are a rare thing to use. Nonetheless, they are valuable and can enable certain compiler optimizations, so we should support them.

Currently, the spelling for a closed class is ‘public’. This makes it very easy for library authors to create them. However, since they are a niche feature and most of the time ‘final’ is a better choice, we do not need to dedicate the entire ‘public’ keyword to them.

Moreover, object-oriented programming is just as much a first-class citizen in Swift as protocol-oriented programming is, so we should treat it as such. Classes are inherently inheritable: when one writes “class Foo {}”, then Foo has a default visibility of ‘internal’, and by default it can have subclasses. That is a straightforward model, and it is easy to work with.

If subclasses are to be disallowed, then Foo should be marked ‘final’; if Foo is exported to clients then it should be marked ‘public’; and if both are true then Foo should be ‘public final’. This covers all the common cases, and leaves only the narrow corner of closed classes to consider. Per the motivation of SE-0117, that case is worth handling. Per our collective experience with Swift 3, however, it is not worth the added complexity of its own access modifier keyword. We need a better way to spell it.

One of the reasons ‘public’ was previously chosen for closed classes is to provide a “soft default” for library authors, so they can prevent subclassing until they decide later whether to allow it in a future release. This is a misguided decision, as it prioritizes the convenience of library authors over the productivity of application developers. Library authors have a responsibility to decide what interfaces they present, and we should not encourage them to release libraries without making those decisions.

Moreover, we need to trust client programmers to make reasonable choices. If a library mistakenly allows subclassing when it shouldn’t, all a client has to do to work with it correctly is *not make subclasses*. The library is still usable. Conversely, if a library mistakenly prohibits subclassing, then there are things a client *should* be able to do but cannot. The harm to the users of a library is greater in this last case, because the ability to use the library is compromised, and that diminishes their productivity.

We should not make “soft defaults” that tend to negatively impact the clients of a library for the dubious benefit of enabling its author to procrastinate on a basic design decision. If someone truly wants to publish a library with a closed class, then we should support that. But it should be an intentional decision, not a default.

Motivation – Rethinking ‘final’

The question then comes to spelling. It is evident that preventing subclasses is closely related to being ‘final’. One possibility, then, is to allow the ‘final’ keyword to take a parameter. The parameter would be an access level, to indicate that the type acts like it is final when accessed from at or above that level.

In particular, ‘final(public)’ would mean “this class cannot be subclassed from outside the module”, or in other words “this class appears final publicly, although it is nonfinal internally”. This approach is more powerful than a ‘closed’ keyword because it also allows ‘final(internal)’, meaning “this class appears final to the rest of the module, although it can be subclassed privately”.

Motivation – Rethinking ‘private’

Now let us return to ‘private’, which as discussed earlier should be the only modifier that is tighter than ‘internal’. The purpose of ‘private’ is to enable encapsulation of related code, without revealing implementation details to the rest of the module. It should be compatible with using extensions to build up types, and it should not encourage overly-long files.

The natural definition, therefore, is that ‘private’ should mean “visible in a small group of files which belong together as a unit”. Of course Swift does not yet have submodules, and is not likely to gain them this year. However, if we say that each file is implicitly its own submodule unless otherwise specified, then the model works. In that view, ‘private’ will mean “visible in this submodule”, and for the time being that is synonymous with “visible in this file”.

Although this does not immediately enable lengthy files to be separated along natural divisions, it does lay the groundwork to allow doing so in the future when submodules arrive.

Motivation – Summary

By looking at access control in its entirety, we can adopt a system that empowers both library authors and client programmers to organize their code in a principled way, and to expose the interfaces they want in the places they need. The complexity of the Swift 3 visibility story, which many people now regret creating, will be replaced by a far simpler model which in several respects is even more powerful.

Notably, being able to parameterize ‘final’ lets classes be closed not just externally, but also in the rest of the module outside the ‘private’ scope if desired. Furthermore, defining ‘private’ as being scoped to a group of related files means that, as soon as we get the ability to create such groups, it will no longer be necessary to write large files just to keep implementation details hidden.

Recommendations

To recap, the ideas presented here focus on simplifying access control while still supporting important use cases such as closed class hierarchies. The indicated design uses just three familiar access keywords:

‘private’, to restrict visibility to a group of files, or just one file until we get that capability.

‘internal’, which is the default and does not have to be written, for module-wide visibility.

‘public’, to make visible outside the module, including the ability to subclass.

Additionally, the design allows ‘final’ to take any one of those visibility levels as a parameter, to indicate that the type should be treated as ‘final’ at and above the specified scope. Thus ‘final(public)’ prevents subclassing outside the module, while ‘final(internal)’ prevents it outside the ‘private’ scope. For consistency, ‘final(private)’ is also permitted, although it means the same thing as ‘final’ by itself.

Conclusion

The Swift 3 access situation is harmful—as evidenced by the myriad calls to fix it—not just because of its excessive complexity, but also because it prioritizes convenience for library authors over utility for their clients, and because it has no natural way to accommodate splitting large files into smaller ones while preserving encapsulation.

We have an opportunity now to correct a mistake we have made, and to set a precedent that we *will* correct our mistakes, rather than continue down an undesirable path simply because it seemed like a good idea at the time. When real-world experience demonstrates that a change has taken us in the wrong direction, we can and should update our decisions based on that new experience.

Therefore, in the situation at hand, we should reconsider our access modifier story and choose a model which is both simple and powerful. I have presented here my best efforts at describing such a system, and I offer it as one possible way to move forward.

– Nevin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


#9

Thanks for the feedback David,

The whole design and motivation behind SE-0117 is that it is not rare to

want classes to be freely subclass-able internally while being closed to
subclassing externally. Do you have arguments to contradict the lengthy
points mentioned in the SE-0117 motivation section?

In fact I *agree* with the lengthy points mentioned in the SE-0117
motivation section!

That document does an excellent job laying out why it is important for
Swift to support closed classes, and I am wholly on board with that
rationale. It is a major reason I dedicated so much space to explaining
that we *should* continue to support closed classes. However, SE-0117 makes
no attempt to dispute the fact that closed classes are rarely needed. After
all, one can often use structs or final classes instead.

Only at the very end of its motivation section does SE-0117 contain a few
brief lines about “excess annotations”, and it is there I take a different
view. Since closed classes are quite infrequent, it is acceptable to place
a short annotation on the declaration in order to specify, “Yes, this
really is supposed to be a closed class.”

The loss is small because it just means writing ‘final(public)’ on an
uncommon sort of declaration, and the win is large because it dramatically
simplifies our access control system.

The reasons for the rarity above do not make much sense to me:

   - *"it must first of all be using classes rather than a protocol with
   conforming structs”: *we are discussing the merits of the different
   forms of subclassing prevention (public vs final), so I don’t see the
   reason for mentioning structs.

Any library which models its features using structs, does not need classes

at all, so it certainly does not need a closed class hierarchy. This is
part of the reason why closed classes are rare in Swift.

We should not make “soft defaults” that tend to negatively impact the

clients of a library for the dubious benefit of enabling its author to
procrastinate on a basic design decision. If someone truly wants to publish
a library with a closed class, then we should support that. But it should
be an intentional decision, not a default.

That’s a bit harsh. For me, it has nothing to do with procrastination, but
with being *safe-by-default*, which is an important Swift goal. You are
arguing for *useful-by-default*, which is only a different priority.

This is not an issue of safety. Classes are safe in Swift regardless of
their visibility or finality.

I am putting forward the idea that Swift, being an opinionated language,
should take the stance, “Classes can be subclassed, unless they are
‘final’.” Then if a library author wants to restrict subclassing, they can
use ‘final’ or ‘final(public)’ to do so.

This model allows progressive disclosure:

   1. Classes are subclassable
   2. ‘final’ prevents subclassing
   3. ‘final(public)’ prevents external subclassing

I’m a proponent for going back to a file-based private but your solution

has two big disadvantages for me:

   - it makes impossible to have a file-based scope in submodules

This is intentional. If you want certain files to share private details,

then put them in the same submodule. Otherwise, don’t.

The benefits are that it encourages people to keep submodules small and
focused, while supporting the pattern of building up types using
extensions, and without requiring lengthy files.

   - *private* would have different semantics depending if its in a
   submodule or not.

I think my description may have been unclear. In my model there is only

one semantic meaning for ‘private’, and that is “visible in the submodule”.
Every file would naturally be its own submodule unless otherwise specified,
so the file-scope aspect is really just a consequence of having a submodule
with only one file in it.

   - it would make copy-pasting/moving files from a submodule to a
   top-module potentially breaking:

Anytime code is moved out of a given visibility region, it will obviously

not be able to access things restricted to its previous location. This is
true of every access control system ever devised, and if I may be blunt it
is kind of the whole point.

– Nevin


(Matthew Johnson) #10

The prevailing view from recent discussions is that there should be just one access level more fine-grained than ‘internal’, and it should be spelled ‘private’. Let us leave aside for the moment what its exact meaning should be, and consider the other end of the scale.

I think the submodule discussion now goes against this being a prevailing view - submodules are directly tied to be an access control level *above* private/fileprivate.

My expectation is that with submodules, “fileprivate” would go away.

Moreover, object-oriented programming is just as much a first-class citizen in Swift as protocol-oriented programming is, so we should treat it as such. Classes are inherently inheritable: when one writes “class Foo {}”, then Foo has a default visibility of ‘internal’, and by default it can have subclasses. That is a straightforward model, and it is easy to work with.

There is nothing about OOP which requires subclass-based polymorphism. Rust and Go are two examples of modern languages that do not support subclassing at all.

In reality, there is less agreement on what “Object Oriented Programming” means to technologists than there is about the real meaning of “Moore’s Law"

One of the reasons ‘public’ was previously chosen for closed classes is to provide a “soft default” for library authors, so they can prevent subclassing until they decide later whether to allow it in a future release. This is a misguided decision, as it prioritizes the convenience of library authors over the productivity of application developers. Library authors have a responsibility to decide what interfaces they present, and we should not encourage them to release libraries without making those decisions.

I think SE-0117 and the discussion behind it is pretty clear about why this is the default. It is not about convenience, but safety.

Moreover, we need to trust client programmers to make reasonable choices. If a library mistakenly allows subclassing when it shouldn’t, all a client has to do to work with it correctly is *not make subclasses*. The library is still usable. Conversely, if a library mistakenly prohibits subclassing, then there are things a client *should* be able to do but cannot. The harm to the users of a library is greater in this last case, because the ability to use the library is compromised, and that diminishes their productivity.

If a client uses a library type which was never intended to be subclassed and subclasses it, the only options to take the library forward are to:
1. break the client
2. make that usage part of your supported API, and work around any issues that causes

I sympathize with app developers under deadlines. I wouldn’t mind Swift having an ‘escape hatch’ to let an app developer take responsibility and to then abuse a library’s access levels (if that were possible without restricting optimizations in the Swift compiler). But closed-by-default is safer, and lets poorly specified libraries evolve to be higher quality libraries without breaking existing usage that they never intended to support.

We should not make “soft defaults” that tend to negatively impact the clients of a library for the dubious benefit of enabling its author to procrastinate on a basic design decision. If someone truly wants to publish a library with a closed class, then we should support that. But it should be an intentional decision, not a default.

If you were proposing the default for public classes be final rather than closed, I would be on-board with that line of thinking.

If you proposed that final was redundant and should go away, leaving just public and public open, I’d find that defensible (I think even with closed as a default, it is useful to indicate within am module that a class is subclassed or not)

If you proposed that library authors should be forced into a decision, and should get a warning with a default of final until they label public classes as open or final or sealsed, I’d also be on-board with that option.

However, “open by default” is quite different than “final by default”. The default for structs and enums are final by default. Protocols are implementable, but they by definition should have defined semantics for doing so. Open-by-default subclasses have the potential to be modified in ways you were never expected, then handed back to you. Exposing a class the same way you would expose a struct, enum, or protocol and having it directly impact your resiliency because you forgot to also add “final” is just not as safe. I think the SE-0117 discussion covered this *very* well.

Motivation – Rethinking ‘private’

Now let us return to ‘private’, which as discussed earlier should be the only modifier that is tighter than ‘internal’. The purpose of ‘private’ is to enable encapsulation of related code, without revealing implementation details to the rest of the module. It should be compatible with using extensions to build up types, and it should not encourage overly-long files.

As said before, I think this is an incorrect conclusion.

My personal thinking has been swayed over the last few months, and I’ve gone from vehemently opposing private to seeing merit in it. I believe my opposition was not against scoped private, but that it both kept fileprivate as a required access level and a redundant one, because it was not part of a larger change.

Swift need a modifier between private and internal, and “fileprivate” is neither the feature we want, nor the spelling. Fileprivate should have been replaced with a more appropriate access level between private and internal, to represent ‘friend” relationships. I think we may be now looking at doing that, with submodules.

I would love to hear your feedback on my submodule proposal David! I believe it addresses the use cases you're describing very well.

···

Sent from my iPad

On Feb 25, 2017, at 12:12 AM, David Waite via swift-evolution <swift-evolution@swift.org> wrote:

On Feb 23, 2017, at 2:56 PM, Nevin Brackett-Rozinsky via swift-evolution <swift-evolution@swift.org> wrote:

My tact is that keywords should first and foremost document developer intent.

If I were to say which members of a type *should* be private, it would be the members that are not properly designed to uphold class invariance. Allowing access to write a negative integer to a property that should only ever be positive would be a good example. Calling a member without holding a lock or first dispatching onto the right GCD queue would be another.

In that context, a private which restricts even same-file, build-up extensions could be a positive language feature, because the extensions are being built using the safer interface. This is a trade-off, as some interfaces (NSCoding is a prime example) may require access to members otherwise made private.

The natural definition, therefore, is that ‘private’ should mean “visible in a small group of files which belong together as a unit”. Of course Swift does not yet have submodules, and is not likely to gain them this year. However, if we say that each file is implicitly its own submodule unless otherwise specified, then the model works. In that view, ‘private’ will mean “visible in this submodule”, and for the time being that is synonymous with “visible in this file”.

I think internal is a more appropriate visibility within a submodule.

-DW

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Dimitri Racordon) #11

I have to admit I don’t get why there’s so much push back about the whole public/open debate. I understand that completely removing the possibility to declare internally subclassable but publicly closed classes (like it did) might be harmful, but I’m still convinced that this use-case isn’t worth the complexity introduced by an additional access level modifier. This proposal offers a great compromise in that regards.

I think everybody agrees (at the very least to some degree) that the current access level system is too complicated. I also think that everybody agrees that the main reason behind that is the current set of keywords, or more specifically their spelling and/or semantics. This proposal does a great job at reducing this set to the ones that are in my opinion the most easily understandable: public, private and internal. The two former are generally well understood, and easy to explain to novices. The latter is in practice never written, but is also quite easy to teach later, for the sake of completeness. On the contrary, open feels like an odd exception, only applicable to a subset of the language entities, for a use-case that *does* happen but *isn’t* the norm. Why are we fighting so hard to keep it.

SE-0117 states that designing classes for subclassing requires more care, but that’s also true *within* a library. Just because a library writer is able to put public in front of his/her class doesn’t mean the complexity to design it properly for subclassing goes away. If we really wanted to label Swift's sublcassing system “safe by default”, we’d have to make classes *final* by default (then we’d drop final and change open so it'd remove the default restriction). Instead, we have a solution that enforces a concept externally, but doesn’t do the same internally. We should choose a direction and commit to it. SE-0117 didn’t do that fully, this proposal does (but in another direction).

Is one direction better that the other? Now *that* is debatable. But people generally understand classes to be subclassable, and it is also generally understood that this ability can be denied when classes are marked final. That’s the case in C++, Java, C# or Scala to cite few other languages. I couldn’t agree more that we shouldn’t adopt a design just because language X also does, but here we’re talking about the whole OO paradigm and its general understanding. The current behaviour (for publicly exposed classes) goes against this general understanding, which I think is harmful. I personally have no problem with putting the burden on the application writer, first because I find it is unrealistic to think that nothing can ever be used improperly when importing a framework, second because I think that experimented users should be able to find workarounds to use a library that may not have foreseen all possible corner cases. People might very well disagree on that, and I’d accept their position **if** they also stand on the “final by default” side. In either case, one keyword has to be eliminated (open or final).


(Matthew Johnson) #12

+1000

This is the best of the proposals I have seen. I think it works by itself as a complete proposal, but since we are talking "comprehensive rethink", there is one use case which most of the proposals seem to miss.

That is, what if I want to make a class which is both Public and Subclassable, but I want to keep some of the implementation details private to the class, while still allowing subclasses and extensions (across the module boundary) to have access to those details. A concrete example of this is UIGestureRecognizer, which is intended to be subclassed outside of its framework, but hides potentially catastrophic things (like being able to set the state) from callers. This isn’t currently possible in Swift AFAICT without importing from ObjC.

My solution to this has been to allow something to be marked ‘public hidden’ or ‘hidden', which means that it is public/internal, but has it’s visibility limited only to files which opt-in to seeing it with ‘import hidden filename’ (instead of just ‘import filename’). I think this is the simplest way to provide this feature, which is sometimes very necessary. It also captures authorial intent very nicely.

My submodule proposal is able to support this use case. You would just put the symbols intended for subclasses in a separate submodule. These submodules could be exposed to users with any name you desire, regardless of the internal structure of your module. You could even arrange for users to get all of the non-subclass-only symbols automatically when they import your module but require explicit import of each individual subclass-only submodule in places where it is needed.

···

Sent from my iPad

On Feb 25, 2017, at 2:54 PM, Jonathan Hull via swift-evolution <swift-evolution@swift.org> wrote:

Thanks,
Jon

On Feb 23, 2017, at 1:56 PM, Nevin Brackett-Rozinsky via swift-evolution <swift-evolution@swift.org> wrote:

Introduction

There has been a deluge of discussion about access levels lately, all attempting to simplify the situation. Shortly after Swift 3 was released, many people realized that the new access modifier story was far more complex than the old one, and expressed the belief that the changes may have been a mistake.

In the months that followed, more and more people came to share the same view, and stage 2 of Swift 4 has seen a profusion of proposals addressing the issue. These proposals are generally small and focus on changing just one aspect of access control. However, given the situation we are in now, it is important to look at the problem in its entirety and arrive at a cohesive solution.

Background

During the Swift 3 timeframe there were lengthy debates about access control. The end results were to adopt SE-0025, which introduced the ‘fileprivate’ keyword and made ‘private’ scope-based, and SE-0117, which made ‘public’ classes closed by default and introduced the ‘open’ keyword. At the time, there was broad agreement (and some dissent) that these were the right changes to make.

That belief, as well as the numerous arguments which led to it, were well-informed and thoughtfully considered. However, due to the inevitable linear nature of time, they were not based on first-hand experience with the new changes. Upon the release of Swift 3, we all gained that first-hand experience, and it quickly became apparent to many people that the new access control system was needlessly complicated, and not at all the improvement it had been heralded as.

Rather than make it easy to encapsulate implementation details of related types across multiple files, we had instead doubled down on requiring that many things go in a single file or else reveal their secrets to the entire module. Even worse, the new scope-based ‘private’ discouraged the preferred style of using extensions to build up a type. To cap it off, we went from needing to know two access modifier keywords (‘public’ and ‘private’) to needing to know four of them (‘private’, ‘fileprivate’, ‘public’, and ‘open’) without even providing a way to share details across a small number of related files.

Motivation – Overview

Many different ideas for access control have been expressed on the Swift Evolution mailing list. Some people want ‘protected’ or ‘friend’ or ‘extensible’ or various other combinations of type-based visibility. Other people (*cough* Slava) see no need for any levels finer than ‘internal’ at all. The common points of agreement, though, are that ‘fileprivate’ is an awkward spelling, and that the whole system is too complicated.

It is important that we as the Swift community are able to recognize our mistakes, and even more important that we fix them. We originally thought that the Swift 3 access control changes would be beneficial. However, experience with them in practice has shown that not to be the case. Instead, the language became more complex, and that has real costs. It is time for a simplification.

The prevailing view from recent discussions is that there should be just one access level more fine-grained than ‘internal’, and it should be spelled ‘private’. Let us leave aside for the moment what its exact meaning should be, and consider the other end of the scale.

Motivation – Rethinking ‘public’

Prior to Swift 3 we had just one access level more broad than ‘internal’, and for simplicity of the model it would be desirable to achieve that again. However, SE-0117 raised the point that certain library designs require a class to have subclasses within the defining module, but not outside. In other words, client code should not be able to create subclasses, even though they exist in the library. Let us be clear: this is a niche use-case, but it is important.

The most common situations are that a class either should not be subclassable at all—in which case it is ‘final’—or that it should be subclassable anywhere including client code. In order for a library to need a publicly closed class, it must first of all be using classes rather than a protocol with conforming structs, it must have a hierarchy with a parent class that is exposed outside the module, it must have subclasses of that parent class within the module, and it must also require that no external subclasses can exist. Putting all those criteria together, we see that closed classes are a rare thing to use. Nonetheless, they are valuable and can enable certain compiler optimizations, so we should support them.

Currently, the spelling for a closed class is ‘public’. This makes it very easy for library authors to create them. However, since they are a niche feature and most of the time ‘final’ is a better choice, we do not need to dedicate the entire ‘public’ keyword to them.

Moreover, object-oriented programming is just as much a first-class citizen in Swift as protocol-oriented programming is, so we should treat it as such. Classes are inherently inheritable: when one writes “class Foo {}”, then Foo has a default visibility of ‘internal’, and by default it can have subclasses. That is a straightforward model, and it is easy to work with.

If subclasses are to be disallowed, then Foo should be marked ‘final’; if Foo is exported to clients then it should be marked ‘public’; and if both are true then Foo should be ‘public final’. This covers all the common cases, and leaves only the narrow corner of closed classes to consider. Per the motivation of SE-0117, that case is worth handling. Per our collective experience with Swift 3, however, it is not worth the added complexity of its own access modifier keyword. We need a better way to spell it.

One of the reasons ‘public’ was previously chosen for closed classes is to provide a “soft default” for library authors, so they can prevent subclassing until they decide later whether to allow it in a future release. This is a misguided decision, as it prioritizes the convenience of library authors over the productivity of application developers. Library authors have a responsibility to decide what interfaces they present, and we should not encourage them to release libraries without making those decisions.

Moreover, we need to trust client programmers to make reasonable choices. If a library mistakenly allows subclassing when it shouldn’t, all a client has to do to work with it correctly is *not make subclasses*. The library is still usable. Conversely, if a library mistakenly prohibits subclassing, then there are things a client *should* be able to do but cannot. The harm to the users of a library is greater in this last case, because the ability to use the library is compromised, and that diminishes their productivity.

We should not make “soft defaults” that tend to negatively impact the clients of a library for the dubious benefit of enabling its author to procrastinate on a basic design decision. If someone truly wants to publish a library with a closed class, then we should support that. But it should be an intentional decision, not a default.

Motivation – Rethinking ‘final’

The question then comes to spelling. It is evident that preventing subclasses is closely related to being ‘final’. One possibility, then, is to allow the ‘final’ keyword to take a parameter. The parameter would be an access level, to indicate that the type acts like it is final when accessed from at or above that level.

In particular, ‘final(public)’ would mean “this class cannot be subclassed from outside the module”, or in other words “this class appears final publicly, although it is nonfinal internally”. This approach is more powerful than a ‘closed’ keyword because it also allows ‘final(internal)’, meaning “this class appears final to the rest of the module, although it can be subclassed privately”.

Motivation – Rethinking ‘private’

Now let us return to ‘private’, which as discussed earlier should be the only modifier that is tighter than ‘internal’. The purpose of ‘private’ is to enable encapsulation of related code, without revealing implementation details to the rest of the module. It should be compatible with using extensions to build up types, and it should not encourage overly-long files.

The natural definition, therefore, is that ‘private’ should mean “visible in a small group of files which belong together as a unit”. Of course Swift does not yet have submodules, and is not likely to gain them this year. However, if we say that each file is implicitly its own submodule unless otherwise specified, then the model works. In that view, ‘private’ will mean “visible in this submodule”, and for the time being that is synonymous with “visible in this file”.

Although this does not immediately enable lengthy files to be separated along natural divisions, it does lay the groundwork to allow doing so in the future when submodules arrive.

Motivation – Summary

By looking at access control in its entirety, we can adopt a system that empowers both library authors and client programmers to organize their code in a principled way, and to expose the interfaces they want in the places they need. The complexity of the Swift 3 visibility story, which many people now regret creating, will be replaced by a far simpler model which in several respects is even more powerful.

Notably, being able to parameterize ‘final’ lets classes be closed not just externally, but also in the rest of the module outside the ‘private’ scope if desired. Furthermore, defining ‘private’ as being scoped to a group of related files means that, as soon as we get the ability to create such groups, it will no longer be necessary to write large files just to keep implementation details hidden.

Recommendations

To recap, the ideas presented here focus on simplifying access control while still supporting important use cases such as closed class hierarchies. The indicated design uses just three familiar access keywords:

‘private’, to restrict visibility to a group of files, or just one file until we get that capability.

‘internal’, which is the default and does not have to be written, for module-wide visibility.

‘public’, to make visible outside the module, including the ability to subclass.

Additionally, the design allows ‘final’ to take any one of those visibility levels as a parameter, to indicate that the type should be treated as ‘final’ at and above the specified scope. Thus ‘final(public)’ prevents subclassing outside the module, while ‘final(internal)’ prevents it outside the ‘private’ scope. For consistency, ‘final(private)’ is also permitted, although it means the same thing as ‘final’ by itself.

Conclusion

The Swift 3 access situation is harmful—as evidenced by the myriad calls to fix it—not just because of its excessive complexity, but also because it prioritizes convenience for library authors over utility for their clients, and because it has no natural way to accommodate splitting large files into smaller ones while preserving encapsulation.

We have an opportunity now to correct a mistake we have made, and to set a precedent that we *will* correct our mistakes, rather than continue down an undesirable path simply because it seemed like a good idea at the time. When real-world experience demonstrates that a change has taken us in the wrong direction, we can and should update our decisions based on that new experience.

Therefore, in the situation at hand, we should reconsider our access modifier story and choose a model which is both simple and powerful. I have presented here my best efforts at describing such a system, and I offer it as one possible way to move forward.

– Nevin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Matthew Johnson) #13


Additionally, the design allows ‘final’ to take any one of those visibility levels as a parameter, to indicate that the type should be treated as ‘final’ at and above the specified scope. Thus ‘final(public)’ prevents subclassing outside the module, while ‘final(internal)’ prevents it outside the ‘private’ scope. For consistency, ‘final(private)’ is also permitted, although it means the same thing as ‘final’ by itself.

I feel final(public) could be confusing given how we currently have private(get/set). What about public(final)? That’s at least consistent with current access syntax.

The syntax that most closely aligns with private(set) is internal(inherit). The parameter expresses a capability not a limitation and it is used with an access modifier that specifies the *maximum* access level that is allowed to have this capability. `final` is also misleading in this context because it implies to most people that there are *no* subclasses (inside or outside the module).

The only way to express `final` in Swift’s access control system would to have a `never` access modifier allowing you to say `never(inherit)`. I can’ think of any other uses for an access level like this. `final` is much more direct and is the term of art. I don’t see a reason to change the spelling of `final` and parameterizing it doesn’t make any sense - if something is `final` it is `final` in *all* scopes.

···

On Feb 25, 2017, at 3:27 PM, Kevin Nattinger via swift-evolution <swift-evolution@swift.org> wrote:

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Jon Hull) #14

+1000

This is the best of the proposals I have seen. I think it works by itself as a complete proposal, but since we are talking "comprehensive rethink", there is one use case which most of the proposals seem to miss.

That is, what if I want to make a class which is both Public and Subclassable, but I want to keep some of the implementation details private to the class, while still allowing subclasses and extensions (across the module boundary) to have access to those details. A concrete example of this is UIGestureRecognizer, which is intended to be subclassed outside of its framework, but hides potentially catastrophic things (like being able to set the state) from callers. This isn’t currently possible in Swift AFAICT without importing from ObjC.

My solution to this has been to allow something to be marked ‘public hidden’ or ‘hidden', which means that it is public/internal, but has it’s visibility limited only to files which opt-in to seeing it with ‘import hidden filename’ (instead of just ‘import filename’). I think this is the simplest way to provide this feature, which is sometimes very necessary. It also captures authorial intent very nicely.

My submodule proposal is able to support this use case. You would just put the symbols intended for subclasses in a separate submodule. These submodules could be exposed to users with any name you desire, regardless of the internal structure of your module. You could even arrange for users to get all of the non-subclass-only symbols automatically when they import your module but require explicit import of each individual subclass-only submodule in places where it is needed.

I agree that this could also be solved with nested submodules, but the current proposals all seem to add a lot of complexity. This complexity does give you much finer grain control over visibility, but what I like about Nevin’s proposal is that it is ‘just enough’ control with minimal complexity. What I like about ‘hidden’ vs export of nested submodules is that you can freely intermix those declarations in your code (like you can with private now). There is also the question of how to separate a single type into multiple submodules, when we can’t declare storage in an extension.

Thanks,
Jon

···

On Feb 25, 2017, at 1:19 PM, Matthew Johnson <matthew@anandabits.com> wrote:

On Feb 25, 2017, at 2:54 PM, Jonathan Hull via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Thanks,
Jon

On Feb 23, 2017, at 1:56 PM, Nevin Brackett-Rozinsky via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Introduction

There has been a deluge of discussion about access levels lately, all attempting to simplify the situation. Shortly after Swift 3 was released, many people realized that the new access modifier story was far more complex than the old one, and expressed the belief that the changes may have been a mistake.

In the months that followed, more and more people came to share the same view, and stage 2 of Swift 4 has seen a profusion of proposals addressing the issue. These proposals are generally small and focus on changing just one aspect of access control. However, given the situation we are in now, it is important to look at the problem in its entirety and arrive at a cohesive solution.

Background

During the Swift 3 timeframe there were lengthy debates about access control. The end results were to adopt SE-0025 <https://github.com/apple/swift-evolution/blob/master/proposals/0025-scoped-access-level.md>, which introduced the ‘fileprivate’ keyword and made ‘private’ scope-based, and SE-0117 <https://github.com/apple/swift-evolution/blob/master/proposals/0117-non-public-subclassable-by-default.md>, which made ‘public’ classes closed by default and introduced the ‘open’ keyword. At the time, there was broad agreement (and some dissent) that these were the right changes to make.

That belief, as well as the numerous arguments which led to it, were well-informed and thoughtfully considered. However, due to the inevitable linear nature of time, they were not based on first-hand experience with the new changes. Upon the release of Swift 3, we all gained that first-hand experience, and it quickly became apparent to many people that the new access control system was needlessly complicated, and not at all the improvement it had been heralded as.

Rather than make it easy to encapsulate implementation details of related types across multiple files, we had instead doubled down on requiring that many things go in a single file or else reveal their secrets to the entire module. Even worse, the new scope-based ‘private’ discouraged the preferred style of using extensions to build up a type. To cap it off, we went from needing to know two access modifier keywords (‘public’ and ‘private’) to needing to know four of them (‘private’, ‘fileprivate’, ‘public’, and ‘open’) without even providing a way to share details across a small number of related files.

Motivation – Overview

Many different ideas for access control have been expressed on the Swift Evolution mailing list. Some people want ‘protected’ or ‘friend’ or ‘extensible’ or various other combinations of type-based visibility. Other people (*cough* Slava) see no need for any levels finer than ‘internal’ at all. The common points of agreement, though, are that ‘fileprivate’ is an awkward spelling, and that the whole system is too complicated.

It is important that we as the Swift community are able to recognize our mistakes, and even more important that we fix them. We originally thought that the Swift 3 access control changes would be beneficial. However, experience with them in practice has shown that not to be the case. Instead, the language became more complex, and that has real costs. It is time for a simplification.

The prevailing view from recent discussions is that there should be just one access level more fine-grained than ‘internal’, and it should be spelled ‘private’. Let us leave aside for the moment what its exact meaning should be, and consider the other end of the scale.

Motivation – Rethinking ‘public’

Prior to Swift 3 we had just one access level more broad than ‘internal’, and for simplicity of the model it would be desirable to achieve that again. However, SE-0117 raised the point that certain library designs require a class to have subclasses within the defining module, but not outside. In other words, client code should not be able to create subclasses, even though they exist in the library. Let us be clear: this is a niche use-case, but it is important.

The most common situations are that a class either should not be subclassable at all—in which case it is ‘final’—or that it should be subclassable anywhere including client code. In order for a library to need a publicly closed class, it must first of all be using classes rather than a protocol with conforming structs, it must have a hierarchy with a parent class that is exposed outside the module, it must have subclasses of that parent class within the module, and it must also require that no external subclasses can exist. Putting all those criteria together, we see that closed classes are a rare thing to use. Nonetheless, they are valuable and can enable certain compiler optimizations, so we should support them.

Currently, the spelling for a closed class is ‘public’. This makes it very easy for library authors to create them. However, since they are a niche feature and most of the time ‘final’ is a better choice, we do not need to dedicate the entire ‘public’ keyword to them.

Moreover, object-oriented programming is just as much a first-class citizen in Swift as protocol-oriented programming is, so we should treat it as such. Classes are inherently inheritable: when one writes “class Foo {}”, then Foo has a default visibility of ‘internal’, and by default it can have subclasses. That is a straightforward model, and it is easy to work with.

If subclasses are to be disallowed, then Foo should be marked ‘final’; if Foo is exported to clients then it should be marked ‘public’; and if both are true then Foo should be ‘public final’. This covers all the common cases, and leaves only the narrow corner of closed classes to consider. Per the motivation of SE-0117, that case is worth handling. Per our collective experience with Swift 3, however, it is not worth the added complexity of its own access modifier keyword. We need a better way to spell it.

One of the reasons ‘public’ was previously chosen for closed classes is to provide a “soft default” for library authors, so they can prevent subclassing until they decide later whether to allow it in a future release. This is a misguided decision, as it prioritizes the convenience of library authors over the productivity of application developers. Library authors have a responsibility to decide what interfaces they present, and we should not encourage them to release libraries without making those decisions.

Moreover, we need to trust client programmers to make reasonable choices. If a library mistakenly allows subclassing when it shouldn’t, all a client has to do to work with it correctly is *not make subclasses*. The library is still usable. Conversely, if a library mistakenly prohibits subclassing, then there are things a client *should* be able to do but cannot. The harm to the users of a library is greater in this last case, because the ability to use the library is compromised, and that diminishes their productivity.

We should not make “soft defaults” that tend to negatively impact the clients of a library for the dubious benefit of enabling its author to procrastinate on a basic design decision. If someone truly wants to publish a library with a closed class, then we should support that. But it should be an intentional decision, not a default.

Motivation – Rethinking ‘final’

The question then comes to spelling. It is evident that preventing subclasses is closely related to being ‘final’. One possibility, then, is to allow the ‘final’ keyword to take a parameter. The parameter would be an access level, to indicate that the type acts like it is final when accessed from at or above that level.

In particular, ‘final(public)’ would mean “this class cannot be subclassed from outside the module”, or in other words “this class appears final publicly, although it is nonfinal internally”. This approach is more powerful than a ‘closed’ keyword because it also allows ‘final(internal)’, meaning “this class appears final to the rest of the module, although it can be subclassed privately”.

Motivation – Rethinking ‘private’

Now let us return to ‘private’, which as discussed earlier should be the only modifier that is tighter than ‘internal’. The purpose of ‘private’ is to enable encapsulation of related code, without revealing implementation details to the rest of the module. It should be compatible with using extensions to build up types, and it should not encourage overly-long files.

The natural definition, therefore, is that ‘private’ should mean “visible in a small group of files which belong together as a unit”. Of course Swift does not yet have submodules, and is not likely to gain them this year. However, if we say that each file is implicitly its own submodule unless otherwise specified, then the model works. In that view, ‘private’ will mean “visible in this submodule”, and for the time being that is synonymous with “visible in this file”.

Although this does not immediately enable lengthy files to be separated along natural divisions, it does lay the groundwork to allow doing so in the future when submodules arrive.

Motivation – Summary

By looking at access control in its entirety, we can adopt a system that empowers both library authors and client programmers to organize their code in a principled way, and to expose the interfaces they want in the places they need. The complexity of the Swift 3 visibility story, which many people now regret creating, will be replaced by a far simpler model which in several respects is even more powerful.

Notably, being able to parameterize ‘final’ lets classes be closed not just externally, but also in the rest of the module outside the ‘private’ scope if desired. Furthermore, defining ‘private’ as being scoped to a group of related files means that, as soon as we get the ability to create such groups, it will no longer be necessary to write large files just to keep implementation details hidden.

Recommendations

To recap, the ideas presented here focus on simplifying access control while still supporting important use cases such as closed class hierarchies. The indicated design uses just three familiar access keywords:

‘private’, to restrict visibility to a group of files, or just one file until we get that capability.

‘internal’, which is the default and does not have to be written, for module-wide visibility.

‘public’, to make visible outside the module, including the ability to subclass.

Additionally, the design allows ‘final’ to take any one of those visibility levels as a parameter, to indicate that the type should be treated as ‘final’ at and above the specified scope. Thus ‘final(public)’ prevents subclassing outside the module, while ‘final(internal)’ prevents it outside the ‘private’ scope. For consistency, ‘final(private)’ is also permitted, although it means the same thing as ‘final’ by itself.

Conclusion

The Swift 3 access situation is harmful—as evidenced by the myriad calls to fix it—not just because of its excessive complexity, but also because it prioritizes convenience for library authors over utility for their clients, and because it has no natural way to accommodate splitting large files into smaller ones while preserving encapsulation.

We have an opportunity now to correct a mistake we have made, and to set a precedent that we *will* correct our mistakes, rather than continue down an undesirable path simply because it seemed like a good idea at the time. When real-world experience demonstrates that a change has taken us in the wrong direction, we can and should update our decisions based on that new experience.

Therefore, in the situation at hand, we should reconsider our access modifier story and choose a model which is both simple and powerful. I have presented here my best efforts at describing such a system, and I offer it as one possible way to move forward.

– Nevin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution


(Jon Hull) #15

hmm… ‘semifinal' for things that are closed outside the submodule?

Thanks,
Jon

···

On Feb 25, 2017, at 1:43 PM, Matthew Johnson via swift-evolution <swift-evolution@swift.org> wrote:

On Feb 25, 2017, at 3:27 PM, Kevin Nattinger via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:


Additionally, the design allows ‘final’ to take any one of those visibility levels as a parameter, to indicate that the type should be treated as ‘final’ at and above the specified scope. Thus ‘final(public)’ prevents subclassing outside the module, while ‘final(internal)’ prevents it outside the ‘private’ scope. For consistency, ‘final(private)’ is also permitted, although it means the same thing as ‘final’ by itself.

I feel final(public) could be confusing given how we currently have private(get/set). What about public(final)? That’s at least consistent with current access syntax.

The syntax that most closely aligns with private(set) is internal(inherit). The parameter expresses a capability not a limitation and it is used with an access modifier that specifies the *maximum* access level that is allowed to have this capability. `final` is also misleading in this context because it implies to most people that there are *no* subclasses (inside or outside the module).

The only way to express `final` in Swift’s access control system would to have a `never` access modifier allowing you to say `never(inherit)`. I can’ think of any other uses for an access level like this. `final` is much more direct and is the term of art. I don’t see a reason to change the spelling of `final` and parameterizing it doesn’t make any sense - if something is `final` it is `final` in *all* scopes.


(Matthew Johnson) #16

+1000

This is the best of the proposals I have seen. I think it works by itself as a complete proposal, but since we are talking "comprehensive rethink", there is one use case which most of the proposals seem to miss.

That is, what if I want to make a class which is both Public and Subclassable, but I want to keep some of the implementation details private to the class, while still allowing subclasses and extensions (across the module boundary) to have access to those details. A concrete example of this is UIGestureRecognizer, which is intended to be subclassed outside of its framework, but hides potentially catastrophic things (like being able to set the state) from callers. This isn’t currently possible in Swift AFAICT without importing from ObjC.

My solution to this has been to allow something to be marked ‘public hidden’ or ‘hidden', which means that it is public/internal, but has it’s visibility limited only to files which opt-in to seeing it with ‘import hidden filename’ (instead of just ‘import filename’). I think this is the simplest way to provide this feature, which is sometimes very necessary. It also captures authorial intent very nicely.

My submodule proposal is able to support this use case. You would just put the symbols intended for subclasses in a separate submodule. These submodules could be exposed to users with any name you desire, regardless of the internal structure of your module. You could even arrange for users to get all of the non-subclass-only symbols automatically when they import your module but require explicit import of each individual subclass-only submodule in places where it is needed.

I agree that this could also be solved with nested submodules

A minor note, but you would not need to nest them. You could do it either way.

, but the current proposals all seem to add a lot of complexity. This complexity does give you much finer grain control over visibility, but what I like about Nevin’s proposal is that it is ‘just enough’ control with minimal complexity.

Determining what “just enough” is is obviously a very subjective thing. :slight_smile: Everyone seems to have their own opinion about this (more than usual) which is going to make it one of the more complicated discussions we’ll have on the list.

I worked very hard on my proposal to try to design a system that stays out of your way until you need the tools it offers. That way nobody is faced with complexity unless they are deriving direct benefit from it, but when they need powerful tools those tools are available. This is the idea of progressive disclosure (as I understand it).

What I like about ‘hidden’ vs export of nested submodules is that you can freely intermix those declarations in your code (like you can with private now).

Yeah, this is an advantage. It’s not a bad idea, but it’s not a replacement for submodules either. They are complementary.

One thing `hidden` doesn’t do is allow you to say *why* the symbols are hidden and make that clear when they are imported. I can imagine allowing it to take a parameter that is a user-defined identifier stating *why* the symbols are hidden. For example, I might have `hidden(subclass)` and `hidden(extension)`. Making it a user-defined identifier would encourage names that mean something, require users to have read the documentation telling them what the identifier is, and make it clear to readers of the code why hidden symbols are imported (if the identifier used does not match any available symbols it would result in a compiler error).

A user-defined identifier would also be better than a small set of language-defined arguments because users would expect language-defined identifiers like “subclass” or “extension” to be *enforced* by the language. There has been strong pushback against doing that for several reasons. A parameterized `hidden` is a general feature that could express what is desired very well while not over-promising and under-delivering `protected` or `typeprivate` are extremely permeable restrictions and therefore don’t offer any real guarantees that `hidden` or a factored out submodule wouldn’t offer. It’s better to have a more general feature that does exactly what it says and leads users to believe it does.

There is also the question of how to separate a single type into multiple submodules, when we can’t declare storage in an extension.

We have discussed allowing storage to be declared in an extension in the past on the list. There is reasonable support for adding this eventually and we have ideas about how to do it (the complexity is primarily around insuring proper initialization of storage declared in extensions). Introducing a submodules feature could well drive demand for it by demonstrating concrete use cases where it becomes more necessary.

Even without that feature it is possible to separate a type into separate submodules using the features in my proposal. The details of how you do that would depend on concrete use cases.

···

On Feb 25, 2017, at 4:01 PM, Jonathan Hull <jhull@gbis.com> wrote:

On Feb 25, 2017, at 1:19 PM, Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:

On Feb 25, 2017, at 2:54 PM, Jonathan Hull via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Thanks,
Jon

Thanks,
Jon

On Feb 23, 2017, at 1:56 PM, Nevin Brackett-Rozinsky via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Introduction

There has been a deluge of discussion about access levels lately, all attempting to simplify the situation. Shortly after Swift 3 was released, many people realized that the new access modifier story was far more complex than the old one, and expressed the belief that the changes may have been a mistake.

In the months that followed, more and more people came to share the same view, and stage 2 of Swift 4 has seen a profusion of proposals addressing the issue. These proposals are generally small and focus on changing just one aspect of access control. However, given the situation we are in now, it is important to look at the problem in its entirety and arrive at a cohesive solution.

Background

During the Swift 3 timeframe there were lengthy debates about access control. The end results were to adopt SE-0025 <https://github.com/apple/swift-evolution/blob/master/proposals/0025-scoped-access-level.md>, which introduced the ‘fileprivate’ keyword and made ‘private’ scope-based, and SE-0117 <https://github.com/apple/swift-evolution/blob/master/proposals/0117-non-public-subclassable-by-default.md>, which made ‘public’ classes closed by default and introduced the ‘open’ keyword. At the time, there was broad agreement (and some dissent) that these were the right changes to make.

That belief, as well as the numerous arguments which led to it, were well-informed and thoughtfully considered. However, due to the inevitable linear nature of time, they were not based on first-hand experience with the new changes. Upon the release of Swift 3, we all gained that first-hand experience, and it quickly became apparent to many people that the new access control system was needlessly complicated, and not at all the improvement it had been heralded as.

Rather than make it easy to encapsulate implementation details of related types across multiple files, we had instead doubled down on requiring that many things go in a single file or else reveal their secrets to the entire module. Even worse, the new scope-based ‘private’ discouraged the preferred style of using extensions to build up a type. To cap it off, we went from needing to know two access modifier keywords (‘public’ and ‘private’) to needing to know four of them (‘private’, ‘fileprivate’, ‘public’, and ‘open’) without even providing a way to share details across a small number of related files.

Motivation – Overview

Many different ideas for access control have been expressed on the Swift Evolution mailing list. Some people want ‘protected’ or ‘friend’ or ‘extensible’ or various other combinations of type-based visibility. Other people (*cough* Slava) see no need for any levels finer than ‘internal’ at all. The common points of agreement, though, are that ‘fileprivate’ is an awkward spelling, and that the whole system is too complicated.

It is important that we as the Swift community are able to recognize our mistakes, and even more important that we fix them. We originally thought that the Swift 3 access control changes would be beneficial. However, experience with them in practice has shown that not to be the case. Instead, the language became more complex, and that has real costs. It is time for a simplification.

The prevailing view from recent discussions is that there should be just one access level more fine-grained than ‘internal’, and it should be spelled ‘private’. Let us leave aside for the moment what its exact meaning should be, and consider the other end of the scale.

Motivation – Rethinking ‘public’

Prior to Swift 3 we had just one access level more broad than ‘internal’, and for simplicity of the model it would be desirable to achieve that again. However, SE-0117 raised the point that certain library designs require a class to have subclasses within the defining module, but not outside. In other words, client code should not be able to create subclasses, even though they exist in the library. Let us be clear: this is a niche use-case, but it is important.

The most common situations are that a class either should not be subclassable at all—in which case it is ‘final’—or that it should be subclassable anywhere including client code. In order for a library to need a publicly closed class, it must first of all be using classes rather than a protocol with conforming structs, it must have a hierarchy with a parent class that is exposed outside the module, it must have subclasses of that parent class within the module, and it must also require that no external subclasses can exist. Putting all those criteria together, we see that closed classes are a rare thing to use. Nonetheless, they are valuable and can enable certain compiler optimizations, so we should support them.

Currently, the spelling for a closed class is ‘public’. This makes it very easy for library authors to create them. However, since they are a niche feature and most of the time ‘final’ is a better choice, we do not need to dedicate the entire ‘public’ keyword to them.

Moreover, object-oriented programming is just as much a first-class citizen in Swift as protocol-oriented programming is, so we should treat it as such. Classes are inherently inheritable: when one writes “class Foo {}”, then Foo has a default visibility of ‘internal’, and by default it can have subclasses. That is a straightforward model, and it is easy to work with.

If subclasses are to be disallowed, then Foo should be marked ‘final’; if Foo is exported to clients then it should be marked ‘public’; and if both are true then Foo should be ‘public final’. This covers all the common cases, and leaves only the narrow corner of closed classes to consider. Per the motivation of SE-0117, that case is worth handling. Per our collective experience with Swift 3, however, it is not worth the added complexity of its own access modifier keyword. We need a better way to spell it.

One of the reasons ‘public’ was previously chosen for closed classes is to provide a “soft default” for library authors, so they can prevent subclassing until they decide later whether to allow it in a future release. This is a misguided decision, as it prioritizes the convenience of library authors over the productivity of application developers. Library authors have a responsibility to decide what interfaces they present, and we should not encourage them to release libraries without making those decisions.

Moreover, we need to trust client programmers to make reasonable choices. If a library mistakenly allows subclassing when it shouldn’t, all a client has to do to work with it correctly is *not make subclasses*. The library is still usable. Conversely, if a library mistakenly prohibits subclassing, then there are things a client *should* be able to do but cannot. The harm to the users of a library is greater in this last case, because the ability to use the library is compromised, and that diminishes their productivity.

We should not make “soft defaults” that tend to negatively impact the clients of a library for the dubious benefit of enabling its author to procrastinate on a basic design decision. If someone truly wants to publish a library with a closed class, then we should support that. But it should be an intentional decision, not a default.

Motivation – Rethinking ‘final’

The question then comes to spelling. It is evident that preventing subclasses is closely related to being ‘final’. One possibility, then, is to allow the ‘final’ keyword to take a parameter. The parameter would be an access level, to indicate that the type acts like it is final when accessed from at or above that level.

In particular, ‘final(public)’ would mean “this class cannot be subclassed from outside the module”, or in other words “this class appears final publicly, although it is nonfinal internally”. This approach is more powerful than a ‘closed’ keyword because it also allows ‘final(internal)’, meaning “this class appears final to the rest of the module, although it can be subclassed privately”.

Motivation – Rethinking ‘private’

Now let us return to ‘private’, which as discussed earlier should be the only modifier that is tighter than ‘internal’. The purpose of ‘private’ is to enable encapsulation of related code, without revealing implementation details to the rest of the module. It should be compatible with using extensions to build up types, and it should not encourage overly-long files.

The natural definition, therefore, is that ‘private’ should mean “visible in a small group of files which belong together as a unit”. Of course Swift does not yet have submodules, and is not likely to gain them this year. However, if we say that each file is implicitly its own submodule unless otherwise specified, then the model works. In that view, ‘private’ will mean “visible in this submodule”, and for the time being that is synonymous with “visible in this file”.

Although this does not immediately enable lengthy files to be separated along natural divisions, it does lay the groundwork to allow doing so in the future when submodules arrive.

Motivation – Summary

By looking at access control in its entirety, we can adopt a system that empowers both library authors and client programmers to organize their code in a principled way, and to expose the interfaces they want in the places they need. The complexity of the Swift 3 visibility story, which many people now regret creating, will be replaced by a far simpler model which in several respects is even more powerful.

Notably, being able to parameterize ‘final’ lets classes be closed not just externally, but also in the rest of the module outside the ‘private’ scope if desired. Furthermore, defining ‘private’ as being scoped to a group of related files means that, as soon as we get the ability to create such groups, it will no longer be necessary to write large files just to keep implementation details hidden.

Recommendations

To recap, the ideas presented here focus on simplifying access control while still supporting important use cases such as closed class hierarchies. The indicated design uses just three familiar access keywords:

‘private’, to restrict visibility to a group of files, or just one file until we get that capability.

‘internal’, which is the default and does not have to be written, for module-wide visibility.

‘public’, to make visible outside the module, including the ability to subclass.

Additionally, the design allows ‘final’ to take any one of those visibility levels as a parameter, to indicate that the type should be treated as ‘final’ at and above the specified scope. Thus ‘final(public)’ prevents subclassing outside the module, while ‘final(internal)’ prevents it outside the ‘private’ scope. For consistency, ‘final(private)’ is also permitted, although it means the same thing as ‘final’ by itself.

Conclusion

The Swift 3 access situation is harmful—as evidenced by the myriad calls to fix it—not just because of its excessive complexity, but also because it prioritizes convenience for library authors over utility for their clients, and because it has no natural way to accommodate splitting large files into smaller ones while preserving encapsulation.

We have an opportunity now to correct a mistake we have made, and to set a precedent that we *will* correct our mistakes, rather than continue down an undesirable path simply because it seemed like a good idea at the time. When real-world experience demonstrates that a change has taken us in the wrong direction, we can and should update our decisions based on that new experience.

Therefore, in the situation at hand, we should reconsider our access modifier story and choose a model which is both simple and powerful. I have presented here my best efforts at describing such a system, and I offer it as one possible way to move forward.

– Nevin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution


(Matthew Johnson) #17


Additionally, the design allows ‘final’ to take any one of those visibility levels as a parameter, to indicate that the type should be treated as ‘final’ at and above the specified scope. Thus ‘final(public)’ prevents subclassing outside the module, while ‘final(internal)’ prevents it outside the ‘private’ scope. For consistency, ‘final(private)’ is also permitted, although it means the same thing as ‘final’ by itself.

I feel final(public) could be confusing given how we currently have private(get/set). What about public(final)? That’s at least consistent with current access syntax.

The syntax that most closely aligns with private(set) is internal(inherit). The parameter expresses a capability not a limitation and it is used with an access modifier that specifies the *maximum* access level that is allowed to have this capability. `final` is also misleading in this context because it implies to most people that there are *no* subclasses (inside or outside the module).

The only way to express `final` in Swift’s access control system would to have a `never` access modifier allowing you to say `never(inherit)`. I can’ think of any other uses for an access level like this. `final` is much more direct and is the term of art. I don’t see a reason to change the spelling of `final` and parameterizing it doesn’t make any sense - if something is `final` it is `final` in *all* scopes.

hmm… ‘semifinal' for things that are closed outside the submodule?

The name that has usually been used for this is `closed`. But no word that expresses a *restriction* is going to fit well into Swift’s access control system. The approach of Swift’s access control system (which I believe to be the best approach) is to express an upper bound on a *capability*. The basic capability is visibility of the symbol. That model is extended by talking about specific things a user might *do* with that symbol and setting an upper bound on that specific use: `internal private(set)`. `internal` is the bound on visibility and `private` is the bound on the ability to `set` the property.

`closed` is most naturally expressed in this system as `public internal(inherit)`. One interesting thing to observe is that `internal` is the default level of visibility. If we extend that default to capabilities as well as visibility the result is to give us exactly the behavior we want from a naked `public` annotation. We want to require an annotation for exposing anything (a symbol *or* a capability) outside the module. This means that `open` is really an alias for `public(inherit)` if we follow the principles underlying the model we have for access control.

Following this line of reasoning to its natural conclusion we can observe that the current interaction of `var` properties with access control *do not* follow the principles we have set forth for Swift. `public` variables *automatically* have `public` getters violating the principle that `internal` is the default. We could fix this by requiring users to say `public(set)`. I imagine there would be a lot of screaming in the short term if we proposed this. But it would put Swift’s access control system on a more consistent and principled footing.

I think there are three major reasons people have trouble with the current access control system. The `private` / `fileprivate` name situation is an obvious one. But I think perhaps more important is the fact that the system has idiosyncrasies such as introducing `open` rather than following the capability model of `set` (which would have resulted in `public(inherit)`). Another idiosyncrasy noted above is that `set` defaults to the level of the getter rather than `internal` (but bounded by visibility). Finally, the principle that underlies the access modifiers (modulo the idiosyncrasy of `open`) is that of a bounded scope. But this principle is implicit - each keyword is defined on an ad-hoc basis rather than making the underlying principle explicit and defining any shorthand we want in terms of that (as syntactic sugar for common cases and recommended defaults).

This is why I strongly believe the best approach is to make the underlying principle of scopes and capabilities with an `internal` default explicit. I think this principle is easy to teach and easy to understand once it is made explicit. It is a wonderful (but currently rather hidden) aspect of Swift. If we can teach users to understand this principle and to learn the names of scopes and capabilities they will not find it difficult and complex. They will find it as easy to learn as a library function: we would have `scope(<capbability name>, <scope name>)`**. The capability would be defaulted to basic visibility. The scope would be defaulted to the current scope.

Consider scope(inherit, public)`. It would read like “the scope of inheritance is public”. It tells the reader *exactly* what is happening. We could conceptualize shortened like `open` as something like `accessalias open = scope(inherit, public)`. Tools could even allow you to cmd-Click to the definition to see a definition like this. Users would learn something that is no more complicated than a single function call and any time they are confused by shorthand they could cmd-Click or look up documentation that explains it clearly in terms of this "function call”. Note: `accessalias` wouldn’t be a real thing, just a way to document and explain the shorthand keywords we have for “soft defaults” and common conveniences.

This seems to me like a very simple and elegant yet extraordinarily powerful system. I think it’s much better than a small collection of ad-hoc definitions of very specific behaviors.

**Anyone who has been reading my thoughts on this might notice that I am using `scope` rather than `scoped` in this post. As I was writing this I realized that when we think of access in terms of capability it reads better with `scope` than it does with `scoped` (i.e. “inheritance is scoped public” does not read as well as “the scope of inheritance is public".

···

On Feb 25, 2017, at 4:23 PM, Jonathan Hull <jhull@gbis.com> wrote:

On Feb 25, 2017, at 1:43 PM, Matthew Johnson via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Feb 25, 2017, at 3:27 PM, Kevin Nattinger via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Thanks,
Jon


(Tino) #18

I think everybody agrees (at the very least to some degree) that the current access level system is too complicated.

I'm not sure about that — discussions about adding even more access levels aren't that uncommon, and peoples opinions tend to be surprisingly diverse.
I guess that the rate of proponents of more sophisticated (and complicated) rules is even higher on the list than out in the wild.


(Charles Srstka) #19

The nice thing about that is that we could declare something as private public(inherit), making it not callable but possible to override, which would be a great fit for methods that are only intended as override points and are not meant to be called directly, like the many such methods on NSView.

Charles

···

On Feb 25, 2017, at 5:27 PM, Matthew Johnson via swift-evolution <swift-evolution@swift.org> wrote:

The name that has usually been used for this is `closed`. But no word that expresses a *restriction* is going to fit well into Swift’s access control system. The approach of Swift’s access control system (which I believe to be the best approach) is to express an upper bound on a *capability*. The basic capability is visibility of the symbol. That model is extended by talking about specific things a user might *do* with that symbol and setting an upper bound on that specific use: `internal private(set)`. `internal` is the bound on visibility and `private` is the bound on the ability to `set` the property.

`closed` is most naturally expressed in this system as `public internal(inherit)`. One interesting thing to observe is that `internal` is the default level of visibility. If we extend that default to capabilities as well as visibility the result is to give us exactly the behavior we want from a naked `public` annotation. We want to require an annotation for exposing anything (a symbol *or* a capability) outside the module. This means that `open` is really an alias for `public(inherit)` if we follow the principles underlying the model we have for access control.

Following this line of reasoning to its natural conclusion we can observe that the current interaction of `var` properties with access control *do not* follow the principles we have set forth for Swift. `public` variables *automatically* have `public` getters violating the principle that `internal` is the default. We could fix this by requiring users to say `public(set)`. I imagine there would be a lot of screaming in the short term if we proposed this. But it would put Swift’s access control system on a more consistent and principled footing.

I think there are three major reasons people have trouble with the current access control system. The `private` / `fileprivate` name situation is an obvious one. But I think perhaps more important is the fact that the system has idiosyncrasies such as introducing `open` rather than following the capability model of `set` (which would have resulted in `public(inherit)`). Another idiosyncrasy noted above is that `set` defaults to the level of the getter rather than `internal` (but bounded by visibility). Finally, the principle that underlies the access modifiers (modulo the idiosyncrasy of `open`) is that of a bounded scope. But this principle is implicit - each keyword is defined on an ad-hoc basis rather than making the underlying principle explicit and defining any shorthand we want in terms of that (as syntactic sugar for common cases and recommended defaults).

This is why I strongly believe the best approach is to make the underlying principle of scopes and capabilities with an `internal` default explicit. I think this principle is easy to teach and easy to understand once it is made explicit. It is a wonderful (but currently rather hidden) aspect of Swift. If we can teach users to understand this principle and to learn the names of scopes and capabilities they will not find it difficult and complex. They will find it as easy to learn as a library function: we would have `scope(<capbability name>, <scope name>)`**. The capability would be defaulted to basic visibility. The scope would be defaulted to the current scope.

Consider scope(inherit, public)`. It would read like “the scope of inheritance is public”. It tells the reader *exactly* what is happening. We could conceptualize shortened like `open` as something like `accessalias open = scope(inherit, public)`. Tools could even allow you to cmd-Click to the definition to see a definition like this. Users would learn something that is no more complicated than a single function call and any time they are confused by shorthand they could cmd-Click or look up documentation that explains it clearly in terms of this "function call”. Note: `accessalias` wouldn’t be a real thing, just a way to document and explain the shorthand keywords we have for “soft defaults” and common conveniences.

This seems to me like a very simple and elegant yet extraordinarily powerful system. I think it’s much better than a small collection of ad-hoc definitions of very specific behaviors.

**Anyone who has been reading my thoughts on this might notice that I am using `scope` rather than `scoped` in this post. As I was writing this I realized that when we think of access in terms of capability it reads better with `scope` than it does with `scoped` (i.e. “inheritance is scoped public” does not read as well as “the scope of inheritance is public".


(Jon Hull) #20

+1000

This is the best of the proposals I have seen. I think it works by itself as a complete proposal, but since we are talking "comprehensive rethink", there is one use case which most of the proposals seem to miss.

That is, what if I want to make a class which is both Public and Subclassable, but I want to keep some of the implementation details private to the class, while still allowing subclasses and extensions (across the module boundary) to have access to those details. A concrete example of this is UIGestureRecognizer, which is intended to be subclassed outside of its framework, but hides potentially catastrophic things (like being able to set the state) from callers. This isn’t currently possible in Swift AFAICT without importing from ObjC.

My solution to this has been to allow something to be marked ‘public hidden’ or ‘hidden', which means that it is public/internal, but has it’s visibility limited only to files which opt-in to seeing it with ‘import hidden filename’ (instead of just ‘import filename’). I think this is the simplest way to provide this feature, which is sometimes very necessary. It also captures authorial intent very nicely.

My submodule proposal is able to support this use case. You would just put the symbols intended for subclasses in a separate submodule. These submodules could be exposed to users with any name you desire, regardless of the internal structure of your module. You could even arrange for users to get all of the non-subclass-only symbols automatically when they import your module but require explicit import of each individual subclass-only submodule in places where it is needed.

I agree that this could also be solved with nested submodules

A minor note, but you would not need to nest them. You could do it either way.

I can’t think of how to do this, but I trust you that it can.

, but the current proposals all seem to add a lot of complexity. This complexity does give you much finer grain control over visibility, but what I like about Nevin’s proposal is that it is ‘just enough’ control with minimal complexity.

Determining what “just enough” is is obviously a very subjective thing. :slight_smile: Everyone seems to have their own opinion about this (more than usual) which is going to make it one of the more complicated discussions we’ll have on the list.

I worked very hard on my proposal to try to design a system that stays out of your way until you need the tools it offers. That way nobody is faced with complexity unless they are deriving direct benefit from it, but when they need powerful tools those tools are available. This is the idea of progressive disclosure (as I understand it).

True. I would personally define “just enough” as easy for the 80% and possible for the 98%.

I don’t think your proposal is bad at all. I just like the simplicity of Nevin’s a little more (especially with the confusion around ‘open’). In fact, I would like to see you combine them a bit (see below).

What I like about ‘hidden’ vs export of nested submodules is that you can freely intermix those declarations in your code (like you can with private now).

Yeah, this is an advantage. It’s not a bad idea, but it’s not a replacement for submodules either. They are complementary.

Agreed.

What I would really like to see is Nevin’s proposal with a simplified version of yours providing the submodules, and something like ‘hidden’ replacing the idea of ‘export’.

In more detail, I really like just having ‘private’, ‘internal’, and ‘public’ where private means private to the submodule. It is very simple and understandable. I think having a single level of submodule using your ‘submodule’ syntax is the simplest way to provide those.

I think I would be ok with nested submodules as well, again using your syntax, as long as we have ‘private’ mean private within the current submodule (and children). If we find we really do need parametrized private (or equivalent) to provide visibility to a specific parent scope, that is something that can be an additive (non-breaking) change later. I think we will be able do pretty much everything we need to without it though.

One thing `hidden` doesn’t do is allow you to say *why* the symbols are hidden and make that clear when they are imported. I can imagine allowing it to take a parameter that is a user-defined identifier stating *why* the symbols are hidden. For example, I might have `hidden(subclass)` and `hidden(extension)`. Making it a user-defined identifier would encourage names that mean something, require users to have read the documentation telling them what the identifier is, and make it clear to readers of the code why hidden symbols are imported (if the identifier used does not match any available symbols it would result in a compiler error).

A user-defined identifier would also be better than a small set of language-defined arguments because users would expect language-defined identifiers like “subclass” or “extension” to be *enforced* by the language. There has been strong pushback against doing that for several reasons. A parameterized `hidden` is a general feature that could express what is desired very well while not over-promising and under-delivering `protected` or `typeprivate` are extremely permeable restrictions and therefore don’t offer any real guarantees that `hidden` or a factored out submodule wouldn’t offer. It’s better to have a more general feature that does exactly what it says and leads users to believe it does.

I disagree. The whole point of ‘hidden’ is to protect the *callers* of a type from accidentally invoking its deliberate extension points. It basically means “don’t touch these unless you understand them”, and having to ‘import hidden’ is explicitly taking responsibility for this. I don’t think making extenders type a secret code would actually help anything, and would just annoy/complicate. There is only so much you can do to protect people from themselves.

Right now, anything that would be marked ‘public hidden’ has to be marked ‘public’, so I think it is a thorough improvement over the status quo.

There is also the question of how to separate a single type into multiple submodules, when we can’t declare storage in an extension.

We have discussed allowing storage to be declared in an extension in the past on the list. There is reasonable support for adding this eventually and we have ideas about how to do it (the complexity is primarily around insuring proper initialization of storage declared in extensions). Introducing a submodules feature could well drive demand for it by demonstrating concrete use cases where it becomes more necessary.

Even without that feature it is possible to separate a type into separate submodules using the features in my proposal. The details of how you do that would depend on concrete use cases.

True, and I do want this feature… but I don’t want to rely on it being there when it is not yet on the roadmap.

Thanks,
Jon

···

On Feb 25, 2017, at 2:41 PM, Matthew Johnson <matthew@anandabits.com> wrote:

On Feb 25, 2017, at 4:01 PM, Jonathan Hull <jhull@gbis.com <mailto:jhull@gbis.com>> wrote:

On Feb 25, 2017, at 1:19 PM, Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:

On Feb 25, 2017, at 2:54 PM, Jonathan Hull via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote: