[Proposal][Discussion] Modular Swift

I would honestly love to see something which just maps modules to folders/groups for simplicity sake.

There is nothing about this scheme that prevents you from organizing your code this way. However, modulo that particular method of organization, you don’t really gain much as a user of the language by imposing this restriction.

I think the big gain is that users of IDEs would just be able to graphically organize modules in Xcode. Non-IDE users could use the finder to organize things into nested folders. This maps pretty well with the way I organize my code already (others may be different), so it doesn’t add much complexity.

Xcode today is able to do this with nested classes and structures with the hierarchy navigator. Non-IDE users can use grep or (hopefully) some syntax-aware plugin system to help them find these declarations. Really, again, this is a design pattern and one that can be enforced by linters. If we were to impose this restriction at the language level it ties this proposal down to file system structure and we are explicitly trying to avoid that.

On a side note, I do think that people will quickly want a way to reference a particular submodule boundary with access modifiers. It may not be part of this proposal, but it is somewhat inevitable. We should consider that as we consider this proposal…

Modules are not types, access control makes no sense here. APIs may be exported (or not) across module boundaries, but modules themselves are not arbitrary programming constructs nor do we consider them to carry semantic weight as in some other ML-likes. A public module is no different from a private module if you can import it. A private module is no different from a public one if you cannot.

···

On Feb 21, 2017, at 1:46 AM, Jonathan Hull <jhull@gbis.com> wrote:

On Feb 20, 2017, at 6:44 PM, Robert Widmann <devteam.codafi@gmail.com <mailto:devteam.codafi@gmail.com>> wrote:

Thanks,
Jon

More generally, we need to move access control away as far away from filesystems as possible. One day, the compiler is going to get ported over to a platform with some bonkers rules that are going to turn around and bite us.

This is the key thing which I think is being debated right now. Swift 2 defined the file as a unit of compilation and based its elegant system of access modifiers on that. With Swift 3 we have a mix of type/scope based modifiers and file based modifiers… and it is super confusing to everyone because we have mixed our metaphors.

As much as I want modules, I am now convinced that this proposal will only make that situation worse. We need to pick one way (file based) or the other (scope based) and commit to it… but either way, it will require a larger overhaul to make the system consistent/usable/teachable again.

As an analogy, it is like some people are trying to play rock music during a classical music concert. Both are great independently, and different people may prefer one or the other… but trying to play them on top of one another just results in noise.

Is there anything specifically about this proposal that turns you off, or is it access control as it stands today?

···

On Feb 21, 2017, at 2:01 AM, Jonathan Hull <jhull@gbis.com> wrote:

On Feb 20, 2017, at 10:46 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Thanks,
Jon

I think you're missing my point.

What you appear to be saying is: "If two things need privileged access to each other, they should be in the same module; if they should hide implementation details from each other, they should be in different modules." But it's perfectly possible for two things to *both* need privileged access to some APIs *and* hide implementation details from one another.

Basically, my objection is, this proposal requires that parent modules only communicate with submodules through fully `public` APIs which are also available outside the module. If you want to encapsulate certain implementation details by putting them in a submodule, you must paradoxically expose other details—the APIs through which the parent module is supposed to interact with the submodule—to the whole world. That restriction conflicts with Swift's important goal of ensuring that libraries only expose API surface they're committed to supporting indefinitely.

Basically, I think that either we need a new access level meaning "visible to the top-level module and all of its submodules, but not to other modules", or we need a way to specify that a submodule is internal-only and shouldn't be importable by outside code. My submodule sketch the other day took the second approach, but I think one or the other is necessary if we want submodules to serve both as public units of API surface and private encapsulators of implementation detail.

···

On Feb 21, 2017, at 6:09 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

If I'm reading this correctly, you're proposing that the `internal` APIs in a submodule should *not* be accessible to enclosing modules. I also don't see any indication that you can control who is allowed to import a particular submodule.

That means that, if you use a submodule to encapsulate internal state, the APIs that are available to the parent module are *also* available to any rando who feels like importing your submodule. I don't think that's going to be a tenable design.

If the state is truly internal, and you are using internal access control, this is impossible. The internal state cannot cross the module boundary unless it is marked public. If a “random” feels like importing my submodule, they will not have access to any symbols.

--
Brent Royal-Gordon
Architechies

If I'm reading this correctly, you're proposing that the `internal` APIs in a submodule should *not* be accessible to enclosing modules. I also don't see any indication that you can control who is allowed to import a particular submodule.

That means that, if you use a submodule to encapsulate internal state, the APIs that are available to the parent module are *also* available to any rando who feels like importing your submodule. I don't think that's going to be a tenable design.

If the state is truly internal, and you are using internal access control, this is impossible. The internal state cannot cross the module boundary unless it is marked public. If a “random” feels like importing my submodule, they will not have access to any symbols.

I think you're missing my point.

What you appear to be saying is: "If two things need privileged access to each other, they should be in the same module; if they should hide implementation details from each other, they should be in different modules." But it's perfectly possible for two things to *both* need privileged access to some APIs *and* hide implementation details from one another.

Basically, my objection is, this proposal requires that parent modules only communicate with submodules through fully `public` APIs which are also available outside the module. If you want to encapsulate certain implementation details by putting them in a submodule, you must paradoxically expose other details—the APIs through which the parent module is supposed to interact with the submodule—to the whole world. That restriction conflicts with Swift's important goal of ensuring that libraries only expose API surface they're committed to supporting indefinitely.

That is not what this proposal requires. A public API is ripe for re(export), but if the parent wishes to communicate with its children without exporting across the module boundary, see the definition of `internal`.

···

On Feb 21, 2017, at 9:38 PM, Brent Royal-Gordon <brent@architechies.com> wrote:

On Feb 21, 2017, at 6:09 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

Basically, I think that either we need a new access level meaning "visible to the top-level module and all of its submodules, but not to other modules", or we need a way to specify that a submodule is internal-only and shouldn't be importable by outside code. My submodule sketch the other day took the second approach, but I think one or the other is necessary if we want submodules to serve both as public units of API surface and private encapsulators of implementation detail.

--
Brent Royal-Gordon
Architechies

Good question! This behavior is actually the behavior that exists today. For example, open a playground and type

import Foundation.NSDebug

let s : NSString = “"

You’ll notice no matter which submodule you try to visit (Darwin.uuid is another good example), Swift has decided to insert a top-level import. We decided not to change this behavior to maintain source compatibility.

So there is no way to import a single submodule? If that is the case why not just disallow the syntax which implies only a single submodule is getting imported and require users to say `import Foo`? I don’t know what the rationale was for this in the past, but I think it’s undesirable and we’re trying to design something for the future here.

···

On Feb 21, 2017, at 10:25 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

On Feb 21, 2017, at 11:21 PM, Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:

On Feb 20, 2017, at 7:56 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Good Evening All,

Jaden Geller and I have been considering a (sub)module system for Swift that would complement the existing language but also provide sorely needed modularity. A draft of the proposal is attached to this email, but it can also be read as a gist <https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a&gt; if you desire.

Cheers,

~Robert Widmann

Modular Swift

Proposal: SE-NNNN <https://gist.github.com/CodaFi/NNNN-filename.md&gt;
Authors: Robert Widmann <https://github.com/codafi&gt;, Jaden Geller <https://github.com/JadenGeller&gt;
Review Manager: TBD
Status: Awaiting review
<Proposal.md · GitHub

Almost every major programming language supports some form of modular programming through constructs like (sub)modules, packages, or interfaces. Swift, though it provides top-level modules to organize code under, does not provide a complete implementation of any of these concepts, which has led instead to the proliferation of access control levels. This has not proven an effective way to decompose programs into manageable parts, and exposes the need for a real system of modules to solve this modularity problem once and for all.

Separation of code into distinct islands of functionality should be a first-class construct in the language, not dependent on external files and tools or filesystems. To that end, we propose the introduction of a lightweight module system for Swift.

Swift-evolution thread <applewebdata://0EF07D19-5B94-45DD-87E5-5CB1D22E752F>
<Proposal.md · GitHub

Swift has reached a point in its evolution where rich libraries and large projects that take on many dependencies have matured significantly. To accomodate the information-hiding and semantics-signalling needs of these users at the time, Swift began its access control story with just three access modifiers: public, private, and internal then grew fileprivate and open as the need to express locality of implementation and "subclassability" arose respectively. In doing so, Swift's access control scheme has become anti-modular.

<Proposal.md · GitHub solution

We propose the introduction of a lightweight module system for Swift. More than simply namspaces, a module declaration interacts with Swift's access control to provide an API boundary that allows better control over an interface's design.

<Proposal.md · GitHub design

<Proposal.md · GitHub

A module is a named region that introduces a lexical scope into which declarations may be nested. The name of the module can be used to access these member declarations. A module, like other aggregate structures in Swift, may be extended with new declarations over one or more translation units (files).

We propose a new declaration kind, module-decl be added to the language. A proposed grammar using the new modulekeyword is given below:

GRAMMAR OF A MODULE DECLARATION

module-declaration -> `module` module-identifier module-body
module-name -> identifier
module-body -> { module-members(opt) }
module-members -> module-member module-members(opt)
module-member -> declaration | compiler-control-statement
GRAMMAR OF A DECLARATION

+ declaration -> module-declaration
<Proposal.md · GitHub Semantics

Syntax and semantics for imports, as it already supports referencing submodules imported from C and Objective-C modules, remains unchanged:

// The outermost module is given explicitly
// by passing `-module-name=Foo` or exists implicitly, as today.
// module Foo {
public class A {}

module Bar {
  module Baz {
    public class C {}
  }

  public class B {}
}

let message = "Hello, Wisconsin!"
// } // End declarations added to module Foo.
To consume this interface:

// imports all of Foo, Foo.Bar, and Foo.Bar.Baz
import Foo.Bar.Baz

I’ve read this a couple of times now and I keep getting hung up on this. Is the comment a mistake? I would only expect to get Foo.Bar.Baz here, not Foo.Bar or Foo itself. If it’s not a mistake, why did you choose this behavior?

// imports Foo.A as A
import class Foo.A
// imports Foo.Bar.B as B
import class Foo.Bar.B
// imports Foo.Bar.Baz.C as C
import class Foo.Bar.Baz.C
A module declaration may only appear as a top-level entity or as a member of another module declaration. The following code is therefore invalid:

module Foo {
  class Bar {
    module Baz {} // error: module declaration cannot be nested inside type 'Bar'
  }
}
To extend an existing module declaration, simply reference its module name in an extension declaration.

// In module 'Foo'
module Bar {
  public class A {}

  module Baz {}
}

extension Bar {
  public struct B {}
}

extension Bar.Baz {
  public enum C { case D }
}
<Proposal.md · GitHub and Access Control

The semantics of some existing access control modifiers shall also be extended to support module declarations:

open and public declarations are exported by a module for consumption by clients of the module.
internal declarations scope over the entire module and any derived submodules.
By default, to preserve encapsulation of interfaces, modules are "sealed" and may only be "opened" by explicit named import. However, it is often desirable to export a module and a set of submodules or even modules from external dependencies along with a given interface. We propose the public keyword be used for this purpose:

// Defines top-level module "Foo"
//module Foo {
public import Foo.Bar.Baz
public import Foundation.Date
//}
Which then causes the following (sub)modules to be imported into scope along with Foo:

// imports Foo, Foo.Bar.Baz, and Foundation.Date
import Foo
To support existing Swift packages that cannot have opted into modules, and to preserve the scriptable nature of Swift, module declarations shall be optional. Any Swift program that does not declare at least one top-level module explicitly is considered part of an unnamed special "Global Module" with the same rules of access control as today. To give declarations in the Global Module an explicit module without using a module declaration, use the -module-name flag.

<Proposal.md · GitHub on Existing Code

This proposal is intentionally additive. There is no impact on existing code.

<Proposal.md · GitHub considered

<Proposal.md · GitHub Modules Everywhere

Declarations in the top-level of a program exist today in the top-level of the corresponding module. If desired, this module declaration could be required to be explicit like so:

module Foo {
  module Bar {
    module Baz {}
  }
}
However, we feel that imposing such a requirement not only complicates the outermost scope, it requires inserting needless extension Foo {} scopes in every file. It also violates the principle of progressive disclosure by forcing all new adoptees of Swift to learn what a module is without actually using the module system.

<Proposal.md · GitHub Extensions

Nested module extensions may be "expanded" as it were to the following:

module Foo {
  module Bar {}
}

extension Foo {
  extension Bar {}
}
However, this syntax is currently not enabled in general in Swift. This problem should be revisted in a future proposal.

<Proposal.md · GitHub (Source-Breaking Changes)

The system described above is intended to be entirely source and binary compatible. Nonetheless, in its design we feel we have obviated certain existing features and recommend their deprecation in future proposals:

fileprivate access can be recreated by creating a private "utility submodule" containing declarations of at least internal access.
@_exported, the private directive to re-export modules today, should be deprecated and removed.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

TJ and I had a proposal <https://gist.github.com/CodaFi/42e5e5e94d857547abc381d9a9d0afd6&gt; that would have closed this loophole by overhauling qualified imports syntax.

···

On Feb 21, 2017, at 11:36 PM, Matthew Johnson <matthew@anandabits.com> wrote:

On Feb 21, 2017, at 10:25 PM, Robert Widmann <devteam.codafi@gmail.com <mailto:devteam.codafi@gmail.com>> wrote:

Good question! This behavior is actually the behavior that exists today. For example, open a playground and type

import Foundation.NSDebug

let s : NSString = “"

You’ll notice no matter which submodule you try to visit (Darwin.uuid is another good example), Swift has decided to insert a top-level import. We decided not to change this behavior to maintain source compatibility.

So there is no way to import a single submodule? If that is the case why not just disallow the syntax which implies only a single submodule is getting imported and require users to say `import Foo`? I don’t know what the rationale was for this in the past, but I think it’s undesirable and we’re trying to design something for the future here.

On Feb 21, 2017, at 11:21 PM, Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:

On Feb 20, 2017, at 7:56 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Good Evening All,

Jaden Geller and I have been considering a (sub)module system for Swift that would complement the existing language but also provide sorely needed modularity. A draft of the proposal is attached to this email, but it can also be read as a gist <https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a&gt; if you desire.

Cheers,

~Robert Widmann

Modular Swift

Proposal: SE-NNNN <https://gist.github.com/CodaFi/NNNN-filename.md&gt;
Authors: Robert Widmann <https://github.com/codafi&gt;, Jaden Geller <https://github.com/JadenGeller&gt;
Review Manager: TBD
Status: Awaiting review
<Proposal.md · GitHub

Almost every major programming language supports some form of modular programming through constructs like (sub)modules, packages, or interfaces. Swift, though it provides top-level modules to organize code under, does not provide a complete implementation of any of these concepts, which has led instead to the proliferation of access control levels. This has not proven an effective way to decompose programs into manageable parts, and exposes the need for a real system of modules to solve this modularity problem once and for all.

Separation of code into distinct islands of functionality should be a first-class construct in the language, not dependent on external files and tools or filesystems. To that end, we propose the introduction of a lightweight module system for Swift.

Swift-evolution thread <applewebdata://8691CAB6-B77F-4439-B723-BA0D69F0045D>
<Proposal.md · GitHub

Swift has reached a point in its evolution where rich libraries and large projects that take on many dependencies have matured significantly. To accomodate the information-hiding and semantics-signalling needs of these users at the time, Swift began its access control story with just three access modifiers: public, private, and internal then grew fileprivate and open as the need to express locality of implementation and "subclassability" arose respectively. In doing so, Swift's access control scheme has become anti-modular.

<Proposal.md · GitHub solution

We propose the introduction of a lightweight module system for Swift. More than simply namspaces, a module declaration interacts with Swift's access control to provide an API boundary that allows better control over an interface's design.

<Proposal.md · GitHub design

<Proposal.md · GitHub

A module is a named region that introduces a lexical scope into which declarations may be nested. The name of the module can be used to access these member declarations. A module, like other aggregate structures in Swift, may be extended with new declarations over one or more translation units (files).

We propose a new declaration kind, module-decl be added to the language. A proposed grammar using the new modulekeyword is given below:

GRAMMAR OF A MODULE DECLARATION

module-declaration -> `module` module-identifier module-body
module-name -> identifier
module-body -> { module-members(opt) }
module-members -> module-member module-members(opt)
module-member -> declaration | compiler-control-statement
GRAMMAR OF A DECLARATION

+ declaration -> module-declaration
<Proposal.md · GitHub Semantics

Syntax and semantics for imports, as it already supports referencing submodules imported from C and Objective-C modules, remains unchanged:

// The outermost module is given explicitly
// by passing `-module-name=Foo` or exists implicitly, as today.
// module Foo {
public class A {}

module Bar {
  module Baz {
    public class C {}
  }

  public class B {}
}

let message = "Hello, Wisconsin!"
// } // End declarations added to module Foo.
To consume this interface:

// imports all of Foo, Foo.Bar, and Foo.Bar.Baz
import Foo.Bar.Baz

I’ve read this a couple of times now and I keep getting hung up on this. Is the comment a mistake? I would only expect to get Foo.Bar.Baz here, not Foo.Bar or Foo itself. If it’s not a mistake, why did you choose this behavior?

// imports Foo.A as A
import class Foo.A
// imports Foo.Bar.B as B
import class Foo.Bar.B
// imports Foo.Bar.Baz.C as C
import class Foo.Bar.Baz.C
A module declaration may only appear as a top-level entity or as a member of another module declaration. The following code is therefore invalid:

module Foo {
  class Bar {
    module Baz {} // error: module declaration cannot be nested inside type 'Bar'
  }
}
To extend an existing module declaration, simply reference its module name in an extension declaration.

// In module 'Foo'
module Bar {
  public class A {}

  module Baz {}
}

extension Bar {
  public struct B {}
}

extension Bar.Baz {
  public enum C { case D }
}
<Proposal.md · GitHub and Access Control

The semantics of some existing access control modifiers shall also be extended to support module declarations:

open and public declarations are exported by a module for consumption by clients of the module.
internal declarations scope over the entire module and any derived submodules.
By default, to preserve encapsulation of interfaces, modules are "sealed" and may only be "opened" by explicit named import. However, it is often desirable to export a module and a set of submodules or even modules from external dependencies along with a given interface. We propose the public keyword be used for this purpose:

// Defines top-level module "Foo"
//module Foo {
public import Foo.Bar.Baz
public import Foundation.Date
//}
Which then causes the following (sub)modules to be imported into scope along with Foo:

// imports Foo, Foo.Bar.Baz, and Foundation.Date
import Foo
To support existing Swift packages that cannot have opted into modules, and to preserve the scriptable nature of Swift, module declarations shall be optional. Any Swift program that does not declare at least one top-level module explicitly is considered part of an unnamed special "Global Module" with the same rules of access control as today. To give declarations in the Global Module an explicit module without using a module declaration, use the -module-name flag.

<Proposal.md · GitHub on Existing Code

This proposal is intentionally additive. There is no impact on existing code.

<Proposal.md · GitHub considered

<Proposal.md · GitHub Modules Everywhere

Declarations in the top-level of a program exist today in the top-level of the corresponding module. If desired, this module declaration could be required to be explicit like so:

module Foo {
  module Bar {
    module Baz {}
  }
}
However, we feel that imposing such a requirement not only complicates the outermost scope, it requires inserting needless extension Foo {} scopes in every file. It also violates the principle of progressive disclosure by forcing all new adoptees of Swift to learn what a module is without actually using the module system.

<Proposal.md · GitHub Extensions

Nested module extensions may be "expanded" as it were to the following:

module Foo {
  module Bar {}
}

extension Foo {
  extension Bar {}
}
However, this syntax is currently not enabled in general in Swift. This problem should be revisted in a future proposal.

<Proposal.md · GitHub (Source-Breaking Changes)

The system described above is intended to be entirely source and binary compatible. Nonetheless, in its design we feel we have obviated certain existing features and recommend their deprecation in future proposals:

fileprivate access can be recreated by creating a private "utility submodule" containing declarations of at least internal access.
@_exported, the private directive to re-export modules today, should be deprecated and removed.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

The one thing i don't understand is the point of using extensions to extend modules across multiple files, why not just allow declaring the submodule in multiple files?

···

Le 20 févr. 2017 à 22:48, Jaden Geller via swift-evolution <swift-evolution@swift.org> a écrit :

Oh, I see. You’re questioning the motivation of having scope-granularity submodules at all! My misunderstanding.

I actually hadn’t considered this as added complexity. In my mind, a scoped module declaration seems more Swifty than a file module declaration. It builds on the existing syntax in Swift for defining other sorts of scopes, e.g. types.

Cheers,
Jaden Geller

On Feb 20, 2017, at 10:41 PM, Jonathan Hull via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

I think my question is: Why do we want to allow submodules that are smaller than a file? What does that give us to offset the added complexity?

Thanks,
Jon

On Feb 20, 2017, at 6:44 PM, Robert Widmann <devteam.codafi@gmail.com <mailto:devteam.codafi@gmail.com>> wrote:

On Feb 20, 2017, at 9:36 PM, Jonathan Hull <jhull@gbis.com <mailto:jhull@gbis.com>> wrote:

What is the rational for having modules covering only part of a file? Wouldn’t it be less clutter to have an annotation which worked for the whole file. At the very least it would be nice to have an option to spell it in a way that applies to the whole file. Otherwise, everything will be indented another level.

That is a valid spelling (Rust, IIRC, allows that spelling), but one that is easy to miss sitting in a file and makes it confusing to introduce submodules. If you include the annotation then define a submodule later down in the file, suddenly you have to remember whether you annotated the file or whether the submodule you’ve just written is going into the top-level module. See:

// -module-name=Foo
// module Foo {
module Bar; // Shorthand for “This file defines Foo.Bar”

/* Code */

// This defines “Foo.Bar.Baz”, but would you know that if it appeared below the fold?
module Baz {}
//}

If anything, this can be added later if evolution converges on it.

I would honestly love to see something which just maps modules to folders/groups for simplicity sake.

There is nothing about this scheme that prevents you from organizing your code this way. However, modulo that particular method of organization, you don’t really gain much as a user of the language by imposing this restriction.

I haven’t thought about it too much yet, so I could easily be missing something obvious...

Thanks,
Jon

On Feb 20, 2017, at 5:56 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Good Evening All,

Jaden Geller and I have been considering a (sub)module system for Swift that would complement the existing language but also provide sorely needed modularity. A draft of the proposal is attached to this email, but it can also be read as a gist <https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a&gt; if you desire.

Cheers,

~Robert Widmann

Modular Swift

Proposal: SE-NNNN <https://gist.github.com/CodaFi/NNNN-filename.md&gt;
Authors: Robert Widmann <https://github.com/codafi&gt;, Jaden Geller <https://github.com/JadenGeller&gt;
Review Manager: TBD
Status: Awaiting review
<Proposal.md · GitHub

Almost every major programming language supports some form of modular programming through constructs like (sub)modules, packages, or interfaces. Swift, though it provides top-level modules to organize code under, does not provide a complete implementation of any of these concepts, which has led instead to the proliferation of access control levels. This has not proven an effective way to decompose programs into manageable parts, and exposes the need for a real system of modules to solve this modularity problem once and for all.

Separation of code into distinct islands of functionality should be a first-class construct in the language, not dependent on external files and tools or filesystems. To that end, we propose the introduction of a lightweight module system for Swift.

Swift-evolution thread <applewebdata://2ED871CE-7283-4720-945D-299214ADB1B4>
<Proposal.md · GitHub

Swift has reached a point in its evolution where rich libraries and large projects that take on many dependencies have matured significantly. To accomodate the information-hiding and semantics-signalling needs of these users at the time, Swift began its access control story with just three access modifiers: public, private, and internal then grew fileprivate and open as the need to express locality of implementation and "subclassability" arose respectively. In doing so, Swift's access control scheme has become anti-modular.

<Proposal.md · GitHub solution

We propose the introduction of a lightweight module system for Swift. More than simply namspaces, a module declaration interacts with Swift's access control to provide an API boundary that allows better control over an interface's design.

<Proposal.md · GitHub design

<Proposal.md · GitHub

A module is a named region that introduces a lexical scope into which declarations may be nested. The name of the module can be used to access these member declarations. A module, like other aggregate structures in Swift, may be extended with new declarations over one or more translation units (files).

We propose a new declaration kind, module-decl be added to the language. A proposed grammar using the new modulekeyword is given below:

GRAMMAR OF A MODULE DECLARATION

module-declaration -> `module` module-identifier module-body
module-name -> identifier
module-body -> { module-members(opt) }
module-members -> module-member module-members(opt)
module-member -> declaration | compiler-control-statement
GRAMMAR OF A DECLARATION

+ declaration -> module-declaration
<Proposal.md · GitHub Semantics

Syntax and semantics for imports, as it already supports referencing submodules imported from C and Objective-C modules, remains unchanged:

// The outermost module is given explicitly
// by passing `-module-name=Foo` or exists implicitly, as today.
// module Foo {
public class A {}

module Bar {
  module Baz {
    public class C {}
  }

  public class B {}
}

let message = "Hello, Wisconsin!"
// } // End declarations added to module Foo.
To consume this interface:

// imports all of Foo, Foo.Bar, and Foo.Bar.Baz
import Foo.Bar.Baz

// imports Foo.A as A
import class Foo.A
// imports Foo.Bar.B as B
import class Foo.Bar.B
// imports Foo.Bar.Baz.C as C
import class Foo.Bar.Baz.C
A module declaration may only appear as a top-level entity or as a member of another module declaration. The following code is therefore invalid:

module Foo {
  class Bar {
    module Baz {} // error: module declaration cannot be nested inside type 'Bar'
  }
}
To extend an existing module declaration, simply reference its module name in an extension declaration.

// In module 'Foo'
module Bar {
  public class A {}

  module Baz {}
}

extension Bar {
  public struct B {}
}

extension Bar.Baz {
  public enum C { case D }
}
<Proposal.md · GitHub and Access Control

The semantics of some existing access control modifiers shall also be extended to support module declarations:

open and public declarations are exported by a module for consumption by clients of the module.
internal declarations scope over the entire module and any derived submodules.
By default, to preserve encapsulation of interfaces, modules are "sealed" and may only be "opened" by explicit named import. However, it is often desirable to export a module and a set of submodules or even modules from external dependencies along with a given interface. We propose the public keyword be used for this purpose:

// Defines top-level module "Foo"
//module Foo {
public import Foo.Bar.Baz
public import Foundation.Date
//}
Which then causes the following (sub)modules to be imported into scope along with Foo:

// imports Foo, Foo.Bar.Baz, and Foundation.Date
import Foo
To support existing Swift packages that cannot have opted into modules, and to preserve the scriptable nature of Swift, module declarations shall be optional. Any Swift program that does not declare at least one top-level module explicitly is considered part of an unnamed special "Global Module" with the same rules of access control as today. To give declarations in the Global Module an explicit module without using a module declaration, use the -module-name flag.

<Proposal.md · GitHub on Existing Code

This proposal is intentionally additive. There is no impact on existing code.

<Proposal.md · GitHub considered

<Proposal.md · GitHub Modules Everywhere

Declarations in the top-level of a program exist today in the top-level of the corresponding module. If desired, this module declaration could be required to be explicit like so:

module Foo {
  module Bar {
    module Baz {}
  }
}
However, we feel that imposing such a requirement not only complicates the outermost scope, it requires inserting needless extension Foo {} scopes in every file. It also violates the principle of progressive disclosure by forcing all new adoptees of Swift to learn what a module is without actually using the module system.

<Proposal.md · GitHub Extensions

Nested module extensions may be "expanded" as it were to the following:

module Foo {
  module Bar {}
}

extension Foo {
  extension Bar {}
}
However, this syntax is currently not enabled in general in Swift. This problem should be revisted in a future proposal.

<Proposal.md · GitHub (Source-Breaking Changes)

The system described above is intended to be entirely source and binary compatible. Nonetheless, in its design we feel we have obviated certain existing features and recommend their deprecation in future proposals:

fileprivate access can be recreated by creating a private "utility submodule" containing declarations of at least internal access.
@_exported, the private directive to re-export modules today, should be deprecated and removed.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

We avoid forcing users to organize code in such an opinionated manner just to please the compiler. Perhaps some submodules deserve a separate file, but I think that choice should not be forced by the language. I don’t have data on the popularity, but I personally very much dislike the similar restriction Java places on public classes and files.

Given that we want Swift to be a fantastic scripting language, I feel we ought not place artificial restrictions on code organization. Many scripts are a single file (for convenience) but may still benefit from the organization modules offer.

Best,
Jaden Geller

···

On Feb 20, 2017, at 10:51 PM, Jonathan Hull <jhull@gbis.com> wrote:

Part of it. My concern is that the brackets will push everything in the file to the right.

My question is: What do we get in return for that? (as opposed to a system which acted on the whole file)

Thanks,
Jon

On Feb 20, 2017, at 10:45 PM, Jaden Geller <jaden.geller@gmail.com <mailto:jaden.geller@gmail.com>> wrote:

Oh, you’re referring to the fact that a submodule under this proposal must be contained within braces? It would be purely additive to, in the future, annotate that the outer scope of a file is part of some given submodule.

Consider the following the following straw-man syntax that might be equivalent to `module Bar { func foo() { } }`:

module Bar follows // <- at top of file, indicating rest of file is submodule

func foo() { }

Does this address your question?

Thanks,
Jaden Geller

On Feb 20, 2017, at 10:39 PM, Jonathan Hull <jhull@gbis.com <mailto:jhull@gbis.com>> wrote:

On Feb 20, 2017, at 6:42 PM, Jaden Geller <jaden.geller@gmail.com <mailto:jaden.geller@gmail.com>> wrote:

Jon,

I think we might have miscommunicated. It is intended that outermost module is implicit; no `module` declaration is required to wrap every file. We tried to show this in the first code snippet.

What do you mean “covering only part of a file”?

I am assuming that the ModuleName { … } only affects things within the brackets. Thus it is possible for only part of a file to be within a module. What are the benefits of allowing this, and are they worth the added complexity?

Thanks,
Jon

Cheers,
Jaden Geller

On Feb 20, 2017, at 6:36 PM, Jonathan Hull via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

What is the rational for having modules covering only part of a file? Wouldn’t it be less clutter to have an annotation which worked for the whole file. At the very least it would be nice to have an option to spell it in a way that applies to the whole file. Otherwise, everything will be indented another level.

I would honestly love to see something which just maps modules to folders/groups for simplicity sake.

I haven’t thought about it too much yet, so I could easily be missing something obvious...

Thanks,
Jon

What I mean is that someone will eventually want to expose a member within a submodule that is a parent of the innermost submodule, but not make it public. They will want to say something like ‘private(ModuleName)’. Note, I am not proposing that syntax, just that it is an inevitable thing that people will ask for…

Thanks,
Jon

···

On Feb 20, 2017, at 10:51 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

On a side note, I do think that people will quickly want a way to reference a particular submodule boundary with access modifiers. It may not be part of this proposal, but it is somewhat inevitable. We should consider that as we consider this proposal…

Modules are not types, access control makes no sense here. APIs may be exported (or not) across module boundaries, but modules themselves are not arbitrary programming constructs nor do we consider them to carry semantic weight as in some other ML-likes. A public module is no different from a private module if you can import it. A private module is no different from a public one if you cannot.

It is how it interacts with our current access control. The scope based nature of this will conflict with the file based nature of the stuff left from Swift 2. I am convinced we need to overhaul it in one sweep to fit one metaphor or the other. The more we mix them, the harder it is to wrap your head around. We can’t keep rehashing it every 6 months, but that is what will happen so long as it stays in this confused state.

I think I would prefer overall trying swift’s file based system because I think we can completely close that with only 1-2 additional concepts (and it would stay conceptually simple). I would also be fine with switching to a purely type/scope based system with protected, etc…, but I feel like that will need more complexity to get it fully working (e.g. friends), and will end up being a more complex system overall.

Thanks,
Jon

···

On Feb 20, 2017, at 11:03 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

On Feb 21, 2017, at 2:01 AM, Jonathan Hull <jhull@gbis.com <mailto:jhull@gbis.com>> wrote:

On Feb 20, 2017, at 10:46 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

More generally, we need to move access control away as far away from filesystems as possible. One day, the compiler is going to get ported over to a platform with some bonkers rules that are going to turn around and bite us.

This is the key thing which I think is being debated right now. Swift 2 defined the file as a unit of compilation and based its elegant system of access modifiers on that. With Swift 3 we have a mix of type/scope based modifiers and file based modifiers… and it is super confusing to everyone because we have mixed our metaphors.

As much as I want modules, I am now convinced that this proposal will only make that situation worse. We need to pick one way (file based) or the other (scope based) and commit to it… but either way, it will require a larger overhaul to make the system consistent/usable/teachable again.

As an analogy, it is like some people are trying to play rock music during a classical music concert. Both are great independently, and different people may prefer one or the other… but trying to play them on top of one another just results in noise.

Is there anything specifically about this proposal that turns you off, or is it access control as it stands today?

Thanks,
Jon

It is how it interacts with our current access control. The scope based nature of this will conflict with the file based nature of the stuff left from Swift 2. I am convinced we need to overhaul it in one sweep to fit one metaphor or the other. The more we mix them, the harder it is to wrap your head around. We can’t keep rehashing it every 6 months, but that is what will happen so long as it stays in this confused state.

It has been my hope that a lightweight module system will remove the need for `private` *and* `fileprivate`. I doubt a clean "solution" to the access control story can emerge without it. FWIW.

···

On Feb 20, 2017, at 11:16 PM, Jonathan Hull via swift-evolution <swift-evolution@swift.org> wrote:

I think I would prefer overall trying swift’s file based system because I think we can completely close that with only 1-2 additional concepts (and it would stay conceptually simple). I would also be fine with switching to a purely type/scope based system with protected, etc…, but I feel like that will need more complexity to get it fully working (e.g. friends), and will end up being a more complex system overall.

Thanks,
Jon

On Feb 20, 2017, at 11:03 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

On Feb 21, 2017, at 2:01 AM, Jonathan Hull <jhull@gbis.com> wrote:

On Feb 20, 2017, at 10:46 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org> wrote:

More generally, we need to move access control away as far away from filesystems as possible. One day, the compiler is going to get ported over to a platform with some bonkers rules that are going to turn around and bite us.

This is the key thing which I think is being debated right now. Swift 2 defined the file as a unit of compilation and based its elegant system of access modifiers on that. With Swift 3 we have a mix of type/scope based modifiers and file based modifiers… and it is super confusing to everyone because we have mixed our metaphors.

As much as I want modules, I am now convinced that this proposal will only make that situation worse. We need to pick one way (file based) or the other (scope based) and commit to it… but either way, it will require a larger overhaul to make the system consistent/usable/teachable again.

As an analogy, it is like some people are trying to play rock music during a classical music concert. Both are great independently, and different people may prefer one or the other… but trying to play them on top of one another just results in noise.

Is there anything specifically about this proposal that turns you off, or is it access control as it stands today?

Thanks,
Jon

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Sounds distressingly anti-modular!

Really, these kinds of hypotheticals can be addressed when the need arises. For now it's a hypothetical fear about a hypothetical issue that isn't expressible and highly discouraged by this system - why produce an inverted dependency when you could simply rebalance the module by moving the declaration into a more appropriate scope? I can assure you, the proposal seeks compatibility with the existing access control modifiers, not to have to contort them to fit its own goals.

~Robert Widmann

2017/02/21 2:07、Jonathan Hull <jhull@gbis.com> のメッセージ:

···

On Feb 20, 2017, at 10:51 PM, Robert Widmann

~Robert Widmann

2017/02/21 2:16、Jonathan Hull <jhull@gbis.com> のメッセージ:

It is how it interacts with our current access control. The scope based nature of this will conflict with the file based nature of the stuff left from Swift 2. I am convinced we need to overhaul it in one sweep to fit one metaphor or the other. The more we mix them, the harder it is to wrap your head around. We can’t keep rehashing it every 6 months, but that is what will happen so long as it stays in this confused state.

How does this conflict? Moreover, what do you feel we are adding here that crosses the line?

This is not a rehash of existing concepts. We do not deprecate the access control scheme, and we don't extend it except to define how symbols can indicate they wish to cross/not cross the module boundary. This is genuinely a system to enable modularity at as little cost as we could suss out given inspiration from other programming languages.

tl;dr I don't want to confuse this with the fileprivate debate. It is possible to have physical access control and modularity, they're not mutually exclusive. And the existing access control scheme cannot express the same concepts a real module system can, so it's not enough to paint this as a syntactic/physical argument.

I think I would prefer overall trying swift’s file based system because I think we can completely close that with only 1-2 additional concepts (and it would stay conceptually simple). I would also be fine with switching to a purely type/scope based system with protected, etc…, but I feel like that will need more complexity to get it fully working (e.g. friends), and will end up being a more complex system overall.

The semantics we intend are all there in the proposal. You can see for yourself just how much this will complect the language versus a different proposal to redo access entirely.

···

Thanks,
Jon

On Feb 20, 2017, at 11:03 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

On Feb 21, 2017, at 2:01 AM, Jonathan Hull <jhull@gbis.com> wrote:

On Feb 20, 2017, at 10:46 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org> wrote:

More generally, we need to move access control away as far away from filesystems as possible. One day, the compiler is going to get ported over to a platform with some bonkers rules that are going to turn around and bite us.

This is the key thing which I think is being debated right now. Swift 2 defined the file as a unit of compilation and based its elegant system of access modifiers on that. With Swift 3 we have a mix of type/scope based modifiers and file based modifiers… and it is super confusing to everyone because we have mixed our metaphors.

As much as I want modules, I am now convinced that this proposal will only make that situation worse. We need to pick one way (file based) or the other (scope based) and commit to it… but either way, it will require a larger overhaul to make the system consistent/usable/teachable again.

As an analogy, it is like some people are trying to play rock music during a classical music concert. Both are great independently, and different people may prefer one or the other… but trying to play them on top of one another just results in noise.

Is there anything specifically about this proposal that turns you off, or is it access control as it stands today?

Thanks,
Jon

I really doubt it will. `private`/`fileprivate` works because you can also access `internal` at the same time.

What I mean by that is, think about code like this:

  // Foo.swift
  public class Foo {
    public init() { … }

    func doBar() -> Quux {
      return helper(in: randomRange())
    }

    private func helper(in range: Range<Int>) -> Quux {
      …
    }
  }

  // Bar.swift
  public class Bar {
    public static let shared = Bar()

    func baz(with foo: Foo) {
      let quux = foo.doBar()
      process(quux)
    }
    
    private func process(_ quux: Quux) {
      …
    }
  }

These classes have `public` APIs that are externally visible, `internal` APIs for communicating with each other, and `private` APIs for implementation details. Now try to reproduce the same design with submodules and `public`/`internal` only:

  public import MyMod.Foo
  public import MyMod.Bar

  module Foo {
    public class Foo {
      public init() { … }

      ??? func doBar() -> Quux {
        return helper(in: randomRange())
      }

      func helper(in range: Range<Int>) -> Quux {
        …
      }
    }
  }

  // Bar.swift
  module Bar {
    public class Bar {
      public static let shared = Bar()
      
      ??? func baz(with foo: Foo) {
        let quux = foo.doBar()
        process(quux)
      }
    
      func process(_ quux: Quux) {
        …
      }
    }
  }

The `doBar()` and `baz()` methods have to be either exposed to third parties or kept away from yourself. That's just not viable.

···

On Feb 21, 2017, at 1:28 AM, Daniel Duan via swift-evolution <swift-evolution@swift.org> wrote:

It has been my hope that a lightweight module system will remove the need for `private` *and* `fileprivate`.

--
Brent Royal-Gordon
Architechies

I'm not sure whether I'm misunderstanding or we're talking past each other.

Let me state this really simply. You have some code in a top-level module, `MyMod`:

  import MyMod.Submodule
  
  func foo() {
    bar()
  }

And you have some other code in a submodule:

  module Submodule {
    ??? func bar() {
      baz()
    }
  }

And then—perhaps in a separate file—you have some other code in an extension of the submodule:

  extension Submodule {
    ??? func baz() {
      …
    }
  }

What access modifiers do I put on `bar()` and `baz()` so that `MyMod` can access `bar()` but not `baz()`, and code outside `MyMod` can access neither `bar()` nor `baz()`?

···

On Feb 21, 2017, at 6:43 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

That is not what this proposal requires. A public API is ripe for re(export), but if the parent wishes to communicate with its children without exporting across the module boundary, see the definition of `internal`.

--
Brent Royal-Gordon
Architechies

That is not what this proposal requires. A public API is ripe for re(export), but if the parent wishes to communicate with its children without exporting across the module boundary, see the definition of `internal`.

I'm not sure whether I'm misunderstanding or we're talking past each other.

Let me state this really simply. You have some code in a top-level module, `MyMod`:

  import MyMod.Submodule
  
  func foo() {
    bar()
  }

And you have some other code in a submodule:

  module Submodule {
    ??? func bar() {
      baz()
    }
  }

And then—perhaps in a separate file—you have some other code in an extension of the submodule:

  extension Submodule {
    ??? func baz() {
      …
    }
  }

What access modifiers do I put on `bar()` and `baz()` so that `MyMod` can access `bar()` but not `baz()`, and code outside `MyMod` can access neither `bar()` nor `baz()`?

internal

open and public declarations are exported by a module for consumption by clients of the module.
internal declarations scope over the entire module and any derived submodules.
This way you can consume your own interface without it crossing the module boundary.

···

On Feb 21, 2017, at 9:49 PM, Brent Royal-Gordon <brent@architechies.com> wrote:

On Feb 21, 2017, at 6:43 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

--
Brent Royal-Gordon
Architechies

What is implied by the definition of `internal` in the proposal (which should, as this thread has shown, be stated explicitly) is that by “export” we mean “export to clients” across the outermost module-boundary. Any API that is not public is, by definition, not able to cross this outermost boundary. Internal API is allowed to cross internal boundaries if it is imported internally. Private and fileprivate API may not, as the name implies, cross file boundaries and so cannot be allowed to cross either boundary.

···

On Feb 21, 2017, at 9:49 PM, Brent Royal-Gordon <brent@architechies.com> wrote:

On Feb 21, 2017, at 6:43 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

That is not what this proposal requires. A public API is ripe for re(export), but if the parent wishes to communicate with its children without exporting across the module boundary, see the definition of `internal`.

I'm not sure whether I'm misunderstanding or we're talking past each other.

Let me state this really simply. You have some code in a top-level module, `MyMod`:

  import MyMod.Submodule
  
  func foo() {
    bar()
  }

And you have some other code in a submodule:

  module Submodule {
    ??? func bar() {
      baz()
    }
  }

And then—perhaps in a separate file—you have some other code in an extension of the submodule:

  extension Submodule {
    ??? func baz() {
      …
    }
  }

What access modifiers do I put on `bar()` and `baz()` so that `MyMod` can access `bar()` but not `baz()`, and code outside `MyMod` can access neither `bar()` nor `baz()`?

--
Brent Royal-Gordon
Architechies

This was syntax I initially considered, but Jaden swayed me with the idea that we already have an `extension` keyword so we may as well use it to match the rest of the language. Repeating the `module` declaration also lends itself more to C++-style namespaces, which these are not. I’m not saying it’s out of the question, far from it. It is an alternative we will keep in mind.

~Robert Widmann

···

On Feb 21, 2017, at 1:53 AM, Psycho Hedgehog via swift-evolution <swift-evolution@swift.org> wrote:

The one thing i don't understand is the point of using extensions to extend modules across multiple files, why not just allow declaring the submodule in multiple files?

Le 20 févr. 2017 à 22:48, Jaden Geller via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> a écrit :

Oh, I see. You’re questioning the motivation of having scope-granularity submodules at all! My misunderstanding.

I actually hadn’t considered this as added complexity. In my mind, a scoped module declaration seems more Swifty than a file module declaration. It builds on the existing syntax in Swift for defining other sorts of scopes, e.g. types.

Cheers,
Jaden Geller

On Feb 20, 2017, at 10:41 PM, Jonathan Hull via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

I think my question is: Why do we want to allow submodules that are smaller than a file? What does that give us to offset the added complexity?

Thanks,
Jon

On Feb 20, 2017, at 6:44 PM, Robert Widmann <devteam.codafi@gmail.com <mailto:devteam.codafi@gmail.com>> wrote:

On Feb 20, 2017, at 9:36 PM, Jonathan Hull <jhull@gbis.com <mailto:jhull@gbis.com>> wrote:

What is the rational for having modules covering only part of a file? Wouldn’t it be less clutter to have an annotation which worked for the whole file. At the very least it would be nice to have an option to spell it in a way that applies to the whole file. Otherwise, everything will be indented another level.

That is a valid spelling (Rust, IIRC, allows that spelling), but one that is easy to miss sitting in a file and makes it confusing to introduce submodules. If you include the annotation then define a submodule later down in the file, suddenly you have to remember whether you annotated the file or whether the submodule you’ve just written is going into the top-level module. See:

// -module-name=Foo
// module Foo {
module Bar; // Shorthand for “This file defines Foo.Bar”

/* Code */

// This defines “Foo.Bar.Baz”, but would you know that if it appeared below the fold?
module Baz {}
//}

If anything, this can be added later if evolution converges on it.

I would honestly love to see something which just maps modules to folders/groups for simplicity sake.

There is nothing about this scheme that prevents you from organizing your code this way. However, modulo that particular method of organization, you don’t really gain much as a user of the language by imposing this restriction.

I haven’t thought about it too much yet, so I could easily be missing something obvious...

Thanks,
Jon

On Feb 20, 2017, at 5:56 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Good Evening All,

Jaden Geller and I have been considering a (sub)module system for Swift that would complement the existing language but also provide sorely needed modularity. A draft of the proposal is attached to this email, but it can also be read as a gist <https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a&gt; if you desire.

Cheers,

~Robert Widmann

Modular Swift

Proposal: SE-NNNN <https://gist.github.com/CodaFi/NNNN-filename.md&gt;
Authors: Robert Widmann <https://github.com/codafi&gt;, Jaden Geller <https://github.com/JadenGeller&gt;
Review Manager: TBD
Status: Awaiting review
<Proposal.md · GitHub

Almost every major programming language supports some form of modular programming through constructs like (sub)modules, packages, or interfaces. Swift, though it provides top-level modules to organize code under, does not provide a complete implementation of any of these concepts, which has led instead to the proliferation of access control levels. This has not proven an effective way to decompose programs into manageable parts, and exposes the need for a real system of modules to solve this modularity problem once and for all.

Separation of code into distinct islands of functionality should be a first-class construct in the language, not dependent on external files and tools or filesystems. To that end, we propose the introduction of a lightweight module system for Swift.

Swift-evolution thread <applewebdata://958D4EFC-6E24-4386-84D4-120E1C2769EC>
<Proposal.md · GitHub

Swift has reached a point in its evolution where rich libraries and large projects that take on many dependencies have matured significantly. To accomodate the information-hiding and semantics-signalling needs of these users at the time, Swift began its access control story with just three access modifiers: public, private, and internal then grew fileprivate and open as the need to express locality of implementation and "subclassability" arose respectively. In doing so, Swift's access control scheme has become anti-modular.

<Proposal.md · GitHub solution

We propose the introduction of a lightweight module system for Swift. More than simply namspaces, a module declaration interacts with Swift's access control to provide an API boundary that allows better control over an interface's design.

<Proposal.md · GitHub design

<Proposal.md · GitHub

A module is a named region that introduces a lexical scope into which declarations may be nested. The name of the module can be used to access these member declarations. A module, like other aggregate structures in Swift, may be extended with new declarations over one or more translation units (files).

We propose a new declaration kind, module-decl be added to the language. A proposed grammar using the new modulekeyword is given below:

GRAMMAR OF A MODULE DECLARATION

module-declaration -> `module` module-identifier module-body
module-name -> identifier
module-body -> { module-members(opt) }
module-members -> module-member module-members(opt)
module-member -> declaration | compiler-control-statement
GRAMMAR OF A DECLARATION

+ declaration -> module-declaration
<Proposal.md · GitHub Semantics

Syntax and semantics for imports, as it already supports referencing submodules imported from C and Objective-C modules, remains unchanged:

// The outermost module is given explicitly
// by passing `-module-name=Foo` or exists implicitly, as today.
// module Foo {
public class A {}

module Bar {
  module Baz {
    public class C {}
  }

  public class B {}
}

let message = "Hello, Wisconsin!"
// } // End declarations added to module Foo.
To consume this interface:

// imports all of Foo, Foo.Bar, and Foo.Bar.Baz
import Foo.Bar.Baz

// imports Foo.A as A
import class Foo.A
// imports Foo.Bar.B as B
import class Foo.Bar.B
// imports Foo.Bar.Baz.C as C
import class Foo.Bar.Baz.C
A module declaration may only appear as a top-level entity or as a member of another module declaration. The following code is therefore invalid:

module Foo {
  class Bar {
    module Baz {} // error: module declaration cannot be nested inside type 'Bar'
  }
}
To extend an existing module declaration, simply reference its module name in an extension declaration.

// In module 'Foo'
module Bar {
  public class A {}

  module Baz {}
}

extension Bar {
  public struct B {}
}

extension Bar.Baz {
  public enum C { case D }
}
<Proposal.md · GitHub and Access Control

The semantics of some existing access control modifiers shall also be extended to support module declarations:

open and public declarations are exported by a module for consumption by clients of the module.
internal declarations scope over the entire module and any derived submodules.
By default, to preserve encapsulation of interfaces, modules are "sealed" and may only be "opened" by explicit named import. However, it is often desirable to export a module and a set of submodules or even modules from external dependencies along with a given interface. We propose the public keyword be used for this purpose:

// Defines top-level module "Foo"
//module Foo {
public import Foo.Bar.Baz
public import Foundation.Date
//}
Which then causes the following (sub)modules to be imported into scope along with Foo:

// imports Foo, Foo.Bar.Baz, and Foundation.Date
import Foo
To support existing Swift packages that cannot have opted into modules, and to preserve the scriptable nature of Swift, module declarations shall be optional. Any Swift program that does not declare at least one top-level module explicitly is considered part of an unnamed special "Global Module" with the same rules of access control as today. To give declarations in the Global Module an explicit module without using a module declaration, use the -module-name flag.

<Proposal.md · GitHub on Existing Code

This proposal is intentionally additive. There is no impact on existing code.

<Proposal.md · GitHub considered

<Proposal.md · GitHub Modules Everywhere

Declarations in the top-level of a program exist today in the top-level of the corresponding module. If desired, this module declaration could be required to be explicit like so:

module Foo {
  module Bar {
    module Baz {}
  }
}
However, we feel that imposing such a requirement not only complicates the outermost scope, it requires inserting needless extension Foo {} scopes in every file. It also violates the principle of progressive disclosure by forcing all new adoptees of Swift to learn what a module is without actually using the module system.

<Proposal.md · GitHub Extensions

Nested module extensions may be "expanded" as it were to the following:

module Foo {
  module Bar {}
}

extension Foo {
  extension Bar {}
}
However, this syntax is currently not enabled in general in Swift. This problem should be revisted in a future proposal.

<Proposal.md · GitHub (Source-Breaking Changes)

The system described above is intended to be entirely source and binary compatible. Nonetheless, in its design we feel we have obviated certain existing features and recommend their deprecation in future proposals:

fileprivate access can be recreated by creating a private "utility submodule" containing declarations of at least internal access.
@_exported, the private directive to re-export modules today, should be deprecated and removed.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

One pattern that lightweight module enables is a lot more free functions in place of methods. Because the functions would be encapsulated by a module and lose the polluting effect.

···

On Feb 21, 2017, at 4:46 AM, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> wrote:

On Feb 21, 2017, at 1:28 AM, Daniel Duan via swift-evolution <swift-evolution@swift.org> wrote:

It has been my hope that a lightweight module system will remove the need for `private` *and* `fileprivate`.

I really doubt it will. `private`/`fileprivate` works because you can also access `internal` at the same time.

What I mean by that is, think about code like this:

   // Foo.swift
   public class Foo {
       public init() { … }

       func doBar() -> Quux {
           return helper(in: randomRange())
       }

       private func helper(in range: Range<Int>) -> Quux {
           …
       }
   }

   // Bar.swift
   public class Bar {
       public static let shared = Bar()

       func baz(with foo: Foo) {
           let quux = foo.doBar()
           process(quux)
       }
       
       private func process(_ quux: Quux) {
           …
       }
   }

These classes have `public` APIs that are externally visible, `internal` APIs for communicating with each other, and `private` APIs for implementation details. Now try to reproduce the same design with submodules and `public`/`internal` only:

   public import MyMod.Foo
   public import MyMod.Bar

   module Foo {
       public class Foo {
           public init() { … }

           ??? func doBar() -> Quux {
               return helper(in: randomRange())
           }

           func helper(in range: Range<Int>) -> Quux {
               …
           }
       }
   }

   // Bar.swift
   module Bar {
       public class Bar {
           public static let shared = Bar()
           
           ??? func baz(with foo: Foo) {
               let quux = foo.doBar()
               process(quux)
           }
       
           func process(_ quux: Quux) {
               …
           }
       }
   }

The `doBar()` and `baz()` methods have to be either exposed to third parties or kept away from yourself. That's just not viable.

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

We avoid forcing users to organize code in such an opinionated manner just to please the compiler. Perhaps some submodules deserve a separate file, but I think that choice should not be forced by the language. I don’t have data on the popularity, but I personally very much dislike the similar restriction Java places on public classes and files.

On the other hand, having such standardisation means that you know exactly where a type is based on its module membership. One of the outcomes of Scala's move away from enforcing a relationship between files and their module structure means that it is almost impossible to find where a type is defined using the filesystem, and you resort to using 'git grep' or other tools to find out where the implementation is located.

Given that we want Swift to be a fantastic scripting language, I feel we ought not place artificial restrictions on code organization. Many scripts are a single file (for convenience) but may still benefit from the organization modules offer.

Is there a reason why a script should need to be in any module, as opposed to a top-level/unnamed module?

Alex

···

On 21 Feb 2017, at 07:00, Jaden Geller via swift-evolution <swift-evolution@swift.org> wrote: