[Proposal][Discussion] Modular Swift


(Robert Widmann) #1

Good Evening All,

Jaden Geller and I have been considering a (sub)module system for Swift that would complement the existing language but also provide sorely needed modularity. A draft of the proposal is attached to this email, but it can also be read as a gist <https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a> if you desire.

Cheers,

~Robert Widmann

Modular Swift

Proposal: SE-NNNN <https://gist.github.com/CodaFi/NNNN-filename.md>
Authors: Robert Widmann <https://github.com/codafi>, Jaden Geller <https://github.com/JadenGeller>
Review Manager: TBD
Status: Awaiting review
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#introduction>Introduction

Almost every major programming language supports some form of modular programming through constructs like (sub)modules, packages, or interfaces. Swift, though it provides top-level modules to organize code under, does not provide a complete implementation of any of these concepts, which has led instead to the proliferation of access control levels. This has not proven an effective way to decompose programs into manageable parts, and exposes the need for a real system of modules to solve this modularity problem once and for all.

Separation of code into distinct islands of functionality should be a first-class construct in the language, not dependent on external files and tools or filesystems. To that end, we propose the introduction of a lightweight module system for Swift.

Swift-evolution thread <applewebdata://3F1C8A2E-18D5-44D8-8789-9D20A3226324>
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#motivation>Motivation

Swift has reached a point in its evolution where rich libraries and large projects that take on many dependencies have matured significantly. To accomodate the information-hiding and semantics-signalling needs of these users at the time, Swift began its access control story with just three access modifiers: public, private, and internal then grew fileprivate and open as the need to express locality of implementation and "subclassability" arose respectively. In doing so, Swift's access control scheme has become anti-modular.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#proposed-solution>Proposed solution

We propose the introduction of a lightweight module system for Swift. More than simply namspaces, a module declaration interacts with Swift's access control to provide an API boundary that allows better control over an interface's design.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#detailed-design>Detailed design

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#syntax>Syntax

A module is a named region that introduces a lexical scope into which declarations may be nested. The name of the module can be used to access these member declarations. A module, like other aggregate structures in Swift, may be extended with new declarations over one or more translation units (files).

We propose a new declaration kind, module-decl be added to the language. A proposed grammar using the new modulekeyword is given below:

GRAMMAR OF A MODULE DECLARATION

module-declaration -> `module` module-identifier module-body
module-name -> identifier
module-body -> { module-members(opt) }
module-members -> module-member module-members(opt)
module-member -> declaration | compiler-control-statement
GRAMMAR OF A DECLARATION

+ declaration -> module-declaration
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#general-semantics>General Semantics

Syntax and semantics for imports, as it already supports referencing submodules imported from C and Objective-C modules, remains unchanged:

// The outermost module is given explicitly
// by passing `-module-name=Foo` or exists implicitly, as today.
// module Foo {
public class A {}

module Bar {
  module Baz {
    public class C {}
  }

  public class B {}
}

let message = "Hello, Wisconsin!"
// } // End declarations added to module Foo.
To consume this interface:

// imports all of Foo, Foo.Bar, and Foo.Bar.Baz
import Foo.Bar.Baz

// imports Foo.A as A
import class Foo.A
// imports Foo.Bar.B as B
import class Foo.Bar.B
// imports Foo.Bar.Baz.C as C
import class Foo.Bar.Baz.C
A module declaration may only appear as a top-level entity or as a member of another module declaration. The following code is therefore invalid:

module Foo {
  class Bar {
    module Baz {} // error: module declaration cannot be nested inside type 'Bar'
  }
}
To extend an existing module declaration, simply reference its module name in an extension declaration.

// In module 'Foo'
module Bar {
  public class A {}

  module Baz {}
}

extension Bar {
  public struct B {}
}

extension Bar.Baz {
  public enum C { case D }
}
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#modules-and-access-control>Modules and Access Control

The semantics of some existing access control modifiers shall also be extended to support module declarations:

open and public declarations are exported by a module for consumption by clients of the module.
internal declarations scope over the entire module and any derived submodules.
By default, to preserve encapsulation of interfaces, modules are "sealed" and may only be "opened" by explicit named import. However, it is often desirable to export a module and a set of submodules or even modules from external dependencies along with a given interface. We propose the public keyword be used for this purpose:

// Defines top-level module "Foo"
//module Foo {
public import Foo.Bar.Baz
public import Foundation.Date
//}
Which then causes the following (sub)modules to be imported into scope along with Foo:

// imports Foo, Foo.Bar.Baz, and Foundation.Date
import Foo
To support existing Swift packages that cannot have opted into modules, and to preserve the scriptable nature of Swift, module declarations shall be optional. Any Swift program that does not declare at least one top-level module explicitly is considered part of an unnamed special "Global Module" with the same rules of access control as today. To give declarations in the Global Module an explicit module without using a module declaration, use the -module-name flag.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#impact-on-existing-code>Impact on Existing Code

This proposal is intentionally additive. There is no impact on existing code.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#alternatives-considered>Alternatives considered

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#explicit-modules-everywhere>Explicit Modules Everywhere

Declarations in the top-level of a program exist today in the top-level of the corresponding module. If desired, this module declaration could be required to be explicit like so:

module Foo {
  module Bar {
    module Baz {}
  }
}
However, we feel that imposing such a requirement not only complicates the outermost scope, it requires inserting needless extension Foo {} scopes in every file. It also violates the principle of progressive disclosure by forcing all new adoptees of Swift to learn what a module is without actually using the module system.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#nested-extensions>Nested Extensions

Nested module extensions may be "expanded" as it were to the following:

module Foo {
  module Bar {}
}

extension Foo {
  extension Bar {}
}
However, this syntax is currently not enabled in general in Swift. This problem should be revisted in a future proposal.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#deprecations-source-breaking-changes>Deprecations (Source-Breaking Changes)

The system described above is intended to be entirely source and binary compatible. Nonetheless, in its design we feel we have obviated certain existing features and recommend their deprecation in future proposals:

fileprivate access can be recreated by creating a private "utility submodule" containing declarations of at least internal access.
@_exported, the private directive to re-export modules today, should be deprecated and removed.


The One Stop Shop for Previous Submodule Pitches
(Jon Hull) #2

What is the rational for having modules covering only part of a file? Wouldn’t it be less clutter to have an annotation which worked for the whole file. At the very least it would be nice to have an option to spell it in a way that applies to the whole file. Otherwise, everything will be indented another level.

I would honestly love to see something which just maps modules to folders/groups for simplicity sake.

I haven’t thought about it too much yet, so I could easily be missing something obvious...

Thanks,
Jon

···

On Feb 20, 2017, at 5:56 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org> wrote:

Good Evening All,

Jaden Geller and I have been considering a (sub)module system for Swift that would complement the existing language but also provide sorely needed modularity. A draft of the proposal is attached to this email, but it can also be read as a gist <https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a> if you desire.

Cheers,

~Robert Widmann

Modular Swift

Proposal: SE-NNNN <https://gist.github.com/CodaFi/NNNN-filename.md>
Authors: Robert Widmann <https://github.com/codafi>, Jaden Geller <https://github.com/JadenGeller>
Review Manager: TBD
Status: Awaiting review
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#introduction>Introduction

Almost every major programming language supports some form of modular programming through constructs like (sub)modules, packages, or interfaces. Swift, though it provides top-level modules to organize code under, does not provide a complete implementation of any of these concepts, which has led instead to the proliferation of access control levels. This has not proven an effective way to decompose programs into manageable parts, and exposes the need for a real system of modules to solve this modularity problem once and for all.

Separation of code into distinct islands of functionality should be a first-class construct in the language, not dependent on external files and tools or filesystems. To that end, we propose the introduction of a lightweight module system for Swift.

Swift-evolution thread <applewebdata://9F5918B7-3058-4707-9B07-BBBADDDE00AE>
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#motivation>Motivation

Swift has reached a point in its evolution where rich libraries and large projects that take on many dependencies have matured significantly. To accomodate the information-hiding and semantics-signalling needs of these users at the time, Swift began its access control story with just three access modifiers: public, private, and internal then grew fileprivate and open as the need to express locality of implementation and "subclassability" arose respectively. In doing so, Swift's access control scheme has become anti-modular.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#proposed-solution>Proposed solution

We propose the introduction of a lightweight module system for Swift. More than simply namspaces, a module declaration interacts with Swift's access control to provide an API boundary that allows better control over an interface's design.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#detailed-design>Detailed design

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#syntax>Syntax

A module is a named region that introduces a lexical scope into which declarations may be nested. The name of the module can be used to access these member declarations. A module, like other aggregate structures in Swift, may be extended with new declarations over one or more translation units (files).

We propose a new declaration kind, module-decl be added to the language. A proposed grammar using the new modulekeyword is given below:

GRAMMAR OF A MODULE DECLARATION

module-declaration -> `module` module-identifier module-body
module-name -> identifier
module-body -> { module-members(opt) }
module-members -> module-member module-members(opt)
module-member -> declaration | compiler-control-statement
GRAMMAR OF A DECLARATION

+ declaration -> module-declaration
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#general-semantics>General Semantics

Syntax and semantics for imports, as it already supports referencing submodules imported from C and Objective-C modules, remains unchanged:

// The outermost module is given explicitly
// by passing `-module-name=Foo` or exists implicitly, as today.
// module Foo {
public class A {}

module Bar {
  module Baz {
    public class C {}
  }

  public class B {}
}

let message = "Hello, Wisconsin!"
// } // End declarations added to module Foo.
To consume this interface:

// imports all of Foo, Foo.Bar, and Foo.Bar.Baz
import Foo.Bar.Baz

// imports Foo.A as A
import class Foo.A
// imports Foo.Bar.B as B
import class Foo.Bar.B
// imports Foo.Bar.Baz.C as C
import class Foo.Bar.Baz.C
A module declaration may only appear as a top-level entity or as a member of another module declaration. The following code is therefore invalid:

module Foo {
  class Bar {
    module Baz {} // error: module declaration cannot be nested inside type 'Bar'
  }
}
To extend an existing module declaration, simply reference its module name in an extension declaration.

// In module 'Foo'
module Bar {
  public class A {}

  module Baz {}
}

extension Bar {
  public struct B {}
}

extension Bar.Baz {
  public enum C { case D }
}
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#modules-and-access-control>Modules and Access Control

The semantics of some existing access control modifiers shall also be extended to support module declarations:

open and public declarations are exported by a module for consumption by clients of the module.
internal declarations scope over the entire module and any derived submodules.
By default, to preserve encapsulation of interfaces, modules are "sealed" and may only be "opened" by explicit named import. However, it is often desirable to export a module and a set of submodules or even modules from external dependencies along with a given interface. We propose the public keyword be used for this purpose:

// Defines top-level module "Foo"
//module Foo {
public import Foo.Bar.Baz
public import Foundation.Date
//}
Which then causes the following (sub)modules to be imported into scope along with Foo:

// imports Foo, Foo.Bar.Baz, and Foundation.Date
import Foo
To support existing Swift packages that cannot have opted into modules, and to preserve the scriptable nature of Swift, module declarations shall be optional. Any Swift program that does not declare at least one top-level module explicitly is considered part of an unnamed special "Global Module" with the same rules of access control as today. To give declarations in the Global Module an explicit module without using a module declaration, use the -module-name flag.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#impact-on-existing-code>Impact on Existing Code

This proposal is intentionally additive. There is no impact on existing code.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#alternatives-considered>Alternatives considered

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#explicit-modules-everywhere>Explicit Modules Everywhere

Declarations in the top-level of a program exist today in the top-level of the corresponding module. If desired, this module declaration could be required to be explicit like so:

module Foo {
  module Bar {
    module Baz {}
  }
}
However, we feel that imposing such a requirement not only complicates the outermost scope, it requires inserting needless extension Foo {} scopes in every file. It also violates the principle of progressive disclosure by forcing all new adoptees of Swift to learn what a module is without actually using the module system.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#nested-extensions>Nested Extensions

Nested module extensions may be "expanded" as it were to the following:

module Foo {
  module Bar {}
}

extension Foo {
  extension Bar {}
}
However, this syntax is currently not enabled in general in Swift. This problem should be revisted in a future proposal.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#deprecations-source-breaking-changes>Deprecations (Source-Breaking Changes)

The system described above is intended to be entirely source and binary compatible. Nonetheless, in its design we feel we have obviated certain existing features and recommend their deprecation in future proposals:

fileprivate access can be recreated by creating a private "utility submodule" containing declarations of at least internal access.
@_exported, the private directive to re-export modules today, should be deprecated and removed.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Jaden Geller) #3

Jon,

I think we might have miscommunicated. It is intended that outermost module is implicit; no `module` declaration is required to wrap every file. We tried to show this in the first code snippet.

What do you mean ‚Äúcovering only part of a file‚ÄĚ?

Cheers,
Jaden Geller

···

On Feb 20, 2017, at 6:36 PM, Jonathan Hull via swift-evolution <swift-evolution@swift.org> wrote:

What is the rational for having modules covering only part of a file? Wouldn’t it be less clutter to have an annotation which worked for the whole file. At the very least it would be nice to have an option to spell it in a way that applies to the whole file. Otherwise, everything will be indented another level.

I would honestly love to see something which just maps modules to folders/groups for simplicity sake.

I haven’t thought about it too much yet, so I could easily be missing something obvious...

Thanks,
Jon

On Feb 20, 2017, at 5:56 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Good Evening All,

Jaden Geller and I have been considering a (sub)module system for Swift that would complement the existing language but also provide sorely needed modularity. A draft of the proposal is attached to this email, but it can also be read as a gist <https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a> if you desire.

Cheers,

~Robert Widmann

Modular Swift

Proposal: SE-NNNN <https://gist.github.com/CodaFi/NNNN-filename.md>
Authors: Robert Widmann <https://github.com/codafi>, Jaden Geller <https://github.com/JadenGeller>
Review Manager: TBD
Status: Awaiting review
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#introduction>Introduction

Almost every major programming language supports some form of modular programming through constructs like (sub)modules, packages, or interfaces. Swift, though it provides top-level modules to organize code under, does not provide a complete implementation of any of these concepts, which has led instead to the proliferation of access control levels. This has not proven an effective way to decompose programs into manageable parts, and exposes the need for a real system of modules to solve this modularity problem once and for all.

Separation of code into distinct islands of functionality should be a first-class construct in the language, not dependent on external files and tools or filesystems. To that end, we propose the introduction of a lightweight module system for Swift.

Swift-evolution thread <applewebdata://52AC29C3-8E1C-4564-B33A-C68D57885F31>
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#motivation>Motivation

Swift has reached a point in its evolution where rich libraries and large projects that take on many dependencies have matured significantly. To accomodate the information-hiding and semantics-signalling needs of these users at the time, Swift began its access control story with just three access modifiers: public, private, and internal then grew fileprivate and open as the need to express locality of implementation and "subclassability" arose respectively. In doing so, Swift's access control scheme has become anti-modular.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#proposed-solution>Proposed solution

We propose the introduction of a lightweight module system for Swift. More than simply namspaces, a module declaration interacts with Swift's access control to provide an API boundary that allows better control over an interface's design.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#detailed-design>Detailed design

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#syntax>Syntax

A module is a named region that introduces a lexical scope into which declarations may be nested. The name of the module can be used to access these member declarations. A module, like other aggregate structures in Swift, may be extended with new declarations over one or more translation units (files).

We propose a new declaration kind, module-decl be added to the language. A proposed grammar using the new modulekeyword is given below:

GRAMMAR OF A MODULE DECLARATION

module-declaration -> `module` module-identifier module-body
module-name -> identifier
module-body -> { module-members(opt) }
module-members -> module-member module-members(opt)
module-member -> declaration | compiler-control-statement
GRAMMAR OF A DECLARATION

+ declaration -> module-declaration
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#general-semantics>General Semantics

Syntax and semantics for imports, as it already supports referencing submodules imported from C and Objective-C modules, remains unchanged:

// The outermost module is given explicitly
// by passing `-module-name=Foo` or exists implicitly, as today.
// module Foo {
public class A {}

module Bar {
  module Baz {
    public class C {}
  }

  public class B {}
}

let message = "Hello, Wisconsin!"
// } // End declarations added to module Foo.
To consume this interface:

// imports all of Foo, Foo.Bar, and Foo.Bar.Baz
import Foo.Bar.Baz

// imports Foo.A as A
import class Foo.A
// imports Foo.Bar.B as B
import class Foo.Bar.B
// imports Foo.Bar.Baz.C as C
import class Foo.Bar.Baz.C
A module declaration may only appear as a top-level entity or as a member of another module declaration. The following code is therefore invalid:

module Foo {
  class Bar {
    module Baz {} // error: module declaration cannot be nested inside type 'Bar'
  }
}
To extend an existing module declaration, simply reference its module name in an extension declaration.

// In module 'Foo'
module Bar {
  public class A {}

  module Baz {}
}

extension Bar {
  public struct B {}
}

extension Bar.Baz {
  public enum C { case D }
}
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#modules-and-access-control>Modules and Access Control

The semantics of some existing access control modifiers shall also be extended to support module declarations:

open and public declarations are exported by a module for consumption by clients of the module.
internal declarations scope over the entire module and any derived submodules.
By default, to preserve encapsulation of interfaces, modules are "sealed" and may only be "opened" by explicit named import. However, it is often desirable to export a module and a set of submodules or even modules from external dependencies along with a given interface. We propose the public keyword be used for this purpose:

// Defines top-level module "Foo"
//module Foo {
public import Foo.Bar.Baz
public import Foundation.Date
//}
Which then causes the following (sub)modules to be imported into scope along with Foo:

// imports Foo, Foo.Bar.Baz, and Foundation.Date
import Foo
To support existing Swift packages that cannot have opted into modules, and to preserve the scriptable nature of Swift, module declarations shall be optional. Any Swift program that does not declare at least one top-level module explicitly is considered part of an unnamed special "Global Module" with the same rules of access control as today. To give declarations in the Global Module an explicit module without using a module declaration, use the -module-name flag.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#impact-on-existing-code>Impact on Existing Code

This proposal is intentionally additive. There is no impact on existing code.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#alternatives-considered>Alternatives considered

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#explicit-modules-everywhere>Explicit Modules Everywhere

Declarations in the top-level of a program exist today in the top-level of the corresponding module. If desired, this module declaration could be required to be explicit like so:

module Foo {
  module Bar {
    module Baz {}
  }
}
However, we feel that imposing such a requirement not only complicates the outermost scope, it requires inserting needless extension Foo {} scopes in every file. It also violates the principle of progressive disclosure by forcing all new adoptees of Swift to learn what a module is without actually using the module system.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#nested-extensions>Nested Extensions

Nested module extensions may be "expanded" as it were to the following:

module Foo {
  module Bar {}
}

extension Foo {
  extension Bar {}
}
However, this syntax is currently not enabled in general in Swift. This problem should be revisted in a future proposal.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#deprecations-source-breaking-changes>Deprecations (Source-Breaking Changes)

The system described above is intended to be entirely source and binary compatible. Nonetheless, in its design we feel we have obviated certain existing features and recommend their deprecation in future proposals:

fileprivate access can be recreated by creating a private "utility submodule" containing declarations of at least internal access.
@_exported, the private directive to re-export modules today, should be deprecated and removed.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Jaden Geller) #4

To further clarify‚ÄĒI think it would be accurate to say that this proposal introduces 2 things:

1) submodules, without changing the existing module system
2) `public` keyword for re-exporting modules

It is both unnecessary and not allowed to name the outermost module in the file. The `module` keyword simply introduces submodules. Notice that this works well with the notion of progress disclosure.

···

On Feb 20, 2017, at 6:42 PM, Jaden Geller <jaden.geller@gmail.com> wrote:

Jon,

I think we might have miscommunicated. It is intended that outermost module is implicit; no `module` declaration is required to wrap every file. We tried to show this in the first code snippet.

What do you mean ‚Äúcovering only part of a file‚ÄĚ?

Cheers,
Jaden Geller

On Feb 20, 2017, at 6:36 PM, Jonathan Hull via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

What is the rational for having modules covering only part of a file? Wouldn’t it be less clutter to have an annotation which worked for the whole file. At the very least it would be nice to have an option to spell it in a way that applies to the whole file. Otherwise, everything will be indented another level.

I would honestly love to see something which just maps modules to folders/groups for simplicity sake.

I haven’t thought about it too much yet, so I could easily be missing something obvious...

Thanks,
Jon

On Feb 20, 2017, at 5:56 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Good Evening All,

Jaden Geller and I have been considering a (sub)module system for Swift that would complement the existing language but also provide sorely needed modularity. A draft of the proposal is attached to this email, but it can also be read as a gist <https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a> if you desire.

Cheers,

~Robert Widmann

Modular Swift

Proposal: SE-NNNN <https://gist.github.com/CodaFi/NNNN-filename.md>
Authors: Robert Widmann <https://github.com/codafi>, Jaden Geller <https://github.com/JadenGeller>
Review Manager: TBD
Status: Awaiting review
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#introduction>Introduction

Almost every major programming language supports some form of modular programming through constructs like (sub)modules, packages, or interfaces. Swift, though it provides top-level modules to organize code under, does not provide a complete implementation of any of these concepts, which has led instead to the proliferation of access control levels. This has not proven an effective way to decompose programs into manageable parts, and exposes the need for a real system of modules to solve this modularity problem once and for all.

Separation of code into distinct islands of functionality should be a first-class construct in the language, not dependent on external files and tools or filesystems. To that end, we propose the introduction of a lightweight module system for Swift.

Swift-evolution thread <applewebdata://F5974A32-2DD6-4262-BB95-DC6E8EB054B6>
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#motivation>Motivation

Swift has reached a point in its evolution where rich libraries and large projects that take on many dependencies have matured significantly. To accomodate the information-hiding and semantics-signalling needs of these users at the time, Swift began its access control story with just three access modifiers: public, private, and internal then grew fileprivate and open as the need to express locality of implementation and "subclassability" arose respectively. In doing so, Swift's access control scheme has become anti-modular.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#proposed-solution>Proposed solution

We propose the introduction of a lightweight module system for Swift. More than simply namspaces, a module declaration interacts with Swift's access control to provide an API boundary that allows better control over an interface's design.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#detailed-design>Detailed design

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#syntax>Syntax

A module is a named region that introduces a lexical scope into which declarations may be nested. The name of the module can be used to access these member declarations. A module, like other aggregate structures in Swift, may be extended with new declarations over one or more translation units (files).

We propose a new declaration kind, module-decl be added to the language. A proposed grammar using the new modulekeyword is given below:

GRAMMAR OF A MODULE DECLARATION

module-declaration -> `module` module-identifier module-body
module-name -> identifier
module-body -> { module-members(opt) }
module-members -> module-member module-members(opt)
module-member -> declaration | compiler-control-statement
GRAMMAR OF A DECLARATION

+ declaration -> module-declaration
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#general-semantics>General Semantics

Syntax and semantics for imports, as it already supports referencing submodules imported from C and Objective-C modules, remains unchanged:

// The outermost module is given explicitly
// by passing `-module-name=Foo` or exists implicitly, as today.
// module Foo {
public class A {}

module Bar {
  module Baz {
    public class C {}
  }

  public class B {}
}

let message = "Hello, Wisconsin!"
// } // End declarations added to module Foo.
To consume this interface:

// imports all of Foo, Foo.Bar, and Foo.Bar.Baz
import Foo.Bar.Baz

// imports Foo.A as A
import class Foo.A
// imports Foo.Bar.B as B
import class Foo.Bar.B
// imports Foo.Bar.Baz.C as C
import class Foo.Bar.Baz.C
A module declaration may only appear as a top-level entity or as a member of another module declaration. The following code is therefore invalid:

module Foo {
  class Bar {
    module Baz {} // error: module declaration cannot be nested inside type 'Bar'
  }
}
To extend an existing module declaration, simply reference its module name in an extension declaration.

// In module 'Foo'
module Bar {
  public class A {}

  module Baz {}
}

extension Bar {
  public struct B {}
}

extension Bar.Baz {
  public enum C { case D }
}
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#modules-and-access-control>Modules and Access Control

The semantics of some existing access control modifiers shall also be extended to support module declarations:

open and public declarations are exported by a module for consumption by clients of the module.
internal declarations scope over the entire module and any derived submodules.
By default, to preserve encapsulation of interfaces, modules are "sealed" and may only be "opened" by explicit named import. However, it is often desirable to export a module and a set of submodules or even modules from external dependencies along with a given interface. We propose the public keyword be used for this purpose:

// Defines top-level module "Foo"
//module Foo {
public import Foo.Bar.Baz
public import Foundation.Date
//}
Which then causes the following (sub)modules to be imported into scope along with Foo:

// imports Foo, Foo.Bar.Baz, and Foundation.Date
import Foo
To support existing Swift packages that cannot have opted into modules, and to preserve the scriptable nature of Swift, module declarations shall be optional. Any Swift program that does not declare at least one top-level module explicitly is considered part of an unnamed special "Global Module" with the same rules of access control as today. To give declarations in the Global Module an explicit module without using a module declaration, use the -module-name flag.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#impact-on-existing-code>Impact on Existing Code

This proposal is intentionally additive. There is no impact on existing code.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#alternatives-considered>Alternatives considered

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#explicit-modules-everywhere>Explicit Modules Everywhere

Declarations in the top-level of a program exist today in the top-level of the corresponding module. If desired, this module declaration could be required to be explicit like so:

module Foo {
  module Bar {
    module Baz {}
  }
}
However, we feel that imposing such a requirement not only complicates the outermost scope, it requires inserting needless extension Foo {} scopes in every file. It also violates the principle of progressive disclosure by forcing all new adoptees of Swift to learn what a module is without actually using the module system.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#nested-extensions>Nested Extensions

Nested module extensions may be "expanded" as it were to the following:

module Foo {
  module Bar {}
}

extension Foo {
  extension Bar {}
}
However, this syntax is currently not enabled in general in Swift. This problem should be revisted in a future proposal.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#deprecations-source-breaking-changes>Deprecations (Source-Breaking Changes)

The system described above is intended to be entirely source and binary compatible. Nonetheless, in its design we feel we have obviated certain existing features and recommend their deprecation in future proposals:

fileprivate access can be recreated by creating a private "utility submodule" containing declarations of at least internal access.
@_exported, the private directive to re-export modules today, should be deprecated and removed.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution


(Robert Widmann) #5

What is the rational for having modules covering only part of a file? Wouldn’t it be less clutter to have an annotation which worked for the whole file. At the very least it would be nice to have an option to spell it in a way that applies to the whole file. Otherwise, everything will be indented another level.

That is a valid spelling (Rust, IIRC, allows that spelling), but one that is easy to miss sitting in a file and makes it confusing to introduce submodules. If you include the annotation then define a submodule later down in the file, suddenly you have to remember whether you annotated the file or whether the submodule you’ve just written is going into the top-level module. See:

// -module-name=Foo
// module Foo {
module Bar; // Shorthand for ‚ÄúThis file defines Foo.Bar‚ÄĚ

/* Code */

// This defines ‚ÄúFoo.Bar.Baz‚ÄĚ, but would you know that if it appeared below the fold?
module Baz {}
//}

If anything, this can be added later if evolution converges on it.

I would honestly love to see something which just maps modules to folders/groups for simplicity sake.

There is nothing about this scheme that prevents you from organizing your code this way. However, modulo that particular method of organization, you don’t really gain much as a user of the language by imposing this restriction.

···

On Feb 20, 2017, at 9:36 PM, Jonathan Hull <jhull@gbis.com> wrote:

I haven’t thought about it too much yet, so I could easily be missing something obvious...

Thanks,
Jon

On Feb 20, 2017, at 5:56 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Good Evening All,

Jaden Geller and I have been considering a (sub)module system for Swift that would complement the existing language but also provide sorely needed modularity. A draft of the proposal is attached to this email, but it can also be read as a gist <https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a> if you desire.

Cheers,

~Robert Widmann

Modular Swift

Proposal: SE-NNNN <https://gist.github.com/CodaFi/NNNN-filename.md>
Authors: Robert Widmann <https://github.com/codafi>, Jaden Geller <https://github.com/JadenGeller>
Review Manager: TBD
Status: Awaiting review
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#introduction>Introduction

Almost every major programming language supports some form of modular programming through constructs like (sub)modules, packages, or interfaces. Swift, though it provides top-level modules to organize code under, does not provide a complete implementation of any of these concepts, which has led instead to the proliferation of access control levels. This has not proven an effective way to decompose programs into manageable parts, and exposes the need for a real system of modules to solve this modularity problem once and for all.

Separation of code into distinct islands of functionality should be a first-class construct in the language, not dependent on external files and tools or filesystems. To that end, we propose the introduction of a lightweight module system for Swift.

Swift-evolution thread <applewebdata://32B8028F-83F6-4547-817C-897FD96FD281>
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#motivation>Motivation

Swift has reached a point in its evolution where rich libraries and large projects that take on many dependencies have matured significantly. To accomodate the information-hiding and semantics-signalling needs of these users at the time, Swift began its access control story with just three access modifiers: public, private, and internal then grew fileprivate and open as the need to express locality of implementation and "subclassability" arose respectively. In doing so, Swift's access control scheme has become anti-modular.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#proposed-solution>Proposed solution

We propose the introduction of a lightweight module system for Swift. More than simply namspaces, a module declaration interacts with Swift's access control to provide an API boundary that allows better control over an interface's design.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#detailed-design>Detailed design

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#syntax>Syntax

A module is a named region that introduces a lexical scope into which declarations may be nested. The name of the module can be used to access these member declarations. A module, like other aggregate structures in Swift, may be extended with new declarations over one or more translation units (files).

We propose a new declaration kind, module-decl be added to the language. A proposed grammar using the new modulekeyword is given below:

GRAMMAR OF A MODULE DECLARATION

module-declaration -> `module` module-identifier module-body
module-name -> identifier
module-body -> { module-members(opt) }
module-members -> module-member module-members(opt)
module-member -> declaration | compiler-control-statement
GRAMMAR OF A DECLARATION

+ declaration -> module-declaration
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#general-semantics>General Semantics

Syntax and semantics for imports, as it already supports referencing submodules imported from C and Objective-C modules, remains unchanged:

// The outermost module is given explicitly
// by passing `-module-name=Foo` or exists implicitly, as today.
// module Foo {
public class A {}

module Bar {
  module Baz {
    public class C {}
  }

  public class B {}
}

let message = "Hello, Wisconsin!"
// } // End declarations added to module Foo.
To consume this interface:

// imports all of Foo, Foo.Bar, and Foo.Bar.Baz
import Foo.Bar.Baz

// imports Foo.A as A
import class Foo.A
// imports Foo.Bar.B as B
import class Foo.Bar.B
// imports Foo.Bar.Baz.C as C
import class Foo.Bar.Baz.C
A module declaration may only appear as a top-level entity or as a member of another module declaration. The following code is therefore invalid:

module Foo {
  class Bar {
    module Baz {} // error: module declaration cannot be nested inside type 'Bar'
  }
}
To extend an existing module declaration, simply reference its module name in an extension declaration.

// In module 'Foo'
module Bar {
  public class A {}

  module Baz {}
}

extension Bar {
  public struct B {}
}

extension Bar.Baz {
  public enum C { case D }
}
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#modules-and-access-control>Modules and Access Control

The semantics of some existing access control modifiers shall also be extended to support module declarations:

open and public declarations are exported by a module for consumption by clients of the module.
internal declarations scope over the entire module and any derived submodules.
By default, to preserve encapsulation of interfaces, modules are "sealed" and may only be "opened" by explicit named import. However, it is often desirable to export a module and a set of submodules or even modules from external dependencies along with a given interface. We propose the public keyword be used for this purpose:

// Defines top-level module "Foo"
//module Foo {
public import Foo.Bar.Baz
public import Foundation.Date
//}
Which then causes the following (sub)modules to be imported into scope along with Foo:

// imports Foo, Foo.Bar.Baz, and Foundation.Date
import Foo
To support existing Swift packages that cannot have opted into modules, and to preserve the scriptable nature of Swift, module declarations shall be optional. Any Swift program that does not declare at least one top-level module explicitly is considered part of an unnamed special "Global Module" with the same rules of access control as today. To give declarations in the Global Module an explicit module without using a module declaration, use the -module-name flag.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#impact-on-existing-code>Impact on Existing Code

This proposal is intentionally additive. There is no impact on existing code.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#alternatives-considered>Alternatives considered

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#explicit-modules-everywhere>Explicit Modules Everywhere

Declarations in the top-level of a program exist today in the top-level of the corresponding module. If desired, this module declaration could be required to be explicit like so:

module Foo {
  module Bar {
    module Baz {}
  }
}
However, we feel that imposing such a requirement not only complicates the outermost scope, it requires inserting needless extension Foo {} scopes in every file. It also violates the principle of progressive disclosure by forcing all new adoptees of Swift to learn what a module is without actually using the module system.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#nested-extensions>Nested Extensions

Nested module extensions may be "expanded" as it were to the following:

module Foo {
  module Bar {}
}

extension Foo {
  extension Bar {}
}
However, this syntax is currently not enabled in general in Swift. This problem should be revisted in a future proposal.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#deprecations-source-breaking-changes>Deprecations (Source-Breaking Changes)

The system described above is intended to be entirely source and binary compatible. Nonetheless, in its design we feel we have obviated certain existing features and recommend their deprecation in future proposals:

fileprivate access can be recreated by creating a private "utility submodule" containing declarations of at least internal access.
@_exported, the private directive to re-export modules today, should be deprecated and removed.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution


(Matthew Johnson) #6

Thanks for bringing up submodules Robert. I agree it's a good time to discuss them.

There seem to be two broad approaches to submodules. One is more of a physical / encapsulation based view and the other is more of a logical / namespace based view. This proposal falls into the latter group (Brent's pitch falls into the former).

Before I offer feedback on this proposal I'm wondering if you can provide some solid examples of what benefits you see in the ability of a single file to participate in more than one submodule. And also, what problems do you foresee if a file were restricted to being in a single submodule (or at the top level of the module)?

···

Sent from my iPad

On Feb 20, 2017, at 7:56 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org> wrote:

Good Evening All,

Jaden Geller and I have been considering a (sub)module system for Swift that would complement the existing language but also provide sorely needed modularity. A draft of the proposal is attached to this email, but it can also be read as a gist if you desire.

Cheers,

~Robert Widmann

Modular Swift
Proposal: SE-NNNN
Authors: Robert Widmann, Jaden Geller
Review Manager: TBD
Status: Awaiting review
Introduction

Almost every major programming language supports some form of modular programming through constructs like (sub)modules, packages, or interfaces. Swift, though it provides top-level modules to organize code under, does not provide a complete implementation of any of these concepts, which has led instead to the proliferation of access control levels. This has not proven an effective way to decompose programs into manageable parts, and exposes the need for a real system of modules to solve this modularity problem once and for all.

Separation of code into distinct islands of functionality should be a first-class construct in the language, not dependent on external files and tools or filesystems. To that end, we propose the introduction of a lightweight module system for Swift.

Swift-evolution thread

Motivation

Swift has reached a point in its evolution where rich libraries and large projects that take on many dependencies have matured significantly. To accomodate the information-hiding and semantics-signalling needs of these users at the time, Swift began its access control story with just three access modifiers: public, private, and internal then grew fileprivate and open as the need to express locality of implementation and "subclassability" arose respectively. In doing so, Swift's access control scheme has become anti-modular.

Proposed solution

We propose the introduction of a lightweight module system for Swift. More than simply namspaces, a module declaration interacts with Swift's access control to provide an API boundary that allows better control over an interface's design.

Detailed design

Syntax

A module is a named region that introduces a lexical scope into which declarations may be nested. The name of the module can be used to access these member declarations. A module, like other aggregate structures in Swift, may be extended with new declarations over one or more translation units (files).

We propose a new declaration kind, module-decl be added to the language. A proposed grammar using the new modulekeyword is given below:

GRAMMAR OF A MODULE DECLARATION

module-declaration -> `module` module-identifier module-body
module-name -> identifier
module-body -> { module-members(opt) }
module-members -> module-member module-members(opt)
module-member -> declaration | compiler-control-statement
GRAMMAR OF A DECLARATION

+ declaration -> module-declaration
General Semantics

Syntax and semantics for imports, as it already supports referencing submodules imported from C and Objective-C modules, remains unchanged:

// The outermost module is given explicitly
// by passing `-module-name=Foo` or exists implicitly, as today.
// module Foo {
public class A {}

module Bar {
  module Baz {
    public class C {}
  }

  public class B {}
}

let message = "Hello, Wisconsin!"
// } // End declarations added to module Foo.
To consume this interface:

// imports all of Foo, Foo.Bar, and Foo.Bar.Baz
import Foo.Bar.Baz

// imports Foo.A as A
import class Foo.A
// imports Foo.Bar.B as B
import class Foo.Bar.B
// imports Foo.Bar.Baz.C as C
import class Foo.Bar.Baz.C
A module declaration may only appear as a top-level entity or as a member of another module declaration. The following code is therefore invalid:

module Foo {
  class Bar {
    module Baz {} // error: module declaration cannot be nested inside type 'Bar'
  }
}
To extend an existing module declaration, simply reference its module name in an extension declaration.

// In module 'Foo'
module Bar {
  public class A {}

  module Baz {}
}

extension Bar {
  public struct B {}
}

extension Bar.Baz {
  public enum C { case D }
}
Modules and Access Control

The semantics of some existing access control modifiers shall also be extended to support module declarations:

open and public declarations are exported by a module for consumption by clients of the module.
internal declarations scope over the entire module and any derived submodules.
By default, to preserve encapsulation of interfaces, modules are "sealed" and may only be "opened" by explicit named import. However, it is often desirable to export a module and a set of submodules or even modules from external dependencies along with a given interface. We propose the public keyword be used for this purpose:

// Defines top-level module "Foo"
//module Foo {
public import Foo.Bar.Baz
public import Foundation.Date
//}
Which then causes the following (sub)modules to be imported into scope along with Foo:

// imports Foo, Foo.Bar.Baz, and Foundation.Date
import Foo
To support existing Swift packages that cannot have opted into modules, and to preserve the scriptable nature of Swift, module declarations shall be optional. Any Swift program that does not declare at least one top-level module explicitly is considered part of an unnamed special "Global Module" with the same rules of access control as today. To give declarations in the Global Module an explicit module without using a module declaration, use the -module-name flag.

Impact on Existing Code

This proposal is intentionally additive. There is no impact on existing code.

Alternatives considered

Explicit Modules Everywhere

Declarations in the top-level of a program exist today in the top-level of the corresponding module. If desired, this module declaration could be required to be explicit like so:

module Foo {
  module Bar {
    module Baz {}
  }
}
However, we feel that imposing such a requirement not only complicates the outermost scope, it requires inserting needless extension Foo {} scopes in every file. It also violates the principle of progressive disclosure by forcing all new adoptees of Swift to learn what a module is without actually using the module system.

Nested Extensions

Nested module extensions may be "expanded" as it were to the following:

module Foo {
  module Bar {}
}

extension Foo {
  extension Bar {}
}
However, this syntax is currently not enabled in general in Swift. This problem should be revisted in a future proposal.

Deprecations (Source-Breaking Changes)

The system described above is intended to be entirely source and binary compatible. Nonetheless, in its design we feel we have obviated certain existing features and recommend their deprecation in future proposals:

fileprivate access can be recreated by creating a private "utility submodule" containing declarations of at least internal access.
@_exported, the private directive to re-export modules today, should be deprecated and removed.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Jon Hull) #7

Jon,

I think we might have miscommunicated. It is intended that outermost module is implicit; no `module` declaration is required to wrap every file. We tried to show this in the first code snippet.

What do you mean ‚Äúcovering only part of a file‚ÄĚ?

I am assuming that the ModuleName { … } only affects things within the brackets. Thus it is possible for only part of a file to be within a module. What are the benefits of allowing this, and are they worth the added complexity?

Thanks,
Jon

···

On Feb 20, 2017, at 6:42 PM, Jaden Geller <jaden.geller@gmail.com> wrote:

Cheers,
Jaden Geller

On Feb 20, 2017, at 6:36 PM, Jonathan Hull via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

What is the rational for having modules covering only part of a file? Wouldn’t it be less clutter to have an annotation which worked for the whole file. At the very least it would be nice to have an option to spell it in a way that applies to the whole file. Otherwise, everything will be indented another level.

I would honestly love to see something which just maps modules to folders/groups for simplicity sake.

I haven’t thought about it too much yet, so I could easily be missing something obvious...

Thanks,
Jon


(Brent Royal-Gordon) #8

If I'm reading this correctly, you're proposing that the `internal` APIs in a submodule should *not* be accessible to enclosing modules. I also don't see any indication that you can control who is allowed to import a particular submodule.

That means that, if you use a submodule to encapsulate internal state, the APIs that are available to the parent module are *also* available to any rando who feels like importing your submodule. I don't think that's going to be a tenable design.

(I have a couple other objections‚ÄĒI think the keyword ought to be `submodule` if you don't need a top-level `module` declaration, I think there's a lot to be said for a single declaration covering the entire file, and I'm pretty iffy on this entire approach anyway‚ÄĒbut this seems like the most serious problem of the bunch.)

···

On Feb 20, 2017, at 5:56 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org> wrote:

The semantics of some existing access control modifiers shall also be extended to support module declarations:

¬†¬†‚ÄĘ open and public declarations are exported by a module for consumption by clients of the module.
¬†¬†‚ÄĘ internal declarations scope over the entire module and any derived submodules.

--
Brent Royal-Gordon
Architechies


(Matthew Johnson) #9

Good Evening All,

Jaden Geller and I have been considering a (sub)module system for Swift that would complement the existing language but also provide sorely needed modularity. A draft of the proposal is attached to this email, but it can also be read as a gist <https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a> if you desire.

Cheers,

~Robert Widmann

Modular Swift

Proposal: SE-NNNN <https://gist.github.com/CodaFi/NNNN-filename.md>
Authors: Robert Widmann <https://github.com/codafi>, Jaden Geller <https://github.com/JadenGeller>
Review Manager: TBD
Status: Awaiting review
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#introduction>Introduction

Almost every major programming language supports some form of modular programming through constructs like (sub)modules, packages, or interfaces. Swift, though it provides top-level modules to organize code under, does not provide a complete implementation of any of these concepts, which has led instead to the proliferation of access control levels. This has not proven an effective way to decompose programs into manageable parts, and exposes the need for a real system of modules to solve this modularity problem once and for all.

Separation of code into distinct islands of functionality should be a first-class construct in the language, not dependent on external files and tools or filesystems. To that end, we propose the introduction of a lightweight module system for Swift.

Swift-evolution thread <applewebdata://D267D5BF-B7AA-49E6-B592-9BE742F01507>
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#motivation>Motivation

Swift has reached a point in its evolution where rich libraries and large projects that take on many dependencies have matured significantly. To accomodate the information-hiding and semantics-signalling needs of these users at the time, Swift began its access control story with just three access modifiers: public, private, and internal then grew fileprivate and open as the need to express locality of implementation and "subclassability" arose respectively. In doing so, Swift's access control scheme has become anti-modular.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#proposed-solution>Proposed solution

We propose the introduction of a lightweight module system for Swift. More than simply namspaces, a module declaration interacts with Swift's access control to provide an API boundary that allows better control over an interface's design.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#detailed-design>Detailed design

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#syntax>Syntax

A module is a named region that introduces a lexical scope into which declarations may be nested. The name of the module can be used to access these member declarations. A module, like other aggregate structures in Swift, may be extended with new declarations over one or more translation units (files).

We propose a new declaration kind, module-decl be added to the language. A proposed grammar using the new modulekeyword is given below:

GRAMMAR OF A MODULE DECLARATION

module-declaration -> `module` module-identifier module-body
module-name -> identifier
module-body -> { module-members(opt) }
module-members -> module-member module-members(opt)
module-member -> declaration | compiler-control-statement
GRAMMAR OF A DECLARATION

+ declaration -> module-declaration
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#general-semantics>General Semantics

Syntax and semantics for imports, as it already supports referencing submodules imported from C and Objective-C modules, remains unchanged:

// The outermost module is given explicitly
// by passing `-module-name=Foo` or exists implicitly, as today.
// module Foo {
public class A {}

module Bar {
  module Baz {
    public class C {}
  }

  public class B {}
}

let message = "Hello, Wisconsin!"
// } // End declarations added to module Foo.
To consume this interface:

// imports all of Foo, Foo.Bar, and Foo.Bar.Baz
import Foo.Bar.Baz

I’ve read this a couple of times now and I keep getting hung up on this. Is the comment a mistake? I would only expect to get Foo.Bar.Baz here, not Foo.Bar or Foo itself. If it’s not a mistake, why did you choose this behavior?

···

On Feb 20, 2017, at 7:56 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org> wrote:

// imports Foo.A as A
import class Foo.A
// imports Foo.Bar.B as B
import class Foo.Bar.B
// imports Foo.Bar.Baz.C as C
import class Foo.Bar.Baz.C
A module declaration may only appear as a top-level entity or as a member of another module declaration. The following code is therefore invalid:

module Foo {
  class Bar {
    module Baz {} // error: module declaration cannot be nested inside type 'Bar'
  }
}
To extend an existing module declaration, simply reference its module name in an extension declaration.

// In module 'Foo'
module Bar {
  public class A {}

  module Baz {}
}

extension Bar {
  public struct B {}
}

extension Bar.Baz {
  public enum C { case D }
}
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#modules-and-access-control>Modules and Access Control

The semantics of some existing access control modifiers shall also be extended to support module declarations:

open and public declarations are exported by a module for consumption by clients of the module.
internal declarations scope over the entire module and any derived submodules.
By default, to preserve encapsulation of interfaces, modules are "sealed" and may only be "opened" by explicit named import. However, it is often desirable to export a module and a set of submodules or even modules from external dependencies along with a given interface. We propose the public keyword be used for this purpose:

// Defines top-level module "Foo"
//module Foo {
public import Foo.Bar.Baz
public import Foundation.Date
//}
Which then causes the following (sub)modules to be imported into scope along with Foo:

// imports Foo, Foo.Bar.Baz, and Foundation.Date
import Foo
To support existing Swift packages that cannot have opted into modules, and to preserve the scriptable nature of Swift, module declarations shall be optional. Any Swift program that does not declare at least one top-level module explicitly is considered part of an unnamed special "Global Module" with the same rules of access control as today. To give declarations in the Global Module an explicit module without using a module declaration, use the -module-name flag.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#impact-on-existing-code>Impact on Existing Code

This proposal is intentionally additive. There is no impact on existing code.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#alternatives-considered>Alternatives considered

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#explicit-modules-everywhere>Explicit Modules Everywhere

Declarations in the top-level of a program exist today in the top-level of the corresponding module. If desired, this module declaration could be required to be explicit like so:

module Foo {
  module Bar {
    module Baz {}
  }
}
However, we feel that imposing such a requirement not only complicates the outermost scope, it requires inserting needless extension Foo {} scopes in every file. It also violates the principle of progressive disclosure by forcing all new adoptees of Swift to learn what a module is without actually using the module system.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#nested-extensions>Nested Extensions

Nested module extensions may be "expanded" as it were to the following:

module Foo {
  module Bar {}
}

extension Foo {
  extension Bar {}
}
However, this syntax is currently not enabled in general in Swift. This problem should be revisted in a future proposal.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#deprecations-source-breaking-changes>Deprecations (Source-Breaking Changes)

The system described above is intended to be entirely source and binary compatible. Nonetheless, in its design we feel we have obviated certain existing features and recommend their deprecation in future proposals:

fileprivate access can be recreated by creating a private "utility submodule" containing declarations of at least internal access.
@_exported, the private directive to re-export modules today, should be deprecated and removed.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Jon Hull) #10

I think my question is: Why do we want to allow submodules that are smaller than a file? What does that give us to offset the added complexity?

Thanks,
Jon

···

On Feb 20, 2017, at 6:44 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

On Feb 20, 2017, at 9:36 PM, Jonathan Hull <jhull@gbis.com <mailto:jhull@gbis.com>> wrote:

What is the rational for having modules covering only part of a file? Wouldn’t it be less clutter to have an annotation which worked for the whole file. At the very least it would be nice to have an option to spell it in a way that applies to the whole file. Otherwise, everything will be indented another level.

That is a valid spelling (Rust, IIRC, allows that spelling), but one that is easy to miss sitting in a file and makes it confusing to introduce submodules. If you include the annotation then define a submodule later down in the file, suddenly you have to remember whether you annotated the file or whether the submodule you’ve just written is going into the top-level module. See:

// -module-name=Foo
// module Foo {
module Bar; // Shorthand for ‚ÄúThis file defines Foo.Bar‚ÄĚ

/* Code */

// This defines ‚ÄúFoo.Bar.Baz‚ÄĚ, but would you know that if it appeared below the fold?
module Baz {}
//}

If anything, this can be added later if evolution converges on it.

I would honestly love to see something which just maps modules to folders/groups for simplicity sake.

There is nothing about this scheme that prevents you from organizing your code this way. However, modulo that particular method of organization, you don’t really gain much as a user of the language by imposing this restriction.

I haven’t thought about it too much yet, so I could easily be missing something obvious...

Thanks,
Jon

On Feb 20, 2017, at 5:56 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Good Evening All,

Jaden Geller and I have been considering a (sub)module system for Swift that would complement the existing language but also provide sorely needed modularity. A draft of the proposal is attached to this email, but it can also be read as a gist <https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a> if you desire.

Cheers,

~Robert Widmann

Modular Swift

Proposal: SE-NNNN <https://gist.github.com/CodaFi/NNNN-filename.md>
Authors: Robert Widmann <https://github.com/codafi>, Jaden Geller <https://github.com/JadenGeller>
Review Manager: TBD
Status: Awaiting review
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#introduction>Introduction

Almost every major programming language supports some form of modular programming through constructs like (sub)modules, packages, or interfaces. Swift, though it provides top-level modules to organize code under, does not provide a complete implementation of any of these concepts, which has led instead to the proliferation of access control levels. This has not proven an effective way to decompose programs into manageable parts, and exposes the need for a real system of modules to solve this modularity problem once and for all.

Separation of code into distinct islands of functionality should be a first-class construct in the language, not dependent on external files and tools or filesystems. To that end, we propose the introduction of a lightweight module system for Swift.

Swift-evolution thread <applewebdata://4C05112E-CB0F-4EA2-93D5-BF8C6BD617E3>
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#motivation>Motivation

Swift has reached a point in its evolution where rich libraries and large projects that take on many dependencies have matured significantly. To accomodate the information-hiding and semantics-signalling needs of these users at the time, Swift began its access control story with just three access modifiers: public, private, and internal then grew fileprivate and open as the need to express locality of implementation and "subclassability" arose respectively. In doing so, Swift's access control scheme has become anti-modular.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#proposed-solution>Proposed solution

We propose the introduction of a lightweight module system for Swift. More than simply namspaces, a module declaration interacts with Swift's access control to provide an API boundary that allows better control over an interface's design.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#detailed-design>Detailed design

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#syntax>Syntax

A module is a named region that introduces a lexical scope into which declarations may be nested. The name of the module can be used to access these member declarations. A module, like other aggregate structures in Swift, may be extended with new declarations over one or more translation units (files).

We propose a new declaration kind, module-decl be added to the language. A proposed grammar using the new modulekeyword is given below:

GRAMMAR OF A MODULE DECLARATION

module-declaration -> `module` module-identifier module-body
module-name -> identifier
module-body -> { module-members(opt) }
module-members -> module-member module-members(opt)
module-member -> declaration | compiler-control-statement
GRAMMAR OF A DECLARATION

+ declaration -> module-declaration
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#general-semantics>General Semantics

Syntax and semantics for imports, as it already supports referencing submodules imported from C and Objective-C modules, remains unchanged:

// The outermost module is given explicitly
// by passing `-module-name=Foo` or exists implicitly, as today.
// module Foo {
public class A {}

module Bar {
  module Baz {
    public class C {}
  }

  public class B {}
}

let message = "Hello, Wisconsin!"
// } // End declarations added to module Foo.
To consume this interface:

// imports all of Foo, Foo.Bar, and Foo.Bar.Baz
import Foo.Bar.Baz

// imports Foo.A as A
import class Foo.A
// imports Foo.Bar.B as B
import class Foo.Bar.B
// imports Foo.Bar.Baz.C as C
import class Foo.Bar.Baz.C
A module declaration may only appear as a top-level entity or as a member of another module declaration. The following code is therefore invalid:

module Foo {
  class Bar {
    module Baz {} // error: module declaration cannot be nested inside type 'Bar'
  }
}
To extend an existing module declaration, simply reference its module name in an extension declaration.

// In module 'Foo'
module Bar {
  public class A {}

  module Baz {}
}

extension Bar {
  public struct B {}
}

extension Bar.Baz {
  public enum C { case D }
}
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#modules-and-access-control>Modules and Access Control

The semantics of some existing access control modifiers shall also be extended to support module declarations:

open and public declarations are exported by a module for consumption by clients of the module.
internal declarations scope over the entire module and any derived submodules.
By default, to preserve encapsulation of interfaces, modules are "sealed" and may only be "opened" by explicit named import. However, it is often desirable to export a module and a set of submodules or even modules from external dependencies along with a given interface. We propose the public keyword be used for this purpose:

// Defines top-level module "Foo"
//module Foo {
public import Foo.Bar.Baz
public import Foundation.Date
//}
Which then causes the following (sub)modules to be imported into scope along with Foo:

// imports Foo, Foo.Bar.Baz, and Foundation.Date
import Foo
To support existing Swift packages that cannot have opted into modules, and to preserve the scriptable nature of Swift, module declarations shall be optional. Any Swift program that does not declare at least one top-level module explicitly is considered part of an unnamed special "Global Module" with the same rules of access control as today. To give declarations in the Global Module an explicit module without using a module declaration, use the -module-name flag.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#impact-on-existing-code>Impact on Existing Code

This proposal is intentionally additive. There is no impact on existing code.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#alternatives-considered>Alternatives considered

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#explicit-modules-everywhere>Explicit Modules Everywhere

Declarations in the top-level of a program exist today in the top-level of the corresponding module. If desired, this module declaration could be required to be explicit like so:

module Foo {
  module Bar {
    module Baz {}
  }
}
However, we feel that imposing such a requirement not only complicates the outermost scope, it requires inserting needless extension Foo {} scopes in every file. It also violates the principle of progressive disclosure by forcing all new adoptees of Swift to learn what a module is without actually using the module system.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#nested-extensions>Nested Extensions

Nested module extensions may be "expanded" as it were to the following:

module Foo {
  module Bar {}
}

extension Foo {
  extension Bar {}
}
However, this syntax is currently not enabled in general in Swift. This problem should be revisted in a future proposal.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#deprecations-source-breaking-changes>Deprecations (Source-Breaking Changes)

The system described above is intended to be entirely source and binary compatible. Nonetheless, in its design we feel we have obviated certain existing features and recommend their deprecation in future proposals:

fileprivate access can be recreated by creating a private "utility submodule" containing declarations of at least internal access.
@_exported, the private directive to re-export modules today, should be deprecated and removed.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution


(Jaden Geller) #11

Oh, you’re referring to the fact that a submodule under this proposal must be contained within braces? It would be purely additive to, in the future, annotate that the outer scope of a file is part of some given submodule.

Consider the following the following straw-man syntax that might be equivalent to `module Bar { func foo() { } }`:

module Bar follows // <- at top of file, indicating rest of file is submodule

func foo() { }

Does this address your question?

Thanks,
Jaden Geller

···

On Feb 20, 2017, at 10:39 PM, Jonathan Hull <jhull@gbis.com> wrote:

On Feb 20, 2017, at 6:42 PM, Jaden Geller <jaden.geller@gmail.com <mailto:jaden.geller@gmail.com>> wrote:

Jon,

I think we might have miscommunicated. It is intended that outermost module is implicit; no `module` declaration is required to wrap every file. We tried to show this in the first code snippet.

What do you mean ‚Äúcovering only part of a file‚ÄĚ?

I am assuming that the ModuleName { … } only affects things within the brackets. Thus it is possible for only part of a file to be within a module. What are the benefits of allowing this, and are they worth the added complexity?

Thanks,
Jon

Cheers,
Jaden Geller

On Feb 20, 2017, at 6:36 PM, Jonathan Hull via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

What is the rational for having modules covering only part of a file? Wouldn’t it be less clutter to have an annotation which worked for the whole file. At the very least it would be nice to have an option to spell it in a way that applies to the whole file. Otherwise, everything will be indented another level.

I would honestly love to see something which just maps modules to folders/groups for simplicity sake.

I haven’t thought about it too much yet, so I could easily be missing something obvious...

Thanks,
Jon


(Jon Hull) #12

I think the big gain is that users of IDEs would just be able to graphically organize modules in Xcode. Non-IDE users could use the finder to organize things into nested folders. This maps pretty well with the way I organize my code already (others may be different), so it doesn’t add much complexity.

On a side note, I do think that people will quickly want a way to reference a particular submodule boundary with access modifiers. It may not be part of this proposal, but it is somewhat inevitable. We should consider that as we consider this proposal…

Thanks,
Jon

···

On Feb 20, 2017, at 6:44 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

I would honestly love to see something which just maps modules to folders/groups for simplicity sake.

There is nothing about this scheme that prevents you from organizing your code this way. However, modulo that particular method of organization, you don’t really gain much as a user of the language by imposing this restriction.


(Robert Widmann) #13

Thanks for bringing up submodules Robert. I agree it's a good time to discuss them.

There seem to be two broad approaches to submodules. One is more of a physical / encapsulation based view and the other is more of a logical / namespace based view. This proposal falls into the latter group (Brent's pitch falls into the former).

Before I offer feedback on this proposal I'm wondering if you can provide some solid examples of what benefits you see in the ability of a single file to participate in more than one submodule.

A single file defining more than one submodule is like a single file defining more than one data structure/extension. If you decide that particular functionality deserves to reside in separate areas of concern, but don’t wish to break it out into separate files for whatever reason, just don’t. Can you be more specific about the kind of thing you’re looking for?

And also, what problems do you foresee if a file were restricted to being in a single submodule (or at the top level of the module)?

The top level case is called out in the proposal. We felt that requiring a top-level module declaration in every file to opt-in would be a needless complication to the semantics and would introduce an identifier-addressing problem. Specifically, because modules aren’t namespaces we don’t have a way to refer to the “anonymous/global" top-level namespace as in C++ with a bare ::, so this would become ambiguous without additional syntax

// Explicit declaration of top-level module Foo
module Foo {
  // Decls...
}

// Where does this constant live? How can I address it from within Foo?
// Why isn’t it just a part of Foo?
let string = ‚ÄúHello World!‚ÄĚ

For one-file-per-module, that kind of restriction represents a particular way of organizing code and is a design pattern that is supported under this proposal. We just happen to not enforce that particular pattern, and feel that it is the job of a linter to do so. Really, this kind of restriction is to ease the mental burden on compiler writers who use it to build compilation unit dependency graphs. Swift already considers all files when building its modules because you can extend any type (and now, any module) from any one of them so it doesn’t buy us anything other than an arbitrary restriction.

More generally, we need to move access control away as far away from filesystems as possible. One day, the compiler is going to get ported over to a platform with some bonkers rules that are going to turn around and bite us.

~ Robert Widmann

···

On Feb 20, 2017, at 11:39 PM, Matthew Johnson <matthew@anandabits.com> wrote:

Sent from my iPad

On Feb 20, 2017, at 7:56 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Good Evening All,

Jaden Geller and I have been considering a (sub)module system for Swift that would complement the existing language but also provide sorely needed modularity. A draft of the proposal is attached to this email, but it can also be read as a gist <https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a> if you desire.

Cheers,

~Robert Widmann

Modular Swift

Proposal: SE-NNNN <https://gist.github.com/CodaFi/NNNN-filename.md>
Authors: Robert Widmann <https://github.com/codafi>, Jaden Geller <https://github.com/JadenGeller>
Review Manager: TBD
Status: Awaiting review
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#introduction>Introduction

Almost every major programming language supports some form of modular programming through constructs like (sub)modules, packages, or interfaces. Swift, though it provides top-level modules to organize code under, does not provide a complete implementation of any of these concepts, which has led instead to the proliferation of access control levels. This has not proven an effective way to decompose programs into manageable parts, and exposes the need for a real system of modules to solve this modularity problem once and for all.

Separation of code into distinct islands of functionality should be a first-class construct in the language, not dependent on external files and tools or filesystems. To that end, we propose the introduction of a lightweight module system for Swift.

Swift-evolution thread <applewebdata://CC9A198B-9963-4B27-A62C-22EC5FD1B099>
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#motivation>Motivation

Swift has reached a point in its evolution where rich libraries and large projects that take on many dependencies have matured significantly. To accomodate the information-hiding and semantics-signalling needs of these users at the time, Swift began its access control story with just three access modifiers: public, private, and internal then grew fileprivate and open as the need to express locality of implementation and "subclassability" arose respectively. In doing so, Swift's access control scheme has become anti-modular.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#proposed-solution>Proposed solution

We propose the introduction of a lightweight module system for Swift. More than simply namspaces, a module declaration interacts with Swift's access control to provide an API boundary that allows better control over an interface's design.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#detailed-design>Detailed design

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#syntax>Syntax

A module is a named region that introduces a lexical scope into which declarations may be nested. The name of the module can be used to access these member declarations. A module, like other aggregate structures in Swift, may be extended with new declarations over one or more translation units (files).

We propose a new declaration kind, module-decl be added to the language. A proposed grammar using the new modulekeyword is given below:

GRAMMAR OF A MODULE DECLARATION

module-declaration -> `module` module-identifier module-body
module-name -> identifier
module-body -> { module-members(opt) }
module-members -> module-member module-members(opt)
module-member -> declaration | compiler-control-statement
GRAMMAR OF A DECLARATION

+ declaration -> module-declaration
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#general-semantics>General Semantics

Syntax and semantics for imports, as it already supports referencing submodules imported from C and Objective-C modules, remains unchanged:

// The outermost module is given explicitly
// by passing `-module-name=Foo` or exists implicitly, as today.
// module Foo {
public class A {}

module Bar {
  module Baz {
    public class C {}
  }

  public class B {}
}

let message = "Hello, Wisconsin!"
// } // End declarations added to module Foo.
To consume this interface:

// imports all of Foo, Foo.Bar, and Foo.Bar.Baz
import Foo.Bar.Baz

// imports Foo.A as A
import class Foo.A
// imports Foo.Bar.B as B
import class Foo.Bar.B
// imports Foo.Bar.Baz.C as C
import class Foo.Bar.Baz.C
A module declaration may only appear as a top-level entity or as a member of another module declaration. The following code is therefore invalid:

module Foo {
  class Bar {
    module Baz {} // error: module declaration cannot be nested inside type 'Bar'
  }
}
To extend an existing module declaration, simply reference its module name in an extension declaration.

// In module 'Foo'
module Bar {
  public class A {}

  module Baz {}
}

extension Bar {
  public struct B {}
}

extension Bar.Baz {
  public enum C { case D }
}
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#modules-and-access-control>Modules and Access Control

The semantics of some existing access control modifiers shall also be extended to support module declarations:

open and public declarations are exported by a module for consumption by clients of the module.
internal declarations scope over the entire module and any derived submodules.
By default, to preserve encapsulation of interfaces, modules are "sealed" and may only be "opened" by explicit named import. However, it is often desirable to export a module and a set of submodules or even modules from external dependencies along with a given interface. We propose the public keyword be used for this purpose:

// Defines top-level module "Foo"
//module Foo {
public import Foo.Bar.Baz
public import Foundation.Date
//}
Which then causes the following (sub)modules to be imported into scope along with Foo:

// imports Foo, Foo.Bar.Baz, and Foundation.Date
import Foo
To support existing Swift packages that cannot have opted into modules, and to preserve the scriptable nature of Swift, module declarations shall be optional. Any Swift program that does not declare at least one top-level module explicitly is considered part of an unnamed special "Global Module" with the same rules of access control as today. To give declarations in the Global Module an explicit module without using a module declaration, use the -module-name flag.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#impact-on-existing-code>Impact on Existing Code

This proposal is intentionally additive. There is no impact on existing code.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#alternatives-considered>Alternatives considered

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#explicit-modules-everywhere>Explicit Modules Everywhere

Declarations in the top-level of a program exist today in the top-level of the corresponding module. If desired, this module declaration could be required to be explicit like so:

module Foo {
  module Bar {
    module Baz {}
  }
}
However, we feel that imposing such a requirement not only complicates the outermost scope, it requires inserting needless extension Foo {} scopes in every file. It also violates the principle of progressive disclosure by forcing all new adoptees of Swift to learn what a module is without actually using the module system.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#nested-extensions>Nested Extensions

Nested module extensions may be "expanded" as it were to the following:

module Foo {
  module Bar {}
}

extension Foo {
  extension Bar {}
}
However, this syntax is currently not enabled in general in Swift. This problem should be revisted in a future proposal.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#deprecations-source-breaking-changes>Deprecations (Source-Breaking Changes)

The system described above is intended to be entirely source and binary compatible. Nonetheless, in its design we feel we have obviated certain existing features and recommend their deprecation in future proposals:

fileprivate access can be recreated by creating a private "utility submodule" containing declarations of at least internal access.
@_exported, the private directive to re-export modules today, should be deprecated and removed.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution


(Jaden Geller) #14

The semantics of some existing access control modifiers shall also be extended to support module declarations:

¬†¬†‚ÄĘ open and public declarations are exported by a module for consumption by clients of the module.
¬†¬†‚ÄĘ internal declarations scope over the entire module and any derived submodules.

If I'm reading this correctly, you're proposing that the `internal` APIs in a submodule should *not* be accessible to enclosing modules. I also don't see any indication that you can control who is allowed to import a particular submodule.

That means that, if you use a submodule to encapsulate internal state, the APIs that are available to the parent module are *also* available to any rando who feels like importing your submodule. I don't think that's going to be a tenable design.

It might be reasonable to allow access control modifiers to be used with modules.

‚ÄĒ `private` module declarations can only be imported in the scope in which they were defined
‚ÄĒ `fileprivate` module declarations can only be imported in the file in which they were defined
‚ÄĒ `internal` module declarations can be imported anywhere in the outermost (?) module
‚ÄĒ `public` module declaration are exported to consumers of the outermost module

Definitely more thought needs to go into this, but it might address this desire.

(I have a couple other objections‚ÄĒI think the keyword ought to be `submodule` if you don't need a top-level `module` declaration, I think there's a lot to be said for a single declaration covering the entire file, and I'm pretty iffy on this entire approach anyway‚ÄĒbut this seems like the most serious problem of the bunch.)

I think using `submodule` as a keyword is both reasonable and a good idea. It does become problematic if/when we eventually want to allow entire files to be part of a submodule though…

···

On Feb 21, 2017, at 5:00 AM, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> wrote:

On Feb 20, 2017, at 5:56 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org> wrote:

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Robert Widmann) #15

The semantics of some existing access control modifiers shall also be extended to support module declarations:

¬†¬†‚ÄĘ open and public declarations are exported by a module for consumption by clients of the module.
¬†¬†‚ÄĘ internal declarations scope over the entire module and any derived submodules.

If I'm reading this correctly, you're proposing that the `internal` APIs in a submodule should *not* be accessible to enclosing modules. I also don't see any indication that you can control who is allowed to import a particular submodule.

That means that, if you use a submodule to encapsulate internal state, the APIs that are available to the parent module are *also* available to any rando who feels like importing your submodule. I don't think that's going to be a tenable design.

If the state is truly internal, and you are using internal access control, this is impossible. The internal state cannot cross the module boundary unless it is marked public. If a ‚Äúrandom‚ÄĚ feels like importing my submodule, they will not have access to any symbols.

···

On Feb 21, 2017, at 8:00 AM, Brent Royal-Gordon <brent@architechies.com> wrote:

On Feb 20, 2017, at 5:56 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org> wrote:

(I have a couple other objections‚ÄĒI think the keyword ought to be `submodule` if you don't need a top-level `module` declaration, I think there's a lot to be said for a single declaration covering the entire file, and I'm pretty iffy on this entire approach anyway‚ÄĒbut this seems like the most serious problem of the bunch.)

--
Brent Royal-Gordon
Architechies


(Robert Widmann) #16

Good question! This behavior is actually the behavior that exists today. For example, open a playground and type

import Foundation.NSDebug

let s : NSString = “"

You’ll notice no matter which submodule you try to visit (Darwin.uuid is another good example), Swift has decided to insert a top-level import. We decided not to change this behavior to maintain source compatibility.

···

On Feb 21, 2017, at 11:21 PM, Matthew Johnson <matthew@anandabits.com> wrote:

On Feb 20, 2017, at 7:56 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Good Evening All,

Jaden Geller and I have been considering a (sub)module system for Swift that would complement the existing language but also provide sorely needed modularity. A draft of the proposal is attached to this email, but it can also be read as a gist <https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a> if you desire.

Cheers,

~Robert Widmann

Modular Swift

Proposal: SE-NNNN <https://gist.github.com/CodaFi/NNNN-filename.md>
Authors: Robert Widmann <https://github.com/codafi>, Jaden Geller <https://github.com/JadenGeller>
Review Manager: TBD
Status: Awaiting review
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#introduction>Introduction

Almost every major programming language supports some form of modular programming through constructs like (sub)modules, packages, or interfaces. Swift, though it provides top-level modules to organize code under, does not provide a complete implementation of any of these concepts, which has led instead to the proliferation of access control levels. This has not proven an effective way to decompose programs into manageable parts, and exposes the need for a real system of modules to solve this modularity problem once and for all.

Separation of code into distinct islands of functionality should be a first-class construct in the language, not dependent on external files and tools or filesystems. To that end, we propose the introduction of a lightweight module system for Swift.

Swift-evolution thread <applewebdata://0EA05864-29DB-470C-BC04-091D903C80C6>
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#motivation>Motivation

Swift has reached a point in its evolution where rich libraries and large projects that take on many dependencies have matured significantly. To accomodate the information-hiding and semantics-signalling needs of these users at the time, Swift began its access control story with just three access modifiers: public, private, and internal then grew fileprivate and open as the need to express locality of implementation and "subclassability" arose respectively. In doing so, Swift's access control scheme has become anti-modular.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#proposed-solution>Proposed solution

We propose the introduction of a lightweight module system for Swift. More than simply namspaces, a module declaration interacts with Swift's access control to provide an API boundary that allows better control over an interface's design.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#detailed-design>Detailed design

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#syntax>Syntax

A module is a named region that introduces a lexical scope into which declarations may be nested. The name of the module can be used to access these member declarations. A module, like other aggregate structures in Swift, may be extended with new declarations over one or more translation units (files).

We propose a new declaration kind, module-decl be added to the language. A proposed grammar using the new modulekeyword is given below:

GRAMMAR OF A MODULE DECLARATION

module-declaration -> `module` module-identifier module-body
module-name -> identifier
module-body -> { module-members(opt) }
module-members -> module-member module-members(opt)
module-member -> declaration | compiler-control-statement
GRAMMAR OF A DECLARATION

+ declaration -> module-declaration
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#general-semantics>General Semantics

Syntax and semantics for imports, as it already supports referencing submodules imported from C and Objective-C modules, remains unchanged:

// The outermost module is given explicitly
// by passing `-module-name=Foo` or exists implicitly, as today.
// module Foo {
public class A {}

module Bar {
  module Baz {
    public class C {}
  }

  public class B {}
}

let message = "Hello, Wisconsin!"
// } // End declarations added to module Foo.
To consume this interface:

// imports all of Foo, Foo.Bar, and Foo.Bar.Baz
import Foo.Bar.Baz

I’ve read this a couple of times now and I keep getting hung up on this. Is the comment a mistake? I would only expect to get Foo.Bar.Baz here, not Foo.Bar or Foo itself. If it’s not a mistake, why did you choose this behavior?

// imports Foo.A as A
import class Foo.A
// imports Foo.Bar.B as B
import class Foo.Bar.B
// imports Foo.Bar.Baz.C as C
import class Foo.Bar.Baz.C
A module declaration may only appear as a top-level entity or as a member of another module declaration. The following code is therefore invalid:

module Foo {
  class Bar {
    module Baz {} // error: module declaration cannot be nested inside type 'Bar'
  }
}
To extend an existing module declaration, simply reference its module name in an extension declaration.

// In module 'Foo'
module Bar {
  public class A {}

  module Baz {}
}

extension Bar {
  public struct B {}
}

extension Bar.Baz {
  public enum C { case D }
}
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#modules-and-access-control>Modules and Access Control

The semantics of some existing access control modifiers shall also be extended to support module declarations:

open and public declarations are exported by a module for consumption by clients of the module.
internal declarations scope over the entire module and any derived submodules.
By default, to preserve encapsulation of interfaces, modules are "sealed" and may only be "opened" by explicit named import. However, it is often desirable to export a module and a set of submodules or even modules from external dependencies along with a given interface. We propose the public keyword be used for this purpose:

// Defines top-level module "Foo"
//module Foo {
public import Foo.Bar.Baz
public import Foundation.Date
//}
Which then causes the following (sub)modules to be imported into scope along with Foo:

// imports Foo, Foo.Bar.Baz, and Foundation.Date
import Foo
To support existing Swift packages that cannot have opted into modules, and to preserve the scriptable nature of Swift, module declarations shall be optional. Any Swift program that does not declare at least one top-level module explicitly is considered part of an unnamed special "Global Module" with the same rules of access control as today. To give declarations in the Global Module an explicit module without using a module declaration, use the -module-name flag.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#impact-on-existing-code>Impact on Existing Code

This proposal is intentionally additive. There is no impact on existing code.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#alternatives-considered>Alternatives considered

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#explicit-modules-everywhere>Explicit Modules Everywhere

Declarations in the top-level of a program exist today in the top-level of the corresponding module. If desired, this module declaration could be required to be explicit like so:

module Foo {
  module Bar {
    module Baz {}
  }
}
However, we feel that imposing such a requirement not only complicates the outermost scope, it requires inserting needless extension Foo {} scopes in every file. It also violates the principle of progressive disclosure by forcing all new adoptees of Swift to learn what a module is without actually using the module system.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#nested-extensions>Nested Extensions

Nested module extensions may be "expanded" as it were to the following:

module Foo {
  module Bar {}
}

extension Foo {
  extension Bar {}
}
However, this syntax is currently not enabled in general in Swift. This problem should be revisted in a future proposal.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#deprecations-source-breaking-changes>Deprecations (Source-Breaking Changes)

The system described above is intended to be entirely source and binary compatible. Nonetheless, in its design we feel we have obviated certain existing features and recommend their deprecation in future proposals:

fileprivate access can be recreated by creating a private "utility submodule" containing declarations of at least internal access.
@_exported, the private directive to re-export modules today, should be deprecated and removed.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution


(Jaden Geller) #17

Oh, I see. You’re questioning the motivation of having scope-granularity submodules at all! My misunderstanding.

I actually hadn’t considered this as added complexity. In my mind, a scoped module declaration seems more Swifty than a file module declaration. It builds on the existing syntax in Swift for defining other sorts of scopes, e.g. types.

Cheers,
Jaden Geller

···

On Feb 20, 2017, at 10:41 PM, Jonathan Hull via swift-evolution <swift-evolution@swift.org> wrote:

I think my question is: Why do we want to allow submodules that are smaller than a file? What does that give us to offset the added complexity?

Thanks,
Jon

On Feb 20, 2017, at 6:44 PM, Robert Widmann <devteam.codafi@gmail.com <mailto:devteam.codafi@gmail.com>> wrote:

On Feb 20, 2017, at 9:36 PM, Jonathan Hull <jhull@gbis.com <mailto:jhull@gbis.com>> wrote:

What is the rational for having modules covering only part of a file? Wouldn’t it be less clutter to have an annotation which worked for the whole file. At the very least it would be nice to have an option to spell it in a way that applies to the whole file. Otherwise, everything will be indented another level.

That is a valid spelling (Rust, IIRC, allows that spelling), but one that is easy to miss sitting in a file and makes it confusing to introduce submodules. If you include the annotation then define a submodule later down in the file, suddenly you have to remember whether you annotated the file or whether the submodule you’ve just written is going into the top-level module. See:

// -module-name=Foo
// module Foo {
module Bar; // Shorthand for ‚ÄúThis file defines Foo.Bar‚ÄĚ

/* Code */

// This defines ‚ÄúFoo.Bar.Baz‚ÄĚ, but would you know that if it appeared below the fold?
module Baz {}
//}

If anything, this can be added later if evolution converges on it.

I would honestly love to see something which just maps modules to folders/groups for simplicity sake.

There is nothing about this scheme that prevents you from organizing your code this way. However, modulo that particular method of organization, you don’t really gain much as a user of the language by imposing this restriction.

I haven’t thought about it too much yet, so I could easily be missing something obvious...

Thanks,
Jon

On Feb 20, 2017, at 5:56 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Good Evening All,

Jaden Geller and I have been considering a (sub)module system for Swift that would complement the existing language but also provide sorely needed modularity. A draft of the proposal is attached to this email, but it can also be read as a gist <https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a> if you desire.

Cheers,

~Robert Widmann

Modular Swift

Proposal: SE-NNNN <https://gist.github.com/CodaFi/NNNN-filename.md>
Authors: Robert Widmann <https://github.com/codafi>, Jaden Geller <https://github.com/JadenGeller>
Review Manager: TBD
Status: Awaiting review
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#introduction>Introduction

Almost every major programming language supports some form of modular programming through constructs like (sub)modules, packages, or interfaces. Swift, though it provides top-level modules to organize code under, does not provide a complete implementation of any of these concepts, which has led instead to the proliferation of access control levels. This has not proven an effective way to decompose programs into manageable parts, and exposes the need for a real system of modules to solve this modularity problem once and for all.

Separation of code into distinct islands of functionality should be a first-class construct in the language, not dependent on external files and tools or filesystems. To that end, we propose the introduction of a lightweight module system for Swift.

Swift-evolution thread <applewebdata://6211E639-228B-457B-B598-8F026C4DC4FA>
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#motivation>Motivation

Swift has reached a point in its evolution where rich libraries and large projects that take on many dependencies have matured significantly. To accomodate the information-hiding and semantics-signalling needs of these users at the time, Swift began its access control story with just three access modifiers: public, private, and internal then grew fileprivate and open as the need to express locality of implementation and "subclassability" arose respectively. In doing so, Swift's access control scheme has become anti-modular.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#proposed-solution>Proposed solution

We propose the introduction of a lightweight module system for Swift. More than simply namspaces, a module declaration interacts with Swift's access control to provide an API boundary that allows better control over an interface's design.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#detailed-design>Detailed design

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#syntax>Syntax

A module is a named region that introduces a lexical scope into which declarations may be nested. The name of the module can be used to access these member declarations. A module, like other aggregate structures in Swift, may be extended with new declarations over one or more translation units (files).

We propose a new declaration kind, module-decl be added to the language. A proposed grammar using the new modulekeyword is given below:

GRAMMAR OF A MODULE DECLARATION

module-declaration -> `module` module-identifier module-body
module-name -> identifier
module-body -> { module-members(opt) }
module-members -> module-member module-members(opt)
module-member -> declaration | compiler-control-statement
GRAMMAR OF A DECLARATION

+ declaration -> module-declaration
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#general-semantics>General Semantics

Syntax and semantics for imports, as it already supports referencing submodules imported from C and Objective-C modules, remains unchanged:

// The outermost module is given explicitly
// by passing `-module-name=Foo` or exists implicitly, as today.
// module Foo {
public class A {}

module Bar {
  module Baz {
    public class C {}
  }

  public class B {}
}

let message = "Hello, Wisconsin!"
// } // End declarations added to module Foo.
To consume this interface:

// imports all of Foo, Foo.Bar, and Foo.Bar.Baz
import Foo.Bar.Baz

// imports Foo.A as A
import class Foo.A
// imports Foo.Bar.B as B
import class Foo.Bar.B
// imports Foo.Bar.Baz.C as C
import class Foo.Bar.Baz.C
A module declaration may only appear as a top-level entity or as a member of another module declaration. The following code is therefore invalid:

module Foo {
  class Bar {
    module Baz {} // error: module declaration cannot be nested inside type 'Bar'
  }
}
To extend an existing module declaration, simply reference its module name in an extension declaration.

// In module 'Foo'
module Bar {
  public class A {}

  module Baz {}
}

extension Bar {
  public struct B {}
}

extension Bar.Baz {
  public enum C { case D }
}
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#modules-and-access-control>Modules and Access Control

The semantics of some existing access control modifiers shall also be extended to support module declarations:

open and public declarations are exported by a module for consumption by clients of the module.
internal declarations scope over the entire module and any derived submodules.
By default, to preserve encapsulation of interfaces, modules are "sealed" and may only be "opened" by explicit named import. However, it is often desirable to export a module and a set of submodules or even modules from external dependencies along with a given interface. We propose the public keyword be used for this purpose:

// Defines top-level module "Foo"
//module Foo {
public import Foo.Bar.Baz
public import Foundation.Date
//}
Which then causes the following (sub)modules to be imported into scope along with Foo:

// imports Foo, Foo.Bar.Baz, and Foundation.Date
import Foo
To support existing Swift packages that cannot have opted into modules, and to preserve the scriptable nature of Swift, module declarations shall be optional. Any Swift program that does not declare at least one top-level module explicitly is considered part of an unnamed special "Global Module" with the same rules of access control as today. To give declarations in the Global Module an explicit module without using a module declaration, use the -module-name flag.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#impact-on-existing-code>Impact on Existing Code

This proposal is intentionally additive. There is no impact on existing code.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#alternatives-considered>Alternatives considered

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#explicit-modules-everywhere>Explicit Modules Everywhere

Declarations in the top-level of a program exist today in the top-level of the corresponding module. If desired, this module declaration could be required to be explicit like so:

module Foo {
  module Bar {
    module Baz {}
  }
}
However, we feel that imposing such a requirement not only complicates the outermost scope, it requires inserting needless extension Foo {} scopes in every file. It also violates the principle of progressive disclosure by forcing all new adoptees of Swift to learn what a module is without actually using the module system.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#nested-extensions>Nested Extensions

Nested module extensions may be "expanded" as it were to the following:

module Foo {
  module Bar {}
}

extension Foo {
  extension Bar {}
}
However, this syntax is currently not enabled in general in Swift. This problem should be revisted in a future proposal.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#deprecations-source-breaking-changes>Deprecations (Source-Breaking Changes)

The system described above is intended to be entirely source and binary compatible. Nonetheless, in its design we feel we have obviated certain existing features and recommend their deprecation in future proposals:

fileprivate access can be recreated by creating a private "utility submodule" containing declarations of at least internal access.
@_exported, the private directive to re-export modules today, should be deprecated and removed.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Jon Hull) #18

Part of it. My concern is that the brackets will push everything in the file to the right.

My question is: What do we get in return for that? (as opposed to a system which acted on the whole file)

Thanks,
Jon

···

On Feb 20, 2017, at 10:45 PM, Jaden Geller <jaden.geller@gmail.com> wrote:

Oh, you’re referring to the fact that a submodule under this proposal must be contained within braces? It would be purely additive to, in the future, annotate that the outer scope of a file is part of some given submodule.

Consider the following the following straw-man syntax that might be equivalent to `module Bar { func foo() { } }`:

module Bar follows // <- at top of file, indicating rest of file is submodule

func foo() { }

Does this address your question?

Thanks,
Jaden Geller

On Feb 20, 2017, at 10:39 PM, Jonathan Hull <jhull@gbis.com <mailto:jhull@gbis.com>> wrote:

On Feb 20, 2017, at 6:42 PM, Jaden Geller <jaden.geller@gmail.com <mailto:jaden.geller@gmail.com>> wrote:

Jon,

I think we might have miscommunicated. It is intended that outermost module is implicit; no `module` declaration is required to wrap every file. We tried to show this in the first code snippet.

What do you mean ‚Äúcovering only part of a file‚ÄĚ?

I am assuming that the ModuleName { … } only affects things within the brackets. Thus it is possible for only part of a file to be within a module. What are the benefits of allowing this, and are they worth the added complexity?

Thanks,
Jon

Cheers,
Jaden Geller

On Feb 20, 2017, at 6:36 PM, Jonathan Hull via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

What is the rational for having modules covering only part of a file? Wouldn’t it be less clutter to have an annotation which worked for the whole file. At the very least it would be nice to have an option to spell it in a way that applies to the whole file. Otherwise, everything will be indented another level.

I would honestly love to see something which just maps modules to folders/groups for simplicity sake.

I haven’t thought about it too much yet, so I could easily be missing something obvious...

Thanks,
Jon


(Jon Hull) #19

This is the key thing which I think is being debated right now. Swift 2 defined the file as a unit of compilation and based its elegant system of access modifiers on that. With Swift 3 we have a mix of type/scope based modifiers and file based modifiers… and it is super confusing to everyone because we have mixed our metaphors.

As much as I want modules, I am now convinced that this proposal will only make that situation worse. We need to pick one way (file based) or the other (scope based) and commit to it… but either way, it will require a larger overhaul to make the system consistent/usable/teachable again.

As an analogy, it is like some people are trying to play rock music during a classical music concert. Both are great independently, and different people may prefer one or the other… but trying to play them on top of one another just results in noise.

Thanks,
Jon

···

On Feb 20, 2017, at 10:46 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org> wrote:

More generally, we need to move access control away as far away from filesystems as possible. One day, the compiler is going to get ported over to a platform with some bonkers rules that are going to turn around and bite us.


(Robert Widmann) #20

The same benefits afforded by multiple extensions contained in the same file: sometimes you wish for your concerns not to overlap with one another, but feel that they do not necessarily warrant being split into separate files. There really isn’t a reason we should enforce separation of submodules in the language if we can help it, it’s more the job of a linter.

~Robert Widmann

···

On Feb 21, 2017, at 1:41 AM, Jonathan Hull <jhull@gbis.com> wrote:

I think my question is: Why do we want to allow submodules that are smaller than a file? What does that give us to offset the added complexity?

Thanks,
Jon

On Feb 20, 2017, at 6:44 PM, Robert Widmann <devteam.codafi@gmail.com <mailto:devteam.codafi@gmail.com>> wrote:

On Feb 20, 2017, at 9:36 PM, Jonathan Hull <jhull@gbis.com <mailto:jhull@gbis.com>> wrote:

What is the rational for having modules covering only part of a file? Wouldn’t it be less clutter to have an annotation which worked for the whole file. At the very least it would be nice to have an option to spell it in a way that applies to the whole file. Otherwise, everything will be indented another level.

That is a valid spelling (Rust, IIRC, allows that spelling), but one that is easy to miss sitting in a file and makes it confusing to introduce submodules. If you include the annotation then define a submodule later down in the file, suddenly you have to remember whether you annotated the file or whether the submodule you’ve just written is going into the top-level module. See:

// -module-name=Foo
// module Foo {
module Bar; // Shorthand for ‚ÄúThis file defines Foo.Bar‚ÄĚ

/* Code */

// This defines ‚ÄúFoo.Bar.Baz‚ÄĚ, but would you know that if it appeared below the fold?
module Baz {}
//}

If anything, this can be added later if evolution converges on it.

I would honestly love to see something which just maps modules to folders/groups for simplicity sake.

There is nothing about this scheme that prevents you from organizing your code this way. However, modulo that particular method of organization, you don’t really gain much as a user of the language by imposing this restriction.

I haven’t thought about it too much yet, so I could easily be missing something obvious...

Thanks,
Jon

On Feb 20, 2017, at 5:56 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Good Evening All,

Jaden Geller and I have been considering a (sub)module system for Swift that would complement the existing language but also provide sorely needed modularity. A draft of the proposal is attached to this email, but it can also be read as a gist <https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a> if you desire.

Cheers,

~Robert Widmann

Modular Swift

Proposal: SE-NNNN <https://gist.github.com/CodaFi/NNNN-filename.md>
Authors: Robert Widmann <https://github.com/codafi>, Jaden Geller <https://github.com/JadenGeller>
Review Manager: TBD
Status: Awaiting review
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#introduction>Introduction

Almost every major programming language supports some form of modular programming through constructs like (sub)modules, packages, or interfaces. Swift, though it provides top-level modules to organize code under, does not provide a complete implementation of any of these concepts, which has led instead to the proliferation of access control levels. This has not proven an effective way to decompose programs into manageable parts, and exposes the need for a real system of modules to solve this modularity problem once and for all.

Separation of code into distinct islands of functionality should be a first-class construct in the language, not dependent on external files and tools or filesystems. To that end, we propose the introduction of a lightweight module system for Swift.

Swift-evolution thread <applewebdata://82CEF250-170B-4B51-B538-1715E9963C3E>
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#motivation>Motivation

Swift has reached a point in its evolution where rich libraries and large projects that take on many dependencies have matured significantly. To accomodate the information-hiding and semantics-signalling needs of these users at the time, Swift began its access control story with just three access modifiers: public, private, and internal then grew fileprivate and open as the need to express locality of implementation and "subclassability" arose respectively. In doing so, Swift's access control scheme has become anti-modular.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#proposed-solution>Proposed solution

We propose the introduction of a lightweight module system for Swift. More than simply namspaces, a module declaration interacts with Swift's access control to provide an API boundary that allows better control over an interface's design.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#detailed-design>Detailed design

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#syntax>Syntax

A module is a named region that introduces a lexical scope into which declarations may be nested. The name of the module can be used to access these member declarations. A module, like other aggregate structures in Swift, may be extended with new declarations over one or more translation units (files).

We propose a new declaration kind, module-decl be added to the language. A proposed grammar using the new modulekeyword is given below:

GRAMMAR OF A MODULE DECLARATION

module-declaration -> `module` module-identifier module-body
module-name -> identifier
module-body -> { module-members(opt) }
module-members -> module-member module-members(opt)
module-member -> declaration | compiler-control-statement
GRAMMAR OF A DECLARATION

+ declaration -> module-declaration
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#general-semantics>General Semantics

Syntax and semantics for imports, as it already supports referencing submodules imported from C and Objective-C modules, remains unchanged:

// The outermost module is given explicitly
// by passing `-module-name=Foo` or exists implicitly, as today.
// module Foo {
public class A {}

module Bar {
  module Baz {
    public class C {}
  }

  public class B {}
}

let message = "Hello, Wisconsin!"
// } // End declarations added to module Foo.
To consume this interface:

// imports all of Foo, Foo.Bar, and Foo.Bar.Baz
import Foo.Bar.Baz

// imports Foo.A as A
import class Foo.A
// imports Foo.Bar.B as B
import class Foo.Bar.B
// imports Foo.Bar.Baz.C as C
import class Foo.Bar.Baz.C
A module declaration may only appear as a top-level entity or as a member of another module declaration. The following code is therefore invalid:

module Foo {
  class Bar {
    module Baz {} // error: module declaration cannot be nested inside type 'Bar'
  }
}
To extend an existing module declaration, simply reference its module name in an extension declaration.

// In module 'Foo'
module Bar {
  public class A {}

  module Baz {}
}

extension Bar {
  public struct B {}
}

extension Bar.Baz {
  public enum C { case D }
}
<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#modules-and-access-control>Modules and Access Control

The semantics of some existing access control modifiers shall also be extended to support module declarations:

open and public declarations are exported by a module for consumption by clients of the module.
internal declarations scope over the entire module and any derived submodules.
By default, to preserve encapsulation of interfaces, modules are "sealed" and may only be "opened" by explicit named import. However, it is often desirable to export a module and a set of submodules or even modules from external dependencies along with a given interface. We propose the public keyword be used for this purpose:

// Defines top-level module "Foo"
//module Foo {
public import Foo.Bar.Baz
public import Foundation.Date
//}
Which then causes the following (sub)modules to be imported into scope along with Foo:

// imports Foo, Foo.Bar.Baz, and Foundation.Date
import Foo
To support existing Swift packages that cannot have opted into modules, and to preserve the scriptable nature of Swift, module declarations shall be optional. Any Swift program that does not declare at least one top-level module explicitly is considered part of an unnamed special "Global Module" with the same rules of access control as today. To give declarations in the Global Module an explicit module without using a module declaration, use the -module-name flag.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#impact-on-existing-code>Impact on Existing Code

This proposal is intentionally additive. There is no impact on existing code.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#alternatives-considered>Alternatives considered

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#explicit-modules-everywhere>Explicit Modules Everywhere

Declarations in the top-level of a program exist today in the top-level of the corresponding module. If desired, this module declaration could be required to be explicit like so:

module Foo {
  module Bar {
    module Baz {}
  }
}
However, we feel that imposing such a requirement not only complicates the outermost scope, it requires inserting needless extension Foo {} scopes in every file. It also violates the principle of progressive disclosure by forcing all new adoptees of Swift to learn what a module is without actually using the module system.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#nested-extensions>Nested Extensions

Nested module extensions may be "expanded" as it were to the following:

module Foo {
  module Bar {}
}

extension Foo {
  extension Bar {}
}
However, this syntax is currently not enabled in general in Swift. This problem should be revisted in a future proposal.

<https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a#deprecations-source-breaking-changes>Deprecations (Source-Breaking Changes)

The system described above is intended to be entirely source and binary compatible. Nonetheless, in its design we feel we have obviated certain existing features and recommend their deprecation in future proposals:

fileprivate access can be recreated by creating a private "utility submodule" containing declarations of at least internal access.
@_exported, the private directive to re-export modules today, should be deprecated and removed.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution