[Pitch] Let's talk about submodules

Brent_Royal-Gordon · February 21, 2017, 1:36am

Okay, lots of people want to have some kind of submodule feature, so I'd like to sketch one out so we can hopefully agree on what submodules might look like.

···

***

Any group of Swift files can be grouped together to form a submodule. Submodules belong within a particular module, and have a dotted name: If `ModKit` is a module, it might have a submodule called `ModKit.Foo`. Submodules can be nested within one another: `ModKit.Foo.Bar` is a submodule of `ModKit.Foo`, which is a submodule of `ModKit`.

No new access levels are necessary. `internal` APIs are only visible within the submodule they're declared in; a module cannot see its submodules' `internal` APIs, and a submodule cannot see its parent module's `internal` APIs. If a submodule wants to expose some of its APIs to its parent or sibling modules, it must mark them as `public` or `open`. Then they can import the submodule to see its APIs:

import ModKit.Foo

By default, outside modules cannot import a submodule. But an import in the parent module can be decorated by an access control keyword to allow that:

/// Any module outside ModKit can import ModKit.Foo and access its `public` and `open` APIs.
open import ModKit.Foo

  /// Any module outside ModKit can import ModKit.Foo and access its `public` and `open` APIs,
  /// except `open` APIs are treated as `public`.
  public import ModKit.Foo

Imports may also be decorated by the `@exported` attribute, which exposes the submodule's APIs as though they were parent module APIs:

@exported open import ModKit.Foo

@exported public import ModKit.Foo

(This is sort of half-implemented already in a buggy `@_exported` attribute.)

Finally, the standard syntax for importing individual symbols can be used to cherry-pick types to treat differently:

// Most ModKit.Foo APIs are not importable...
import ModKit.Foo

// ...but SomeEnum can be imported as public...
public import enum ModKit.Foo.SomeEnum

// ...SomeClass can be imported as open...
open import class ModKit.Foo.SomeClass

// And ImportantStruct will import whenever you import ModKit.
@exported public import struct ModKit.Foo.ImportantStruct

(This syntax should be enhanced to allow cherry-picked importing of global functions, constants, and variables.)

If there are several different `import`s covering the same submodule or submodule symbol, the most permissive one wins.

(In large projects, `public`, `open`, and `@exported` imports will most likely all be put in a single Policy.swift file or something, but this is not enforced by the language.)

A submodule may not import any direct parent module (parent, grandparent, etc.), but may import any other submodule in the same module. This list shows permitted imports for a project with four modules/submodules:

  ModKit
    - ModKit.Foo
    - ModKit.Foo.Bar
    - ModKit.Quux
  ModKit.Foo
    - ModKit.Foo.Bar
    - ModKit.Quux
  ModKit.Foo.Bar
    - ModKit.Quux
  ModKit.Quux
    - ModKit.Foo
    - ModKit.Foo.Bar

However, submodules may not form circular dependencies through imports—if `ModKit.Quux` imports `ModKit.Foo`, then `ModKit.Foo` cannot import `ModKit.Quux`. The `#if canImport()` feature cannot be used to probe for other submodules within the same top-level module you're in.

At the compiler driver level, a submodule is specified by giving a `-module-name` parameter with a dot in it. When a file is compiled, only the filenames of the other .swift files in the same module are specified, along with .o files for any submodules; then all the .o files within that submodule are linked into a single .o file for the whole submodule. So files in `ModKit.Foo` would be compiled with only the .swift files in `ModKit.Foo` and the .o file for `ModKit.Foo.Bar`; then all the `ModKit.Foo` .o files would be linked into one .o file for the top-level `ModKit` to use. None of `ModKit.Foo`'s .swift files would be included in the command line when compiling the top-level `ModKit` module.

(That bit is kind of speculative—particularly the idea of linking submodule files into a single .o file—but I think something like what I'm describing could work.)

Because the compiler driver is used to group submodules together, Xcode can specify submodules in project file metadata and calculate a submodule dependency graph, while SwiftPM can use folders and compile submodules whenever the compiler emits an error indicating that a file tried to import a nonexistent submodule. Other build systems can do whatever best suits their style.

***

Thoughts?

--
Brent Royal-Gordon
Architechies

Brent_Royal-Gordon · February 21, 2017, 1:38am

Ten seconds after sending this, I realized I forgot an important caveat: I don't expect this to be a feature of Swift 4, no matter how much we might want it. I just think that having a design in mind may help inform our thinking about access modifiers.

···

On Feb 20, 2017, at 5:36 PM, Brent Royal-Gordon <brent@architechies.com> wrote:

Okay, lots of people want to have some kind of submodule feature, so I'd like to sketch one out so we can hopefully agree on what submodules might look like.

--
Brent Royal-Gordon
Architechies

codafi · February 21, 2017, 1:47am

You’ll be delighted to know, then, that I’ve been thinking about this for a few weeks now and have a draft proposal that will be submitted for discussion shortly. I believe this can be an additive feature and still preserve all the goodness you would expect of a real module system.

~Robert Widmann

···

On Feb 20, 2017, at 8:36 PM, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> wrote:

Okay, lots of people want to have some kind of submodule feature, so I'd like to sketch one out so we can hopefully agree on what submodules might look like.

***

Any group of Swift files can be grouped together to form a submodule. Submodules belong within a particular module, and have a dotted name: If `ModKit` is a module, it might have a submodule called `ModKit.Foo`. Submodules can be nested within one another: `ModKit.Foo.Bar` is a submodule of `ModKit.Foo`, which is a submodule of `ModKit`.

No new access levels are necessary. `internal` APIs are only visible within the submodule they're declared in; a module cannot see its submodules' `internal` APIs, and a submodule cannot see its parent module's `internal` APIs. If a submodule wants to expose some of its APIs to its parent or sibling modules, it must mark them as `public` or `open`. Then they can import the submodule to see its APIs:

  import ModKit.Foo

By default, outside modules cannot import a submodule. But an import in the parent module can be decorated by an access control keyword to allow that:

  /// Any module outside ModKit can import ModKit.Foo and access its `public` and `open` APIs.
  open import ModKit.Foo

  /// Any module outside ModKit can import ModKit.Foo and access its `public` and `open` APIs,
  /// except `open` APIs are treated as `public`.
  public import ModKit.Foo

Imports may also be decorated by the `@exported` attribute, which exposes the submodule's APIs as though they were parent module APIs:

  @exported open import ModKit.Foo

  @exported public import ModKit.Foo

(This is sort of half-implemented already in a buggy `@_exported` attribute.)

Finally, the standard syntax for importing individual symbols can be used to cherry-pick types to treat differently:

  // Most ModKit.Foo APIs are not importable...
  import ModKit.Foo

  // ...but SomeEnum can be imported as public...
  public import enum ModKit.Foo.SomeEnum

  // ...SomeClass can be imported as open...
  open import class ModKit.Foo.SomeClass

  // And ImportantStruct will import whenever you import ModKit.
  @exported public import struct ModKit.Foo.ImportantStruct

(This syntax should be enhanced to allow cherry-picked importing of global functions, constants, and variables.)

If there are several different `import`s covering the same submodule or submodule symbol, the most permissive one wins.

(In large projects, `public`, `open`, and `@exported` imports will most likely all be put in a single Policy.swift file or something, but this is not enforced by the language.)

A submodule may not import any direct parent module (parent, grandparent, etc.), but may import any other submodule in the same module. This list shows permitted imports for a project with four modules/submodules:

  ModKit
    - ModKit.Foo
    - ModKit.Foo.Bar
    - ModKit.Quux
  ModKit.Foo
    - ModKit.Foo.Bar
    - ModKit.Quux
  ModKit.Foo.Bar
    - ModKit.Quux
  ModKit.Quux
    - ModKit.Foo
    - ModKit.Foo.Bar

However, submodules may not form circular dependencies through imports—if `ModKit.Quux` imports `ModKit.Foo`, then `ModKit.Foo` cannot import `ModKit.Quux`. The `#if canImport()` feature cannot be used to probe for other submodules within the same top-level module you're in.

At the compiler driver level, a submodule is specified by giving a `-module-name` parameter with a dot in it. When a file is compiled, only the filenames of the other .swift files in the same module are specified, along with .o files for any submodules; then all the .o files within that submodule are linked into a single .o file for the whole submodule. So files in `ModKit.Foo` would be compiled with only the .swift files in `ModKit.Foo` and the .o file for `ModKit.Foo.Bar`; then all the `ModKit.Foo` .o files would be linked into one .o file for the top-level `ModKit` to use. None of `ModKit.Foo`'s .swift files would be included in the command line when compiling the top-level `ModKit` module.

(That bit is kind of speculative—particularly the idea of linking submodule files into a single .o file—but I think something like what I'm describing could work.)

Because the compiler driver is used to group submodules together, Xcode can specify submodules in project file metadata and calculate a submodule dependency graph, while SwiftPM can use folders and compile submodules whenever the compiler emits an error indicating that a file tried to import a nonexistent submodule. Other build systems can do whatever best suits their style.

***

Thoughts?

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

anandabits · February 21, 2017, 3:59am

Okay, lots of people want to have some kind of submodule feature, so I'd like to sketch one out so we can hopefully agree on what submodules might look like.

***

Any group of Swift files can be grouped together to form a submodule.

It isn't clear to me how you express that a particular group of files forms a submodule. Can you elaborate?

···

Sent from my iPad

On Feb 20, 2017, at 7:36 PM, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> wrote:

Submodules belong within a particular module, and have a dotted name: If `ModKit` is a module, it might have a submodule called `ModKit.Foo`. Submodules can be nested within one another: `ModKit.Foo.Bar` is a submodule of `ModKit.Foo`, which is a submodule of `ModKit`.

No new access levels are necessary. `internal` APIs are only visible within the submodule they're declared in; a module cannot see its submodules' `internal` APIs, and a submodule cannot see its parent module's `internal` APIs. If a submodule wants to expose some of its APIs to its parent or sibling modules, it must mark them as `public` or `open`. Then they can import the submodule to see its APIs:

   import ModKit.Foo

By default, outside modules cannot import a submodule. But an import in the parent module can be decorated by an access control keyword to allow that:

   /// Any module outside ModKit can import ModKit.Foo and access its `public` and `open` APIs.
   open import ModKit.Foo

   /// Any module outside ModKit can import ModKit.Foo and access its `public` and `open` APIs,
   /// except `open` APIs are treated as `public`.
   public import ModKit.Foo

Imports may also be decorated by the `@exported` attribute, which exposes the submodule's APIs as though they were parent module APIs:

   @exported open import ModKit.Foo

   @exported public import ModKit.Foo

(This is sort of half-implemented already in a buggy `@_exported` attribute.)

Finally, the standard syntax for importing individual symbols can be used to cherry-pick types to treat differently:

   // Most ModKit.Foo APIs are not importable...
   import ModKit.Foo

   // ...but SomeEnum can be imported as public...
   public import enum ModKit.Foo.SomeEnum

   // ...SomeClass can be imported as open...
   open import class ModKit.Foo.SomeClass

   // And ImportantStruct will import whenever you import ModKit.
   @exported public import struct ModKit.Foo.ImportantStruct

(This syntax should be enhanced to allow cherry-picked importing of global functions, constants, and variables.)

If there are several different `import`s covering the same submodule or submodule symbol, the most permissive one wins.

(In large projects, `public`, `open`, and `@exported` imports will most likely all be put in a single Policy.swift file or something, but this is not enforced by the language.)

A submodule may not import any direct parent module (parent, grandparent, etc.), but may import any other submodule in the same module. This list shows permitted imports for a project with four modules/submodules:

   ModKit
       - ModKit.Foo
       - ModKit.Foo.Bar
       - ModKit.Quux
   ModKit.Foo
       - ModKit.Foo.Bar
       - ModKit.Quux
   ModKit.Foo.Bar
       - ModKit.Quux
   ModKit.Quux
       - ModKit.Foo
       - ModKit.Foo.Bar

However, submodules may not form circular dependencies through imports—if `ModKit.Quux` imports `ModKit.Foo`, then `ModKit.Foo` cannot import `ModKit.Quux`. The `#if canImport()` feature cannot be used to probe for other submodules within the same top-level module you're in.

At the compiler driver level, a submodule is specified by giving a `-module-name` parameter with a dot in it. When a file is compiled, only the filenames of the other .swift files in the same module are specified, along with .o files for any submodules; then all the .o files within that submodule are linked into a single .o file for the whole submodule. So files in `ModKit.Foo` would be compiled with only the .swift files in `ModKit.Foo` and the .o file for `ModKit.Foo.Bar`; then all the `ModKit.Foo` .o files would be linked into one .o file for the top-level `ModKit` to use. None of `ModKit.Foo`'s .swift files would be included in the command line when compiling the top-level `ModKit` module.

(That bit is kind of speculative—particularly the idea of linking submodule files into a single .o file—but I think something like what I'm describing could work.)

Because the compiler driver is used to group submodules together, Xcode can specify submodules in project file metadata and calculate a submodule dependency graph, while SwiftPM can use folders and compile submodules whenever the compiler emits an error indicating that a file tried to import a nonexistent submodule. Other build systems can do whatever best suits their style.

***

Thoughts?

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Rien · February 21, 2017, 7:20am

Maybe we should try to collect what people want from submodules first.

I wanted modules for organisational purposes, however with the arrival of SPM that need has been almost completely removed. At least to the point that I do not feel that they are absolutely necessary.

Have the people who want modules tried SPM yet?

PS: be prepared for additional access levels requests when modules are available… :-0

Regards,
Rien

Site: http://balancingrock.nl
Blog: http://swiftrien.blogspot.com
Github: Balancingrock (Rien) · GitHub
Project: http://swiftfire.nl

···

On 21 Feb 2017, at 02:36, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> wrote:

Okay, lots of people want to have some kind of submodule feature, so I'd like to sketch one out so we can hopefully agree on what submodules might look like.

***

Any group of Swift files can be grouped together to form a submodule. Submodules belong within a particular module, and have a dotted name: If `ModKit` is a module, it might have a submodule called `ModKit.Foo`. Submodules can be nested within one another: `ModKit.Foo.Bar` is a submodule of `ModKit.Foo`, which is a submodule of `ModKit`.

No new access levels are necessary. `internal` APIs are only visible within the submodule they're declared in; a module cannot see its submodules' `internal` APIs, and a submodule cannot see its parent module's `internal` APIs. If a submodule wants to expose some of its APIs to its parent or sibling modules, it must mark them as `public` or `open`. Then they can import the submodule to see its APIs:

  import ModKit.Foo

By default, outside modules cannot import a submodule. But an import in the parent module can be decorated by an access control keyword to allow that:

  /// Any module outside ModKit can import ModKit.Foo and access its `public` and `open` APIs.
  open import ModKit.Foo

  /// Any module outside ModKit can import ModKit.Foo and access its `public` and `open` APIs,
  /// except `open` APIs are treated as `public`.
  public import ModKit.Foo

Imports may also be decorated by the `@exported` attribute, which exposes the submodule's APIs as though they were parent module APIs:

  @exported open import ModKit.Foo

  @exported public import ModKit.Foo

(This is sort of half-implemented already in a buggy `@_exported` attribute.)

Finally, the standard syntax for importing individual symbols can be used to cherry-pick types to treat differently:

  // Most ModKit.Foo APIs are not importable...
  import ModKit.Foo

  // ...but SomeEnum can be imported as public...
  public import enum ModKit.Foo.SomeEnum

  // ...SomeClass can be imported as open...
  open import class ModKit.Foo.SomeClass

  // And ImportantStruct will import whenever you import ModKit.
  @exported public import struct ModKit.Foo.ImportantStruct

(This syntax should be enhanced to allow cherry-picked importing of global functions, constants, and variables.)

If there are several different `import`s covering the same submodule or submodule symbol, the most permissive one wins.

(In large projects, `public`, `open`, and `@exported` imports will most likely all be put in a single Policy.swift file or something, but this is not enforced by the language.)

A submodule may not import any direct parent module (parent, grandparent, etc.), but may import any other submodule in the same module. This list shows permitted imports for a project with four modules/submodules:

  ModKit
    - ModKit.Foo
    - ModKit.Foo.Bar
    - ModKit.Quux
  ModKit.Foo
    - ModKit.Foo.Bar
    - ModKit.Quux
  ModKit.Foo.Bar
    - ModKit.Quux
  ModKit.Quux
    - ModKit.Foo
    - ModKit.Foo.Bar

However, submodules may not form circular dependencies through imports—if `ModKit.Quux` imports `ModKit.Foo`, then `ModKit.Foo` cannot import `ModKit.Quux`. The `#if canImport()` feature cannot be used to probe for other submodules within the same top-level module you're in.

At the compiler driver level, a submodule is specified by giving a `-module-name` parameter with a dot in it. When a file is compiled, only the filenames of the other .swift files in the same module are specified, along with .o files for any submodules; then all the .o files within that submodule are linked into a single .o file for the whole submodule. So files in `ModKit.Foo` would be compiled with only the .swift files in `ModKit.Foo` and the .o file for `ModKit.Foo.Bar`; then all the `ModKit.Foo` .o files would be linked into one .o file for the top-level `ModKit` to use. None of `ModKit.Foo`'s .swift files would be included in the command line when compiling the top-level `ModKit` module.

(That bit is kind of speculative—particularly the idea of linking submodule files into a single .o file—but I think something like what I'm describing could work.)

Because the compiler driver is used to group submodules together, Xcode can specify submodules in project file metadata and calculate a submodule dependency graph, while SwiftPM can use folders and compile submodules whenever the compiler emits an error indicating that a file tried to import a nonexistent submodule. Other build systems can do whatever best suits their style.

***

Thoughts?

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

anandabits · February 22, 2017, 4:33am

Okay, lots of people want to have some kind of submodule feature, so I'd like to sketch one out so we can hopefully agree on what submodules might look like.

***

Any group of Swift files can be grouped together to form a submodule. Submodules belong within a particular module, and have a dotted name: If `ModKit` is a module, it might have a submodule called `ModKit.Foo`. Submodules can be nested within one another: `ModKit.Foo.Bar` is a submodule of `ModKit.Foo`, which is a submodule of `ModKit`.

No new access levels are necessary. `internal` APIs are only visible within the submodule they're declared in; a module cannot see its submodules' `internal` APIs, and a submodule cannot see its parent module's `internal` APIs. If a submodule wants to expose some of its APIs to its parent or sibling modules, it must mark them as `public` or `open`. Then they can import the submodule to see its APIs:

  import ModKit.Foo

By default, outside modules cannot import a submodule. But an import in the parent module can be decorated by an access control keyword to allow that:

  /// Any module outside ModKit can import ModKit.Foo and access its `public` and `open` APIs.
  open import ModKit.Foo

  /// Any module outside ModKit can import ModKit.Foo and access its `public` and `open` APIs,
  /// except `open` APIs are treated as `public`.
  public import ModKit.Foo

Imports may also be decorated by the `@exported` attribute, which exposes the submodule's APIs as though they were parent module APIs:

  @exported open import ModKit.Foo

  @exported public import ModKit.Foo

(This is sort of half-implemented already in a buggy `@_exported` attribute.)

Finally, the standard syntax for importing individual symbols can be used to cherry-pick types to treat differently:

  // Most ModKit.Foo APIs are not importable...
  import ModKit.Foo

  // ...but SomeEnum can be imported as public...
  public import enum ModKit.Foo.SomeEnum

  // ...SomeClass can be imported as open...
  open import class ModKit.Foo.SomeClass

  // And ImportantStruct will import whenever you import ModKit.
  @exported public import struct ModKit.Foo.ImportantStruct

(This syntax should be enhanced to allow cherry-picked importing of global functions, constants, and variables.)

If there are several different `import`s covering the same submodule or submodule symbol, the most permissive one wins.

(In large projects, `public`, `open`, and `@exported` imports will most likely all be put in a single Policy.swift file or something, but this is not enforced by the language.)

A submodule may not import any direct parent module (parent, grandparent, etc.), but may import any other submodule in the same module. This list shows permitted imports for a project with four modules/submodules:

  ModKit
    - ModKit.Foo
    - ModKit.Foo.Bar
    - ModKit.Quux
  ModKit.Foo
    - ModKit.Foo.Bar
    - ModKit.Quux
  ModKit.Foo.Bar
    - ModKit.Quux
  ModKit.Quux
    - ModKit.Foo
    - ModKit.Foo.Bar

Am I understanding this correctly? It looks like ModKit.Foo.Bar cannot see or import any symbols declared in ModKit.Foo or ModKit. Is that correct? Descendents cannot see or import symbols declared in ancestors (including public symbols) but they can import any submodule that is not their ancestor (and therefore see public symbols in any submodule that’s not an ancestor by way of importing that submodule)?

I think I understand why you might have specified this design but am interested in hearing you elaborate a bit further on the rationale.

···

On Feb 20, 2017, at 7:36 PM, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> wrote:

However, submodules may not form circular dependencies through imports—if `ModKit.Quux` imports `ModKit.Foo`, then `ModKit.Foo` cannot import `ModKit.Quux`. The `#if canImport()` feature cannot be used to probe for other submodules within the same top-level module you're in.

At the compiler driver level, a submodule is specified by giving a `-module-name` parameter with a dot in it. When a file is compiled, only the filenames of the other .swift files in the same module are specified, along with .o files for any submodules; then all the .o files within that submodule are linked into a single .o file for the whole submodule. So files in `ModKit.Foo` would be compiled with only the .swift files in `ModKit.Foo` and the .o file for `ModKit.Foo.Bar`; then all the `ModKit.Foo` .o files would be linked into one .o file for the top-level `ModKit` to use. None of `ModKit.Foo`'s .swift files would be included in the command line when compiling the top-level `ModKit` module.

(That bit is kind of speculative—particularly the idea of linking submodule files into a single .o file—but I think something like what I'm describing could work.)

Because the compiler driver is used to group submodules together, Xcode can specify submodules in project file metadata and calculate a submodule dependency graph, while SwiftPM can use folders and compile submodules whenever the compiler emits an error indicating that a file tried to import a nonexistent submodule. Other build systems can do whatever best suits their style.

***

Thoughts?

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Derrick_Ho · February 21, 2017, 1:40am

Oh, I thought this would be another discussion about namespaces.

···

On Mon, Feb 20, 2017 at 8:39 PM Brent Royal-Gordon via swift-evolution < swift-evolution@swift.org> wrote:

> On Feb 20, 2017, at 5:36 PM, Brent Royal-Gordon <brent@architechies.com> > wrote:
>
> Okay, lots of people want to have some kind of submodule feature, so I'd
like to sketch one out so we can hopefully agree on what submodules might
look like.

Ten seconds after sending this, I realized I forgot an important caveat: I
don't expect this to be a feature of Swift 4, no matter how much we might
want it. I just think that having a design in mind may help inform our
thinking about access modifiers.

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Brent_Royal-Gordon · February 21, 2017, 10:01am

I discuss this later, but maybe don't describe it that clearly. Basically, submodule membership is not directly expressed in the source code; rather, it's a function of how the build system invokes the compiler.

To understand what I'm suggesting, you must first understand how the Swift compiler is invoked to build code today. (The following is very simplified—there are some steps I've left out—but it's basically accurate.) Suppose your app, `PersonTalk`, has four files:

  AppDelegate.swift
  RootViewController.swift
  Person.swift
  Message.swift

When Xcode or SwiftPM or whatever goes to compile your app, the build system will invoke the compiler once for each file, passing *all* of the files but specifying only one as primary:

  swiftc -emit-object -module-name PersonTalk -primary-file AppDelegate.swift RootViewController.swift Person.swift Message.swift ...
  swiftc -emit-object -module-name PersonTalk AppDelegate.swift -primary-file RootViewController.swift Person.swift Message.swift ...
  swiftc -emit-object -module-name PersonTalk AppDelegate.swift RootViewController.swift -primary-file Person.swift Message.swift ...
  swiftc -emit-object -module-name PersonTalk AppDelegate.swift RootViewController.swift Person.swift -primary-file Message.swift ...

Each of these compiler invocations compiles its primary file into a .o file. The build system then asks Clang to link them all together, and you get a fully built module of some sort:

clang -o PersonTalk AppDelegate.o RootViewController.o Person.o Message.o …

In my proposal, if you wanted Person.swift and Message.swift to be in a `PersonTalk.Model` submodule, you would do that by invoking the build tools in a slightly different way. First you would compile the two files in the submodule with one another:

swiftc -emit-object -module-name PersonTalk.Model -primary-file Person.swift Message.swift ...
swiftc -emit-object -module-name PersonTalk.Model Person.swift -primary-file Message.swift ...

Then you would emit a single PersonTalk.Model.o file encompassing both of them:

swiftc -emit-submodule -module-name PersonTalk.Model Person.o Message.o ...

Then you would compile the other two files, providing the PersonTalk.Model.o file as context:

swiftc -emit-object -module-name PersonTalk -primary-file AppDelegate.swift RootViewController.swift -with-submodule PersonTalk.Model.o ...
swiftc -emit-object -module-name PersonTalk AppDelegate.swift -primary-file RootViewController.swift -with-submodule PersonTalk.Model.o ...

And finally, you would link everything together:

clang -o PersonTalk AppDelegate.o RootViewController.o PersonTalk.Model.o …

So whether files belong to a submodule or the main module is not a property of the files themselves, but rather of how the build system chooses to combine them during compilation. That leaves the build system free to specify submodules in a way appropriate to its design.

I assume that Xcode would group files into submodules with per-file or per-target metadata. For instance, there might be a "Submodule" field in the File inspector of each Swift source file, or an additional tab in the Project editor. Upon building, it would examine all its metadata, create a list of submodules and files in each one, examine each submodule to determine its dependencies, order the submodules so that each one's dependencies will be available, and invoke the compiler dance properly to build everything.

SwiftPM, which relies mainly on the directory structure to organize your code, would probably have each submodule be a subdirectory of the Sources directory, using the presence of dots in the filename to match submodules:

  Package.swift
  Sources/
    PersonTalk/
      AppDelegate.swift
      RootViewController.swift
    PersonTalk.Model/
      Person.swift
      Message.swift
    TalkKit/
      Talker.swift
      TalkUser.swift
      TalkMessage.swift

Or perhaps it would use nested subdirectories to represent the submodules:

  Package.swift
  Sources/
    PersonTalk/
      AppDelegate.swift
      RootViewController.swift
      Model/
        Person.swift
        Message.swift
    TalkKit/
      Talker.swift
      TalkUser.swift
      TalkMessage.swift

In either case, I imagine that SwiftPM would try to build `PersonTalk`, detect the error indicating that the `PersonTalk.Model` submodule was not available, recurse in order to build `PersonTalk.Model`, and then try to build `PersonTalk` again.

Other build systems would do whatever made sense for them. If you were using `make`, for instance, you'd probably have either a target for each submodule, or a directory and `Makefile` for each submodule. If you used Ant, you'd write some sort of XML horror. The point is, specifying which files belong to which submodules is a build system responsibility, not a language responsibility. This keeps source code portable, avoids boilerplate, and keeps submodule information from being duplicated (and thus potentially becoming inconsistent) in file system structure or build system metadata.

Hope this helps,

···

On Feb 20, 2017, at 7:59 PM, Matthew Johnson <matthew@anandabits.com> wrote:

Any group of Swift files can be grouped together to form a submodule.

It isn't clear to me how you express that a particular group of files forms a submodule. Can you elaborate?

--
Brent Royal-Gordon
Architechies

Karl · February 21, 2017, 2:33am

That’s an interesting proposal. Here are the issues I see:

- I don’t like nesting everything within a large “module” block. I would like a top-level “@module Foo” declaration for the entire file. Also, since any sub-modules would be nested within the implicit top-level module, I’d prefer the keyword “submodule”. Otherwise, what happens if I write:

module Foundation {
// Is this a module-extension? Can I insert new types or top-level functions in to Foundation?
// Am I creating a new top-level module named “Foundation”?
}

Looking at this thread:

- There are certainly issues with access control of submodules. OTOH, I don’t really like the idea that one incorrect import in one file can expose an entire submodule as part of your module’s API.

More generally:

- Access control of submodules seems like something which could benefit from a single source of truth.
- Can submodules have independent versions? Where would we declare them?

I wonder if we should have something like “module manifest” for all of a library’s public submodules, similar to SwiftPM's package manifest. So, strawman syntax…

Module(Foo, version: 1.2.2, description: “A library for foo-ing around with”, submodules: [
Module(Foo.Maths, version: 1.5.0, description: “A maths library supporting Foo”),
Module(Foo.Formatters, version: 1.5.0, description: “Formatters for Foo types”)
])

So, in this case we had a module Foo, then we updated FooMaths with some new APIs (say some new operations/types were added, and we updated FooFormatters accordingly). However, the API of Foo itself hasn’t changed; it’s the same as it was ages ago. Would that be possible?

···

On 21 Feb 2017, at 02:47, Robert Widmann via swift-evolution <swift-evolution@swift.org> wrote:

You’ll be delighted to know, then, that I’ve been thinking about this for a few weeks now and have a draft proposal that will be submitted for discussion shortly. I believe this can be an additive feature and still preserve all the goodness you would expect of a real module system.

~Robert Widmann

On Feb 20, 2017, at 8:36 PM, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> wrote:

Okay, lots of people want to have some kind of submodule feature, so I'd like to sketch one out so we can hopefully agree on what submodules might look like.

***

Any group of Swift files can be grouped together to form a submodule. Submodules belong within a particular module, and have a dotted name: If `ModKit` is a module, it might have a submodule called `ModKit.Foo`. Submodules can be nested within one another: `ModKit.Foo.Bar` is a submodule of `ModKit.Foo`, which is a submodule of `ModKit`.

No new access levels are necessary. `internal` APIs are only visible within the submodule they're declared in; a module cannot see its submodules' `internal` APIs, and a submodule cannot see its parent module's `internal` APIs. If a submodule wants to expose some of its APIs to its parent or sibling modules, it must mark them as `public` or `open`. Then they can import the submodule to see its APIs:

  import ModKit.Foo

By default, outside modules cannot import a submodule. But an import in the parent module can be decorated by an access control keyword to allow that:

  /// Any module outside ModKit can import ModKit.Foo and access its `public` and `open` APIs.
  open import ModKit.Foo

  /// Any module outside ModKit can import ModKit.Foo and access its `public` and `open` APIs,
  /// except `open` APIs are treated as `public`.
  public import ModKit.Foo

Imports may also be decorated by the `@exported` attribute, which exposes the submodule's APIs as though they were parent module APIs:

  @exported open import ModKit.Foo

  @exported public import ModKit.Foo

(This is sort of half-implemented already in a buggy `@_exported` attribute.)

Finally, the standard syntax for importing individual symbols can be used to cherry-pick types to treat differently:

  // Most ModKit.Foo APIs are not importable...
  import ModKit.Foo

  // ...but SomeEnum can be imported as public...
  public import enum ModKit.Foo.SomeEnum

  // ...SomeClass can be imported as open...
  open import class ModKit.Foo.SomeClass

  // And ImportantStruct will import whenever you import ModKit.
  @exported public import struct ModKit.Foo.ImportantStruct

(This syntax should be enhanced to allow cherry-picked importing of global functions, constants, and variables.)

If there are several different `import`s covering the same submodule or submodule symbol, the most permissive one wins.

(In large projects, `public`, `open`, and `@exported` imports will most likely all be put in a single Policy.swift file or something, but this is not enforced by the language.)

A submodule may not import any direct parent module (parent, grandparent, etc.), but may import any other submodule in the same module. This list shows permitted imports for a project with four modules/submodules:

  ModKit
    - ModKit.Foo
    - ModKit.Foo.Bar
    - ModKit.Quux
  ModKit.Foo
    - ModKit.Foo.Bar
    - ModKit.Quux
  ModKit.Foo.Bar
    - ModKit.Quux
  ModKit.Quux
    - ModKit.Foo
    - ModKit.Foo.Bar

However, submodules may not form circular dependencies through imports—if `ModKit.Quux` imports `ModKit.Foo`, then `ModKit.Foo` cannot import `ModKit.Quux`. The `#if canImport()` feature cannot be used to probe for other submodules within the same top-level module you're in.

At the compiler driver level, a submodule is specified by giving a `-module-name` parameter with a dot in it. When a file is compiled, only the filenames of the other .swift files in the same module are specified, along with .o files for any submodules; then all the .o files within that submodule are linked into a single .o file for the whole submodule. So files in `ModKit.Foo` would be compiled with only the .swift files in `ModKit.Foo` and the .o file for `ModKit.Foo.Bar`; then all the `ModKit.Foo` .o files would be linked into one .o file for the top-level `ModKit` to use. None of `ModKit.Foo`'s .swift files would be included in the command line when compiling the top-level `ModKit` module.

(That bit is kind of speculative—particularly the idea of linking submodule files into a single .o file—but I think something like what I'm describing could work.)

Because the compiler driver is used to group submodules together, Xcode can specify submodules in project file metadata and calculate a submodule dependency graph, while SwiftPM can use folders and compile submodules whenever the compiler emits an error indicating that a file tried to import a nonexistent submodule. Other build systems can do whatever best suits their style.

***

Thoughts?

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Brent_Royal-Gordon · February 22, 2017, 7:31am

A submodule may not import any direct parent module (parent, grandparent, etc.), but may import any other submodule in the same module. This list shows permitted imports for a project with four modules/submodules:

  ModKit
    - ModKit.Foo
    - ModKit.Foo.Bar
    - ModKit.Quux
  ModKit.Foo
    - ModKit.Foo.Bar
    - ModKit.Quux
  ModKit.Foo.Bar
    - ModKit.Quux
  ModKit.Quux
    - ModKit.Foo
    - ModKit.Foo.Bar

Am I understanding this correctly? It looks like ModKit.Foo.Bar cannot see or import any symbols declared in ModKit.Foo or ModKit. Is that correct? Descendents cannot see or import symbols declared in ancestors (including public symbols) but they can import any submodule that is not their ancestor (and therefore see public symbols in any submodule that’s not an ancestor by way of importing that submodule)?

Yes, that's correct.

I think I understand why you might have specified this design but am interested in hearing you elaborate a bit further on the rationale.

A few reasons:

1. It creates a stricter layering between parent modules and submodules: submodules are for lower layers, while parent modules are for higher layers. Layering reduces spaghetti, and less spaghetti is generally a good thing.

2. It limits the amount of code the compiler has to consider at once.

3. It helps avoid (but doesn't completely prevent) circular dependencies between submodules, ensuring there's an order in which they can be compiled separately.

On the other hand, a design which considered all submodules at once would certainly be more flexible. You could call back and forth between submodules and their parents freely, and organize your code more flexibly. But I'm not sure it can be made to work with this "build system decides what's in a submodule" approach—I'm trying to imagine how you would tell the compiler about the submodule mappings for fifty files without that information being in the source code, and I'm not liking any of the solutions I'm coming up with.

···

On Feb 21, 2017, at 8:33 PM, Matthew Johnson <matthew@anandabits.com> wrote:

--
Brent Royal-Gordon
Architechies

anandabits · February 21, 2017, 1:24pm

Any group of Swift files can be grouped together to form a submodule.

It isn't clear to me how you express that a particular group of files forms a submodule. Can you elaborate?

I discuss this later, but maybe don't describe it that clearly. Basically, submodule membership is not directly expressed in the source code; rather, it's a function of how the build system invokes the compiler.

To understand what I'm suggesting, you must first understand how the Swift compiler is invoked to build code today. (The following is very simplified—there are some steps I've left out—but it's basically accurate.) Suppose your app, `PersonTalk`, has four files:

   AppDelegate.swift
   RootViewController.swift
   Person.swift
   Message.swift

When Xcode or SwiftPM or whatever goes to compile your app, the build system will invoke the compiler once for each file, passing *all* of the files but specifying only one as primary:

   swiftc -emit-object -module-name PersonTalk -primary-file AppDelegate.swift RootViewController.swift Person.swift Message.swift ...
   swiftc -emit-object -module-name PersonTalk AppDelegate.swift -primary-file RootViewController.swift Person.swift Message.swift ...
   swiftc -emit-object -module-name PersonTalk AppDelegate.swift RootViewController.swift -primary-file Person.swift Message.swift ...
   swiftc -emit-object -module-name PersonTalk AppDelegate.swift RootViewController.swift Person.swift -primary-file Message.swift ...

Each of these compiler invocations compiles its primary file into a .o file. The build system then asks Clang to link them all together, and you get a fully built module of some sort:

   clang -o PersonTalk AppDelegate.o RootViewController.o Person.o Message.o …

In my proposal, if you wanted Person.swift and Message.swift to be in a `PersonTalk.Model` submodule, you would do that by invoking the build tools in a slightly different way. First you would compile the two files in the submodule with one another:

   swiftc -emit-object -module-name PersonTalk.Model -primary-file Person.swift Message.swift ...
   swiftc -emit-object -module-name PersonTalk.Model Person.swift -primary-file Message.swift ...

Then you would emit a single PersonTalk.Model.o file encompassing both of them:

   swiftc -emit-submodule -module-name PersonTalk.Model Person.o Message.o ...

Then you would compile the other two files, providing the PersonTalk.Model.o file as context:

   swiftc -emit-object -module-name PersonTalk -primary-file AppDelegate.swift RootViewController.swift -with-submodule PersonTalk.Model.o ...
   swiftc -emit-object -module-name PersonTalk AppDelegate.swift -primary-file RootViewController.swift -with-submodule PersonTalk.Model.o ...

And finally, you would link everything together:

   clang -o PersonTalk AppDelegate.o RootViewController.o PersonTalk.Model.o …

So whether files belong to a submodule or the main module is not a property of the files themselves, but rather of how the build system chooses to combine them during compilation. That leaves the build system free to specify submodules in a way appropriate to its design.

Hmm interesting. I'll have to give this further thought but I can see how this approach could have advantages. Would WMO still work across the whole module or would it be restricted to individual submodules? I think we would definitely want to see it work across all submodules. I think this would work because it runs during linking no, right?

I assume that Xcode would group files into submodules with per-file or per-target metadata. For instance, there might be a "Submodule" field in the File inspector of each Swift source file, or an additional tab in the Project editor. Upon building, it would examine all its metadata, create a list of submodules and files in each one, examine each submodule to determine its dependencies, order the submodules so that each one's dependencies will be available, and invoke the compiler dance properly to build everything.

SwiftPM, which relies mainly on the directory structure to organize your code, would probably have each submodule be a subdirectory of the Sources directory, using the presence of dots in the filename to match submodules:

   Package.swift
   Sources/
       PersonTalk/
           AppDelegate.swift
           RootViewController.swift
       PersonTalk.Model/
           Person.swift
           Message.swift
       TalkKit/
           Talker.swift
           TalkUser.swift
           TalkMessage.swift

Or perhaps it would use nested subdirectories to represent the submodules:

   Package.swift
   Sources/
       PersonTalk/
           AppDelegate.swift
           RootViewController.swift
           Model/
               Person.swift
               Message.swift
       TalkKit/
           Talker.swift
           TalkUser.swift
           TalkMessage.swift

In either case, I imagine that SwiftPM would try to build `PersonTalk`, detect the error indicating that the `PersonTalk.Model` submodule was not available, recurse in order to build `PersonTalk.Model`, and then try to build `PersonTalk` again.

Other build systems would do whatever made sense for them. If you were using `make`, for instance, you'd probably have either a target for each submodule, or a directory and `Makefile` for each submodule. If you used Ant, you'd write some sort of XML horror. The point is, specifying which files belong to which submodules is a build system responsibility, not a language responsibility. This keeps source code portable, avoids boilerplate, and keeps submodule information from being duplicated (and thus potentially becoming inconsistent) in file system structure or build system metadata.

Like I said earlier, I'll have to give this more thought. In some ways I think it fits pretty well with what I want out of submodules.

The downside is that it's potentially harder to move code between build systems or have it work with multiple build systems simultaneously (i.e. Xcode and SPM). Have you given any thought to how to address this? It would be unfortunate to have to duplicate submodule structure information for each build system.

Hope this helps,

Very much, thanks!

···

Sent from my iPad
On Feb 21, 2017, at 4:01 AM, Brent Royal-Gordon <brent@architechies.com> wrote:

On Feb 20, 2017, at 7:59 PM, Matthew Johnson <matthew@anandabits.com> wrote:

--
Brent Royal-Gordon
Architechies

Alvae · February 21, 2017, 11:18am

Maybe we should try to collect what people want from submodules first.

I’d like a simple method to group my sources files into some kind lexical scope. Nothing more actually =D

The point is, specifying which files belong to which submodules is a build system responsibility, not a language responsibility. This keeps source code portable, avoids boilerplate, and keeps submodule information from being duplicated (and thus potentially becoming inconsistent) in file system structure or build system metadata.

Although I think this assertion makes a lot of sense, wouldn’t it be better for the language to be a bit more opinionated about that? The problem I foresee is when people will start making frameworks with Xcode while others will opt for SwiftPM. I guess compatibility between the two default architectures could easily become hellish. For instance, building/linking C/C++ libraries is a hell as soon as you have more than a single dependency. Everybody has another system to do build their library (make, cmake, autotools, Xcode, …), and people end up rewriting unified build configuration scripts for whatever tool they use.

Ideally, I’d prefer an opinionated submodule system that clearly distinguish what is part of a submodule from what is not, would it be filesystem-based (my personal favoured) or meta-data based by the means of a manifest or some other directives within the source files (my personal least favoured).

I like filesystem-based approaches as they are simple to teach and simple to put in place, with and IDE **as well as** with a simple shell. I personally find Python’s approach to be very successful in that regard. The only improvement I would like for Swift is not having to write a dozen of `import foo.bar.baz.koala` at the beginning of my sources. Maybe that could be achieved if importing one module would also make all the public entities of its submodules visible.

I wonder if we should have something like “module manifest” for all of a library’s public submodules, similar to SwiftPM's package manifest. So, strawman syntax…

Writing manifest files is tedious and error-prone when done by hand, I’ll also add that the usability of language shouldn’t depend on some IDE (yes I’m looking at you Java). Writing one and one only manifest for the top-level module is fine and makes sense (specifying targets, dependencies, ...). But I’d argue that writing one for every submodule would be annoying to create and maintain. Also, I’m not convinced that everything need that much information. The multiple versions of a submodule are probably already stored in some versioning tool and the description will most often be a wordy version of the module’s name.

Have the people who want modules tried SPM yet?

Yes I did and to be honest it is close to a perfect solution for me. It’s mostly filesystem-based, easy to teach and easy put in place. In my dreams, submodules would just be an easier way to bundle a bunch of sources together, than having to manually link modules via a dependency list in their respective in manifest files.

Best regards,
Dimitri

anandabits · February 22, 2017, 3:16pm

A submodule may not import any direct parent module (parent, grandparent, etc.), but may import any other submodule in the same module. This list shows permitted imports for a project with four modules/submodules:

  ModKit
    - ModKit.Foo
    - ModKit.Foo.Bar
    - ModKit.Quux
  ModKit.Foo
    - ModKit.Foo.Bar
    - ModKit.Quux
  ModKit.Foo.Bar
    - ModKit.Quux
  ModKit.Quux
    - ModKit.Foo
    - ModKit.Foo.Bar

Am I understanding this correctly? It looks like ModKit.Foo.Bar cannot see or import any symbols declared in ModKit.Foo or ModKit. Is that correct? Descendents cannot see or import symbols declared in ancestors (including public symbols) but they can import any submodule that is not their ancestor (and therefore see public symbols in any submodule that’s not an ancestor by way of importing that submodule)?

Yes, that's correct.

I think I understand why you might have specified this design but am interested in hearing you elaborate a bit further on the rationale.

A few reasons:

1. It creates a stricter layering between parent modules and submodules: submodules are for lower layers, while parent modules are for higher layers. Layering reduces spaghetti, and less spaghetti is generally a good thing.

2. It limits the amount of code the compiler has to consider at once.

3. It helps avoid (but doesn't completely prevent) circular dependencies between submodules, ensuring there's an order in which they can be compiled separately.

These are the reasons I suspected. Glad we’re on the same page.

On the other hand, a design which considered all submodules at once would certainly be more flexible. You could call back and forth between submodules and their parents freely, and organize your code more flexibly. But I'm not sure it can be made to work with this "build system decides what's in a submodule" approach—I'm trying to imagine how you would tell the compiler about the submodule mappings for fifty files without that information being in the source code, and I'm not liking any of the solutions I'm coming up with.

Yes, this requirement does seem to be implied by the build system approach you suggested. For the sake of discussion, would you still desire these rules if the files in a submodule was determined with a mechanism that allowed for this flexibility. Let’s say for discussion even circular dependencies were possible to implement. What rules would you consider valuable enough to limit flexibility of code organization? I’m not arguing for maximum flexibility at all (I don’t think that would be the right design). But I am interested in whether you would change the rules without the limitation of the build system mechanism or not.

···

On Feb 22, 2017, at 1:31 AM, Brent Royal-Gordon <brent@architechies.com> wrote:

On Feb 21, 2017, at 8:33 PM, Matthew Johnson <matthew@anandabits.com> wrote:

--
Brent Royal-Gordon
Architechies

anandabits · February 21, 2017, 1:27pm

Maybe we should try to collect what people want from submodules first.

I’d like a simple method to group my sources files into some kind lexical scope. Nothing more actually =D

I think we have very similar desires for a submodule system. I'm goi to write up some thoughts as soon as I have a chance, but today is very busy so it may not be until tomorrow.

···

Sent from my iPad
On Feb 21, 2017, at 5:18 AM, Dimitri Racordon <Dimitri.Racordon@unige.ch> wrote:

The point is, specifying which files belong to which submodules is a build system responsibility, not a language responsibility. This keeps source code portable, avoids boilerplate, and keeps submodule information from being duplicated (and thus potentially becoming inconsistent) in file system structure or build system metadata.

Although I think this assertion makes a lot of sense, wouldn’t it be better for the language to be a bit more opinionated about that? The problem I foresee is when people will start making frameworks with Xcode while others will opt for SwiftPM. I guess compatibility between the two default architectures could easily become hellish. For instance, building/linking C/C++ libraries is a hell as soon as you have more than a single dependency. Everybody has another system to do build their library (make, cmake, autotools, Xcode, …), and people end up rewriting unified build configuration scripts for whatever tool they use.

Ideally, I’d prefer an opinionated submodule system that clearly distinguish what is part of a submodule from what is not, would it be filesystem-based (my personal favoured) or meta-data based by the means of a manifest or some other directives within the source files (my personal least favoured).

I like filesystem-based approaches as they are simple to teach and simple to put in place, with and IDE **as well as** with a simple shell. I personally find Python’s approach to be very successful in that regard. The only improvement I would like for Swift is not having to write a dozen of `import foo.bar.baz.koala` at the beginning of my sources. Maybe that could be achieved if importing one module would also make all the public entities of its submodules visible.

I wonder if we should have something like “module manifest” for all of a library’s public submodules, similar to SwiftPM's package manifest. So, strawman syntax…

Writing manifest files is tedious and error-prone when done by hand, I’ll also add that the usability of language shouldn’t depend on some IDE (yes I’m looking at you Java). Writing one and one only manifest for the top-level module is fine and makes sense (specifying targets, dependencies, ...). But I’d argue that writing one for every submodule would be annoying to create and maintain. Also, I’m not convinced that everything need that much information. The multiple versions of a submodule are probably already stored in some versioning tool and the description will most often be a wordy version of the module’s name.

Have the people who want modules tried SPM yet?

Yes I did and to be honest it is close to a perfect solution for me. It’s mostly filesystem-based, easy to teach and easy put in place. In my dreams, submodules would just be an easier way to bundle a bunch of sources together, than having to manually link modules via a dependency list in their respective in manifest files.

Best regards,
Dimitri

Nevin · February 21, 2017, 11:40pm

I think entities declared as “internal” should be visible throughout the
entire *module* just as they are today. In particular, if I write “struct
Foo {}” inside a submodule, then Foo should have internal visibility (the
default when no access level is specified) and thus should be available to
the entire module.

Similarly, if I write “public struct Bar {}” inside a submodule, then Bar
should be exported from the module and available to client code. In other
words, the submodule should exist to *organize* code, not to change its
meaning.

Moreover, the ongoing access level discussions elsewhere on the list make
clear that many people believe there should be just one visibility more
restricted than “internal” (which of course should be named “private”). If
we go that route, then “private” would mean “visible in this submodule
only”. For files which are not in a submodule this acts like “fileprivate”,
and for files which *are* in a submodule it means they can share their
implementation details with closely related code (usually types and
extensions) while allowing that code to be in separate files as appropriate.

That is most of what I want out of submodules: to provide all the benefits
of Swift-2-era “private” without requiring that everything be stuffed
inside one single file. For all other purposes a submodule would be
transparent, it just lets you split up a large file along its natural
divisions. Consequently, I see no need or purpose for nested submodules,
because a submodule simply “acts like” a single file.

Nevin

Jon_Hull · February 22, 2017, 7:11am

I think I could get behind this. It definitely takes care of the 80%.

It might even enable adding storage in extensions *within the submodule*

There is one major use case which I see all of these proposals failing to handle (although that may be intentional for some designs), is where you have something internal to the type, but you still want to allow subclasses/extensions outside the framework (which need that information) to access it. An actual example of this is UIGestureRecognizer. This class is designed to be subclassed outside of the framework, but it hides things like setting the ‘state' from client code (without hiding it from subclasses). This is important, because if a client sets the state directly, then it results in either undefined behavior, or an infinite loop. Other languages use ‘protected’ and ‘friend’ to accomplish this. I am wondering if there is something we can do to solve that in Swift without the complication of protected/friend...

Thanks,
Jon

···

On Feb 21, 2017, at 3:40 PM, Nevin Brackett-Rozinsky via swift-evolution <swift-evolution@swift.org> wrote:

I think entities declared as “internal” should be visible throughout the entire *module* just as they are today. In particular, if I write “struct Foo {}” inside a submodule, then Foo should have internal visibility (the default when no access level is specified) and thus should be available to the entire module.

Similarly, if I write “public struct Bar {}” inside a submodule, then Bar should be exported from the module and available to client code. In other words, the submodule should exist to *organize* code, not to change its meaning.

Moreover, the ongoing access level discussions elsewhere on the list make clear that many people believe there should be just one visibility more restricted than “internal” (which of course should be named “private”). If we go that route, then “private” would mean “visible in this submodule only”. For files which are not in a submodule this acts like “fileprivate”, and for files which *are* in a submodule it means they can share their implementation details with closely related code (usually types and extensions) while allowing that code to be in separate files as appropriate.

That is most of what I want out of submodules: to provide all the benefits of Swift-2-era “private” without requiring that everything be stuffed inside one single file. For all other purposes a submodule would be transparent, it just lets you split up a large file along its natural divisions. Consequently, I see no need or purpose for nested submodules, because a submodule simply “acts like” a single file.

Nevin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Brent_Royal-Gordon · February 22, 2017, 8:04am

Yeah, that's a use case I've thought about. I *think* that, if extensions in other submodules can add `open` methods, it can be done:

1. Create a submodule (or several) containing `UIGestureRecognizer`'s normal APIs.

2. Create a `UIKit.UIGestureRecognizerSubclass` submodule containing the subclass-only APIs in an extension.

3. In the top-level module, declare the main submodule `@exported open`, and declare `UIKit.UIGestureRecognizerSubclass` `open`.

There are holes in this idea—particularly, the setter on the `state` property—but I *think* you could do something along these lines.

···

On Feb 21, 2017, at 11:11 PM, Jonathan Hull via swift-evolution <swift-evolution@swift.org> wrote:

There is one major use case which I see all of these proposals failing to handle (although that may be intentional for some designs), is where you have something internal to the type, but you still want to allow subclasses/extensions outside the framework (which need that information) to access it. An actual example of this is UIGestureRecognizer. This class is designed to be subclassed outside of the framework, but it hides things like setting the ‘state' from client code (without hiding it from subclasses). This is important, because if a client sets the state directly, then it results in either undefined behavior, or an infinite loop. Other languages use ‘protected’ and ‘friend’ to accomplish this.

--
Brent Royal-Gordon
Architechies

anandabits · February 22, 2017, 3:20pm

There is one major use case which I see all of these proposals failing to handle (although that may be intentional for some designs), is where you have something internal to the type, but you still want to allow subclasses/extensions outside the framework (which need that information) to access it. An actual example of this is UIGestureRecognizer. This class is designed to be subclassed outside of the framework, but it hides things like setting the ‘state' from client code (without hiding it from subclasses). This is important, because if a client sets the state directly, then it results in either undefined behavior, or an infinite loop. Other languages use ‘protected’ and ‘friend’ to accomplish this.

Yeah, that's a use case I've thought about. I *think* that, if extensions in other submodules can add `open` methods, it can be done:

1. Create a submodule (or several) containing `UIGestureRecognizer`'s normal APIs.

2. Create a `UIKit.UIGestureRecognizerSubclass` submodule containing the subclass-only APIs in an extension.

3. In the top-level module, declare the main submodule `@exported open`, and declare `UIKit.UIGestureRecognizerSubclass` `open`.

There are holes in this idea—particularly, the setter on the `state` property—but I *think* you could do something along these lines.

If I understand this strategy correctly it doesn’t prevent client code from seeing `UIGestureRecognizerSubclass` but it does require an explicit `import UIKit.UIGestureRecognizerSubclass` in code that depends on it making the inappropriate dependency easy to detect. Is that what you have in mind?

···

On Feb 22, 2017, at 2:04 AM, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> wrote:

On Feb 21, 2017, at 11:11 PM, Jonathan Hull via swift-evolution <swift-evolution@swift.org> wrote:

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Jon_Hull · February 22, 2017, 5:28pm

Yeah, there is something possible in there somewhere.

I guess you could also allow some change in visibility of submodules to set when they are importable or not (i.e. ‘public submodule’). There are some issues there around how it would interact with the visibility of types though.

Looking at the complexity of all this, I am back to thinking my idea of adding ‘hidden’ is the simplest way to accomplish this. Things would have private/internal/public modifiers, just as Nevin describes, but you could also mark something hidden (e.g. ‘public hidden’), which would reduce it’s visibility to the current file... and any file with an ‘import hidden FileName’ statement (as allowed by its private/internal/public declaration).

Thanks,
Jon

···

On Feb 22, 2017, at 12:04 AM, Brent Royal-Gordon <brent@architechies.com> wrote:

On Feb 21, 2017, at 11:11 PM, Jonathan Hull via swift-evolution <swift-evolution@swift.org> wrote:

There is one major use case which I see all of these proposals failing to handle (although that may be intentional for some designs), is where you have something internal to the type, but you still want to allow subclasses/extensions outside the framework (which need that information) to access it. An actual example of this is UIGestureRecognizer. This class is designed to be subclassed outside of the framework, but it hides things like setting the ‘state' from client code (without hiding it from subclasses). This is important, because if a client sets the state directly, then it results in either undefined behavior, or an infinite loop. Other languages use ‘protected’ and ‘friend’ to accomplish this.

Yeah, that's a use case I've thought about. I *think* that, if extensions in other submodules can add `open` methods, it can be done:

1. Create a submodule (or several) containing `UIGestureRecognizer`'s normal APIs.

2. Create a `UIKit.UIGestureRecognizerSubclass` submodule containing the subclass-only APIs in an extension.

3. In the top-level module, declare the main submodule `@exported open`, and declare `UIKit.UIGestureRecognizerSubclass` `open`.

There are holes in this idea—particularly, the setter on the `state` property—but I *think* you could do something along these lines.