[Proposal][Discussion] Modular Swift

I specified two different behaviors for `bar()` and `baz()`. I see now that you describe `internal` as having the behavior I want for `bar()`. Is there a way I can get the behavior I want for `baz()`?

···

--
Brent Royal-Gordon
Sent from my iPhone

On Feb 21, 2017, at 6:51 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

What access modifiers do I put on `bar()` and `baz()` so that `MyMod` can access `bar()` but not `baz()`, and code outside `MyMod` can access neither `bar()` nor `baz()`?

internal

That is not what this proposal requires. A public API is ripe for
re(export), but if the parent wishes to communicate with its children
without exporting across the module boundary, see the definition of
`internal`.

I'm not sure whether I'm misunderstanding or we're talking past each other.

Let me state this really simply. You have some code in a top-level module,
`MyMod`:

import MyMod.Submodule

func foo() {
bar()
}

And you have some other code in a submodule:

module Submodule {
??? func bar() {
baz()
}
}

And then—perhaps in a separate file—you have some other code in an
extension of the submodule:

extension Submodule {
??? func baz() {

}
}

What access modifiers do I put on `bar()` and `baz()` so that `MyMod` can
access `bar()` but not `baz()`, and code outside `MyMod` can access neither
`bar()` nor `baz()`?

internal

Huh? Brent wants `bar()` to be visible inside `foo()`, but not `baz()`. How
can they both use `internal`?

···

On Tue, Feb 21, 2017 at 8:51 PM, Robert Widmann via swift-evolution < swift-evolution@swift.org> wrote:

On Feb 21, 2017, at 9:49 PM, Brent Royal-Gordon <brent@architechies.com> > wrote:
On Feb 21, 2017, at 6:43 PM, Robert Widmann <devteam.codafi@gmail.com> > wrote:

   - open and public declarations are exported by a module for
   consumption by clients of the module.

   - internal declarations scope over the entire module and any derived
   submodules.

This way you can consume your own interface without it crossing the module
boundary.

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

It has been my hope that a lightweight module system will remove the need for `private` *and* `fileprivate`.

I really doubt it will. `private`/`fileprivate` works because you can also access `internal` at the same time.

What I mean by that is, think about code like this:

  // Foo.swift
  public class Foo {
    public init() { … }

    func doBar() -> Quux {
      return helper(in: randomRange())
    }

    private func helper(in range: Range<Int>) -> Quux {
      …
    }
  }

  // Bar.swift
  public class Bar {
    public static let shared = Bar()

    func baz(with foo: Foo) {
      let quux = foo.doBar()
      process(quux)
    }
    
    private func process(_ quux: Quux) {
      …
    }
  }

These classes have `public` APIs that are externally visible, `internal` APIs for communicating with each other, and `private` APIs for implementation details. Now try to reproduce the same design with submodules and `public`/`internal` only:

  public import MyMod.Foo
  public import MyMod.Bar

  module Foo {
    public class Foo {
      public init() { … }

      ??? func doBar() -> Quux {
        return helper(in: randomRange())
      }

      func helper(in range: Range<Int>) -> Quux {
        …
      }
    }
  }

  // Bar.swift
  module Bar {
    public class Bar {
      public static let shared = Bar()
      
      ??? func baz(with foo: Foo) {
        let quux = foo.doBar()
        process(quux)
      }
    
      func process(_ quux: Quux) {
        …
      }
    }
  }

The `doBar()` and `baz()` methods have to be either exposed to third parties or kept away from yourself. That's just not viable.

If they must communicate, they can be a part of the same (sub)module. This makes filling in these annotations trivial. Nobody actually uses modules to wall off their own APIs from themselves like this, they use submodules to encapsulate the internal parts and surface public APIs in the parent.

  module Bar {
    public class Foo {
      public init() { … }

      internal func doBar() -> Quux {
        return helper(in: randomRange())
      }

      internal func helper(in range: Range<Int>) -> Quux {
        …
      }
    }
  }

  // Bar.swift
  extension Bar {
    public class Bar {
      public static let shared = Bar()
      
      internal func baz(with foo: Foo) {
        let quux = foo.doBar()
        process(quux)
      }
    
      internal func process(_ quux: Quux) {
        …
      }
    }
  }

···

On Feb 21, 2017, at 7:46 AM, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> wrote:

On Feb 21, 2017, at 1:28 AM, Daniel Duan via swift-evolution <swift-evolution@swift.org> wrote:

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Sorry, been replying to multiple sub-threads today.

For bar(), because you wish to be able to

1) Not export it across the outermost module boundary
2) But still use it internally

Internal access is required. Any higher and you would export (violating 1), any lower and you wouldn’t be able to internally import (violating 2).

For baz(), because you wish to be able to

1) Not export it across the outermost module boundary,
2) Or even your own internal submodule boundary

Private or fileprivate suffices depending on the scoping you wish for it to have within the file/interface it’s a part of relative to the other APIs in the submodule.

···

On Feb 21, 2017, at 10:04 PM, Brent Royal-Gordon <brent@architechies.com> wrote:

I specified two different behaviors for `bar()` and `baz()`. I see now that you describe `internal` as having the behavior I want for `bar()`. Is there a way I can get the behavior I want for `baz()`?

--
Brent Royal-Gordon
Sent from my iPhone

On Feb 21, 2017, at 6:51 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

What access modifiers do I put on `bar()` and `baz()` so that `MyMod` can access `bar()` but not `baz()`, and code outside `MyMod` can access neither `bar()` nor `baz()`?

internal

It’s probably best we explain this motivation under “Alternatives Considered”. I forked and updated the gist—feel free to “pull” in the changes… <cough>copy and paste the document</cough>

https://gist.github.com/JadenGeller/c18b1be8e7b9fa3023e98b84fca423ca#multiple-declarations-instead-of-extensions

Cheers,
Jaden Geller

···

On Feb 20, 2017, at 10:57 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

This was syntax I initially considered, but Jaden swayed me with the idea that we already have an `extension` keyword so we may as well use it to match the rest of the language. Repeating the `module` declaration also lends itself more to C++-style namespaces, which these are not. I’m not saying it’s out of the question, far from it. It is an alternative we will keep in mind.

~Robert Widmann

On Feb 21, 2017, at 1:53 AM, Psycho Hedgehog via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

The one thing i don't understand is the point of using extensions to extend modules across multiple files, why not just allow declaring the submodule in multiple files?

Le 20 févr. 2017 à 22:48, Jaden Geller via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> a écrit :

Oh, I see. You’re questioning the motivation of having scope-granularity submodules at all! My misunderstanding.

I actually hadn’t considered this as added complexity. In my mind, a scoped module declaration seems more Swifty than a file module declaration. It builds on the existing syntax in Swift for defining other sorts of scopes, e.g. types.

Cheers,
Jaden Geller

On Feb 20, 2017, at 10:41 PM, Jonathan Hull via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

I think my question is: Why do we want to allow submodules that are smaller than a file? What does that give us to offset the added complexity?

Thanks,
Jon

On Feb 20, 2017, at 6:44 PM, Robert Widmann <devteam.codafi@gmail.com <mailto:devteam.codafi@gmail.com>> wrote:

On Feb 20, 2017, at 9:36 PM, Jonathan Hull <jhull@gbis.com <mailto:jhull@gbis.com>> wrote:

What is the rational for having modules covering only part of a file? Wouldn’t it be less clutter to have an annotation which worked for the whole file. At the very least it would be nice to have an option to spell it in a way that applies to the whole file. Otherwise, everything will be indented another level.

That is a valid spelling (Rust, IIRC, allows that spelling), but one that is easy to miss sitting in a file and makes it confusing to introduce submodules. If you include the annotation then define a submodule later down in the file, suddenly you have to remember whether you annotated the file or whether the submodule you’ve just written is going into the top-level module. See:

// -module-name=Foo
// module Foo {
module Bar; // Shorthand for “This file defines Foo.Bar”

/* Code */

// This defines “Foo.Bar.Baz”, but would you know that if it appeared below the fold?
module Baz {}
//}

If anything, this can be added later if evolution converges on it.

I would honestly love to see something which just maps modules to folders/groups for simplicity sake.

There is nothing about this scheme that prevents you from organizing your code this way. However, modulo that particular method of organization, you don’t really gain much as a user of the language by imposing this restriction.

I haven’t thought about it too much yet, so I could easily be missing something obvious...

Thanks,
Jon

On Feb 20, 2017, at 5:56 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Good Evening All,

Jaden Geller and I have been considering a (sub)module system for Swift that would complement the existing language but also provide sorely needed modularity. A draft of the proposal is attached to this email, but it can also be read as a gist <https://gist.github.com/CodaFi/cd66b7d70b5cd8e4e8b433fa2ace378a&gt; if you desire.

Cheers,

~Robert Widmann

Modular Swift

Proposal: SE-NNNN <https://gist.github.com/CodaFi/NNNN-filename.md&gt;
Authors: Robert Widmann <https://github.com/codafi&gt;, Jaden Geller <https://github.com/JadenGeller&gt;
Review Manager: TBD
Status: Awaiting review
<Proposal.md · GitHub

Almost every major programming language supports some form of modular programming through constructs like (sub)modules, packages, or interfaces. Swift, though it provides top-level modules to organize code under, does not provide a complete implementation of any of these concepts, which has led instead to the proliferation of access control levels. This has not proven an effective way to decompose programs into manageable parts, and exposes the need for a real system of modules to solve this modularity problem once and for all.

Separation of code into distinct islands of functionality should be a first-class construct in the language, not dependent on external files and tools or filesystems. To that end, we propose the introduction of a lightweight module system for Swift.

Swift-evolution thread <applewebdata://DA662173-61C2-4A11-9298-89E9CF47D2FE>
<Proposal.md · GitHub

Swift has reached a point in its evolution where rich libraries and large projects that take on many dependencies have matured significantly. To accomodate the information-hiding and semantics-signalling needs of these users at the time, Swift began its access control story with just three access modifiers: public, private, and internal then grew fileprivate and open as the need to express locality of implementation and "subclassability" arose respectively. In doing so, Swift's access control scheme has become anti-modular.

<Proposal.md · GitHub solution

We propose the introduction of a lightweight module system for Swift. More than simply namspaces, a module declaration interacts with Swift's access control to provide an API boundary that allows better control over an interface's design.

<Proposal.md · GitHub design

<Proposal.md · GitHub

A module is a named region that introduces a lexical scope into which declarations may be nested. The name of the module can be used to access these member declarations. A module, like other aggregate structures in Swift, may be extended with new declarations over one or more translation units (files).

We propose a new declaration kind, module-decl be added to the language. A proposed grammar using the new modulekeyword is given below:

GRAMMAR OF A MODULE DECLARATION

module-declaration -> `module` module-identifier module-body
module-name -> identifier
module-body -> { module-members(opt) }
module-members -> module-member module-members(opt)
module-member -> declaration | compiler-control-statement
GRAMMAR OF A DECLARATION

+ declaration -> module-declaration
<Proposal.md · GitHub Semantics

Syntax and semantics for imports, as it already supports referencing submodules imported from C and Objective-C modules, remains unchanged:

// The outermost module is given explicitly
// by passing `-module-name=Foo` or exists implicitly, as today.
// module Foo {
public class A {}

module Bar {
  module Baz {
    public class C {}
  }

  public class B {}
}

let message = "Hello, Wisconsin!"
// } // End declarations added to module Foo.
To consume this interface:

// imports all of Foo, Foo.Bar, and Foo.Bar.Baz
import Foo.Bar.Baz

// imports Foo.A as A
import class Foo.A
// imports Foo.Bar.B as B
import class Foo.Bar.B
// imports Foo.Bar.Baz.C as C
import class Foo.Bar.Baz.C
A module declaration may only appear as a top-level entity or as a member of another module declaration. The following code is therefore invalid:

module Foo {
  class Bar {
    module Baz {} // error: module declaration cannot be nested inside type 'Bar'
  }
}
To extend an existing module declaration, simply reference its module name in an extension declaration.

// In module 'Foo'
module Bar {
  public class A {}

  module Baz {}
}

extension Bar {
  public struct B {}
}

extension Bar.Baz {
  public enum C { case D }
}
<Proposal.md · GitHub and Access Control

The semantics of some existing access control modifiers shall also be extended to support module declarations:

open and public declarations are exported by a module for consumption by clients of the module.
internal declarations scope over the entire module and any derived submodules.
By default, to preserve encapsulation of interfaces, modules are "sealed" and may only be "opened" by explicit named import. However, it is often desirable to export a module and a set of submodules or even modules from external dependencies along with a given interface. We propose the public keyword be used for this purpose:

// Defines top-level module "Foo"
//module Foo {
public import Foo.Bar.Baz
public import Foundation.Date
//}
Which then causes the following (sub)modules to be imported into scope along with Foo:

// imports Foo, Foo.Bar.Baz, and Foundation.Date
import Foo
To support existing Swift packages that cannot have opted into modules, and to preserve the scriptable nature of Swift, module declarations shall be optional. Any Swift program that does not declare at least one top-level module explicitly is considered part of an unnamed special "Global Module" with the same rules of access control as today. To give declarations in the Global Module an explicit module without using a module declaration, use the -module-name flag.

<Proposal.md · GitHub on Existing Code

This proposal is intentionally additive. There is no impact on existing code.

<Proposal.md · GitHub considered

<Proposal.md · GitHub Modules Everywhere

Declarations in the top-level of a program exist today in the top-level of the corresponding module. If desired, this module declaration could be required to be explicit like so:

module Foo {
  module Bar {
    module Baz {}
  }
}
However, we feel that imposing such a requirement not only complicates the outermost scope, it requires inserting needless extension Foo {} scopes in every file. It also violates the principle of progressive disclosure by forcing all new adoptees of Swift to learn what a module is without actually using the module system.

<Proposal.md · GitHub Extensions

Nested module extensions may be "expanded" as it were to the following:

module Foo {
  module Bar {}
}

extension Foo {
  extension Bar {}
}
However, this syntax is currently not enabled in general in Swift. This problem should be revisted in a future proposal.

<Proposal.md · GitHub (Source-Breaking Changes)

The system described above is intended to be entirely source and binary compatible. Nonetheless, in its design we feel we have obviated certain existing features and recommend their deprecation in future proposals:

fileprivate access can be recreated by creating a private "utility submodule" containing declarations of at least internal access.
@_exported, the private directive to re-export modules today, should be deprecated and removed.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

We avoid forcing users to organize code in such an opinionated manner just to please the compiler. Perhaps some submodules deserve a separate file, but I think that choice should not be forced by the language. I don’t have data on the popularity, but I personally very much dislike the similar restriction Java places on public classes and files.

On the other hand, having such standardisation means that you know exactly where a type is based on its module membership. One of the outcomes of Scala's move away from enforcing a relationship between files and their module structure means that it is almost impossible to find where a type is defined using the filesystem, and you resort to using 'git grep' or other tools to find out where the implementation is located.

Given that we want Swift to be a fantastic scripting language, I feel we ought not place artificial restrictions on code organization. Many scripts are a single file (for convenience) but may still benefit from the organization modules offer.

Is there a reason why a script should need to be in any module, as opposed to a top-level/unnamed module?

Submodules within a script provide encapsulation that can be useful for code organization. They likely would use an implicit (unnamed) outer module.

Alex

Cheers,
Jaden

···

On Feb 21, 2017, at 4:44 AM, Alex Blewitt <alblue@apple.com> wrote:

On 21 Feb 2017, at 07:00, Jaden Geller via swift-evolution <swift-evolution@swift.org> wrote:

So are private and/or fileprivate changing to mean "visible within current submodule"? Because I specified that baz() is in a type extension, possibly in another file.

···

--
Brent Royal-Gordon
Sent from my iPhone

On Feb 21, 2017, at 7:08 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

For baz(), because you wish to be able to

1) Not export it across the outermost module boundary,
2) Or even your own internal submodule boundary

Private or fileprivate suffices depending on the scoping you wish for it to have within the file/interface it’s a part of relative to the other APIs in the submodule.

This proposal is definitely *not* what I want from a submodule system,
because it will exacerbate the problem of developers stuffing large amounts
of code into a single file.

The “fileprivate” keyword already encourages long files, because there is
no way to split up related components into separate files while retaining
encapsulation. And scope-based “private” is even worse of an offender,
because it requires nesting code inside the same *type declaration*, so you
can’t even benefit from extensions.

To my mind, any submodule system for Swift should be designed to relieve
the pressure for long files, and make it easy to group tightly related
files into a single unit with shared visibility. That way developers can
easily organize their code into smaller files while utilizing Swift’s
pattern of providing protocol conformances in extensions and keeping
implementation details hidden from the rest of the module at large.

For example, one possible design would enable us to replace both “private”
and “fileprivate” with a single new access level—probably spelled
“private— which restricts visibility to just the current submodule. That
way it provides all the benefits of “fileprivate” (implementation hiding
and the ability to use extensions) while also allowing code to be placed in
separate files rather than one large file.

Nevin

It has been my hope that a lightweight module system will remove the need
for `private` *and* `fileprivate`.

I really doubt it will. `private`/`fileprivate` works because you can also
access `internal` at the same time.

What I mean by that is, think about code like this:

// Foo.swift
public class Foo {
public init() { … }

func doBar() -> Quux {
return helper(in: randomRange())
}

private func helper(in range: Range<Int>) -> Quux {

}
}

// Bar.swift
public class Bar {
public static let shared = Bar()

func baz(with foo: Foo) {
let quux = foo.doBar()
process(quux)
}

private func process(_ quux: Quux) {

}
}

These classes have `public` APIs that are externally visible, `internal`
APIs for communicating with each other, and `private` APIs for
implementation details. Now try to reproduce the same design with
submodules and `public`/`internal` only:

public import MyMod.Foo
public import MyMod.Bar

module Foo {
public class Foo {
public init() { … }

??? func doBar() -> Quux {
return helper(in: randomRange())
}

func helper(in range: Range<Int>) -> Quux {

}
}
}

// Bar.swift
module Bar {
public class Bar {
public static let shared = Bar()

??? func baz(with foo: Foo) {
let quux = foo.doBar()
process(quux)
}

func process(_ quux: Quux) {

}
}
}

The `doBar()` and `baz()` methods have to be either exposed to third
parties or kept away from yourself. That's just not viable.

If they must communicate, they can be a part of the same (sub)module.
This makes filling in these annotations trivial. Nobody actually uses
modules to wall off their own APIs from themselves like this, they use
submodules to encapsulate the internal parts and surface public APIs in the
parent.

I think you'll find a ton of people on this list who would want to use
submodules precisely to wall off their own APIs from themselves. Witness
the hundreds of messages about new syntax to do just that.

···

On Tue, Feb 21, 2017 at 8:15 PM, Robert Widmann via swift-evolution < swift-evolution@swift.org> wrote:

On Feb 21, 2017, at 7:46 AM, Brent Royal-Gordon via swift-evolution < > swift-evolution@swift.org> wrote:
On Feb 21, 2017, at 1:28 AM, Daniel Duan via swift-evolution < > swift-evolution@swift.org> wrote:

module Bar {

public class Foo {

public init() { … }

internal func doBar() -> Quux {

return helper(in: randomRange())

}

internal func helper(in range: Range<Int>) -> Quux {

}

}

}

// Bar.swift

extension Bar {

public class Bar {

public static let shared = Bar()

internal func baz(with foo: Foo) {

let quux = foo.doBar()

process(quux)

}

internal func process(_ quux: Quux) {

}

}
}

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

There’s an important distinction between the ability to wall off, and this
particular instance of doing so. By definition, a module allows for the
encapsulation and export of a subset of an API, the same way access
modifiers allow types to expose an interface. If the sub-parts of that API
must interact, they must by definition share the same concerns and belong
together under the same submodule. If their implementations require
separation then further subdivisions can be made with further submodules.
This is the design practice encouraged, and put to good use, by every other
language with modules since the heyday of Modula, and breaking that
contract by creating unnecessary parent divisions to introduce headaches
for yourself is just another case of anti-modular use of a modular system.
There’s nothing we can do to stop you from asking for this, but that also
doesn’t excuse the poor design choice here - especially when it can be
rectified by reshuffling your own dependency graph.

Indeed, I won't disagree with you wrt _modules_. I notice you titled this
thread "submodules" but propose a syntax that uses the word `module`. Left
unsaid, I'd imagine, is that you regard a submodule as a
module-within-a-module.

This is *not*, as I understand it, what most people on this list are asking
for wrt _submodules_. Instead, they are asking for a unit of code greater
than a file but less than a module. To parallel that new facility, they
want an access level greater than fileprivate but less than internal. This
draft proposal (in its failure to acknowledge this frequent ask) explicitly
but _silently_ rejects those motivations.

···

On Tue, Feb 21, 2017 at 8:32 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

On Feb 21, 2017, at 9:19 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Tue, Feb 21, 2017 at 8:15 PM, Robert Widmann via swift-evolution < > swift-evolution@swift.org> wrote:

On Feb 21, 2017, at 7:46 AM, Brent Royal-Gordon via swift-evolution < >> swift-evolution@swift.org> wrote:

On Feb 21, 2017, at 1:28 AM, Daniel Duan via swift-evolution < >> swift-evolution@swift.org> wrote:

It has been my hope that a lightweight module system will remove the need
for `private` *and* `fileprivate`.

I really doubt it will. `private`/`fileprivate` works because you can
also access `internal` at the same time.

What I mean by that is, think about code like this:

// Foo.swift
public class Foo {
public init() { … }

func doBar() -> Quux {
return helper(in: randomRange())
}

private func helper(in range: Range<Int>) -> Quux {

}
}

// Bar.swift
public class Bar {
public static let shared = Bar()

func baz(with foo: Foo) {
let quux = foo.doBar()
process(quux)
}

private func process(_ quux: Quux) {

}
}

These classes have `public` APIs that are externally visible, `internal`
APIs for communicating with each other, and `private` APIs for
implementation details. Now try to reproduce the same design with
submodules and `public`/`internal` only:

public import MyMod.Foo
public import MyMod.Bar

module Foo {
public class Foo {
public init() { … }

??? func doBar() -> Quux {
return helper(in: randomRange())
}

func helper(in range: Range<Int>) -> Quux {

}
}
}

// Bar.swift
module Bar {
public class Bar {
public static let shared = Bar()

??? func baz(with foo: Foo) {
let quux = foo.doBar()
process(quux)
}

func process(_ quux: Quux) {

}
}
}

The `doBar()` and `baz()` methods have to be either exposed to third
parties or kept away from yourself. That's just not viable.

If they must communicate, they can be a part of the same (sub)module.
This makes filling in these annotations trivial. Nobody actually uses
modules to wall off their own APIs from themselves like this, they use
submodules to encapsulate the internal parts and surface public APIs in the
parent.

I think you'll find a ton of people on this list who would want to use
submodules precisely to wall off their own APIs from themselves. Witness
the hundreds of messages about new syntax to do just that.

module Bar {

public class Foo {

public init() { … }

internal func doBar() -> Quux {

return helper(in: randomRange())

}

internal func helper(in range: Range<Int>) -> Quux {

}

}

}

// Bar.swift

extension Bar {

public class Bar {

public static let shared = Bar()

internal func baz(with foo: Foo) {

let quux = foo.doBar()

process(quux)

}

internal func process(_ quux: Quux) {

}

}
}

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Sorry, been replying to multiple sub-threads today.

For bar(), because you wish to be able to

1) Not export it across the outermost module boundary
2) But still use it internally

Internal access is required. Any higher and you would export (violating
1), any lower and you wouldn’t be able to internally import (violating 2).

For baz(), because you wish to be able to

1) Not export it across the outermost module boundary,
2) Or even your own internal submodule boundary

3) But still use it within the same submodule, across different file
boundaries: this is the feature that many people have stated they want to
emerge out of a submodule design.

Private or fileprivate suffices depending on the scoping you wish for it to

···

On Tue, Feb 21, 2017 at 9:08 PM, Robert Widmann via swift-evolution < swift-evolution@swift.org> wrote:

have within the file/interface it’s a part of relative to the other APIs in
the submodule.

> On Feb 21, 2017, at 10:04 PM, Brent Royal-Gordon <brent@architechies.com> > wrote:
>
> I specified two different behaviors for `bar()` and `baz()`. I see now
that you describe `internal` as having the behavior I want for `bar()`. Is
there a way I can get the behavior I want for `baz()`?
>
> --
> Brent Royal-Gordon
> Sent from my iPhone
>
> On Feb 21, 2017, at 6:51 PM, Robert Widmann <devteam.codafi@gmail.com> > wrote:
>
>>> What access modifiers do I put on `bar()` and `baz()` so that `MyMod`
can access `bar()` but not `baz()`, and code outside `MyMod` can access
neither `bar()` nor `baz()`?
>>>
>>
>> internal

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

There’s an important distinction between the ability to wall off, and this particular instance of doing so. By definition, a module allows for the encapsulation and export of a subset of an API, the same way access modifiers allow types to expose an interface. If the sub-parts of that API must interact, they must by definition share the same concerns and belong together under the same submodule. If their implementations require separation then further subdivisions can be made with further submodules. This is the design practice encouraged, and put to good use, by every other language with modules since the heyday of Modula, and breaking that contract by creating unnecessary parent divisions to introduce headaches for yourself is just another case of anti-modular use of a modular system. There’s nothing we can do to stop you from asking for this, but that also doesn’t excuse the poor design choice here - especially when it can be rectified by reshuffling your own dependency graph.

Indeed, I won't disagree with you wrt _modules_. I notice you titled this thread "submodules" but propose a syntax that uses the word `module`. Left unsaid, I'd imagine, is that you regard a submodule as a module-within-a-module.

This is *not*, as I understand it, what most people on this list are asking for wrt _submodules_. Instead, they are asking for a unit of code greater than a file but less than a module. To parallel that new facility, they want an access level greater than fileprivate but less than internal. This draft proposal (in its failure to acknowledge this frequent ask) explicitly but _silently_ rejects those motivations.

So they want to be able to aggregate interfaces and define their exportability, while also maintaining the ability to scope a declaration to a particular grouping. This is, quite literally, the semantics we have defined - with modules enabling aggregation and `internal` stretching to become this “new” access control kind.

···

On Feb 21, 2017, at 9:44 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
On Tue, Feb 21, 2017 at 8:32 PM, Robert Widmann <devteam.codafi@gmail.com <mailto:devteam.codafi@gmail.com>> wrote:

On Feb 21, 2017, at 9:19 PM, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> wrote:

On Tue, Feb 21, 2017 at 8:15 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Feb 21, 2017, at 7:46 AM, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Feb 21, 2017, at 1:28 AM, Daniel Duan via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

It has been my hope that a lightweight module system will remove the need for `private` *and* `fileprivate`.

I really doubt it will. `private`/`fileprivate` works because you can also access `internal` at the same time.

What I mean by that is, think about code like this:

  // Foo.swift
  public class Foo {
    public init() { … }

    func doBar() -> Quux {
      return helper(in: randomRange())
    }

    private func helper(in range: Range<Int>) -> Quux {
      …
    }
  }

  // Bar.swift
  public class Bar {
    public static let shared = Bar()

    func baz(with foo: Foo) {
      let quux = foo.doBar()
      process(quux)
    }
    
    private func process(_ quux: Quux) {
      …
    }
  }

These classes have `public` APIs that are externally visible, `internal` APIs for communicating with each other, and `private` APIs for implementation details. Now try to reproduce the same design with submodules and `public`/`internal` only:

  public import MyMod.Foo
  public import MyMod.Bar

  module Foo {
    public class Foo {
      public init() { … }

      ??? func doBar() -> Quux {
        return helper(in: randomRange())
      }

      func helper(in range: Range<Int>) -> Quux {
        …
      }
    }
  }

  // Bar.swift
  module Bar {
    public class Bar {
      public static let shared = Bar()
      
      ??? func baz(with foo: Foo) {
        let quux = foo.doBar()
        process(quux)
      }
    
      func process(_ quux: Quux) {
        …
      }
    }
  }

The `doBar()` and `baz()` methods have to be either exposed to third parties or kept away from yourself. That's just not viable.

If they must communicate, they can be a part of the same (sub)module. This makes filling in these annotations trivial. Nobody actually uses modules to wall off their own APIs from themselves like this, they use submodules to encapsulate the internal parts and surface public APIs in the parent.

I think you'll find a ton of people on this list who would want to use submodules precisely to wall off their own APIs from themselves. Witness the hundreds of messages about new syntax to do just that.

  module Bar {
    public class Foo {
      public init() { … }

      internal func doBar() -> Quux {
        return helper(in: randomRange())
      }

      internal func helper(in range: Range<Int>) -> Quux {
        …
      }
    }
  }

  // Bar.swift
  extension Bar {
    public class Bar {
      public static let shared = Bar()
      
      internal func baz(with foo: Foo) {
        let quux = foo.doBar()
        process(quux)
      }
    
      internal func process(_ quux: Quux) {
        …
      }
    }
  }

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

Fileprivate and private are not changing at all. Their meaning now extends from “private to this file” and “private to this declaration” respectively to those meanings plus “unexportable across any module boundary”. One implication of this is it is now possible to create module-scoped private constants, functions, and data structures, which is one of the use-cases that Daniel Duan mentioned earlier down in the thread.

···

On Feb 21, 2017, at 10:12 PM, Brent Royal-Gordon <brent@architechies.com> wrote:

So are private and/or fileprivate changing to mean "visible within current submodule"? Because I specified that baz() is in a type extension, possibly in another file.

--
Brent Royal-Gordon
Sent from my iPhone

On Feb 21, 2017, at 7:08 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

For baz(), because you wish to be able to

1) Not export it across the outermost module boundary,
2) Or even your own internal submodule boundary

Private or fileprivate suffices depending on the scoping you wish for it to have within the file/interface it’s a part of relative to the other APIs in the submodule.

Once again, internal is your keyword. An extension allows you to “open and expand” the module boundary here, which is exactly what you want here - to extend this module across file boundaries without showing your cards to an external consumer of your framework.

···

On Feb 21, 2017, at 10:26 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Tue, Feb 21, 2017 at 9:08 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
Sorry, been replying to multiple sub-threads today.

For bar(), because you wish to be able to

1) Not export it across the outermost module boundary
2) But still use it internally

Internal access is required. Any higher and you would export (violating 1), any lower and you wouldn’t be able to internally import (violating 2).

For baz(), because you wish to be able to

1) Not export it across the outermost module boundary,
2) Or even your own internal submodule boundary

3) But still use it within the same submodule, across different file boundaries: this is the feature that many people have stated they want to emerge out of a submodule design.

Private or fileprivate suffices depending on the scoping you wish for it to have within the file/interface it’s a part of relative to the other APIs in the submodule.

> On Feb 21, 2017, at 10:04 PM, Brent Royal-Gordon <brent@architechies.com <mailto:brent@architechies.com>> wrote:
>
> I specified two different behaviors for `bar()` and `baz()`. I see now that you describe `internal` as having the behavior I want for `bar()`. Is there a way I can get the behavior I want for `baz()`?
>
> --
> Brent Royal-Gordon
> Sent from my iPhone
>
> On Feb 21, 2017, at 6:51 PM, Robert Widmann <devteam.codafi@gmail.com <mailto:devteam.codafi@gmail.com>> wrote:
>
>>> What access modifiers do I put on `bar()` and `baz()` so that `MyMod` can access `bar()` but not `baz()`, and code outside `MyMod` can access neither `bar()` nor `baz()`?
>>>
>>
>> internal

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

So what you are saying is that, in my example:

  // foo.swift
  import MyMod.Submodule
  func foo() {
    bar()
  }

  // bar.swift
  module Submodule {
    internal func bar() {
      baz()
    }
  }

  // baz.swift
  extension Submodule {
    ??? func baz() {
      …
    }
  }

There is nothing I can put in the `???` slot that will expose `baz()` to `bar()`, but not to `foo()`. Correct?

···

On Feb 21, 2017, at 7:14 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

Fileprivate and private are not changing at all. Their meaning now extends from “private to this file” and “private to this declaration” respectively to those meanings plus “unexportable across any module boundary”. One implication of this is it is now possible to create module-scoped private constants, functions, and data structures, which is one of the use-cases that Daniel Duan mentioned earlier down in the thread.

--
Brent Royal-Gordon
Architechies

I agree with Nevin’s points in large strokes, but I don’t think large files are the end of the world. I think an important takeaway idea from all the access control threads is that we need not provide facilities in the language to support antipatterns; we should more focus on how to encourage good patterns better.

On the face of it, I’m excited by the proposal; even if this isn’t the answer, we should seek to go down any road that removes fileprivate (and I mean remove - not just a new spelling) but preserves aspect the feature to please whatever it is that people like about that feature.

···

On Feb 21, 2017, at 7:36 PM, Nevin Brackett-Rozinsky via swift-evolution <swift-evolution@swift.org> wrote:

This proposal is definitely *not* what I want from a submodule system, because it will exacerbate the problem of developers stuffing large amounts of code into a single file.

The “fileprivate” keyword already encourages long files, because there is no way to split up related components into separate files while retaining encapsulation. And scope-based “private” is even worse of an offender, because it requires nesting code inside the same *type declaration*, so you can’t even benefit from extensions.

To my mind, any submodule system for Swift should be designed to relieve the pressure for long files, and make it easy to group tightly related files into a single unit with shared visibility. That way developers can easily organize their code into smaller files while utilizing Swift’s pattern of providing protocol conformances in extensions and keeping implementation details hidden from the rest of the module at large.

For example, one possible design would enable us to replace both “private” and “fileprivate” with a single new access level—probably spelled “private— which restricts visibility to just the current submodule. That way it provides all the benefits of “fileprivate” (implementation hiding and the ability to use extensions) while also allowing code to be placed in separate files rather than one large file.

Nevin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

To my mind, any submodule system for Swift should be designed to relieve the pressure for long files, and make it easy to group tightly related files into a single unit with shared visibility. That way developers can easily organize their code into smaller files while utilizing Swift’s pattern of providing protocol conformances in extensions and keeping implementation details hidden from the rest of the module at large.

Wonderful, because that’s absolutely supported by this proposal. To group tightly related files into a single unit, simply declare a submodule for them and extend it in each of your related files. Any variables defined with `internal` access will be visible across those files to those extensions and only those extensions (see the section on access control and modules). Any variables declared fileprivate or private will, obviously, not be visible across these files. As an example:

// FooUtilities.swift
//
// -module-name=Foo
// module Foo {
// Defines Foo.Utilities
module Utilities {
  public func exportableOutsideThisSubmodule() {}
  func visibleInThisSubmodule() {}
  private func invisibleToOtherFiles() {}
}
//}

// FooUtilities+MoreUtilities.swift
extension Utilities {
  private func privateHelper() {
    visibleInThisSubmodule()
  }
}

I’m not sure where you got the impression that we were just trying to make another fileprivate happen.

···

On Feb 21, 2017, at 7:36 PM, Nevin Brackett-Rozinsky via swift-evolution <swift-evolution@swift.org> wrote:

Nevin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Once again, internal is your keyword. An extension allows you to “open
and expand” the module boundary here, which is exactly what you want here -
to extend this module across file boundaries without showing your cards to
an external consumer of your framework.

Sorry, I don't understand. Does your design support the use case below? I
don't think it does. Are you replying that supporting the use case below is
not a goal of your proposal? If so, please just say so.

···

On Tue, Feb 21, 2017 at 9:29 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

On Feb 21, 2017, at 10:26 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Tue, Feb 21, 2017 at 9:08 PM, Robert Widmann via swift-evolution < > swift-evolution@swift.org> wrote:

Sorry, been replying to multiple sub-threads today.

For bar(), because you wish to be able to

1) Not export it across the outermost module boundary
2) But still use it internally

Internal access is required. Any higher and you would export (violating
1), any lower and you wouldn’t be able to internally import (violating 2).

For baz(), because you wish to be able to

1) Not export it across the outermost module boundary,
2) Or even your own internal submodule boundary

3) But still use it within the same submodule, across different file
boundaries: this is the feature that many people have stated they want to
emerge out of a submodule design.

Private or fileprivate suffices depending on the scoping you wish for it

to have within the file/interface it’s a part of relative to the other APIs
in the submodule.

> On Feb 21, 2017, at 10:04 PM, Brent Royal-Gordon < >> brent@architechies.com> wrote:
>
> I specified two different behaviors for `bar()` and `baz()`. I see now
that you describe `internal` as having the behavior I want for `bar()`. Is
there a way I can get the behavior I want for `baz()`?
>
> --
> Brent Royal-Gordon
> Sent from my iPhone
>
> On Feb 21, 2017, at 6:51 PM, Robert Widmann <devteam.codafi@gmail.com> >> wrote:
>
>>> What access modifiers do I put on `bar()` and `baz()` so that `MyMod`
can access `bar()` but not `baz()`, and code outside `MyMod` can access
neither `bar()` nor `baz()`?
>>>
>>
>> internal

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Let’s use some code to illustrate things.

// FooUtilities.swift
//
// -module-name=Foo
// module Foo {
// Defines Foo.Utilities
module Utilities {
  internal func visibleInThisSubmodule() {}
}
//}

// FooUtilities+MoreUtilities.swift
extension Foo.Utilities {
  private func privateHelper() {
    visibleInThisSubmodule()
  }
}

···

On Feb 21, 2017, at 10:31 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Tue, Feb 21, 2017 at 9:29 PM, Robert Widmann <devteam.codafi@gmail.com <mailto:devteam.codafi@gmail.com>> wrote:
Once again, internal is your keyword. An extension allows you to “open and expand” the module boundary here, which is exactly what you want here - to extend this module across file boundaries without showing your cards to an external consumer of your framework.

Sorry, I don't understand. Does your design support the use case below? I don't think it does. Are you replying that supporting the use case below is not a goal of your proposal? If so, please just say so.

On Feb 21, 2017, at 10:26 PM, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> wrote:

On Tue, Feb 21, 2017 at 9:08 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
Sorry, been replying to multiple sub-threads today.

For bar(), because you wish to be able to

1) Not export it across the outermost module boundary
2) But still use it internally

Internal access is required. Any higher and you would export (violating 1), any lower and you wouldn’t be able to internally import (violating 2).

For baz(), because you wish to be able to

1) Not export it across the outermost module boundary,
2) Or even your own internal submodule boundary

3) But still use it within the same submodule, across different file boundaries: this is the feature that many people have stated they want to emerge out of a submodule design.

Private or fileprivate suffices depending on the scoping you wish for it to have within the file/interface it’s a part of relative to the other APIs in the submodule.

> On Feb 21, 2017, at 10:04 PM, Brent Royal-Gordon <brent@architechies.com <mailto:brent@architechies.com>> wrote:
>
> I specified two different behaviors for `bar()` and `baz()`. I see now that you describe `internal` as having the behavior I want for `bar()`. Is there a way I can get the behavior I want for `baz()`?
>
> --
> Brent Royal-Gordon
> Sent from my iPhone
>
> On Feb 21, 2017, at 6:51 PM, Robert Widmann <devteam.codafi@gmail.com <mailto:devteam.codafi@gmail.com>> wrote:
>
>>> What access modifiers do I put on `bar()` and `baz()` so that `MyMod` can access `bar()` but not `baz()`, and code outside `MyMod` can access neither `bar()` nor `baz()`?
>>>
>>
>> internal

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

Correct. Because, in dividing the submodule across an extension, you have placed what should be a private API into a differently-scoped location.

···

On Feb 21, 2017, at 10:34 PM, Brent Royal-Gordon <brent@architechies.com> wrote:

On Feb 21, 2017, at 7:14 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

Fileprivate and private are not changing at all. Their meaning now extends from “private to this file” and “private to this declaration” respectively to those meanings plus “unexportable across any module boundary”. One implication of this is it is now possible to create module-scoped private constants, functions, and data structures, which is one of the use-cases that Daniel Duan mentioned earlier down in the thread.

So what you are saying is that, in my example:

  // foo.swift
  import MyMod.Submodule
  func foo() {
    bar()
  }

  // bar.swift
  module Submodule {
    internal func bar() {
      baz()
    }
  }

  // baz.swift
  extension Submodule {
    ??? func baz() {
      …
    }
  }

There is nothing I can put in the `???` slot that will expose `baz()` to `bar()`, but not to `foo()`. Correct?

--
Brent Royal-Gordon
Architechies