[Proposal][Discussion] Modular Swift

Okay. So is your submodule design not intended to address the "I want to encapsulate implementation details so they're only visible to several units of code in different files, but not the entire module" use case? Because if there's no way to scope a symbol to "everything inside this submodule, but nothing outside this submodule", I think it leaves that use case unserved.

···

On Feb 21, 2017, at 7:38 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

Correct. Because, in dividing the submodule across an extension, you have placed what should be a private API into a differently-scoped location.

--
Brent Royal-Gordon
Architechies

To my mind, any submodule system for Swift should be designed to relieve
the pressure for long files, and make it easy to group tightly related
files into a single unit with shared visibility. That way developers can
easily organize their code into smaller files while utilizing Swift’s
pattern of providing protocol conformances in extensions and keeping
implementation details hidden from the rest of the module at large.

Wonderful, because that’s absolutely supported by this proposal. To group
tightly related files into a single unit, simply declare a submodule for
them and extend it in each of your related files.

It's supported, but it isn't first-class. By this I mean: there are two
distinguishable uses supported by your proposal, lumped together by the
fact that they are both about grouping units of code together. Put crudely,
one use case is grouping lines of code, while the other is about grouping
files of code. The merits of supporting both have already been debated in
this discussion. The issue I'll touch on is supporting both with the same
syntax. The chief drawbacks here are:

- It makes sense to use braces to group lines of code, but it makes no
sense to use braces to group files of code; this just causes entire files
to be indented.

- Because some lines of code necessarily precede some other lines of code,
it makes sense to declare the first group using `module` and to extend that
with the second group using `extension`. However, because a file of code
does not necessarily precede another file of code, it is arbitrary which
file is surrounded with a `module` declaration and which one is surrounded
with an `extension` declaration.

In both of these cases, your proposal has chosen to accommodate grouping
lines of code over the ergonomics of grouping files of code. Therefore,
while Nevin's use case is "absolutely supported by this proposal," I agree
with him that your choices promote something else entirely.

Any variables defined with `internal` access will be visible across those

···

On Tue, Feb 21, 2017 at 7:59 PM, Robert Widmann via swift-evolution < swift-evolution@swift.org> wrote:

On Feb 21, 2017, at 7:36 PM, Nevin Brackett-Rozinsky via swift-evolution < > swift-evolution@swift.org> wrote:
files to those extensions and only those extensions (see the section on
access control and modules). Any variables declared fileprivate or private
will, obviously, not be visible across these files. As an example:

// FooUtilities.swift
//
// -module-name=Foo
// module Foo {
// Defines Foo.Utilities
module Utilities {
  public func exportableOutsideThisSubmodule() {}
  func visibleInThisSubmodule() {}
  private func invisibleToOtherFiles() {}
}
//}

// FooUtilities+MoreUtilities.swift
extension Utilities {
  private func privateHelper() {
    visibleInThisSubmodule()
  }
}

I’m not sure where you got the impression that we were just trying to make
another fileprivate happen.

Nevin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

To my mind, any submodule system for Swift should be designed to relieve the pressure for long files, and make it easy to group tightly related files into a single unit with shared visibility. That way developers can easily organize their code into smaller files while utilizing Swift’s pattern of providing protocol conformances in extensions and keeping implementation details hidden from the rest of the module at large.

Wonderful, because that’s absolutely supported by this proposal. To group tightly related files into a single unit, simply declare a submodule for them and extend it in each of your related files.

It's supported, but it isn't first-class. By this I mean: there are two distinguishable uses supported by your proposal, lumped together by the fact that they are both about grouping units of code together. Put crudely, one use case is grouping lines of code, while the other is about grouping files of code. The merits of supporting both have already been debated in this discussion. The issue I'll touch on is supporting both with the same syntax. The chief drawbacks here are:

What exactly would be required to make it first class? Referencing file names in the module declaration?

- It makes sense to use braces to group lines of code, but it makes no sense to use braces to group files of code; this just causes entire files to be indented.

If braces aren’t used to demarcate scopes, nesting modules becomes ambiguous. It’s important to note that indentation is one particular style. LLVM code style, in particular, chooses not to indent after namespace declarations. This issue also crops up when dealing with nested type declarations, and I distinctly remember it not being a big enough deal to "fix this" at the time when a proposal to “flatten” these declaration was brought up.

- Because some lines of code necessarily precede some other lines of code, it makes sense to declare the first group using `module` and to extend that with the second group using `extension`. However, because a file of code does not necessarily precede another file of code, it is arbitrary which file is surrounded with a `module` declaration and which one is surrounded with an `extension` declaration.

Absolutely. But it is similarly arbitrary which public APIs are exposed in a type declaration and which are exposed in an extension declaration. My hope is that the module declaration itself will become the one-stop-shop for re-exports and general public bookkeeping just as aggregate declarations are today. Module extensions exist to accommodate users that wish to break related functionality across files or into separate independent regions within the same file for the same reasons type extensions exist.

···

On Feb 21, 2017, at 9:13 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
On Tue, Feb 21, 2017 at 7:59 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Feb 21, 2017, at 7:36 PM, Nevin Brackett-Rozinsky via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Any variables defined with `internal` access will be visible across those files to those extensions and only those extensions (see the section on access control and modules). Any variables declared fileprivate or private will, obviously, not be visible across these files. As an example:

// FooUtilities.swift
//
// -module-name=Foo
// module Foo {
// Defines Foo.Utilities
module Utilities {
  public func exportableOutsideThisSubmodule() {}
  func visibleInThisSubmodule() {}
  private func invisibleToOtherFiles() {}
}
//}

// FooUtilities+MoreUtilities.swift
extension Utilities {
  private func privateHelper() {
    visibleInThisSubmodule()
  }
}

I’m not sure where you got the impression that we were just trying to make another fileprivate happen.

Nevin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

Let’s use some code to illustrate things.

// FooUtilities.swift
//
// -module-name=Foo
// module Foo {
// Defines Foo.Utilities
module Utilities {
  internal func visibleInThisSubmodule() {}
}
//}

// FooUtilities+MoreUtilities.swift
extension Foo.Utilities {
  private func privateHelper() {
    visibleInThisSubmodule()
  }
}

Either I'm entirely misunderstanding what you're trying to illustrate, or
this is totally unresponsive to Brent's question. In either case, I'll let
Brent ask his own question from here.

···

On Tue, Feb 21, 2017 at 9:34 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

On Feb 21, 2017, at 10:31 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Tue, Feb 21, 2017 at 9:29 PM, Robert Widmann <devteam.codafi@gmail.com> > wrote:

Once again, internal is your keyword. An extension allows you to “open
and expand” the module boundary here, which is exactly what you want here -
to extend this module across file boundaries without showing your cards to
an external consumer of your framework.

Sorry, I don't understand. Does your design support the use case below? I
don't think it does. Are you replying that supporting the use case below is
not a goal of your proposal? If so, please just say so.

On Feb 21, 2017, at 10:26 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Tue, Feb 21, 2017 at 9:08 PM, Robert Widmann via swift-evolution < >> swift-evolution@swift.org> wrote:

Sorry, been replying to multiple sub-threads today.

For bar(), because you wish to be able to

1) Not export it across the outermost module boundary
2) But still use it internally

Internal access is required. Any higher and you would export (violating
1), any lower and you wouldn’t be able to internally import (violating 2).

For baz(), because you wish to be able to

1) Not export it across the outermost module boundary,
2) Or even your own internal submodule boundary

3) But still use it within the same submodule, across different file
boundaries: this is the feature that many people have stated they want to
emerge out of a submodule design.

Private or fileprivate suffices depending on the scoping you wish for it

to have within the file/interface it’s a part of relative to the other APIs
in the submodule.

> On Feb 21, 2017, at 10:04 PM, Brent Royal-Gordon < >>> brent@architechies.com> wrote:
>
> I specified two different behaviors for `bar()` and `baz()`. I see now
that you describe `internal` as having the behavior I want for `bar()`. Is
there a way I can get the behavior I want for `baz()`?
>
> --
> Brent Royal-Gordon
> Sent from my iPhone
>
> On Feb 21, 2017, at 6:51 PM, Robert Widmann <devteam.codafi@gmail.com> >>> wrote:
>
>>> What access modifiers do I put on `bar()` and `baz()` so that
`MyMod` can access `bar()` but not `baz()`, and code outside `MyMod` can
access neither `bar()` nor `baz()`?
>>>
>>
>> internal

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Correct. Because, in dividing the submodule across an extension, you have placed what should be a private API into a differently-scoped location.

Okay. So is your submodule design not intended to address the "I want to encapsulate implementation details so they're only visible to several units of code in different files, but not the entire module" use case? Because if there's no way to scope a symbol to "everything inside this submodule, but nothing outside this submodule", I think it leaves that use case unserved.

Unless I’m missing something there is also another encapsulation-related problem with the proposed design. Let’s suppose for the sake of discussion there was a `submoduleprivate` access modifier (intentionally ungainly and not realistic).

// File 1
module Foo {
    // internal, visible to the whole module
    class Bar { submoduleprivate var protectedState: Int = 0 }
}

// File 2 - Has nothing to do with Foo at all
import MyModule.Foo

module NotFoo {
   // Hey, I need to see Bar.protectedState!!!
   func totallyNotFoo() {
       var bar = Bar()
       bar.foosExposedPrivates = 42
   }
}

// ok, I’ll just add an extension to Foo so I can see submoduleprivate and wrap what I need
module Foo {
    // Hey, I’ll be nice and keep it fileprivate, but I could make it public if I wanted to.
    extension Bar {
         fileprivate var foosExposedPrivates: Int {
            // Yep, I’m inside Foo so I can see it’s submoduleprivate stuff
            get { return protectedState }
            set { protectedState = newValue }
         }
    }
}

···

On Feb 21, 2017, at 9:47 PM, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> wrote:

On Feb 21, 2017, at 7:38 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Correct. Because, in dividing the submodule across an extension, you have placed what should be a private API into a differently-scoped location.

Okay. So is your submodule design not intended to address the "I want to encapsulate implementation details so they're only visible to several units of code in different files, but not the entire module" use case? Because if there's no way to scope a symbol to "everything inside this submodule, but nothing outside this submodule", I think it leaves that use case unserved.

Unless I’m missing something there is also another encapsulation-related problem with the proposed design. Let’s suppose for the sake of discussion there was a `submoduleprivate` access modifier (intentionally ungainly and not realistic).

// File 1
module Foo {
   // internal, visible to the whole module
   class Bar { submoduleprivate var protectedState: Int = 0 }
}

// File 2 - Has nothing to do with Foo at all
import MyModule.Foo

module NotFoo {
  // Hey, I need to see Bar.protectedState!!!
  func totallyNotFoo() {
      var bar = Bar()
      bar.foosExposedPrivates = 42
  }
}

// ok, I’ll just add an extension to Foo so I can see submoduleprivate and wrap what I need
module Foo {

Oops, this should have been `extension Foo`, but otherwise I believe it is valid under this proposal.

···

On Feb 21, 2017, at 10:11 PM, Matthew Johnson <matthew@anandabits.com> wrote:

On Feb 21, 2017, at 9:47 PM, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> wrote:

On Feb 21, 2017, at 7:38 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

   // Hey, I’ll be nice and keep it fileprivate, but I could make it public if I wanted to.
   extension Bar {
        fileprivate var foosExposedPrivates: Int {
           // Yep, I’m inside Foo so I can see it’s submoduleprivate stuff
           get { return protectedState }
           set { protectedState = newValue }
        }
   }
}

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

This case is unserved because it is anti-modular and a nightmare to maintain. The actual way to structure this is to factor bar() and baz() into their own utility submodule, or even deeper if necessary, that has as a parent the primary module that wishes to consume them both but not re-export them out to the parent. For example,

  // foo.swift
  import MyMod.Submodule
  func foo() {
    bar()
  }

  // bar.swift

        // Make my utilities visible in Foo.Submodule, but invisible to the parent.
        import Foo.Submodule.UtilitySubmodule

  module Submodule {
    module UtilitySubmodule {
                   internal func bar() {
      baz()
      }
               }
  }

  // baz.swift
  extension Submodule.UtilitySubmodule {
    internal func baz() {
      …
    }
  }

The thought is that it should be as cheap to create submodules to organize interfaces under as it is to create new directories to organize code under.

Though, this example is a little odd given that you’re defining and importing the same utility submodule. Really, what you would want to do is declare the utility in its own file/files outside of bar.swift to really take full advantage of the separation afforded here, then consume it internally with the import as written here.

···

On Feb 21, 2017, at 10:47 PM, Brent Royal-Gordon <brent@architechies.com> wrote:

On Feb 21, 2017, at 7:38 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

Correct. Because, in dividing the submodule across an extension, you have placed what should be a private API into a differently-scoped location.

Okay. So is your submodule design not intended to address the "I want to encapsulate implementation details so they're only visible to several units of code in different files, but not the entire module" use case? Because if there's no way to scope a symbol to "everything inside this submodule, but nothing outside this submodule", I think it leaves that use case unserved.

--
Brent Royal-Gordon
Architechies

This level of access, the “private to this submodule except to the select set of interfaces I want to see it” level, is the equivalent of friend classes in C++. I don’t consider leaving this out to be a hole, nor is it an "encapsulation-related problem” because at no point can you break the API boundary and re-export anything here with a higher level of access than it had previously.

···

On Feb 21, 2017, at 11:13 PM, Matthew Johnson <matthew@anandabits.com> wrote:

On Feb 21, 2017, at 10:11 PM, Matthew Johnson <matthew@anandabits.com> wrote:

On Feb 21, 2017, at 9:47 PM, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> wrote:

On Feb 21, 2017, at 7:38 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

Correct. Because, in dividing the submodule across an extension, you have placed what should be a private API into a differently-scoped location.

Okay. So is your submodule design not intended to address the "I want to encapsulate implementation details so they're only visible to several units of code in different files, but not the entire module" use case? Because if there's no way to scope a symbol to "everything inside this submodule, but nothing outside this submodule", I think it leaves that use case unserved.

Unless I’m missing something there is also another encapsulation-related problem with the proposed design. Let’s suppose for the sake of discussion there was a `submoduleprivate` access modifier (intentionally ungainly and not realistic).

// File 1
module Foo {
  // internal, visible to the whole module
  class Bar { submoduleprivate var protectedState: Int = 0 }
}

// File 2 - Has nothing to do with Foo at all
import MyModule.Foo

module NotFoo {
// Hey, I need to see Bar.protectedState!!!
func totallyNotFoo() {
     var bar = Bar()
     bar.foosExposedPrivates = 42
}
}

// ok, I’ll just add an extension to Foo so I can see submoduleprivate and wrap what I need
module Foo {

Oops, this should have been `extension Foo`, but otherwise I believe it is valid under this proposal.

  // Hey, I’ll be nice and keep it fileprivate, but I could make it public if I wanted to.
  extension Foo {
       fileprivate var foosExposedPrivates: Int {
          // Yep, I’m inside Foo so I can see it’s submoduleprivate stuff
          get { return protectedState }
          set { protectedState = newValue }
       }
  }
}

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

To my mind, any submodule system for Swift should be designed to relieve
the pressure for long files, and make it easy to group tightly related
files into a single unit with shared visibility. That way developers can
easily organize their code into smaller files while utilizing Swift’s
pattern of providing protocol conformances in extensions and keeping
implementation details hidden from the rest of the module at large.

Wonderful, because that’s absolutely supported by this proposal. To
group tightly related files into a single unit, simply declare a submodule
for them and extend it in each of your related files.

It's supported, but it isn't first-class. By this I mean: there are two
distinguishable uses supported by your proposal, lumped together by the
fact that they are both about grouping units of code together. Put crudely,
one use case is grouping lines of code, while the other is about grouping
files of code. The merits of supporting both have already been debated in
this discussion. The issue I'll touch on is supporting both with the same
syntax. The chief drawbacks here are:

What exactly would be required to *make* it first class? Referencing
file names in the module declaration?

See below.

- It makes sense to use braces to group lines of code, but it makes no
sense to use braces to group files of code; this just causes entire files
to be indented.

If braces aren’t used to demarcate scopes, nesting modules becomes
ambiguous.

Again, let's observe the distinction about grouping files vs. grouping
lines.

Grouping files does not require braces: if the intended use of your feature
were to label files X, Y, and Z as belonging to one submodule and A, B, and
C to another, it would not matter if X, Y, and Z belonged to Foo.Bar and A,
B, and C to Foo.Bar.Baz: your syntax would not require braces.

It’s important to note that indentation is one particular style. LLVM
code style, in particular, chooses not to indent after namespace
declarations. This issue also crops up when dealing with nested type
declarations, and I distinctly remember it not being a big enough deal to
"fix this" at the time when a proposal to “flatten” these declaration was
brought up.

Mine is not a critique of the syntax itself; I don't particularly care
about indents, nor do I mind not indenting namespaces.

What I'm saying is, you would not have chosen to require braces if your
proposed feature were aimed at making the grouping of files into submodules
as simple as possible. You chose to accommodate grouping lines using the
same syntax as grouping files over the simplest design for grouping files.
Make no mistake, this promotes one use over another.

- Because some lines of code necessarily precede some other lines of code,

it makes sense to declare the first group using `module` and to extend that
with the second group using `extension`. However, because a file of code
does not necessarily precede another file of code, it is arbitrary which
file is surrounded with a `module` declaration and which one is surrounded
with an `extension` declaration.

Absolutely. But it is similarly arbitrary which public APIs are exposed
in a type declaration and which are exposed in an extension declaration.

Not entirely, no. Stored properties must be in the type declaration. Enum
cases must be in the type declaration. Perhaps you regard these as
temporary inconveniences of the current grammar; I see them as quite
reasonable ways to give some consistency as to what's written where in a
language where types can be retroactively extended. In a very real sense,
you must read the type declaration before you read the extensions in order
to understand the latter. By comparison, there is nothing that must be in
your proposed module declaration.

My hope is that the module declaration itself will become the
one-stop-shop for re-exports and general public bookkeeping just as
aggregate declarations are today. Module extensions exist to accommodate
users that wish to break related functionality across files or into
separate independent regions within the same file for the same reasons type
extensions exist.

Indeed, that you phrase it this way supports Nevin's argument. _Module
extensions_ exist to accommodate his use case; however, his use case
(which, mind you, is what I think most people are thinking of when it comes
to submodules, given previous threads on this topic) isn't the raison
d'etre for your submodule proposal. Quite simply, a syntax that
accommodates both grouping lines and grouping files cannot make the latter
first class, because the former necessarily requires more ceremony.

Any variables defined with `internal` access will be visible across those

···

On Tue, Feb 21, 2017 at 8:22 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

On Feb 21, 2017, at 9:13 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
On Tue, Feb 21, 2017 at 7:59 PM, Robert Widmann via swift-evolution < > swift-evolution@swift.org> wrote:

On Feb 21, 2017, at 7:36 PM, Nevin Brackett-Rozinsky via swift-evolution < >> swift-evolution@swift.org> wrote:
files to those extensions and only those extensions (see the section on
access control and modules). Any variables declared fileprivate or private
will, obviously, not be visible across these files. As an example:

// FooUtilities.swift
//
// -module-name=Foo
// module Foo {
// Defines Foo.Utilities
module Utilities {
  public func exportableOutsideThisSubmodule() {}
  func visibleInThisSubmodule() {}
  private func invisibleToOtherFiles() {}
}
//}

// FooUtilities+MoreUtilities.swift
extension Utilities {
  private func privateHelper() {
    visibleInThisSubmodule()
  }
}

I’m not sure where you got the impression that we were just trying to
make another fileprivate happen.

Nevin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

To my mind, any submodule system for Swift should be designed to relieve the pressure for long files, and make it easy to group tightly related files into a single unit with shared visibility. That way developers can easily organize their code into smaller files while utilizing Swift’s pattern of providing protocol conformances in extensions and keeping implementation details hidden from the rest of the module at large.

Wonderful, because that’s absolutely supported by this proposal. To group tightly related files into a single unit, simply declare a submodule for them and extend it in each of your related files.

It's supported, but it isn't first-class. By this I mean: there are two distinguishable uses supported by your proposal, lumped together by the fact that they are both about grouping units of code together. Put crudely, one use case is grouping lines of code, while the other is about grouping files of code. The merits of supporting both have already been debated in this discussion. The issue I'll touch on is supporting both with the same syntax. The chief drawbacks here are:

What exactly would be required to make it first class? Referencing file names in the module declaration?

See below.

- It makes sense to use braces to group lines of code, but it makes no sense to use braces to group files of code; this just causes entire files to be indented.

If braces aren’t used to demarcate scopes, nesting modules becomes ambiguous.

Again, let's observe the distinction about grouping files vs. grouping lines.

Grouping files does not require braces: if the intended use of your feature were to label files X, Y, and Z as belonging to one submodule and A, B, and C to another, it would not matter if X, Y, and Z belonged to Foo.Bar and A, B, and C to Foo.Bar.Baz: your syntax would not require braces.

It’s important to note that indentation is one particular style. LLVM code style, in particular, chooses not to indent after namespace declarations. This issue also crops up when dealing with nested type declarations, and I distinctly remember it not being a big enough deal to "fix this" at the time when a proposal to “flatten” these declaration was brought up.

Mine is not a critique of the syntax itself; I don't particularly care about indents, nor do I mind not indenting namespaces.

What I'm saying is, you would not have chosen to require braces if your proposed feature were aimed at making the grouping of files into submodules as simple as possible. You chose to accommodate grouping lines using the same syntax as grouping files over the simplest design for grouping files. Make no mistake, this promotes one use over another.

Ah, I see. Yes, one of the stated goals is to become filesystem-independent. We certainly cannot do that by encouraging the alternative.

- Because some lines of code necessarily precede some other lines of code, it makes sense to declare the first group using `module` and to extend that with the second group using `extension`. However, because a file of code does not necessarily precede another file of code, it is arbitrary which file is surrounded with a `module` declaration and which one is surrounded with an `extension` declaration.

Absolutely. But it is similarly arbitrary which public APIs are exposed in a type declaration and which are exposed in an extension declaration.

Not entirely, no. Stored properties must be in the type declaration. Enum cases must be in the type declaration. Perhaps you regard these as temporary inconveniences of the current grammar; I see them as quite reasonable ways to give some consistency as to what's written where in a language where types can be retroactively extended. In a very real sense, you must read the type declaration before you read the extensions in order to understand the latter. By comparison, there is nothing that must be in your proposed module declaration.

My hope is that the module declaration itself will become the one-stop-shop for re-exports and general public bookkeeping just as aggregate declarations are today. Module extensions exist to accommodate users that wish to break related functionality across files or into separate independent regions within the same file for the same reasons type extensions exist.

Indeed, that you phrase it this way supports Nevin's argument. _Module extensions_ exist to accommodate his use case; however, his use case (which, mind you, is what I think most people are thinking of when it comes to submodules, given previous threads on this topic) isn't the raison d'etre for your submodule proposal. Quite simply, a syntax that accommodates both grouping lines and grouping files cannot make the latter first class, because the former necessarily requires more ceremony.

Okay, but the question still stands: what do we need to make Nevin's use-case first class? To my mind, we’ve offered a syntax and semantics internal to the language that supports file-only aggregation because file-only aggregation enables a subset of the actual use cases of this module system. We aren’t enforcing this by compiler-fiat because it is a stylistic choice that can be enforced by a linter.

···

On Feb 21, 2017, at 9:37 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
On Tue, Feb 21, 2017 at 8:22 PM, Robert Widmann <devteam.codafi@gmail.com <mailto:devteam.codafi@gmail.com>>wrote:

On Feb 21, 2017, at 9:13 PM, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> wrote:
On Tue, Feb 21, 2017 at 7:59 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Feb 21, 2017, at 7:36 PM, Nevin Brackett-Rozinsky via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Any variables defined with `internal` access will be visible across those files to those extensions and only those extensions (see the section on access control and modules). Any variables declared fileprivate or private will, obviously, not be visible across these files. As an example:

// FooUtilities.swift
//
// -module-name=Foo
// module Foo {
// Defines Foo.Utilities
module Utilities {
  public func exportableOutsideThisSubmodule() {}
  func visibleInThisSubmodule() {}
  private func invisibleToOtherFiles() {}
}
//}

// FooUtilities+MoreUtilities.swift
extension Utilities {
  private func privateHelper() {
    visibleInThisSubmodule()
  }
}

I’m not sure where you got the impression that we were just trying to make another fileprivate happen.

Nevin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

This level of access, the “private to this submodule except to the select set of interfaces I want to see it” level, is the equivalent of friend classes in C++. I don’t consider leaving this out to be a hole, nor is it an "encapsulation-related problem” because at no point can you break the API boundary and re-export anything here with a higher level of access than it had previously.

By API boundary you mean the top-level module, right?

···

On Feb 21, 2017, at 10:29 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

On Feb 21, 2017, at 11:13 PM, Matthew Johnson <matthew@anandabits.com> wrote:

On Feb 21, 2017, at 10:11 PM, Matthew Johnson <matthew@anandabits.com> wrote:

On Feb 21, 2017, at 9:47 PM, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> wrote:

On Feb 21, 2017, at 7:38 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

Correct. Because, in dividing the submodule across an extension, you have placed what should be a private API into a differently-scoped location.

Okay. So is your submodule design not intended to address the "I want to encapsulate implementation details so they're only visible to several units of code in different files, but not the entire module" use case? Because if there's no way to scope a symbol to "everything inside this submodule, but nothing outside this submodule", I think it leaves that use case unserved.

Unless I’m missing something there is also another encapsulation-related problem with the proposed design. Let’s suppose for the sake of discussion there was a `submoduleprivate` access modifier (intentionally ungainly and not realistic).

// File 1
module Foo {
// internal, visible to the whole module
class Bar { submoduleprivate var protectedState: Int = 0 }
}

// File 2 - Has nothing to do with Foo at all
import MyModule.Foo

module NotFoo {
// Hey, I need to see Bar.protectedState!!!
func totallyNotFoo() {
    var bar = Bar()
    bar.foosExposedPrivates = 42
}
}

// ok, I’ll just add an extension to Foo so I can see submoduleprivate and wrap what I need
module Foo {

Oops, this should have been `extension Foo`, but otherwise I believe it is valid under this proposal.

// Hey, I’ll be nice and keep it fileprivate, but I could make it public if I wanted to.
extension Foo {
      fileprivate var foosExposedPrivates: Int {
         // Yep, I’m inside Foo so I can see it’s submoduleprivate stuff
         get { return protectedState }
         set { protectedState = newValue }
      }
}
}

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Whoops, move bar() out of the utility and you get the actual answer here, because you want to be able to see this in foo()!

···

On Feb 21, 2017, at 10:54 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

This case is unserved because it is anti-modular and a nightmare to maintain. The actual way to structure this is to factor bar() and baz() into their own utility submodule, or even deeper if necessary, that has as a parent the primary module that wishes to consume them both but not re-export them out to the parent. For example,

  // foo.swift
  import MyMod.Submodule
  func foo() {
    bar()
  }

  // bar.swift

       // Make my utilities visible in Foo.Submodule, but invisible to the parent.
       import Foo.Submodule.UtilitySubmodule

  module Submodule {
    module UtilitySubmodule {
                  internal func bar() {
      baz()
      }
              }
  }

  // baz.swift
  extension Submodule.UtilitySubmodule {
    internal func baz() {
      …
    }
  }

The thought is that it should be as cheap to create submodules to organize interfaces under as it is to create new directories to organize code under.

Though, this example is a little odd given that you’re defining and importing the same utility submodule. Really, what you would want to do is declare the utility in its own file/files outside of bar.swift to really take full advantage of the separation afforded here, then consume it internally with the import as written here.

On Feb 21, 2017, at 10:47 PM, Brent Royal-Gordon <brent@architechies.com> wrote:

On Feb 21, 2017, at 7:38 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

Correct. Because, in dividing the submodule across an extension, you have placed what should be a private API into a differently-scoped location.

Okay. So is your submodule design not intended to address the "I want to encapsulate implementation details so they're only visible to several units of code in different files, but not the entire module" use case? Because if there's no way to scope a symbol to "everything inside this submodule, but nothing outside this submodule", I think it leaves that use case unserved.

--
Brent Royal-Gordon
Architechies

By API boundaries I mean both the one internal to MyModule.Foo and the one defined by MyModule. Here “the API boundary” is explicitly about the submodule MyModule.Foo, whose internal state may have been “unsealed” in the top level by the extension, but has not been re-exported.

~Robert Widmann

···

On Feb 21, 2017, at 11:38 PM, Matthew Johnson <matthew@anandabits.com> wrote:

On Feb 21, 2017, at 10:29 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

This level of access, the “private to this submodule except to the select set of interfaces I want to see it” level, is the equivalent of friend classes in C++. I don’t consider leaving this out to be a hole, nor is it an "encapsulation-related problem” because at no point can you break the API boundary and re-export anything here with a higher level of access than it had previously.

By API boundary you mean the top-level module, right?

On Feb 21, 2017, at 11:13 PM, Matthew Johnson <matthew@anandabits.com> wrote:

On Feb 21, 2017, at 10:11 PM, Matthew Johnson <matthew@anandabits.com> wrote:

On Feb 21, 2017, at 9:47 PM, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> wrote:

On Feb 21, 2017, at 7:38 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

Correct. Because, in dividing the submodule across an extension, you have placed what should be a private API into a differently-scoped location.

Okay. So is your submodule design not intended to address the "I want to encapsulate implementation details so they're only visible to several units of code in different files, but not the entire module" use case? Because if there's no way to scope a symbol to "everything inside this submodule, but nothing outside this submodule", I think it leaves that use case unserved.

Unless I’m missing something there is also another encapsulation-related problem with the proposed design. Let’s suppose for the sake of discussion there was a `submoduleprivate` access modifier (intentionally ungainly and not realistic).

// File 1
module Foo {
// internal, visible to the whole module
class Bar { submoduleprivate var protectedState: Int = 0 }
}

// File 2 - Has nothing to do with Foo at all
import MyModule.Foo

module NotFoo {
// Hey, I need to see Bar.protectedState!!!
func totallyNotFoo() {
   var bar = Bar()
   bar.foosExposedPrivates = 42
}
}

// ok, I’ll just add an extension to Foo so I can see submoduleprivate and wrap what I need
module Foo {

Oops, this should have been `extension Foo`, but otherwise I believe it is valid under this proposal.

// Hey, I’ll be nice and keep it fileprivate, but I could make it public if I wanted to.
extension Foo {
     fileprivate var foosExposedPrivates: Int {
        // Yep, I’m inside Foo so I can see it’s submoduleprivate stuff
        get { return protectedState }
        set { protectedState = newValue }
     }
}
}

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

To my mind, any submodule system for Swift should be designed to relieve
the pressure for long files, and make it easy to group tightly related
files into a single unit with shared visibility. That way developers can
easily organize their code into smaller files while utilizing Swift’s
pattern of providing protocol conformances in extensions and keeping
implementation details hidden from the rest of the module at large.

Wonderful, because that’s absolutely supported by this proposal. To
group tightly related files into a single unit, simply declare a submodule
for them and extend it in each of your related files.

It's supported, but it isn't first-class. By this I mean: there are two
distinguishable uses supported by your proposal, lumped together by the
fact that they are both about grouping units of code together. Put crudely,
one use case is grouping lines of code, while the other is about grouping
files of code. The merits of supporting both have already been debated in
this discussion. The issue I'll touch on is supporting both with the same
syntax. The chief drawbacks here are:

What exactly would be required to *make* it first class? Referencing
file names in the module declaration?

See below.

- It makes sense to use braces to group lines of code, but it makes no
sense to use braces to group files of code; this just causes entire files
to be indented.

If braces aren’t used to demarcate scopes, nesting modules becomes
ambiguous.

Again, let's observe the distinction about grouping files vs. grouping
lines.

Grouping files does not require braces: if the intended use of your
feature were to label files X, Y, and Z as belonging to one submodule and
A, B, and C to another, it would not matter if X, Y, and Z belonged to
Foo.Bar and A, B, and C to Foo.Bar.Baz: your syntax would not require
braces.

It’s important to note that indentation is one particular style. LLVM
code style, in particular, chooses not to indent after namespace
declarations. This issue also crops up when dealing with nested type
declarations, and I distinctly remember it not being a big enough deal to
"fix this" at the time when a proposal to “flatten” these declaration was
brought up.

Mine is not a critique of the syntax itself; I don't particularly care
about indents, nor do I mind not indenting namespaces.

What I'm saying is, you would not have chosen to require braces if your
proposed feature were aimed at making the grouping of files into submodules
as simple as possible. You chose to accommodate grouping lines using the
same syntax as grouping files over the simplest design for grouping files.
Make no mistake, this promotes one use over another.

Ah, I see. Yes, one of the stated goals is to become
filesystem-independent. We certainly cannot do that by encouraging the
alternative.

Swift's current design is deliberately not file system-independent. A
submodule design built on top of Swift could preserve that. Your draft
proposal makes two changes: it introduces a design for submodules; and, it
eliminates files as a unit of code by default (not least by declaring
`fileprivate` redundant). To my mind, you have presented no justification
for the second change other than to say that it is a stated goal--but why?

- Because some lines of code necessarily precede some other lines of code,

it makes sense to declare the first group using `module` and to extend that
with the second group using `extension`. However, because a file of code
does not necessarily precede another file of code, it is arbitrary which
file is surrounded with a `module` declaration and which one is surrounded
with an `extension` declaration.

Absolutely. But it is similarly arbitrary which public APIs are exposed
in a type declaration and which are exposed in an extension declaration.

Not entirely, no. Stored properties must be in the type declaration. Enum
cases must be in the type declaration. Perhaps you regard these as
temporary inconveniences of the current grammar; I see them as quite
reasonable ways to give some consistency as to what's written where in a
language where types can be retroactively extended. In a very real sense,
you must read the type declaration before you read the extensions in order
to understand the latter. By comparison, there is nothing that must be in
your proposed module declaration.

My hope is that the module declaration itself will become the
one-stop-shop for re-exports and general public bookkeeping just as
aggregate declarations are today. Module extensions exist to accommodate
users that wish to break related functionality across files or into
separate independent regions within the same file for the same reasons type
extensions exist.

Indeed, that you phrase it this way supports Nevin's argument. _Module
extensions_ exist to accommodate his use case; however, his use case
(which, mind you, is what I think most people are thinking of when it comes
to submodules, given previous threads on this topic) isn't the raison
d'etre for your submodule proposal. Quite simply, a syntax that
accommodates both grouping lines and grouping files cannot make the latter
first class, because the former necessarily requires more ceremony.

Okay, but the question still stands: what do we need to make Nevin's
use-case first class?

We need to have either *two* spellings for a submodule feature that
supports both grouping files and grouping lines of code, or we need to have
*two* features, one for grouping files and another for grouping lines of
code. In any case, the spelling for grouping files should:
- Not require braces
- Not require one file to be a `module` and another to be an `extension`

The simplest such spelling would be:

File A:

module Foo
// rest of file

File B:

module Foo
// rest of file

To my mind, we’ve offered a syntax and semantics internal to the language

that supports file-only aggregation because file-only aggregation enables a
subset of the actual use cases of this module system. We aren’t enforcing
this by compiler-fiat because it is a stylistic choice that can be enforced
by a linter.

It's not enough to offer a syntax and semantics that supports it; if it is
the intended major use case, the design ought to reflect that by making
that use case no less cumbersome than necessary.

Any variables defined with `internal` access will be visible across those

···

On Tue, Feb 21, 2017 at 8:41 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

On Feb 21, 2017, at 9:37 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
On Tue, Feb 21, 2017 at 8:22 PM, Robert Widmann <devteam.codafi@gmail.com> > wrote:

On Feb 21, 2017, at 9:13 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
On Tue, Feb 21, 2017 at 7:59 PM, Robert Widmann via swift-evolution < >> swift-evolution@swift.org> wrote:

On Feb 21, 2017, at 7:36 PM, Nevin Brackett-Rozinsky via swift-evolution >>> <swift-evolution@swift.org> wrote:
files to those extensions and only those extensions (see the section on
access control and modules). Any variables declared fileprivate or private
will, obviously, not be visible across these files. As an example:

// FooUtilities.swift
//
// -module-name=Foo
// module Foo {
// Defines Foo.Utilities
module Utilities {
  public func exportableOutsideThisSubmodule() {}
  func visibleInThisSubmodule() {}
  private func invisibleToOtherFiles() {}
}
//}

// FooUtilities+MoreUtilities.swift
extension Utilities {
  private func privateHelper() {
    visibleInThisSubmodule()
  }
}

I’m not sure where you got the impression that we were just trying to
make another fileprivate happen.

Nevin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

To my mind, any submodule system for Swift should be designed to relieve the pressure for long files, and make it easy to group tightly related files into a single unit with shared visibility. That way developers can easily organize their code into smaller files while utilizing Swift’s pattern of providing protocol conformances in extensions and keeping implementation details hidden from the rest of the module at large.

Wonderful, because that’s absolutely supported by this proposal. To group tightly related files into a single unit, simply declare a submodule for them and extend it in each of your related files.

It's supported, but it isn't first-class. By this I mean: there are two distinguishable uses supported by your proposal, lumped together by the fact that they are both about grouping units of code together. Put crudely, one use case is grouping lines of code, while the other is about grouping files of code. The merits of supporting both have already been debated in this discussion. The issue I'll touch on is supporting both with the same syntax. The chief drawbacks here are:

What exactly would be required to make it first class? Referencing file names in the module declaration?

See below.

- It makes sense to use braces to group lines of code, but it makes no sense to use braces to group files of code; this just causes entire files to be indented.

If braces aren’t used to demarcate scopes, nesting modules becomes ambiguous.

Again, let's observe the distinction about grouping files vs. grouping lines.

Grouping files does not require braces: if the intended use of your feature were to label files X, Y, and Z as belonging to one submodule and A, B, and C to another, it would not matter if X, Y, and Z belonged to Foo.Bar and A, B, and C to Foo.Bar.Baz: your syntax would not require braces.

It’s important to note that indentation is one particular style. LLVM code style, in particular, chooses not to indent after namespace declarations. This issue also crops up when dealing with nested type declarations, and I distinctly remember it not being a big enough deal to "fix this" at the time when a proposal to “flatten” these declaration was brought up.

Mine is not a critique of the syntax itself; I don't particularly care about indents, nor do I mind not indenting namespaces.

What I'm saying is, you would not have chosen to require braces if your proposed feature were aimed at making the grouping of files into submodules as simple as possible. You chose to accommodate grouping lines using the same syntax as grouping files over the simplest design for grouping files. Make no mistake, this promotes one use over another.

Ah, I see. Yes, one of the stated goals is to become filesystem-independent. We certainly cannot do that by encouraging the alternative.

Swift's current design is deliberately not file system-independent. A submodule design built on top of Swift could preserve that. Your draft proposal makes two changes: it introduces a design for submodules; and, it eliminates files as a unit of code by default (not least by declaring `fileprivate` redundant). To my mind, you have presented no justification for the second change other than to say that it is a stated goal--but why?

fileprivate access can be recreated by creating a private "utility submodule" containing declarations of at most internal access.

- Because some lines of code necessarily precede some other lines of code, it makes sense to declare the first group using `module` and to extend that with the second group using `extension`. However, because a file of code does not necessarily precede another file of code, it is arbitrary which file is surrounded with a `module` declaration and which one is surrounded with an `extension` declaration.

Absolutely. But it is similarly arbitrary which public APIs are exposed in a type declaration and which are exposed in an extension declaration.

Not entirely, no. Stored properties must be in the type declaration. Enum cases must be in the type declaration. Perhaps you regard these as temporary inconveniences of the current grammar; I see them as quite reasonable ways to give some consistency as to what's written where in a language where types can be retroactively extended. In a very real sense, you must read the type declaration before you read the extensions in order to understand the latter. By comparison, there is nothing that must be in your proposed module declaration.

My hope is that the module declaration itself will become the one-stop-shop for re-exports and general public bookkeeping just as aggregate declarations are today. Module extensions exist to accommodate users that wish to break related functionality across files or into separate independent regions within the same file for the same reasons type extensions exist.

Indeed, that you phrase it this way supports Nevin's argument. _Module extensions_ exist to accommodate his use case; however, his use case (which, mind you, is what I think most people are thinking of when it comes to submodules, given previous threads on this topic) isn't the raison d'etre for your submodule proposal. Quite simply, a syntax that accommodates both grouping lines and grouping files cannot make the latter first class, because the former necessarily requires more ceremony.

Okay, but the question still stands: what do we need to make Nevin's use-case first class?

We need to have either *two* spellings for a submodule feature that supports both grouping files and grouping lines of code, or we need to have *two* features, one for grouping files and another for grouping lines of code. In any case, the spelling for grouping files should:
- Not require braces
- Not require one file to be a `module` and another to be an `extension`

The simplest such spelling would be:

File A:

module Foo
// rest of file

File B:

module Foo
// rest of file

This was mentioned earlier, so I will quote my response to the last poster:

That is a valid spelling (Rust, IIRC, allows that spelling), but one that is easy to miss sitting in a file and makes it confusing to introduce submodules. If you include the annotation then define a submodule later down in the file, suddenly you have to remember whether you annotated the file or whether the submodule you’ve just written is going into the top-level module. See:

// -module-name=Foo
// module Foo {
module Bar; // Shorthand for “This file defines Foo.Bar”

/* Code */

// This defines “Foo.Bar.Baz”, but would you know that if it appeared below the fold?
module Baz {}
//}

But that reply was oriented towards a “why can’t we have nested modules and a shorthand too?” Here, you’re referring more to the one-module-one-file language restrictions, so I will quote another response:

For one-file-per-module, that kind of restriction represents a particular way of organizing code and is a design pattern that is supported under this proposal. We just happen to not enforce that particular pattern, and feel that it is the job of a linter to do so. Really, this kind of restriction is to ease the mental burden on compiler writers who use it to build compilation unit dependency graphs. Swift already considers all files when building its modules because you can extend any type (and now, any module) from any one of them so it doesn’t buy us anything other than an arbitrary restriction.

Realistically, the only difference between the proposal’s syntax and this one is two characters (the braces) and maybe some tabs if you decide to enforce that style. There are few objective organizational benefits from your style, and it creates a bureaucratic rule where none need exist. Subjectively, of course, people may prefer this way, or they may prefer a more ad-hoc approach. But we designed for both cases and showed our hand by favoring non-physical organization because it is a subset of possible organizational styles available to users.

To my mind, we’ve offered a syntax and semantics internal to the language that supports file-only aggregation because file-only aggregation enables a subset of the actual use cases of this module system. We aren’t enforcing this by compiler-fiat because it is a stylistic choice that can be enforced by a linter.

It's not enough to offer a syntax and semantics that supports it; if it is the intended major use case, the design ought to reflect that by making that use case no less cumbersome than necessary.

See above for aesthetic concerns.

···

On Feb 21, 2017, at 10:03 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
On Tue, Feb 21, 2017 at 8:41 PM, Robert Widmann <devteam.codafi@gmail.com <mailto:devteam.codafi@gmail.com>>wrote:

On Feb 21, 2017, at 9:37 PM, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> wrote:
On Tue, Feb 21, 2017 at 8:22 PM, Robert Widmann <devteam.codafi@gmail.com <mailto:devteam.codafi@gmail.com>>wrote:

On Feb 21, 2017, at 9:13 PM, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> wrote:
On Tue, Feb 21, 2017 at 7:59 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Feb 21, 2017, at 7:36 PM, Nevin Brackett-Rozinsky via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Any variables defined with `internal` access will be visible across those files to those extensions and only those extensions (see the section on access control and modules). Any variables declared fileprivate or private will, obviously, not be visible across these files. As an example:

// FooUtilities.swift
//
// -module-name=Foo
// module Foo {
// Defines Foo.Utilities
module Utilities {
  public func exportableOutsideThisSubmodule() {}
  func visibleInThisSubmodule() {}
  private func invisibleToOtherFiles() {}
}
//}

// FooUtilities+MoreUtilities.swift
extension Utilities {
  private func privateHelper() {
    visibleInThisSubmodule()
  }
}

I’m not sure where you got the impression that we were just trying to make another fileprivate happen.

Nevin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

By API boundaries I mean both the one internal to MyModule.Foo and the one defined by MyModule. Here “the API boundary” is explicitly about the submodule MyModule.Foo, whose internal state may have been “unsealed” in the top level by the extension, but has not been re-exported.

I’m sorry, but I just don’t understand how modules form an API boundary in this system. To me a boundary means something that blocks access. In this system `internal` ranges over the entire module and all submodules. The only boundaries I can see besides the module itself are files and lexical scopes (with `fileprivate` and `private`).

This means that it is trivial to put code anywhere within the module that extends the submodule and wraps a symbol in a new name and declares it `public`. They can also trivially add a `public import MyModule.Foo` anywhere at the top level of their file because every file is forced to include top level scope.

In my opinion, we need to identify what goals we have for a submodule system - what problems are we trying to solve and what use cases do we intend to enable.

There are quite a few of us who want the ability to form solid API boundaries inside a module and view this as one of the fundamental features of a submodule system. It’s reasonable to ask why we view this capability as essential.

I can’t speak for anyone else, but here are a few reasons why it’s important to me:

* Solid API boundaries are essential to good design.
* Having access to an entire code base does not reduce the benefits of #1. Some code bases are substantial in size and hard boundaries are important to keeping them manageable.
* Using full-fledged modules to do this is possible, but also involves a bit of ceremony that is incidental, not essential complexity in many cases. It would be better to have a lighter weight mechanism to do this.
* Swift currently only has whole module optimization, not whole program optimization. There is a performance penalty to using full-fledged modules.

···

On Feb 21, 2017, at 10:41 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

~Robert Widmann

On Feb 21, 2017, at 11:38 PM, Matthew Johnson <matthew@anandabits.com> wrote:

On Feb 21, 2017, at 10:29 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

This level of access, the “private to this submodule except to the select set of interfaces I want to see it” level, is the equivalent of friend classes in C++. I don’t consider leaving this out to be a hole, nor is it an "encapsulation-related problem” because at no point can you break the API boundary and re-export anything here with a higher level of access than it had previously.

By API boundary you mean the top-level module, right?

On Feb 21, 2017, at 11:13 PM, Matthew Johnson <matthew@anandabits.com> wrote:

On Feb 21, 2017, at 10:11 PM, Matthew Johnson <matthew@anandabits.com> wrote:

On Feb 21, 2017, at 9:47 PM, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> wrote:

On Feb 21, 2017, at 7:38 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

Correct. Because, in dividing the submodule across an extension, you have placed what should be a private API into a differently-scoped location.

Okay. So is your submodule design not intended to address the "I want to encapsulate implementation details so they're only visible to several units of code in different files, but not the entire module" use case? Because if there's no way to scope a symbol to "everything inside this submodule, but nothing outside this submodule", I think it leaves that use case unserved.

Unless I’m missing something there is also another encapsulation-related problem with the proposed design. Let’s suppose for the sake of discussion there was a `submoduleprivate` access modifier (intentionally ungainly and not realistic).

// File 1
module Foo {
// internal, visible to the whole module
class Bar { submoduleprivate var protectedState: Int = 0 }
}

// File 2 - Has nothing to do with Foo at all
import MyModule.Foo

module NotFoo {
// Hey, I need to see Bar.protectedState!!!
func totallyNotFoo() {
  var bar = Bar()
  bar.foosExposedPrivates = 42
}
}

// ok, I’ll just add an extension to Foo so I can see submoduleprivate and wrap what I need
module Foo {

Oops, this should have been `extension Foo`, but otherwise I believe it is valid under this proposal.

// Hey, I’ll be nice and keep it fileprivate, but I could make it public if I wanted to.
extension Foo {
    fileprivate var foosExposedPrivates: Int {
       // Yep, I’m inside Foo so I can see it’s submoduleprivate stuff
       get { return protectedState }
       set { protectedState = newValue }
    }
}
}

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

By API boundaries I mean both the one internal to MyModule.Foo and the one defined by MyModule. Here “the API boundary” is explicitly about the submodule MyModule.Foo, whose internal state may have been “unsealed” in the top level by the extension, but has not been re-exported.

I’m sorry, but I just don’t understand how modules form an API boundary in this system. To me a boundary means something that blocks access. In this system `internal` ranges over the entire module and all submodules. The only boundaries I can see besides the module itself are files and lexical scopes (with `fileprivate` and `private`).

A module is a named region that introduces a lexical scope into which declarations may be nested. The name of the module can be used to access these member declarations. A module, like other aggregate structures in Swift, may be extended with new declarations over one or more translation units (files).

Your API boundary lives, as it does today, at the edges of each (sub)module declaration. APIs that are public or open in a module defines code that is free to move across this boundary and into the open. APIs that are internal are free to have their modules unsealed into other internal modules to enable modular composition. APIs that are private and fileprivate do not participate in the API boundary because they are not eligible for any kind of export.

If any of that is unclear, please let me know.

This means that it is trivial to put code anywhere within the module that extends the submodule and wraps a symbol in a new name and declares it `public`.

Precisely. That’s the same pattern that good Swift code, arguably good code in any language that enables hiding, uses today.

They can also trivially add a `public import MyModule.Foo` anywhere at the top level of their file because every file is forced to include top level scope.

Perhaps you misunderstand. Say the APIs in MyModule.Foo were all of internal or stricter access: The re-export is a no-op. You cannot change the access level of declarations, you can only do the modular thing and wrap them in a palatable interface for export by a module you want to be user-facing. You have to decide to make an API public, just as today you have to decide to make part of an interface public. I don’t see how this is distinct from the goals of this proposal.

···

On Feb 21, 2017, at 11:59 PM, Matthew Johnson <matthew@anandabits.com> wrote:

On Feb 21, 2017, at 10:41 PM, Robert Widmann <devteam.codafi@gmail.com <mailto:devteam.codafi@gmail.com>> wrote:

In my opinion, we need to identify what goals we have for a submodule system - what problems are we trying to solve and what use cases do we intend to enable.

There are quite a few of us who want the ability to form solid API boundaries inside a module and view this as one of the fundamental features of a submodule system. It’s reasonable to ask why we view this capability as essential.

I can’t speak for anyone else, but here are a few reasons why it’s important to me:

* Solid API boundaries are essential to good design.
* Having access to an entire code base does not reduce the benefits of #1. Some code bases are substantial in size and hard boundaries are important to keeping them manageable.
* Using full-fledged modules to do this is possible, but also involves a bit of ceremony that is incidental, not essential complexity in many cases. It would be better to have a lighter weight mechanism to do this.
* Swift currently only has whole module optimization, not whole program optimization. There is a performance penalty to using full-fledged modules.

~Robert Widmann

On Feb 21, 2017, at 11:38 PM, Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:

On Feb 21, 2017, at 10:29 PM, Robert Widmann <devteam.codafi@gmail.com <mailto:devteam.codafi@gmail.com>> wrote:

This level of access, the “private to this submodule except to the select set of interfaces I want to see it” level, is the equivalent of friend classes in C++. I don’t consider leaving this out to be a hole, nor is it an "encapsulation-related problem” because at no point can you break the API boundary and re-export anything here with a higher level of access than it had previously.

By API boundary you mean the top-level module, right?

On Feb 21, 2017, at 11:13 PM, Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:

On Feb 21, 2017, at 10:11 PM, Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:

On Feb 21, 2017, at 9:47 PM, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Feb 21, 2017, at 7:38 PM, Robert Widmann <devteam.codafi@gmail.com <mailto:devteam.codafi@gmail.com>> wrote:

Correct. Because, in dividing the submodule across an extension, you have placed what should be a private API into a differently-scoped location.

Okay. So is your submodule design not intended to address the "I want to encapsulate implementation details so they're only visible to several units of code in different files, but not the entire module" use case? Because if there's no way to scope a symbol to "everything inside this submodule, but nothing outside this submodule", I think it leaves that use case unserved.

Unless I’m missing something there is also another encapsulation-related problem with the proposed design. Let’s suppose for the sake of discussion there was a `submoduleprivate` access modifier (intentionally ungainly and not realistic).

// File 1
module Foo {
// internal, visible to the whole module
class Bar { submoduleprivate var protectedState: Int = 0 }
}

// File 2 - Has nothing to do with Foo at all
import MyModule.Foo

module NotFoo {
// Hey, I need to see Bar.protectedState!!!
func totallyNotFoo() {
  var bar = Bar()
  bar.foosExposedPrivates = 42
}
}

// ok, I’ll just add an extension to Foo so I can see submoduleprivate and wrap what I need
module Foo {

Oops, this should have been `extension Foo`, but otherwise I believe it is valid under this proposal.

// Hey, I’ll be nice and keep it fileprivate, but I could make it public if I wanted to.
extension Foo {
    fileprivate var foosExposedPrivates: Int {
       // Yep, I’m inside Foo so I can see it’s submoduleprivate stuff
       get { return protectedState }
       set { protectedState = newValue }
    }
}
}

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

To my mind, any submodule system for Swift should be designed to
relieve the pressure for long files, and make it easy to group tightly
related files into a single unit with shared visibility. That way
developers can easily organize their code into smaller files while
utilizing Swift’s pattern of providing protocol conformances in extensions
and keeping implementation details hidden from the rest of the module at
large.

Wonderful, because that’s absolutely supported by this proposal. To
group tightly related files into a single unit, simply declare a submodule
for them and extend it in each of your related files.

It's supported, but it isn't first-class. By this I mean: there are two
distinguishable uses supported by your proposal, lumped together by the
fact that they are both about grouping units of code together. Put crudely,
one use case is grouping lines of code, while the other is about grouping
files of code. The merits of supporting both have already been debated in
this discussion. The issue I'll touch on is supporting both with the same
syntax. The chief drawbacks here are:

What exactly would be required to *make* it first class? Referencing
file names in the module declaration?

See below.

- It makes sense to use braces to group lines of code, but it makes no
sense to use braces to group files of code; this just causes entire files
to be indented.

If braces aren’t used to demarcate scopes, nesting modules becomes
ambiguous.

Again, let's observe the distinction about grouping files vs. grouping
lines.

Grouping files does not require braces: if the intended use of your
feature were to label files X, Y, and Z as belonging to one submodule and
A, B, and C to another, it would not matter if X, Y, and Z belonged to
Foo.Bar and A, B, and C to Foo.Bar.Baz: your syntax would not require
braces.

It’s important to note that indentation is one particular style. LLVM
code style, in particular, chooses not to indent after namespace
declarations. This issue also crops up when dealing with nested type
declarations, and I distinctly remember it not being a big enough deal to
"fix this" at the time when a proposal to “flatten” these declaration was
brought up.

Mine is not a critique of the syntax itself; I don't particularly care
about indents, nor do I mind not indenting namespaces.

What I'm saying is, you would not have chosen to require braces if your
proposed feature were aimed at making the grouping of files into submodules
as simple as possible. You chose to accommodate grouping lines using the
same syntax as grouping files over the simplest design for grouping files.
Make no mistake, this promotes one use over another.

Ah, I see. Yes, one of the stated goals is to become
filesystem-independent. We certainly cannot do that by encouraging the
alternative.

Swift's current design is deliberately not file system-independent. A
submodule design built on top of Swift could preserve that. Your draft
proposal makes two changes: it introduces a design for submodules; and, it
eliminates files as a unit of code by default (not least by declaring
`fileprivate` redundant). To my mind, you have presented no justification
for the second change other than to say that it is a stated goal--but why?

   - fileprivate access can be recreated by creating a private "utility
   submodule" containing declarations of at most internal access.

- Because some lines of code necessarily precede some other lines of

code, it makes sense to declare the first group using `module` and to
extend that with the second group using `extension`. However, because a
file of code does not necessarily precede another file of code, it is
arbitrary which file is surrounded with a `module` declaration and which
one is surrounded with an `extension` declaration.

Absolutely. But it is similarly arbitrary which public APIs are exposed
in a type declaration and which are exposed in an extension declaration.

Not entirely, no. Stored properties must be in the type declaration. Enum
cases must be in the type declaration. Perhaps you regard these as
temporary inconveniences of the current grammar; I see them as quite
reasonable ways to give some consistency as to what's written where in a
language where types can be retroactively extended. In a very real sense,
you must read the type declaration before you read the extensions in order
to understand the latter. By comparison, there is nothing that must be in
your proposed module declaration.

My hope is that the module declaration itself will become the
one-stop-shop for re-exports and general public bookkeeping just as
aggregate declarations are today. Module extensions exist to accommodate
users that wish to break related functionality across files or into
separate independent regions within the same file for the same reasons type
extensions exist.

Indeed, that you phrase it this way supports Nevin's argument. _Module
extensions_ exist to accommodate his use case; however, his use case
(which, mind you, is what I think most people are thinking of when it comes
to submodules, given previous threads on this topic) isn't the raison
d'etre for your submodule proposal. Quite simply, a syntax that
accommodates both grouping lines and grouping files cannot make the latter
first class, because the former necessarily requires more ceremony.

Okay, but the question still stands: what do we need to make Nevin's
use-case first class?

We need to have either *two* spellings for a submodule feature that
supports both grouping files and grouping lines of code, or we need to have
*two* features, one for grouping files and another for grouping lines of
code. In any case, the spelling for grouping files should:
- Not require braces
- Not require one file to be a `module` and another to be an `extension`

The simplest such spelling would be:

File A:

module Foo
// rest of file

File B:

module Foo
// rest of file

This was mentioned earlier, so I will quote my response to the last poster:

That is a valid spelling (Rust, IIRC, allows that spelling), but one that
is easy to miss sitting in a file and makes it confusing to introduce
submodules. If you include the annotation then define a submodule later
down in the file, suddenly you have to remember whether you annotated the
file or whether the submodule you’ve just written is going into the
top-level module. See:

// -module-name=Foo
// module Foo {
module Bar; // Shorthand for “This file defines Foo.Bar”

/* Code */

// This defines “Foo.Bar.Baz”, but would you know that if it appeared
below the fold?
module Baz {}
//}

But that reply was oriented towards a “why can’t we have nested modules
and a shorthand too?” Here, you’re referring more to the
one-module-one-file language restrictions, so I will quote another response:

For one-file-per-module, that kind of restriction represents a particular
way of organizing code and is a design pattern that is supported under this
proposal. We just happen to not enforce that particular pattern, and feel
that it is the job of a linter to do so. Really, this kind of restriction
is to ease the mental burden on compiler writers who use it to build
compilation unit dependency graphs. Swift already considers all files when
building its modules because you can extend any type (and now, any module)
from any one of them so it doesn’t buy us anything other than an arbitrary
restriction.

Realistically, the only difference between the proposal’s syntax and this
one is two characters (the braces) and maybe some tabs if you decide to
enforce that style. There are few objective organizational benefits from
your style, and it creates a bureaucratic rule where none need exist.

I'm sorry, what is the bureaucratic rule?

Subjectively, of course, people may prefer this way, or they may prefer a
more ad-hoc approach. But we designed for both cases and showed our hand
by favoring non-physical organization because it is a subset of possible
organizational styles available to users.

Indeed you did. But you will need to justify why *both* cases are important
to support, and moreover why you would favor the case that's the more
complicated and less often requested, and why the other case should be
burdened with even a single character that's not necessary for its own
support.

To my mind, we’ve offered a syntax and semantics internal to the language

···

On Tue, Feb 21, 2017 at 9:28 PM, Robert Widmann <devteam.codafi@gmail.com> wrote:

On Feb 21, 2017, at 10:03 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
On Tue, Feb 21, 2017 at 8:41 PM, Robert Widmann <devteam.codafi@gmail.com> > wrote:

On Feb 21, 2017, at 9:37 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
On Tue, Feb 21, 2017 at 8:22 PM, Robert Widmann <devteam.codafi@gmail.com >> >wrote:

On Feb 21, 2017, at 9:13 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
On Tue, Feb 21, 2017 at 7:59 PM, Robert Widmann via swift-evolution < >>> swift-evolution@swift.org> wrote:

On Feb 21, 2017, at 7:36 PM, Nevin Brackett-Rozinsky via >>>> swift-evolution <swift-evolution@swift.org> wrote:

that supports file-only aggregation because file-only aggregation enables a
subset of the actual use cases of this module system. We aren’t enforcing
this by compiler-fiat because it is a stylistic choice that can be enforced
by a linter.

It's not enough to offer a syntax and semantics that supports it; if it is
the intended major use case, the design ought to reflect that by making
that use case no less cumbersome than necessary.

See above for aesthetic concerns.

Any variables defined with `internal` access will be visible across those

files to those extensions and only those extensions (see the section on
access control and modules). Any variables declared fileprivate or private
will, obviously, not be visible across these files. As an example:

// FooUtilities.swift
//
// -module-name=Foo
// module Foo {
// Defines Foo.Utilities
module Utilities {
  public func exportableOutsideThisSubmodule() {}
  func visibleInThisSubmodule() {}
  private func invisibleToOtherFiles() {}
}
//}

// FooUtilities+MoreUtilities.swift
extension Utilities {
  private func privateHelper() {
    visibleInThisSubmodule()
  }
}

I’m not sure where you got the impression that we were just trying to
make another fileprivate happen.

Nevin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

To my mind, any submodule system for Swift should be designed to relieve the pressure for long files, and make it easy to group tightly related files into a single unit with shared visibility. That way developers can easily organize their code into smaller files while utilizing Swift’s pattern of providing protocol conformances in extensions and keeping implementation details hidden from the rest of the module at large.

Wonderful, because that’s absolutely supported by this proposal. To group tightly related files into a single unit, simply declare a submodule for them and extend it in each of your related files.

It's supported, but it isn't first-class. By this I mean: there are two distinguishable uses supported by your proposal, lumped together by the fact that they are both about grouping units of code together. Put crudely, one use case is grouping lines of code, while the other is about grouping files of code. The merits of supporting both have already been debated in this discussion. The issue I'll touch on is supporting both with the same syntax. The chief drawbacks here are:

What exactly would be required to make it first class? Referencing file names in the module declaration?

See below.

- It makes sense to use braces to group lines of code, but it makes no sense to use braces to group files of code; this just causes entire files to be indented.

If braces aren’t used to demarcate scopes, nesting modules becomes ambiguous.

Again, let's observe the distinction about grouping files vs. grouping lines.

Grouping files does not require braces: if the intended use of your feature were to label files X, Y, and Z as belonging to one submodule and A, B, and C to another, it would not matter if X, Y, and Z belonged to Foo.Bar and A, B, and C to Foo.Bar.Baz: your syntax would not require braces.

It’s important to note that indentation is one particular style. LLVM code style, in particular, chooses not to indent after namespace declarations. This issue also crops up when dealing with nested type declarations, and I distinctly remember it not being a big enough deal to "fix this" at the time when a proposal to “flatten” these declaration was brought up.

Mine is not a critique of the syntax itself; I don't particularly care about indents, nor do I mind not indenting namespaces.

What I'm saying is, you would not have chosen to require braces if your proposed feature were aimed at making the grouping of files into submodules as simple as possible. You chose to accommodate grouping lines using the same syntax as grouping files over the simplest design for grouping files. Make no mistake, this promotes one use over another.

Ah, I see. Yes, one of the stated goals is to become filesystem-independent. We certainly cannot do that by encouraging the alternative.

Swift's current design is deliberately not file system-independent. A submodule design built on top of Swift could preserve that. Your draft proposal makes two changes: it introduces a design for submodules; and, it eliminates files as a unit of code by default (not least by declaring `fileprivate` redundant). To my mind, you have presented no justification for the second change other than to say that it is a stated goal--but why?

fileprivate access can be recreated by creating a private "utility submodule" containing declarations of at most internal access.

- Because some lines of code necessarily precede some other lines of code, it makes sense to declare the first group using `module` and to extend that with the second group using `extension`. However, because a file of code does not necessarily precede another file of code, it is arbitrary which file is surrounded with a `module` declaration and which one is surrounded with an `extension` declaration.

Absolutely. But it is similarly arbitrary which public APIs are exposed in a type declaration and which are exposed in an extension declaration.

Not entirely, no. Stored properties must be in the type declaration. Enum cases must be in the type declaration. Perhaps you regard these as temporary inconveniences of the current grammar; I see them as quite reasonable ways to give some consistency as to what's written where in a language where types can be retroactively extended. In a very real sense, you must read the type declaration before you read the extensions in order to understand the latter. By comparison, there is nothing that must be in your proposed module declaration.

My hope is that the module declaration itself will become the one-stop-shop for re-exports and general public bookkeeping just as aggregate declarations are today. Module extensions exist to accommodate users that wish to break related functionality across files or into separate independent regions within the same file for the same reasons type extensions exist.

Indeed, that you phrase it this way supports Nevin's argument. _Module extensions_ exist to accommodate his use case; however, his use case (which, mind you, is what I think most people are thinking of when it comes to submodules, given previous threads on this topic) isn't the raison d'etre for your submodule proposal. Quite simply, a syntax that accommodates both grouping lines and grouping files cannot make the latter first class, because the former necessarily requires more ceremony.

Okay, but the question still stands: what do we need to make Nevin's use-case first class?

We need to have either *two* spellings for a submodule feature that supports both grouping files and grouping lines of code, or we need to have *two* features, one for grouping files and another for grouping lines of code. In any case, the spelling for grouping files should:
- Not require braces
- Not require one file to be a `module` and another to be an `extension`

The simplest such spelling would be:

File A:

module Foo
// rest of file

File B:

module Foo
// rest of file

This was mentioned earlier, so I will quote my response to the last poster:

That is a valid spelling (Rust, IIRC, allows that spelling), but one that is easy to miss sitting in a file and makes it confusing to introduce submodules. If you include the annotation then define a submodule later down in the file, suddenly you have to remember whether you annotated the file or whether the submodule you’ve just written is going into the top-level module. See:

// -module-name=Foo
// module Foo {
module Bar; // Shorthand for “This file defines Foo.Bar”

/* Code */

// This defines “Foo.Bar.Baz”, but would you know that if it appeared below the fold?
module Baz {}
//}

But that reply was oriented towards a “why can’t we have nested modules and a shorthand too?” Here, you’re referring more to the one-module-one-file language restrictions, so I will quote another response:

For one-file-per-module, that kind of restriction represents a particular way of organizing code and is a design pattern that is supported under this proposal. We just happen to not enforce that particular pattern, and feel that it is the job of a linter to do so. Really, this kind of restriction is to ease the mental burden on compiler writers who use it to build compilation unit dependency graphs. Swift already considers all files when building its modules because you can extend any type (and now, any module) from any one of them so it doesn’t buy us anything other than an arbitrary restriction.

Realistically, the only difference between the proposal’s syntax and this one is two characters (the braces) and maybe some tabs if you decide to enforce that style. There are few objective organizational benefits from your style, and it creates a bureaucratic rule where none need exist. Subjectively, of course, people may prefer this way, or they may prefer a more ad-hoc approach. But we designed for both cases and showed our hand by favoring non-physical organization because it is a subset of possible organizational styles available to users.

Your earlier post argues that your syntax is better because with the alternative approach it’s too easy to forget whether there is a module statement at the top of the file or not. Now you’re arguing that the difference is just two braces and that they’re relatively insignificant. You can’t have it both ways.

···

On Feb 21, 2017, at 9:28 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org> wrote:

On Feb 21, 2017, at 10:03 PM, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> wrote:
On Tue, Feb 21, 2017 at 8:41 PM, Robert Widmann <devteam.codafi@gmail.com <mailto:devteam.codafi@gmail.com>>wrote:

On Feb 21, 2017, at 9:37 PM, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> wrote:
On Tue, Feb 21, 2017 at 8:22 PM, Robert Widmann <devteam.codafi@gmail.com <mailto:devteam.codafi@gmail.com>>wrote:

On Feb 21, 2017, at 9:13 PM, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> wrote:
On Tue, Feb 21, 2017 at 7:59 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Feb 21, 2017, at 7:36 PM, Nevin Brackett-Rozinsky via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

To my mind, we’ve offered a syntax and semantics internal to the language that supports file-only aggregation because file-only aggregation enables a subset of the actual use cases of this module system. We aren’t enforcing this by compiler-fiat because it is a stylistic choice that can be enforced by a linter.

It's not enough to offer a syntax and semantics that supports it; if it is the intended major use case, the design ought to reflect that by making that use case no less cumbersome than necessary.

See above for aesthetic concerns.

Any variables defined with `internal` access will be visible across those files to those extensions and only those extensions (see the section on access control and modules). Any variables declared fileprivate or private will, obviously, not be visible across these files. As an example:

// FooUtilities.swift
//
// -module-name=Foo
// module Foo {
// Defines Foo.Utilities
module Utilities {
  public func exportableOutsideThisSubmodule() {}
  func visibleInThisSubmodule() {}
  private func invisibleToOtherFiles() {}
}
//}

// FooUtilities+MoreUtilities.swift
extension Utilities {
  private func privateHelper() {
    visibleInThisSubmodule()
  }
}

I’m not sure where you got the impression that we were just trying to make another fileprivate happen.

Nevin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

To my mind, any submodule system for Swift should be designed to relieve the pressure for long files, and make it easy to group tightly related files into a single unit with shared visibility. That way developers can easily organize their code into smaller files while utilizing Swift’s pattern of providing protocol conformances in extensions and keeping implementation details hidden from the rest of the module at large.

Wonderful, because that’s absolutely supported by this proposal. To group tightly related files into a single unit, simply declare a submodule for them and extend it in each of your related files.

It's supported, but it isn't first-class. By this I mean: there are two distinguishable uses supported by your proposal, lumped together by the fact that they are both about grouping units of code together. Put crudely, one use case is grouping lines of code, while the other is about grouping files of code. The merits of supporting both have already been debated in this discussion. The issue I'll touch on is supporting both with the same syntax. The chief drawbacks here are:

What exactly would be required to make it first class? Referencing file names in the module declaration?

See below.

- It makes sense to use braces to group lines of code, but it makes no sense to use braces to group files of code; this just causes entire files to be indented.

If braces aren’t used to demarcate scopes, nesting modules becomes ambiguous.

Again, let's observe the distinction about grouping files vs. grouping lines.

Grouping files does not require braces: if the intended use of your feature were to label files X, Y, and Z as belonging to one submodule and A, B, and C to another, it would not matter if X, Y, and Z belonged to Foo.Bar and A, B, and C to Foo.Bar.Baz: your syntax would not require braces.

It’s important to note that indentation is one particular style. LLVM code style, in particular, chooses not to indent after namespace declarations. This issue also crops up when dealing with nested type declarations, and I distinctly remember it not being a big enough deal to "fix this" at the time when a proposal to “flatten” these declaration was brought up.

Mine is not a critique of the syntax itself; I don't particularly care about indents, nor do I mind not indenting namespaces.

What I'm saying is, you would not have chosen to require braces if your proposed feature were aimed at making the grouping of files into submodules as simple as possible. You chose to accommodate grouping lines using the same syntax as grouping files over the simplest design for grouping files. Make no mistake, this promotes one use over another.

Ah, I see. Yes, one of the stated goals is to become filesystem-independent. We certainly cannot do that by encouraging the alternative.

Swift's current design is deliberately not file system-independent. A submodule design built on top of Swift could preserve that. Your draft proposal makes two changes: it introduces a design for submodules; and, it eliminates files as a unit of code by default (not least by declaring `fileprivate` redundant). To my mind, you have presented no justification for the second change other than to say that it is a stated goal--but why?

fileprivate access can be recreated by creating a private "utility submodule" containing declarations of at most internal access.

- Because some lines of code necessarily precede some other lines of code, it makes sense to declare the first group using `module` and to extend that with the second group using `extension`. However, because a file of code does not necessarily precede another file of code, it is arbitrary which file is surrounded with a `module` declaration and which one is surrounded with an `extension` declaration.

Absolutely. But it is similarly arbitrary which public APIs are exposed in a type declaration and which are exposed in an extension declaration.

Not entirely, no. Stored properties must be in the type declaration. Enum cases must be in the type declaration. Perhaps you regard these as temporary inconveniences of the current grammar; I see them as quite reasonable ways to give some consistency as to what's written where in a language where types can be retroactively extended. In a very real sense, you must read the type declaration before you read the extensions in order to understand the latter. By comparison, there is nothing that must be in your proposed module declaration.

My hope is that the module declaration itself will become the one-stop-shop for re-exports and general public bookkeeping just as aggregate declarations are today. Module extensions exist to accommodate users that wish to break related functionality across files or into separate independent regions within the same file for the same reasons type extensions exist.

Indeed, that you phrase it this way supports Nevin's argument. _Module extensions_ exist to accommodate his use case; however, his use case (which, mind you, is what I think most people are thinking of when it comes to submodules, given previous threads on this topic) isn't the raison d'etre for your submodule proposal. Quite simply, a syntax that accommodates both grouping lines and grouping files cannot make the latter first class, because the former necessarily requires more ceremony.

Okay, but the question still stands: what do we need to make Nevin's use-case first class?

We need to have either *two* spellings for a submodule feature that supports both grouping files and grouping lines of code, or we need to have *two* features, one for grouping files and another for grouping lines of code. In any case, the spelling for grouping files should:
- Not require braces
- Not require one file to be a `module` and another to be an `extension`

The simplest such spelling would be:

File A:

module Foo
// rest of file

File B:

module Foo
// rest of file

This was mentioned earlier, so I will quote my response to the last poster:

That is a valid spelling (Rust, IIRC, allows that spelling), but one that is easy to miss sitting in a file and makes it confusing to introduce submodules. If you include the annotation then define a submodule later down in the file, suddenly you have to remember whether you annotated the file or whether the submodule you’ve just written is going into the top-level module. See:

// -module-name=Foo
// module Foo {
module Bar; // Shorthand for “This file defines Foo.Bar”

/* Code */

// This defines “Foo.Bar.Baz”, but would you know that if it appeared below the fold?
module Baz {}
//}

But that reply was oriented towards a “why can’t we have nested modules and a shorthand too?” Here, you’re referring more to the one-module-one-file language restrictions, so I will quote another response:

For one-file-per-module, that kind of restriction represents a particular way of organizing code and is a design pattern that is supported under this proposal. We just happen to not enforce that particular pattern, and feel that it is the job of a linter to do so. Really, this kind of restriction is to ease the mental burden on compiler writers who use it to build compilation unit dependency graphs. Swift already considers all files when building its modules because you can extend any type (and now, any module) from any one of them so it doesn’t buy us anything other than an arbitrary restriction.

Realistically, the only difference between the proposal’s syntax and this one is two characters (the braces) and maybe some tabs if you decide to enforce that style. There are few objective organizational benefits from your style, and it creates a bureaucratic rule where none need exist. Subjectively, of course, people may prefer this way, or they may prefer a more ad-hoc approach. But we designed for both cases and showed our hand by favoring non-physical organization because it is a subset of possible organizational styles available to users.

Your earlier post argues that your syntax is better because with the alternative approach it’s too easy to forget whether there is a module statement at the top of the file or not. Now you’re arguing that the difference is just two braces and that they’re relatively insignificant. You can’t have it both ways.

Now I’m context switching… I will clarify: Syntactically the delta between the proposal and Xiaodi’s syntax is two braces. Organizationally it means the difference between Java-style packages that divide named modules into files and approaches that allow for freedom by ignoring this constraint.

···

On Feb 21, 2017, at 10:36 PM, Matthew Johnson <matthew@anandabits.com> wrote:

On Feb 21, 2017, at 9:28 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Feb 21, 2017, at 10:03 PM, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> wrote:
On Tue, Feb 21, 2017 at 8:41 PM, Robert Widmann <devteam.codafi@gmail.com <mailto:devteam.codafi@gmail.com>>wrote:

On Feb 21, 2017, at 9:37 PM, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> wrote:
On Tue, Feb 21, 2017 at 8:22 PM, Robert Widmann <devteam.codafi@gmail.com <mailto:devteam.codafi@gmail.com>>wrote:

On Feb 21, 2017, at 9:13 PM, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> wrote:
On Tue, Feb 21, 2017 at 7:59 PM, Robert Widmann via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Feb 21, 2017, at 7:36 PM, Nevin Brackett-Rozinsky via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

To my mind, we’ve offered a syntax and semantics internal to the language that supports file-only aggregation because file-only aggregation enables a subset of the actual use cases of this module system. We aren’t enforcing this by compiler-fiat because it is a stylistic choice that can be enforced by a linter.

It's not enough to offer a syntax and semantics that supports it; if it is the intended major use case, the design ought to reflect that by making that use case no less cumbersome than necessary.

See above for aesthetic concerns.

Any variables defined with `internal` access will be visible across those files to those extensions and only those extensions (see the section on access control and modules). Any variables declared fileprivate or private will, obviously, not be visible across these files. As an example:

// FooUtilities.swift
//
// -module-name=Foo
// module Foo {
// Defines Foo.Utilities
module Utilities {
  public func exportableOutsideThisSubmodule() {}
  func visibleInThisSubmodule() {}
  private func invisibleToOtherFiles() {}
}
//}

// FooUtilities+MoreUtilities.swift
extension Utilities {
  private func privateHelper() {
    visibleInThisSubmodule()
  }
}

I’m not sure where you got the impression that we were just trying to make another fileprivate happen.

Nevin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution