What happens if you don't pass -package-name? Does the compiler reject the package keyword, or does it use a default package name? (You don't need an answer right now, but the actual proposal should have one.)
With this design, there doesn't seem to be any "security"—that is, someone else could create a module of their own and build it in the same "package" as yours, thereby gaining permission to use all of your package APIs. Are we okay with that, or do we want the package author to have a little more control here?
Relatedly, are package APIs printed into both module interfaces, or only the private one, or neither?
You seem to be contemplating a totally from-scratch implementation of this feature, but I could imagine using this as sugar for @_spi:
The package keyword is equivalent to @_spi(<package-name>) public or @_spi(<packageName>) open (depending on the declaration and its enclosing type).
Every import implicitly has an @_spi(<package-name>) on it, so it automatically imports package API from modules in the same package.
The current public interface/private interface distinction could then provide the kind of security I was asking about above.
Have you considered a design like this? Are there reasons to not use it?
Do you imagine that another module in the same package would be able to subclass a public declaration? If not, that puts us in kind of a strange spot where the compiler can make stronger assumptions about a public class having subclasses than a package class. It also makes public package(set) var a bit paradoxical, since the setter is in theory overridable but the getter is not.
Makes me wonder if there should be a packageopen level too...
For that matter, do we need a @usableFromPackageInline or something, so that a declaration can be used from @inlinable package but not from @inlinable public?
It was actually originally considered as a sugar for @_spi but having package as its own access modifier makes it clearer that the scope only applies to modules within a package (also a package name could become part of mangling in the future). It also has a different distribution logic from SPI in that package symbols will not be printed in .swiftinterface or .private.swiftinterface; they'll be stored only in .swiftmodule. If .swiftinterface or .private.swiftinterface is required to build a binary and .swiftmodule doesn't exist, package symbols will not be visible. It would be possible to have something like .package.swiftinterface, but that is TBD.
public and open would continue to behave as is today; only package will allow accessing and subclassing cross-modules as long as they are in the same package. If the symbol needs to be visible to a client outside of the package but still needs to be subclassed within the same package, then it will need to be made open. This behavior is parallel to internal; if internal symbol needs to be visible from another module but also subclassed from some other module, it needs to be made open. @Douglas_Gregor any other thoughts on this?
That's a good idea; could also be with an argument like @usableFromInline(package:package_name)
I see what Becca's saying here, and we have a couple of choices on what to do about subclassing. As it stands today, a public class has no subclasses outside of its defining module, and a public class member has no overrides outside of its defining module. The optimizer can take advantage of this information in whole-module builds to (for example) replace dynamic dispatch with static dispatch.
When we add package visibility into the mix, we have an initial choice: we could allow package-visible classes to have subclasses within the package, and package-visible class members to have overrides within the package. If we do that, it has a couple of implications:
We don't get those "all subclasses of a public class are in the same module"-style optimizations for package classes, because we can't reasonably have whole-package visibility into subclasses. (Same thing for public class members and overrides)
We need to say that public declarations are also overridable within the package, because otherwise making a package class public would break subclasses defined in other modules within the same package. (That's Becca's package(set) example).
Combining those two, we don't get those "all subclasses of a public class are in the same module"-style optimizations even for public classes. That would mean that the presence of package in the language could mean we get less optimal code, which is a concern.
So we might not want to extend subclassing of a package class to other modules in the package. What are the ramifications of that?
The optimization model for public remains unchanged (that's good).
package has the same optimization model as public for subclassing/overrides. (that's probably good?)
We will almost surely be asked to introduce packageopen (as Becca suggests) to indicate "subclass able or overridable from other modules in my package".
After writing this out, I think this latter approach is the better one, because it means the introduction of package into the language has no effect on existing code. And I like the better optimization module for package.
Moreover, I'd want to acknowledge that there is a gap here because we can't express "open but only in my package", but I don't think we should fill that gap, ever. Right now, we have a linear progression from private through open where each step is offers strictly more capabilities to a wider range of clients. A package that does not allow subclassing/overrides outside of its defining module maintains that linear progression. Supporting "public, but open only in my package" breaks that linear progression, which would leave us in a weird spot: either we have to go redesign open to be orthogonal to visibility (imagine public open(package), or public open(internal)), or introduce one-off affordances like packageopen. I don't think it's worth the effort or the code churn to do the redesign, and I don't think it's worth the effort or complexity increase from having something that's a one-off point in the space like packageopen.
I'd like to hear your reasoning for this. Why would it be advantageous to do so?
This sounds like it would be a breaking change. Currently, I can just open my terminal and say
echo 'print("Hello, World!")' > example.swift
swift example.swift
and see the second command output "Hello, World!". Would this break?
If this were ever seriously considered I'd like to see more about the interaction of the new open with final. I like the idea of making everything final (or open(never)?) by default, like in Kotlin, but it would be massively source-breaking to do so.
Okay, this is the clarification I was looking for: you can omit -package-name, but if you do, you'll get a diagnostic when you use the package keyword. Please do specify that in the proposal, as well as whether it should be an error or a warning, and how it behaves if you use a warning (does it degrade to internal? public? Use the module name as a package name?).
Hmm, interesting. So package is intentionally, explicitly designed to only be used among a closed set of modules that are built together, whereas @_spi gives access to an open set of modules that may be built separately.
How does package interact with library evolution? It occurs to me that if same-package clients need access to swiftmodules, that means they don't need to be independently rebuildable and could have an unstable ABI. That means they could avoid resilience overhead, as well as language rules like the @unknown default requirement.
Also, I'm thinking it may be a good idea to have swiftinterfaces remember the package they were in, but have the compiler refuse to use a swiftinterface for a same-package module. Otherwise, if the swiftmodule is missing or unreadable, Swift would fall back to the interface and the import would appear to work, but all of the package APIs would be missing. This will make using a scheme like reverse-DNS package names even more crucial, because if your -package-name is Game, then a project whose -package-name is also Game would not be able to use your module, even if there was no other conflict between them.
You hit the nail on the head :) Even with the library evolution enabled, modules within a package could be treated non-resilient and overhead like indirection could be avoided.
That's a good idea. We could throw an error if the package name of the importing module and the package name in the swiftinterface it tries to import match.
Additionally, we could add a check for duplicate package names and add a prefix if needed, e.g. Xcode.Game.
What about the new keyword though? Will it clash with other code called package and we should start use back ticks or is it avoidable for access modifiers?
It's informative here to write out the possibility space:
Accessible from...
___anywhere___|____package____|___module___
anywhere | open | (illegal) | (illegal)
Subclassable package | (A) | (B) | (illegal)
from... module | public | (C) | internal
nowhere | public final | package final | internal final
Some of these boxes are naturally illegal: it of course doesn't make sense for a class to be subclassable when it can't be named at all. Others have existing or obvious answers.
package on a non-final class could mean either box (B) or box (C). I'm not sure from Becca's post whether she was thinking of packageopen as box (A) or box (B); probably (B). Your recommendation, as I understand it, is that package means box (C), that box (A) should be intentionally left empty, and that box (B) is not worthwhile to pursue. (You also argue that public should be where I've put it and not in (A); I hope that's uncontroversial enough that nobody will dispute me just filling it in that way.)
I think you are understating the importance of box (B). Box (B) is what a traditional OO programmer would naturally want if they took an internal class and extracted it into its own module. For example, it is what you would want for a custom UIView subclass that you expect to be further customizable by its clients. Swift does push value types and protocols pretty hard, and I generally agree with that, but I think not having a spelling for (B) is a step too far in that direction.
Maybe it's better to solve the initial problem by introducing a mechanism to Package.swift file? So that you can specify when internal symbols can be used from another module in the same package.
There's a significant code size impact to making all internal symbols available cross module just in case they might be used. For packages where all the modules will be statically linked together this can be mitigated with dead stripping, but the design of this feature should also to take into account packages made up of separately compiled modules distributed as dynamic libraries.
It’s also still valuable to maintain access control between modules in a package. In fact, a lot of the access-control proposals we’ve received over the years have been requests for intra-module access control; making internal just apply at the package level would just make that a bigger problem.
I really like the idea of this pitch. But I dislike the name package, it doesn't fit into the current list of keywords, which to me are pretty clearly stating some level of "visibility": public, internal, private (open has always been a special case in my mind)
I could imagine to name it one of these, which are all more consistent:
packageinternal (clearest, but a bit long)
packagewide (clear enough, a bit shorter)
widened (compared to the default internal, the scope is "widened")
Alternatively, we could keep package and rename internal to module, then things would also be consistent. But that would be source-breaking, so I don't think it's the best idea.
We could also simply create a new word, combined of things that make sense. Such as packternal for package internal. Not sure if this is wise though.
As hindsight showed us for the much-maligned fileprivate, the perfect stratification of access control can be the enemy of pragmatic development.
In fact, a package keyword to internal has almost the same analogy as fileprivate to private, and I think we'll see the same results here and eventual regret of another keyword.
Let's learn from the mistake we already made with fileprivate and lean for pragmatism and simplicity here; the value add of a new access control modifier is not worth the minor benefits over internal accessible for all package modules.