Here’s a proposal for more control over dependencies and imports in Swift. With this feature you can mark imports as public (the current regular import), internal for implementation details of the module, and private or fileprivate for implementation details to the source file. Additionally, the newer package access-level allows to mark a dependency as being visible only to modules of the same package. This is then enforced like the typical access-levels on declarations in the source file. Declarations imported as internal can only be referenced from internal declarations or lower, uses in public or package declarations are reported as an error.
Here’s a typical use case where a dependency is an implementation detail we don’t want to expose to clients in the module API, and the expected diagnostics:
internal import DatabaseAdapter
internal func internalFunc() -> DatabaseAdapter.Entry { ... } // Ok
public func publicFunc(entry: DatabaseAdapter.Entry) { ... }
// error: function cannot be declared public because its parameter uses an internal type
public func useInBody() {
DatabaseAdapter.foo() // Ok
}
@inlinable
public func useInInlinableBody() {
DatabaseAdapter.foo()
// error: global function 'foo()' is internal and cannot be referenced from an '@inlinable' function
}
The proposal also defines a set of conditions where a dependency can be hidden from clients. This offers a strong way to fully hide implementation details and can speed up build time in clients.
This proposal aims to offer an official and cleaner replacement for @_implementationOnly. In contrast, this version provides familiar diagnostics, more levels of controls, as well as better compatibility with non-resilient modules and @testable clients.
Depending on the community's response to the suggested Swift 6 behavior we could make it part of this proposal.
This is long overdue, thanks for proposing this Alexis! I think changing the default import visibility to internal in Swift 6 language mode makes a lot of sense; as you note in the proposal, that brings imports in line with the default behavior for other declarations and reduces accidental API leakage, but the migration path for adopting the change in Swift 6 mode also seems straightforward enough, since a mechanical s/import/public import/ seems like it would always suffice to get the Swift 5 behavior back.
You mention dependency visibility but how does this help when all Swift dependencies are public anyway (i.e. they must all participate in the same dependency tree)? Perhaps it's just a matter of language, but it would be good to clarify where you mean "import visibility" and where you mean "module visibility". As far as I can tell this won't help with overall module visibility.
Also, what does public import mean, aside from letting you use imported symbols in public declarations?
I also agree that an internal default for Swift 6 makes sense.
IIUC, this proposal reads as if it's primarily about affecting the compile-time semantic visibility of the imported symbols. What you call module visibility is a bit intertwined with the rest of the build process, particularly how the various modules are built and linked together. Using library evolution and @_implementationOnly import today, to build a dynamic library, you ought to get a good degree of module isolation for your dependencies in addition to semantic isolation, since only the public module's API would be exported through the dynamic library. A similar level of isolation could be achieved for static library or package builds, but would require either changing how we build them to use something like incremental linking, and/or doing some additional compile-time mangling of symbols and runtime data structures for internal imports. I think it would be good to be able to do that, but having this semantic-level isolation seems like a necessary first step no matter what.
Big +1. Has been a gap in the current access level story and the underscored attributes have been adopted widely throughout the ecosystem. I would wish that we make @_exported a stable feature in the same proposal though. That would solve all the current usage of underscored import attributes besides SPI in the server ecosystem.
I'm also very excited about this. Bazel builds encourage users to write very many, small modules, and so the amount of work the compiler has to do to load them is significant the higher up we get in the build graph.
However, the currently described inability to hide dependencies of non-resilient modules would still be a showstopper for our use cases. We don't use library evolution at all since we're building from tip-of-tree and statically linking everything into the executable, but we still desperately want the dependency tree pruning that this feature provides. The described restriction also seems stronger than @_implementationOnly implements today. With @_implementationOnly, I can at least reference the imported module in places that don't affect ABI, such as inside function/property bodies; for example, to hide an imported type inside an Any and cast it in/out to get around the storage layout limitations.
So yes, absolutely we should move forward with this. But do you have more insight on how difficult it would be to follow-up this work with storage layout abstractions to remove this limitation?
The proposal uses the term dependency for both a source file dependency and a module dependency. I'll see if I can clarify that wording.
I should note that when the required criteria are met, we can hide transitive module dependencies which does allow to hide a part of the dependency tree when compiling one of the modules in the tree. However, when building all the modules in the tree from source, then all dependencies are needed. It's only if some of the dependency tree is pre-built or rebuilt from swiftinterfaces that their non-public dependencies can be hidden when compiling indirect clients.
That's the meaning at the file level. At the module level, using a public import forces the dependency to remain visible to all clients. I may be missing other implications but this is the behavior we're used to with the current imports in Swift 5, so adding public on an import won't change anything from the behavior before this feature is introduced.
The knowledge of the storage layout is a part of the work known to me. There are other levels of the compiler we would need to look into and see what's needed. This would be a valuable change however it is a feature in itself.
This is something I'll have to look into. In part because it would be useful for any migration tool. However, we have to take into account the behavior we pick for Swift 6. This warning may be more useful with the Swift 5 behavior where it's easy to mistakenly make an import public when it doesn't need to be, but then we may not want to ask to add internal keywords on all imports and then remove them with Swift 6.
This has been suggested before, we preferred explicitly declaring the intended visibility on each import which allows us to enforce that intention in the source file. Inferring that intent from the use in the file could end up changing an internal dependency to a public one from a mistakenly added reference in a public declaration, all this without the compiler being able to diagnose the inconsistency as the intent is not declared separately.
I suppose the question is what is the user's intention or source of truth for what should be public? Probably a user considers the API of the module the source of truth for what should be public/internal/private, rather than the import statements.
This is something we've needed for a while, and accesslevel import Foo is the natural way to spell it. I especially like the future direction of making imports default to internal.
@xymus Can we have a frontend emit mode that produces some kind of parsable output with summary: list of imported modules, list of used entities, and list of uses for each entity for each level of access? I.e.
{
imports: {
"Swift": {
"Dictonary": {
public: [
"MyModule.myFunction(_:Int)" // FQN of the site of use `Swift.Dictionay`
]
}
}
}
}
Such report would be a great tool to analyze the level of connectivity in the dependency graph. It could be used to plan work for improving incremental build times. And it's not an easy task without the compiler, because it requires the result of the type checker. swift-syntax alone is not enough.
It might be quite a bit of undesirable extra work for the compiler to group references by the thing being referred to output that kind of manifest, but much of the data you're looking for can already be found today in the indexstore if you build with -index-store-path <some directory>.
Newbie here... I'm not sure if this concern of mine has to do with the same issue. If it doesn't and there's something else someone can point me to, that would be appreciated; I haven't been able to find a solution I like. I'm interested in having the group that develops DatabaseAdapter in your example being able to easily release the binaries only so that the group developing the consuming module or app have no visibility into the internals of DatabaseAdapter. The only way I understand this to be able to be done now is by building as a framework and copying the binaries to some other repo with different visibility. As a compiled language, I feel like I should be able to easily build and release DatabaseAdapter to my clients without them being able to see the internals. Thanks.
As much as I like the cleanliness of the syntax, it seems like it introduces some ambiguity.
currently when you import a module it's effectively fileprivate, in that only code in that file can refer to symbols in the imported module unless they are implicitly reexported by declaring non-fileprivate APIs that use them.
If I saw internal import Foo I'd expect that to have the effect of making the entire imported module visible to any file in the importing module, but it doesn't seem like that's what's being proposed based on the examples above?
Similarly, I'd expect public import to have an effect equivalent to what @_exported import does now.
If I've misunderstood, and that is what's being proposed, then it seems like the proposal is merging two different module access level concepts into one, which isn't necessarily desirable.
Perhaps there could be a syntax similar to how read-only properties work, where you can set the two access levels independently? e.g
// import the members of Foo with public access
// but limit the access of Foo itself to this file
public private(export) import Foo
I also interpreted the current import Foo as fileprivate import Foo (or private - that's currently the same at the top level) , as no other file can use this import. Each file has to do its own imports.
The @_exported attribute is a step above a public import as clients see the imported module declarations is if they were part of the local module. With this proposal, @_exported is accepted only on public import statements, both with the keyword or the default public visibility.
I don't think we would ever want changing an import statement in one file to impact how other files are type-checked, so that's enough to cross that possible interpretation of <access-level> import off the list in my mind. While it's possible it might be confusing to some users at first, my own opinion is that I think we'd be hard-pressed to find a more concise and clean spelling of the concept being pitched, nor do I think the feature would be improved by trying to come up with something more verbose here.
open isn't just an access level above public; it has a specific meaning around subclassing of types, so I don't think it would be appropriate to use it to support what @_exported currently does. Module A writing @_exported import B really means "when A is imported, B can also be used as if it was imported at the same location", and that's a very specific concept that doesn't map to the access level-based approach being proposed here. If <access-level> import X means "the names in module X can be used by <access-level> or lower declarations in the importing file", then it would be very confusing if open import did something completely different than that.
My way of seeing this is that we need a reasonable duplication of the information for the compiler to enforce the developers intent, in this proposal it's by having the visibility of the API match with the visibility declared on the import statements,. Without the import information the compiler could infer the intent for a dependency from the API and generate the same code, but it won't be able to offer support to the developer at type-checking. In my experience with projects hiding dependencies, the support was critical as a leak of a dependency was undesirable.
The recommended way to distribute a binary framework is the xcframework format. The process sounds generally like what you describe. These two WWDC videos should be a good starting point: