It's hard to keep public-facing modules in a package from never having internal or package declarations. While it's useful to have package-private modules, having a package access modifier also gives a public facing module flexibility, e.g. it could have its own helper function in the module that is package and need to include a package symbol from another module in its signature; if a public function were to include it, it would not be allowed.
2 Likes
To clarify, it's a c99 extended identifier.
Okay, but "c99 extended identifier" is not precisely defined. The C99 standard throws up its hands and just says it's implementation-defined what other characters are allowed in identifiers. It certainly doesn't say that Greek characters are allowed as long as they don't have accents. That's just what your function does, it's not a specification.
That mangling seems reasonable to use, at least by SwiftPM.
It is a public method that would have been package if this proposal had existed way back when it was written, because it is an implementation detail. The method is for mangling an arbitrary string into something that will be accepted as an identifier by the entire toolchain. āc99ā might be an overāsimplified name. I think the result preserves c99 explicitly allowed characters, including c99 implementation defined characters (i.e. excluding c99 explicitly disallowed characters), but the actual transformation of the illegal characters is an arbitrary implementation detail.
I only meant that the nameās structure can be an implementation detail (albeit panātoolchain) and not an API promise unless we are deliberately trying to support compiling something halfway with SwiftPM and halfway with Bazel. Whatever tool is arranging the build graph can decide on its own identities and mangling. (Some sort of mangling is obviously wise if there is worry about Unicode changing while passing between precesses, but if the compiler is doing its own, then it is liable to trip the higher level tool.)
If we do want all client tools to interoperate, then we need to nail down the mangling as an API condition. It would narrow SwiftPMās ability to add new features without direct compiler acknowledgment. It would also constrain sister tools like Bazel to match SwiftPMās model more closely.
The proposal does talk briefly about "sub-modules" in the Alternatives Considered section. I'd be interested in your thoughts about what you think sub-modules would be and how they would be a better feature. In general, I feel that vague proposals sometimes seem attractive precisely because they are vague; they can shift to meet any goals we have at the moment without ever needing to compromise. So it is useful to explore what sub-modules would actually be like in order to understand their implications for the language and their suitability for solving this language problem. Are sub-modules required to be modular, i.e. independent units with acyclic dependencies? Do they create a namespace, and if so, is that namespace exposed as part of the interface of the containing module? How does one specify that something is accessible only within the sub-module vs. the containing module?
4 Likes
A simpler, somewhat similar, solution to some aspects of this problem would be what I call "sub-targets", which would be similar to CocoaPod's subspecs. That is, rather than forcing package authors to split into modules or not split at all, allow parts of a target to be offered separately at a source level. For instance, I could break Alamofire multipart form encoding functionality into a separate sub-target that isn't included by default, simply by putting the functionality in a separate source file. Since I don't want to offer that functionality as a separate, public module, nor some internal module (no matter the form) I wouldn't have to make any source changes, as the functionality would never care it may not always be compiled.
Of course, this doesn't solve the problem of being able to label and separately import the parts of my target, but if the author of the package doesn't want that functionality, letting them split targets, and letting users consume parts of the target, would allow at least some of the advantages of granular consumption without having to formalize your intertarget boundaries.
Of course there are also some things SPM would need to do to keep integrity, like requiring only the whole target to create ABI stable artifacts (this needs to be updated anyway, as SPM really should be able to create a platform's preferred stable distribution form), but it seems a workable solution for smaller packages.
@_spiwould also not be easy to optimize. By design, clients of an SPI can be anywhere, making it effectively part of the public ABI of a module. To avoid exporting an SPI, the build system would have to know about that specific SPI group and promise the compiler that it was only used in the current built image. Recognizing that all of the modules in a package are being linked into the same image and can be optimized together is comparatively easy for a build system and so is a much more feasible future direction.
It seems like the issues with SPI could avoided by formalizing it and allowing a visibility modifier as part of the SPI declaration. Couldn't we simply mark an SPI as package visible to gain these same advantages? That way we gain both a formalized SPI feature (which seems necessary) and don't have to change the access modifiers at all. I could even see this update to SPI enable things like compiler-enforced testing only APIs, or any other visibility we want, without having to modify other parts of the language at all.
5 Likes
In fact, it seems like we could combine access levels for SPI and allow things like package and testing only APIs at the same time, a feature just not possible with access levels without major changes.
The thing about SPI is that that underscore in front of @_spi makes it feel like one of those internal features we really shouldn't be using. I end up using it anyway, because there's no alternative at the moment, but I'd love to be able to switch to something that was officially supported.
As for my use case: I've been splitting a lot of the libraries I'm making in half, so that I have two products in the package: one that has no dependencies on Foundation, and another that adds Foundation-specific features and entry points, usually as an extension on the principal type defined in the first package. This way, my library can be used by clients who want to avoid their product depending on Foundation, but for those who do want to use Foundation, we can still support working with Foundation types like URL and Data. Often it ends up that there are certain implementation details that the Foundation half needs to access, but which really shouldn't be part of the public API. @_spi does the job, but package would seem much more tailor-made for the purpose.
6 Likes
Packages already aren't "us" though. If I'm working on a large project I shouldn't be forced into making "quarantine" sub-package just to prevent leakage to peer teams, if that's even possible in context.
More philosophically, I don't believe (or haven't been convinced) there is a level between internal and public where it's OK to not very intentionally shape your API surface. .run() "rising with the tide" and having ever increasing scope of visibility just because the project gets larger doesn't make sense to me.
I have done work across local modules as you've described above by using spi and naming it SPI (e.x. @spi(SPI)), so I can see the argument for "anonymous" or "package" spi (bikeshed spelling @spi func ... or @spi(#package) func...), although I suspect that would clash with your point below.
I appreciate the extra perspective, but this strikes me as basically the same argument that's made in the proposal; which I still disagree with. Importing spi being noteworthy to the point of suspicion is an Apple-flavored idea, and even then isn't true across the various types of SPI and projects at Apple. Some SPI is as you describe, where every use case should be carefully analyzed, and some is effectively public API that never quite made it through API review, or is still baking. There are popular well understood tools for guarding special things (linters or, if you're Apple, entitlements)ācasting spi as solely the realm of sharp objects is an artificial limitation.
In general I haven't seen (and don't foresee) the import explosion you describe. Taking the proposal's example .run() Would only be imported in one file. .run() should only be imported in one file. Importing lots of SPI in one file and the same SPI being imported in many files are both smells that indicate boundaries are maybe drawn incorrectly, which would not be surfaced with the use of package. The exception being SPI that's more "weak API" than something truly specializedābut we have historic examples for this that suggest it's not something easily missed. #import <objc/runtime.h> is a massive beacon that always calls scrutiny in code review; similarly #import "Foo+Private.h" draws attention more than #import "Foo.h". In my experience imports don't turn into line noise the same way, e.g., warnings do.
I think I've gone in a bit of a circle so I want to try and lay out what I'm saying more clearly. The proposal suggests adding another layer of access control, expressed in terms of containment:
Today:
Public( Internal( FilePrivate( Private( ) ) ) )
Proposal:
Public( Package( Internal( FilePrivate( Private( ) ) ) ) )
I don't like this structure because it removes agency both from the client and from the author in a way other levels don't; instead giving control to whoever decides the project structure. Package organization is as much a factor of institutional politics and time as any individual's specific intent. The meaning of package changes over timeāmodules are added and jettisoned, others are split out into peer packages, etc. This is different from other access levels because I am the owner of my module/file/declaration site. I may be the owner of the package I contribute to.
Ignoring the specifics of spi and package, as an author I have a hard time reasoning about when I would ever choose to write code that is truly package-level. Taking my claim above as true, I have to assume peer modules will change. Either I have a specific client in mind (maybe a single individual, maybe an archetype like "Game Author"), in which case I should have some mechanism to name them so I don't accumulate accidental users who I unexpectedly need to support, or I have to account for every possible client, in which case I am writing API.
The point about spi specifically isn't just that it's more flexible, it is also that it's more expressive. Package-level access control is a step up in complexity that warrants a more capable tool.
8 Likes
I think your argument boils down to āModula-style interfaces are more expressive than Java-style packagesā. Maybe the answer isnāt one or the other, but both? Itās feasible to imagine a silo that cuts through public, package, and internal, grouping related APIs with different levels of exposure. This argues for being able to parameterize any visibility declaration. But since packages are part of the hierarchy defined by SwiftPM (in which āinternalā maps to a target), it makes sense to let people use that scope as a visibility boundary.
I could see a convention arising where public(_Underscored) symbols were stripped from the .swiftinterface automatically, like @_spi() declarations are today.
1 Like
It can be because I'm not a native English speaker, but I don't think package naming makes sense. When I will find package struct Foo, I will wonder "Oh is it a Swift Package?"
open, public, internal, private, or other access modifiers are adjectives, but 'package' is a noun.
I don't have alternative naming suggestion, but package seems really strange.
14 Likes
Il not sure if the following was considered or not, could we stop modules from "leaking" their dependencies and then just reexport manually what you want to export ?
This would mean that we could keep access control as it is. Imported public symbols would become internals symbols in the parent module. We would get the package scope for free by always re-exporting very sub module unless in the top module. This would allow authors to have any number of scopes similaire to package without being limited by the package structure. Thought this would be breaking.
Another similar direction could be in the form of "internal import Submodule". In my mind different team would want different scope and this proposal only add a single scope in addition to existing ones.
Is anyone else concerned about the generality of the word package and the fact that it doesnāt in any way indicate that it's an access modifier like all of the other access modifiers do? Iāve suggested in the past and will raise again my preference for packageprivate.
Possibly relatedly, why all the hate for fileprivate? I use it all the time, primarily to add some specific functionality in an extension on type A that I only need to use in the private implementation of type B inside of file B.swift.
I'm vaguely aware that private might behave the exact same way in this case? Would that fact form part of the argument against fileprivate? I wouldn't find that very sensible. I deliberately use the slightly more verbose fileprivate despite being aware that technically I may be able to make private do what I want because in the case I described it seems like fileprivate is exactly what I'm interested in and if private works too then that seems to me more like a bug than anything else and I don't want to depend on it.
7 Likes
I believe thatās a compromiseābetween clearer name and shorter syntax.
If private(file) is unacceptable because we already have private(set), private<file> should be my personal choice:
-
fileprivate->private<file> -
internal->private<module> -
package->private<package>
We can even extend this syntax for SPI, instead of @_spi(MyProj) public we can have private<_spi(MyProj)> or a simplest private<MyProj>. Fun fact: SPIs are stored in .private.swiftinterface though theyāre marked as public.
4 Likes
Historically, private was file-scoped, but was changed in SE-0025 to be declaration-scoped. This led to the introduction of fileprivate as a file-scoped access level.
The core team later considered that a mistake, although it was also impractical to undo ("is in use within the Swift community and in established patterns, such that it would be harmful to remove the functionality"). There was an idea to rename the keywords, but it was considered too much churn. Thus SE-0159, which would have reverted SE-0025, was rejected.
Sometimes people get confused and think that fileprivate itself is somehow discouraged. AFAICT there was never any significant dissatisfaction with the concept of fileprivate - it's with the spelling; some would like it to be the "default private".
I suppose these people would therefore be in direct disagreement with this position of mine?
I doubt re-litigating private is within the scope of this review.
It seems to me that peopleās issue with fileprivate is playing a fairly central role in the debate, which is why I thought that it was worthwhile to ask about, so perhaps limiting the discussion to the similarities between the two cases would be good
+0. I think this is okay, but it would be better if it were specified in terms of SPI - as an SPI scope defined by the package manager.
I think SPI is a sorely needed feature, and I'm surprised that so many developers here seem to know about it and use it already. I wonder if it has reached the threshold for being a de-facto language feature. We really need to get a grip on that.
I wonder how attributes such as @usableFromPackageInline would look if they were generalised to SPI scopes. Perhaps it would actually be something like @usableFromInline(package).
14 Likes