New Access Modifier: package

My preference for an access level keyword between public and internal is external.

1 Like

@8675309 Your message makes me think about an alternative: What if instead of introducing a new keyword, we simply change the behavior of the existing internal keyword if a new setting for targets called something like extendInternalToPackage is set to true in the Package.swift manifest file of a package?

That would have several benefits:

  1. No need to learn a new keyword.
  2. No need to explicitly state the visibility (as internal is already the default).
  3. No source-breaking change by keeping the default behavior as before.

@Dante-Broggi I agree with @hassila here about external – it sounds too much like it's an alternative to public. It doesn't hold the notion that it's actually "internal to the package".

Just to make sure I understand what you are suggesting correctly, do you mean that instead of adding package we should change the semantic meaning of internal to mean that it's visible for all targets in the package (as package was intended to be)?

Correct.

I think this was pitched in other replies in this thread as well, but there was some input that some use cases may actually want types and methods that are internal to the specific submodule only but not the package so we would want a separate keyword just for that.

As I stated previously, we tried something similar with the fileprivate / private division since there are also use cases for keeping a member accessible to the type but not file, and it didn't really work out well in practice. Same thing seems like it might happen if we introduce package.

At the very least I feel we should start off by trying to keep a single keyword internal and expose to package modules, and we can always make a change later if it proves unwieldy in practice. But if we start by introducing a package keyword then the cat's out of the bag and we're more or less stuck with it like fileprivate.

2 Likes

Alright, I would be happy with that too - it is way more common for us to want to have optimizer visibility cross-module in our packages than wanting to keep accessibility locked down, but perhaps there are issues with that which I don't see. I guess cross-module-optimization would be another approach for that, but we've tested it a few times now and always gotten performance regresssions with it enabled...

1 Like

Speaking just for myself, I think it makes sense to have access levels for both modules (smallest unit of library organization) and packages (smallest unit of code distribution). Both boundaries are important to be able to enforce: modules because we want to encourage small, encapsulated libraries with well-designed interfaces, and packages because it's almost certainly a boundary between different development organizations.

I don't think analogies to intra-module access levels are really on point. Intra-module access levels are designed to promote encapsulation of small subsystems that can't quite be extracted into separate modules. At most, they might reflect organizational differences within a large development group. I do actually think there's space for more levels within modules — I keep toying with the idea of a directoryprivate that would help teams to promote/enforce those subsystem boundaries without requiring code to be broken into separate modules (not always possible) or combined into single large files (distasteful and likely a build-time hit) — but on some level, this is all arbitrary because the code will ultimately be built in a unit. In contrast, modules and packages have clear provider/client relationships with a direct need for access enforcement in order to maintain a well-defined API and avoid unwanted dependencies.

11 Likes

Food for thought: could there be a way/would it be a positive change to have some sort of design for user-definable access modifiers? Perhaps it’s an off-topic idea, since I can’t currently imagine how this would solve the need at hand (i.e., we would still need the package modifier or whatever we end up calling it), but in case it inspires any interesting or important ideas in others I'll at least describe it a little bit.

An example is that it could be useful to make an extension on one type that is private to another type, like this:

private(to: Dashboard) extension Double {
    static var minimumSectionHeight: Self { 40 }
    static var standardCornerRadius: Self { 12 }
}

which can then be used only in the scope of Dashboard:

struct Dashboard: View {
    var body: some View {
        VStack {
            Text("Welcome")
                .padding()
                .background(
                    Color.blue
                        .cornerRadius(.standardCornerRadius)
                )

            Dashboard.FirstSection()
                .frame(minHeight: .minimumSectionHeight)
            
            Dashboard.SecondSection()
                .frame(minHeight: .minimumSectionHeight)
        }
    }
}
1 Like

This reminds me a lot of C++’s friend.

1 Like

Just a quick thought; What about local as the keyword?

Frankly, this feature would be so useful that I'd happily use it even if it were named florglequat.

5 Likes

packages, modules, files, types... these are all conceptually smaller and smaller "subsystems", each with its own local encapsulation.

That would be interesting, I think the excitement around SwiftPM and how easy it seems to be kind of promoted distributed monoliths where you are splitting an app in multiple packages hosted in their separate repos but effectively are not separate at all.

This is an area where to be frank all a lot of devs need is fulfilled by the package visibility you have with Java projects where the only thing you need to do is setup the folder structure to create a “module” where you can be flexible about visibility inside the module and strict between modules without much bureaucracy really.

I'm all for a package access modifier! Maintaining packages will be better for iit.

5 Likes

i’ve been thinking about this for the past week or so and i’ve gradually come around to the conclusion that a new access modifier like package/external is not the right solution to this problem.

because i agree with @Joannis_Orlandos that i too often find myself reaching for something like package and i end up just making it public even though it is not really “for public consumption”, it is only public so that i don’t end up cramming everything into one module. and that just leads to a lot of underscored APIs and doccomments that scream “don’t call this manually!” which i dislike.

but the more i think about why i reach for package in the first place the more i realize it is something of an XY problem because “access control” really encompasses two things that get conflated with one another:

  1. enabling local reasoning (“i can change this thing while being aware of all the other things that could potentially be affected by this thing”)
  2. encouraging correct usage (“i can’t plug the USB(-A) stick backwards because it won’t fit”)

because what package does is provide #1 but a lot of times what i really wanted to accomplish is #2. and i think the local reasoning that package would enable is not actually very useful in practice, because the more code that local refers to the less helpful locality is in the first place.

for example internal can facilitate some amount of local reasoning, but it is nowhere near as powerful as fileprivate. and fileprivate itself is not as powerful as private. and that’s why i think a level above internal, like package/external, would not be helpful because when “all the other things that could potentially be affected by this thing” means “everything in this package”, and the package itself can be quite large, that really doesn’t mean much.

and when i imagine what really would help accomplish #2, i think that solution looks more like @_spi, which would align a bit better with some of the other ideas pitched in this thread, such as:

2 Likes

While in the end I'm not in total agreement with the concept as stated, I appreciate the formulation of it and it led me to some new ideas.

I would change "encouraging correct usage" to "enforcing correct usage", and I would further add that I think the benefit of enforcing correct usage always boils down to local reasoning (e.g., an initializer that must be called just-so is made private, which enforces correct usage of the type by making the user use the robust public initializers, and the purpose of this design is that the author of the type can locally reason that the type's properties will never be configured improperly.)

Therefore, I'm arguing (at least until I'm convinced otherwise) that maybe there's only one reason for access control modifiers, which could be described as "local reasoning".

Since that's what I'm arguing, the next step would be to refute this:

to which I would point out that "a Swift package that I am currently authoring" is a waaay more "local" scope than public (and also of course it's much less "local" than "the code in this single file"), and consequently marking something packageprivate gives me tremendous local reasoning capabilities as compared to public. For example, if I have a packageprivate protocol I have the ability to physically view all of the conformances to that protocol that exist in the entire world, just by browsing the package's source files that I have locally on my computer, which is a noticeably easier task than the impossible task of viewing all conformances if the protocol is public and on GitHub (and, of course, is also a correspondingly harder task than viewing the conformances if the protocol is fileprivate.)

New Idea - General Solution

I'll finish with an idea that has just formed in my mind, which I think would qualify as a more general solution to the problem:

Continuing off the idea I offered above, imagine that we introduce a new access modifier:

private(to: RelativeScope)

where conceptually we can imagine the RelativeScope type like this:

enum RelativeScope {
    case package
    case module
    case folder
    case file
    case type
}

This current proposal would be served by:

private(to: .package) protocol AllConformancesCanBeViewed { }

These current modifiers would desugar in these ways:

internal is private(to: .module)
fileprivate is private(to: .file)
private is private(to: .type)

It would be pretty awesome to get folder to come along for the ride.

Later on we can add new overloads for this access modifier if we determine new scopes that would be useful. For example, earlier in this thread in a pseudo-code example I made use of this overload:

private(to: Any.Type)

As a final new idea, I'll mention the possibility of adding the case named extension to the imaginary RelativeScope enum, which would be useful in this way:

/* we're inside of the somewhat long DashboardView.swift */

/* ... code, code, code ... */
/* ... not visible because it's scrolled off the screen ... */

private extension DashboardView {
    var balanceReadout: some View {
        HStack {
            balanceNumbers
            currencyToggle
        }
    }

    /// I can see at a glance that this subview
    /// is not used anywhere other than in `balanceReadout`
    private(to: .extension) var balanceNumbers: some View {
        Text(viewModel.balance(in: selectedCurrency).description)
            .bold()
    }

    /// The same luxury of local reasoning applies to this one too
    private(to: .extension) var currencyToggle: some View {
        Toggle("Currency", isOn: $selectedCurrency.dollarsOrEuros)
    }
}
6 Likes

"The only clients of this are in this package" vs "the clients of this are external" seems like a very meaningful difference to me in terms of reasoning about the effects any changes might have. If a decl has package level visibility, it means if I change the name, the type, the signature, etc. that I can be confident that all consequences of that change can be addressed by updating code that I have access to and control. For a public declaration, on the other hand, those details may be a contract I need to maintain indefinitely if I vend a package that provides source stability. When I was an SDK framework developer, being able to make this distinction would have been incredibly useful.

9 Likes

you are referring to “public as-in for public consumption” but i was referring to “public as-in uses the public keyword”, which seems to be the main motivation for package.

for example,

import protocol FrameworkCore._Barable

extension Foo:FrameworkCore._Barable
{
    public
    func barableRequirement()
    {
    }
}

is technically going to be public, but it is obviously not public API, and you would probably not be providing any source-stability guarantees for it.

I have mostly developed packages in what some might call a "live at head" environment. In such an environment, relying on the assertion that some of the public surface of my package is source unstable isn't really an option because it's unenforceable. If my package exposes a public decl and there's a chance that some other package in the ecosystem might be using it then I simply cannot change that decl without tracking down the uses and coordinating a staged change with the affected package owners. If I break the build for those other packages, it's my change that's going to get rolled back. And to make matters worse, sometimes just even identifying whether any other package uses an API can be challenging. The package access level solves this by allowing me to build inter-module APIs without needing to worry about the implications of exposing internals.

Even when maintaining source stability for other packages is less of a concern, though, I think that this language feature is well justified. Breaking encapsulation by exposing the internals at all, uglifying the names with underbars, and needing boilerplate documentation to remind users that "this interface isn't actually for you" are significant compromises. This feature is pretty analogous to what Xcode refers to as project headers for C like languages and it was surprising to me as a long-time Objective-C framework developer that there was no analog to this useful encapsulation tool when building a module in Swift.

8 Likes

+1 I’d love to see this. I’m fine with no package open class variant. Just package class (open within package) and package final class is fine.

I concur with @esummers . package class would be open within the package.

1 Like