Feedback on package access level

Evolution Discussion
1

I've been working on a project recently where I've made extensive use of the recent package access control level, and I have some feedback to share.

(Proposal link, if you need to refresh: SE-0386 New access modifier: package)

The Project

The project is a set of libraries and a suite of applications. These are specialised applications deployed on a fleet of managed devices, and there is a desire for some of the technology to be made available as an SDK.

One of the priorities when doing this was to keep the SDK libraries feeling like "our" libraries - they should be able to share implementation details with each other and the first-party apps, as we do now, without making API commitments. Additionally, the working environment should undergo as few changes as possible - we don't want to make it cumbersome to make changes for the first-party apps.

One issue with making this transition is that it would result in public having a different weight in certain files within the same workspace. Previously, the only way for libraries to communicate with each other was via their public interfaces, so we use public a lot and are in the habit of writing it, and for non-published libraries that's fine. But for SDK libraries we want developers to be more conservative - those libraries will be published, so public in those modules makes commitments to external clients.

.
└── Project/
    ├── lib-sdk/
    │   ├── SharedTypes
    │   └── SDKModuleA
    ├── lib/
    │   ├── InternalModuleA
    │   └── InternalModuleB
    └── app/
        ├── AppA
        └── AppB

So my idea was to try making package our new default for internal interfaces between modules and to reserve public for external API commitments across the board. It's a bit of an adjustment, but that way, access levels have a consistent meaning.

And as an added benefit, the proposal notes that the compiler will not use resilient access patterns for package symbols in library evolution mode, since modules in the same package are part of the same resilience domain. Our SDK libraries will be distributed as binary XCFrameworks, so using package everywhere looks like the right approach to get the best performance.

Feedback 1: package is great

For the most part, replacing public with package for the use-case I described just works. You basically don't need any additional code changes, and it's great - it does everything I wanted it to do.

For instance, in the diagram above, SharedTypes is able to contain types and expose implementation details to our internal modules, without making them available to SDK clients:

// In SharedTypes.

public struct Country {
  package init(rawISOCode: String)
}

// SDK Clients can receive a 'Country' from the API,
// but can't construct them.
// Internal modules can construct them as freely as before.

I just made everything that was public now package, everything still built, then I made only the Country type definition public. It's that easy to manage these two levels of exported API.

If I think of trying to implement this without package, it would require making everything public and using #if build-time conditions everywhere. I can imagine we'd sometimes forget to include everything in the right #if blocks and get build failures when building in the other mode. This is much better.

It's been so successful that I think I might adjust my coding style for personal projects to also make package the default for package interfaces, rather than making them public.

Even if you don't have the same kind of needs that I do in this project, and your libraries are never published and used by external clients, I think it's probably a good habit to develop to write your package-level interfaces with package, and to get out of the habit of making them public by default. The language is allowing us to express module interfaces more precisely, so I'd encourage everybody else to also consider adjusting their habits to make use of it.

Then, in the future, if you do ever want to publish parts of the library, you can expose a very minimal API surface with basically no additional barriers impeding your regular workflow - you don't need to break things up in to smaller modules, you're not prevented from sharing implementation details within the package, etc. It's really nice.

However, one thing to be aware of is that if you start wholesale replacing public with package, you need to do it everywhere at once. The compiler doesn't know that your non-published modules won't be published, so uses of package types in nominally-public interfaces will not compile:

// In InternalModuleA

// Error:
// Function cannot be declared public because its type uses a package type
public func doSomething(_: SomePackageType)

This may seem obvious, but this "viral" property of access control becomes an issue later.

Feedback 2: classes are painful

The one area where package falls down is classes.

Swift, of course, favours mutable value semantics, structs and protocols, but classes are still important. Lots of Apple's APIs are object-oriented, and even users of modern frameworks such as SwiftUI make use of classes for things like view-models and @Observable state. As such, it is not unusual for developers to maintain classes, and to refactor those classes in to hierarchies. It is not always feasible to adapt such interfaces to structs and protocols.

Unfortunately, package does not really work with subclassing at all. While an internal class can be subclassed freely throughout the declaring a module, a package class cannot be subclassed throughout the declaring package.

// Cannot be subclassed in other modules of the package.

package class Base {
  package func foo(_: SomePackageType)
}

Of note, protocols do not have this limitation, and can be refined and conformed-to throughout the package:

// Can be refined and conformed to in other modules of the package.

package protocol BaseProtocol {
  func foo(_: SomePackageType)
}

In order to allow subclassing, we need to violate our package-everywhere rule and declare Base as open, which also makes it public. If Base happens to be in a published SDK module, we would be forced in to making an API commitment we don't necessarily want to make.

But let's say we're lucky and Base is in an internal module. It won't be published, so we can violate our rule without really making API commitments. The next problem occurs when we need to override a function which uses package-level types:

// Technically "open", but this whole module is package-use-only.

open class Base {

  // Error:
  // Function cannot be declared open because its type uses a package type
  open func foo(_: SomePackageType) {
  }
}

This is the "viral" nature of access control. If you look at open solely from a subclassing perspective, this seems overly strict.

I understand why this happens, but it's quite annoying and you can't really work around it. It's the one area where adopting package puts barriers in your way that weren't there with public.

I note the proposal mentions subclassing as a "future direction", but I would like to call it out specifically because this aspect didn't get that much attention in the review discussions, although I would like to point out this post from @John_McCall

The current package class is Box C.

In my opinion, it would be more in keeping with the spirit of the proposal if it were Box B. If we're going with the idea that a package is managed by one "team" - it's a unit of distribution, version-locked, resilience domain, etc - there seems little value in making a class accessible to your team-members but artificially restricting their ability to subclass it.

But in any case, the lack of anything in Box B is what I'm struggling with here. We should really add a spelling for this.

Conclusion

In summary, I'm really happy with the new capabilities package gives us. It's early, but I'm confident that it doesn't put obstacles in the everyday work of the development team (beyond having to unlearn a lot of uses of public), and it still allows to make the minimum API commitments possible to clients.

As I mentioned earlier, I encourage all library developers to use it for their package-internal interfaces rather than public, even if you've never had these issues. I think that's probably a new Swift "best practice".

But we do need to sort out classes to make the most of these new capabilities.

27 Likes
2

Taking a cue from the fairly common “private(set)” access level for properties, perhaps classes could optionally specify an access level for subclassing. For example, this might be spelled “package(subclass)”.

3

I understand the desire to avoid making subclassing orthogonal from visibility (although you could argue that it kind of already is with final...), but I think we could come a long way by just changing the meaning of package class.

The thing about Box B/C is that C (the current package class) represents a class which:

  • Is visible to other modules in the package
  • Is subclassed by this module (if you want to ban subclassing outright, you have final)
  • But should not be subclassed by other modules in the package

And this is just such a specific set of circumstances that if there's one of these options which we think is not worth expressing, I'd argue it's this one.

The one thing this gives us over Box B is that we can infer final if you don't actually subclass in the declaring module, without needing to extend Whole Module Optimisation to Whole Package Optimisation. I think that's a bit niche to begin with, but even if it does come up... why not Whole Package Optimisation? :slight_smile:

(I mean, don't we have cross-module optimisation already? Perhaps it can provide this information)

2 Likes
4

We already have a recent precedent for saying "packages should be allowed to do things that used to only be allowed by modules", which is retroactive conformances without having to spell @retroactive. Package subclassability sort of feels like a similar bucket.

I tend to agree that the box C that you call out feels like the kind of finely-grained access control auditing that people love to think they need but that doesn't really buy you anything in practice. It's hard to see any real harm from just letting package classes be subclassed within the same package.

The only potential concern I could imagine is if it would be possible for someone from outside your package to pass the same -package-name identifier to the compiler to "break into" your package and access subclass that they shouldn't. SwiftPM wouldn't allow that because it controls the use of that flag. For any other framework distributed to end-users, they wouldn't even know those declarations are there because they should only get the public .swiftinterface file, not the .package.swiftinterface. So I don't think there's a risk there, either.

(I may have in my past done something similar to access package-restricted Java APIs by declaring a class to be in the same package as the one I needed to access.)

6 Likes
5

If we align with this philosophy, then public should also eventually move to (A) from where it is now (i.e., public types should be subclassable anywhere within a package) and the "default" access level should become package.

Without arguing against any of these excellent points, big picture with this approach is whether the end of the road here is that package simply supplants internal for most purposes, much like fileprivate (being the old private) got supplanted for most purposes by new-private. One also wonders if we're just sort of introducing submodules by incremental feature building.

If so, I'd like to just bite the bullet and do these things in an explicit, designed way (with a vision document, say) rather than by accretion.

7 Likes
6

IMO, it would make sense for us to allow open to be a parameter of access control modifiers. So you could declare:

package(open) class C { ... }

And you would get a class that is visible to and can be subclassed by the entire package. Note that this would make the current open modifier an alias of public(open).

From there, we could also introduce a closed parameter. This would allow us to create closed protocols that cannot be conformed to from outside:

public(closed) protocol P { ... }

With that, you could declare:

public(closed) package(open) class C { ... }

And you would get a class that is publicly visible but can only be subclassed within the package where it's declared.

7

I don't think it makes sense for package to fully supplant internal as the default visibility level. There is still an important distinction there that internal declarations are only accessible to code that can be compiled alongside the declaration itself. Different modules with package visibility into each other may be OK with being version-locked to each other such that a dependent has to take on source changes or recompilation in response to changes in its dependencies, but it can still be compiled separately with only knowledge of the dependencies' module interfaces and not their complete source code.

9 Likes
8

If you do not mind using underscore attributes. You can use @_spi to workaround the open + package issue you mentioned early. (Even Apple is actually using the pattern extensively for their internal SDK as I have noticed eg. SwiftUI and SwiftUICore)

@_spi(InternalOnly)
open class Base {

  @_spi(InternalOnly)
  open func foo(_: SomePackageType) {
  }
}

@_spi(InternalOnly)
public /*package*/ SomePackageType {
    package init() { ... }
}

You can then use @_spi(InternalOnly) import in your other package target or internal apps here then.

Outer client can't access it because you can strip the package.swiftinterface & private.swiftinterface in your shipped SDK [1].

Also changing @_spi public would not break the API here unless you are using @_frozen.

  1. A gist for building swift package to external client Generate xcframework for Swift package · GitHub ↩︎

1 Like
9

If we had namespaces, couldn't a package-wide open class be declared as:

// The namespace Utils is only visible inside this package
package namespace Utils {
   // sub-classable by anyone with access to the namespace
   // i.e. package wide.
   open class Base {
      open func foo(_: SomePackageType) { ... }
   }
   // usable package wide, but not sub-classable except in this module.
   public class Sub: Base {
       override func foo(_: SomePackageType) {
           ...
        }
   }
}

// in another module in the package
// could also import Utils
// OR
// import class Utils::Base
// OR
// typealias Base = Utils::Base
final class Derived: Utils::Base {
   ...
}

Or am I missing something?

10

Since this idea has come up in this thread I'll link to this other thread that I created a little while ago where I was hoping for something like this. I wonder if there's a reasonable way to allow the package developer to opt into package as the default without imposing this choice on everyone.

This would certainly be exciting to me.