Update on implementation-only imports

#1

After the long discussion on the previous thread, @John_McCall and I went away to go prototype the main new feature: implementation-only imports. (I'm staying away from "private imports" to avoid the implications of access control for now, as well as the other experimental "private imports" feature that's in support of dynamic method replacement.) We've been doing this work on the master and 5.1 branches of the compiler so that some Apple projects can live on it for a while to make sure (1) it works, and (2) the design is actually what we want. Note that it's not going to be an official feature of Swift 5.1.

Meanwhile, if you want to play around with it, a reminder that this is unsupported, possibly buggy, and definitely not the final syntax. But other than that, you're welcome to!

Right now the new feature looks like this:

// MyKit.swift
@_implementationOnly import SecretKit

The name is pretty unwieldy right now since we expect that'll be a hot topic of discussion. (We've gotten some feedback that it'd be worth trying to change the default semantics of import; I'm personally skeptical whether we can do this in a non-breaking way.)

What does this do today?

  • Clients of MyKit will not need SecretKit around when using MyKit. SecretKit won't even be loaded, avoiding the problems listed in "@_exported and fixing import visibility".

  • The public API and ABI of MyKit is not allowed to use any declarations from SecretKit. This is enforced by the compiler.

  • The public API and ABI of MyKit is not allowed to rely on conformances declared in SecretKit, even if the types involved are from other modules. This is subtle but probably won't come up in practice. (Also enforced by the compiler.)

  • The public types of MyKit are not allowed to conform to protocols in SecretKit. We consider this a limitation, but it's a tricky restriction to lift given the way the several parts of the compiler work right now.

  • If a public type in MyKit overrides a method defined in SecretKit, the override has to be marked @_implementationOnly as well, so that it can be omitted from the module interface. This feels quite hacky to me; everywhere else things are enforced by the compiler, but here the compiler mostly just trusts the developer to get it right once they've put the attribute on.

  • A generated module interface (.swiftinterface file) will not make any mention of SecretKit, and should not reference anything from SecretKit either.

  • The compiled module (.swiftmodule file) may still have references to SecretKit for debugging and testing purposes. This means that the compiler must know not to rely on the parts that depend on SecretKit when the compiled module is used for building clients. (This is where 90% of the known issues are right now.)

  • @testable import MyKit is probably completely broken. This should probably bring in the implementation-only dependencies of MyKit even though they normally wouldn't be brought in, since the alternative is to figure out what's being used in MyKit's internal types as well.

  • LLDB support is only verrrry lightly tested. (I haven't even managed to land the PR with the tests in it, mostly because I've been busy with other things.)

  • I've tried not to make anything dependent on -enable-library-evolution, because I know people want this for packages, but the adopters within Apple are all using library evolution support, so it wouldn't surprise me if things are worse in "normal" / "fragile" mode.

Again, this is what's currently implemented, not necessarily what we want to propose in the long run. And it doesn't cover a number of other use cases, particularly vending content that's only present if the client asks for it.

23 Likes
@_exported and fixing import visibility
Building .swiftmodules for SourceKit
Swift packages and module dependencies
Unknown attributes should be a warning (not an error)
Backwards-deployable Conformances
#2

Probably also worth mentioning that implementation-only import won't affect symbol mangling and you still can't have two modules named SecretKit that end up in the same "package graph".

#3

Sure, right. It's a modifier on the import only; it doesn't affect the existence or contents of the module being imported.

#4

How a public type can override a method without confirming to a SecretKit protocol or being able to inherit a SecretKit type ?

[edit] I just read the PR link description. This is about overriding methods declared in extension of existing type inside SecretKit [/edit]

1 Like
#5

I don’t understand the why. What is the purpose of this feature?

#6

See the thread @_exported and fixing import visibility for more information.

If you have a dependency that is entirely an implementation detail, such as how you do JSON parsing - I could see this being useful.

It also gives compiler support for enforcing the intended use of the dependency.

1 Like
#7

Yep, @Mordil's got it. There are times where a dependency should not be exposed to clients, whether that's because of Swift's current leaky imports that we hope to fix at some point (@_exported and fixing import visibility), or because you're making something for binary distribution and won't be distributing your dependencies. That latter is Apple's main use case (public frameworks being implemented in terms of private frameworks), but it's also a step in addressing one of the biggest limitations in the Xcode 11 binary framework support: you can't use packages with it, because those become dependencies for your clients as well.

(I compressed a lot into that one sentence; check out the WWDC session I co-presented. The problem here is talked about at 37:16, albeit briefly.)

Now, this alone doesn't solve all the problems there, because you still can't have two copies of the same package in the same address space. But it's a step, and it would at least allow for local packages, prefixed with your organization's name in some way, to be used in XCFrameworks. As long as you're careful about not packaging them into two separate binary frameworks, anyway.

In general we really do want to improve the cross-module dependency story in Swift, both for source packages and for binary packages. We as library authors should be able to separate compilation dependencies from link dependencies and control how both are passed on to a library's clients.

(sorry for not responding sooner)

7 Likes
#8

Here is something that has a difference between -enable-library-evolution and not.

If you have:
something.h:

struct A { ... };

something.swift:

struct B {
  var v: UnsafeMutablePointer<A>;
};

With -enable-library-evolution it works, but without it complains with: "error: cannot use struct 'A' ... has been imported as implementation-only"

1 Like
#9

I just wrote a post on my experiences splitting a package into modules (Swift packages and module dependencies ) where one of the issues I raised was automatic "re-export" of conformances and extensions added to imported public types. My suggestion was

but hopefully this feature will deal with this issue!

1 Like
#10

Any news on this? We really need this feature :(

1 Like
#11

@zienag Jordan Rose is not at Apple anymore maybe someone else at Apple will continue his work maybe not, we shall see.

1 Like
#12

The core work is already present in the compiler; it's just (unfortunately) all using internal, non-stable language features.

2 Likes
Terms of Service

Privacy Policy

Cookie Policy