Cross-import overlays

Evolution Pitches
21

Nice to see this moving along. Assorted comments:

If swiftoverlay files with the same name are present in both the swiftcrossimport directory itself and the module-triple-specific subdirectories, the compiler takes the union of all of their “modules” lists.

What if some of the modules have the same name, but additional keys that differ? It might be better to define priority rules now, in case they ever grow more attributes. Or to just say it's not allowed.

Imports with the @testable or @_private attributes are not included in the cross-import set; nor are their transitive imports.

What's the standard way to test my cross-import overlay, then? Just be explicit? (@testable import KarrKit; @testable import _KarrKit_Combine)

Any imports of one of the declaring modules directly in the source file are disabled, so the compiler behaves as though the declaring module was not directly imported into that source file.

I'm sure there is a good reason for this, but what is it? Since you already have to handle cross-import overlays _KarrKit_Combine and _KarrKit_Tractor alongside each other, it seems like looking into the original KarrKit would be just as easy. (Thus eliminating the dependency on @_exported.)

Also, while the order of lookups into cross-import shouldn't be specified, you should make sure it's deterministic, because I can see it subtly affecting things. (Certainly it can affect diagnostic order.)

  1. Load the list of cross-import overlay modules for that pair of modules.

What happens if a cross-import overlay is named, but missing? Is that a compiler error, or a warning, or ignored?

It's very weird that this proposal depends on @_exported when that hasn't officially been made part of the language, even if, as you say, we can hardly break it. I think it's worth formally acknowledging @_exported at this point, but that would be a separate proposal. Hm.

I also agree with the several people who say this needs SwiftPM support. It may still be a separate, parallel proposal, but it should be considered in parallel with everything else. I suggest that an initial version of the SwiftPM proposal doesn't do anything to reduce dependencies at the package level (so, e.g. you still have to declare the cross-import overlays as explicit dependencies for now, even if they are not imported explicitly in the source).

3 Likes
22

Even later to the party, but I'd just like to underline:

and entirely agree with:

3 Likes
23

I'd like to resurrect this topic. This is still an important feature for the ecosystem, and it would be a shame to let it go stale just because Apple's immediate needs have been taken care of.

(BTW, the gist from the OP has moved. New link).

I have a couple of questions about the implementation, though, apart from SwiftPM support:

A cross-import overlay module wraps its declaring module , re-exporting the declaring module’s contents but also adding extra APIs to it. The compiler imports the cross-import overlay module in place of the declaring module only when an additional bystanding module is imported into the same file.
...
the compiler would essentially rewrite this:

import KarrKit
import Combine

Into this:

import _KarrKit_Combine
import Combine

While making KarrKit an alias for _KarrKit_Combine in that source file.

What I understand by "wrapping" the declaring module is that all types from KarrKit would no longer be part of that module, but instead part of the overlay _KarrKit_Combine. So if I expose any types from KarrKit, any other modules which use those types will also need to use _KarrKit_Combine, else the types would be seen as distinct:

// Module A
import KarrKit
import Combine

public func findVehicle() -> Karr // Looks like KarrKit.Karr, but is really _KarrKit_Combine.Karr
{
  ... 
}

// Module B
import KarrKit
import ModuleA
// Doesn't use Combine, doesn't import it.

let vehicle: Karr = findVehicle() // ERROR: Cannot convert _KarrKit_Combine.Karr to declared type KarrKit.Karr

Is that correct? Or is something else meant by "wrapping"?

I don't think module wrapping is really what we want here - I think what most users are looking for is an implicit import so the KarrKit-Combine integration APIs become available without the syntax burden of adding extra import statements. In other words, when the compiler sees:

import KarrKit
import Combine

Rather than the cross-import overlay outright replacing KarrKit, it should instead become:

import KarrKit
import Combine
import _KarrKit_Combine

To function correctly, a cross-import overlay module should use an @_exported import of the declaring module and a regular import of the bystanding module:

If the cross-import overlay was added to the original import list instead of replacing its declaring module, would this still be necessary?

To hide the cross-import overlay from autocompletion, its module name should begin with an _ . We recommend using _<declaring module>_<bystanding module> as the overlay name, but this is just a convention—the compiler doesn’t ascribe any special meaning to this name.

IMO it's probably better not to hide the name of the cross-import overlay in general. There may be exceptions, like Apple's SDKs, which are a tightly integrated set of libraries, but for random libraries out in the wild, it's probably better if they have clear, descriptive names - such as NIOFoundationCompat. So if they do show up, it doesn't look so cryptic.

2 Likes
24

Instead of leaning on convention only, one suggestion is to explore the opposite tack:

Folks often use + already in file names to denote when one type is implementing a protocol, etc. Suppose instead we explicitly support the syntax <declaring module>+<bystanding module>, such that the library the cross-import overlay here would be named KarrKit+Combine. The name is immediately descriptive and users would not be confused as to why it exists even on first viewing.

10 Likes
25

If _KarrKit_Combine uses @_exported import KarrKit (either explicitly or implied by the wrapping), this Karr will be KarrKit.Karr.

3 Likes
26

Oh, I see, thanks. I'm still a bit hazy on what exactly @_exported does.

So what exactly is the difference between doing this - replacing import KarrKit with a module that exports KarrKit, and leaving the original import intact but adding an implicit import of the overlay module alongside it?

27

Has there been any movement on this?

2 Likes