What modules were you looking at? Do you think they would've reexported if they had a choice?
What types from SwiftShims show up in the standard library ABI?
You could also have a non-exported import of SwiftShims, followed by @_exported imports of specific decls from SwiftShims.
[DNM] Intentionally expose some symbols we think should not be by Catfish-Man · Pull Request #21198 · apple/swift · GitHub is a good example.
Sure, that handles some of the information-hiding, but not the "we want to stop shipping these headers altogether" part.
That's an interesting question. I think the answer is usually "no, because people want Swift to work the same way as Python". I still believe that that's not a good idea in a world with C interop (especially if we get PrintAsObjC on Linux, even if we don't support mixed-source targets).
I sympathize with your desire for consistency, but I worry that "because C interop" is a reason that'll fall on deaf ears for most users, especially those used to other languages' C FFIs that don't infect the host language with C's lack of namespacing. As you note in the linked PR, we'd like to get away from exposing SwiftShims through the standard library today, and that seems like it'd become impossible if we just said that all imports are transitively exported forever. On a somewhat-related note, we often have to tell people to drop down to C for things that Swift doesn't yet natively provide, such as for concurrency primitives or exact struct layout, but they reject that solution because they can't use an internal-only bridging header and would have to expose a Clang module. I know it isn't easy, and there's usually something more important eating up our time, but I feel we ought to figure out a way for Swift library code to be able to use C code without also having to expose the fact that it came from C to the library's clients.
12 Likes
Let's say there are two uses of C code in Swift: use of existing libraries with their own API (like all the system frameworks on Apple platforms), and code written specifically to support a C library because it can't be written in Swift (like SwiftShims or concurrency or struct layout). In the former case, we definitely want to treat these as canonically coming from C, so that two libraries that both use Foundation class types will be able to interoperate.
The latter case is potentially something new. We could treat these entities as "coming from Swift" somehow—altering their mangling, not encoding the source module as "from Clang" in serialization, etc—but that doesn't get you away from having to ship them if they appear in your ABI. They have to live somewhere.
This is true: if you expose something, somebody might use it. But I don't see a way to not expose things if you're using it.
Maybe the answer is to ignore the "ABI" bit. You need to re-export whatever appears in your API, but stuff in your ABI can stay "must be present on a client's machine, but not re-exported". That means we don't get to treat this as a fix to the operator and extension problems, which is a bummer, but it's a valid choice. It also means we end up with three kinds of import right away, instead of just two.
(We really need implementation-only imports; we really want a formalized _exported, and it helps deal with the generated header inconsistency; and you're arguing that there's still a cause for today's import that must be present but isn't made visible.)
A completely separate discussion has sparked an idea here: we've talked about @qualified import Foundation before where you have to say Foundation.URL instead of just URL. What if the "re-export" that happens here is a qualified-only re-export by default? That's closer to "the Python model" in that the top-level namespace isn't polluted, but extensions and stuff are still considered visible.
EDIT: To be clear, there would be three kinds of imports for a framework here:
- "implementation imports" - these are only used in non-inlinable function bodies and do not need to be present on the client's machine.
- "dependency imports" - these are used in a framework's API or ABI, and therefore must be present on the client's machine. The import can be used from client modules when using qualified lookup syntax, and its public extensions and operators are present, but unqualified lookup will not find its declarations. For compatibility with Swift 4.2 (and probably Swift 5) this would be the default behavior of
import X. We might be able to change this or deprecate the syntax in some future release. - "exports" - these imports should be considered part of a framework's API; clients can use everything as if it were part of the framework itself.
5 Likes
I've been really busy and haven't had a time to look closely at the recent discussion in this thread. I'm hoping to soon, but I do want to reply to this right away.
I would like @qualified import but I don't think qualified re-export should be the default. This means that when I import Foo my code can use symbols from Bar if there is code in Foo that has import Bar. I really don't want that to be the case. In general, I want my code to be required to say import Bar before it is able to use symbols from bar.
It would be even better if we could distinguish at the build / package level modules we link (i.e. have an indirect dependency on) from modules want to depend directly on (and be able to import). The ability to centralize this kind of policy is very helpful on large projects. In lieu of direct support in the build system and compiler, if we are required to import Bar then we can at least use a linter rule to ban imports of modules on which we have an indirect dependency but for which we do not wish to take on a direct dependency.
The part about "can be used from client modules" (presumably without the client module saying import Bar) is what makes me uncomfortable. I believe we do not have enough control over symbols available to our code if import Foo makes symbols from Bar available outside of relatively narrow use cases like umbrella frameworks. I know supporting more granular control of symbol visbility makes the implementation more complex but I suspect the complexity is worth the cost - it will help a lot with managing dependencies on large projects.
1 Like
It's not implementation complexity I'm worried about here. It's a difference in behavior between Swift and Objective-C that's necessarily exposed to Swift users on Apple platforms.
I mean, yes, I'd like to not have to come up with a good and also source-compatible way to deal with the extension and operator visibility problems, but even apart from that I want a model that's going to make sense. I can't see a way around "when you import this C module, you get all of its dependencies", and "when you import a Swift module that has a C/ObjC interface, you get at least some of its dependencies".
I think a module should be allowed to reexport the dependencies it wants, allowing you to build umbrella modules. But it'd make more sense to me if by default importing a module did not import its dependencies for name lookups.
This can get tiring in C/C++ when headers import others headers but what those headers import vary between platforms. At some point you realize that code importing only cross-platform headers is not as portable as you thought because it also depends on symbols from platform-specific headers imported by those cross platform ones. You don't have the choice in C to restrict your imported symbols to only the modules you really want to depend on. I'd rather not perpetuate this problem.
That's why symbol visibility should be limited to what I explicitly import. Libraries can reexport other modules when it makes sense, but what they reexport essentially become part of their API so it should be done with explicit intent.
This is also pretty much how modules works in D, and I never felt it made it difficult for D code to interact with C code. If you really need to avoid too many imports, you make an umbrella module publicly importing everything you need.
3 Likes
I have taken time to re-read this whole thread and give it some additional thought. Perhaps I am naive (in which case I would appreciate further explanation) but it seems to me like we are discussing unnecessary complexity in the programmer model here, specifically:
- imports used in API/ABI, which may be present on the client side
- possibly another kind for "imports used in API/ABI that are pretty much required because without them you can't do anything, but still not re-exported"
The model I would like to see as a Swift programmer is simple: a module exposes both exports and link dependencies.
When I use the module I must always link its link dependencies. When I import that module I only get symbols declared in that module and its re-exported modules. Public API in this module (and its exports) which require symbols from other module(s) will only be available if I also import the other module(s).
To make it easier to get the full public API of a module, we could consider introducing @recursive import. The name is a strawman, but @recursive import would place control in the hands of the user of a library and let them say "import whatever modules I need to get the full public API of this library". This places a lot more control in the hands of library users than exports do which is generally more appropriate (IMO) outside of large multi-models frameworks / platforms.
There are a couple of wrinkles to this approach.
Obviously it makes inlining a little trickier. Ideally the implementation would be able to identify symbols from these other modules which are used by inlined code and add tightly scoped imports where necessary. Remember that I don't care about the kind of indirect dependency on these symbols that would be introduced by inlining code provided by the framework I link. I have already taken on the indirect dependency when I imported the module in the first place.
Again, maybe I am naive here but if implementation of this approach is feasible it means that we don't need fine-grained distinctions exposed in the surface of the language. Library authors only need to be concerned with what they import and what they export. They would not be restricted in contexts such as inlinable code.
The other wrinkle here is that as you noted upthread, a module could have a public API that is basically only usable if some other module is also available in user code. I think this is ok. The library author would have the choice of exporting or requiring users to import explicitly. Perhaps a warning would be reasonable in extreme cases.
Re: C and Objective-C, I agree with @Joe_Groff about how this should be handled. If Swift supports re-exporting then we have a way to map that behavior to Swift and should not otherwise let it influence the model we adopt for Swift (especially since it is undesirable in general).
Even if this approach is viable and desirable it still leaves the question of source compatibility. Given that the current behavior is quite different from anything that makes sense as a long-term design (IMO) and the strict policy on source breakage, I wonder if we might need a deprecated attribute to designate current behavior. As a strawman, we can call it @leaky import. This attribute would be deprecated and users would be encouraged to migrate away from it in time.
A migrator could give users the option of updating all existing imports to use this attribute. The migrator could also offer an alternatives of using @recursive import Foo or of adding explicit import Bar for all other modules Bar that are used via implicitly in the current file (either via a Foo API that includes types from Bar or via extensions or operators, etc).
A couple of afterthoughts:
I'm not sure @_exported is the right approach. Exporting feels more like a build target level configuration to me for a couple of reasons. First, it's a global option for the module. Specifying it in code means it is specified in potentially (probably) many files. Some of the imports might be @_exported and others might not. How is the conflict resolved? Second, with code-level export specification there is no centralized location where all exports are visible. A library can try to have a source file which specifies all exports but that is a convention that can easily be violated (even accidentally).
The dual for this "exports" build configuration is the "importable" build configuration on the user side which specifies a subset of linked modules that may be imported by code in the target. This would provide centralized control over the direct dependencies a module takes on (tooling could automatically add modules to this list when they are added to the list of linked modules if this was necessary to make things easier for beginners and small projects).
@qualified import feels like something that should impact the file in which it occurs, not clients which import the library in which it occurs. The way I read @qualified import (and the way I would want to use it) is to have it make symbols available but require explicit, fully-qualified names for all symbols used from that module in the file in which the @qualified import occurred (including operators, extensions, etc).
Under this meaning of @qualified it would be appropriate to also support it in conjunction with recursive imports: @recursive(qualified) Foo would specify that the full public API of Foo should be available, but recursively imported modules would only receive qualified imports.
2 Likes
Okay. Then I need help. Today, we have this behavior:
// MySwiftFramework.swift
import UIKit
import FooKit
import BarKit
public class MyController: UIViewController {
@objc public func foo() -> Foo {}
@nonobjc public func bar() -> Bar {}
}
// Generated ObjC header for MySwiftFramework, simplified
@import UIKit;
@import FooKit;
@interface MyController : UIViewController
- (Foo *)foo;
@end
// MyObjCFramework.h
@import MySwiftFramework;
@interface MyController (MyObjCFrameworkAdditions)
- (void)barrelRoll;
@end
// somewhere in AppDelegate.swift
import MyObjCFramework
let controller: MyController = ...
controller.barrelRoll()
Are the contents of FooKit visible in AppDelegate.swift or not? If not, what should the rule be? If they are, what rule makes FooKit different from BarKit?
I have more I could say in response here, but this is the core problem that needs solving or "not-to-be-fixed"-ing. (Adding @_uncheckedPrivate import allows me to make progress on implementation while we figure out the model.)
My reading of your example is that MyObjCFramework, being Objective-C, re-exports everything it imports. It imports MySwiftFramework so that is automatically re-exported. It can see FooKit because that is imported in the Obj-C header for MySwiftFramework. FooKit is included in the Obj-C header because it is used in the signature @objc public func foo() -> Foo.
I think to the way to square this circle is to say that symbols from imported modules can only be used in @objc declarations (i.e. visible in the Obj-C header) is if the module is exported. In your example and using the @_exported syntax the code would not compile and would error on the method declaration with a fixit to update the import FooKit to @_exported import FooKit. This approach makes it clear that export is forced upon users by Objective-C and otherwise a design decision for the library.
(Using the build-level export I suggested this code might compile depending on how exports are configured in the build settings and the fixit would be to export FooKit in the build config)
What I'd like: in AppDelegate.swift using the name MyController should be an error since the type comes from a module was never imported into that file.
If you want MyObjCFramework to reexport MySwiftFramework, we could add a special flag for the Objective-C importer, something like:
SWIFT_EXPORTED
@import MySwiftFramework;
Or AppDelegate.swift can simply import both frameworks. The code is explicitly referencing symbols from the two frameworks so it makes sense to import both.
That's just not how C works. If you say import UIKit, developers don't want errors saying that they can't use NSObject.
I think it'd be good to separate "how C works" from "how people want Swift interfaces to C work" and "how Apple frameworks work". It's true that the Clang importer can't robustly namespace C symbols, and it's true that Apple frameworks as a matter of policy provides umbrella-ish modules that blanket export their transitive dependencies. That doesn't have to be the only way it works. Apple's frameworks are also special in the amount of work we put into bridging them and trying to hide the distinction between ObjC and Swift frameworks. In the Swift world outside Apple, though, I think that users are generally more aware of what libraries are C versus native Swift libraries, since even as good as the Clang importer is, it's never going to make an arbitrary C library feel "native". Furthermore, it seems to me to be a reasonably consistent model to say that any module with a Clang backing module and an overlay gets to choose what it re-exports. (Outside the Apple ecosystem, the analog would be a swiftpm package trying to provide a Swifty interface on top of a raw Clang module.) Isn't that how it works in fact today, and it's just a matter of policy that Apple library overlays @exported import their underlying module?
4 Likes
No, it isn't; only the overlays write that explicitly. A mixed-source framework always re-exports its Clang module, but whether or not that was the case you'd still run into issues with the example I showed, because there's a pure C module "in the middle" if the import chain.
But I'm really trying to leave mixed-source frameworks out of this and just deal with "Swift libraries with C interfaces", which I think is interesting everywhere.
@anandabits' model is workable, with the new restriction being on things that are both public and @objc. (I'm okay with sacrificing a bit of precision for app targets, which expose @objc internal things too.) It does mean the work here on "implementation imports" won't solve the problems with operators and extension members leaking out, but I do feel better knowing that Python works the same way…even if "extensions" are way less common in Python.
Can you elaborate on what you mean here? I certainly don't want operators and extensions leaking out and didn't mean to propose that (unless you are only referring to things that go through C and therefore must be re-exported).
Hm, is that something that could be changed, so that the Swift side "sits in front" of the Clang side like an overlay from Swift's point of view, and it can choose what it wants to reexport from the Clang side?
It would certainly be nice to have more control over this if it is possible.
I don't think we'd want to change the default, but let's say it is possible. What does that give us? As I said, the problem is about C frameworks that import Swift frameworks, not about mixed-source frameworks.
I know, I don't want it either, but it's not a regression. It's (2) and (3) from the original post.