@_exported and fixing import visibility

Ooh, I was hoping no one would bring up export *. Swift is currently pretty broken when people don't do export *, and I'm not sure Clang works well with it either. But on top of that, the module map for an Xcode-built framework containing Swift code currently always includes export * for the Swift part of the module, with no way to configure it. Part of the reason for this is because it's never a good thing when C code behaves differently with and without modules enabled, and while Swift generated headers currently don't support modules-off very well, they should be able to (SR-805).

We may not get forward declarations in Swift (for probably dozens of reasons), but this thread definitely covers adding the implementation-only imports. The tricky bits are around the things you want to expose as API, and then around the things you want to expose as ABI but not API. (The latter is where forward declarations are "usually good enough" in C.)

My bit to the discussion from user of the Swift and SPM.

I use SPM and I have plenty of targets. SPM requires you to explicitly specify the dependencies for each target inside Package.swift, but on the other side, Swift allows me to use transitive dependencies in my source code without updating Package.swift. Sometimes this breaks the build phase, because, as I think, the compiler meets some import-s of not yet built modules/targets. Then I have to dig into the problem, find the missing dependency and update my module inside Package.swift so build system would build the libraries in the right order. Usually this is a pain, because swift build will just print some not very helpful error about libc module. :slight_smile:

I agree that NSObject should be usable if you import AppKit or UIKit, but for non-Apple modules, I think, it will be beneficial to have the ability to disable the use of the transitive deps.

1 Like

I just ran into an issue related to this. Any new developments?

Not really. I've been more focused on module stability right now, but the general outcome of this discussion is that it's probably not worth doing anything clever for Swift vNext: just add "implementation-only import" and maybe "exported import" and leave the rest alone for now.

5 Likes

Thanks for the quick response. That much improvement would be enough for me.

Sorry to jump in like this but could you clarify whether swift vNext means swift 5 in this case?
I assume not, but I’d really like to have an official exported import and would just like a clear idea if I can expect this soon.

No, anything like this would have to go through the Swift Evolution Process, and it's well past the deadline for Swift 5 changes for that.

Thanks! I thought so, just had to check for sure since the thread’s been going for a year, I’ll keep a lookout on swift evolution

I developed a lib, it consists from two parts.
In first part I have a lot of custom infix operators.
Second part do @_exported import FirstPart.

Then my users do import SecondPart in their projects, by they facing with problem that infix operators from FirstPart are not available with just import SecondPart thought they should. So my users should also do import FirstPart.

Is there any chance if it will be fixed in Swift 5?

1 Like

@_exported is an implementation detail (as indicated by the leading underscore); its semantics are whatever it does, and can be changed at any point, so you should not depend on it doing anything specific.

Have you reported a bug? Bugs that are not reported are unlikely to get fixed in any Swift release. It's also possible that the behavior you're seeing is not a bug (it's hard to tell based on the limited description here), in which case reporting it gives you the opportunity to clarify exactly what you're doing and what behavior you expect to see, and for someone to help you figure out how to get it.

Swift 5 is really only taking fixes for critical regressions at this point, so realistically this is not going to change for that release.

I thought Discourse automatically linked threads that reference a particular thread, but I guess not. Check out "Update on implementation-only imports".

I'm late the party, but this thread was referenced in response to a question I had a couple days ago.

I want to build a Swift wrapper for SDL, an open-source graphics/event-handling library. At one extreme, it would make sense to write a comprehensive apinotes file for it and package it as a module (if only I could find comprehensive docs on how to do that). But that doesn't make it very Swifty, particularly in that you don't get ARC support.

So the next thing would be to wrap the API in a set of Swift classes and structs. I started to do this with SDL, and used the systemLibrary() in the SwiftPM to import the C API. I got stuck because there’s a set of #defines in the C API that I wanted to use early on to prove things out, but they weren’t visible to clients of my SwiftSDL module (I haven't yet tried @_export).

In the long run, this should be an OptionSet anyway, and I built that shim manually, but I feel like this is what I should be able to do (for example): Use apinotes to convert an enum to an OptionSet, and re-export just that OptionSet. Since I don’t control the C API, I need to be able to apply the apinotes as part of my library (SwiftSDL), which should allow the compiler to handle the fine-grained re-export.

Also, I don't like language of public import Foo. “Import” means one thing, “export” means the other, so I’d rather see import Foo; export Foo. But it’s a bit strange to export the imported library at the source file level. What happens when I have several source files? Any one of them could trigger the re-export. Seems like this should exist in a more central file for the module (e.g. Package dscriptor or apinotes or modulemap).

1 Like

+1 Just hit the same problem needing to export internal libraries as part of our framework public API. We're using @_exported for now but hoping for an official solution.

2 Likes

Hi,
Sorry for bring this thread to life but I have encountered a very strange problem with @_exported import and wanted to share here for input.

We have a dynamic wrapper framework that embeds few static libraries from a swift package. This wrapper framework is then linked back with lot of modules and app target in our main project.

This framework is using @_exported import for all of its static dependencies so the consumers won't have to explicitly import them. But this have some side affects. Xcode fails to detect that there is some duplicate code in consumer modules and those static libraries and successfully builds the app. This can cause serious side affects in app business logic.

I uncovered this issue when I tried to remove one of the @_exported import in wrapper framework and explicitly import static library in a consumer module.

Any idea what might be going on and how to fix it?

1 Like

You can't, in general, embed static Swift libraries in a dynamic framework, and then you doubly can't do so and re-export them, for exactly the reason you discovered: a client who uses the static library won't know to link to your dynamic framework instead of another copy of the library.

Now, this kind of thing is certainly desirable—people keep talking about it! But @_exported is not the tool to do it, or at least not the complete solution, and I don't know of a "complete solution" in today's Swift.

1 Like

It looks like swift is totally fine in that sense, it's an xcode and general linking problem. And one can totally experience the same problem outside of xcode and swift.

As far as I understand, here you need to
(1) prevent xcode to link those static libraries into executable
(2) force to link all object files from static lib to dynamic lib, with -all_load or similar
(3) have a diagnostic tool that checks that there is no duplicate symbols in dylibs and executable, so one couldn't accidentally link code from static lib to executable.

Am I wrong?

1 Like

It is a general problem (not Swift-specific), but not one with a general solution, because of step 3. The only way to prevent a library used in two places from being included twice in the final build is to link everything statically or link everything dynamically (in that part of the build graph).

Swift also doesn’t support an option for (1) today because of a bunch of modules that get imported implicitly, which would make (2) very painful. But because of (3) it’s not worth it anyway.

Note that this is why SwiftPM doesn’t specify whether a package will be built statically or dynamically, so that it (or Xcode) can use static linking when it’s probably safe/correct and dynamic linking otherwise. I don’t know to what extent that’s been implemented, though.

@jrose Thank you for your reply
You can't, in general, embed static Swift libraries in a dynamic framework
That is the only way to use swift package libraries right now because Xcode (unfortunately) doesn't provide a way to link package dependencies as dynamic (even dynamic swift package libraries are linked as static).

and then you doubly can't do so and re-export them
this was a refactoring task i.e take break existing large module into several swift package based modules and then reintegrate new modules back into main project. @_exported allowed integration with fewer changes. Otherwise we would have to import new modules everywhere in the project.

Just so I understand correctly.. can @_exported make a separate copy of the static library when used implicitly in other modules. Here is a more accurate description of my current setup

/// SPMDependencyContainer dynamic framework that wraps
/// following static libraries from swift packages

@_exported Networking
@_exported LayoutKit
@_exported CoreUtils
/// Framework1 that was broken into swift package modules Networking, 
/// LayoutKit & CoreUtils. This framework will build fine because the code 
/// that was moved into swift packages is available to internal components
/// via following import that re-exports them. This module need to export 
/// same dependencies again so that it's consumers continue to build fine.

import SPMDependencyContainer
/// Modules that were using Framework1 before refactoring are still
/// using Framework1 and they build fine because they implicit imports
/// from Framework1

import Framework1

This setup is great for quick refactoring and can work fine in production assuming there is no duplicate code.

Now, about this bug or whatever we want to call it, can't Xcode treat swift package based module code just like a static/dynamic framework created inside Xcode project and detect duplicate code/symbols which it can do with normal setup already. Why only hybrid setup (Xcode project frameworks + spm modules) is a problem for build system?

As @zienag points out, it is incorrect in the most general case to link static libraries into a dynamic library if someone else might also use them; your use case is special in that if everybody linked to your dynamic library instead everything would be fine. But there’s currently no way to communicate that to the linker, in Xcode or outside of it. Again, I think this is a valid feature request, if one that needs design on how to integrate with existing autolinking behavior, but there’s no supported way to do that today. EDIT: that emphasis is deliberate; it may be possible to trick the linker into doing the right thing, but a future Swift or Xcode release could break it.

(There’s a small exception for some languages for non-exported static libraries linked into dynamic libraries where you can pretend that part of the library does not exist at all. Swift is not quite such a language, exacerbated by the ObjC support on Apple platforms; it relies on the global uniqueness of mangled names in a program. But that’s not relevant to your situation.)

I see what you mean now and that worries me that my setup can stop working one day due to a newer version of Swift or Xcode blocking private apis.

Regarding valid feature request, I agree with you because what's the point of publicly embedding static libraries in a dynamic one anyway which btw is supported case in terms of .frameworks which can store compiled libraries as submodules.

I just hope Swift does the right thing and come up with something like public/private import XYZ (this was suggested by somebody on swift evolution already).

But thank you for all the replies.

1 Like