SE-0403: Package Manager Mixed Language Target Support

Evolution Proposal Reviews
21

Thinking more about this, I don't think you need to add the generated header to the public product module map at all at the moment. The reason being is that the public headers in the mixed target cannot reference the generated header, as the generated header generation depends on the public headers being built and loaded as a Clang module when the generated header is generated from the Swift module in the mixed target. Thus you don't need to modularize this header, as the dependent Swift modules would not need to include it, and dependent C/ObjC/C++ compilations can load it textually.

1 Like
22

Yes, I think that's a viable approach that's worth investigating, or bridging headers as well. But it depends on the goals you're trying to accomplish for this pitch, do you want to let users have that capability? I think it could potentially make sense to support it the private.module.modulemap in the same include directory next to the custom module map. Then the user could have a consistent approach to referencing the headers in both the public and the private module map that they write. You do need to make sure that this private module map is loadable by the dependent Swift targets though.

Another alternative is using some form of a (potentially modularized) bridging header and letting the user avoid writing a module map. Using modules is preferable for sure for a number of reasons, but we could modularize the bridging header potentially ([Prototype] Import ObjC Bridging Header as module by cachemeifyoucan Ā· Pull Request #67538 Ā· apple/swift Ā· GitHub).

I think it's interesting to think about where this goes in the future. If you were to support a mixed-language executable target, would you use a module map or bridging header? It might be worth thinking about consistent mixing within a target regardless of the target's kind. I know that that's different to what Xcode does but it might be better to simplify and offer users a consistent mixing experience.

1 Like
23

I follow. I sort of assumed we would want to support the case where an ObjC client imports the mixed target module and expects the Objective-C compatible Swift APIs to be included in that import statement. Do you think this is worth supporting? Thinking about it, it does seem simpler and more consistent to only support importing the Swift header via the textual import. I'm working on updating the proposal document and will highlight this change.

24

As a library developer looking to incrementally add more Swift, yes I'd like to have that capability. Whether it makes sense to part of this proposal versus a follow-up, I'm not sure yetā€“ I'll need to quickly explore some of the options.

With regard to an approach featuring a bridging header, I can revisit this. I did once try to expose the non-public headers via a bridging header but the issue I ran into was that by using the -import-underlying-module flag when compiling the Swift module, the target is viewed as a framework and so I can't also pass the -import-objc-header flag (error).

1 Like
25

In the "Motivation" section, when discussing workarounds:

binary dependencies are only available on Apple platforms

but this is also a limitation of your proposed solution?

I think a new MixedSourceModuleTarget type would be needed, but otherwise PackagePlugin support might be mentioned in the "Future Directions" section.

26

Yes, it is a limitation of the proposed solution. The reason was that Swift targets do not emit the generated ObjC header when not running on Darwin. This still seems true on main:

I realize now however with the added goal of supporting C++ interop in this proposal, this may need to be adjusted so that the limitation only affects targets that mix only Swift and Objective-C?

cc: @compnerd

Agreed. With the review period closing, I'm leaning towards adding it to the "Future Directions" section. But, I have begun experimenting with it, so I'll update this post at EOD with a more informed comment here.

edit: I'll add it to the "Future Directions" section in the interest of time.

1 Like
27

I had one more question related to your comments on the design generating intermediary build artifacts that I'd appreciate feedback on.

When creating the generated Swift header, I know the compiler will try to import an umbrella header to resolve symbols that cannot be forward declared in the generated header.

// MixedPkg/Sources/include/Car.h
#import <Foundation/Foundation.h>
@interface Car: NSObject
@end

// MixedPkg/Sources/RaceCar.swift
@objc public class RaceCar: Car {}

// The generated MixedPkg-Swift.h assumes that there is an umbrella
// header to import. This is a problem if the target doesn't specify an
// umbrella header in a subfolder named after the target within the public
// headers directory.
#import <MixedPkg/MixedPkg.h>

I believe this has to do with the compiler thinking of the target as a framework (relevant Apple docs).

Anyhow, I didn't want to enforce package authors mixing Swift/ObjC to alter their public header directory to include such a subdirectory.

MixedPkg/Sources/include/MixedPkg/MixedPkg.h

So the proposal/implementation effectively looks for this structure and generates the directory/umbrella header if it doesn't exist. If generated, it is overlayed over the public headers directory via Clang VFS.

Given some of things I've learned since, I think I'd rather the package author need to conform to this requirement than letting the package manager try to synthesize such artifacts. This is for a few reasons ranging from making the implementation needlessly more complex, concern that this could break down when introducing other languages to the target, and at least one weird issue I documented here proposal.

What do you think?

cc: @NeoNacho @Max_Desiatov

28

Seems reasonable to me to have this be a task for the package author. Do we have any good way of validating that upfront or will this basically manifest as compilation errors?

2 Likes
29

A precise validation is hard since it involves not only checking that an umbrella header exists in the correct subdirectory, but also that the umbrella header imports the header(s) necessary. In my earlier example, we ideally want to validate that Car.h is imported into the umbrella header.

Without any validation, it would manifest as a compilation error. We could do a partial validation in that we issue a warning if the correct public headers structure does not exist, where the correct public headers structure would look like:

Pkg/Sources/PUBLIC_HDRS_DIR_NAME/TARGET_NAME/TARGET_NAME.h

Now, I think this issue is only related to Swift/Objective-C interoperability (as opposed to pure Swift/C++ target). If I'm correct here, the package manager may need to adjust based on what languages the target is mixing. For that reason, I'm thinking that it may be simplest for authors to explicitly opt in by setting the target's SwiftSetting.InteroperabilityMode. Right now, there are two cases in that enum, .C and .Cxx. SPM doesn't seem to do anything for .C yet, so maybe passing .C explicitly opts into such ObjC interop specific behaviors (e.g. warnings, limiting what languages can be used together on non-Apple platforms). If .C is reserved for interop with the C Language, maybe we can add an .ObjectiveC case then?

30

Thinking more about this it would be better to modularize the generated header as well, in the public interface of the mixed target. The reason for that is that dependent C/ObjC/C++ targets can then include it in their modularized public headers, which can then be included from another Swift target. Sorry for the misdirection, but I think it's worth keeping it modularized for the public interface, but there's no need for this modularization within the mixed target itself.

1 Like
31

The isDarwin() part can be removed. The generated header no longer requires Objective-C, and can be used in pure C and C++ contexts as well. It would be great if your proposal made this change.

2 Likes
32

Good question. The #import <MixedPkg/MixedPkg.h> is an implementation detail of the generated header that stems from the fact that this was originally implemented to mix ObjC frameworks that guaranteed that layout. I think we should no longer follow this constraint and make relevant compiler changes instead to generate an include that actually makes sense for the specific scenario. I would be more than willing to make this change in the compiler alongside your proposal, as I think it would make it a better feature, as it wouldn't force users to conform to this requirement that stems purely from Apple's framework layout. How about this:

  • If we know that the headers have an umbrella header that the synthesized module map uses, we can instead emit reference directly to that umbrella header instead. It could be even done automatically in the header with preprocessor checks, for example we can emit:
if __has_include(MixedPkg/MixedPkg.h)
#import <MixedPkg/MixedPkg.h> // import a framework layout umbrella header
#elif __has_include(MixedPkg.h)
#import <MixedPkg.h>         // import an umbrella header
#endif
  • otherwise, we can emit a conditional that includes the framework style header or all textual includes from the user provided module map when they don't have an umbrella header in a module map:
if __has_include(MixedPkg/MixedPkg.h)
#import <MixedPkg/MixedPkg.h> // import a framework layout umbrella header
#elif __has_include(headerA.h) && __has_include(headerB.h)
// import headers present in the user's module map.
#import <headerA.h>
#import <headerB.h>
#endif

There's existing compiler support for emitting these kinds of references in the generated header, so I would just need to make one change to account for this proposal.

Then the dependent targets would just need to make sure the public include path is appropriately set for their Swift and Clang compilations, and both then should be able to pickup the header.

1 Like
33

Hi everyone ā€“ā€“ I've been continuing work to address the feedback raised in the above discussions during the initial formal review period. As I do this, I want to highlight some design decisions that have cropped up while working through the review feedback.

Plugin support for processing mixed language targets

Hi @benrimmington, so I've since dug into this in an effort to include it in the scope of this proposal, with the goal here being that mixed language targets can be processed by package plugins (be it a build or command plugins).

The PackagePlugin public API will need to change to support this as there there is no API that represents a source mixed target. I've added a public MixedSourceModuleTarget by joining together the properties of SwiftSourceModuleTarget and ClangSourceModuleTarget (commit).

  1. In my current approach, joining the properties of SwiftSourceModuleTarget and ClangSourceModuleTarget together to make MixedSourceModuleTarget meant adding a compilationConditions: [String] from the SwiftSourceModuleTarget and preprocessorDefinitions: [String] from the ClangSourceModuleTarget. To make them more distinct I prefixed each property's API with either swift or clang to make it clearer what they pertain to. I'd appreciate any feedback here regarding whether the prefixing is worth doing.
    /// Any custom compilation conditions specified for the target's Swift sources.
     public let swiftCompilationConditions: [String]

     /// Any preprocessor definitions specified for the target's Clang sources.
     public let clangPreprocessorDefinitions: [String]
  1. An entirely different approach I wanted to call out would be to remove/deprecate the language specific {X}SourceModuleTarget types in favor of a general type that encompasses all three cases (Swift, Clang, mixed). This would simplify the API surface. In terms of naming, SourceModuleTarget seems reasonable, though there is already a SourceModuleTarget protocol that the SwiftSourceModuleTarget and ClangSourceModuleTarget types conform to, so this sort of complicates a straightforward deprecateā€“thenā€“replace strategy.

I'd appreciate any feedback on which approach is preferred. Personally, I could lean either way.

cc: @NeoNacho, @Max_Desiatov, and @abertelrud for past Plugins work

3 Likes
34

I think I prefer this approach since the eventual goal for SwiftPM as a whole should be that we don't have three distinct targets types, but that essentially every target would be mixed. Some mixed targets may only contain Swift files, but that seems fine.

From that perspective, we should design the MixedSourceModuleTarget API in such a way that we would be happy with it in a world where that would just be SourceModuleTarget and the other ones weren't there anymore.

To that end, I think instead of prefixing, we should have some kind of nested approach in which we group all the settings pertaining to the Swift compiler together such that one would write e.g. .target.swift.compilationConditions.

1 Like
35

@Alex_L, looking into this a bit more, I wanted to double check that the isDarwin() part (here) can indeed be removed. It's guarding whether or not to add the -emit-objc-header flag, which I assume does require Objective-C.

edit: Chatted with @compnerd about this and I understand your comment better. My assumption is wrong as I learned that -emit-objc-header doesn't have any dependency on Objective-C. The "objc" part in the name was what was throwing me off from realizing this.

36

The preprocessorDefinitions property already seems specific to Clang, so I don't think the prefixing is necessary.

PackagePlugin APIs use any Target or any SourceModuleTarget existential types. Therefore, could the properties be added to the SourceModuleTarget protocol, and could the MixedSourceModuleTarget be an internal type?

Could that simply be:

public struct MixedSourceModuleTarget: SourceModuleTarget {
  public let clang: ClangSourceModuleTarget
  public let swift: SwiftSourceModuleTarget
}

and would it matter that the unique id and name are shared by mixed and nested targets?

37

So if MixedSourceModuleTarget was an internal type, then does that mean clients could cast mixed targets to both SwiftSourceModuleTarget and ClangSourceModuleTarget?

I didn't consider the above approach (I'll call it approach #1). Check out this commit for my go at the nested approach brought up by @NeoNacho (approach #2). They both seem pretty similar. One advantage of approach #2 is that by not referencing {Clang|Swift}SourceModuleTarget, there is more flexibility when it comes to removing them in the future. OTOH, I introduced two new types in approach #2 for the nested properties.

38

No, in this scenario those types would be deprecated and unused. I can only find one PackagePlugin API which needs them:

extension Package {
  public func targets<T: Target>(ofType: T.Type) -> [T]
}

When using "swift-tools-version: 5.9", the following PackagePlugin APIs are available:

extension Package {
  public var sourceModules: [any SourceModuleTarget] { get }
}

extension Product {
  public var sourceModules: [any SourceModuleTarget] { get }
}

extension Target {
  public var sourceModule: (any SourceModuleTarget)? { get }
}

A package manifest can have separate preprocessor definitions (and header search paths) for C and C++:

cSettings: [
  .define("A"),
  .define("B", to: "C"),
],
cxxSettings: [
  .define("D"),
  .define("E", to: "F"),
],

but these are given to the ClangSourceModuleTarget as a single array:

preprocessorDefinitions -> ["A", "B=C", "D", "E=F"]

Should the plugin have separate properties for C and C++ settings, or alternatively should the manifest have a combined clangSettings parameter?

2 Likes
39

I know I'm late to the game, but want to extend my appraisal and thanks for this effort!

We are currently planning to decompose our monolithic mixed ObjC and Swift app (95% ObjC) into more manageable packages, and this turns out to be a crucial requirement. While we have several "clean" features that are either plain Swift or ObjC, we have many that are not. Our team's policy is that any new functionality is to be written in Swift, which means mixing the languages when working on existing components. However, the result is Swift code that is deeply interwoven with ObjC and the other way around.

The current workaround of splitting code into targets by languages will not work. And since most code is still on ObjC, our current take is to back-port those Swift classes to ObjC. Which is exactly the opposite of a gradual migration to Swift. :sweat_smile:

To cut it short: Thanks @ncooke3 and everybody involved! This is really important, crucial work for us.

2 Likes