Relationship, if any, between `-import-underlying-module` and `-emit-objc-header`?

I've been experimenting with manually creating a mixed language module. This is related to the investigation into mixed language support for SPM (pre-pitch). My module contains both Swift and Objective-C sources that are compiled by the Swift and Clang compiler, respectively.

I have figured out the Clang side of things, but could use some help understanding the Swift parts.

When I invoke the swift-frontend to emit the Swift module, I want two things to happen:

  1. A generated Objective-C header ($(ModuleName)–Swift.h) is created so my Swift APIs can be consumed by Objective-C clients. My understanding is that this is what the -emit-objc-header -emit-objc-header-path /path/to/header args are for.
  2. The Swift compiler should know to import the Objective-C half of the module. My understanding is that this is what the -import-underlying-module is for. Here is a great answer that helped me understand what was going on with this flag (cc @jrose in case you have any ideas for this post).

Naturally, I tried adding -emit-objc-header -emit-objc-header-path /path/to/header -import-underlying-module to my swift-frontend invocation in hopes that both goals would be achieved but the error suggests that the -import-underlying-module flag keeps the -emit-objc-header flag from doing its job:

/Users/nickcooke/Developer/MyPackage/.build/x86_64-apple-macosx/debug/MixedPackage.build/module.modulemap:6:12: error: header '/Users/nickcooke/Developer/MyPackage/.build/x86_64-apple-macosx/debug/MixedPackage.build/MixedPackage-Swift.h' not found
    header "/Users/nickcooke/Developer/MyPackage/.build/x86_64-apple-macosx/debug/MixedPackage.build/MixedPackage-Swift.h"
           ^
<unknown>:0: error: could not build Objective-C module 'MixedPackage'

Now, if I remove -import-underlying-module, the $(ModuleName)–Swift.h is created, but the emitted Swift module doesn't import the separately built Clang module.

As a workaround, I can run the Swift compile command twice. One time without the -import-underlying-module flag and then one time with it. When doing that, everything works and I'm able to use my manual mixed language module.

So, my question is: what is the relationship between -import-underlying-module and -emit-objc-header and why does adding -import-underlying-module prevent the generated $(ModuleName)-Swift.h header from being generated? I know it possible to do both in a single Swift compiler command because CocoaPods does so when emitting a Swift module for a mixed language pod.

In case it'd be helpful to see all the emit module command that produces the above error message:

/Applications/Xcode_13.3.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift-frontend 
  -frontend 
  -emit-module 
  -experimental-skip-non-inlinable-function-bodies-without-types 
  /Users/nickcooke/Developer/MixedPackage/Sources/MixedPackage/NewCar.swift 
  -target x86_64-apple-macosx10.13 
  -enable-objc-interop 
  -sdk /Applications/Xcode_13.3.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk 
  -I /Users/nickcooke/Developer/MyPackage/.build/x86_64-apple-macosx/debug 
  -I /Applications/Xcode_13.3.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/lib 
  -F /Applications/Xcode_13.3.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Frameworks 
  -color-diagnostics 
  -enable-testing 
  -g 
  -module-cache-path /Users/nickcooke/Developer/MyPackage/.build/x86_64-apple-macosx/debug/ModuleCache 
  -swift-version 5 
  -Onone 
  -D SWIFT_PACKAGE 
  -D DEBUG 
  -new-driver-path /Applications/Xcode_13.3.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift-driver 
  -resource-dir /Applications/Xcode_13.3.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift 
  -enable-anonymous-context-mangled-names 
  -Xcc -I/Users/nickcooke/Developer/MixedPackage/Sources/MixedPackage
  -module-name MixedPackage 
  -target-sdk-version 12.3 
  -emit-module-doc-path /Users/nickcooke/Developer/MyPackage/.build/x86_64-apple-macosx/debug/MixedPackage.swiftdoc 
  -emit-module-source-info-path /Users/nickcooke/Developer/MyPackage/.build/x86_64-apple-macosx/debug/MixedPackage.swiftsourceinfo 
  -emit-objc-header 
  -emit-objc-header-path /Users/nickcooke/Developer/MyPackage/.build/x86_64-apple-macosx/debug/MixedPackage.build/MixedPackage-Swift.h 
  -import-underlying-module 
  -parse-as-library 
  -o /Users/nickcooke/Developer/MyPackage/.build/x86_64-apple-macosx/debug/MixedPackage.swiftmodule 
  -emit-abi-descriptor-path /Users/nickcooke/Developer/MyPackage/.build/x86_64-apple-macosx/debug/MixedPackage.abi.json 

Thanks!

1 Like

There’s not a direct relation between the two flags, but Xcode does some Hacks to make them work together. When I was at Apple, this involved using Clang’s VFS overlay to substitute a temporary module map for the target being built when compiling its Swift part, so that the generated header wouldn’t be considered an input. There was a bit more to it than that but I’m kind of hoping something’s changed since I left Apple.

2 Likes

Thanks a bunch @jrose! I'll look into the -ivfsoverlay flag and see if I can move forward.

Hopefully someone chimes in with an easier path.

To close the loop here, I was able to get thinks working with a single Swift compile command by creating the following files:

  1. all-product-headers.yaml
  2. unextended-module-overlay.yaml
  3. unextended-module.modulemap (referenced by the above unextended-module-overlay.yaml)

And then adding these args to my swift-frontend command:

swift-frontend /* all the args */ -Xcc -ivfsoverlay -Xcc /path/to/all-product-headers.yaml \
-Xcc -ivfsoverlay -Xcc /path/to/unextended-module-overlay.yaml

In terms of populating the files, I looked at how CocoaPods creates these files for a mixed language pod.

The VFS overlay files seem to follow a straightforward template so programmatically generating them should be feasible.

Thanks again @jrose for the helpful reply!

2 Likes