Where to place custom module.modulemap for third party C library without touching their source code

Development Package Manager
1

Hello World!

I'm wrapping couple of upstream C libraries (XXX in the example below) into swift modules and all works great with the following setup:

Package.swift
Sources/XXX
Sources/XXXSwift/XXXSwift.swift

where XXX is a git submodule of third party C library cloned into my swift module's repo from upstream git sources. This way I can easily track new releases from upstream without touching their code.

In the meantime such packages can be then turned via swift package generate-xcodeproj into projects that can be used as subproject in Xcode. When C library XXX contains either include/XXX.h or include/XXX/XXX.h everything is fine and working great.

But I have one C library that just publishes a couple of XXX/include/YYY.h files defining various features, without defining one big umbrella file, mainly because the library is intended to be used piece by piece by including only what is necessary. The swift package manager (I think) is not finding the umbrella header and therefore is not creating a clang module (I think) and Xcode later fails to compile swift sources that need access to the C code with the error:

error: no such module 'XXX'
import XXX

I know I can workaround this by creating my own module.modulemap for XXX defining what header files should be included, but I struggle to find the right place where to put it.

I know the module.modulemap should be placed in XXX/include/ but given that XXX/include is cloned from a third party git I do not necessarily have a right to put it there without forking XXX into my own repo and adding necessary modifications.

What's the best practice to swift-ify a third party C code without upstream modifications these days?

Where does SPM look for module.modulemap for XXX?

sorry if I'm missing something obvious and thanks for your help,
Martin

2

SwiftPM always generates a modulemap for C libraries, even if they don't have an umbrella header. Try to find the generated modulemap in the .build directory. One possibility is that the module name generated for XXX could be normalized by SwiftPM. The import statement must mention the module name defined in the modulemap.

I've found symlinks to the files inside the submodule directory to work pretty well. The downside is that you have to create them by hand but you can automate that using some script.

3

I have looked into .build and this is what I found:

module xxx {
    umbrella "/Users/mman/Projects/xxx-swift/Sources/xxx/include"
    export *
}

So this looks like it's specifying the whole include dir as umbrella, is that correct?

4

Right. You should be able to do import xxx. Maybe you didn't setup the target dependency?

5

Nope. Did try it again and after swift generate-xcodeproj and dragging the xcodeproj as a subproject and declaring a dependency the xxx itself builds fine but can not be then used in swift files being compiled by Xcode. New build system.

Interestingly enough it works when I use just SPM. So Package A depends on B depends on XXXSwift and swift files in package A can do import 'xxx' but when the same is done in Xcode it does not work.

And one more. It does work from both Xcode and SPM when the package contains the umbrella header, just not in umbrella dir.

Where else could I look?

6

Ah. Yeah, the subprojects won't work without using umbrella headers. This has to do with how "real" frameworks can't include non-modular headers. I suggest creating an umbrella header using the symlink trick.

7

Just one more investigation update.

I have examined all messages of full Xcode rebuild and indeed a working Xcode subproject generated from SPM has a "Copy module.modulemap" step where module.modulemap is in derived files directory, was autogenerated, and includes the umbrella header statement.

For the non working subproject there is no "Copy module.modulemap" step and derived files directory does not contain any module.modulemap. Xcode did not generate one.

So from what I could see, there is an inconsistency:

The swift package manager when using swift build will generate a module.modulemap pointing umbrella to include/ subdirectory.

Whereas Xcode will not do anything when there is no single file umbrella header.

Is my assessment correct? Does it make sense to file this as a bug? If yes then against Xcode or against SPM?

I will continue working around by symlinking or copying my custom module.modulemap around.

Thanks,
Martin

8

Your assessment is correct but this is a feature not a bug :slight_smile:. I really need to writeup some documentation to explain these behaviors and reasons behind them.

See: [Xcodeproj] Use Xcode's modulemap generation when umbrella header is … by aciidb0mb3r · Pull Request #1602 · apple/swift-package-manager · GitHub

9

Yes please do, I will love to learn the details.

I think what got me confused was that staying in SPM land things work as expected, in other words Package A that depends on Package B that depends on Package XXXSwift that contains a pure C target XXX can in fact import XXX but the same does not work in Xcode.

Is it because Xcode uses frameworks and SPM uses dylibs?

Thanks
Martin

10

What you're describing will work in Xcode as well if you generate an Xcode project directly in A. The problem occurs if you try to embed a generated project in another project. SwiftPM and Xcode assume that targets that don't have an umbrella header are non-modular and thus are not real frameworks. The targets that depend on such non-modular frameworks need to add a search path in order to locate the modulemap. When the Xcode project is generated by SwiftPM, it adds the required search paths to all of the dependees but when you embed that project as a subproject, things break because of lack of this search path in the top-level Xcode project.

2 Likes