How to package Swift for consumption by C code

I have a Swift module that I want to make available to C code; in other words, to make a C library that contains some Swift code. Can anyone offer guidance on how to do that?

I imagine that I'll probably be exposing my Swift types as opaque pointers, doing dynamic allocation (or boxing them into classes) and creating @_cdecl functions for the boundary. I'm more curious about how to set up the SPM project structure.

I've run into two problems:

  • the Swift code in the C library needs access to the declarations in the C library's headers, and I don't see a way to do that with SPM other than by .unsafeFlags. With CMake of course that's no problem
  • I don't seem to be able to import the C library into a Swift test. If I do that (uncomment the test target here), I get scads of errors that look like this:
.../TemporaryDirectory.J6j0QN/LotsawaC.pch:3:21: error: source file is not valid UTF-8
   1 | # 1 "<built-in>"

   2 | # 1 "/var/folders/wg/wvvfbgl96f727hzqdtpfrrl80000gn/T/TemporaryDirectory.J6j0QN/LotsawaC.pch"

      1 | # 1 "<built-in>"

   2 | # 1 "/var/folders/wg/wvvfbgl96f727hzqdtpfrrl80000gn/T/TemporaryDirectory.J6j0QN/LotsawaC.pch"

   3 | ����  <more nonsense>

In other words, the .pch file starts with valid text, but then it appears to be binary. Which might be just fine, but then SPM is cosuming it incorrectly.

Thanks in advance,
Dave

2 Likes

I think SwiftPM is not really capable of doing this in a way you're going to find comfortable. SwiftPM doesn't really supported mixed-language targets as a concept, so you can only have the dependency edge pointing in one direction: either Swift depends on C or vice-versa.

It's reasonable to call this a SwiftPM limitation that could be lifted, but for now I'd suggest using CMake.

3 Likes

It's reasonable to call this a SwiftPM limitation that could be lifted,

Better support for creating C libraries with Swift · Issue #8228 · swiftlang/swift-package-manager · GitHub.

but for now I'd suggest using CMake.

Would that solve the 2nd problem somehow?

I believe it should, yes.

@dabrahams Unless I'm misunderstanding the question, SPM is actually capable of doing what you want.

Here's a simple demo project that shows how to "export" a Swift API to C: GitHub - lemonmojo/SwiftCExportTest: Demo project showing how Swift Package Manager (SPM) can be used to write C libraries in Swift

It also uses types (an enum and a typedef) from a C header in Swift. Additionally, it shows how to declare and implement parts of the API in C as well.

So unless I'm missing something, I would argue that it's very much possible to achieve what you want using just SPM.

2 Likes

@lemonmojo this is awesome, thanks!

It seems like, especially because experts like @lukasa don't know how to do it, we need some kind of tutorial, especially on how/why to use all the magic @_... attributes (which isn't explained in the code).

1 Like

I quickly cobbled this together today to also document it for myself but didn't yet get to adding comments or a proper readme. I basically extracted the important parts from a large project I'm working on which isn't open source (yet).
So yes, I agree, there needs to be more documentation around this but at least we have this repo now and I plan on adding some more docs to it. Open to PRs as well of course! :wink:

4 Likes

Would love to but if I knew the answers I wouldn't have needed your help!

1 Like

Added a bunch of comments now: Add comments · lemonmojo/SwiftCExportTest@e95e498 · GitHub
Let me know if this is sufficient or if I should expand on some of the details.

2 Likes

If interop ever reaches the point that we can package C++20 modules, SPM will have become the first decent package manager for C++.

Edit: Is there a reason you used @_implementationOnly instead of internal import?

1 Like

Yes, internal import didn't exist when I originally wrote that code. :wink: Worth a shot now to try it instead of @_implementationOnly.

Ah, I should clarify that I did know SwiftPM could do what is demo'd here, I just didn't want to recommend it because it's extremely limited, and has some sharp edges. It's definitely possible I didn't sufficiently understand what you were looking for here, and led you astray, but let me lay out my concerns with this approach.

The first limitation is that CDefinitions is doing two jobs. It's defining common types, but it's also defining the API for the Swift target.

This is not a huge problem, because that could always be factored out into two targets, but that takes us to our next problem. This is that the C code has to remember to depend on both the Swift module and its interface module, which cannot be the same. Otherwise the C code will happily compile but fail to link, or fail to compile despite apparently having the relevant dependency edge.

The natural way to try to fix that is to make the Swift module a dependency of the C module that defines the interface of the Swift code. That can work too, and this is a bit nicer, but then we hit what I think is the main limitation, which is that @_cdecl is horrible.

Specifically, there is no way in this pattern to confirm that the definition written in the C header file has any correspondence with the declarations in the Swift module. For small amounts of code this is just about tenable, but for a large interface, particularly one that involves callbacks and other recursive patterns, this can get truly awful really quickly. This can lead to horrible memory safety bugs caused by argument number mismatches and other nastiness.

I know @Douglas_Gregor has had thoughts about this problem in the past, and he might have more to illuminate. In my view the big thing that will need to happen to make this usable is that building the Swift module must be able to emit a header file that matches the C declarations, or to validate in the other direction.

6 Likes

@lukasa Yup, I agree with everything you said. We're in fact using this approach in a much larger API surface, including mapping Swift delegates to collections of C function pointers. It's barely tenable in this project because the API changes very rarely - it's mostly additive.

Fun fact: The project I'm referring to is a high level wrapper for a third party C library. So we're putting a nice Swift API on top of existing C code and then go the extra mile by surfacing a simple C API for consumption by other languages (in our case C#).

And to top it all off, it works on all of Apple's platforms, Linux and Windows.

So it's pretty wild and I'm very happy that it works at all but I would love to see something akin to @objc for plain old C which also takes care of generating header files.

In fact, some of the pieces are already in place; -emit-objc-header for a few versions has emitted C function signatures for functions annotated with @_cdecl. But it still assumes/forces a ton of Obj-C-isms. This code,

@_cdecl("foobar")
public func f(x: Int) -> Int { x }

generates the usual boilerplate of Obj-C/Foundation-related preprocessor macros, and then this for the actual decl:

#if defined(__OBJC__)

SWIFT_EXTERN NSInteger foobar(NSInteger x) SWIFT_NOEXCEPT SWIFT_WARN_UNUSED_RESULT;

#endif

So the header still can't even be used by clients not compiling in Objective-C mode (and it uses Foundation's NSInteger for Swift Int instead of something like intptr_t).

In the absence of a more official solution right now, it would be great to at least make the header usable for folks who aren't going to be using it with Obj-C consumers (whether that's on Linux or even on Darwin).

2 Likes