Writing portable C library in SwiftPM question

Using Swift
, ,
1

I have a C library backing the Swift framework. When I attempt to port it to Linux and Windows, the compiler raises an error for the absence of the CoreFoundation header.

#ifndef DemoFlags_h
#define DemoFlags_h
#include <CoreFoundation/CoreFoundation.h>
typedef CF_OPTIONS(uint32_t, DemoFlags) {
    DemoFlags_0 = 0,
    DemoFlags_1 = 1,
};

CF_EXPORT
void greeting(void);
#endif /* DemoFlags_h */

Question 1:

What is the recommended approach for writing a portable C library in SwiftPM?

It will involve extensive utilization of the __attribute(swift) macro.

Moreover, I would prefer not to write a separate header wrapper for this. The extensive use of CF macros aligns perfectly with my requirements.

eg. CF_EXPORT CF_OPTION CF_ASSUME_NONNULL_BEGIN CF_ASSUME_NONNULL_END

I don't think it's necessary to repeat the invention of the wheel on Linux and Windows.

Question 2:

If the first question is by using #include <CoreFoundation/CoreFoundation.h>, then how should we include them in our header code?

Initially, I attempted to create a SwiftPM package - CoreFoundation, copying all the header files and making necessary modifications to facilitate compilation.

However, this resulted in a conflict due to the redefinition of the module.
xx/toolchain/swift-5.9/usr/lib/swift/CoreFoundation/module.map:1:8: error: redefinition of module 'CoreFoundation'

Subsequently, I attempted to directly include it by adding an unsafeFlags entry to the target configuration .unsafeFlags(["-I", "xx/toolchain/swift-5.9/usr/lib/swift/"],.when(platforms: [.linux])),
This, however, still led to a multitude of errors.

toolchain/swift-5.9/usr/lib/swift/CoreFoundation/ForSwiftFoundationOnly.h:93:20: error: declaration of '__CFAllocatorRespectsHintZeroWhenAllocating' has a different language linkage
2

CoreFoundation is an implementation detail of swift-corelibs-foundation. You should not be using CoreFoundation on non-Darwin platforms. Even if some CoreFoundation headers are currently available on non-Darwin platforms, those are private and there's no guarantee they will be available or not change in a breaking manner in the future.

3

I agree we should not depend on it if it is an implementationOnly detail of Foundation.

So how should we build cross-platform C library while providing great Swift API in SwiftPM?

A manually code would lead us to use a lot of __attribute directly. And we may need to write if else macros for a lot of different OS and different arch. This is almost a repeat of the wheels that CoreFoundation has already provided. :smiling_face_with_tear:

4

Is there anything SwiftPM-specific here that wouldn't apply to any C library in abstract? By "build" here do you mean the build system specifically, or writing the library itself in a portable way? Do you already have a C library that utilizes Darwin APIs and want to port it to non-Darwin platform?

Does the library have to be written in C? Why not write it in Swift and expose the C API to it with @_cdecl and corresponding headers? Or expose a C++ API via C++ interop?

I feel like some context here is missing that would clarify your constraints.

5

Is there anything SwiftPM-specific here that wouldn't apply to any C library in abstract?

  1. No. It's not SwiftPM-specific.It's because the C and Swift Package are mostly provided via SwiftPM in Linux.

By "build" here do you mean the build system specifically, or writing the library itself in a portable way?

  1. I mean writing the library itself in a portable way. I do not think I should duplicate such macro in every portable C library I wrote.
    https://github.com/apple/swift-corelibs-foundation/blob/8a9b69b5041b5069360239cd18304fd8e2eb91d9/CoreFoundation/Base.subproj/CFBase.h#L121-L149

Do you already have a C library that utilizes Darwin APIs and want to port it to non-Darwin platform?

  1. I do not depend on specified Darwin Only API(eg. CFLock CFString). Mostly depend on the CF cross-platform C macro(CFOption CFExport).

Does the library have to be written in C? Why not write it in Swift and expose the C API to it with @_cdecl and corresponding headers? Or expose a C++ API via C++ interop?

  1. Some part of the code is very low level. There is some limitation in the current Swift version to do it right.

Formalizing the unavailability of Core Foundation - #12 by Max_Desiatov

  1. Deep sorry for this interruption. I forgot to check the last active time of the past. By the way, can we consider automatically locking posts that have not received a new reply for more than a period of time(eg. 1 year)?
6

Would you be able to clarify what those limitations are? I'd much rather see efforts invested in addressing those limitations than see CoreFoundation as a public API on non-Darwin paltforms.

1 Like
7

I'll spend some time to try it convert the existing C code directly in Swift and summary back the limitations later in this thread.

For now, I’ll just duplicate the CF macro and use __attribute directly in my package.

Thanks for your explanation here.

1 Like
8

For example:

This is a tiny C package I implement in a SwiftPM package which compiles fine on Linux & macOS:

// counter.h
/*CF_EXPORT*/
long long getCounterValue(void);

// counter.c
#include "counter.h"
#include <stdatomic.h>
long long getCounterValue(void) {
    static atomic_llong counter = 1;
    return counter++;
}
.
├── Package.swift
└── Sources
    └── A
        ├── counter.c
        └── include
            └── counter.h

How would you write such code in Swift. Forgive me if the problem is silly for you.

The main problem is that I do not know how to import <stdatomic.h> in Swift. I could add a C shim(module _A) which provide a modulemap and module header and add
#include <stdatomic.h> in the module header.

  1. IMO It seems it is still inevitable to create C Shim. So I will simply use C to complete the whole thing.
  2. The shim will also to be accessible by other C package. So if I write the implementation in Swift, it will add another layer of complexity to expose to C.

Many thanks if you can point out to me how would you implement such code in Swift and expose it to C.

9

Is there a reason you couldn't use the Swift Atomics library instead? Do you have to expose the atomics API to C? Why not expose a higher-level API from Swift to C that doesn't have to deal with atomics?

10

I know that the best practice is to use the "standard library" agreed by the community. Just as we won't build wheels to invent our own CustomString. Instead we use Swift.String.

But things is different here.

  1. Atomics is a "large" library compared to our current tiny C implementation. We only want a single small function here - generate a increased id atomicly.
  2. Not every feature we implemented and will implement later in C has a corresponding official Swift library. In such case we will still face the issue I met in my last post. The issue is not specified for atomic stuff.

Once again. Is there any method available that would allow us to effortlessly convert such C code to Swift?

// Pseudocode
import Darwin

private static var counter = _CAtomic(1)
@cdel
public func getCounterValue() -> Int {
    defer { counter += 1 }
    return counter
}
11

If you need a cross-platform solution and you're not satisfied with the currently available Swift APIs, why not use std::atomic?

12

The problem is not about atomic. But any C header of the existing Unix system(macOS/Linux/Windows). We can easy include and use them in C but not that easy in Swift code if it was not a clang module.

13

I'm not sure I understand the problem well as described, maybe you have a sample project to illustrate this with more context? Is there anything that prevents you from adding Clang modulemaps to your C libraries?

14

Ended up forking CoreFoundation with modification and maintaining a OpenFoundation library myself.

The migration is then simply a regex replacement: "CF" -> "OF".

99% of code works just fine on Linux. Only one issue is found:
The OF_OPTIONS will cause some issue on Linux platform. Will look into it when I had time.

#include <OpenFoundation/OpenFoundation.h>

typedef OF_OPTIONS(uint32_t, Flags) {
    FlagsA = 0,
    FlagsB = 1,
};

error: non-defining declaration of enumeration with a fixed underlying type is only permitted as a standalone declaration; missing list of enumerators?
typedef OF_OPTIONS(uint32_t, Flags) {
        ^
OpenFoundation/Sources/OpenFoundation/include/OpenFoundation/OFAvailability.h:32:69: note: expanded from macro 'OF_OPTIONS'
#define OF_OPTIONS(_type, _name) enum __OF_OPTIONS_ATTRIBUTES _name : _type _name; enum _name : _type
15

Is there anything that prevents you from adding Clang modulemaps to your C libraries?

No. We could do it technically. Just decide to keep our C library for some history and preference reason now.

16

Max is suggesting we should not reply on old thread. So I'll reply your post on this thread.

Formalizing the unavailability of Core Foundation - #13 by compnerd

Yes. So I just create a CoreFoundation fork and try to maintain a cross-platform solution myself.

After all, what I need is just the handy macros of CF not the implementations of the .c file.

#if !defined(CF_EXTERN_C_BEGIN)
#if defined(__cplusplus)
#define CF_EXTERN_C_BEGIN extern "C" {
#define CF_EXTERN_C_END   }
#else
#define CF_EXTERN_C_BEGIN
#define CF_EXTERN_C_END
#endif
#endif

#if TARGET_OS_WIN32
    #if defined(__cplusplus)
        #define _CF_EXTERN extern "C"
    #else
        #define _CF_EXTERN extern
    #endif

    #if defined(_WINDLL)
        #if defined(CoreFoundation_EXPORTS) || defined(CF_BUILDING_CF)
            #define CF_EXPORT _CF_EXTERN __declspec(dllexport)
        #else
            #define CF_EXPORT _CF_EXTERN __declspec(dllimport)
        #endif
    #else
        #define CF_EXPORT _CF_EXTERN
    #endif
#else
#define CF_EXPORT extern
#endif

cc @compnerd

17

I would recommend against that. These are namespaced to CF which means that the official definition should be whatever CoreFoundation decides to use that day.

More importantly, the CF_EXPORT is fundamentally non-portable. This must be defined per module as you cannot reuse the macro across two libraries simultaneously. The __declspec(dllexport) should only be applied to declarations that are ABI for the module, and __declspec(dllimport) should be attributed to declarations that are being provided by a different DLL for the module. If you have two DLLs that are being built and share a header the wrong attribution is applied.

The only bit that is portable is extern "C" { and } which IMO is far from useful.

1 Like
18

These are namespaced to CF which means that the official definition should be whatever CoreFoundation decides to use that day.

So I just renamed them to OF. LOL

More importantly, the CF_EXPORT is fundamentally non-portable.

The example of "OF_EXPORT" may not be appropriate. Let's just see another macro I need. It hides the detail of the underlining attribute magic and is portable to compiler where swift_name is not available.

#if __has_attribute(swift_name)
# define OF_SWIFT_NAME(_name) __attribute__((swift_name(#_name)))
#else
# define OF_SWIFT_NAME(_name)
#endif
19

You can use what I've done for myself here: https://github.com/smumriak/AppKid/blob/main/CCore/include/CCore.h
It goes a little bit beyond EXPORT and EXTERN macro ;)

1 Like
20

Also CF Is essentially dead. Foundation is on it's way to the ground too. There are some things that I would like to exist for myself in future tho, so I'm preserving them (ideologically! not copying code) in https://github.com/smumriak/AppKid/tree/main/TinyFoundation/Sources/TinyFoundation/Threading and https://github.com/smumriak/AppKid/tree/main/TinyFoundation/Sources/TinyFoundation/Experiments/RunLoop
If you want to have a truly portable C library you would not have any CF dependencies anyway ;)

1 Like