C interoperability, combinations of library and OS versions

Using Swift
,
1

Hello Swift community, I'm looking for advice.

I want to write Swift code that adapts to the various versions of SQLite found in various iOS/macOS versions, as well as it adapts to custom SQLite builds (latest release + SQLCipher).

By "adapt", I mean:

  1. At compile-time: expose or not Swift APIs depending on the underlying SQLite version.
  2. At compile-time: branch depending on the underlying SQLite version.
  3. At run-time: branch depending on the underlying SQLite version.

Precisely speaking, I already know how to do this. In the schematic sample code below, we have four functions f1, f2, f3, and f4.

  • f1 is always available for custom builds, but only available on some iOS/macOS version
  • f2 is always available, but branches depending on the iOS/macOS version
  • f3 is available for custom builds with specific SQLite compilation option, and some iOS/macOS versions
  • f4 is always available, but branches depending on the SQLite version
#if CUSTOM_SQLITE_BUILD // Set in OTHER_SWIFT_FLAGS
// Using custom SQLite build

func f1() { 🐵 }

func f2() { 🐶 }

#if SQLITE_ENABLE_FTS5 // Set in OTHER_SWIFT_FLAGS
func f3() { 🦊 }
#endif

#else
// Using SQLite shipped with operating-system

@available(...)
func f1() { 🐵 }

func f2() {
    if #available(...) { 🐶 }
    else { 🦁 }
}

@available(...)
func f3() { 🦊 }

#endif

func f4() {
    if sqlite3_libversion_number() < ... { 🐷 }
    else { 🐰 }
}

So it is totally possible, and it indeed works today. But there are problems:

  • f1 and f2 are declared twice.
  • codes chunks :monkey_face:, :dog: and :fox_face: are duplicated.
  • f2 and f4 are pursuing the same goal. f2 uses compile-time checks, and f4 uses run-time checks. Compile-time checks are supposed to be better, but they create code duplication. So run-time checks are sometimes preferred - for no clear reason.

With C/Objective-C, which have access to C #define, all of those problems are solved:

// C code

#if SQLITE_VERSION_NUMBER > ...
void f1() { 🐵 }
#endif

void f2() {
    #if SQLITE_VERSION_NUMBER > ...
        🐶
    #else
        🦁
    #endif
}

#if SQLITE_ENABLE_FTS5
void f3() { 🦊 }
#endif

void f4() {
    #if SQLITE_VERSION_NUMBER > ...
        🐷
    #else
        🐰
    #endif
}

But I'm not here to complain about the lack of C #define.

I come to you today because until now, I could deal with code duplication (worst examples so far one, two).

But recently a user came with a reasonable request that would duplicate whole files (all files starting with FTS5 in this directory).

Now this becomes serious. This message has three goals:

  • Listen to good advice
  • Raise the awareness that consuming C APIs is not always easy.
  • Maybe get in touch with an available core-team member so that we could discuss in details an eventual solution.

Thanks for reading!

2 Likes
2

Hm. Your simplified version isn't quite correct, because SQLITE_VERSION_NUMBER is (hypothetically) resolved at compile time but @available can be paired with #available. That's a big difference for all of them except f2.

The problem with depending on C macros is it means you can't ship your library as a self-contained unit. That's not supported today, sure, but we don't want to add features to the language that make it harder. You really are choosing whether to build with a bundled SQLite—with a particular configuration—or with the system one, and unlike search paths or imports this is a choice that affects clients of your library as well.

That said, there are still some interesting things that could be done:

  • Allow #if around attributes (something that's been asked for for plenty of other reasons).
  • Some way to get configuration options en masse, rather than having to put each one on the command line. Maybe even from reading C headers (but done more explicitly than with import). I don't know what this looks like.
3

Hello, Jordan, thanks

Your simplified version isn't quite correct, because SQLITE_VERSION_NUMBER is (hypothetically) resolved at compile time but @available can be paired with #available. That's a big difference for all of them except f2.

I'm sorry, I do not understand. The "simplified version" is pseudo-C, which can access C macros, and thus avoid code duplication.

To focus on a single example, can the following code be simplified in order to avoid code duplication?

#if CUSTOM_SQLITE_BUILD // Set in OTHER_SWIFT_FLAGS
func f1() { 🐵 }
#else
@available(...)
func f1() { 🐵 }
#endif

The problem with depending on C macros is it means you can't ship your library as a self-contained unit. That's not supported today, sure, but we don't want to add features to the language that make it harder. You really are choosing whether to build with a bundled SQLite—with a particular configuration—or with the system one, and unlike search paths or imports this is a choice that affects clients of your library as well.

This is true. GRDB ships with three flavors:

  • GRDB, available on CocoaPods and SPM, links against the system SQLite
  • GRDBCipher, available on CocoaPods, links against SQLCipher
  • GRDBCustomSQLite, available through manual Xcode integration, links against a customized SQLite build

Those are all three independent targets and frameworks, which share the same code, though. Their behaviors and public apis depend on OTHER_SWIFT_FLAGS and availability checks.

In particular, GRDBCustomSQLite doesn't depend on C macros (it can't): instead, a remarquable bridge named swiftlyfalling/SQLiteLib allows the host library (GRDB) and even the application to know about the enabled SQLite options. This is how the Swift apis for FTS5 (the latest full-text search engine) are enabled today:

#if SQLITE_ENABLE_FTS5 // Set in OTHER_SWIFT_FLAGS by swiftlyfalling/SQLiteLib
...
#endif

It's hard to imagine the amount of energy which is spent in order to achieve our goals, isn't it? :sweat_smile:

Now FTS5 is enabled by default in macOS 10.13. It would be nice to expose it from the regular GRDB framework, when available. But since C macros are not exposed, the code has to perform availability checks:

#if CUSTOM_SQLITE_BUILD
    // Custom SQLite
    #if SQLITE_ENABLE_FTS5 // Set in OTHER_SWIFT_FLAGS by swiftlyfalling/SQLiteLib
    /* duplicated FTS5 code */
    #endif
#else
    // Stock SQLite
    @available(...)
    /* duplicated FTS5 code */
#endif

This is what I'm somewhat reluctant to do now, because of the huge amount of duplicated code. Meanwhile, FTS5 is still not activated for the library users.

That said, there are still some interesting things that could be done:

  • Allow if around attributes (something that's been asked for for plenty of other reasons).

Yes, this would help a lot:

#if !CUSTOM_SQLITE_BUILD || SQLITE_ENABLE_FTS5
    #if !CUSTOM_SQLITE_BUILD
    @available(...)
    #endif
    /* non-duplicated FTS5 code */
#endif
  • Some way to get configuration options en masse, rather than having to put each one on the command line. Maybe even from reading C headers (but done more explicitly than with import). I don't know what this looks like.

Me neither. But some configuration flags depend the operating system: at 1st sight, allowing if around @available looks more promising.

4

Meanwhile, another silly workaround you may not have thought of: function pointers:

#if CUSTOM_SQLITE_BUILD
func f1() {
  f1Impl(sqlite_foo) // the custom one
}
#else
@available(…)
func f1() {
  f1Impl(sqlite_foo) // the Apple one
}
#endif

private func f1Impl(_ sqlite_foo: @convention(c) (UnsafePointer<CChar>) -> Void) {
  sqlite_foo(🐵)
}

This has its own set of drawbacks, of course, but it can sometimes cut down on code duplication, and the extra function should get optimized away.

4 Likes
5

That's a super tip :+1: !!!