Converting arrays of convertible types?

[jtbandes (Jacob Bandes-Storch)]

Maybe a poorly phrased title, but here’s the problem I’m interested in solving.

Suppose we have a C API with a function that takes an array of C structs:

struct CValue

func c_api_one(value: CValue) -> Void
func c_api_many(values: UnsafePointer<CValue>, values_len: Int) -> Void

To make the API more ergonomic to use from Swift, let's make a Swift wrapper type, with a method to convert to the C type, such as:

struct SwiftValue {
  var foo: String
  // maybe some other fields...

  mutating func withCValue<T>(_ body: (borrowing CValue) throws -> T) rethrows -> T {
    return try foo.withUTF8 { buf in
      return try body(
        CValue(buf)
      )
    }
  }
}

A wrapper for c_api_one can be implemented easily:

func wrappedApiOne(_ value: SwiftValue) {
  var value = value  // since withCValue is mutating
  value.withCValue { cValue in
    c_api_one(cValue)
  }
}

But, the question is, how do we implement a wrapper for c_api_many?

func wrappedApiMany(_ values: [SwiftValue]) {
   c_api_many(values: ???, values_len: values.count)
}

If it were a single value or a fixed-length array, we could do the following — it's the variable length which makes this tricky.

func wrappedApiMany(actuallyJustOne value: SwiftValue) {
  value.withCValue { cValue in
    withUnsafePointer(to: cValue) { ptr in
      c_api_many(values: ptr, values_len: 1)
    }
  }
}

Looks like a similar/related issue was discussed a couple years ago:

Avoiding stack overflow when nesting String.withCString { ... } (how to handle an arbitrary number of temp values in general)

Hello, In the last days I've been improving the low-level layers of GRDB the SQLite toolkit by asking SQLite to not copy C strings or blob buffers arguments when appropriate (SQLITE_STATIC vs. SQLITE_TRANSIENT). // Avoid a useless copy of the "Arthur\0" buffer by // instructing SQLite to use the temporary one created by Swift let sql = "INSERT INTO player (name) VALUES (?)" let statement = try db.makeStatement(sql: sql) try statement.execute(arguments: ["Arthur"]) To this end, I execute the S…

Some ideas that come to mind:

• Recursion
• Don't use a closure-based "with" style for these conversions at all. In which case...we may no longer be able to use withUTF8 in the implementation for converting string fields?

Any better ideas?

[mattcurtis]

So there's two questions I'm reading here (and correct me if I'm wrong):

The first question is an API design question: how should you make a C API more Swift-friendly and ergonomic? And the second question is: what's the best way to implement wrappers for methods that accept variable-length arguments?

For the API design question: my first observation is that your current proposed API does a lot of (potential) on-the-fly conversion. Before a C method can be called, your Swift value has to do some sort of work before it can call the C API. This is fine if the work is small or low-cost, but if you have to do a lot of work to construct values the C API will accept, then that performance cost will add up.

Truthfully, you may not care much about the performance aspect of this question. Whether or not they'll matter will depend on your usage and needs. But if they do matter, than it's important to think about where you move these costs. Sometimes it's cheaper to, say, construct the ideal C version of a value when you create the Swift value, and keep that around, rather than doing conversion every time a method is called. It all depends on how your API is meant to be used, and what sort of performance concerns you're looking at.

Now for the variable-length argument question. The easiest to implement answer could be to loop over the Swift array you have, and each iteration call the single-argument version of an API. That would work if there isn't much difference between the variable and single argument versions of the methods besides convenience, but if there is a performance difference, then "easy to implement" is not necessarily what you care about.

The Array.withUnsafeBuffer APIs are probably the second-easiest answer, but you wouldn't be able to implement it with your [SwiftValue] type, because the C API doesn't expect an array of SwiftValues. You could construct a buffer on the fly when your Swift version of a C method is called and pass that to C, but the performance penalty could be significant if you're allocating and copying values into that buffer... at which point, a better answer would probably be to create a SwiftValueBuffer/Array type from the start and use that instead.

In hindsight, I'm not sure if this clarifies anything. But I did want to attempt to outline some possible considerations and options. Hopefully it helps!