Better C interoperability e.g. with apinotes

Using Swift
,
1

I'm following @Douglas_Gregor s excellent guide Improving the usability of C libraries in Swift | Swift.org to improve how I interop with libsecp256k1 in my K1 (non-trivial Swift wrapper around libsecp256k1 with API's like CryptoKit).

libsecp256k1 has many functions which takes mutable bytes unsigned char * and immutable bytes const unsigned char *, these gets translated into UnsafeMutablePointer<UInt8>! and UnsafePointer<UInt8>! respectively.

Following Doug's guide I've been able to insert labels and remove the nullability (Nullability: N).

However I have not found a way to rid of the UnsafeMutablePointer<UInt8>/UnsafePointer<UInt8> - I want API's using Span and MutableSpan (at least I think...).

Is that somehow possible using apinotes? Or does it require "manual wrapping"?

If manual wrapping would work, is it possible to do it in C land and some annotation (which is not available in apinotes)? Or need I do it in Swift land? Do I need to do the withUnsafeMutableBufferPointer / withUnsafeMutableBytes dance?

I could have sworn I had read something about Span improving C interop, automagically translating from UnsafeMutablePointer to MutableSpan... but maybe I've dreamt this? Or maybe it is mentioned somewhere (link please!) but just not implemented?

Thanks and happy Friday!

/Alex

1 Like
2

Maybe "Annotating Pointers with Bounds Annotations" paragraph could help Safely Mixing Swift and C/C++ | Swift.org

3

interesting, and we can also use ApiNotes to set __counted_by or __sized_by

    BoundsSafety:
      Kind: sized_by
      BoundedBy:"len"

Actually I forgot to say that lots of functions expect fixed size byte arrays of eg length 32, so the best Swift APIs for those would be InlineArray<4, UInt8>. Is it possible to map an unsigned char * (without any other C arg for length) in C to InlineArray? That would be very neat!

4

It's still behind an experimental flag (SafeInteropWrappers) while we finalize the details, and the results may change. With that experimental flag, you also need to provide both bounds (via __counted_by or the API notes equivalent) and lifetime information (via the noescape or lifetimebound attribute or API notes equivalent) to map pointer + length into a Span or MutableSpan.

  • Doug
2 Likes
5

Nice! I got it working:

C code

void fill_with_fives(
	unsigned char *buf,
	int len
) {
	memset(buf, 5, len);
}

.apinotes file:

- Name: fill_with_fives
  SwiftName: fillWithFives(span:)
  Parameters:
  - Position: 0 # buf
    Lifetimebound: true
    BoundsSafety:
      Kind: counted_by
      BoundedBy: "len"
    Nullability: N
  - Position: 1 # len
    Nullability: N

Working Swift test (I had missed the experimental feature flag, I should have read better)

var bytes: InlineArray<7, UInt8> = .init(repeating: 0)

var span = bytes.mutableSpan
fillWithFives(span: &span)
for i in 0..<7 {
	#expect(bytes[i] == 5)
}

The var span = bytes.mutableSpan dance is a little bit unfortunate - naively I want to do:

fillWithFives(span: &bytes.mutableSpan)

which does not compile.

Another thing to notice, is that I lacked the ability to do:

+let count: Int = 7
var bytes: InlineArray<count, UInt8> = .init(repeating: 0)

var span = bytes.mutableSpan
fillWithFives(span: &span)
-for i in 0..<7 {
+for i in 0..<count {
	#expect(bytes[i] == 5)
}

since those 7 should match! Will that be supported in future version of Swift?

1 Like
6

Yes, this is a known limitation right now.

Also a known limitation, until we get a model for constant values in the language.

Doug

1 Like
7

If it helps, you can use the count from the InlineArray type parameter:

func f<let C: Int>(bytes: InlineArray<C, UInt8>) {
  for i in 0..<bytes.count {
    // ...
  }
}
1 Like
8

Yes that's very close to what I ended up doing:


@available(macOS 26.0, *)
func allEqual<let C: Int, Element: Equatable>(
	to expectedElement: Element,
	defaultElement: Element,
	initBytes: (inout InlineArray<C, Element>) -> Void
) -> Bool {
	var bytes: InlineArray<C, Element> = .init(repeating: defaultElement)
	initBytes(&bytes)
	for index in 0..<bytes.count {
		guard bytes[index] == expectedElement else {
			return false
		}
	}
	return true
}

@available(macOS 26.0, *)
func expectAllEqual<let C: Int, Element: AdditiveArithmetic>(
	to expectedElement: Element,
	initBytes: @escaping (inout InlineArray<C, Element>) -> Void,
	file: StaticString = #filePath,
	line: UInt = #line
) {
	#expect(
		allEqual(
			to: expectedElement,
			defaultElement: .zero,
			initBytes: initBytes
		),
		"in file: \(file), line: \(line)"
	)
}

And usage:

@available(macOS 26.0, *)
@Test
func `C get Swiftified by apinotes file with non-nullability and MutableSpan`() {

	expectAllEqual(to: 5) { (bytes: inout InlineArray<3, UInt8>) in
		var span = bytes.mutableSpan
		fillWithFives(span: &span)
	}

}

(btw, what is the correct usage of file/line in Swift testing, there #expect macro did not take any such args, so I passed them as Comment type (which is ExpressibleByStringLiteral))

9

Any plans on supporting fixed size arrays, translating from unsigned char * in C to InlineArray<LEN, UInt8> in Swift?

Given this C function:

void clone_buf_of_len_three(
    unsigned char *destination,
    const unsigned char *source
) {
	memcpy(destination, source, 3);
}

I naively tried this - non working - apinotes:

- Name: clone_buf_of_len_three
  SwiftName: cloneSpanOfLength3Unsafe(into:from:)
  Parameters:
  - Position: 0 # destination
    Lifetimebound: true
    BoundsSafety:
      Kind: counted_by
      BoundedBy: 3
    Nullability: N
  - Position: 1 # source
    Lifetimebound: true
    BoundsSafety:
      Kind: counted_by
      BoundedBy: 3
    Nullability: N

which translates into Swift:

func cloneSpanOfLength3Unsafe(
    into destination: UnsafeMutablePointer<UInt8>,
    from source: UnsafePointer<UInt8>
)

And I have to manually wrap it in Swift to:

@available(macOS 26.0, *)
func cloneSpanOfLength3(
	into destination: inout InlineArray<3, UInt8>,
	from source: InlineArray<3, UInt8>
) {

	var span = destination.mutableSpan
	span.withUnsafeMutableBufferPointer { dst in
		source.span.withUnsafeBufferPointer { src in
				cloneSpanOfLength3Unsafe(
					into: dst.baseAddress!,
					from: src.baseAddress!
				)
		}
	}
}

libsecp256k1 is full of functions with fixed size, so it would be excellent to get it to translate into InlineArray<L, UInt8>, where L is documented according to the header of libsecp256k1 (often times 32 bytes for SHA256 hashes and ECC keys).

@Douglas_Gregor are there any plans to support this translation using apinotes?

Thanks!