C struct in a Swift struct has an unstable address?!

Using Swift
1

I’m losing my mind trying to call zlib from Swift. I’ve ported some C++ code of mine that’s known to work, and I think I have a decent grasp of the twisty maze of unsafe-pointer types, but I’m getting errors back from every call to deflate that basically mean “your context struct is bad”.

After breaking on the entry points of deflateInit2_ and deflate and looking at the registers, I find that the z_stream pointer in $x0 is not the same from call to call. The struct contents at the address are correct, but it’s as though the struct is being copied. I strongly suspect that zlib does not like its struct being moved around and that’s the reason for the error.

Here I should point out that the C z_stream is a member of my Swift struct, which is ~Copyable. So that should prevent Swift from making copies, right?

import zlib
struct ZlibStream: ~Copyable {   // highly simplified
  var stream = z_stream()
  init() { deflatInit2_(&stream, ...) }
}

Moreover, I can see this in Swift by printing the address of the stream member:

withUnsafePointer(to: self.stream) { zPtr in
    print("stream is at \(zPtr)")
}

When I call this at different times I get three different results:

  • In the struct’s init method
  • In the function that created my struct
  • When my struct makes a call to deflate

Is there something deeply weird about having a C struct as a member of a Swift struct? Is Swift somehow forced to make a temporary copy of the struct every time it references it?

By comparison, I’ve found code nearly identical to mine in the swift-gzip library. The only difference is that the z_stream is a local variable of a function, not a member of a struct. I can’t do this, though, because I have to make multiple calls to the stream across async events.

(Platform details: Swift 6.3, Xcode 26.4, Apple Silicon MBP.)

2

If you want a stable address, I believe you need to allocate the memory yourself, eg. with an unsafe pointer.

You can find a lot of discussion about this in regards to implementing locks such as mutex in Swift.

I don’t know much about non-copyable types, but I recall that early on they were called “move-only” types.

1 Like
3

The immutable version of withUnsafePointer was added in SE-0205, which states:

We do not propose removing the inout -taking forms of withUnsafePointer and withUnsafeBytes , both for source compatibility, and because in Swift today, using inout is still the only reliably way to assert exclusive access to storage and suppress implicit copying. Optimized Swift code ought to in principle avoid copying when withUnsafePointer(to: x) is passed a stored let property x that is known to be immutable, and with moveonly types it would be possible to statically guarantee this, but in Swift today this cannot be guaranteed.

That, of course, was written before Swift had non-copyable types (Swift 4.2). Based on that paragraph, theoretically since stream is a member of a ~Copyable type, I would think this would be fine and the function wouldn't make an implicit copy as it must be doing today. I wonder if that behavior simply isn't implemented yet, or if there's some other reason it can't work today?

Does it behave differently in an optimized vs. non-optimized build? Not that that helps you in general, but it would be interesting to see if the optimizer is able to avoid the copy.

@Nevin's advice is your best bet for now; allocate it on the heap yourself to guarantee a stable address.

4

You need to use _Cell for now to get a stable address. This is also what Mutex is using under the hood. cc @Alejandro

2 Likes
5

What I'm currently doing is compression/Sources/Zlib/ZStreamBox.swift at main · brokenhandsio/compression · GitHub for your exact same reason

6

Thanks for the explanation. I’ve switched to using calloc/free and it’s working now.

This must come up a lot with C++ bindings – C++ objects are not necessarily what’s called “trivially moveable”, and standard library containers like vector can’t just use memcpy to move their items around, they have to invoke move constructors. So putting any non-trivially-moveable C++ type in a Swift object will break in the same way.

There really, really needs to be better documentation of C interop. There’s nothing in the Swift Book, the official pages on Apple’s docs are superficial, and even books like Advanced Swift don’t go into enough depth to deal with situations like this.

5 Likes
7

_Cell is still "move on move", this will print 2 different addresses:

func doIt() {
  let cell = _Cell(1)

  func foobar<T>(_ input: consuming _Cell<T>) {
    print(input._address)
  }

  print(cell._address)
  foobar(cell)
}

I just use UnsafeMutablePointer: Use UnsafeMutablePointer for handling z_stream by Cyberbeni · Pull Request #12 · adam-fowler/compress-nio · GitHub

2 Likes
8

That is the correct behavior for _Cell. If you consume it, it will move. But as long as you borrow it, it will stay at the same address.

9

We do actually understand and handle this in C++ interop. The problem is precisely that C structs are formally copyable, and often there's no problem with copying them, and yet sometimes it's critically important that you don't.

7 Likes