Usability of pointers in Swift

I have an app which uses Metal and I find myself having to work with pointers frequently.

The user experience of pointers in Swift is utterly dreadful.

UnsafeMutablePointer, UnsafePointer, UnsafeMutableBufferPointer, UnsafeBufferPointer, UnsafeMutableRawPointer, UnsafeRawPointer, UnsafeMutableRawBufferPointer, UnsafeRawBufferPointer.

Did I get them all? This is just comically bad.

When I work with pointers, I know it's unsafe, I don't need or want the language constantly holding my hand and slowing me down while I have to keep track of these types.

I often find myself dropping to ObjC or C++ just so I don't have to deal with this garbage. Or I find myself just auto-completing my way to getting something working.

At least it's gotten better after working with swift for years now. I can read the tea leaves of compiler errors.

Why was the wheel of pointers reinvented? C++ could have had a mess of high-level pointer types of no use just like Swift. Instead they have super useful ones like unique_ptr and shared_ptr.

Rust, AFAICT, does not have a gaggle of pointer types.

Most swift code doesn't have to deal with pointers, so why try to make them as safe as possible?

Why can't we have, perhaps Pointer<T> and MutablePointer<T> and call it a day? A pointer to some bytes would be Pointer<Int8>. And just have good old pointer arithmetic and subscripting. C pointers baby!

Can I build those pointers on top of swift's mess of pointer-wank?

pointer.advanced(by: 1) instead of pointer+1. Are you kidding me? Ok, maybe it does have the operator, I didn't actually check.

rawPointer.bindMemory(to: Int.self, capacity: count) instead of (int*) rawPointer. Is Swift just trying to make pointers so unpleasant to work with that I will stick to safe APIs? Which I can't of course.

assumingMemoryBound (the verbosity!) vs bindMemory? It's all just the same pointer under the hood, right?

This rather awful verbose MemoryLayout<T>.size instead of sizeof(T).

And then of course there's withUnsafeBytes. I'm sure eventually I'll have to nest that a few times to get multiple pointers. Why can't Swift guarantee a pointer is valid within the scope of what it's pointing to? If I do:

var array = [1,2,3]
var ptr = UnsafePointer<Int>(&array)

I get a warning about a dangling pointer. Is it so hard for Swift to ensure that ptr is valid within the scope (like C/C++/ObjC others) or is this more hand-holding?

Let's look at some code which is importing a ModelIO triangle mesh into my app:

// Transform data.
        vertexData.withUnsafeMutableBytes { buf in
            
            let ptr = buf.baseAddress!.assumingMemoryBound(to: MCVertex.self)
            
            for i in 0..<mesh.vertexCount {
                var v = float3.zero
                withUnsafeMutableBytes(of: &v) { ptr in
                    let start = i*stride+attr.offset
                    _ = vdata.copyBytes(to: ptr, from: start..<start+12)
                }
                
                let p = v * scale + translation
                
                ptr[i].position.0 = p.x
                ptr[i].position.1 = p.y
                ptr[i].position.2 = p.z
            }
            
        }

In ObjC this would look something like:

MCVertex* ptr = (MCVertex*) vertexData.mutableBytes;
for(int i=0; i<mesh.vertexCount;++i) {
   float3 v;
   memcpy(&v, (char*)vdata.bytes + i*stride+attr.offset, 3*sizeof(float));
   p = v * scale + translation;
   ptr[i].position = p;
}

Now, maybe I could improve the Swift. But still, it's too complex!

Pointers are just not safe. Accept it and be at peace with it. Why try to fight it?

I know this reads like a rant. Please just tell me why I'm wrong.

9 Likes

They’re different operations.

In-memory representation can be constructed/deconstructed on an as-needed basis. It’s very apparent with String’s multitude of uft representations. Pretty sure many structures behave the same way.

Before writing a detailed response to this: if you want to just write C, why not just write C? That's a perfectly fine thing to do. Swift doesn't need to re-implement the semantics of C pointers, and shouldn't do so, precisely because you can just use C.

6 Likes

Because I've written an iOS/macOS app. And I do write C/ObjC/C++ (see above).

And because there are other things that I actually do like about Swift.

2 Likes

This would look a lot cleaner if you weren't trying to just transliterate your C code.

outputData.withUnsafeMutableBytes { outputBuffer in
  let outputArray = outputBuffer.bindMemory(to: MCVertex.self)
  inputData.withUnsafeBytes { inputBuffer in
    for i in 0..<mesh.vertexCount {
      let v = inputBuffer.load(fromByteOffset: i*stride+attr.offset, as: float3.self)
      let p = v * scale + translation
      outputBuffer[i].position.0 = p.x
      outputBuffer[i].position.1 = p.y
      outputBuffer[i].position.2 = p.z
   }
}

It's unfortunate that MCVertex.position is a tuple instead of a float3, but if you can't fix that, you could at least write an adapter property in an extension.

5 Likes

Isn't it a testament to the lameness of this that I couldn't simply figure that out?

1 Like

Without diving too deep into it, I'm pretty sure if you import a VData struct with the correct layout, you can do:

vData.withUnsafeBytes { bytes in
  let vData = bytes.baseAddress!.assumingMemoryBound(to: VDATAType.self)
  vertexData.withUnsafeMutableBytes { bytes in
    let vertexData = bytes.baseAddress!.assumingMemoryBound(to: MCVertex.self)

    for i in 0..<mesh.vertexCount {
      let v = vData[i].position * scale + translation
      ptr[i].position = (v.x, v.y, v.z)
    }
  }
}

Well, if you don't mind the Unsafe drop-off setup.

1 Like

The stride and offset are not statically known.

Setting aside the tone of the original post, there are some valid points here.

1. Swift pointer types have unwieldy names. This is intentional, and from my perspective it is not a big deal because any project making extensive use of pointers can declare shorter type-aliases for them, such as Ptr<T>.

2. It is difficult to learn the proper usage of methods like assumingMemoryBound(to:) and bindMemory. This is a documentation issue which I believe is real. There should be an official Swift document that serves as a Pointer Programming Guide, but we do not currently have one that I know of.

3. APIs like withUnsafeBytes can lead to multiply-nested “pyramid of doom” code structures. This is an ergonomic issue which deserves consideration. I don’t know how to solve it, perhaps it will involve variadic-generic versions of those functions, to construct several pointers at once. Or perhaps it will be something analogous to what guard does to eliminate nested ifs. I don’t know the solution, but it’s worth looking for one.

33 Likes

Ok, then I guess @John_McCall's code would be a good one.

There's also this recent init(unsafeUninitializedCapacity:initializingWith:) so that we could have [MCVertex] for computation until offloading it onto the buffer.

2 Likes

I think this is a good assessment. I did need to dig into the Unsafe rabbit hole until I figure out what binding, initialised status are, and how to property handle it.

1 Like

That's quite the mouthful.

1 Like

I did try to solve this issue in my own project. There's a slight problem if you want a mix of Mutable/non-Mutable, though at best it'd split into two withUnsafes.

¯\_(ツ)_/¯ The naming was much debated during the proposal review.

1 Like

Design-by-committee at its finest, I'm sure.

What did I do wrong?

I think the unwieldy naming is definitely a problem. I would like to see us eventually have unsafe { ... } blocks like Rust and D do, so the Unsafe bit can move out of the names of individual unsafe things. Aside from the redundancy, a naming convention isn't a particularly robust unsafe containment mechanism, because there's nothing keeping you from wrapping up unsafe APIs in other unsafe APIs that don't carry on the naming convention.

The ergonomics of the APIs could be improved as well, and I think we're aware of several areas we could make things better. The focus thus far has been establishing the basic memory model for pointers, not so much on making them pleasant to use yet.

18 Likes

Ok fixed up @John_McCall's suggestion:

// Transform data.
        vertexData.withUnsafeMutableBytes { outputBuffer in
          let outputArray = outputBuffer.bindMemory(to: MCVertex.self)
          vdata.withUnsafeBytes { inputBuffer in
            for i in 0..<mesh.vertexCount {
              let v = inputBuffer.load(fromByteOffset: i*stride+attr.offset, as: float3.self)
              let p = v * scale + translation
              outputArray[i].position.0 = p.x
              outputArray[i].position.1 = p.y
              outputArray[i].position.2 = p.z
            }
          }
        }

Pretty hilarious how "unable to infer complex closure return type" actually means "you forgot a }"

2 Likes

Oh never mind!

Fatal error: load from misaligned raw pointer

This is pretty funny.

Ah, right, loads will definitely assume higher alignment, sorry. Your C code happens to be written in a way that avoids that.

Quite odd that the type checker continues, even when the parser couldn't even match the bracket.

2 Likes