UnsafeMutablePointer allocation compatibility with C malloc/free

Using Swift
1

Are the allocate() and deallocate() methods of UnsafeMutablePointer “compatible” with the C memory management functions malloc() and free()?

Concretely: Would the following be safe?

if let ptr = someCFunctionReturningAMallocedPointer() {
    // The C function returns the result of a `malloc()` call
    // ...
    ptr.deallocate()
}

Or this?

let ptr = UnsafeMutablePointer<CChar>.allocate(capacity: 1024)
someCFunctionExpectingAMallocedPointer(ptr)
// The C function calls `free(ptr)` eventually

My feeling is that malloc/free should not be mixed with allocate/deallocate, but some confirmation would be appreciated.

7 Likes
2

I believe we don’t guarantee compatibility, so yes, you should pair allocators correctly.

On Darwin I’m not sure there’s a reason we couldn’t create a malloc zone for any custom allocator we used, so free will probably work indefinitely, but incompatibility is the better portable assumption.

3 Likes
3

So in the general case, if you're passed an UnsafeMutablePointer, it's not possible to know how to destroy it without knowing how it was created? I assume that suggests that the caller should pass you the correct deallocator in any API that takes ownership of a pointer? (This is not a bad practice; I'm just trying to nail down some best practice rules.)

In returned pointers, I assume this means we need to follow the lead from functions like class_copy_IvarList, and include explicit instructions like "You must free the array with free()."

4 Likes
4

Well, in the general case of course you don't even know whether you can deallocate an unsafe pointer. So yeah, I'd say that whatever conventional knowledge tells you that you own the pointer should also specify how to deallocate it, either by documenting the use of a particular function or just by giving you a deallocation function explicitly.

6 Likes
5

Yeah; there just is a long history of the use of copy in the ObjC runtime to mean "you must call free on this." And that raises an interesting problem of new functions that include "copy" but might be allocated with .allocate. It's no longer obvious what "copy" means as precisely as it was.

1 Like
6

Thank you for the clarification. Would it make sense to change the documentation of UnsafeMutablePointer.deallocate() from

This pointer must be a pointer to the start of a previously allocated memory block.

to

This pointer must be a pointer to the start of a memory block that was previously allocated with allocate().

or something similar?

6 Likes
7

Yes, I think that's a reasonable proposal. @Ben_Cohen, @Andrew_Trick?

8

@John_McCall This has come up before. @Andrew_Trick will remember better than me, but I believe we are malloc/free compatible on all platforms except windows. Some context:

https://bugs.swift.org/browse/SR-9536

https://github.com/apple/swift/commit/0b5fa792e1f0c531d9b9e24583a5b55dcaeed2ad

9

We don't guarantee to be malloc/free compatible on any platform. We haven't yet had a reason to diverge on any platform but Windows.

10

And even if we did guarantee compatibility with the system malloc/free, using custom allocators via LD_PRELOAD or DYLD environment variables is a pretty common thing for people to do (whether or not it's advisable), and there's no way we can guarantee compatibility with any malloc/free someone might implement.

3 Likes
11

Right, this is really the only workable strategy in any language that allows allocation via multiple potentially incompatible mechanisms.

1 Like
12

I wasn't suggesting that we were guaranteeing anything. I was just trying to give background about the current state of affairs.

13

Sure, I just want to make that perfectly clear. There's always a hazard that, by saying "this is how it's implemented today", that someone may misunderstand that as stating a guarantee.

7 Likes
14

But the question was what a programmer should do in their code, not what the current implementation is. It can be useful to tell people about the latter, but it's also important to be very clear that that's what you're doing when that information isn't guaranteed to be accurate in the future.

(Oops, didn't mean to pile on.)

15

I think it is worth noting that we do know we're not compatible with free on Windows, and therefore it's best if we can establish Good Swift Programming Practices to pair allocators correctly.

(I'm also skeptical that we'll ever be able to change it on Apple platforms without breaking tons of existing code.)

4 Likes
16

No worries. I didn't take it that way ; ).

17

I agree that users should pair allocators as a portability guarantee.

That said, I think it's clearly a good implementation goal for UnsafeMutablePointer allocation to be "C compatible"; that's just not very well-defined goal. It's impossible for Swift to provide a platform indepedent guarantee of malloc/free compatibility because Swift supports (over)aligned allocation. I do think that if the platform provides a free-compatible aligned_alloc, then Swift should provide a free-compatible allocate/deallocate. If the user wants to dynamically override allocation, then they would need to override the relevant C library entry points used by Swift. Optimizing manual allocation via UnsafeMutablePointer is a secondary concern. It should never be done to the extent that it breaks C compatibility. If there's a need to allocate buffers differently in Swift, that should be done via something like the ManagedBuffer API.

As far as I can tell, C11 and C++17 have the same issue with aligned_alloc. The difference is that the C/C++ standard libraries make this API available conditionally only when supported, whereas Swift always supports alignment with what I would call a reasonable effort at compatibility. If anyone knows more about these issues with stdlib, or knows of any aligned allocation API on Windows that is free-compatible, I'd be interested in that.

1 Like