Unmanaged, and COpaquePointer vs. Unsafe(Mutable)Pointer

One hopes to avoid COpaquePointer and Unsafe(Mutable)Pointer, but in
practice I've had occasion to use them a few times.

Things that are nice:

- Unmanaged.passUnretained/takeRetained/etc. which make memory management
semantics explicit.

Things I've been frustrated by:

- An API takes UnsafeMutablePointer<Void>, but Unmanaged.toOpaque() returns
a COpaquePointer.

- An API gives me UnsafeMutablePointer<Void>, but Unmanaged.fromOpaque()
takes a COpaquePointer.

In practice, I end up with monstrosities like:

Unmanaged.passRetained(CFCopyDescription(Unmanaged<AnyObject>.fromOpaque(COpaquePointer($0)).takeUnretainedValue()))

I think a few things could help:

- Phase out COpaquePointer in favor of UnsafePointer<Void> (is this already
happening?)

- Add implicit conversion from COpaquePointer to
Unsafe(Mutable)Pointer<Void>, and/or vice versa.

- Even better, add implicit conversion from Unmanaged<T> to COpaquePointer
or UnsafePointer<Void>, behaving the way toOpaque() currently does. Also,
replace Unmanaged.fromOpaque() with an initializer Unmanaged(_:
UnsafePointer<Void>).

What are others' experiences? Would it be feasible/favorable to have some
of these conversions?

Jacob Bandes-Storch

One hopes to avoid COpaquePointer and Unsafe(Mutable)Pointer, but in practice I've had occasion to use them a few times.

Things that are nice:

- Unmanaged.passUnretained/takeRetained/etc. which make memory management semantics explicit.

Things I've been frustrated by:

- An API takes UnsafeMutablePointer<Void>, but Unmanaged.toOpaque() returns a COpaquePointer.

- An API gives me UnsafeMutablePointer<Void>, but Unmanaged.fromOpaque() takes a COpaquePointer.

In practice, I end up with monstrosities like:

Unmanaged.passRetained(CFCopyDescription(Unmanaged<AnyObject>.fromOpaque(COpaquePointer($0)).takeUnretainedValue()))

I think a few things could help:

- Phase out COpaquePointer in favor of UnsafePointer<Void> (is this already happening?)

I think this is planned, but it would be good to verify that. We should definitely be consistent about which type we’re using.

- Add implicit conversion from COpaquePointer to Unsafe(Mutable)Pointer<Void>, and/or vice versa.

- Even better, add implicit conversion from Unmanaged<T> to COpaquePointer or UnsafePointer<Void>, behaving the way toOpaque() currently does. Also, replace Unmanaged.fromOpaque() with an initializer Unmanaged(_: UnsafePointer<Void>).

We try very hard to avoid adding new implicit conversions.

John.

···

On Dec 8, 2015, at 12:26 AM, Jacob Bandes-Storch via swift-evolution <swift-evolution@swift.org> wrote:

COpaquePointer is IMO a vestige that should be eliminated completely. We'd ultimately like to import opaque C structs as distinct, non-constructible types in Swift, so that they can still be well-typed UnsafePointer<OpaqueThing> types in Swift.

-Joe

···

On Dec 8, 2015, at 12:26 AM, Jacob Bandes-Storch via swift-evolution <swift-evolution@swift.org> wrote:

One hopes to avoid COpaquePointer and Unsafe(Mutable)Pointer, but in practice I've had occasion to use them a few times.

Things that are nice:

- Unmanaged.passUnretained/takeRetained/etc. which make memory management semantics explicit.

Things I've been frustrated by:

- An API takes UnsafeMutablePointer<Void>, but Unmanaged.toOpaque() returns a COpaquePointer.

- An API gives me UnsafeMutablePointer<Void>, but Unmanaged.fromOpaque() takes a COpaquePointer.

In practice, I end up with monstrosities like:

Unmanaged.passRetained(CFCopyDescription(Unmanaged<AnyObject>.fromOpaque(COpaquePointer($0)).takeUnretainedValue()))

I think a few things could help:

- Phase out COpaquePointer in favor of UnsafePointer<Void> (is this already happening?)

- Add implicit conversion from COpaquePointer to Unsafe(Mutable)Pointer<Void>, and/or vice versa.

- Even better, add implicit conversion from Unmanaged<T> to COpaquePointer or UnsafePointer<Void>, behaving the way toOpaque() currently does. Also, replace Unmanaged.fromOpaque() with an initializer Unmanaged(_: UnsafePointer<Void>).

What are others' experiences? Would it be feasible/favorable to have some of these conversions?

Jacob Bandes-Storch
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

COpaquePointer is IMO a vestige that should be eliminated completely. We'd
ultimately like to import opaque C structs as distinct, non-constructible
types in Swift, so that they can still be well-typed
UnsafePointer<OpaqueThing> types in Swift.

-Joe

That would be nice. But there is still the "context pointer" use case,
where conversions to/from UnsafePointer<Void> are needed. Would it make
sense for the Unmanaged type to deal in UnsafePointer<Void>, rather than
COpaquePointer?

···

On Tue, Dec 8, 2015 at 9:42 AM, Joe Groff <jgroff@apple.com> wrote:

On Tue, Dec 8, 2015 at 9:37 AM, John McCall <rjmccall@apple.com> wrote:

- Add implicit conversion from COpaquePointer to
Unsafe(Mutable)Pointer<Void>, and/or vice versa.

- Even better, add implicit conversion from Unmanaged<T> to COpaquePointer
or UnsafePointer<Void>, behaving the way toOpaque() currently does. Also,
replace Unmanaged.fromOpaque() with an initializer Unmanaged(_:
UnsafePointer<Void>).

We try very hard to avoid adding new implicit conversions.

John.

My impression is that Unmanaged is pretty much only used for cases like
this. It seems a bit redundant given that UnsafePointer exists, and
converting between them is tedious as a user. Would it make sense to move
the passUnretained/takeRetainedValue/etc. functions onto UnsafePointer?

COpaquePointer is IMO a vestige that should be eliminated completely. We'd ultimately like to import opaque C structs as distinct, non-constructible types in Swift, so that they can still be well-typed UnsafePointer<OpaqueThing> types in Swift.

-Joe

That would be nice. But there is still the "context pointer" use case, where conversions to/from UnsafePointer<Void> are needed. Would it make sense for the Unmanaged type to deal in UnsafePointer<Void>, rather than COpaquePointer?

I think so, yeah.

- Add implicit conversion from COpaquePointer to Unsafe(Mutable)Pointer<Void>, and/or vice versa.

- Even better, add implicit conversion from Unmanaged<T> to COpaquePointer or UnsafePointer<Void>, behaving the way toOpaque() currently does. Also, replace Unmanaged.fromOpaque() with an initializer Unmanaged(_: UnsafePointer<Void>).

We try very hard to avoid adding new implicit conversions.

John.

My impression is that Unmanaged is pretty much only used for cases like this. It seems a bit redundant given that UnsafePointer exists, and converting between them is tedious as a user. Would it make sense to move the passUnretained/takeRetainedValue/etc. functions onto UnsafePointer?

It possibly makes sense for the methods to exist on both Unmanaged and UnsafePointer<Void>, since it's common to cross over from the managed to unmanaged world both ways.

-Joe

···

On Dec 8, 2015, at 10:32 AM, Jacob Bandes-Storch <jtbandes@gmail.com> wrote:
On Tue, Dec 8, 2015 at 9:42 AM, Joe Groff <jgroff@apple.com <mailto:jgroff@apple.com>> wrote:
On Tue, Dec 8, 2015 at 9:37 AM, John McCall <rjmccall@apple.com <mailto:rjmccall@apple.com>> wrote:

UnsafePointer<T> expresses an extra layer of indirection beyond what Unmanaged<T> does: in C terms, it is an Object** instead of an Object*. I definitely don’t think we should collapse Unmanaged<T>s down to specifically UnsafePointer<Void> and lose the ability to express a type-safe unmanaged reference.

John.

···

On Dec 8, 2015, at 10:32 AM, Jacob Bandes-Storch via swift-evolution <swift-evolution@swift.org> wrote:

On Tue, Dec 8, 2015 at 9:42 AM, Joe Groff <jgroff@apple.com <mailto:jgroff@apple.com>> wrote:
COpaquePointer is IMO a vestige that should be eliminated completely. We'd ultimately like to import opaque C structs as distinct, non-constructible types in Swift, so that they can still be well-typed UnsafePointer<OpaqueThing> types in Swift.

-Joe

That would be nice. But there is still the "context pointer" use case, where conversions to/from UnsafePointer<Void> are needed. Would it make sense for the Unmanaged type to deal in UnsafePointer<Void>, rather than COpaquePointer?

On Tue, Dec 8, 2015 at 9:37 AM, John McCall <rjmccall@apple.com <mailto:rjmccall@apple.com>> wrote:

- Add implicit conversion from COpaquePointer to Unsafe(Mutable)Pointer<Void>, and/or vice versa.

- Even better, add implicit conversion from Unmanaged<T> to COpaquePointer or UnsafePointer<Void>, behaving the way toOpaque() currently does. Also, replace Unmanaged.fromOpaque() with an initializer Unmanaged(_: UnsafePointer<Void>).

We try very hard to avoid adding new implicit conversions.

John.

My impression is that Unmanaged is pretty much only used for cases like this. It seems a bit redundant given that UnsafePointer exists, and converting between them is tedious as a user. Would it make sense to move the passUnretained/takeRetainedValue/etc. functions onto UnsafePointer?

Confirming that this is the direction we should go. We can do this independent of any changes to COpaquePointer, since you'll (almost) never want to pass a class reference through an opaque struct pointer. Feel free to make this part a formal proposal!

Jordan

···

On Dec 8, 2015, at 10:42, Joe Groff via swift-evolution <swift-evolution@swift.org> wrote:

On Dec 8, 2015, at 10:32 AM, Jacob Bandes-Storch <jtbandes@gmail.com <mailto:jtbandes@gmail.com>> wrote:

On Tue, Dec 8, 2015 at 9:42 AM, Joe Groff <jgroff@apple.com <mailto:jgroff@apple.com>> wrote:
COpaquePointer is IMO a vestige that should be eliminated completely. We'd ultimately like to import opaque C structs as distinct, non-constructible types in Swift, so that they can still be well-typed UnsafePointer<OpaqueThing> types in Swift.

-Joe

That would be nice. But there is still the "context pointer" use case, where conversions to/from UnsafePointer<Void> are needed. Would it make sense for the Unmanaged type to deal in UnsafePointer<Void>, rather than COpaquePointer?

I think so, yeah.

Thanks, Jordan. I'll write one up tonight.

Should it use UnsafePointer or UnsafeMutablePointer? I've seen that C APIs
frequently get imported as UnsafeMutablePointer, when it doesn't
necessarily match the semantics of the API. Is that just the default?

Jacob

···

On Tue, Dec 8, 2015 at 3:53 PM, Jordan Rose <jordan_rose@apple.com> wrote:

On Dec 8, 2015, at 10:42, Joe Groff via swift-evolution < > swift-evolution@swift.org> wrote:

On Dec 8, 2015, at 10:32 AM, Jacob Bandes-Storch <jtbandes@gmail.com> > wrote:

On Tue, Dec 8, 2015 at 9:42 AM, Joe Groff <jgroff@apple.com> wrote:

COpaquePointer is IMO a vestige that should be eliminated completely.
We'd ultimately like to import opaque C structs as distinct,
non-constructible types in Swift, so that they can still be well-typed
UnsafePointer<OpaqueThing> types in Swift.

-Joe

That would be nice. But there is still the "context pointer" use case,
where conversions to/from UnsafePointer<Void> are needed. Would it make
sense for the Unmanaged type to deal in UnsafePointer<Void>, rather than
COpaquePointer?

I think so, yeah.

Confirming that this is the direction we should go. We can do this
independent of any changes to COpaquePointer, since you'll (almost) never
want to pass a class reference through an opaque struct pointer. Feel free
to make this part a formal proposal!

Jordan

I’m not sure we want to retire the idea of a separate opaque pointer type As noted here <Proposal to change Unmanaged API to use UnsafePointer by jtbandes · Pull Request #44 · apple/swift-evolution · GitHub, pointers to Void and incomplete types are not in any sense “unsafe” (once you restrict the interface as appropriate for incomplete types), and so maybe we want OpaquePointer<T>.

-Dave

···

On Dec 8, 2015, at 3:53 PM, Jordan Rose via swift-evolution <swift-evolution@swift.org> wrote:

On Dec 8, 2015, at 10:42, Joe Groff via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Dec 8, 2015, at 10:32 AM, Jacob Bandes-Storch <jtbandes@gmail.com <mailto:jtbandes@gmail.com>> wrote:

On Tue, Dec 8, 2015 at 9:42 AM, Joe Groff <jgroff@apple.com <mailto:jgroff@apple.com>> wrote:
COpaquePointer is IMO a vestige that should be eliminated completely. We'd ultimately like to import opaque C structs as distinct, non-constructible types in Swift, so that they can still be well-typed UnsafePointer<OpaqueThing> types in Swift.

-Joe

That would be nice. But there is still the "context pointer" use case, where conversions to/from UnsafePointer<Void> are needed. Would it make sense for the Unmanaged type to deal in UnsafePointer<Void>, rather than COpaquePointer?

I think so, yeah.

Confirming that this is the direction we should go. We can do this independent of any changes to COpaquePointer, since you'll (almost) never want to pass a class reference through an opaque struct pointer. Feel free to make this part a formal proposal!

A 'void *' in C is mutable by default, yeah.

-Joe

···

On Dec 8, 2015, at 4:03 PM, Jacob Bandes-Storch <jtbandes@gmail.com> wrote:

Thanks, Jordan. I'll write one up tonight.

Should it use UnsafePointer or UnsafeMutablePointer? I've seen that C APIs frequently get imported as UnsafeMutablePointer, when it doesn't necessarily match the semantics of the API. Is that just the default?

Either way we're going to lose in some cases. We can probably make the "from" case work for either, but I think the "to" case should just use UnsafeMutablePointer, because (a) "void *" is more common than "const void *" in C APIs*, and (b) if you do need to convert, "UnsafePointer(...)" is shorter. :-)

* citation needed

(Swift does support overloading on return type, but the downside is you need to always provide context, which makes it harder to break things up into multiple statements. So we generally avoid it unless there's a compelling reason.)

Jordan

···

On Dec 8, 2015, at 16:03, Jacob Bandes-Storch <jtbandes@gmail.com> wrote:

Thanks, Jordan. I'll write one up tonight.

Should it use UnsafePointer or UnsafeMutablePointer? I've seen that C APIs frequently get imported as UnsafeMutablePointer, when it doesn't necessarily match the semantics of the API. Is that just the default?

Proposed: Proposal to change Unmanaged API to use UnsafePointer by jtbandes · Pull Request #44 · apple/swift-evolution · GitHub

Jacob

···

On Tue, Dec 8, 2015 at 4:30 PM, Jordan Rose <jordan_rose@apple.com> wrote:

On Dec 8, 2015, at 16:03, Jacob Bandes-Storch <jtbandes@gmail.com> wrote:

Thanks, Jordan. I'll write one up tonight.

Should it use UnsafePointer or UnsafeMutablePointer? I've seen that C
APIs frequently get imported as UnsafeMutablePointer, when it doesn't
necessarily match the semantics of the API. Is that just the default?

Either way we're going to lose in some cases. We can probably make the
"from" case work for either, but I think the "to" case should just use
UnsafeMutablePointer, because (a) "void *" is more common than "const void
*" in C APIs*, and (b) if you *do* need to convert, "UnsafePointer(...)"
is shorter. :-)

* citation needed

(Swift does support overloading on return type, but the downside is you
need to always provide context, which makes it harder to break things up
into multiple statements. So we generally avoid it unless there's a
compelling reason.)

Jordan

A related topic that would be great to discuss for Swift 3: right now nullable C pointers import directly as UnsafePointer, and UnsafePointer are therefore nullable. While it is true that they are unsafe :-), it would be more true to the Swift model to import them as optional unsafe pointers.

There are tradeoffs on both sides, just something to consider.

-Chris

···

On Dec 8, 2015, at 8:07 PM, Jacob Bandes-Storch via swift-evolution <swift-evolution@swift.org> wrote:

Proposed: Proposal to change Unmanaged API to use UnsafePointer by jtbandes · Pull Request #44 · apple/swift-evolution · GitHub

I haven't seen much feedback here. Are there any objections?

What's needed for a proposal to go from pull-request to "Awaiting Review"?

Jacob

···

On Tue, Dec 8, 2015 at 9:52 PM, Chris Lattner <clattner@apple.com> wrote:

On Dec 8, 2015, at 8:07 PM, Jacob Bandes-Storch via swift-evolution < > swift-evolution@swift.org> wrote:

Proposed: Proposal to change Unmanaged API to use UnsafePointer by jtbandes · Pull Request #44 · apple/swift-evolution · GitHub

A related topic that would be great to discuss for Swift 3: right now
nullable C pointers import directly as UnsafePointer, and UnsafePointer are
therefore nullable. While it is true that they are unsafe :-), it would be
more true to the Swift model to import them as optional unsafe pointers.

There are tradeoffs on both sides, just something to consider.

-Chris

This LGTM as far as being a well thought out proposal, and I’d certainly like to see COpaquePointer go away. :-)

One thing to consider incorporating into your proposal, we’re trying to keep Swift 2.2 source compatible with Swift 2 (thought providing migration warnings where it makes sense). Should your proposal wait for swift 3, or is there some piece that would be good to go into swift 2.2 to aid migration?

-Chris

···

On Dec 10, 2015, at 10:58 PM, Jacob Bandes-Storch <jtbandes@gmail.com> wrote:

I haven't seen much feedback here. Are there any objections?

What's needed for a proposal to go from pull-request to "Awaiting Review"?

Jacob

On Tue, Dec 8, 2015 at 9:52 PM, Chris Lattner <clattner@apple.com <mailto:clattner@apple.com>> wrote:
On Dec 8, 2015, at 8:07 PM, Jacob Bandes-Storch via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Proposed: Proposal to change Unmanaged API to use UnsafePointer by jtbandes · Pull Request #44 · apple/swift-evolution · GitHub

A related topic that would be great to discuss for Swift 3: right now nullable C pointers import directly as UnsafePointer, and UnsafePointer are therefore nullable. While it is true that they are unsafe :-), it would be more true to the Swift model to import them as optional unsafe pointers.

There are tradeoffs on both sides, just something to consider.

-Chris

We could add @availability(*, deprecated=...) to the existing declarations
but keep them around until Swift 3. I'm not sure whether this is preferable
to simply waiting until Swift 3.

I'm also not sure how the migration stuff usually works, whether it'd be
worth adding a migration step for this or just letting users see & fix the
deprecation warnings themselves.

Earlier in this thread, Jordan said

Swift does support overloading on return type, but the downside is you need

to always provide context, which makes it harder to break things up into
multiple statements. So we generally avoid it unless there's a compelling
reason.

Either way, I'd be happy to prepare a patch for this, but I'm guessing I
should wait until the proposal is accepted to do so.

Jacob

···

On Thu, Dec 10, 2015 at 11:07 PM, Chris Lattner <clattner@apple.com> wrote:

This LGTM as far as being a well thought out proposal, and I’d certainly
like to see COpaquePointer go away. :-)

One thing to consider incorporating into your proposal, we’re trying to
keep Swift 2.2 source compatible with Swift 2 (thought providing migration
warnings where it makes sense). Should your proposal wait for swift 3, or
is there some piece that would be good to go into swift 2.2 to aid
migration?

-Chris

On Dec 10, 2015, at 10:58 PM, Jacob Bandes-Storch <jtbandes@gmail.com> > wrote:

I haven't seen much feedback here. Are there any objections?

What's needed for a proposal to go from pull-request to "Awaiting Review"?

Jacob

On Tue, Dec 8, 2015 at 9:52 PM, Chris Lattner <clattner@apple.com> wrote:

On Dec 8, 2015, at 8:07 PM, Jacob Bandes-Storch via swift-evolution < >> swift-evolution@swift.org> wrote:

Proposed: Proposal to change Unmanaged API to use UnsafePointer by jtbandes · Pull Request #44 · apple/swift-evolution · GitHub

A related topic that would be great to discuss for Swift 3: right now
nullable C pointers import directly as UnsafePointer, and UnsafePointer are
therefore nullable. While it is true that they are unsafe :-), it would be
more true to the Swift model to import them as optional unsafe pointers.

There are tradeoffs on both sides, just something to consider.

-Chris