[Draft] UnsafeRawPointer API


(Andrew Trick) #1

I sent two RFC's for this proposal over the past couple months (see Early swift-evolution threads). High-level feedback was fairly light. This version is a final draft, as I expect it to go through the review process next week. There is a lot more explanation and detail in this proposal now, and the memory model has been simplified and clarified.

https://github.com/atrick/swift-evolution/blob/voidpointer/proposals/XXXX-unsaferawpointer.md

If you have opinions or suggestions on API syntax, please make yourself heard. You can jump straight to the naming discussion here:

https://github.com/atrick/swift-evolution/blob/voidpointer/proposals/XXXX-unsaferawpointer.md#variations-under-consideration

Of particular interest may be the API names for:

- Memory allocation/deallocation: fairly fundamental to the language.

- Unsafe casting from raw pointers to typed pointers. This is going to impact a lot of code that needs C interoperability.

Keep in mind that we will make additive API improvements later for convenience. We want the fundamentals to be clear, explicit, and reasonably safe.

-Andy

XXXX-unsaferawpointer.md (56.1 KB)


(L Mihalkovic) #2

Very cool...

Couple thoughts

UnsafeMutableRawPointer:
func store<T>(, WITH: T)
does not flow very well
Fill:with: seems nicer or write(, from:T) which means changing 'load' into 'read'
func read<T>(_ : T.Type) -> T
func write<T>(_: T.T.Type, from: T) (write even match the method doc)

UnsafeRawPointer.toType():
Should it nit be something like typed(as:) instead

Regards
LM
(From mobile)

···

On Jun 24, 2016, at 3:40 AM, Andrew Trick via swift-evolution <swift-evolution@swift.org> wrote:

I sent two RFC's for this proposal over the past couple months (see Early swift-evolution threads). High-level feedback was fairly light. This version is a final draft, as I expect it to go through the review process next week. There is a lot more explanation and detail in this proposal now, and the memory model has been simplified and clarified.

https://github.com/atrick/swift-evolution/blob/voidpointer/proposals/XXXX-unsaferawpointer.md

If you have opinions or suggestions on API syntax, please make yourself heard. You can jump straight to the naming discussion here:

https://github.com/atrick/swift-evolution/blob/voidpointer/proposals/XXXX-unsaferawpointer.md#variations-under-consideration

Of particular interest may be the API names for:

- Memory allocation/deallocation: fairly fundamental to the language.

- Unsafe casting from raw pointers to typed pointers. This is going to impact a lot of code that needs C interoperability.

Keep in mind that we will make additive API improvements later for convenience. We want the fundamentals to be clear, explicit, and reasonably safe.

-Andy

<XXXX-unsaferawpointer.md>
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Matthew Johnson) #3

Andrew, thank you for working on this. The latest draft is much improved!

I have a few questions.

Why do you require explicitly passing the type in these signatures?

func initialize<T>(_: T.Type, with: T, count: Int = 1) -> UnsafeMutablePointer<T>
func initialize<T>(toContiguous: T.Type, atIndex: Int, with: T) -> UnsafeMutablePointer<T>
func storeRaw<T>(_: T.Type, with: T)
func storeRaw<T>(toContiguous: T.Type, atIndex: Int, with: T)

There is probably a good reason, but it seems redundant at first glance and isn’t obvious from my reading of the proposal. The alternatives would be something like this:

func initialize<T>(with: T, count: Int = 1) -> UnsafeMutablePointer<T>

The parameter order in this signature is the opposite of the order in UnsafeMutablePointer. Is that intentional? If so, what is the rationale? You might want to elaborate this in the proposal.

public func + (lhs: Int, rhs: UnsafeRawPointer) -> UnsafeRawPointer

Shouldn’t the precondition be that memory for all elements is *uninitialized* / *deinitialized* in this example?

// - precondition: memory for all elements is initialized.
func freeCBuffer() {
  UnsafeRawPointer(ptrToA).deallocate(capacity: eltCount, of: A.self)
}

It looks like there is a type in this example:

var anyT = T(...)
takesTPtr(&anyT)
takesVoidPtr(&any)

Should the last line say `&anyT`?

Other than these few questions all I can say is that this looks great! I believe it will add important clarity to code that works with unsafe pointers.

-Matthew

···

On Jun 23, 2016, at 8:40 PM, Andrew Trick via swift-evolution <swift-evolution@swift.org> wrote:

I sent two RFC's for this proposal over the past couple months (see Early swift-evolution threads). High-level feedback was fairly light. This version is a final draft, as I expect it to go through the review process next week. There is a lot more explanation and detail in this proposal now, and the memory model has been simplified and clarified.

https://github.com/atrick/swift-evolution/blob/voidpointer/proposals/XXXX-unsaferawpointer.md

If you have opinions or suggestions on API syntax, please make yourself heard. You can jump straight to the naming discussion here:

https://github.com/atrick/swift-evolution/blob/voidpointer/proposals/XXXX-unsaferawpointer.md#variations-under-consideration

Of particular interest may be the API names for:

- Memory allocation/deallocation: fairly fundamental to the language.

- Unsafe casting from raw pointers to typed pointers. This is going to impact a lot of code that needs C interoperability.

Keep in mind that we will make additive API improvements later for convenience. We want the fundamentals to be clear, explicit, and reasonably safe.

-Andy

<XXXX-unsaferawpointer.md>_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Russ Bishop) #4

I really like this proposal. I think it brings a lot of clarity. Nice work!

Russ

···

On Jun 23, 2016, at 6:40 PM, Andrew Trick via swift-evolution <swift-evolution@swift.org> wrote:

I sent two RFC's for this proposal over the past couple months (see Early swift-evolution threads). High-level feedback was fairly light. This version is a final draft, as I expect it to go through the review process next week. There is a lot more explanation and detail in this proposal now, and the memory model has been simplified and clarified.

https://github.com/atrick/swift-evolution/blob/voidpointer/proposals/XXXX-unsaferawpointer.md

-Andy


(Andrew Trick) #5

Would anyone like to bikeshed the allocation API? Here are two options with a slight stylistic difference:

# Option 1:

extension UnsafeMutableRawPointer {
    init<T>(allocatingCapacity: Int, of: T.Type)

    func deallocate<T>(capacity: Int, of: T.Type)
}

let ptrToA = UnsafeMutableRawPointer(allocatingCapacity: 1, of: A.self)
  .initialize(A.self, with: A())
ptrToA.deinitialize(count: 1).deallocate(capacity: 1, of: A.self)

# Option 2:

extension UnsafeMutableRawPointer {
    static allocate<T>(capacity: Int, of: T.Type) -> UnsafeMutableRawPointer

    func deallocate<T>(capacity: Int, of: T.Type)
}

let ptrToA = UnsafeMutableRawPointer.allocate(capacity: 1, of: A.self)
  .initialize(A.self, with: A())
ptrToA.deinitialize(count: 1).deallocate(capacity: 1, of: A.self)

-Andy

···

On Jun 23, 2016, at 6:40 PM, Andrew Trick via swift-evolution <swift-evolution@swift.org> wrote:

https://github.com/atrick/swift-evolution/blob/voidpointer/proposals/XXXX-unsaferawpointer.md


(Dmitri Gribenko) #6

Hi Andy,

Thank you for the proposal! A few comments from me.

- In the "Custom memory allocation section" you write:

Note: The same allocated raw memory cannot be used both for this custom memory allocation case and for the C buffer case above because the C buffer requries that the allocated raw memory is always initialized to the same type.

Could you provide more explanation? I'm not quite getting it. Is
this because of binding -- are you saying that there is no way to
un-bind the type?

- In the "Accessing uninitialized memory with a typed pointer (binding
the type)" section you write

This cast explicitly signals the intention to bind the raw memory to the destination type.

I think that "signals the intention" is not a strong enough wording,
it is open to interpretations. Either it is a no-op "intention" (that
can be retracted) or it is the actual binding. From other discussions
in this thread, I think you are proposing that the .toType() method
actually binds the memory to a type. Is this right? Here's what made
me think this way:

The following code is undefined:

ptrA = rawPtr.cast(to: UnsafePointer<A>.self)
ptrA.initialize(with: A())
ptrA.deinitialize()
ptrB = rawPtr.cast(to: UnsafePointer<B>.self)
ptrB.initialize(with: B())

It is hard to spot the difference between the two styles without drawing attention to the unsafe cast.

- In the same section, in the table, it is not clear whether the
"tptr.deinitialize" operation un-binds the memory type, or does not
have effect on it. Which way is it? Can I replace
"tptr.initialize(t2: T)" with "tptr.initialize(u1: U)"?

- Is it valid to access an ARC reference to a class MyClass that
conforms to MyProtocol with aliasing pointers,
UnsafeMutablePointer<MyClass>, UnsafeMutablePointer<MyProtocol>, and
'UnsafeMutablePointer<AnyObject>' ? What about
'UnsafeMutablePointer<AnyObject?>' ?

- There's no API to convert from UnsafeMutableRawPointer to
UnsafeMutablePointer<T> without either doing an initialization, or
binding the type. Is this on purpose? The reason why I'm asking is
that initialization does not seem to be binding the type (I couldn't
find that in the proposal), but still performs the conversion,
allowing further code to use typed memory access. If this allows the
optimizer to get the desired guarantees about memory, why is binding
important? (I'm probably completely confused about this point.)

- Just wanted to mention that we'd probably need 'raw' variants of
atomic operations for stdlib-internal use, but you probably already
noticed that while working on the branch.

Dmitri

···

--
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com>*/


(Andrew Trick) #7

Very cool...

Couple thoughts

UnsafeMutableRawPointer:
func store<T>(, WITH: T)
does not flow very well
Fill:with: seems nicer or write(, from:T) which means changing 'load' into 'read'
func read<T>(_ : T.Type) -> T
func write<T>(_: T.T.Type, from: T) (write even match the method doc)

Yes but...

- I was parrotting the current initialize(_: T.Type, with: T) style.

- I was trying to establish consistency that `from` is used to copy from a pointer which points to a range of elements.

- Doesn't `fill` imply assigning a range of elements? That would make sense for `storeRaw(contiguous:)` but not the others.

- `store` by itself may imply assignment. Any previous value will not be destroyed (we don't even know its type). The user needs to be aware of this, at the expense of awkward naming. Safety is more important than convenience here. Hence we need `storeRaw` or `storeBits`. There is a deliberate assymetry between store and load because `load` will initialize its returned value. `store` will not initialize the stored value. `initialize` should be used for that.

- `writeRaw` sounds a little weird. `writeBits` sounds better.

UnsafeRawPointer.toType():
Should it nit be something like typed(as:) instead

I like "typed(as:)" better than toType(_). I'm debating whether it should be:
"unsafeCast(toType:)". It's a clarity/safety vs. verbosity tradeoff.

-Andy

···

On Jun 23, 2016, at 10:10 PM, L. Mihalkovic <laurent.mihalkovic@gmail.com> wrote:

Regards
LM
(From mobile)

On Jun 24, 2016, at 3:40 AM, Andrew Trick via swift-evolution <swift-evolution@swift.org> wrote:

I sent two RFC's for this proposal over the past couple months (see Early swift-evolution threads). High-level feedback was fairly light. This version is a final draft, as I expect it to go through the review process next week. There is a lot more explanation and detail in this proposal now, and the memory model has been simplified and clarified.

https://github.com/atrick/swift-evolution/blob/voidpointer/proposals/XXXX-unsaferawpointer.md

If you have opinions or suggestions on API syntax, please make yourself heard. You can jump straight to the naming discussion here:

https://github.com/atrick/swift-evolution/blob/voidpointer/proposals/XXXX-unsaferawpointer.md#variations-under-consideration

Of particular interest may be the API names for:

- Memory allocation/deallocation: fairly fundamental to the language.

- Unsafe casting from raw pointers to typed pointers. This is going to impact a lot of code that needs C interoperability.

Keep in mind that we will make additive API improvements later for convenience. We want the fundamentals to be clear, explicit, and reasonably safe.

-Andy

<XXXX-unsaferawpointer.md>
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Andrew Trick) #8

Andrew, thank you for working on this. The latest draft is much improved!

I have a few questions.

Why do you require explicitly passing the type in these signatures?

func initialize<T>(_: T.Type, with: T, count: Int = 1) -> UnsafeMutablePointer<T>
func initialize<T>(toContiguous: T.Type, atIndex: Int, with: T) -> UnsafeMutablePointer<T>
func storeRaw<T>(_: T.Type, with: T)
func storeRaw<T>(toContiguous: T.Type, atIndex: Int, with: T)

There is probably a good reason, but it seems redundant at first glance and isn’t obvious from my reading of the proposal. The alternatives would be something like this:

func initialize<T>(with: T, count: Int = 1) -> UnsafeMutablePointer<T>

Good question. It is deliberately, and unfortunately redundant. We're trading convenience for safety. I added this note to the proposal:

Note that the `T.Type` argument on `initialize` is redundant because
it may be inferred from the `with` argument. However, relying on type
inferrence at this point is dangerous. The user needs to ensure that
the raw pointer has the necessary size and alignment for the
initialized type. Explicitly spelling the type at initialization
prevents bugs in which the user has incorrectly guessed the inferred
type.

The parameter order in this signature is the opposite of the order in UnsafeMutablePointer. Is that intentional? If so, what is the rationale? You might want to elaborate this in the proposal.

public func + (lhs: Int, rhs: UnsafeRawPointer) -> UnsafeRawPointer

Fixed.

Shouldn’t the precondition be that memory for all elements is *uninitialized* / *deinitialized* in this example?

// - precondition: memory for all elements is initialized.
func freeCBuffer() {
  UnsafeRawPointer(ptrToA).deallocate(capacity: eltCount, of: A.self)
}

Typo. Thanks!

It looks like there is a type in this example:

var anyT = T(...)
takesTPtr(&anyT)
takesVoidPtr(&any)

Should the last line say `&anyT`?

Typo. Thanks!

-Andy

···

On Jun 24, 2016, at 8:19 AM, Matthew Johnson <matthew@anandabits.com> wrote:

Other than these few questions all I can say is that this looks great! I believe it will add important clarity to code that works with unsafe pointers.

-Matthew

On Jun 23, 2016, at 8:40 PM, Andrew Trick via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

I sent two RFC's for this proposal over the past couple months (see Early swift-evolution threads). High-level feedback was fairly light. This version is a final draft, as I expect it to go through the review process next week. There is a lot more explanation and detail in this proposal now, and the memory model has been simplified and clarified.

https://github.com/atrick/swift-evolution/blob/voidpointer/proposals/XXXX-unsaferawpointer.md

If you have opinions or suggestions on API syntax, please make yourself heard. You can jump straight to the naming discussion here:

https://github.com/atrick/swift-evolution/blob/voidpointer/proposals/XXXX-unsaferawpointer.md#variations-under-consideration

Of particular interest may be the API names for:

- Memory allocation/deallocation: fairly fundamental to the language.

- Unsafe casting from raw pointers to typed pointers. This is going to impact a lot of code that needs C interoperability.

Keep in mind that we will make additive API improvements later for convenience. We want the fundamentals to be clear, explicit, and reasonably safe.

-Andy

<XXXX-unsaferawpointer.md>_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Andrew Trick) #9

Hi Andy,

Thank you for the proposal! A few comments from me.

Thanks for the feedback (again)! I updated the language in the proposal (again).

- In the "Custom memory allocation section" you write:

Note: The same allocated raw memory cannot be used both for this custom memory allocation case and for the C buffer case above because the C buffer requries that the allocated raw memory is always initialized to the same type.

Could you provide more explanation? I'm not quite getting it. Is
this because of binding -- are you saying that there is no way to
un-bind the type?

It now reads:

Note: The same allocated raw memory cannot be used both for this
custom memory allocation case and for the C buffer case above because
the C buffer binds the allocated memory to an element type. Binding
the type applies to the allocation lifetime and requries that the
allocated raw memory is always initialized to the same type.

This should make sense after reading the earlier section on initializing memory that I’ve updated as explained below...

- In the "Accessing uninitialized memory with a typed pointer (binding
the type)" section you write

This cast explicitly signals the intention to bind the raw memory to the destination type.

I think that "signals the intention" is not a strong enough wording,
it is open to interpretations. Either it is a no-op "intention" (that
can be retracted) or it is the actual binding. From other discussions
in this thread, I think you are proposing that the .toType() method
actually binds the memory to a type. Is this right? Here's what made
me think this way:

The following code is undefined:

ptrA = rawPtr.cast(to: UnsafePointer<A>.self)
ptrA.initialize(with: A())
ptrA.deinitialize()
ptrB = rawPtr.cast(to: UnsafePointer<B>.self)
ptrB.initialize(with: B())

It is hard to spot the difference between the two styles without drawing attention to the unsafe cast.

- In the same section, in the table, it is not clear whether the
"tptr.deinitialize" operation un-binds the memory type, or does not
have effect on it. Which way is it? Can I replace
"tptr.initialize(t2: T)" with "tptr.initialize(u1: U)”?

I reworded this section:

https://github.com/atrick/swift-evolution/blob/voidpointer/proposals/XXXX-unsaferawpointer.md#initializing-memory-with-a-typed-pointer-binding-the-type

The simple rule is: if you use a typed pointer to initialize memory, that memory is bound for the duration of its lifetime. The semantics are not temporal.

I think that's very clear in the proposal now.

To fully answer your question “can memory be unbound", technically if you reinitialize the memory using a raw pointer, then accesses on either side of that initialization point are protected from strict aliasing guarantees. But that's just a confusing and mostly useless artifact of the implementation. That's not the memory model as it as specified in the proposal. So forget I said that.

- Is it valid to access an ARC reference to a class MyClass that
conforms to MyProtocol with aliasing pointers,
UnsafeMutablePointer<MyClass>, UnsafeMutablePointer<MyProtocol>, and
'UnsafeMutablePointer<AnyObject>' ? What about
'UnsafeMutablePointer<AnyObject?>’ ?

That's discussed in the "Type Safe Memory Access" documentation. I wrote this doc for the sake of discussion, but it’s a little out of date and needs to be updated before putting it up for review again:

https://github.com/atrick/swift/blob/type-safe-mem-docs/docs/TypeSafeMemory.rst

There are two aspects of the type to consider: whether they are related for the purpose of strict aliasing, and whether they are mutually layout compatible.

MyClass and MyProtocol are related, so there's no problem with aliasing. If MyProtocol is an AnyObject existential, then they also have the same representation (layout), so it's safe. If MyProtocol is not class constrained, then they are not layout compatible.

Regarding AnyObject and AnyObject?:

They are related because "one type may be a tuple, enum, or struct that contains the other type as part of its own storage"

They are mutually layout compatible because we know per the ABI that Optional class types have the same representation as class types. It's a bit of a special case. In general, fragile enums with single payloads have some layout guarantees. You can read the payload from the enum type, but can't read the enum from the payload type.

Layout compatible also pertains to the location of references within the type (they need to be ARC-compatible). I have not adequately explained that in the doc but have it on my TODO list.

- There's no API to convert from UnsafeMutableRawPointer to
UnsafeMutablePointer<T> without either doing an initialization, or
binding the type. Is this on purpose? The reason why I'm asking is
that initialization does not seem to be binding the type (I couldn't
find that in the proposal), but still performs the conversion,
allowing further code to use typed memory access. If this allows the
optimizer to get the desired guarantees about memory, why is binding
important? (I'm probably completely confused about this point.)

This is important. I think it is clear now in the proposal (if not, please suggest some better language to use):

https://github.com/atrick/swift-evolution/blob/voidpointer/proposals/XXXX-unsaferawpointer.md#initializing-memory-with-a-typed-pointer-binding-the-type

In short, initializing via a raw pointer has different semantics than initializing via a typed pointer (just like other operations on raw pointer have different semantics). Initializing via a raw pointer changes the memory state to "initialized with some type" for the lifetime of that value in memory. Deinitializing the memory then returns it to a pristine state. It does not impose any type on the allocated memory. I propose that this should be the normal, "type safe" way to work with unsafe pointers.

Initializing via a typed pointer, in addition to changing the temporal memory state, also imposes a type on the allocated memory for the entire lifetime of the memory itself, from allocation to deallocation. This is effectively a performance optimization and works well for an important use case (C buffer), but it is less safe, which is why casting a raw to a typed pointer needs to be an explicit cast.

As I keep saying, the type safe way to get a typed pointer is by initializing the raw pointer:

  let ptrToA = rawPtr.initialize(A.self, A())

Explicit pointer casts should only be used for optimizing certain data structures. Unfortunately, interoperability is another reason that developers will need to cast pointers in practice. Some reasoning about type safety is needed in those cases, but at least with this proposal it will be much easier to audit the risky pointer casts.

- Just wanted to mention that we'd probably need 'raw' variants of
atomic operations for stdlib-internal use, but you probably already
noticed that while working on the branch.

Yes, I saw that. Thanks.
-Andy

···

On Jun 27, 2016, at 10:18 PM, Dmitri Gribenko <gribozavr@gmail.com> wrote:

Dmitri

--
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com>*/


(Andrew Trick) #10

Since no one else has weighed in, and DaveA and I are in violent agreement, I'm going to revise the proposal to change the allocation API from:

  let rawPtr = UnsafeRawPointer(allocatingCapacity: n, of: T.self)

to

  let rawPtr = UnsafeRawPointer.allocate(capacity: n, of: T.self)

The only reason I had proposed the former was to mimic the current
UnsafePointer API. However, an initializer's primary purpose should be to
construct an instance. This functions primary purpose is to allocate
memory, returning an new instance as a useful side effect. This also
provides natural symmetry with:

  rawPtr.deallocate(capacity: n, of: T.self)

-Andy

···

On Jun 27, 2016, at 4:53 PM, Andrew Trick <atrick@apple.com> wrote:

On Jun 23, 2016, at 6:40 PM, Andrew Trick via swift-evolution <swift-evolution@swift.org> wrote:

https://github.com/atrick/swift-evolution/blob/voidpointer/proposals/XXXX-unsaferawpointer.md

Would anyone like to bikeshed the allocation API? Here are two options with a slight stylistic difference:

# Option 1:

extension UnsafeMutableRawPointer {
   init<T>(allocatingCapacity: Int, of: T.Type)

   func deallocate<T>(capacity: Int, of: T.Type)
}

let ptrToA = UnsafeMutableRawPointer(allocatingCapacity: 1, of: A.self)
.initialize(A.self, with: A())
ptrToA.deinitialize(count: 1).deallocate(capacity: 1, of: A.self)

# Option 2:

extension UnsafeMutableRawPointer {
   static allocate<T>(capacity: Int, of: T.Type) -> UnsafeMutableRawPointer

   func deallocate<T>(capacity: Int, of: T.Type)
}

let ptrToA = UnsafeMutableRawPointer.allocate(capacity: 1, of: A.self)
.initialize(A.self, with: A())
ptrToA.deinitialize(count: 1).deallocate(capacity: 1, of: A.self)


(John McCall) #11

Andrew, thank you for working on this. The latest draft is much improved!

I have a few questions.

Why do you require explicitly passing the type in these signatures?

func initialize<T>(_: T.Type, with: T, count: Int = 1) -> UnsafeMutablePointer<T>
func initialize<T>(toContiguous: T.Type, atIndex: Int, with: T) -> UnsafeMutablePointer<T>
func storeRaw<T>(_: T.Type, with: T)
func storeRaw<T>(toContiguous: T.Type, atIndex: Int, with: T)

There is probably a good reason, but it seems redundant at first glance and isn’t obvious from my reading of the proposal. The alternatives would be something like this:

func initialize<T>(with: T, count: Int = 1) -> UnsafeMutablePointer<T>

Good question. It is deliberately, and unfortunately redundant. We're trading convenience for safety. I added this note to the proposal:

Note that the `T.Type` argument on `initialize` is redundant because
it may be inferred from the `with` argument. However, relying on type
inferrence at this point is dangerous. The user needs to ensure that
the raw pointer has the necessary size and alignment for the
initialized type. Explicitly spelling the type at initialization
prevents bugs in which the user has incorrectly guessed the inferred
type.

One major problematic case: the value could have a defaulted type, e.g.:
  pointer.initialize(with: 0, count: 1024)

John.

···

On Jun 24, 2016, at 10:58 AM, Andrew Trick via swift-evolution <swift-evolution@swift.org> wrote:

On Jun 24, 2016, at 8:19 AM, Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:

The parameter order in this signature is the opposite of the order in UnsafeMutablePointer. Is that intentional? If so, what is the rationale? You might want to elaborate this in the proposal.

public func + (lhs: Int, rhs: UnsafeRawPointer) -> UnsafeRawPointer

Fixed.

Shouldn’t the precondition be that memory for all elements is *uninitialized* / *deinitialized* in this example?

// - precondition: memory for all elements is initialized.
func freeCBuffer() {
  UnsafeRawPointer(ptrToA).deallocate(capacity: eltCount, of: A.self)
}

Typo. Thanks!

It looks like there is a type in this example:

var anyT = T(...)
takesTPtr(&anyT)
takesVoidPtr(&any)

Should the last line say `&anyT`?

Typo. Thanks!

-Andy

Other than these few questions all I can say is that this looks great! I believe it will add important clarity to code that works with unsafe pointers.

-Matthew

On Jun 23, 2016, at 8:40 PM, Andrew Trick via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

I sent two RFC's for this proposal over the past couple months (see Early swift-evolution threads). High-level feedback was fairly light. This version is a final draft, as I expect it to go through the review process next week. There is a lot more explanation and detail in this proposal now, and the memory model has been simplified and clarified.

https://github.com/atrick/swift-evolution/blob/voidpointer/proposals/XXXX-unsaferawpointer.md

If you have opinions or suggestions on API syntax, please make yourself heard. You can jump straight to the naming discussion here:

https://github.com/atrick/swift-evolution/blob/voidpointer/proposals/XXXX-unsaferawpointer.md#variations-under-consideration

Of particular interest may be the API names for:

- Memory allocation/deallocation: fairly fundamental to the language.

- Unsafe casting from raw pointers to typed pointers. This is going to impact a lot of code that needs C interoperability.

Keep in mind that we will make additive API improvements later for convenience. We want the fundamentals to be clear, explicit, and reasonably safe.

-Andy

<XXXX-unsaferawpointer.md>_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Matthew Johnson) #12

Andrew, thank you for working on this. The latest draft is much improved!

I have a few questions.

Why do you require explicitly passing the type in these signatures?

func initialize<T>(_: T.Type, with: T, count: Int = 1) -> UnsafeMutablePointer<T>
func initialize<T>(toContiguous: T.Type, atIndex: Int, with: T) -> UnsafeMutablePointer<T>
func storeRaw<T>(_: T.Type, with: T)
func storeRaw<T>(toContiguous: T.Type, atIndex: Int, with: T)

There is probably a good reason, but it seems redundant at first glance and isn’t obvious from my reading of the proposal. The alternatives would be something like this:

func initialize<T>(with: T, count: Int = 1) -> UnsafeMutablePointer<T>

Good question. It is deliberately, and unfortunately redundant. We're trading convenience for safety. I added this note to the proposal:

Note that the `T.Type` argument on `initialize` is redundant because
it may be inferred from the `with` argument. However, relying on type
inferrence at this point is dangerous. The user needs to ensure that
the raw pointer has the necessary size and alignment for the
initialized type. Explicitly spelling the type at initialization
prevents bugs in which the user has incorrectly guessed the inferred
type.

I suspected it was something like this. Thanks for explaining and updating the proposal!

···

Sent from my iPad

On Jun 24, 2016, at 12:58 PM, Andrew Trick <atrick@apple.com> wrote:

On Jun 24, 2016, at 8:19 AM, Matthew Johnson <matthew@anandabits.com> wrote:

The parameter order in this signature is the opposite of the order in UnsafeMutablePointer. Is that intentional? If so, what is the rationale? You might want to elaborate this in the proposal.

public func + (lhs: Int, rhs: UnsafeRawPointer) -> UnsafeRawPointer

Fixed.

Shouldn’t the precondition be that memory for all elements is *uninitialized* / *deinitialized* in this example?

// - precondition: memory for all elements is initialized.
func freeCBuffer() {
  UnsafeRawPointer(ptrToA).deallocate(capacity: eltCount, of: A.self)
}

Typo. Thanks!

It looks like there is a type in this example:

var anyT = T(...)
takesTPtr(&anyT)
takesVoidPtr(&any)

Should the last line say `&anyT`?

Typo. Thanks!

-Andy

Other than these few questions all I can say is that this looks great! I believe it will add important clarity to code that works with unsafe pointers.

-Matthew

On Jun 23, 2016, at 8:40 PM, Andrew Trick via swift-evolution <swift-evolution@swift.org> wrote:

I sent two RFC's for this proposal over the past couple months (see Early swift-evolution threads). High-level feedback was fairly light. This version is a final draft, as I expect it to go through the review process next week. There is a lot more explanation and detail in this proposal now, and the memory model has been simplified and clarified.

https://github.com/atrick/swift-evolution/blob/voidpointer/proposals/XXXX-unsaferawpointer.md

If you have opinions or suggestions on API syntax, please make yourself heard. You can jump straight to the naming discussion here:

https://github.com/atrick/swift-evolution/blob/voidpointer/proposals/XXXX-unsaferawpointer.md#variations-under-consideration

Of particular interest may be the API names for:

- Memory allocation/deallocation: fairly fundamental to the language.

- Unsafe casting from raw pointers to typed pointers. This is going to impact a lot of code that needs C interoperability.

Keep in mind that we will make additive API improvements later for convenience. We want the fundamentals to be clear, explicit, and reasonably safe.

-Andy

<XXXX-unsaferawpointer.md>_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(L Mihalkovic) #13

Very cool...

Couple thoughts

UnsafeMutableRawPointer:
func store<T>(, WITH: T)
does not flow very well
Fill:with: seems nicer or write(, from:T) which means changing 'load' into 'read'
func read<T>(_ : T.Type) -> T
func write<T>(_: T.T.Type, from: T) (write even match the method doc)

Yes but...

- I was parrotting the current initialize(_: T.Type, with: T) style.

- I was trying to establish consistency that `from` is used to copy from a pointer which points to a range of elements.

- Doesn't `fill` imply assigning a range of elements? That would make sense for `storeRaw(contiguous:)` but not the others.

- `store` by itself may imply assignment. Any previous value will not be destroyed (we don't even know its type). The user needs to be aware of this, at the expense of awkward naming. Safety is more important than convenience here. Hence we need `storeRaw` or `storeBits`. There is a deliberate assymetry between store and load because `load` will initialize its returned value. `store` will not initialize the stored value. `initialize` should be used for that.

- `writeRaw` sounds a little weird. `writeBits` sounds better.

all makes sense.

UnsafeRawPointer.toType():
Should it nit be something like typed(as:) instead

I like "typed(as:)" better than toType(_). I'm debating whether it should be:
"unsafeCast(toType:)". It's a clarity/safety vs. verbosity tradeoff.

I like the watch-what-you-wish-for warning of unsafeCast.

I think it was really brilliant to introduce the extra step... basically echos the alloc/init dichotomy of objc, so it should feel familiar to people with this bkgnd

···

On Jun 24, 2016, at 7:43 PM, Andrew Trick <atrick@apple.com> wrote:

On Jun 23, 2016, at 10:10 PM, L. Mihalkovic <laurent.mihalkovic@gmail.com> wrote:

-Andy

Regards
LM
(From mobile)

On Jun 24, 2016, at 3:40 AM, Andrew Trick via swift-evolution <swift-evolution@swift.org> wrote:

I sent two RFC's for this proposal over the past couple months (see Early swift-evolution threads). High-level feedback was fairly light. This version is a final draft, as I expect it to go through the review process next week. There is a lot more explanation and detail in this proposal now, and the memory model has been simplified and clarified.

https://github.com/atrick/swift-evolution/blob/voidpointer/proposals/XXXX-unsaferawpointer.md

If you have opinions or suggestions on API syntax, please make yourself heard. You can jump straight to the naming discussion here:

https://github.com/atrick/swift-evolution/blob/voidpointer/proposals/XXXX-unsaferawpointer.md#variations-under-consideration

Of particular interest may be the API names for:

- Memory allocation/deallocation: fairly fundamental to the language.

- Unsafe casting from raw pointers to typed pointers. This is going to impact a lot of code that needs C interoperability.

Keep in mind that we will make additive API improvements later for convenience. We want the fundamentals to be clear, explicit, and reasonably safe.

-Andy

<XXXX-unsaferawpointer.md>
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Dmitri Gribenko) #14

Hi Andy,

Everything is clear now, thank you!

···

On Tue, Jun 28, 2016 at 1:02 PM, Andrew Trick <atrick@apple.com> wrote:

Initializing via a typed pointer, in addition to changing the temporal memory state, also imposes a type on the allocated memory for the entire lifetime of the memory itself, from allocation to deallocation.

I see. Given that UnsafeMutablePoiner.initialize() has this very
important difference in semantics, did you consider reflecting it in
the name? Something like '.bindTypeAndInitialize()' -- but I'm sure a
better wording is possible.

Dmitri

--
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com>*/


(Andrew Trick) #15

I like the watch-what-you-wish-for warning of unsafeCast.

I’ll try porting stdlib to the “UnsafeRawPointer.unsafeCast(to: T.Type)” syntax and see how bad it is.

I think it was really brilliant to introduce the extra step... basically echos the alloc/init dichotomy of objc, so it should feel familiar to people with this

Dave Abrahams advocated for this and it ended up making the memory model easier to explain in simple terms that correspond to the API.

-Andy

···

On Jun 24, 2016, at 11:17 AM, L. Mihalkovic <laurent.mihalkovic@gmail.com> wrote:


(Matthew Johnson) #16

Andrew, thank you for working on this. The latest draft is much improved!

I have a few questions.

Why do you require explicitly passing the type in these signatures?

func initialize<T>(_: T.Type, with: T, count: Int = 1) -> UnsafeMutablePointer<T>
func initialize<T>(toContiguous: T.Type, atIndex: Int, with: T) -> UnsafeMutablePointer<T>
func storeRaw<T>(_: T.Type, with: T)
func storeRaw<T>(toContiguous: T.Type, atIndex: Int, with: T)

There is probably a good reason, but it seems redundant at first glance and isn’t obvious from my reading of the proposal. The alternatives would be something like this:

func initialize<T>(with: T, count: Int = 1) -> UnsafeMutablePointer<T>

Good question. It is deliberately, and unfortunately redundant. We're trading convenience for safety. I added this note to the proposal:

Note that the `T.Type` argument on `initialize` is redundant because
it may be inferred from the `with` argument. However, relying on type
inferrence at this point is dangerous. The user needs to ensure that
the raw pointer has the necessary size and alignment for the
initialized type. Explicitly spelling the type at initialization
prevents bugs in which the user has incorrectly guessed the inferred
type.

One major problematic case: the value could have a defaulted type, e.g.:
  pointer.initialize(with: 0, count: 1024)

Yes, this one crossed my mind. :slight_smile:

It would be more convenient to not have to explicitly pass the type in every case even if we had to explicitly specify it here like 'pointer.initialize(with: Int(0), count: 1024)' but there probably isn't a good way to require it only in cases like this. And given the safety concerns I think the minor inconvenience is very warranted. In any case, it's nice to have the rationale documented in the proposal.

···

Sent from my iPad

On Jun 24, 2016, at 1:27 PM, John McCall <rjmccall@apple.com> wrote:

On Jun 24, 2016, at 10:58 AM, Andrew Trick via swift-evolution <swift-evolution@swift.org> wrote:
On Jun 24, 2016, at 8:19 AM, Matthew Johnson <matthew@anandabits.com> wrote:

John.

The parameter order in this signature is the opposite of the order in UnsafeMutablePointer. Is that intentional? If so, what is the rationale? You might want to elaborate this in the proposal.

public func + (lhs: Int, rhs: UnsafeRawPointer) -> UnsafeRawPointer

Fixed.

Shouldn’t the precondition be that memory for all elements is *uninitialized* / *deinitialized* in this example?

// - precondition: memory for all elements is initialized.
func freeCBuffer() {
  UnsafeRawPointer(ptrToA).deallocate(capacity: eltCount, of: A.self)
}

Typo. Thanks!

It looks like there is a type in this example:

var anyT = T(...)
takesTPtr(&anyT)
takesVoidPtr(&any)

Should the last line say `&anyT`?

Typo. Thanks!

-Andy

Other than these few questions all I can say is that this looks great! I believe it will add important clarity to code that works with unsafe pointers.

-Matthew

On Jun 23, 2016, at 8:40 PM, Andrew Trick via swift-evolution <swift-evolution@swift.org> wrote:

I sent two RFC's for this proposal over the past couple months (see Early swift-evolution threads). High-level feedback was fairly light. This version is a final draft, as I expect it to go through the review process next week. There is a lot more explanation and detail in this proposal now, and the memory model has been simplified and clarified.

https://github.com/atrick/swift-evolution/blob/voidpointer/proposals/XXXX-unsaferawpointer.md

If you have opinions or suggestions on API syntax, please make yourself heard. You can jump straight to the naming discussion here:

https://github.com/atrick/swift-evolution/blob/voidpointer/proposals/XXXX-unsaferawpointer.md#variations-under-consideration

Of particular interest may be the API names for:

- Memory allocation/deallocation: fairly fundamental to the language.

- Unsafe casting from raw pointers to typed pointers. This is going to impact a lot of code that needs C interoperability.

Keep in mind that we will make additive API improvements later for convenience. We want the fundamentals to be clear, explicit, and reasonably safe.

-Andy

<XXXX-unsaferawpointer.md>_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Andrew Trick) #17

I don't think there's a clear winner here. Let me enumerate some
options.

Option (1) UnsafePointer<T>(cast: UnsafeRawPointer)

Option (2) UnsafePointer<T>(_: UnsafeRawPointer, to: T.self)

Option (3) UnsafeRawPointer.unsafeCast<T>(to: T.Type) -> UnsafePointer<T>

Option (4) unsafeCast(rawPointer: UnsafeRawPointer, to: T.self) -> UnsafePointer<T>

···

On Jun 24, 2016, at 11:22 AM, Andrew Trick via swift-evolution <swift-evolution@swift.org> wrote:

On Jun 24, 2016, at 11:17 AM, L. Mihalkovic <laurent.mihalkovic@gmail.com <mailto:laurent.mihalkovic@gmail.com>> wrote:

I like the watch-what-you-wish-for warning of unsafeCast.

I’ll try porting stdlib to the “UnsafeRawPointer.unsafeCast(to: T.Type)” syntax and see how bad it is.

---
Option (3) is the most explicit and searchable, and forces
UnsafeRawPointer to be spelled out in the conversion (unless you
already have a raw pointer). I like this because conceptually, you
need to cast to a raw pointer before casting to a new pointee
type, and casting a raw pointer to a typed pointer carries important
semantics beyond simply converting to a typed pointer. The main problem
with Option (3) is that optional raw pointers can't be converted
naturally (without using `map`).

Another thing I'm a little nervous about is confusing a cast of the
pointer value with a cast of the pointee type:

  `unsafeBitCast(rawPtr, to: Int.self)`

is very different from

  `rawPtr.unsafeCast(to: Int.self)`

Does this need to be clarified? If so, we can go back to the
`toPointee` label that I proposed earlier.

With that in mind, Option(4) is starting to look pretty good.

Examples:

---
Case 1: casting a raw pointer as an argument

func foo(_: UnsafePointer<A>)

let rawPtr = UnsafeRawPointer(...)

(1) foo(UnsafePointer(cast: rawPtr))

(2) foo(UnsafePointer(rawPtr, to: A.self))

(3) foo(rawPtr.unsafeCast(to: A.self))

(4) foo(unsafeCast(rawPointer: rawPtr, to: A.self))

---
Case 2: "recasting" a typed pointer argument

Note that typed pointer arguments are implicitly cast to raw pointer
arguments, so the conversion from PtrB to raw is implicit.

func foo(_: UnsafePointer<A>)

let ptrB = UnsafePointer<B>(...)

(1) foo(UnsafePointer(cast: ptrB))

(2) foo(UnsafePointer(ptrB, to: A.self))

(3) foo(UnsafeRawPointer(ptrB).unsafeCast(to: A.self))

(4) foo(unsafeCast(rawPointer: ptrB, to: A.self))

---
Case 3: Optional argument (only Option 3 is affected)

func nullableFoo(_: UnsafePointer<Int>?)

let ptrB: UnsafePointer<UInt>? = ...

(1) nullableFoo(UnsafePointer(cast: ptrB))

(2) nullableFoo(UnsafePointer(ptrB, to: A.self))

(3) nullableFoo(UnsafeRawPointer(ptrB).map { $0.unsafeCast(to: A.self) })

(4) nullableFoo(unsafeCast(rawPointer: ptrB, to: A.self))

---
Case 4: Return values

func foo() -> UnsafePointer<A>

func caller() -> UnsafePointer<B> { ...

(1) return UnsafePointer(cast: foo())

(2) return UnsafePointer(foo(), to: B.self)

(3) let rawPtr = UnsafeRawPointer(foo())
    return rawPtr.unsafeCast(to: B.self)

(4) return unsafeCast(rawPointer: foo(), to: B.self)

-Andy


(Dave Abrahams) #18

I like the watch-what-you-wish-for warning of unsafeCast.

I’ll try porting stdlib to the “UnsafeRawPointer.unsafeCast(to:
T.Type)” syntax and see how bad it is.

I don't think there's a clear winner here. Let me enumerate some
options.

Option (1) UnsafePointer<T>(cast: UnsafeRawPointer)

The problem with this one is that T can be deduced based on type
context. I think we ought to move away from that for operations like
this one.

Option (2) UnsafePointer<T>(_: UnsafeRawPointer, to: T.self)

I think you mean T.Type, not T.self, because this looks like a declaration.

To evaluate, you have to look at the use-site:

    let p = UnsafePointer(r, to: Int.self)

I don't find “to” to be descriptive enough. Maybe

    let p = UnsafePointer(r, pointee: Int.self)

is better. But I hate that the language doesn't give us a way to say
“don't deduce generic parameters here.” This is the only syntax that
feels right, IMO:

    let p = UnsafePointer<Int>(r)

Option (3) UnsafeRawPointer.unsafeCast<T>(to: T.Type) ->
UnsafePointer<T>

    r.unsafeCast(to: Int.self)

I don't see adding “unsafe” to the name of the operation as adding
anything. It isn't any more unsafe than other UnsafeRawPointer
operations. Also, it reads like we're casting the raw pointer to an
Int, rather than to an UnsafePointer<Int>. Also, how do you get an
UnsafeMutablePointer?

Option (4) unsafeCast(rawPointer: UnsafeRawPointer, to: T.self) ->
UnsafePointer<T>

This one won't read correctly for the same reasons as #3.

    r.cast(to: UnsafePointer<Int>.self)

works better for me than any of the alternatives given our inability to
get the One True Syntax.

---
Option (3) is the most explicit and searchable, and forces
UnsafeRawPointer to be spelled out in the conversion (unless you
already have a raw pointer).

Huh? I'm confused here. What you wrote looks like it's intended to be
a regular method, in which case of course invoking it would require a raw
pointer and wouldn't force you to write UnsafeRawPointer out anywhere.

The only way it could force you to write UnsafeRawPointer would be if it
was a static method, but in that case it has too few arguments.

I like this because conceptually, you need to cast to a raw pointer
before casting to a new pointee type, and casting a raw pointer to a
typed pointer carries important semantics beyond simply converting to
a typed pointer. The main problem with Option (3) is that optional raw
pointers can't be converted naturally (without using `map`).

  r ?? someExpressionUsing(r!)

best I can do.

Another thing I'm a little nervous about is confusing a cast of the
pointer value with a cast of the pointee type:

  `unsafeBitCast(rawPtr, to: Int.self)`

is very different from

  `rawPtr.unsafeCast(to: Int.self)`

Does this need to be clarified?

Yes!

If so, we can go back to the `toPointee` label that I proposed
earlier.

With that in mind, Option(4) is starting to look pretty good.

Examples:

---
Case 1: casting a raw pointer as an argument

Use sites! (yay)...

func foo(_: UnsafePointer<A>)

let rawPtr = UnsafeRawPointer(...)

(1) foo(UnsafePointer(cast: rawPtr))

(2) foo(UnsafePointer(rawPtr, to: A.self))

(3) foo(rawPtr.unsafeCast(to: A.self))

(4) foo(unsafeCast(rawPointer: rawPtr, to: A.self))

foo(rawPtr.cast(to: UnsafePointer<A>.self))

---
Case 2: "recasting" a typed pointer argument

Note that typed pointer arguments are implicitly cast to raw pointer
arguments, so the conversion from PtrB to raw is implicit.

func foo(_: UnsafePointer<A>)

let ptrB = UnsafePointer<B>(...)

(1) foo(UnsafePointer(cast: ptrB))

(2) foo(UnsafePointer(ptrB, to: A.self))

(3) foo(UnsafeRawPointer(ptrB).unsafeCast(to: A.self))

(4) foo(unsafeCast(rawPointer: ptrB, to: A.self))

foo(UnsafeRawPointer(ptrB).cast(to: UnsafePointer<A>.self))

I don't believe in making these “double-hops” concise.

---
Case 3: Optional argument (only Option 3 is affected)

func nullableFoo(_: UnsafePointer<Int>?)

let ptrB: UnsafePointer<UInt>? = ...

(1) nullableFoo(UnsafePointer(cast: ptrB))

(2) nullableFoo(UnsafePointer(ptrB, to: A.self))

(3) nullableFoo(UnsafeRawPointer(ptrB).map { $0.unsafeCast(to: A.self) })

(4) nullableFoo(unsafeCast(rawPointer: ptrB, to: A.self))

nullableFoo(UnsafeRawPointer(ptrB)?.cast(to: UnsafePointer<A>.self))

You do the above with a failable init on UnsafeRawPointer that takes an
optional UnsafePointer.

---
Case 4: Return values

func foo() -> UnsafePointer<A>

func caller() -> UnsafePointer<B> { ...

(1) return UnsafePointer(cast: foo())

(2) return UnsafePointer(foo(), to: B.self)

(3) let rawPtr = UnsafeRawPointer(foo())
    return rawPtr.unsafeCast(to: B.self)

(4) return unsafeCast(rawPointer: foo(), to: B.self)

return UnsafeRawPointer(foo()).cast(to: UnsafePointer<B>.self)

IMO-ly y'rs,

···

on Fri Jun 24 2016, Andrew Trick <atrick-AT-apple.com> wrote:

On Jun 24, 2016, at 11:22 AM, Andrew Trick via swift-evolution >> <swift-evolution@swift.org> wrote:

On Jun 24, 2016, at 11:17 AM, L. Mihalkovic >>> <laurent.mihalkovic@gmail.com >>> <mailto:laurent.mihalkovic@gmail.com>> wrote:

--
-Dave


(Andrew Trick) #19

Hi Andy,

Everything is clear now, thank you!

Initializing via a typed pointer, in addition to changing the temporal memory state, also imposes a type on the allocated memory for the entire lifetime of the memory itself, from allocation to deallocation.

I see. Given that UnsafeMutablePoiner.initialize() has this very
important difference in semantics, did you consider reflecting it in
the name? Something like '.bindTypeAndInitialize()' -- but I'm sure a
better wording is possible.

Yes, I did consider that. I’m still open to it--maybe ‘.typedInitialize(with:). But...

(1) It’s awkward. The developer isn’t interested in binding the type at that point. It’s just a side effect of the way their unsafe pointer is being used.

(2) It would imply that the ‘.bindAndInitialize' entry point is the only way to bind the type of allocated memory. But once you have a typed pointer, it’s easy to initialize memory via a simple assignment:
ptrToA[0] = A() // where A is trivial
If ptrToA was in an uninitialized state, then that also binds the type.

Instead, I tried to focus on discouraging the unsafe pointer cast that leads to this situation. The important thing is that we have an alternate “safe” API so that most developers just don’t need to think about it.

I think the best way to make this clear is through API comments and code examples. There are just a handful of UnsafePointer idioms that will be prevalent (Expected use cases) , and as long as people follow normal patterns with the proposed API, they’re unlikely to run afoul of the rules.

If the proposed API still isn’t safe enough, and developers are running into problems, then that calls for development of a pointer type safety sanitizer.

-Andy

···

On Jun 28, 2016, at 1:53 PM, Dmitri Gribenko <gribozavr@gmail.com> wrote:
On Tue, Jun 28, 2016 at 1:02 PM, Andrew Trick <atrick@apple.com> wrote:

Dmitri

--
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com>*/


(Alejandro Martinez) #20

Awesome changes! Thanks for working on this.
The final section address all the concerns that I was having while
reading which is really interesting for a proposal of this kind.
I’m glad to see improvements on this part of the API.
I pointed out a minor typo on twitter, sorry didn’t saw this thread
before :wink: https://twitter.com/alexito4/status/746420742661771265

Can’t wait to see be able to use this :wink:
Cheers,

···

On Fri, Jun 24, 2016 at 8:38 PM, Matthew Johnson via swift-evolution <swift-evolution@swift.org> wrote:

Sent from my iPad

On Jun 24, 2016, at 1:27 PM, John McCall <rjmccall@apple.com> wrote:

On Jun 24, 2016, at 10:58 AM, Andrew Trick via swift-evolution > <swift-evolution@swift.org> wrote:

On Jun 24, 2016, at 8:19 AM, Matthew Johnson <matthew@anandabits.com> wrote:

Andrew, thank you for working on this. The latest draft is much improved!

I have a few questions.

Why do you require explicitly passing the type in these signatures?

func initialize<T>(_: T.Type, with: T, count: Int = 1) ->
UnsafeMutablePointer<T>
func initialize<T>(toContiguous: T.Type, atIndex: Int, with: T) ->
UnsafeMutablePointer<T>
func storeRaw<T>(_: T.Type, with: T)
func storeRaw<T>(toContiguous: T.Type, atIndex: Int, with: T)

There is probably a good reason, but it seems redundant at first glance and
isn’t obvious from my reading of the proposal. The alternatives would be
something like this:

func initialize<T>(with: T, count: Int = 1) -> UnsafeMutablePointer<T>

Good question. It is deliberately, and unfortunately redundant. We're
trading convenience for safety. I added this note to the proposal:

Note that the `T.Type` argument on `initialize` is redundant because
it may be inferred from the `with` argument. However, relying on type
inferrence at this point is dangerous. The user needs to ensure that
the raw pointer has the necessary size and alignment for the
initialized type. Explicitly spelling the type at initialization
prevents bugs in which the user has incorrectly guessed the inferred
type.

One major problematic case: the value could have a defaulted type, e.g.:
  pointer.initialize(with: 0, count: 1024)

Yes, this one crossed my mind. :slight_smile:

It would be more convenient to not have to explicitly pass the type in every
case even if we had to explicitly specify it here like
'pointer.initialize(with: Int(0), count: 1024)' but there probably isn't a
good way to require it only in cases like this. And given the safety
concerns I think the minor inconvenience is very warranted. In any case,
it's nice to have the rationale documented in the proposal.

John.

The parameter order in this signature is the opposite of the order in
UnsafeMutablePointer. Is that intentional? If so, what is the rationale?
You might want to elaborate this in the proposal.

public func + (lhs: Int, rhs: UnsafeRawPointer) -> UnsafeRawPointer

Fixed.

Shouldn’t the precondition be that memory for all elements is
*uninitialized* / *deinitialized* in this example?

// - precondition: memory for all elements is initialized.
func freeCBuffer() {
  UnsafeRawPointer(ptrToA).deallocate(capacity: eltCount, of: A.self)
}

Typo. Thanks!

It looks like there is a type in this example:

var anyT = T(...)
takesTPtr(&anyT)
takesVoidPtr(&any)

Should the last line say `&anyT`?

Typo. Thanks!

-Andy

Other than these few questions all I can say is that this looks great! I
believe it will add important clarity to code that works with unsafe
pointers.

-Matthew

On Jun 23, 2016, at 8:40 PM, Andrew Trick via swift-evolution > <swift-evolution@swift.org> wrote:

I sent two RFC's for this proposal over the past couple months (see Early
swift-evolution threads). High-level feedback was fairly light. This version
is a final draft, as I expect it to go through the review process next week.
There is a lot more explanation and detail in this proposal now, and the
memory model has been simplified and clarified.

https://github.com/atrick/swift-evolution/blob/voidpointer/proposals/XXXX-unsaferawpointer.md

If you have opinions or suggestions on API syntax, please make yourself
heard. You can jump straight to the naming discussion here:

https://github.com/atrick/swift-evolution/blob/voidpointer/proposals/XXXX-unsaferawpointer.md#variations-under-consideration

Of particular interest may be the API names for:

- Memory allocation/deallocation: fairly fundamental to the language.

- Unsafe casting from raw pointers to typed pointers. This is going to
impact a lot of code that needs C interoperability.

Keep in mind that we will make additive API improvements later for
convenience. We want the fundamentals to be clear, explicit, and reasonably
safe.

-Andy

<XXXX-unsaferawpointer.md>_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
Alejandro Martinez
http://alejandromp.com