[Review] SE-0107: UnsafeRawPointer API

Evolution Announcements
21

Thanks for reviewing!

* What is your evaluation of the proposal?

This is an excellent proposal, and a step forward in balancing safety and un-safety in an understandable way.

I have no issues about the substance, but two documentation issues:

- the compiler’s strict aliasing rules: not clearly defined in this document.
I *think* I know mostly what that means, but I’m not completely sure.

That’s punted to a separate doc:

  For details of the compiler's rules for memory aliasing, see proposed Type Safe Memory Access documentation.
  https://github.com/atrick/swift/blob/type-safe-mem-docs/docs/TypeSafeMemory.rst

That document needs to be rewritten now, but should be good enough for discussion. It’s important to focus on the source-breaking aspect of the proposal now. The memory model spec will be rewritten and clarified as we go. There’s just a limited bandwidth for people to review these kind of specs, and there are actually no changes to the strict aliasing rules being proposed here!

I did recently add a simplified discussion about strict aliasing to the revised proposal, but it’s buried here:

I should create a link at the top.

- so-called “trivial” types:
Coming from physics, where trivial means “not worthy of consideration” (an English dictionary concurs.)
It made me wonder why integers deserved such a putdown; the word “simple” would be just fine.
[y’ = y has solutions y(x) = e^x and y = 0; the latter is trivial as it generally isn’t interesting, despite being valid.]
It’s probably meant as “easily proven”, but since no proofs are shown, it feels like an inappropriate use.

Point taken. That term is used throughout the implementation and was first officially documented here:

and again here:

-Andy

···

On Jun 30, 2016, at 9:37 AM, Guillaume Lessard via swift-evolution <swift-evolution@swift.org> wrote:

* Is the problem being addressed significant enough to warrant a change to Swift?

Absolutely.

* Does this proposal fit well with the feel and direction of Swift?

I think so.

* If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?

I’ve only done the kind of things enabled by this in C/C++, where it just feels like gambling. [feels like testing can only show that something works on one particular C/C++ compiler]. Clarity is good.

* How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

I’ve followed this discussion from the start, and asked questions. My interest was informed by experimental attempts to push the memory model; those attempts would have been covered by this document.

Cheers,
Guillaume Lessard

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

22

Being consistent with existing convention is good; I also agree it happens
to make perfect sense anyway.

···

On Sun, Jul 3, 2016 at 03:18 Andrew Trick via swift-evolution < swift-evolution@swift.org> wrote:

On Jul 2, 2016, at 8:10 PM, Brent Royal-Gordon via swift-evolution < > swift-evolution@swift.org> wrote:

I have a pile of naming quibbles; rather than describe them all in prose
(which turned into a mess), I've annotated parts of the "Full
UnsafeRawPointer API" section in a gist: <
https://gist.github.com/brentdax/8f4ed4decafc1d18c4441092baa13cfe&gt;\.

Let's bikeshed this easy one now... I’m curious what others think:

  // In general, I think you "initialize to" a value, not
  // "initialize with" a value. "with" is needlessly vacuous.
  //
  // func initialize<T>(_: T.Type, with: T, count: Int = 1)
  // -> UnsafeMutablePointer<T>
  func initialize<T>(_: T.Type, to: T, count: Int = 1)
    -> UnsafeMutablePointer<T>

`initialize` was recently renamed to `initialized(with:)`.

commit d96b051d28b6042adcc8b8692a918abddf211aec
Author: Dave Abrahams <dabrahams@apple.com>
Date: Tue Feb 23 15:12:24 2016 -0800

    stdlib: initializePointee(_) => initialize(with:)

    Tacking "Pointee" on just for unary operations (and especially
    operations with an optional count) created inconsistency.

So Swift 3 users have already migrated to this “better” name.

I agree that initialize(to:) is consistent with the language we use for
assigning values. But grammatically, I think initialize(with:) also makes
perfect sense and is just as common.

In general, if there’s controversy, I’ll stick with the existing
conventions because there’s already enough to debate in this proposal.

-Andy
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

23

I have a pile of naming quibbles; rather than describe them all in
prose (which turned into a mess), I've annotated parts of the "Full
UnsafeRawPointer API" section in a gist:
<https://gist.github.com/brentdax/8f4ed4decafc1d18c4441092baa13cfe

<https://gist.github.com/brentdax/8f4ed4decafc1d18c4441092baa13cfe&gt;&gt;\.

Let's bikeshed this easy one now... I’m curious what others think:

  // In general, I think you "initialize to" a value, not
  // "initialize with" a value. "with" is needlessly vacuous.
  //
  // func initialize<T>(_: T.Type, with: T, count: Int = 1)
  // -> UnsafeMutablePointer<T>
  func initialize<T>(_: T.Type, to: T, count: Int = 1)
    -> UnsafeMutablePointer<T>

`initialize` was recently renamed to `initialized(with:)`.

commit d96b051d28b6042adcc8b8692a918abddf211aec
Author: Dave Abrahams
<dabrahams@apple.com>
Date: Tue Feb 23 15:12:24 2016 -0800

    stdlib: initializePointee(_) => initialize(with:)

    Tacking "Pointee" on just for unary operations (and especially
    operations with an optional count) created inconsistency.

So Swift 3 users have already migrated to this “better” name.

I agree that initialize(to:) is consistent with the language we use
for assigning values. But grammatically, I think initialize(with:)
also makes perfect sense and is just as common.

“With” is a weak preposition with many possible interpretations, so we'd
like to avoid it. If I used “with” where “to” would have worked, I
regret it.

In general, if there’s controversy, I’ll stick with the existing
conventions because there’s already enough to debate in this proposal.

Let's not compound the mistake, though, please!

···

on Sun Jul 03 2016, Andrew Trick <swift-evolution@swift.org> wrote:

On Jul 2, 2016, at 8:10 PM, Brent Royal-Gordon via swift-evolution >> <swift-evolution@swift.org> wrote:

--
-Dave

24

Regards
(From mobile)

I have a pile of naming quibbles; rather than describe them all in prose (which turned into a mess), I've annotated parts of the "Full UnsafeRawPointer API" section in a gist: <https://gist.github.com/brentdax/8f4ed4decafc1d18c4441092baa13cfe&gt;\.

Let's bikeshed this easy one now... I’m curious what others think:

  // In general, I think you "initialize to" a value, not
  // "initialize with" a value. "with" is needlessly vacuous.
  //
  // func initialize<T>(_: T.Type, with: T, count: Int = 1)
  // -> UnsafeMutablePointer<T>
  func initialize<T>(_: T.Type, to: T, count: Int = 1)
    -> UnsafeMutablePointer<T>

`initialize` was recently renamed to `initialized(with:)`.

commit d96b051d28b6042adcc8b8692a918abddf211aec
Author: Dave Abrahams <dabrahams@apple.com>
Date: Tue Feb 23 15:12:24 2016 -0800

    stdlib: initializePointee(_) => initialize(with:)
    
    Tacking "Pointee" on just for unary operations (and especially
    operations with an optional count) created inconsistency.

So Swift 3 users have already migrated to this “better” name.

I agree that initialize(to:) is consistent with the language we use for assigning values. But grammatically, I think initialize(with:) also makes perfect sense and is just as common.

+1

···

On Jul 3, 2016, at 10:18 AM, Andrew Trick via swift-evolution <swift-evolution@swift.org> wrote:

On Jul 2, 2016, at 8:10 PM, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> wrote:

In general, if there’s controversy, I’ll stick with the existing conventions because there’s already enough to debate in this proposal.

-Andy
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

25

This is the first version of this proposal which I've had time to read. I
like it a lot overall. If I have some more time, I may try pulling the
branch and writing some code with it to see how it feels. (If we could get
a toolchain built from the branch, that might help others review it.)

Here are a handful of minor comments:

- Naming: "bindMemory(to:capacity:)", being "verb-ish", seems incongruous
with "assumingMemoryBound(to:)" and "withMemoryRebound(to:capacity:_:)".
How about "bindingMemory(to:capacity:)" ?

That was intentional. It has side-effects in the user model, so it is
actually appropriate that it forms an active verb phrase.

- Would it be possible for "+(UnsafeRawPointer, Int) -> UnsafeRawPointer"
to accept any Integer or FixedWidthInteger, rather than only Int?

- Why allow/encourage multiple calls to bindMemory on the same RawPointer?

The reason to allow it are:

0. Typed access is faster and we don't want to relegate important use
   cases to untyped access.

1. We want to be able to use a single piece of memory for typed access
   to T and then reused for typed access to U as long as it is properly
   aligned and the T has been destroyed, without having to return it to
   the allocator.

2. We want to support the use of known layout compatibility to safely
   deal with type impedance mismatches (like CChar/Int8/UInt8 and
   (Double,Double)/Complex) at API boundaries.

I don't know where you see it encouraged.

These APIs do a good job of making aliasing explicit by introducing data
dependencies between pointers (you can only use a typed pointer after
obtaining it from a raw pointer, and you can recover the raw pointer for
later use by deinitializing the typed pointer). So, I would think the
guidelines should prefer bindMemory to be used only once on a particular
RawPointer value.

If you stick to the UnsafePointer APIs, that will be the effect. The
use of UnsafeRawPointer is something you only need to get into in
specialized circumstances.

And minor notes about the proposal itself:

- strideof(Int.self) is used in most examples, but sizeof(Int.self) appears
in one of them (the "normalLifetime()" example).

- I think there must be a mistake in this example, because pA is already
bound and bindMemory was only defined for untyped RawPointers:

    func testInitAB() {
      // Get a raw pointer to (A, B).
      let p = initAB()

      let pA = p.bindMemory(to: A.self, capacity: 1)
      printA(pA)

      printB((pA + 1).bindMemory(to: B.self, capacity: 1)) //<<< should
this be (p+1) rather than (pA+1)?
    }

Good eye, but it's not (p + 1). You actually need to move p by the
least value >= MemoryLayout<A>.size that leaves the pointer properly
aligned for B.

···

on Mon Jul 04 2016, Jacob Bandes-Storch <swift-evolution@swift.org> wrote:

Jacob

On Mon, Jul 4, 2016 at 3:32 PM, Andrew Trick via swift-evolution < > swift-evolution@swift.org> wrote:

On Jun 28, 2016, at 11:05 PM, Chris Lattner <clattner@apple.com> wrote:

Hello Swift community,

The review of “SE-0107: UnsafeRawPointer API” begins now and runs through
July 4, 2016. The proposal is available here:

https://github.com/apple/swift-evolution/blob/master/proposals/0107-unsaferawpointer.md

I've revised the proposal again based on extremely helpful feedback from
DaveA and Jordan.

This revision expands on the concept of formally binding the memory type
that I was recently working on with Dmitri. Now we can clearly define pre
and post conditions on memory operations and pointer casts that can be used
to prove the type safety. The model is now simpler, more complete, and easy
to reason about locally. This will help developers reason about correctness
and make it easy to implement a sanitizer that verifies the type safety of
UnsafePointer operations.

Adding safety to pointer "casts" made it possible for me to actually
simplify the allocation and initialization APIs. I think both camps,
convenience and safety, will be happy.

You can see what changed in this pull request:
Revise SE-0107: UnsafeRawPointer API (type binding) by atrick · Pull Request #408 · apple/swift-evolution · GitHub

Brief summary:

- Memory is dynamically bound to a single type.

- All typed access to memory, whether via a typed pointer or regular
  language construct, must be consistent with the memory's bound type
  (the access type must be related to the bound type). Typed access
  includes initialization, assignment, or deinitialization via a typed
  pointer.

- Memory remains bound after being deinitialized.

- Memory is implicitly bound or rebound to a type by initializing it
  via a raw pointer.

- A separate API now exists for explicity binding or rebinding memory
  to a type. This allows binding to be decoupled from initialization
  for convenience and efficiency. It also supports safe
  interoperability between APIs that used different, but layout
  compatible types.

- Using an API that accesses memory as a different type can now be
  accomplished by rebinding the memory. This effectively changes the
  type of any initialized values in memory. The compiler is still
  permitted to assume strict aliasing for accesses on either side of
  the operation that rebinds memory.

Andy

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
-Dave

26

I have a pile of naming quibbles; rather than describe them all in
prose (which turned into a mess), I've annotated parts of the "Full
UnsafeRawPointer API" section in a gist:
<https://gist.github.com/brentdax/8f4ed4decafc1d18c4441092baa13cfe

<https://gist.github.com/brentdax/8f4ed4decafc1d18c4441092baa13cfe&gt;&gt;\.

I want to call this out separately because it’s not specific to my
proposal and changes the existing UnsafePointer API.

Brent’s suggestion is to change `initialize(from:count:)` to
`initialize(from:forwardToCount:)` for symmetry with the backward
operations. It makes perfect sense to me, and it does a better job of
conveying that a sequence of elements will be read out of “from”.

Any objections?

Well, maybe we ought to think about this again. I'm not 100% sure we
should have separate APIs for these; we could compare pointers and
decide which direction to go in. The branch should be optimizable-out
in most situations, right? And where it couldn't be optimized out,
you'd currently need to branch manually.

The concern I have with “backward/forward” is that when you're shifting
something backwards in memory, you (may) need to proceed forwards, and
vice-versa. So these names are easily misinterpreted as having the
opposite to their actual meaning.

I suppose there's an argument to be made that CPUs/caches are better at
going forward than backward, and when the memory regions don't overlap
you want to go forwards. But even that is detectable at runtime. What
do you think, Andy?

···

on Tue Jul 05 2016, Andrew Trick <swift-evolution@swift.org> wrote:

On Jul 2, 2016, at 10:10 PM, Brent Royal-Gordon via swift-evolution >> <swift-evolution@swift.org> wrote:

  // I'm not happy with the asymmetry of these forwards/backwards
  // pairs.
  //
  // func initialize<T>(from: UnsafePointer<T>, count: Int)
  // -> UnsafeMutablePointer<T>
  // func initializeBackward<T>(from: UnsafePointer<T>, count: Int)
  // -> UnsafeMutablePointer<T>
  func initialize<T>(from: UnsafePointer<T>, forwardToCount: Int)
    -> UnsafeMutablePointer<T>
  func initialize<T>(from: UnsafePointer<T>, backwardFromCount: Int)
    -> UnsafeMutablePointer<T>

  // More detailed thoughts on redesigining `move` methods in the
  // email, but for now:
  //
  // func moveInitialize<T>(from: UnsafePointer<T>, count: Int)
  // -> UnsafeMutablePointer<T>
  // func moveInitializeBackward<T>(from: UnsafePointer<T>, count:
Int)
  // -> UnsafeMutablePointer<T>
  func moveInitialize<T>(from: UnsafePointer<T>, forwardToCount: Int)
    -> UnsafeMutablePointer<T>
  func moveInitialize<T>(from: UnsafePointer<T>, backwardFromCount:
Int)
    -> UnsafeMutablePointer<T>

Andy
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
-Dave

27

Typed pointers are returned only for the memory that was initialized. In this case, a single element:

  /// Initialize the memory location at `index` with `newValue` as if this
  /// memory holds at least `index` + 1 contiguous values of type `T`.
  ///
  /// Returns an `UnsafeMutablePointer<T>` the single intialized
  /// in-memory element.
  ///
  /// - Precondition: The memory at `index` is not initialized.
  ///
  /// - Precondition: The underlying pointer is properly aligned for
  /// accessing `T`.
  ///
  /// - Postcondition: The memory is initialized; the value should eventually
  /// be destroyed or moved from to avoid leaks.
  @discardableResult
  public func initialize<T>(toContiguous _: T.Type, atIndex index: Int,
    with value: T)
    -> UnsafeMutablePointer<T> {
    return (self + strideof(T.self) * index).initialize(T.self, with: value)
  }

Andy

···

On Jul 5, 2016, at 2:06 PM, Andrew Trick <atrick@apple.com> wrote:

* I don't think I understand `initialize(toContiguous:atIndex:with:)`. Does it return a typed pointer to the whole buffer, or just the one instance it initialized?

28

I agree.

The only value in having 2 APIs is that many times the user knows the regions are non-overlapping and may not want a runtime check. In that case forward copying makes more sense.

If the regions may overlap, and if the user somehow knows the relationship between pointers, then the compiler should also. Otherwise, if the pointer comparison is not done in the stdlib, we’re just forcing the user to do it, or allowing them to forget to do it.

Note that there will be no branch in the pure `initialize` case, because the regions are not allowed to overlap anyway. It’s only an issue for moveInitialize and assignInitialize.

How about removing the “BackwardFrom” APIs, adding runtime checks to the forward APIs, and possibly adding non-overlapping “performance” API’s as a separate proposal if needed.

-Andy

···

On Jul 5, 2016, at 10:48 AM, Dave Abrahams via swift-evolution <swift-evolution@swift.org> wrote:

I want to call this out separately because it’s not specific to my
proposal and changes the existing UnsafePointer API.

Brent’s suggestion is to change `initialize(from:count:)` to
`initialize(from:forwardToCount:)` for symmetry with the backward
operations. It makes perfect sense to me, and it does a better job of
conveying that a sequence of elements will be read out of “from”.

Any objections?

Well, maybe we ought to think about this again. I'm not 100% sure we
should have separate APIs for these; we could compare pointers and
decide which direction to go in. The branch should be optimizable-out
in most situations, right? And where it couldn't be optimized out,
you'd currently need to branch manually.

The concern I have with “backward/forward” is that when you're shifting
something backwards in memory, you (may) need to proceed forwards, and
vice-versa. So these names are easily misinterpreted as having the
opposite to their actual meaning.

I suppose there's an argument to be made that CPUs/caches are better at
going forward than backward, and when the memory regions don't overlap
you want to go forwards. But even that is detectable at runtime. What
do you think, Andy?

29

Comments made on that page.

···

on Tue Jul 05 2016, Andrew Trick <swift-evolution@swift.org> wrote:

On Jul 2, 2016, at 10:10 PM, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> wrote:

I have a pile of naming quibbles; rather than describe them all in
prose (which turned into a mess), I've annotated parts of the "Full
UnsafeRawPointer API" section in a gist:
<https://gist.github.com/brentdax/8f4ed4decafc1d18c4441092baa13cfe
<https://gist.github.com/brentdax/8f4ed4decafc1d18c4441092baa13cfe&gt;&gt;\.

I mostly agree with Brent's naming suggestions below. Anyone else want to weigh in?
[WIP] Revised SE-0107: UnsafeRawPointer API, initialization method names. by atrick · Pull Request #410 · apple/swift-evolution · GitHub

--
Dave

30

I don't mind `initialize(from:forwardToCount:)`, but I do have trouble with
Brent's suggestion of `initialize(from:backwardFromCount:)`. It adds
ambiguity as to whether the pointer in the first argument points to the 0th
element or the (count - 1)th element from which initializing is proceeding
backward, a problem that does not exist with the currently proposed version
`initializeBackward(from:count:)`. I don't find the symmetry wins
compelling enough to overcome that additional ambiguity.

···

On Tue, Jul 5, 2016 at 10:23 Andrew Trick via swift-evolution < swift-evolution@swift.org> wrote:

On Jul 2, 2016, at 10:10 PM, Brent Royal-Gordon via swift-evolution < > swift-evolution@swift.org> wrote:

I have a pile of naming quibbles; rather than describe them all in prose
(which turned into a mess), I've annotated parts of the "Full
UnsafeRawPointer API" section in a gist: <
https://gist.github.com/brentdax/8f4ed4decafc1d18c4441092baa13cfe&gt;\.

I want to call this out separately because it’s not specific to my
proposal and changes the existing UnsafePointer API.

Brent’s suggestion is to change `initialize(from:count:)` to
`initialize(from:forwardToCount:)` for symmetry with the backward
operations. It makes perfect sense to me, and it does a better job of
conveying that a sequence of elements will be read out of “from”.

Any objections?

  // I'm not happy with the asymmetry of these forwards/backwards
  // pairs.
  //
  // func initialize<T>(from: UnsafePointer<T>, count: Int)
  // -> UnsafeMutablePointer<T>
  // func initializeBackward<T>(from: UnsafePointer<T>, count: Int)
  // -> UnsafeMutablePointer<T>
  func initialize<T>(from: UnsafePointer<T>, forwardToCount: Int)
    -> UnsafeMutablePointer<T>
  func initialize<T>(from: UnsafePointer<T>, backwardFromCount: Int)
    -> UnsafeMutablePointer<T>

  // More detailed thoughts on redesigining `move` methods in the
  // email, but for now:
  //
  // func moveInitialize<T>(from: UnsafePointer<T>, count: Int)
  // -> UnsafeMutablePointer<T>
  // func moveInitializeBackward<T>(from: UnsafePointer<T>, count: Int)
  // -> UnsafeMutablePointer<T>
  func moveInitialize<T>(from: UnsafePointer<T>, forwardToCount: Int)
    -> UnsafeMutablePointer<T>
  func moveInitialize<T>(from: UnsafePointer<T>, backwardFromCount: Int)
    -> UnsafeMutablePointer<T>

Andy
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

31

This is the first version of this proposal which I've had time to read. I like it a lot overall. If I have some more time, I may try pulling the branch and writing some code with it to see how it feels. (If we could get a toolchain built from the branch, that might help others review it.)

The latest UnsafeRawPointer definition is on this branch:
https://github.com/atrick/swift/tree/rawptr

I plan to update it with the latest round of feedback soonish and send out a PR.
Unfortunately, the type system/stdlib changes are not on that branch, so you won't get any implicit conversions.

Here are a handful of minor comments:

- Naming: "bindMemory(to:capacity:)", being "verb-ish", seems incongruous with "assumingMemoryBound(to:)" and "withMemoryRebound(to:capacity:_:)". How about "bindingMemory(to:capacity:)” ?

DaveA is right, "bind" is active, "assumingMemoryBound" is passive. "withMemoryRebound" temporarily rebinds the type and implies that the closure may mutate state.

- Would it be possible for "+(UnsafeRawPointer, Int) -> UnsafeRawPointer" to accept any Integer or FixedWidthInteger, rather than only Int?

I'm not sure what all the tradeoffs are, but I don't think we want to index memory with non-Int sized integers. I think the user should be required to convert, which may trap.

- Why allow/encourage multiple calls to bindMemory on the same RawPointer? These APIs do a good job of making aliasing explicit by introducing data dependencies between pointers (you can only use a typed pointer after obtaining it from a raw pointer, and you can recover the raw pointer for later use by deinitializing the typed pointer). So, I would think the guidelines should prefer bindMemory to be used only once on a particular RawPointer value.

DaveA's answer was good. "bindMemory" is not really encouraged by the API. It's just mentioned a lot in the proposal because the semantics are interesting.

"bindMemory" is important because it's the only safe way to reinterpret in-memory values. So you need to use it to pass a UInt8 array off as a CChar, and so forth, which is actually quite common.

It is certainly encouraged more than unsafeBitCast(ptr, to: ...) because `bindMemory` is actually safe as long as you know the layout of the types and don't reuse your old typed pointers.

And minor notes about the proposal itself:

- strideof(Int.self) is used in most examples, but sizeof(Int.self) appears in one of them (the "normalLifetime()" example).

That example is actually allocating memory for a single value. Should it be changed to `strideof`? If so, then what's the point of `sizeof`?

- I think there must be a mistake in this example, because pA is already bound and bindMemory was only defined for untyped RawPointers:

    func testInitAB() {
      // Get a raw pointer to (A, B).
      let p = initAB()

      let pA = p.bindMemory(to: A.self, capacity: 1)
      printA(pA)

      printB((pA + 1).bindMemory(to: B.self, capacity: 1)) //<<< should this be (p+1) rather than (pA+1)?
    }

Thanks. That was supposed to be cast to raw pointer like the other examples:

  // Knowing the `B` has the same alignment as `A`...
  printB(UnsafeRawPointer(pA + 1).bindMemory(to: B.self, capacity: 1))

-Andy

···

On Jul 4, 2016, at 7:52 PM, Jacob Bandes-Storch <jtbandes@gmail.com> wrote:

Jacob

On Mon, Jul 4, 2016 at 3:32 PM, Andrew Trick via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Jun 28, 2016, at 11:05 PM, Chris Lattner <clattner@apple.com <mailto:clattner@apple.com>> wrote:

Hello Swift community,

The review of “SE-0107: UnsafeRawPointer API” begins now and runs through July 4, 2016. The proposal is available here:

  https://github.com/apple/swift-evolution/blob/master/proposals/0107-unsaferawpointer.md

I've revised the proposal again based on extremely helpful feedback from DaveA and Jordan.

This revision expands on the concept of formally binding the memory type that I was recently working on with Dmitri. Now we can clearly define pre and post conditions on memory operations and pointer casts that can be used to prove the type safety. The model is now simpler, more complete, and easy to reason about locally. This will help developers reason about correctness and make it easy to implement a sanitizer that verifies the type safety of UnsafePointer operations.

Adding safety to pointer "casts" made it possible for me to actually simplify the allocation and initialization APIs. I think both camps, convenience and safety, will be happy.

You can see what changed in this pull request:
Revise SE-0107: UnsafeRawPointer API (type binding) by atrick · Pull Request #408 · apple/swift-evolution · GitHub

Brief summary:

- Memory is dynamically bound to a single type.

- All typed access to memory, whether via a typed pointer or regular
  language construct, must be consistent with the memory's bound type
  (the access type must be related to the bound type). Typed access
  includes initialization, assignment, or deinitialization via a typed
  pointer.

- Memory remains bound after being deinitialized.

- Memory is implicitly bound or rebound to a type by initializing it
  via a raw pointer.

- A separate API now exists for explicity binding or rebinding memory
  to a type. This allows binding to be decoupled from initialization
  for convenience and efficiency. It also supports safe
  interoperability between APIs that used different, but layout
  compatible types.

- Using an API that accesses memory as a different type can now be
  accomplished by rebinding the memory. This effectively changes the
  type of any initialized values in memory. The compiler is still
  permitted to assume strict aliasing for accesses on either side of
  the operation that rebinds memory.

Andy

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

32

Using `bytes` to label the count / length / size would be inconsistent with:

  Foundation.Data.init(bytes:count:)
  <https://developer.apple.com/reference/foundation/data/1780158-init&gt;

  Foundation.Data.copyBytes(to:count:)
  <https://developer.apple.com/reference/foundation/data/1780297-copybytes&gt;

UnsafeMutableRawPointer could use a `size` or `sizeInBytes` label.
(This also applies to the `allocate` and `deallocate` methods).

-- Ben

···

On 10 Jul 2016, at 14:41, Andrew Trick via swift-evolution <swift-evolution@swift.org> wrote:

I'm revising this proposal based on last week's feedback. A few of the
additive APIs are removed and a number of UnsafePointer and
UnsafeRawPointer methods are renamed.

Here is a PR for the revision. Note that the examples in the proposal
text still need to be updated:
Revise SE-0107: rename UnsafePointer & UnsafeRawPointer methods. by atrick · Pull Request #420 · apple/swift-evolution · GitHub

I updated the short-form summary of the API:
https://github.com/atrick/swift-evolution/blob/3122ace9d2fb55072ebd7395c7353fcbf497318a/proposals/0107-unsaferawpointer.md#full-unsaferawpointer-api

The full UnsafeRawPointer API with doc comments is here:
https://github.com/atrick/swift/blob/22e3a2885e4236888ec447a7148acf633d8544f5/stdlib/public/core/UnsafeRawPointer.swift.gyb

The UnsafePointer and UnsafeRawPointer changes are on this branch:
https://github.com/atrick/swift/commits/rawptr

If you wish to comment line-by-line on the detailed docs or
implementation, you can do so here:
[WIP] SE-0107: UnsafePointer and UnsafeRawPointer API changes. by atrick · Pull Request #3437 · apple/swift · GitHub

---
The only concern I have about this version of the proposal is this method name:

  func copyBytes(from: UnsafeRawPointer, count: Int)

because `count` usually refers to a number of values. I think it should be:

  func copy(bytes: Int, from: UnsafeRawPointer)

33

I merged the last of my edits to this proposal (see the PR below), if anyone it interested in seeing where things ended up and doing last minute reviews.

-Andy

···

On Jul 10, 2016, at 6:41 AM, Andrew Trick <atrick@apple.com> wrote:

On Jul 4, 2016, at 5:32 PM, Andrew Trick via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Jun 28, 2016, at 11:05 PM, Chris Lattner <clattner@apple.com <mailto:clattner@apple.com>> wrote:

Hello Swift community,

The review of “SE-0107: UnsafeRawPointer API” begins now and runs through July 4, 2016. The proposal is available here:

  https://github.com/apple/swift-evolution/blob/master/proposals/0107-unsaferawpointer.md

I'm revising this proposal based on last week's feedback. A few of the
additive APIs are removed and a number of UnsafePointer and
UnsafeRawPointer methods are renamed.

Here is a PR for the revision. Note that the examples in the proposal
text still need to be updated:
Revise SE-0107: rename UnsafePointer & UnsafeRawPointer methods. by atrick · Pull Request #420 · apple/swift-evolution · GitHub

I updated the short-form summary of the API:
https://github.com/atrick/swift-evolution/blob/3122ace9d2fb55072ebd7395c7353fcbf497318a/proposals/0107-unsaferawpointer.md#full-unsaferawpointer-api

The full UnsafeRawPointer API with doc comments is here:
https://github.com/atrick/swift/blob/22e3a2885e4236888ec447a7148acf633d8544f5/stdlib/public/core/UnsafeRawPointer.swift.gyb

The UnsafePointer and UnsafeRawPointer changes are on this branch:
https://github.com/atrick/swift/commits/rawptr

If you wish to comment line-by-line on the detailed docs or
implementation, you can do so here:
[WIP] SE-0107: UnsafePointer and UnsafeRawPointer API changes. by atrick · Pull Request #3437 · apple/swift · GitHub

---
I should preemptively answer the question "why do UnsafeRawPointer
methods take an explicit type argument when it can be inferred?". Such
as:

  rawPtr.initializeMemory(as: Int.self, ...)

These methods don't simply operate on values of some type, they
actively bind memory to that type. At the point of use, that type
needs to be explicit to convey that fact. It's important for
readability, comprehension, and correctness. We cannot rely on type
inferrence on some expression which can change without the original
author's intervention resulting in subtle miscompiles.

---
The only concern I have about this version of the proposal is this method name:

  func copyBytes(from: UnsafeRawPointer, count: Int)

because `count` usually refers to a number of values. I think it should be:

  func copy(bytes: Int, from: UnsafeRawPointer)

-Andy

34

That looks like a +1 from DaveA.

So, my only objection is that I’m trying to establish a convention where “from:” reads from memory at a pointer and “to:” writes to memory at a pointer. Here “to” is backward because the object of the preposition is not being modified.

-Andy

···

On Jul 4, 2016, at 6:19 PM, Dave Abrahams via swift-evolution <swift-evolution@swift.org> wrote:

So Swift 3 users have already migrated to this “better” name.

I agree that initialize(to:) is consistent with the language we use
for assigning values. But grammatically, I think initialize(with:)
also makes perfect sense and is just as common.

“With” is a weak preposition with many possible interpretations, so we'd
like to avoid it. If I used “with” where “to” would have worked, I
regret it.

35

That’s a good point, but I think both forms are equally ambiguous.

-Andy

···

On Jul 5, 2016, at 11:05 AM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

I don't mind `initialize(from:forwardToCount:)`, but I do have trouble with Brent's suggestion of `initialize(from:backwardFromCount:)`. It adds ambiguity as to whether the pointer in the first argument points to the 0th element or the (count - 1)th element from which initializing is proceeding backward, a problem that does not exist with the currently proposed version `initializeBackward(from:count:)`. I don't find the symmetry wins compelling enough to overcome that additional ambiguity.

36

func initialize<T>(_: T.Type, to: T, count: Int = 1)
    -> UnsafeMutablePointer<T>

I wonder why the first parameter is needed. If one is passing literals,
it's always more Swift'y to use 'as'.

···

2016-07-05 6:00 GMT+03:00 L. Mihalkovic via swift-evolution < swift-evolution@swift.org>:

Regards
(From mobile)

On Jul 3, 2016, at 10:18 AM, Andrew Trick via swift-evolution < > swift-evolution@swift.org> wrote:

On Jul 2, 2016, at 8:10 PM, Brent Royal-Gordon via swift-evolution < > swift-evolution@swift.org> wrote:

I have a pile of naming quibbles; rather than describe them all in prose
(which turned into a mess), I've annotated parts of the "Full
UnsafeRawPointer API" section in a gist: <
https://gist.github.com/brentdax/8f4ed4decafc1d18c4441092baa13cfe&gt;\.

Let's bikeshed this easy one now... I’m curious what others think:

  // In general, I think you "initialize to" a value, not
  // "initialize with" a value. "with" is needlessly vacuous.
  //
  // func initialize<T>(_: T.Type, with: T, count: Int = 1)
  // -> UnsafeMutablePointer<T>
  func initialize<T>(_: T.Type, to: T, count: Int = 1)
    -> UnsafeMutablePointer<T>

`initialize` was recently renamed to `initialized(with:)`.

commit d96b051d28b6042adcc8b8692a918abddf211aec
Author: Dave Abrahams <dabrahams@apple.com>
Date: Tue Feb 23 15:12:24 2016 -0800

    stdlib: initializePointee(_) => initialize(with:)

    Tacking "Pointee" on just for unary operations (and especially
    operations with an optional count) created inconsistency.

So Swift 3 users have already migrated to this “better” name.

I agree that initialize(to:) is consistent with the language we use for
assigning values. But grammatically, I think initialize(with:) also makes
perfect sense and is just as common.

+1

In general, if there’s controversy, I’ll stick with the existing
conventions because there’s already enough to debate in this proposal.

-Andy

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

37

+1

···

on Tue Jul 05 2016, Andrew Trick <atrick-AT-apple.com> wrote:

On Jul 5, 2016, at 10:48 AM, Dave Abrahams via swift-evolution <swift-evolution@swift.org> wrote:

I want to call this out separately because it’s not specific to my
proposal and changes the existing UnsafePointer API.

Brent’s suggestion is to change `initialize(from:count:)` to
`initialize(from:forwardToCount:)` for symmetry with the backward
operations. It makes perfect sense to me, and it does a better job of
conveying that a sequence of elements will be read out of “from”.

Any objections?

Well, maybe we ought to think about this again. I'm not 100% sure we
should have separate APIs for these; we could compare pointers and
decide which direction to go in. The branch should be optimizable-out
in most situations, right? And where it couldn't be optimized out,
you'd currently need to branch manually.

The concern I have with “backward/forward” is that when you're shifting
something backwards in memory, you (may) need to proceed forwards, and
vice-versa. So these names are easily misinterpreted as having the
opposite to their actual meaning.

I suppose there's an argument to be made that CPUs/caches are better at
going forward than backward, and when the memory regions don't overlap
you want to go forwards. But even that is detectable at runtime. What
do you think, Andy?

I agree.

The only value in having 2 APIs is that many times the user knows the
regions are non-overlapping and may not want a runtime check. In that
case forward copying makes more sense.

If the regions may overlap, and if the user somehow knows the
relationship between pointers, then the compiler should
also. Otherwise, if the pointer comparison is not done in the stdlib,
we’re just forcing the user to do it, or allowing them to forget to do
it.

Note that there will be no branch in the pure `initialize` case,
because the regions are not allowed to overlap anyway. It’s only an
issue for moveInitialize and assignInitialize.

How about removing the “BackwardFrom” APIs, adding runtime checks to
the forward APIs, and possibly adding non-overlapping “performance”
API’s as a separate proposal if needed.

--
Dave

38

I'm not sure about that. "Initialize backward from x, count y" is
unambiguous as to how initialization starts and iterates (the first
argument), and it is clear that `count` is an end condition dissociated
from anything to do with how initialization starts and iterates.

By contrast, "Initialize from x, backward from y" associates the direction
of movement with y instead of x. Thus, y becomes the start condition (the
end condition being implicitly "to zero"), thus raising the question of
what position x is in relative to the count y.

···

On Tue, Jul 5, 2016 at 11:10 Andrew Trick <atrick@apple.com> wrote:

On Jul 5, 2016, at 11:05 AM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

I don't mind `initialize(from:forwardToCount:)`, but I do have trouble
with Brent's suggestion of `initialize(from:backwardFromCount:)`. It adds
ambiguity as to whether the pointer in the first argument points to the 0th
element or the (count - 1)th element from which initializing is proceeding
backward, a problem that does not exist with the currently proposed version
`initializeBackward(from:count:)`. I don't find the symmetry wins
compelling enough to overcome that additional ambiguity.

That’s a good point, but I think both forms are equally ambiguous.

-Andy

39

Thanks for pointing that out.

My concern is code like:

  let ptrToInt: UnsafePointer<Int32> = …
  rawPtr.copyBytes(from: ptrToInt, count: 4)

which looks a lot like 4 Int32s will be copied when only 1 Int32 will actually be copied.

Anyone care to vote on this?

Current:

  let rawPtr = UnsafeMutableRawPointer.allocate(bytes: 24)
  rawPtr.copyBytes(from: ptrToInt, count: 24)
  rawPtr.deallocate(bytes: 24)

Proposed:

  let rawPtr = UnsafeMutableRawPointer.allocate(sizeInBytes: 24)
  rawPtr.copyBytes(from: ptrToInt, sizeInBytes: 24)
  rawPtr.deallocate(sizeInBytes: 24)

-Andy

···

On Jul 11, 2016, at 12:50 AM, Ben Rimmington <me@benrimmington.com> wrote:

On 10 Jul 2016, at 14:41, Andrew Trick via swift-evolution <swift-evolution@swift.org> wrote:

I'm revising this proposal based on last week's feedback. A few of the
additive APIs are removed and a number of UnsafePointer and
UnsafeRawPointer methods are renamed.

Here is a PR for the revision. Note that the examples in the proposal
text still need to be updated:
Revise SE-0107: rename UnsafePointer & UnsafeRawPointer methods. by atrick · Pull Request #420 · apple/swift-evolution · GitHub

I updated the short-form summary of the API:
https://github.com/atrick/swift-evolution/blob/3122ace9d2fb55072ebd7395c7353fcbf497318a/proposals/0107-unsaferawpointer.md#full-unsaferawpointer-api

The full UnsafeRawPointer API with doc comments is here:
https://github.com/atrick/swift/blob/22e3a2885e4236888ec447a7148acf633d8544f5/stdlib/public/core/UnsafeRawPointer.swift.gyb

The UnsafePointer and UnsafeRawPointer changes are on this branch:
https://github.com/atrick/swift/commits/rawptr

If you wish to comment line-by-line on the detailed docs or
implementation, you can do so here:
[WIP] SE-0107: UnsafePointer and UnsafeRawPointer API changes. by atrick · Pull Request #3437 · apple/swift · GitHub

---
The only concern I have about this version of the proposal is this method name:

func copyBytes(from: UnsafeRawPointer, count: Int)

because `count` usually refers to a number of values. I think it should be:

func copy(bytes: Int, from: UnsafeRawPointer)

Using `bytes` to label the count / length / size would be inconsistent with:

  Foundation.Data.init(bytes:count:)
  <https://developer.apple.com/reference/foundation/data/1780158-init&gt;

  Foundation.Data.copyBytes(to:count:)
  <https://developer.apple.com/reference/foundation/data/1780297-copybytes&gt;

UnsafeMutableRawPointer could use a `size` or `sizeInBytes` label.
(This also applies to the `allocate` and `deallocate` methods).

— Ben

40

The use "as"?

···

Sent from my moss-covered three-handled family gradunza

On Jul 5, 2016, at 6:48 AM, Andrew Trick <atrick@apple.com> wrote:

On Jul 4, 2016, at 6:19 PM, Dave Abrahams via swift-evolution <swift-evolution@swift.org> wrote:

So Swift 3 users have already migrated to this “better” name.

I agree that initialize(to:) is consistent with the language we use
for assigning values. But grammatically, I think initialize(with:)
also makes perfect sense and is just as common.

“With” is a weak preposition with many possible interpretations, so we'd
like to avoid it. If I used “with” where “to” would have worked, I
regret it.

That looks like a +1 from DaveA.

So, my only objection is that I’m trying to establish a convention where “from:” reads from memory at a pointer and “to:” writes to memory at a pointer. Here “to” is backward because the object of the preposition is not being modified.

-Andy