[Pitch] Normalize Slice Types for Unsafe Buffers


(Nate Cook) #1

Hello all—

This is a proposal for a fairly minor change in slicing behavior for unsafe buffers.

Nate

···

------

This proposal changes Swift's typed UnsafeBufferPointers to be their own slice type, like the UnsafeRawBufferPointer types. This is a minor change in the subscript API of UnsafeBufferPointer and UnsafeMutableBufferPointer, but constitutes a change to the standard library's ABI, as it can't be solved through type aliasing.
<https://gist.github.com/natecook1000/a239f4db2704db9c5d54d540c0a09499#motivation>Motivation

The standard library has parallel pointer and buffer types for working with raw and typed memory. These types have broadly similar APIs that streamline working with pointers, as some kinds of memory manipulation involve moving back and forth between the two. One significant difference between the two groups of buffer types, however, is that while UnsafeRawBufferPointers are their own slice type, UnsafeBufferPointers use the default Slice type as a wrapper.

Using a Slice wrapper is a needless addition when working with buffers—the wrapper is most useful when used to prevent copying of a collection's stored data, but since UnsafeBufferPointers aren't owners of the memory they reference, there is no copying performed when simply creating a new buffer over a subrange of the memory. Moreover, the overhead of a Slice wrapper around an UnsafeBufferPointer is almost certainly higher than another UnsafeBufferPointer. instance.

The Slice wrapper makes using buffer pointers as parameters more cumbersome than necessary. To pass a slice of a buffer to a function taking a buffer, you need to create a new buffer manually:

func _operateOnBuffer<T>(_ buffer: UnsafeMutableBufferPointer<T>) {
    // ...
}

let buffer: UnsafeMutableBufferPointer<Int> = ...
_operateOnBuffer(buffer) // okay
_operateOnBuffer(buffer[0..<16]) // error: type mismatch
let subBuffer = UnsafeMutableBufferPointer(start: buffer, count: 16)
_operateOnBuffer(subBuffer) // okay
The wrapper complicates subscript assignment, as well. Instead of using simple assignment to copy all the elements of one buffer into a memory range of another, you must either manually create a slice or subscript the source buffer with its full range:

let biggerBuffer: UnsafeMutableBufferPointer<Int> = ...
let smallerBuffer: UnsafeMutableBufferPointer<Int> = ...

biggerBuffer[0..<smallerBuffer.count] =
    smallerBuffer[0..<smallerBuffer.count]
<https://gist.github.com/natecook1000/a239f4db2704db9c5d54d540c0a09499#proposed-solution>Proposed solution

The proposed solution is to switch the UnsafeBufferPointers to be their own slice type. This uses less overhead than the Slice type, which needs to store both the original buffer and a bounding range.

The operations above are simpler with this change:

_operateOnBuffer(buffer[0..<16]) // subscripting okay

// no need to subscript 'smallerBuffer'
biggerBuffer[0..<smallerBuffer.count] = smallerBuffer
<https://gist.github.com/natecook1000/a239f4db2704db9c5d54d540c0a09499#detailed-design>Detailed design

The change follows the example of the raw buffer pointer types:

struct UnsafeBufferPointer<Element> : Collection, ... {
    // other declarations
    subscript(bounds: Range<Int>) -> UnsafeBufferPointer {
        get {
            // check bounds
            return UnsafeMutableBufferPointer(
                start: self + bounds.lowerBound,
                count: bounds.count)
        }
    }
}

struct UnsafeMutableBufferPointer<Element> : Collection, ... {
    // other declarations
    subscript(bounds: Range<Int>) -> UnsafeMutableBufferPointer {
        get {
            // check bounds
            return UnsafeMutableBufferPointer(
                start: self + bounds.lowerBound,
                count: bounds.count)
        }
        set {
            // check bounds
            _writeBackMutableSlice(&self, bounds: bounds, slice: newValue)
        }
    }
}
<https://gist.github.com/natecook1000/a239f4db2704db9c5d54d540c0a09499#impact-on-existing-code>Impact on existing code

Any existing code that works with slices of UnsafeMutableBufferPointer and specifies the Slice type explicitly will need to change that specification. This isn't a terribly common thing to do (I can't find any in the standard library or test suite), so the impact of the change should be minor.


(Lily Ballard) #2

This sounds like a sensible idea. But there is one behavioral change you
haven't addressed, which is that this changes how indexes work on the
slice. With all other slice types that come to mind, the slice shares
the same indexes as the base, e.g.

  let ary = Array(0..<10)

  print(ary[3]) // prints 3

  print(ary[2..<5][3]) // still prints 3

UnsafeBufferPointer is indexed using 0-based integers, so with your
proposal, slicing an UnsafeBufferPointer produces a value that uses
different indexes. We could solve this by adding a new field, but that
would break the expectation that startIndex is always zero. But we can't
just ignore this problem, because algorithms that are designed around
collections may assume that slices preserve indexes.

In addition, since you point out that UnsafeRawBufferPointer is already
its own subsequence, and that type also guarantees that startIndex is
always zero, it sounds like we already have an instance of this problem
in the stdlib, and so this needs to be addressed with
UnsafeRawBufferPointer as well.

-Kevin Ballard

···

On Wed, Nov 30, 2016, at 09:15 AM, Nate Cook via swift-evolution wrote:

Hello all—

This is a proposal for a fairly minor change in slicing behavior for
unsafe buffers.
Nate

------

This proposal changes Swift's typed UnsafeBufferPointers to be their
own slice type, like the UnsafeRawBufferPointer types. This is a minor
change in the subscript API of UnsafeBufferPointer and
UnsafeMutableBufferPointer, but constitutes a change to the standard
library's ABI, as it can't be solved through type aliasing.
Motivation

The standard library has parallel pointer and buffer types for working
with raw and typed memory. These types have broadly similar APIs that
streamline working with pointers, as some kinds of memory manipulation
involve moving back and forth between the two. One significant
difference between the two groups of buffer types, however, is that
while UnsafeRawBufferPointers are their own slice type,
UnsafeBufferPointers use the default Slice type as a wrapper.
Using a Slice wrapper is a needless addition when working with
buffers—the wrapper is most useful when used to prevent copying of a
collection's stored data, but since UnsafeBufferPointers aren't owners
of the memory they reference, there is no copying performed when
simply creating a new buffer over a subrange of the memory. Moreover,
the overhead of a Slice wrapper around an UnsafeBufferPointer is
almost certainly higher than another UnsafeBufferPointer. instance.
The Slice wrapper makes using buffer pointers as parameters more
cumbersome than necessary. To pass a slice of a buffer to a function
taking a buffer, you need to create a new buffer manually:
func _operateOnBuffer<T>(_ buffer: UnsafeMutableBufferPointer<T>) {
// ... }

let buffer: UnsafeMutableBufferPointer<Int> = ...
_operateOnBuffer(buffer) // okay
_operateOnBuffer(buffer[..<16]) // error: type mismatch let
subBuffer = UnsafeMutableBufferPointer(start: buffer, count: 16)
_operateOnBuffer(subBuffer) // okay
The wrapper complicates subscript assignment, as well. Instead of
using simple assignment to copy all the elements of one buffer into a
memory range of another, you must either manually create a slice or
subscript the source buffer with its full range:
let biggerBuffer: UnsafeMutableBufferPointer<Int> = ... let
smallerBuffer: UnsafeMutableBufferPointer<Int> = ...
biggerBuffer[..<smallerBuffer.count] =
smallerBuffer[..<smallerBuffer.count]
Proposed solution

The proposed solution is to switch the UnsafeBufferPointers to be
their own slice type. This uses less overhead than the Slice type,
which needs to store both the original buffer and a bounding range.
The operations above are simpler with this change:

_operateOnBuffer(buffer[..<16]) // subscripting okay

// no need to subscript 'smallerBuffer'
biggerBuffer[..<smallerBuffer.count] = smallerBuffer
Detailed design

The change follows the example of the raw buffer pointer types:

struct UnsafeBufferPointer<Element> : Collection, ... { // other
declarations subscript(bounds: Range<Int>) -> UnsafeBufferPointer {
get { // check bounds return UnsafeMutableBufferPointer(
start: self + bounds.lowerBound, count: bounds.count) } } }

struct UnsafeMutableBufferPointer<Element> : Collection, ... { //
other declarations subscript(bounds: Range<Int>) ->
UnsafeMutableBufferPointer { get { // check bounds return
UnsafeMutableBufferPointer( start: self + bounds.lowerBound, count:
bounds.count) } set { // check bounds
_writeBackMutableSlice(&self, bounds: bounds, slice: newValue) } } }
Impact on existing code

Any existing code that works with slices of UnsafeMutableBufferPointer
and specifies the Slice type explicitly will need to change that
specification. This isn't a terribly common thing to do (I can't find
any in the standard library or test suite), so the impact of the
change should be minor.

_________________________________________________

swift-evolution mailing list

swift-evolution@swift.org

https://lists.swift.org/mailman/listinfo/swift-evolution


(Karl) #3

+1. Sensible change.

···

On 30 Nov 2016, at 18:15, Nate Cook via swift-evolution <swift-evolution@swift.org> wrote:

Hello all—

This is a proposal for a fairly minor change in slicing behavior for unsafe buffers.

Nate

------

This proposal changes Swift's typed UnsafeBufferPointers to be their own slice type, like the UnsafeRawBufferPointer types. This is a minor change in the subscript API of UnsafeBufferPointer and UnsafeMutableBufferPointer, but constitutes a change to the standard library's ABI, as it can't be solved through type aliasing.
<https://gist.github.com/natecook1000/a239f4db2704db9c5d54d540c0a09499#motivation>Motivation

The standard library has parallel pointer and buffer types for working with raw and typed memory. These types have broadly similar APIs that streamline working with pointers, as some kinds of memory manipulation involve moving back and forth between the two. One significant difference between the two groups of buffer types, however, is that while UnsafeRawBufferPointers are their own slice type, UnsafeBufferPointers use the default Slice type as a wrapper.

Using a Slice wrapper is a needless addition when working with buffers—the wrapper is most useful when used to prevent copying of a collection's stored data, but since UnsafeBufferPointers aren't owners of the memory they reference, there is no copying performed when simply creating a new buffer over a subrange of the memory. Moreover, the overhead of a Slice wrapper around an UnsafeBufferPointer is almost certainly higher than another UnsafeBufferPointer. instance.

The Slice wrapper makes using buffer pointers as parameters more cumbersome than necessary. To pass a slice of a buffer to a function taking a buffer, you need to create a new buffer manually:

func _operateOnBuffer<T>(_ buffer: UnsafeMutableBufferPointer<T>) {
    // ...
}

let buffer: UnsafeMutableBufferPointer<Int> = ...
_operateOnBuffer(buffer) // okay
_operateOnBuffer(buffer[0..<16]) // error: type mismatch
let subBuffer = UnsafeMutableBufferPointer(start: buffer, count: 16)
_operateOnBuffer(subBuffer) // okay
The wrapper complicates subscript assignment, as well. Instead of using simple assignment to copy all the elements of one buffer into a memory range of another, you must either manually create a slice or subscript the source buffer with its full range:

let biggerBuffer: UnsafeMutableBufferPointer<Int> = ...
let smallerBuffer: UnsafeMutableBufferPointer<Int> = ...

biggerBuffer[0..<smallerBuffer.count] =
    smallerBuffer[0..<smallerBuffer.count]
<https://gist.github.com/natecook1000/a239f4db2704db9c5d54d540c0a09499#proposed-solution>Proposed solution

The proposed solution is to switch the UnsafeBufferPointers to be their own slice type. This uses less overhead than the Slice type, which needs to store both the original buffer and a bounding range.

The operations above are simpler with this change:

_operateOnBuffer(buffer[0..<16]) // subscripting okay

// no need to subscript 'smallerBuffer'
biggerBuffer[0..<smallerBuffer.count] = smallerBuffer
<https://gist.github.com/natecook1000/a239f4db2704db9c5d54d540c0a09499#detailed-design>Detailed design

The change follows the example of the raw buffer pointer types:

struct UnsafeBufferPointer<Element> : Collection, ... {
    // other declarations
    subscript(bounds: Range<Int>) -> UnsafeBufferPointer {
        get {
            // check bounds
            return UnsafeMutableBufferPointer(
                start: self + bounds.lowerBound,
                count: bounds.count)
        }
    }
}

struct UnsafeMutableBufferPointer<Element> : Collection, ... {
    // other declarations
    subscript(bounds: Range<Int>) -> UnsafeMutableBufferPointer {
        get {
            // check bounds
            return UnsafeMutableBufferPointer(
                start: self + bounds.lowerBound,
                count: bounds.count)
        }
        set {
            // check bounds
            _writeBackMutableSlice(&self, bounds: bounds, slice: newValue)
        }
    }
}
<https://gist.github.com/natecook1000/a239f4db2704db9c5d54d540c0a09499#impact-on-existing-code>Impact on existing code

Any existing code that works with slices of UnsafeMutableBufferPointer and specifies the Slice type explicitly will need to change that specification. This isn't a terribly common thing to do (I can't find any in the standard library or test suite), so the impact of the change should be minor.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Dave Abrahams) #4

This sounds like a sensible idea. But there is one behavioral change you
haven't addressed, which is that this changes how indexes work on the
slice. With all other slice types that come to mind, the slice shares
the same indexes as the base, e.g.

  let ary = Array(0..<10)

  print(ary[3]) // prints 3

  print(ary[2..<5][3]) // still prints 3

This is an important invariant that we need to maintain.

UnsafeBufferPointer is indexed using 0-based integers, so with your
proposal, slicing an UnsafeBufferPointer produces a value that uses
different indexes. We could solve this by adding a new field, but that
would break the expectation that startIndex is always zero.

I'm not sure that's an expectation we're obligated to honor. Of course,
once you get into “unsafe” territory like this, breaking even the
expectations that aren't based on documented guarantees can be really
dangerous.

We probably ought to have wrapped those integers in some Index type
specific to UnsafeBufferPointer, so zero wasn't even a value.

But we can't just ignore this problem, because algorithms that are
designed around collections may assume that slices preserve indexes.

In addition, since you point out that UnsafeRawBufferPointer is already
its own subsequence, and that type also guarantees that startIndex is
always zero, it sounds like we already have an instance of this problem
in the stdlib, and so this needs to be addressed with
UnsafeRawBufferPointer as well.

Sounds like it!

···

on Wed Nov 30 2016, Kevin Ballard <swift-evolution@swift.org> wrote:

-Kevin Ballard

On Wed, Nov 30, 2016, at 09:15 AM, Nate Cook via swift-evolution wrote:

Hello all—

This is a proposal for a fairly minor change in slicing behavior for
unsafe buffers.
Nate

------

This proposal changes Swift's typed UnsafeBufferPointers to be their
own slice type, like the UnsafeRawBufferPointer types. This is a minor
change in the subscript API of UnsafeBufferPointer and
UnsafeMutableBufferPointer, but constitutes a change to the standard
library's ABI, as it can't be solved through type aliasing.
Motivation

The standard library has parallel pointer and buffer types for working
with raw and typed memory. These types have broadly similar APIs that
streamline working with pointers, as some kinds of memory manipulation
involve moving back and forth between the two. One significant
difference between the two groups of buffer types, however, is that
while UnsafeRawBufferPointers are their own slice type,
UnsafeBufferPointers use the default Slice type as a wrapper.
Using a Slice wrapper is a needless addition when working with
buffers—the wrapper is most useful when used to prevent copying of a
collection's stored data, but since UnsafeBufferPointers aren't owners
of the memory they reference, there is no copying performed when
simply creating a new buffer over a subrange of the memory. Moreover,
the overhead of a Slice wrapper around an UnsafeBufferPointer is
almost certainly higher than another UnsafeBufferPointer. instance.
The Slice wrapper makes using buffer pointers as parameters more
cumbersome than necessary. To pass a slice of a buffer to a function
taking a buffer, you need to create a new buffer manually:
func _operateOnBuffer<T>(_ buffer: UnsafeMutableBufferPointer<T>) {
// ... }

let buffer: UnsafeMutableBufferPointer<Int> = ...
_operateOnBuffer(buffer) // okay
_operateOnBuffer(buffer[..<16]) // error: type mismatch let
subBuffer = UnsafeMutableBufferPointer(start: buffer, count: 16)
_operateOnBuffer(subBuffer) // okay
The wrapper complicates subscript assignment, as well. Instead of
using simple assignment to copy all the elements of one buffer into a
memory range of another, you must either manually create a slice or
subscript the source buffer with its full range:
let biggerBuffer: UnsafeMutableBufferPointer<Int> = ... let
smallerBuffer: UnsafeMutableBufferPointer<Int> = ...
biggerBuffer[..<smallerBuffer.count] =
smallerBuffer[..<smallerBuffer.count]
Proposed solution

The proposed solution is to switch the UnsafeBufferPointers to be
their own slice type. This uses less overhead than the Slice type,
which needs to store both the original buffer and a bounding range.
The operations above are simpler with this change:

_operateOnBuffer(buffer[..<16]) // subscripting okay

// no need to subscript 'smallerBuffer'
biggerBuffer[..<smallerBuffer.count] = smallerBuffer
Detailed design

The change follows the example of the raw buffer pointer types:

struct UnsafeBufferPointer<Element> : Collection, ... { // other
declarations subscript(bounds: Range<Int>) -> UnsafeBufferPointer {
get { // check bounds return UnsafeMutableBufferPointer(
start: self + bounds.lowerBound, count: bounds.count) } } }

struct UnsafeMutableBufferPointer<Element> : Collection, ... { //
other declarations subscript(bounds: Range<Int>) ->
UnsafeMutableBufferPointer { get { // check bounds return
UnsafeMutableBufferPointer( start: self + bounds.lowerBound, count:
bounds.count) } set { // check bounds
_writeBackMutableSlice(&self, bounds: bounds, slice: newValue) } } }
Impact on existing code

Any existing code that works with slices of UnsafeMutableBufferPointer
and specifies the Slice type explicitly will need to change that
specification. This isn't a terribly common thing to do (I can't find
any in the standard library or test suite), so the impact of the
change should be minor.

_________________________________________________

swift-evolution mailing list

swift-evolution@swift.org

https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
-Dave


(Jordan Rose) #5

Since UnsafeBufferPointer is like an Array, I think it is supposed to have integer indexes from the start of the buffer. (It's what we're telling people to use as the currency type for what would be pointer/size pairs in C.) So I think Kevin's point is valid, and UnsafeBufferPointer cannot be its own slice type.

Jordan

···

On Nov 30, 2016, at 16:15, Dave Abrahams via swift-evolution <swift-evolution@swift.org> wrote:

on Wed Nov 30 2016, Kevin Ballard <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

This sounds like a sensible idea. But there is one behavioral change you
haven't addressed, which is that this changes how indexes work on the
slice. With all other slice types that come to mind, the slice shares
the same indexes as the base, e.g.

let ary = Array(0..<10)

print(ary[3]) // prints 3

print(ary[2..<5][3]) // still prints 3

This is an important invariant that we need to maintain.

UnsafeBufferPointer is indexed using 0-based integers, so with your
proposal, slicing an UnsafeBufferPointer produces a value that uses
different indexes. We could solve this by adding a new field, but that
would break the expectation that startIndex is always zero.

I'm not sure that's an expectation we're obligated to honor. Of course,
once you get into “unsafe” territory like this, breaking even the
expectations that aren't based on documented guarantees can be really
dangerous.

We probably ought to have wrapped those integers in some Index type
specific to UnsafeBufferPointer, so zero wasn't even a value.


(Nate Cook) #6

Argh, thanks to Kevin for pointing all this out!

I can see at least three approaches to resolving the inconsistency between the two sets of buffer types and correcting index sharing for raw buffer pointers.

1) Switch to using Slice as a wrapper for UnsafeRawBufferPointer.
2) Make all buffer pointers their own slices, keeping integer-based indices, but remove the zero-based index expectation. With this we'd need to add an additional stored property to keep track of the relative offset from the "original" buffer.
3) Make all buffer pointers their own slices but use a different index type. If the indices were just wrapped pointers, that would handle the index sharing without needing an additional property on the buffer. We could also maintain integer-based stridable conformance (which greatly simplifies index arithmetic), since the indices would just offset by a byte for raw buffers or a stride for typed buffers.

The first option would certainly be the smallest change; the third seems like it would do the most good, for the reasons Dave laid out. I've tried the third option out here for raw buffers:
  https://github.com/apple/swift/compare/master...natecook1000:nc-buffer-indices

Thanks!
Nate

···

On Nov 30, 2016, at 6:15 PM, Dave Abrahams via swift-evolution <swift-evolution@swift.org> wrote:

on Wed Nov 30 2016, Kevin Ballard <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

This sounds like a sensible idea. But there is one behavioral change you
haven't addressed, which is that this changes how indexes work on the
slice. With all other slice types that come to mind, the slice shares
the same indexes as the base, e.g.

let ary = Array(0..<10)

print(ary[3]) // prints 3

print(ary[2..<5][3]) // still prints 3

This is an important invariant that we need to maintain.

UnsafeBufferPointer is indexed using 0-based integers, so with your
proposal, slicing an UnsafeBufferPointer produces a value that uses
different indexes. We could solve this by adding a new field, but that
would break the expectation that startIndex is always zero.

I'm not sure that's an expectation we're obligated to honor. Of course,
once you get into “unsafe” territory like this, breaking even the
expectations that aren't based on documented guarantees can be really
dangerous.

We probably ought to have wrapped those integers in some Index type
specific to UnsafeBufferPointer, so zero wasn't even a value.

But we can't just ignore this problem, because algorithms that are
designed around collections may assume that slices preserve indexes.

In addition, since you point out that UnsafeRawBufferPointer is already
its own subsequence, and that type also guarantees that startIndex is
always zero, it sounds like we already have an instance of this problem
in the stdlib, and so this needs to be addressed with
UnsafeRawBufferPointer as well.

Sounds like it!


(Dave Abrahams) #7

I'm not sure it's important that the indices be integers. Once we get
into this unsafe corner of the language, it might be better if they
weren't.

···

on Thu Dec 01 2016, Jordan Rose <swift-evolution@swift.org> wrote:

On Nov 30, 2016, at 16:15, Dave Abrahams via swift-evolution > <swift-evolution@swift.org> wrote:

on Wed Nov 30 2016, Kevin Ballard >> <swift-evolution@swift.org > >> <mailto:swift-evolution@swift.org>> >> wrote:

This sounds like a sensible idea. But there is one behavioral change you
haven't addressed, which is that this changes how indexes work on the
slice. With all other slice types that come to mind, the slice shares
the same indexes as the base, e.g.

let ary = Array(0..<10)

print(ary[3]) // prints 3

print(ary[2..<5][3]) // still prints 3

This is an important invariant that we need to maintain.

UnsafeBufferPointer is indexed using 0-based integers, so with your
proposal, slicing an UnsafeBufferPointer produces a value that uses
different indexes. We could solve this by adding a new field, but that
would break the expectation that startIndex is always zero.

I'm not sure that's an expectation we're obligated to honor. Of course,
once you get into “unsafe” territory like this, breaking even the
expectations that aren't based on documented guarantees can be really
dangerous.

We probably ought to have wrapped those integers in some Index type
specific to UnsafeBufferPointer, so zero wasn't even a value.

Since UnsafeBufferPointer is like an Array, I think it is supposed to
have integer indexes from the start of the buffer.

--
-Dave


(Hooman Mehr) #8

+1 on the third option it being clearly the best way to model pointers. Making the index strideable makes its use in index computations easy. It takes a bit care to make it work as conveniently for UnsafeBufferPointer where the stride will need to be special.

···

On Dec 1, 2016, at 11:33 PM, Nate Cook via swift-evolution <swift-evolution@swift.org> wrote:

3) Make all buffer pointers their own slices but use a different index type. If the indices were just wrapped pointers, that would handle the index sharing without needing an additional property on the buffer. We could also maintain integer-based stridable conformance (which greatly simplifies index arithmetic), since the indices would just offset by a byte for raw buffers or a stride for typed buffers.

The first option would certainly be the smallest change; the third seems like it would do the most good, for the reasons Dave laid out. I've tried the third option out here for raw buffers:
  https://github.com/apple/swift/compare/master...natecook1000:nc-buffer-indices

Thanks!
Nate


(Ben Cohen) #9

Unfortunately, switching to non-integer indices would change this from being mildly source-breaking to being extremely source-breaking, as there’s lots of code out there using buffers today indexing them with integers (including integer literals).

The big win with UnsafeBufferPointer having an integer index is it’s a drop-in replacement for arrays, so when you hit a performance problem using an array you can quickly switch to using a buffer under most circumstances instead without having to change much of your code – including code that uses for i in 0..<myArray.count, of which there is a lot out there in the wild. Switching to an opaque index would break anyone doing that.

···

On Dec 1, 2016, at 11:33 PM, Nate Cook via swift-evolution <swift-evolution@swift.org> wrote:

3) Make all buffer pointers their own slices but use a different index type. If the indices were just wrapped pointers, that would handle the index sharing without needing an additional property on the buffer. We could also maintain integer-based stridable conformance (which greatly simplifies index arithmetic), since the indices would just offset by a byte for raw buffers or a stride for typed buffers.


(Hooman Mehr) #10

How about providing an Int subscript in addition to switching the index type for the purpose of Subsequence?

···

On Dec 2, 2016, at 2:12 PM, Ben Cohen via swift-evolution <swift-evolution@swift.org> wrote:

On Dec 1, 2016, at 11:33 PM, Nate Cook via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

3) Make all buffer pointers their own slices but use a different index type. If the indices were just wrapped pointers, that would handle the index sharing without needing an additional property on the buffer. We could also maintain integer-based stridable conformance (which greatly simplifies index arithmetic), since the indices would just offset by a byte for raw buffers or a stride for typed buffers.

Unfortunately, switching to non-integer indices would change this from being mildly source-breaking to being extremely source-breaking, as there’s lots of code out there using buffers today indexing them with integers (including integer literals).

The big win with UnsafeBufferPointer having an integer index is it’s a drop-in replacement for arrays, so when you hit a performance problem using an array you can quickly switch to using a buffer under most circumstances instead without having to change much of your code – including code that uses for i in 0..<myArray.count, of which there is a lot out there in the wild. Switching to an opaque index would break anyone doing that.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Nate Cook) #11

It is definitely very source-breaking, though with relatively simple fixits:

  buf[0] ---> buf[buf.startIndex]
  buf[3] ---> buf[buf.startIndex + 3]
  buf[i] ---> buf[buf.startIndex + i]

Any integer arithmetic happening outside the subscript could be left unchanged. If that cost isn't worth the benefit, then making UnsafeRawBufferPointer use Slice as its slice type is probably the best way to resolve that issue.

Nate

···

On Dec 2, 2016, at 2:12 PM, Ben Cohen via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Dec 1, 2016, at 11:33 PM, Nate Cook via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

3) Make all buffer pointers their own slices but use a different index type. If the indices were just wrapped pointers, that would handle the index sharing without needing an additional property on the buffer. We could also maintain integer-based stridable conformance (which greatly simplifies index arithmetic), since the indices would just offset by a byte for raw buffers or a stride for typed buffers.

Unfortunately, switching to non-integer indices would change this from being mildly source-breaking to being extremely source-breaking, as there’s lots of code out there using buffers today indexing them with integers (including integer literals).

The big win with UnsafeBufferPointer having an integer index is it’s a drop-in replacement for arrays, so when you hit a performance problem using an array you can quickly switch to using a buffer under most circumstances instead without having to change much of your code – including code that uses for i in 0..<myArray.count, of which there is a lot out there in the wild. Switching to an opaque index would break anyone doing that.


(Ben Cohen) #12

3) Make all buffer pointers their own slices but use a different index type. If the indices were just wrapped pointers, that would handle the index sharing without needing an additional property on the buffer. We could also maintain integer-based stridable conformance (which greatly simplifies index arithmetic), since the indices would just offset by a byte for raw buffers or a stride for typed buffers.

Unfortunately, switching to non-integer indices would change this from being mildly source-breaking to being extremely source-breaking, as there’s lots of code out there using buffers today indexing them with integers (including integer literals).

The big win with UnsafeBufferPointer having an integer index is it’s a drop-in replacement for arrays, so when you hit a performance problem using an array you can quickly switch to using a buffer under most circumstances instead without having to change much of your code – including code that uses for i in 0..<myArray.count, of which there is a lot out there in the wild. Switching to an opaque index would break anyone doing that.

It is definitely very source-breaking, though with relatively simple fixits:

  buf[0] ---> buf[buf.startIndex]
  buf[3] ---> buf[buf.startIndex + 3]
  buf[i] ---> buf[buf.startIndex + i]

Any integer arithmetic happening outside the subscript could be left unchanged. If that cost isn't worth the benefit, then making UnsafeRawBufferPointer use Slice as its slice type is probably the best way to resolve that issue.

Nate

The fixits aren’t quite that simple for slices, though:

  let slice = buf[3..<6]
  slice[3] —> slice[slice.startIndex + 0] // fixit would somehow need to know this is 0 not 3
  slice[i] —> slice[slice.startIndex + ??] // or even need to know this is, erm, I haven’t had enough coffee this morning

The other downside is it would thwart speculatively switching an Array to an UnsafeBuffer to see if that was a bottleneck, then switching back.

···

On Dec 2, 2016, at 8:27 PM, Nate Cook <natecook@gmail.com> wrote:

On Dec 2, 2016, at 2:12 PM, Ben Cohen via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Dec 1, 2016, at 11:33 PM, Nate Cook via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Dec 1, 2016, at 11:33 PM, Nate Cook via swift-evolution <swift-evolution@swift.org> wrote:

1) Switch to using Slice as a wrapper for UnsafeRawBufferPointer.

Based on the above, it seems like this is the least bad option, and we need to do this ASAP as currently UnsafeRawBufferPointer is non-compliant with the requirements of slicing and needs changing before it’s more widely adopted.


(Dave Abrahams) #13

Or we could say that UnsafeRawBufferPointer isn't a Collection. Making
it a Collection in the first place has always seemed suspect to me.

···

on Thu Dec 08 2016, Ben Cohen <ben_cohen-AT-apple.com> wrote:

On Dec 2, 2016, at 8:27 PM, Nate Cook <natecook@gmail.com> wrote:

On Dec 2, 2016, at 2:12 PM, Ben Cohen via swift-evolution >>> <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> >>> wrote:

On Dec 1, 2016, at 11:33 PM, Nate Cook via swift-evolution >>>> <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> >>>> wrote:

3) Make all buffer pointers their own slices but use a different
index type. If the indices were just wrapped pointers, that would
handle the index sharing without needing an additional property on
the buffer. We could also maintain integer-based stridable
conformance (which greatly simplifies index arithmetic), since the
indices would just offset by a byte for raw buffers or a stride
for typed buffers.

Unfortunately, switching to non-integer indices would change this
from being mildly source-breaking to being extremely
source-breaking, as there’s lots of code out there using buffers
today indexing them with integers (including integer literals).

The big win with UnsafeBufferPointer having an integer index is
it’s a drop-in replacement for arrays, so when you hit a
performance problem using an array you can quickly switch to using
a buffer under most circumstances instead without having to change
much of your code – including code that uses for i in
0..<myArray.count, of which there is a lot out there in the
wild. Switching to an opaque index would break anyone doing that.

It is definitely very source-breaking, though with relatively simple fixits:

  buf[0] ---> buf[buf.startIndex]
  buf[3] ---> buf[buf.startIndex + 3]
  buf[i] ---> buf[buf.startIndex + i]

Any integer arithmetic happening outside the subscript could be left
unchanged. If that cost isn't worth the benefit, then making
UnsafeRawBufferPointer use Slice as its slice type is probably the
best way to resolve that issue.

Nate

The fixits aren’t quite that simple for slices, though:

  let slice = buf[3..<6]
  slice[3] —> slice[slice.startIndex + 0] // fixit would somehow need to know this is 0 not 3
  slice[i] —> slice[slice.startIndex + ??] // or even need to
know this is, erm, I haven’t had enough coffee this morning

The other downside is it would thwart speculatively switching an Array
to an UnsafeBuffer to see if that was a bottleneck, then switching
back.

On Dec 1, 2016, at 11:33 PM, Nate Cook via swift-evolution <swift-evolution@swift.org> wrote:

1) Switch to using Slice as a wrapper for UnsafeRawBufferPointer.

Based on the above, it seems like this is the least bad option, and we
need to do this ASAP as currently UnsafeRawBufferPointer is
non-compliant with the requirements of slicing and needs changing
before it’s more widely adopted.

--
-Dave


(Alexis) #14

3) Make all buffer pointers their own slices but use a different
index type. If the indices were just wrapped pointers, that would
handle the index sharing without needing an additional property on
the buffer. We could also maintain integer-based stridable
conformance (which greatly simplifies index arithmetic), since the
indices would just offset by a byte for raw buffers or a stride
for typed buffers.

Unfortunately, switching to non-integer indices would change this
from being mildly source-breaking to being extremely
source-breaking, as there’s lots of code out there using buffers
today indexing them with integers (including integer literals).

The big win with UnsafeBufferPointer having an integer index is
it’s a drop-in replacement for arrays, so when you hit a
performance problem using an array you can quickly switch to using
a buffer under most circumstances instead without having to change
much of your code – including code that uses for i in
0..<myArray.count, of which there is a lot out there in the
wild. Switching to an opaque index would break anyone doing that.

It is definitely very source-breaking, though with relatively simple fixits:

   buf[0] ---> buf[buf.startIndex]
   buf[3] ---> buf[buf.startIndex + 3]
   buf[i] ---> buf[buf.startIndex + i]

Any integer arithmetic happening outside the subscript could be left
unchanged. If that cost isn't worth the benefit, then making
UnsafeRawBufferPointer use Slice as its slice type is probably the
best way to resolve that issue.

Nate

The fixits aren’t quite that simple for slices, though:

   let slice = buf[3..<6]
   slice[3] —> slice[slice.startIndex + 0] // fixit would somehow need to know this is 0 not 3
   slice[i] —> slice[slice.startIndex + ??] // or even need to
know this is, erm, I haven’t had enough coffee this morning

The other downside is it would thwart speculatively switching an Array
to an UnsafeBuffer to see if that was a bottleneck, then switching
back.

1) Switch to using Slice as a wrapper for UnsafeRawBufferPointer.

Based on the above, it seems like this is the least bad option, and we
need to do this ASAP as currently UnsafeRawBufferPointer is
non-compliant with the requirements of slicing and needs changing
before it’s more widely adopted.

Or we could say that UnsafeRawBufferPointer isn't a Collection. Making
it a Collection in the first place has always seemed suspect to me.

If this is considered a viable option, it's the one I want. Passing types without bounds checks into generic "safe" code shouldn't be this easy. You should need to explicitly wrap it up in something safe. And I really don't want the known-to-be-error-prone indexing model in concrete unsafe code.

···

On Dec 8, 2016, at 3:50 PM, Dave Abrahams via swift-evolution <swift-evolution@swift.org> wrote:
on Thu Dec 08 2016, Ben Cohen <ben_cohen-AT-apple.com> wrote:

On Dec 2, 2016, at 8:27 PM, Nate Cook <natecook@gmail.com> wrote:

On Dec 2, 2016, at 2:12 PM, Ben Cohen via swift-evolution >>>> <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> >>>> wrote:

On Dec 1, 2016, at 11:33 PM, Nate Cook via swift-evolution >>>>> <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> >>>>> wrote:

On Dec 1, 2016, at 11:33 PM, Nate Cook via swift-evolution <swift-evolution@swift.org> wrote:

--
-Dave
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Andrew Trick) #15

UnsafeRawBufferPointer does not need to be a Collection, but should at least be a Sequence. It is a Collection now simply because it fits the criteria (nondestructively accessed and subscriptable).

In practice, it needs to be able to interoperate with [UInt8] and be interchangeable in the same generic context.

e.g. `byteBuffer += rawBuffer[payloadIndex..<endIndex]` is typical.

I think Sequence is sufficient for that purpose.

I can't see any reason that generic Collection algorithms (aside from simply copying elements) would apply to UnsafeRawBufferPointer. If you don't know the element's type, you can't really apply any logic to the collection. e.g. filtering makes no sense.

We definitely do not want Collection's semantics for Slice indices, because they obviously make no sense when working with integer indices.

So, I would be happy to make UnsafeRawBufferPointer a Sequence rather than a Collection. That does fix the correctness issue with generic algorithms. But it leaves some chaos in the world.

Range subscript is an important feature for manual data layout, so we should not get rid of that. And you would never want to see that subsequence of bytes as an unnormalized view over the original buffer. (That's blatanly wrong).

With range subscript working the way it should, we still have an inconsistency between [UInt8] and UnsafeRawBufferPointer in a nongeneric context.

buffer[i..<n][0] == buffer[i]
array[i..<n][0] -> out-of-bounds

To some extent, this ship has sailed. I can see a few of options now:

1. We make UnsafeRawBufferPointer a Sequence and just live with that inconsistency. Users need discern the importance of range subscript's result type. If it's a Slice, then they should not be using integer offsets.

2. We allow UnsafeRawBufferPointer to be a Collection, use the Slice wrapper for range subscript, but add a conversion from Slice back to UnsafeRawBufferPointer that normalizes byte offsets. That way normal use cases are somewhat supported (otherwise there's no point in supporting range subscript):

readBuffer(UnsafeRawBufferPointer(subRange: buffer[i..<n]))

Is at least better than:

readBuffer(UnsafeRawBufferPointer(
  start: buffer.baseAddress?.advanced(by: i),
  count: n - i))

3. We disallow range subscript in UnsafeRawBufferPointer and add some kind of `subRange` getter:

Before:
  buffer[i..<n] = buffer[j..<(j + n - i)]

After:
  buffer.subRange(i..<n).copyBytes(from: buffer.subRange(j..<(j + n - i))

This hinges on a question that I can't answer. What's the intention behind the Slice API? Is it effectively a bug to use integer indices on a Slice in a nongeneric context? If so, then we can live with the inconcsistency in #1. If not, then I think UnsafeRawBufferPointer's range subscript should work the same way as other Collections, and for maximum consistency with other API's it really should be a Collection (#2).

-Andy

···

On Dec 8, 2016, at 12:50 PM, Dave Abrahams via swift-evolution <swift-evolution@swift.org> wrote:

on Thu Dec 08 2016, Ben Cohen <ben_cohen-AT-apple.com> wrote:

On Dec 2, 2016, at 8:27 PM, Nate Cook <natecook@gmail.com> wrote:

On Dec 2, 2016, at 2:12 PM, Ben Cohen via swift-evolution >>>> <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> >>>> wrote:

On Dec 1, 2016, at 11:33 PM, Nate Cook via swift-evolution >>>>> <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> >>>>> wrote:

3) Make all buffer pointers their own slices but use a different
index type. If the indices were just wrapped pointers, that would
handle the index sharing without needing an additional property on
the buffer. We could also maintain integer-based stridable
conformance (which greatly simplifies index arithmetic), since the
indices would just offset by a byte for raw buffers or a stride
for typed buffers.

Unfortunately, switching to non-integer indices would change this
from being mildly source-breaking to being extremely
source-breaking, as there’s lots of code out there using buffers
today indexing them with integers (including integer literals).

The big win with UnsafeBufferPointer having an integer index is
it’s a drop-in replacement for arrays, so when you hit a
performance problem using an array you can quickly switch to using
a buffer under most circumstances instead without having to change
much of your code – including code that uses for i in
0..<myArray.count, of which there is a lot out there in the
wild. Switching to an opaque index would break anyone doing that.

It is definitely very source-breaking, though with relatively simple fixits:

  buf[0] ---> buf[buf.startIndex]
  buf[3] ---> buf[buf.startIndex + 3]
  buf[i] ---> buf[buf.startIndex + i]

Any integer arithmetic happening outside the subscript could be left
unchanged. If that cost isn't worth the benefit, then making
UnsafeRawBufferPointer use Slice as its slice type is probably the
best way to resolve that issue.

Nate

The fixits aren’t quite that simple for slices, though:

  let slice = buf[3..<6]
  slice[3] —> slice[slice.startIndex + 0] // fixit would somehow need to know this is 0 not 3
  slice[i] —> slice[slice.startIndex + ??] // or even need to
know this is, erm, I haven’t had enough coffee this morning

The other downside is it would thwart speculatively switching an Array
to an UnsafeBuffer to see if that was a bottleneck, then switching
back.

On Dec 1, 2016, at 11:33 PM, Nate Cook via swift-evolution <swift-evolution@swift.org> wrote:

1) Switch to using Slice as a wrapper for UnsafeRawBufferPointer.

Based on the above, it seems like this is the least bad option, and we
need to do this ASAP as currently UnsafeRawBufferPointer is
non-compliant with the requirements of slicing and needs changing
before it’s more widely adopted.

Or we could say that UnsafeRawBufferPointer isn't a Collection. Making
it a Collection in the first place has always seemed suspect to me.

--
-Dave


(Dave Abrahams) #16

3) Make all buffer pointers their own slices but use a different
index type. If the indices were just wrapped pointers, that would
handle the index sharing without needing an additional property on
the buffer. We could also maintain integer-based stridable
conformance (which greatly simplifies index arithmetic), since the
indices would just offset by a byte for raw buffers or a stride
for typed buffers.

Unfortunately, switching to non-integer indices would change this
from being mildly source-breaking to being extremely
source-breaking, as there’s lots of code out there using buffers
today indexing them with integers (including integer literals).

The big win with UnsafeBufferPointer having an integer index is
it’s a drop-in replacement for arrays, so when you hit a
performance problem using an array you can quickly switch to using
a buffer under most circumstances instead without having to change
much of your code – including code that uses for i in
0..<myArray.count, of which there is a lot out there in the
wild. Switching to an opaque index would break anyone doing that.

It is definitely very source-breaking, though with relatively simple fixits:

   buf[0] ---> buf[buf.startIndex]
   buf[3] ---> buf[buf.startIndex + 3]
   buf[i] ---> buf[buf.startIndex + i]

Any integer arithmetic happening outside the subscript could be left
unchanged. If that cost isn't worth the benefit, then making
UnsafeRawBufferPointer use Slice as its slice type is probably the
best way to resolve that issue.

Nate

The fixits aren’t quite that simple for slices, though:

   let slice = buf[3..<6]
   slice[3] —> slice[slice.startIndex + 0] // fixit would somehow need to know this is 0 not 3
   slice[i] —> slice[slice.startIndex + ??] // or even need to
know this is, erm, I haven’t had enough coffee this morning

The other downside is it would thwart speculatively switching an Array
to an UnsafeBuffer to see if that was a bottleneck, then switching
back.

1) Switch to using Slice as a wrapper for UnsafeRawBufferPointer.

Based on the above, it seems like this is the least bad option, and we
need to do this ASAP as currently UnsafeRawBufferPointer is
non-compliant with the requirements of slicing and needs changing
before it’s more widely adopted.

Or we could say that UnsafeRawBufferPointer isn't a Collection. Making
it a Collection in the first place has always seemed suspect to me.

If this is considered a viable option, it's the one I want. Passing
types without bounds checks into generic "safe" code shouldn't be this
easy.

I don't think I agree, but that's a separate argument, about
UnsafeBufferPointer. This is about passing *untyped* memory without
bounds checks.

···

on Thu Dec 08 2016, Alexis Beingessner <swift-evolution@swift.org> wrote:

On Dec 8, 2016, at 3:50 PM, Dave Abrahams via swift-evolution <swift-evolution@swift.org> wrote:
on Thu Dec 08 2016, Ben Cohen <ben_cohen-AT-apple.com> wrote:

On Dec 2, 2016, at 8:27 PM, Nate Cook <natecook@gmail.com> wrote:

On Dec 2, 2016, at 2:12 PM, Ben Cohen via swift-evolution >>>>> <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> >>>>> wrote:

On Dec 1, 2016, at 11:33 PM, Nate Cook via swift-evolution >>>>>> <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> >>>>>> wrote:

On Dec 1, 2016, at 11:33 PM, Nate Cook via swift-evolution <swift-evolution@swift.org> wrote:

You should need to explicitly wrap it up in something safe. And I
really don't want the known-to-be-error-prone indexing model in
concrete unsafe code.

--
-Dave


(Jordan Rose) #17

Um, Sequence doesn’t have a subscript (or indexes). Sequences are single-pass. So if this is important, it needs to stay a Collection.

Jordan

···

On Dec 8, 2016, at 16:22, Andrew Trick via swift-evolution <swift-evolution@swift.org> wrote:

In practice, it needs to be able to interoperate with [UInt8] and be interchangeable in the same generic context.

e.g. `byteBuffer += rawBuffer[payloadIndex..<endIndex]` is typical.

I think Sequence is sufficient for that purpose.


(Xiaodi Wu) #18

Can you elaborate on this? Why aren't you sure this is a wise idea?

···

On Thu, Dec 8, 2016 at 16:19 Dave Abrahams via swift-evolution < swift-evolution@swift.org> wrote:

on Thu Dec 08 2016, Alexis Beingessner <swift-evolution@swift.org> wrote:

>> On Dec 8, 2016, at 3:50 PM, Dave Abrahams via swift-evolution < > swift-evolution@swift.org> wrote:
>>
>>
>> on Thu Dec 08 2016, Ben Cohen <ben_cohen-AT-apple.com> wrote:
>>
>
>>>> On Dec 2, 2016, at 8:27 PM, Nate Cook <natecook@gmail.com> wrote:
>>>>
>>>>> On Dec 2, 2016, at 2:12 PM, Ben Cohen via swift-evolution > >>>>> <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> > >>>>> wrote:
>>>
>>>>>
>>>>>> On Dec 1, 2016, at 11:33 PM, Nate Cook via swift-evolution > >>>>>> <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> > >>>>>> wrote:
>>>>>>
>>>>>> 3) Make all buffer pointers their own slices but use a different
>>>>>> index type. If the indices were just wrapped pointers, that would
>>>>>> handle the index sharing without needing an additional property on
>>>>>> the buffer. We could also maintain integer-based stridable
>>>>>> conformance (which greatly simplifies index arithmetic), since the
>>>>>> indices would just offset by a byte for raw buffers or a stride
>>>>>> for typed buffers.
>>>>>>
>>>>>
>>>>> Unfortunately, switching to non-integer indices would change this
>>>>> from being mildly source-breaking to being extremely
>>>>> source-breaking, as there’s lots of code out there using buffers
>>>>> today indexing them with integers (including integer literals).
>>>>>
>>>>> The big win with UnsafeBufferPointer having an integer index is
>>>>> it’s a drop-in replacement for arrays, so when you hit a
>>>>> performance problem using an array you can quickly switch to using
>>>>> a buffer under most circumstances instead without having to change
>>>>> much of your code – including code that uses for i in
>>>>> 0..<myArray.count, of which there is a lot out there in the
>>>>> wild. Switching to an opaque index would break anyone doing that.
>>>>
>>>> It is definitely very source-breaking, though with relatively simple
fixits:
>>>>
>>>> buf[0] ---> buf[buf.startIndex]
>>>> buf[3] ---> buf[buf.startIndex + 3]
>>>> buf[i] ---> buf[buf.startIndex + i]
>>>>
>>>> Any integer arithmetic happening outside the subscript could be left
>>>> unchanged. If that cost isn't worth the benefit, then making
>>>> UnsafeRawBufferPointer use Slice as its slice type is probably the
>>>> best way to resolve that issue.
>>>>
>>>> Nate
>>>
>>> The fixits aren’t quite that simple for slices, though:
>>>
>>> let slice = buf[3..<6]
>>> slice[3] —> slice[slice.startIndex + 0] // fixit would somehow need
to know this is 0 not 3
>>> slice[i] —> slice[slice.startIndex + ??] // or even need to
>>> know this is, erm, I haven’t had enough coffee this morning
>>>
>>> The other downside is it would thwart speculatively switching an Array
>>> to an UnsafeBuffer to see if that was a bottleneck, then switching
>>> back.
>>>
>>>> On Dec 1, 2016, at 11:33 PM, Nate Cook via swift-evolution < > swift-evolution@swift.org> wrote:
>>>>
>>>> 1) Switch to using Slice as a wrapper for UnsafeRawBufferPointer.
>>>>
>>>
>>> Based on the above, it seems like this is the least bad option, and we
>>> need to do this ASAP as currently UnsafeRawBufferPointer is
>>> non-compliant with the requirements of slicing and needs changing
>>> before it’s more widely adopted.
>>
>> Or we could say that UnsafeRawBufferPointer isn't a Collection. Making
>> it a Collection in the first place has always seemed suspect to me.
>>
>
> If this is considered a viable option, it's the one I want. Passing
> types without bounds checks into generic "safe" code shouldn't be this
> easy.

I don't think I agree, but that's a separate argument, about
UnsafeBufferPointer. This is about passing *untyped* memory without
bounds checks.

> You should need to explicitly wrap it up in something safe. And I
> really don't want the known-to-be-error-prone indexing model in
> concrete unsafe code.

--
-Dave

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Dave Abrahams) #19

Whom are you asking? What is “this?”

···

on Thu Dec 08 2016, Xiaodi Wu <xiaodi.wu-AT-gmail.com> wrote:

Can you elaborate on this? Why aren't you sure this is a wise idea?

--
-Dave


(Ben Cohen) #20

Just because something fulfills one of the requirements of a Collection does not mean it should be one. It needs to tick all the boxes before its allowed to be elevated.

But it’s still allowed to have subscripts (UnsafePointer has subscripting but isn’t a collection) or be multi-pass (strides are multiples but are only sequences). That’s OK

In this case, yes it’s multi-pass, yes it has a subscript, but no it isn’t a collection because it doesn’t meet the requirements for slicing i.e. that indices of the slice be indices of the parent.
(relatedly… it appears this requirement is documented on the concrete Slice type rather than on Collection… which is a documentation bug we should fix).

···

On Dec 8, 2016, at 4:35 PM, Jordan Rose via swift-evolution <swift-evolution@swift.org> wrote:

Um, Sequence doesn’t have a subscript (or indexes). Sequences are single-pass. So if this is important, it needs to stay a Collection.