Proposal: One-past-end array pointers and convertibility of nil for interaction with C APIs

Evolution Discussion
1

Hi everyone,

I was recently trying to use a C API (LLVM for the record) that required
passing an array to a function in the form of a pointer and a size. I
couldn't find a straightforward way to pass a null pointer to the function
in question conditionally (when the array is empty), since the following –
simplified – code doesn't currently typecheck:

    // C function with signature: void foo(T *ptr, unsigned size)
    // imported into Swift as: (UnsafeMutablePointer<T>, UInt32) -> ()
    var arr: [T] =
    foo(arr.count > 0 ? &arr[0] : nil, UInt32(arr.count))

The error is: result values in '? :' expression have mismatching types
'inout T' and '_'

This does not make sense since although `nil` is typeless by itself, its
concrete type should still be able to be inferred from the context (just
like it is inferred correctly if one writes

    condition ? 1 as Optional<Int> : nil

which is an analogous scenario.)

Since the inout operator (&) can only be used in function call arguments
(so it's not exactly C's address-of), I believe that there's no easy way of
elegantly passing a null pointer when the array is empty. (Yes, I could
write two almost-identical calls, but meh…) And even if there is one (and
I'm just missing it), the fact that the above code does not work seems
inconsistent to me.

I also realized that this specific issue generalizes to the (in)ability of
passing one-past-end pointers – which would be equally valid and even more
convenient in the above case, as the callee does not dereference the passed
pointer when the count is 0, but in general, it can be applied to functions
accepting [begin, end + 1) ranges.

The problem here is that a one-past-end pointer does not reside at a valid
index (pretty much by definition), so bounds checking kicks in and kills
the program.

My proposed solutions:

– Extend type inference for unsafe pointers and nil, so that when a value
is passed by address to a function, it's not only the result of an
&-expression that has its type inferred to be (or implicitly converted to)
Unsafe[Mutable]Pointer, but if there's a nil somewhere around, such as the
one in the example above, it gets promoted to that type too, just like NULL
in C or nullptr in C++.

– Stop overloading the inout '&' operator and using it for C-style
address-of operations. I could imagine a similar, but distinct operator or
even a library function (something along the lines of unsafeAddressOf) that
specifically yields the physical address of its operand as an unsafe C
pointer, and which is thus first-class in the sense that it may be used
anywhere other expressions may be, not just as immediate call arguments.

– Make array bounds checking more lenient when passing pointers to array
elements into C functions. Bounds checking should, in these cases, allow
indexing the one-past-end element of an array if (and only if) it is the
argument of the address-of operator.

Comments and questions are welcome (you might need clarification, as it's
2:35 AM here when I'm writing this…)

Cheers,

···

--
Author of the Sparkling language
http://h2co3.org/

2

The & operator isn't exactly an address-of operator. Does &arr[0] even return a pointer to the inner buffer? When you use & with properties (and possibly with subscripts as well), Swift may create a local, copy the property value to it, pass a pointer to that local, and copy back the output to the property.

Anyway, you are probably looking for Array.withUnsafe(Mutable?)BufferPointer:

arr.withUnsafeMutableBufferPointer { foo($0, $0.count) }

For most purposes, foo(arr, arr.count) (or foo(&arr, arr.count) if you need to mutate the array) does the same thing.

Félix

Hi everyone,

I was recently trying to use a C API (LLVM for the record) that required passing an array to a function in the form of a pointer and a size. I couldn't find a straightforward way to pass a null pointer to the function in question conditionally (when the array is empty), since the following – simplified – code doesn't currently typecheck:

    // C function with signature: void foo(T *ptr, unsigned size)
    // imported into Swift as: (UnsafeMutablePointer<T>, UInt32) -> ()
    var arr: [T] =
    foo(arr.count > 0 ? &arr[0] : nil, UInt32(arr.count))

The error is: result values in '? :' expression have mismatching types 'inout T' and '_'

This does not make sense since although `nil` is typeless by itself, its concrete type should still be able to be inferred from the context (just like it is inferred correctly if one writes

    condition ? 1 as Optional<Int> : nil

which is an analogous scenario.)

Since the inout operator (&) can only be used in function call arguments (so it's not exactly C's address-of), I believe that there's no easy way of elegantly passing a null pointer when the array is empty. (Yes, I could write two almost-identical calls, but meh…) And even if there is one (and I'm just missing it), the fact that the above code does not work seems inconsistent to me.

I also realized that this specific issue generalizes to the (in)ability of passing one-past-end pointers – which would be equally valid and even more convenient in the above case, as the callee does not dereference the passed pointer when the count is 0, but in general, it can be applied to functions accepting [begin, end + 1) ranges.

The problem here is that a one-past-end pointer does not reside at a valid index (pretty much by definition), so bounds checking kicks in and kills the program.

My proposed solutions:

– Extend type inference for unsafe pointers and nil, so that when a value is passed by address to a function, it's not only the result of an &-expression that has its type inferred to be (or implicitly converted to) Unsafe[Mutable]Pointer, but if there's a nil somewhere around, such as the one in the example above, it gets promoted to that type too, just like NULL in C or nullptr in C++.

– Stop overloading the inout '&' operator and using it for C-style address-of operations. I could imagine a similar, but distinct operator or even a library function (something along the lines of unsafeAddressOf) that specifically yields the physical address of its operand as an unsafe C pointer, and which is thus first-class in the sense that it may be used anywhere other expressions may be, not just as immediate call arguments.

– Make array bounds checking more lenient when passing pointers to array elements into C functions. Bounds checking should, in these cases, allow indexing the one-past-end element of an array if (and only if) it is the argument of the address-of operator.

Comments and questions are welcome (you might need clarification, as it's 2:35 AM here when I'm writing this…)

Cheers,

--
Author of the Sparkling language
http://h2co3.org/

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

-Dave

···

On Dec 23, 2015, at 5:50 PM, Félix Cloutier via swift-evolution <swift-evolution@swift.org> wrote:

Le 23 déc. 2015 à 20:35:09, Árpád Goretity via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> a écrit :

3

The & operator isn't exactly an address-of operator. Does &arr[0] even return a pointer to the inner buffer? When you use & with properties (and possibly with subscripts as well), Swift may create a local, copy the property value to it, pass a pointer to that local, and copy back the output to the property.

Anyway, you are probably looking for Array.withUnsafe(Mutable?)BufferPointer:

arr.withUnsafeMutableBufferPointer { foo($0, $0.count) }

Félix

···

Le 23 déc. 2015 à 20:35:09, Árpád Goretity via swift-evolution <swift-evolution@swift.org> a écrit :

Hi everyone,

I was recently trying to use a C API (LLVM for the record) that required passing an array to a function in the form of a pointer and a size. I couldn't find a straightforward way to pass a null pointer to the function in question conditionally (when the array is empty), since the following – simplified – code doesn't currently typecheck:

    // C function with signature: void foo(T *ptr, unsigned size)
    // imported into Swift as: (UnsafeMutablePointer<T>, UInt32) -> ()
    var arr: [T] =
    foo(arr.count > 0 ? &arr[0] : nil, UInt32(arr.count))

The error is: result values in '? :' expression have mismatching types 'inout T' and '_'

This does not make sense since although `nil` is typeless by itself, its concrete type should still be able to be inferred from the context (just like it is inferred correctly if one writes

    condition ? 1 as Optional<Int> : nil

which is an analogous scenario.)

Since the inout operator (&) can only be used in function call arguments (so it's not exactly C's address-of), I believe that there's no easy way of elegantly passing a null pointer when the array is empty. (Yes, I could write two almost-identical calls, but meh…) And even if there is one (and I'm just missing it), the fact that the above code does not work seems inconsistent to me.

I also realized that this specific issue generalizes to the (in)ability of passing one-past-end pointers – which would be equally valid and even more convenient in the above case, as the callee does not dereference the passed pointer when the count is 0, but in general, it can be applied to functions accepting [begin, end + 1) ranges.

The problem here is that a one-past-end pointer does not reside at a valid index (pretty much by definition), so bounds checking kicks in and kills the program.

My proposed solutions:

– Extend type inference for unsafe pointers and nil, so that when a value is passed by address to a function, it's not only the result of an &-expression that has its type inferred to be (or implicitly converted to) Unsafe[Mutable]Pointer, but if there's a nil somewhere around, such as the one in the example above, it gets promoted to that type too, just like NULL in C or nullptr in C++.

– Stop overloading the inout '&' operator and using it for C-style address-of operations. I could imagine a similar, but distinct operator or even a library function (something along the lines of unsafeAddressOf) that specifically yields the physical address of its operand as an unsafe C pointer, and which is thus first-class in the sense that it may be used anywhere other expressions may be, not just as immediate call arguments.

– Make array bounds checking more lenient when passing pointers to array elements into C functions. Bounds checking should, in these cases, allow indexing the one-past-end element of an array if (and only if) it is the argument of the address-of operator.

Comments and questions are welcome (you might need clarification, as it's 2:35 AM here when I'm writing this…)

Cheers,

--
Author of the Sparkling language
http://h2co3.org/

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

4

The comments in the generated header for UnsafeMutablePointer claim that its regular init() method constructs a null pointer. Therefore, I think you should just be able to:

foo(UnsafeMutablePointer<T>(), 0)

Charles

···

On Dec 23, 2015, at 7:50 PM, Félix Cloutier via swift-evolution <swift-evolution@swift.org> wrote:

The & operator isn't exactly an address-of operator. Does &arr[0] even return a pointer to the inner buffer? When you use & with properties (and possibly with subscripts as well), Swift may create a local, copy the property value to it, pass a pointer to that local, and copy back the output to the property.

Anyway, you are probably looking for Array.withUnsafe(Mutable?)BufferPointer:

arr.withUnsafeMutableBufferPointer { foo($0, $0.count) }

Félix

5

The & operator isn't exactly an address-of operator

Yes, exactly! And that's why I think it should not sometimes work like that.

From what I can tell, when passing the address of an array element to a C

function via unsafe pointers, no temporary is created (as it is the case
when passing an inout parameter to other Swift functions).

By the way, passing the array as inout also works in my particular case; it
doesn't address the issue of one-past-end pointers, though.

···

On Thu, Dec 24, 2015 at 3:01 AM, Árpád Goretity <arpad.goretity@gmail.com> wrote:

> The & operator isn't exactly an address-of operator

Yes, exactly! And that's why I think it should not sometimes work like
that.

From what I can tell, when passing the address of an array element to a C
function via unsafe pointers, no temporary is created (as it is the case
when passing an inout parameter to other Swift functions).

On Thu, Dec 24, 2015 at 2:48 AM, Félix Cloutier < > felix.cloutier.1@ens.etsmtl.ca> wrote:

The & operator isn't exactly an address-of operator. Does &arr[0] even
return a pointer to the inner buffer? When you use & with properties (and
possibly with subscripts as well), Swift may create a local, copy the
property value to it, pass a pointer to that local, and copy back the
output to the property.

Anyway, you are probably looking for
Array.withUnsafe(Mutable?)BufferPointer:

arr.withUnsafeMutableBufferPointer { foo($0, $0.count) }

Le 23 déc. 2015 à 20:35:09, Árpád Goretity via swift-evolution < >> swift-evolution@swift.org> a écrit :

Hi everyone,

I was recently trying to use a C API (LLVM for the record) that required
passing an array to a function in the form of a pointer and a size. I
couldn't find a straightforward way to pass a null pointer to the function
in question conditionally (when the array is empty), since the following –
simplified – code doesn't currently typecheck:

    // C function with signature: void foo(T *ptr, unsigned size)
    // imported into Swift as: (UnsafeMutablePointer<T>, UInt32) -> ()
    var arr: [T] =
    foo(arr.count > 0 ? &arr[0] : nil, UInt32(arr.count))

The error is: result values in '? :' expression have mismatching types
'inout T' and '_'

This does not make sense since although `nil` is typeless by itself, its
concrete type should still be able to be inferred from the context (just
like it is inferred correctly if one writes

    condition ? 1 as Optional<Int> : nil

which is an analogous scenario.)

Since the inout operator (&) can only be used in function call arguments
(so it's not exactly C's address-of), I believe that there's no easy way of
elegantly passing a null pointer when the array is empty. (Yes, I could
write two almost-identical calls, but meh…) And even if there is one (and
I'm just missing it), the fact that the above code does not work seems
inconsistent to me.

I also realized that this specific issue generalizes to the (in)ability
of passing one-past-end pointers – which would be equally valid and even
more convenient in the above case, as the callee does not dereference the
passed pointer when the count is 0, but in general, it can be applied to
functions accepting [begin, end + 1) ranges.

The problem here is that a one-past-end pointer does not reside at a
valid index (pretty much by definition), so bounds checking kicks in and
kills the program.

My proposed solutions:

– Extend type inference for unsafe pointers and nil, so that when a
value is passed by address to a function, it's not only the result of an
&-expression that has its type inferred to be (or implicitly converted to)
Unsafe[Mutable]Pointer, but if there's a nil somewhere around, such as the
one in the example above, it gets promoted to that type too, just like NULL
in C or nullptr in C++.

– Stop overloading the inout '&' operator and using it for C-style
address-of operations. I could imagine a similar, but distinct operator or
even a library function (something along the lines of unsafeAddressOf) that
specifically yields the physical address of its operand as an unsafe C
pointer, and which is thus first-class in the sense that it may be used
anywhere other expressions may be, not just as immediate call arguments.

– Make array bounds checking more lenient when passing pointers to array
elements into C functions. Bounds checking should, in these cases, allow
indexing the one-past-end element of an array if (and only if) it is the
argument of the address-of operator.

Comments and questions are welcome (you might need clarification, as it's
2:35 AM here when I'm writing this…)

Cheers,

--
Author of the Sparkling language
http://h2co3.org/

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
Author of the Sparkling language
http://h2co3.org/

--
Author of the Sparkling language
http://h2co3.org/

6

The comments in the generated header for UnsafeMutablePointer claim that

its regular init() method constructs a null pointer.

That's right, and I have tried it as well. Yet, type inference doesn't like
like it, and I get back an error similar to what I have already described,
but this time with inout and UnsafeMutablePointer:

    result values in '? :' expression have mismatching types 'inout T' and
'UnsafeMutablePointer<T>'

···

On Thu, Dec 24, 2015 at 7:56 AM, Charles Srstka <cocoadev@charlessoft.com> wrote:

On Dec 23, 2015, at 7:50 PM, Félix Cloutier via swift-evolution < > swift-evolution@swift.org> wrote:

The & operator isn't exactly an address-of operator. Does &arr[0] even
return a pointer to the inner buffer? When you use & with properties (and
possibly with subscripts as well), Swift may create a local, copy the
property value to it, pass a pointer to that local, and copy back the
output to the property.

Anyway, you are probably looking for
Array.withUnsafe(Mutable?)BufferPointer:

arr.withUnsafeMutableBufferPointer { foo($0, $0.count) }

Félix

The comments in the generated header for UnsafeMutablePointer claim that
its regular init() method constructs a null pointer. Therefore, I think you
should just be able to:

foo(UnsafeMutablePointer<T>(), 0)

Charles

--
Author of the Sparkling language
http://h2co3.org/

7

Hi everyone,

I was recently trying to use a C API (LLVM for the record) that required passing an array to a function in the form of a pointer and a size. I couldn't find a straightforward way to pass a null pointer to the function in question conditionally (when the array is empty), since the following – simplified – code doesn't currently typecheck:

    // C function with signature: void foo(T *ptr, unsigned size)
    // imported into Swift as: (UnsafeMutablePointer<T>, UInt32) -> ()
    var arr: [T] =
    foo(arr.count > 0 ? &arr[0] : nil, UInt32(arr.count))

The error is: result values in '? :' expression have mismatching types 'inout T' and '_'

The diagnostic here sucks. The inout-to-pointer is only available to function argument expressions; it's not even considered in a ternary expression here, so the type checker can't find any way to match 'nil' and an inout.

Since the inout operator (&) can only be used in function call arguments (so it's not exactly C's address-of), I believe that there's no easy way of elegantly passing a null pointer when the array is empty. (Yes, I could write two almost-identical calls, but meh…) And even if there is one (and I'm just missing it), the fact that the above code does not work seems inconsistent to me.

I also realized that this specific issue generalizes to the (in)ability of passing one-past-end pointers – which would be equally valid and even more convenient in the above case, as the callee does not dereference the passed pointer when the count is 0, but in general, it can be applied to functions accepting [begin, end + 1) ranges.

The problem here is that a one-past-end pointer does not reside at a valid index (pretty much by definition), so bounds checking kicks in and kills the program.

Past-the-end indices are valid pointers (and valid in Swift collections in general). That's not the problem. `&arr[0]` fails because it's providing a temporary buffer connected only to the *element* &arr[0], rather than a buffer representing the entire array. This won't do what you expect for any Swift array, even if it's non-empty.

In general, we can't support fully first-class pointers into managed Swift entities like Array and properties, without breaking the encapsulation of those abstractions. We can provide scoped operations like `withUnsafePointer` that give you a pointer to a possibly-temporary buffer that represents the value of that array or value for the duration of a block. When you say `CFunctionThatTakesPointer(&a)`, Swift's really wrapping that call in the equivalent of `withUnsafeMutableBufferPointer` on your behalf. You can see how that would be problematic if the wrapping needs to be conditional, such as if it appeared in a ternary or &&/|| expression. I think the thing to do is to make two calls:

if arr.empty {
  foo(nil, 0)
} else {
  foo(&arr, arr.count)
}

since preparing the buffer for the pointer itself isn't necessarily free, and you'd want to avoid that work if you don't need it.

It might be OK to have the pointer produced for an empty array be null to begin with, which would avoid the need for this conditional at all. In most cases, you can't safely dereference a pointer to nothing anyway. I'm also sympathetic to the idea of disconnecting "address-of" and "inout", since it often leads to confusion like this.

-Joe

···

On Dec 23, 2015, at 5:35 PM, Árpád Goretity via swift-evolution <swift-evolution@swift.org> wrote:

My proposed solutions:

– Extend type inference for unsafe pointers and nil, so that when a value is passed by address to a function, it's not only the result of an &-expression that has its type inferred to be (or implicitly converted to) Unsafe[Mutable]Pointer, but if there's a nil somewhere around, such as the one in the example above, it gets promoted to that type too, just like NULL in C or nullptr in C++.

– Stop overloading the inout '&' operator and using it for C-style address-of operations. I could imagine a similar, but distinct operator or even a library function (something along the lines of unsafeAddressOf) that specifically yields the physical address of its operand as an unsafe C pointer, and which is thus first-class in the sense that it may be used anywhere other expressions may be, not just as immediate call arguments.

– Make array bounds checking more lenient when passing pointers to array elements into C functions. Bounds checking should, in these cases, allow indexing the one-past-end element of an array if (and only if) it is the argument of the address-of operator.

Comments and questions are welcome (you might need clarification, as it's 2:35 AM here when I'm writing this…)

Cheers,

--
Author of the Sparkling language
http://h2co3.org/

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

8

That’s because you’re putting it in a ternary expression. Swift has magic to automatically turn inout arguments (which is what you get when you put a & in front of an argument) into Unsafe(Mutable)Pointers if that’s what a function asks for. However, you’re not getting to that point, because you’re putting an inout argument and an UnsafeMutablePointer on opposite sides of the ternary operator. That won’t work, because the ternary operator requires both objects to be of the same type, and since your inout argument hasn’t been sent to the function yet, it hasn’t been automagically translated into a pointer.

What Félix said is also true; you can’t trust that &array[0] will get you a pointer to the array’s internal storage. It’s better to use .withUnsafeMutableBufferPointer for that.

Charles

···

On Dec 24, 2015, at 1:11 AM, Árpád Goretity <arpad.goretity@gmail.com> wrote:

> The comments in the generated header for UnsafeMutablePointer claim that its regular init() method constructs a null pointer.

That's right, and I have tried it as well. Yet, type inference doesn't like like it, and I get back an error similar to what I have already described, but this time with inout and UnsafeMutablePointer:

    result values in '? :' expression have mismatching types 'inout T' and 'UnsafeMutablePointer<T>'

9

Thanks for your answer!

I think the thing to do is to make two calls:

Eeeek, that's exactly what I wanted to avoid (which I actually think is the
worst possible solution).

The inout-to-pointer is only available to function argument expressions;

it's not even considered in a ternary expression here.

Yes, I realized that (in fact I think the diagnostic makes this quite
clear). I understand *why* it does not currently work, I'm merely
suggesting that I think it *should* work. (It might be too hard to
implement, just my opinion.)

In general, we can't support fully first-class pointers into managed

Swift entities like Array and properties, without breaking the
encapsulation of those abstractions.

Yes, that's kind of obvious – what I am actually trying to suggest is that
unsafe is already unsafe, so it might be OK to "break the encapsulation",
for example by returning a pointer to the internal buffer without
introducing a temporary.

However, if you agree about sharply distinguishing Swift inout and unsafe +
C address of (possibly in a way that my examples would work with the new
unsafe address-of operator), I'm happy to narrow down my proposal to that
specific case.

···

On Thu, Dec 24, 2015 at 9:43 PM, Árpád Goretity <arpad.goretity@gmail.com> wrote:

Thanks for your answer!

> I think the thing to do is to make two calls:

Eeeek, that's exactly which I wanted to avoid (and which I think is the
worst possible solution).

On Thu, Dec 24, 2015 at 5:26 PM, Joe Groff <jgroff@apple.com> wrote:

On Dec 23, 2015, at 5:35 PM, Árpád Goretity via swift-evolution < >> swift-evolution@swift.org> wrote:

Hi everyone,

I was recently trying to use a C API (LLVM for the record) that required
passing an array to a function in the form of a pointer and a size. I
couldn't find a straightforward way to pass a null pointer to the function
in question conditionally (when the array is empty), since the following –
simplified – code doesn't currently typecheck:

    // C function with signature: void foo(T *ptr, unsigned size)
    // imported into Swift as: (UnsafeMutablePointer<T>, UInt32) -> ()
    var arr: [T] =
    foo(arr.count > 0 ? &arr[0] : nil, UInt32(arr.count))

The error is: result values in '? :' expression have mismatching types
'inout T' and '_'

The diagnostic here sucks. The inout-to-pointer is only available to
function argument expressions; it's not even considered in a ternary
expression here, so the type checker can't find any way to match 'nil' and
an inout.

Since the inout operator (&) can only be used in function call arguments
(so it's not exactly C's address-of), I believe that there's no easy way of
elegantly passing a null pointer when the array is empty. (Yes, I could
write two almost-identical calls, but meh…) And even if there is one (and
I'm just missing it), the fact that the above code does not work seems
inconsistent to me.

I also realized that this specific issue generalizes to the (in)ability
of passing one-past-end pointers – which would be equally valid and even
more convenient in the above case, as the callee does not dereference the
passed pointer when the count is 0, but in general, it can be applied to
functions accepting [begin, end + 1) ranges.

The problem here is that a one-past-end pointer does not reside at a
valid index (pretty much by definition), so bounds checking kicks in and
kills the program.

Past-the-end indices are valid pointers (and valid in Swift collections
in general). That's not the problem. `&arr[0]` fails because it's providing
a temporary buffer connected only to the *element* &arr[0], rather than a
buffer representing the entire array. This won't do what you expect for any
Swift array, even if it's non-empty.

In general, we can't support fully first-class pointers into managed
Swift entities like Array and properties, without breaking the
encapsulation of those abstractions. We can provide scoped operations like
`withUnsafePointer` that give you a pointer to a possibly-temporary buffer
that represents the value of that array or value for the duration of a
block. When you say `CFunctionThatTakesPointer(&a)`, Swift's really
wrapping that call in the equivalent of `withUnsafeMutableBufferPointer` on
your behalf. You can see how that would be problematic if the wrapping
needs to be conditional, such as if it appeared in a ternary or &&/||
expression. I think the thing to do is to make two calls:

if arr.empty {
  foo(nil, 0)
} else {
  foo(&arr, arr.count)
}

since preparing the buffer for the pointer itself isn't necessarily free,
and you'd want to avoid that work if you don't need it.

It might be OK to have the pointer produced for an empty array be null to
begin with, which would avoid the need for this conditional at all. In most
cases, you can't safely dereference a pointer to nothing anyway. I'm also
sympathetic to the idea of disconnecting "address-of" and "inout", since it
often leads to confusion like this.

-Joe

My proposed solutions:

– Extend type inference for unsafe pointers and nil, so that when a
value is passed by address to a function, it's not only the result of an
&-expression that has its type inferred to be (or implicitly converted to)
Unsafe[Mutable]Pointer, but if there's a nil somewhere around, such as the
one in the example above, it gets promoted to that type too, just like NULL
in C or nullptr in C++.

– Stop overloading the inout '&' operator and using it for C-style
address-of operations. I could imagine a similar, but distinct operator or
even a library function (something along the lines of unsafeAddressOf) that
specifically yields the physical address of its operand as an unsafe C
pointer, and which is thus first-class in the sense that it may be used
anywhere other expressions may be, not just as immediate call arguments.

– Make array bounds checking more lenient when passing pointers to array
elements into C functions. Bounds checking should, in these cases, allow
indexing the one-past-end element of an array if (and only if) it is the
argument of the address-of operator.

Comments and questions are welcome (you might need clarification, as it's
2:35 AM here when I'm writing this…)

Cheers,

--
Author of the Sparkling language
http://h2co3.org/

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
Author of the Sparkling language
http://h2co3.org/

--
Author of the Sparkling language
http://h2co3.org/

10

Thanks for your answer!

> I think the thing to do is to make two calls:

Eeeek, that's exactly what I wanted to avoid (which I actually think is the worst possible solution).

> The inout-to-pointer is only available to function argument expressions; it's not even considered in a ternary expression here.

Yes, I realized that (in fact I think the diagnostic makes this quite clear). I understand *why* it does not currently work, I'm merely suggesting that I think it *should* work. (It might be too hard to implement, just my opinion.)

Well, it might be a bit more humane to have a rule similar to C++ temporaries, where temporary references are guaranteed valid for the duration of the statement in which the temporary was produced. We could say that pointer bridging conversions guarantee the buffer for the duration of the statement rather than the immediate call.

> In general, we can't support fully first-class pointers into managed Swift entities like Array and properties, without breaking the encapsulation of those abstractions.

Yes, that's kind of obvious – what I am actually trying to suggest is that unsafe is already unsafe, so it might be OK to "break the encapsulation", for example by returning a pointer to the internal buffer without introducing a temporary.

The withUnsafe*Pointer operations do try to reuse existing buffers when there is a buffer, and it's safe to mutate. Copy-on-write means we might need to copy before yielding the pointer to avoid mutating shared state, and ObjC bridging means that an array might really be an opaque NSArray subclass we can't get a buffer from. By fencing the pointer's validity to a particular scope we can also keep array value semantics safe when the array ends up shared after the pointer's been taken.

-Joe

···

On Dec 24, 2015, at 12:48 PM, Árpád Goretity via swift-evolution <swift-evolution@swift.org> wrote:

11

ObjC bridging means that an array might really be an opaque NSArray

subclass we can't get a buffer from

Ah, I totally forgot about that. (not trying to be a pain, but NSArray
could as well expose its buffer… if and only if the subclass in use doesn't
do the usual circular buffer optimization. But then again, I see why this
is difficult, so let's just leave it there.)

By fencing the pointer's validity to a particular scope we can also keep

array value semantics safe

Sure thing. So does this then mean that the kind of "unsafe/real
address-of" operation can't be generally realized?

···

On Fri, Dec 25, 2015 at 9:00 PM, Joe Groff <jgroff@apple.com> wrote:

On Dec 24, 2015, at 12:48 PM, Árpád Goretity via swift-evolution < > swift-evolution@swift.org> wrote:

Thanks for your answer!

> I think the thing to do is to make two calls:

Eeeek, that's exactly what I wanted to avoid (which I actually think is
the worst possible solution).

> The inout-to-pointer is only available to function argument
expressions; it's not even considered in a ternary expression here.

Yes, I realized that (in fact I think the diagnostic makes this quite
clear). I understand *why* it does not currently work, I'm merely
suggesting that I think it *should* work. (It might be too hard to
implement, just my opinion.)

Well, it might be a bit more humane to have a rule similar to C++
temporaries, where temporary references are guaranteed valid for the
duration of the statement in which the temporary was produced. We could say
that pointer bridging conversions guarantee the buffer for the duration of
the statement rather than the immediate call.

> In general, we can't support fully first-class pointers into managed
Swift entities like Array and properties, without breaking the
encapsulation of those abstractions.

Yes, that's kind of obvious – what I am actually trying to suggest is that
unsafe is already unsafe, so it might be OK to "break the encapsulation",
for example by returning a pointer to the internal buffer without
introducing a temporary.

The withUnsafe*Pointer operations do try to reuse existing buffers when
there is a buffer, and it's safe to mutate. Copy-on-write means we might
need to copy before yielding the pointer to avoid mutating shared state,
and ObjC bridging means that an array might really be an opaque NSArray
subclass we can't get a buffer from. By fencing the pointer's validity to a
particular scope we can also keep array value semantics safe when the array
ends up shared after the pointer's been taken.

-Joe

--
Author of the Sparkling language
http://h2co3.org/

12

> ObjC bridging means that an array might really be an opaque NSArray subclass we can't get a buffer from

Ah, I totally forgot about that. (not trying to be a pain, but NSArray could as well expose its buffer… if and only if the subclass in use doesn't do the usual circular buffer optimization. But then again, I see why this is difficult, so let's just leave it there.)

The default NS/CFArray implementation does provide internal interfaces to get at its buffer which we take advantage of. In general, though, an NSArray subclass doesn't need to offer anything more than `objectAtIndex:` and `count`, so we have to be able to deal with a potentially fully opaque interface.

> By fencing the pointer's validity to a particular scope we can also keep array value semantics safe

Sure thing. So does this then mean that the kind of "unsafe/real address-of" operation can't be generally realized?

It might be possible, but the interactions between pointer-level and high-level accesses would get pretty complex. After a certain point of pointer and lifetime complexity, the high level semantics are just getting in your way IMO, and malloc/free start to be more appealing.

-Joe

···

On Dec 25, 2015, at 12:29 PM, Árpád Goretity <arpad.goretity@gmail.com> wrote: