On Friday, 18 March 2016, Jordan Rose via swift-evolution < swift-evolution@swift.org> wrote:
Hey, everyone. If you're like me, you're sick of the fact that
'UnsafePointer<Int>' doesn't tell you whether or not the pointer can be
nil. Why do we need to suffer this indignity when reference types—including
function pointers!—can distinguish "present" from "absent" with the
standard type 'Optional'? Well, good news: here's a proposal to make
pointer nullability explicit. 'UnsafePointer<Int>?' can be null (nil),
while 'UnsafePointer<Int>' cannot. Read on for details!
https://github.com/jrose-apple/swift-evolution/blob/optional-pointers/proposals/nnnn-optional-pointers.md
Bonus good news: I've implemented this locally and updated nearly all the
tests already. Assuming this is accepting, the actual changes will go
through review as a PR on GitHub, although it's mostly going to be one big
mega-patch because the core change has a huge ripple effect.
Jordan
---
Make pointer nullability explicit using Optional
- Proposal: SE-NNNN
<https://github.com/apple/swift-evolution/blob/master/proposals/NNNN-name.md>
- Author(s): Jordan Rose <https://github.com/jrose-apple>
- Status: *Awaiting review*
- Review manager: TBD
<https://github.com/jrose-apple/swift-evolution/tree/optional-pointers#introduction>
Introduction
In Objective-C, pointers (whether to objects or to a non-object type) can
be marked as nullable or nonnull, depending on whether the pointer value
can ever be null. In Swift, however, there is no such way to make this
distinction for pointers to non-object types: an UnsafePointer<Int> might
be null, or it might never be.
We already have a way to describe this: Optionals. This proposal makes
UnsafePointer<Int> represent a non-nullable pointer, and
UnsafePointer<Int>? a nullable pointer. This also allows us to preserve
information about pointer nullability available in header files for
imported C and Objective-C APIs.
<https://github.com/jrose-apple/swift-evolution/tree/optional-pointers#motivation>
Motivation
Today, UnsafePointer and friends suffer from a problem inherited from C:
every pointer value could potentially be null, and code that works with
pointers may or may not expect this. Failing to take the null pointer case
into account can lead to assertion failures or crashes. For example, pretty
much every operation on UnsafePointer itself requires a valid pointer
(reading, writing, and initializing the pointee or performing arithmetic
operations).
Fortunately, when a type has a single invalid value for which no
operations are valid, Swift already has a solution: Optionals. Applying
this to pointer types makes things very clear: if the type is non-optional,
the pointer will never be null, and if it *is*optional, the developer
must take the "null pointer" case into account. This clarity has already
been appreciated in Apple's Objective-C headers, which include nullability
annotations for all pointer types (not just object pointers).
This change also allows developers working with pointers to take advantage
of the many syntactic conveniences already built around optionals. For
example, the standard library currently has a helper method on
UnsafeMutablePointer called _setIfNonNil; with "optional pointers" this
can be written simply and clearly:
ptr?.pointee = newValue
Finally, this change also reduces the number of types that conform to
NilLiteralConvertible, a source of confusion for newcomers who (reasonably)
associate nil directly with optionals. Currently the standard library
includes the following NilLiteralConvertible types:
- Optional
- ImplicitlyUnwrappedOptional (subject of a separate proposal by Chris
Willmore)
- _OptionalNilComparisonType (used for optionalValue == nil)
- *UnsafePointer*
- *UnsafeMutablePointer*
- *AutoreleasingUnsafeMutablePointer*
- *OpaquePointer*
plus these Objective-C-specific types:
- *Selector*
- *NSZone* (only used to pass nil in Swift)
All of the italicized types would drop their conformance to
NilLiteralConvertible; the "null pointer" would be represented by a nil
optional of a particular type.
<https://github.com/jrose-apple/swift-evolution/tree/optional-pointers#proposed-solution>Proposed
solution
1.
Have the compiler assume that all values with pointer type (the
italicized types listed above) are non-null. This allows the representation
of Optional.none for a pointer type to be a null pointer value.
2.
Drop NilLiteralConvertible conformance for all pointer types.
3.
Teach the Clang importer to treat _Nullable pointers as Optional (and
_Null_unspecified pointers as ImplicitlyUnwrappedOptional).
4.
Deal with the fallout, i.e. adjust the compiler and the standard
library to handle this new behavior.
5.
Test migration and improve the migrator as necessary.
This proposal does not include the removal of the NilLiteralConvertible
protocol altogether; besides still having two distinct optional types,
we've seen people wanting to use nil for their own types (e.g. JSON
values). (Changing this in the future is not out of the question; it's just
out of scope for this proposal.)
<https://github.com/jrose-apple/swift-evolution/tree/optional-pointers#detailed-design>Detailed
design
<https://github.com/jrose-apple/swift-evolution/tree/optional-pointers#api-changes>API
Changes
-
Conformance to NilLiteralConvertible is removed from all types except
Optional, ImplicitlyUnwrappedOptional, and _OptionalNilComparisonType,
along with the implementation of init(nilLiteral:).
-
init(bitPattern: Int) and init(bitPattern: UInt) on all pointer types
become failable; if the bit pattern represents a null pointer, nil is
returned.
-
Process.unsafeArgv is a pointer to a null-terminated C array of C
strings, so its type changes from
UnsafeMutablePointer<UnsafeMutablePointer<Int8>> to
UnsafeMutablePointer<UnsafeMutablePointer<Int8>?>, i.e. the inner
pointer type becomes optional. It is then an error to access
Process.unsafeArgv before entering main. (Previously you would get a
null pointer value.)
-
NSErrorPointer becomes optional:
-public typealias NSErrorPointer = AutoreleasingUnsafeMutablePointer<NSError?>+public typealias NSErrorPointer = AutoreleasingUnsafeMutablePointer<NSError?>?
- A number of methods on String that came from NSString now have
optional parameters:
public func completePathIntoString(- outputName: UnsafeMutablePointer<String> = nil,+ outputName: UnsafeMutablePointer<String>? = nil,
caseSensitive: Bool,- matchesIntoArray: UnsafeMutablePointer<[String]> = nil,+ matchesIntoArray: UnsafeMutablePointer<[String]>? = nil,
filterTypes: [String]? = nil
) -> Int {
public init(
contentsOfFile path: String,- usedEncoding: UnsafeMutablePointer<NSStringEncoding> = nil+ usedEncoding: UnsafeMutablePointer<NSStringEncoding>? = nil
) throws {
public init(
contentsOfURL url: NSURL,- usedEncoding enc: UnsafeMutablePointer<NSStringEncoding> = nil+ usedEncoding enc: UnsafeMutablePointer<NSStringEncoding>? = nil
) throws {
public func linguisticTags(
in range: Range<Index>,
scheme tagScheme: String,
options opts: NSLinguisticTaggerOptions = ,
orthography: NSOrthography? = nil,- tokenRanges: UnsafeMutablePointer<[Range<Index>]> = nil+ tokenRanges: UnsafeMutablePointer<[Range<Index>]>? = nil
) -> [String] {
-
NSZone's no-argument initializer is gone. (It probably should have
been removed already as part of the Swift 3 naming cleanup.)
-
A small regression: optional pointers can no longer be passed using
withVaList because it would require a conditional conformance to the
CVarArg protocol. For now, using unsafeBitCast to reinterpret the
optional pointer as an Int is the best alternative; Int has the same C
variadic calling conventions as a pointer on all supported platforms.
<https://github.com/jrose-apple/swift-evolution/tree/optional-pointers#conversion-between-pointers>Conversion
between pointers
Currently each pointer type has initializers of this form:
init<OtherPointee>(_ otherPointer: UnsafePointer<OtherPointee>)
This simply makes a pointer with a different type but the same address as
otherPointer. However, in making pointer nullability explicit, this now
only converts non-nil pointers to non-nil pointers. In my experiments, this
has led to this idiom becoming very common:
// Before:let untypedPointer = UnsafePointer<Void>(ptr)
// After:let untypedPointer = ptr.map(UnsafePointer<Void>.init)
// Usually the pointee type is actually inferred:
foo(ptr.map(UnsafePointer.init))
I consider this a bit more difficult to understand than the original code,
at least at a glance. We should therefore add new initializers of the
following form:
init?<OtherPointee>(_ otherPointer: UnsafePointer<OtherPointee>?) {
guard let nonnullPointer = otherPointer else {
return nil
}
self.init(nonnullPointer)
}
The body is for explanation purposes only; we'll make sure the actual
implementation does not require an extra comparison.
(This would need to be an overload rather than replacing the previous
initializer because the "non-null-ness" should be preserved through the
type conversion.)
The alternative is to leave this initializer out, and require the nil case
to be explicitly handled or mapped away.
<https://github.com/jrose-apple/swift-evolution/tree/optional-pointers#open-issue-unsafebufferpointer>Open
Issue: UnsafeBufferPointer
The type UnsafeBufferPointer represents a bounded typed memory region
with no ownership or lifetime semantics; it is logically a bare typed
pointer (its baseAddress) and a length (count). For a buffer with 0
elements, however, there's no need to provide the address of allocated
memory, since it can't be read from. Previously this case would be
represented as a nil base address and a count of 0.
With optional pointers, this now imposes a cost on clients that want to
access the base address: they need to consider the nil case explicitly,
where previously they wouldn't have had to. There are several possibilities
here, each with their own possible implementations:
1.
Like UnsafePointer, UnsafeBufferPointer should always have a valid
base address, even when the count is 0. An UnsafeBufferPointer with a
potentially-nil base address should be optional.
1.
UnsafeBufferPointer's initializer accepts an optional pointer and
becomes failable, returning nil if the input pointer is nil.
2.
UnsafeBufferPointer's initializer accepts an optional pointer and
synthesizes a non-null aligned pointer value if given nil as a base address.
3.
UnsafeBufferPointer's initializer only accepts non-optional
pointers. Clients such as withUnsafeBufferPointermust synthesize a
non-null aligned pointer value if they do not have a valid pointer to
provide.
4.
UnsafeBufferPointer's initializer only accepts non-optional
pointers. Clients *using* withUnsafeBufferPointermust handle a nil
buffer.
2.
UnsafeBufferPointer should allow nil base addresses, i.e. the
baseAddress property will be optional. Clients will need to handle
this case explicitly.
1.
UnsafeBufferPointer's initializer accepts an optional pointer, but
no other changes are made.
2.
UnsafeBufferPointer's initializer accepts an optional pointer.
Additionally, any buffers initialized with a count of 0 will be
canonicalized to having a base address of nil.
I'm currently leaning towards option (2i). Clients that expect a pointer
and length probably shouldn't require the pointer to be non-null, but if
they do then perhaps there's a reason for it. It's also the least work.
Chris (Lattner) is leaning towards option (1ii), which treats
UnsafeBufferPointer similar to UnsafePointer while not penalizing the
common case of withUnsafeBufferPointer.
<https://github.com/jrose-apple/swift-evolution/tree/optional-pointers#impact-on-existing-code>Impact
on existing code
Any code that uses a pointer type (including Selector or NSZone) may be
affected by this change. For the most part our existing logic to handle
last year's nullability audit should cover this, but the implementer should
test migration of several projects to see what issues might arise.
Anecdotally, in migrating the standard library to use this new logic I've
been quite happy with nullability being made explicit. There are many
places where a pointer really *can't* be nil.
<https://github.com/jrose-apple/swift-evolution/tree/optional-pointers#alternatives-considered>Alternatives
considered
The primary alternative here would be to leave everything as it is today,
with UnsafePointer and friends including the null pointer as one of their
normal values. This has obviously worked just fine for nearly two years of
Swift, but it is leaving information on the table that can help avoid bugs,
and is strange in a language that makes fluent use of Optional. As a fairly
major source-breaking change, it is also something that we probably should
do sooner rather than later in the language's evolution.
