SR-104: Improve Crash-Safety when Importing Objective-C Code Without Nullability Attributes

Fabian_Ehrentraud · December 8, 2015, 12:09pm

Swift code accessing Objective-C methods can easily crash - if the Objective-C code does not include nullability attributes, the code is brought as implicitly unwrapped into Swift, where unsafe accesses do not produce compiler warnings.

I created issue SR-104 for this, but as suggested by Jordan Rose it should be discussed on this mailing list first.

Short example:

// Objective-C
- (NSString *)giveMeAString { return nil; }

// Swift
func thisWillCrash() {
let string = someObjectiveCObject.giveMeAString()
let length = string.length // crash
}

The ClangImporter could handle this issue in several safer ways:

1. Only import declarations that are nullability annotated (as proposed by Greg Parker)
Positive side effect would be that it would motivate to write nullability annotations.

2. Import un-annotated code as Optional
Values would need to be unwrapped every time, which does not hurt too much due to the easy use of the `?` syntactic sugar.

What do you think? Would this make Swift more safe when used together with Objective-C?

Javi · December 8, 2015, 3:45pm

I think the decision to import Obj-C APIs with implicitly unwrapped
optionals was a very pragmatic decision at the time. However, given it's
been over a year (I think?) since the nullability specifiers have been
available, I agree with you this could now be improved.

I personally prefer #2: it mimics the old Obj-C behavior ("calling a method
on nil doesn't do anything and any method can take or return nil") if you
use "?." everywhere. This would be somewhat cumbersome, but like you say
would encourage library developers to add these specifiers.
In practice, many developers will work with Apple's frameworks, and Apple
has done a great job tagged all/most of their APIs!

···

On Tue, Dec 8, 2015 at 4:09 AM Fabian Ehrentraud via swift-evolution < swift-evolution@swift.org> wrote:

Swift code accessing Objective-C methods can easily crash - if the
Objective-C code does not include nullability attributes, the code is
brought as implicitly unwrapped into Swift, where unsafe accesses do not
produce compiler warnings.

I created issue SR-104 for this, but as suggested by Jordan Rose it should
be discussed on this mailing list first.

Short example:

// Objective-C
- (NSString *)giveMeAString { return nil; }

// Swift
func thisWillCrash() {
let string = someObjectiveCObject.giveMeAString()
let length = string.length // crash
}

The ClangImporter could handle this issue in several safer ways:

1. Only import declarations that are nullability annotated (as proposed by
Greg Parker)
Positive side effect would be that it would motivate to write nullability
annotations.

2. Import un-annotated code as Optional
Values would need to be unwrapped every time, which does not hurt too much
due to the easy use of the `?` syntactic sugar.

What do you think? Would this make Swift more safe when used together with
Objective-C?
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
Javier Soto

Chris_Lattner · December 9, 2015, 6:11am

Swift code accessing Objective-C methods can easily crash - if the Objective-C code does not include nullability attributes, the code is brought as implicitly unwrapped into Swift, where unsafe accesses do not produce compiler warnings.

I created issue SR-104 for this, but as suggested by Jordan Rose it should be discussed on this mailing list first.

Short example:

// Objective-C
- (NSString *)giveMeAString { return nil; }

// Swift
func thisWillCrash() {
let string = someObjectiveCObject.giveMeAString()
let length = string.length // crash
}

The ClangImporter could handle this issue in several safer ways:

1. Only import declarations that are nullability annotated (as proposed by Greg Parker)
Positive side effect would be that it would motivate to write nullability annotations.

This seems unfortunate to me, because it would severely hamper interoperating with Objective-C and *C* APIs that have not been audited. While Apple is keen to audit their APIs, I doubt that all the linux system headers will get updated in a timely manner and users don’t generally have a way to do anything about that.

2. Import un-annotated code as Optional
Values would need to be unwrapped every time, which does not hurt too much due to the easy use of the `?` syntactic sugar.

This is a very interesting idea, one I haven’t considered recently. I agree with you that this is worth considering, and I would love to see IUO just get summarily deleted :-)

-Chris

···

On Dec 8, 2015, at 4:09 AM, Fabian Ehrentraud via swift-evolution <swift-evolution@swift.org> wrote:

Michael_Buckley · December 8, 2015, 4:41pm

Given the availability and prevalence of nullability annotations, I think
it's reasonable to update this behavior, but while I suspect most Swift
projects that only use Apple-supplied Objective-C frameworks, there will
still be a considerable number that rely on third-party, possibly
closed-source frameworks. It may not be possible or economical for those
projects to get updated versions of those frameworks with nullability
attributes. If we choose option #1, these projects may be forced to rewrite
their Swift code in Objective-C, or write extra Objective-C wrappers with
nullability annotations around their existing code. For this reason, I
recommend option #2.

Because this change will already require these projects to change their
swift code, we should make the process as painless as possible, so this
change should include support for automatic code migration, as well as nice
error messages which both explain the change and how to fix it.

···

On Tue, Dec 8, 2015 at 8:31 AM, Michael Buckley <thebuckley@gmail.com> wrote:

Given the availability and prevalence of nullability annotations, I think
it's reasonable to update this behavior, but while I suspect most Swift
projects that only use Apple-supplied Objective-C frameworks, there will
still be a considerable number that rely on third-party, possibly
closed-source frameworks. It may not be possible or economical for those
projects to get updated versions of those frameworks with nullability
attributes. If we choose option #1, these projects may be forced to rewrite
their Swift code in Objective-C, or write extra Objective-C wrappers with
nullability annotations around their existing code. For this reason, I
recommend option #2.

Because this change will already require these projects to change their
swift code, we should make the process as painless as possible, so this
change should include support for automatic code migration, as well as nice
error messages which both explain the change and how to fix it.

On Tue, Dec 8, 2015 at 7:45 AM, Javier Soto via swift-evolution < > swift-evolution@swift.org> wrote:

I think the decision to import Obj-C APIs with implicitly unwrapped
optionals was a very pragmatic decision at the time. However, given it's
been over a year (I think?) since the nullability specifiers have been
available, I agree with you this could now be improved.

I personally prefer #2: it mimics the old Obj-C behavior ("calling a
method on nil doesn't do anything and any method can take or return nil")
if you use "?." everywhere. This would be somewhat cumbersome, but like you
say would encourage library developers to add these specifiers.
In practice, many developers will work with Apple's frameworks, and Apple
has done a great job tagged all/most of their APIs!

On Tue, Dec 8, 2015 at 4:09 AM Fabian Ehrentraud via swift-evolution < >> swift-evolution@swift.org> wrote:

Swift code accessing Objective-C methods can easily crash - if the
Objective-C code does not include nullability attributes, the code is
brought as implicitly unwrapped into Swift, where unsafe accesses do not
produce compiler warnings.

I created issue SR-104 for this, but as suggested by Jordan Rose it
should be discussed on this mailing list first.

Short example:

// Objective-C
- (NSString *)giveMeAString { return nil; }

// Swift
func thisWillCrash() {
let string = someObjectiveCObject.giveMeAString()
let length = string.length // crash
}

The ClangImporter could handle this issue in several safer ways:

1. Only import declarations that are nullability annotated (as proposed
by Greg Parker)
Positive side effect would be that it would motivate to write
nullability annotations.

2. Import un-annotated code as Optional
Values would need to be unwrapped every time, which does not hurt too
much due to the easy use of the `?` syntactic sugar.

What do you think? Would this make Swift more safe when used together
with Objective-C?
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
Javier Soto
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Brent_Royal-Gordon · December 8, 2015, 8:45pm

It may not be possible or economical for those projects to get updated versions of those frameworks with nullability attributes.

Objective-C nullability attributes don’t affect their binary compatibility, though, right? As long as Swift’s bridging behavior is *safe*, anyone can fix them being *inconvenient* by editing the headers in the crappy old frameworks they’re using.

···

--
Brent Royal-Gordon
Architechies

Michael_Buckley · December 8, 2015, 8:51pm

In an ideal world, absolutely. In the business world, that's contingent on
what contracts have been signed. Some commercial frameworks require that
customers not edit any part of the framework, including headers.

···

On Tue, Dec 8, 2015 at 12:45 PM, Brent Royal-Gordon <brent@architechies.com> wrote:

> It may not be possible or economical for those projects to get updated
versions of those frameworks with nullability attributes.

Objective-C nullability attributes don’t affect their binary
compatibility, though, right? As long as Swift’s bridging behavior is
*safe*, anyone can fix them being *inconvenient* by editing the headers in
the crappy old frameworks they’re using.

Brent_Royal-Gordon · December 8, 2015, 10:04pm

In an ideal world, absolutely. In the business world, that's contingent on what contracts have been signed. Some commercial frameworks require that customers not edit any part of the framework, including headers.

Hmm. Does anyone know if apinotes can modify the nullability of a framework’s symbols? I haven’t really given them a close look yet.

···

--
Brent Royal-Gordon
Architechies

jrose · December 9, 2015, 12:09am

They can, but we really don't want to get into a situation where the average Swift developer has to install extra apinotes files, or worse yet make their own. I don't think we'd take any feature that required that.

Jordan

···

On Dec 8, 2015, at 14:04, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> wrote:

In an ideal world, absolutely. In the business world, that's contingent on what contracts have been signed. Some commercial frameworks require that customers not edit any part of the framework, including headers.

Hmm. Does anyone know if apinotes can modify the nullability of a framework’s symbols? I haven’t really given them a close look yet.

Fabian_Ehrentraud · December 8, 2015, 5:06pm

Good point about old frameworks in case of #1. Maybe a compiler option to deactivate the new behavior would help in the transition phase until the external dependencies are updated with nullability specifiers? Nonethelesd #2 does sound better in this case.

···

Am 08.12.2015 um 17:41 schrieb Michael Buckley via swift-evolution <swift-evolution@swift.org<mailto:swift-evolution@swift.org>>:

Given the availability and prevalence of nullability annotations, I think it's reasonable to update this behavior, but while I suspect most Swift projects that only use Apple-supplied Objective-C frameworks, there will still be a considerable number that rely on third-party, possibly closed-source frameworks. It may not be possible or economical for those projects to get updated versions of those frameworks with nullability attributes. If we choose option #1, these projects may be forced to rewrite their Swift code in Objective-C, or write extra Objective-C wrappers with nullability annotations around their existing code. For this reason, I recommend option #2.

Because this change will already require these projects to change their swift code, we should make the process as painless as possible, so this change should include support for automatic code migration, as well as nice error messages which both explain the change and how to fix it.

On Tue, Dec 8, 2015 at 8:31 AM, Michael Buckley <thebuckley@gmail.com<mailto:thebuckley@gmail.com>> wrote:
Given the availability and prevalence of nullability annotations, I think it's reasonable to update this behavior, but while I suspect most Swift projects that only use Apple-supplied Objective-C frameworks, there will still be a considerable number that rely on third-party, possibly closed-source frameworks. It may not be possible or economical for those projects to get updated versions of those frameworks with nullability attributes. If we choose option #1, these projects may be forced to rewrite their Swift code in Objective-C, or write extra Objective-C wrappers with nullability annotations around their existing code. For this reason, I recommend option #2.

Because this change will already require these projects to change their swift code, we should make the process as painless as possible, so this change should include support for automatic code migration, as well as nice error messages which both explain the change and how to fix it.

On Tue, Dec 8, 2015 at 7:45 AM, Javier Soto via swift-evolution <swift-evolution@swift.org<mailto:swift-evolution@swift.org>> wrote:
I think the decision to import Obj-C APIs with implicitly unwrapped optionals was a very pragmatic decision at the time. However, given it's been over a year (I think?) since the nullability specifiers have been available, I agree with you this could now be improved.

I personally prefer #2: it mimics the old Obj-C behavior ("calling a method on nil doesn't do anything and any method can take or return nil") if you use "?." everywhere. This would be somewhat cumbersome, but like you say would encourage library developers to add these specifiers.
In practice, many developers will work with Apple's frameworks, and Apple has done a great job tagged all/most of their APIs!

On Tue, Dec 8, 2015 at 4:09 AM Fabian Ehrentraud via swift-evolution <swift-evolution@swift.org<mailto:swift-evolution@swift.org>> wrote:
Swift code accessing Objective-C methods can easily crash - if the Objective-C code does not include nullability attributes, the code is brought as implicitly unwrapped into Swift, where unsafe accesses do not produce compiler warnings.

I created issue SR-104 for this, but as suggested by Jordan Rose it should be discussed on this mailing list first.

Short example:

// Objective-C
- (NSString *)giveMeAString { return nil; }

// Swift
func thisWillCrash() {
let string = someObjectiveCObject.giveMeAString()
let length = string.length // crash
}

The ClangImporter could handle this issue in several safer ways:

1. Only import declarations that are nullability annotated (as proposed by Greg Parker)
Positive side effect would be that it would motivate to write nullability annotations.

2. Import un-annotated code as Optional
Values would need to be unwrapped every time, which does not hurt too much due to the easy use of the `?` syntactic sugar.

What do you think? Would this make Swift more safe when used together with Objective-C?
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org<mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution
--
Javier Soto
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org<mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org<mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

gparker42 · December 9, 2015, 7:04am

I'm not sure that Apple will get all of its headers audited in a timely manner either. Platform APIs have long tails.

This would make bring-up of a new platform awfully painful, too. You wouldn't be able to do anything with your shiny new port of Swift without annotating a bunch of that platform's headers somehow.

Measuring the impact of not-importing or ugly-importing unannotated API should be a straightforward experiment using a modified Swift importer.
1. Hack the importer to record every un-annotated API that it sees.
2. Run the custom importer against some platform's SDK.
3. Look at the results and try to imagine how bad the world would be if those APIs were harder to use or missing entirely.

···

On Dec 8, 2015, at 10:11 PM, Chris Lattner via swift-evolution <swift-evolution@swift.org> wrote:

On Dec 8, 2015, at 4:09 AM, Fabian Ehrentraud via swift-evolution <swift-evolution@swift.org> wrote:

1. Only import declarations that are nullability annotated (as proposed by Greg Parker)
Positive side effect would be that it would motivate to write nullability annotations.

This seems unfortunate to me, because it would severely hamper interoperating with Objective-C and *C* APIs that have not been audited. While Apple is keen to audit their APIs, I doubt that all the linux system headers will get updated in a timely manner and users don’t generally have a way to do anything about that.

--
Greg Parker gparker@apple.com Runtime Wrangler

Fabian_Ehrentraud · December 10, 2015, 11:42am

Are there any objections to variant 2 "Import un-annotated code as Optional"? If not, I would create the proposal.

···

Am 09.12.2015 um 07:11 schrieb Chris Lattner <clattner@apple.com>:

On Dec 8, 2015, at 4:09 AM, Fabian Ehrentraud via swift-evolution <swift-evolution@swift.org> wrote:

Swift code accessing Objective-C methods can easily crash - if the Objective-C code does not include nullability attributes, the code is brought as implicitly unwrapped into Swift, where unsafe accesses do not produce compiler warnings.

I created issue SR-104 for this, but as suggested by Jordan Rose it should be discussed on this mailing list first.

Short example:

// Objective-C
- (NSString *)giveMeAString { return nil; }

// Swift
func thisWillCrash() {
let string = someObjectiveCObject.giveMeAString()
let length = string.length // crash
}

The ClangImporter could handle this issue in several safer ways:

1. Only import declarations that are nullability annotated (as proposed by Greg Parker)
Positive side effect would be that it would motivate to write nullability annotations.

This seems unfortunate to me, because it would severely hamper interoperating with Objective-C and *C* APIs that have not been audited. While Apple is keen to audit their APIs, I doubt that all the linux system headers will get updated in a timely manner and users don’t generally have a way to do anything about that.

2. Import un-annotated code as Optional
Values would need to be unwrapped every time, which does not hurt too much due to the easy use of the `?` syntactic sugar.

This is a very interesting idea, one I haven’t considered recently. I agree with you that this is worth considering, and I would love to see IUO just get summarily deleted :-)

-Chris

Fabian_Ehrentraud · December 9, 2015, 3:21pm

Would it be possible to output meaningful compile errors when calling an API method from Swift, that was not imported due to missing nullability annotations? If not, this could easily result in a lot of misspent developer time trying to find the issue why this method could not be called. There is a similar issue right now when trying to access a Swift class/method missing the @objc specifier from Objective-C.
Importing un-annotated code as Optional would not have these issues.

···

On 09.12.2015, at 08:04, Greg Parker <gparker@apple.com> wrote:

On Dec 8, 2015, at 10:11 PM, Chris Lattner via swift-evolution <swift-evolution@swift.org> wrote:

On Dec 8, 2015, at 4:09 AM, Fabian Ehrentraud via swift-evolution <swift-evolution@swift.org> wrote:

1. Only import declarations that are nullability annotated (as proposed by Greg Parker)
Positive side effect would be that it would motivate to write nullability annotations.

This seems unfortunate to me, because it would severely hamper interoperating with Objective-C and *C* APIs that have not been audited. While Apple is keen to audit their APIs, I doubt that all the linux system headers will get updated in a timely manner and users don’t generally have a way to do anything about that.

I'm not sure that Apple will get all of its headers audited in a timely manner either. Platform APIs have long tails.

This would make bring-up of a new platform awfully painful, too. You wouldn't be able to do anything with your shiny new port of Swift without annotating a bunch of that platform's headers somehow.

Measuring the impact of not-importing or ugly-importing unannotated API should be a straightforward experiment using a modified Swift importer.
1. Hack the importer to record every un-annotated API that it sees.
2. Run the custom importer against some platform's SDK.
3. Look at the results and try to imagine how bad the world would be if those APIs were harder to use or missing entirely.

--
Greg Parker gparker@apple.com Runtime Wrangler

Douglas_Gregor · December 10, 2015, 6:27pm

Are there any objections to variant 2 "Import un-annotated code as Optional"? If not, I would create the proposal.

I’m opposed to this.

First of all, this affects a lot of APIs. I went ahead and grepped through the API dumps over at

GitHub - apple/swift-3-api-guidelines-review

to see how often implicitly-unwrapped optionals came through in the Cocoa APIs, and there are lots of them. For example, on OS X, grepping for !’s finds:

  4132 functions/methods
  2281 properties
  409 initializers

On iOS, which has a smaller number of overall APIs, the numbers are still pretty big:
  1071 functions/methods
  320 properties
  134 initializers

This may be a slight oversampling of the data—some of those might be explicitly annotated as _Null_unspecified in the (Objective-)C headers, some might be unavailable—but the scale of the impact here is very, very large.

Most of the implicitly unwrapped optionals we currently get from (Objective-)C APIs can, in fact, never be nil. Forcing programmers to deal with all of these values as full optionals leads to a ton of optional-related boilerplate that doesn’t actual help users write better code. Rather, it dilutes the effectiveness of optionals as a language feature: optionals should mean “nil matters here, think about it carefully!”. But if some large number of optionals people deal with on a day-to-day basis are instead “the compiler didn’t know, so you have to write !”, the useful meaning of the “true” optionals gets lost in the noise.

That said, I’d love for implicitly-unwrapped optionals to be used less often, and getting them further out of the type system would be beneficial for the Swift experience.

- Doug

···

On Dec 10, 2015, at 3:42 AM, Fabian Ehrentraud via swift-evolution <swift-evolution@swift.org> wrote:

Am 09.12.2015 um 07:11 schrieb Chris Lattner <clattner@apple.com>:

On Dec 8, 2015, at 4:09 AM, Fabian Ehrentraud via swift-evolution <swift-evolution@swift.org> wrote:

Swift code accessing Objective-C methods can easily crash - if the Objective-C code does not include nullability attributes, the code is brought as implicitly unwrapped into Swift, where unsafe accesses do not produce compiler warnings.

I created issue SR-104 for this, but as suggested by Jordan Rose it should be discussed on this mailing list first.

Short example:

// Objective-C
- (NSString *)giveMeAString { return nil; }

// Swift
func thisWillCrash() {
let string = someObjectiveCObject.giveMeAString()
let length = string.length // crash
}

The ClangImporter could handle this issue in several safer ways:

1. Only import declarations that are nullability annotated (as proposed by Greg Parker)
Positive side effect would be that it would motivate to write nullability annotations.

This seems unfortunate to me, because it would severely hamper interoperating with Objective-C and *C* APIs that have not been audited. While Apple is keen to audit their APIs, I doubt that all the linux system headers will get updated in a timely manner and users don’t generally have a way to do anything about that.

2. Import un-annotated code as Optional
Values would need to be unwrapped every time, which does not hurt too much due to the easy use of the `?` syntactic sugar.

This is a very interesting idea, one I haven’t considered recently. I agree with you that this is worth considering, and I would love to see IUO just get summarily deleted :-)

-Chris

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Fabian_Ehrentraud · December 10, 2015, 4:34pm

I created the proposal, no PR yet. Is something missing / everything clear?
https://github.com/fabb/swift-evolution/blob/import_as_optional_by_default/proposals/0008-import-as-optional-by-default.md

···

On 10.12.2015, at 12:42, Fabian Ehrentraud via swift-evolution <swift-evolution@swift.org<mailto:swift-evolution@swift.org>> wrote:

Are there any objections to variant 2 "Import un-annotated code as Optional"? If not, I would create the proposal.

Am 09.12.2015 um 07:11 schrieb Chris Lattner <clattner@apple.com<mailto:clattner@apple.com>>:

On Dec 8, 2015, at 4:09 AM, Fabian Ehrentraud via swift-evolution <swift-evolution@swift.org<mailto:swift-evolution@swift.org>> wrote:

Swift code accessing Objective-C methods can easily crash - if the Objective-C code does not include nullability attributes, the code is brought as implicitly unwrapped into Swift, where unsafe accesses do not produce compiler warnings.

I created issue SR-104 for this, but as suggested by Jordan Rose it should be discussed on this mailing list first.

Short example:

// Objective-C
- (NSString *)giveMeAString { return nil; }

// Swift
func thisWillCrash() {
let string = someObjectiveCObject.giveMeAString()
let length = string.length // crash
}

The ClangImporter could handle this issue in several safer ways:

1. Only import declarations that are nullability annotated (as proposed by Greg Parker)
Positive side effect would be that it would motivate to write nullability annotations.

This seems unfortunate to me, because it would severely hamper interoperating with Objective-C and *C* APIs that have not been audited. While Apple is keen to audit their APIs, I doubt that all the linux system headers will get updated in a timely manner and users don’t generally have a way to do anything about that.

2. Import un-annotated code as Optional
Values would need to be unwrapped every time, which does not hurt too much due to the easy use of the `?` syntactic sugar.

This is a very interesting idea, one I haven’t considered recently. I agree with you that this is worth considering, and I would love to see IUO just get summarily deleted :-)

-Chris
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org<mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

Fabian_Ehrentraud · December 10, 2015, 9:22pm

Too bad to hear you are opposed to the change.
Thank you for the hint at the API dumps - this is really something that should be referenced in the proposal.

First of all, this affects a lot of APIs. I went ahead and grepped through the API dumps [...] to see how often implicitly-unwrapped optionals came through in the Cocoa APIs, and there are lots of them. For example, on OS X, grepping for !’s finds:

We should come to a common way to count the occurrences of affected APIs (despite that users would also need to adapt for un-annotated 3rd-party frameworks, and the own mixed&matched application code).
When I counted, it resulted in less than half the numbers for initializers and functions/methods.

There are many comment lines in the API dumps that contain /*! - we should not count those.
Also for functions/methods, input-parameters do not matter, due to automatic Optional wrapping on the Swift side. Maybe closure parameters that have implicitly unwrapped optional arguments themselves should be counted too. (anything else?)

Initializers:

Should we only count "init!" instances?
$ grep init! . | wc -l
=> OSX 152, iOS 42

Other initializers containing ! in another place (I guess not relevant for the proposal)
$ grep -r ! . | grep init | grep -v '\/\*\!' | grep -v init!

Functions/Methods:

Only counting those that have a return type with an implicitly unwrapped optional. This does not take into account closure parameters that have implicitly unwrapped optional arguments themselves though.
$ grep -r ! . | grep func | grep '\->' | grep -v '\/\*\!' | grep -e '\!$' | wc -l
=> OSX 1577, iOS 452

Properties:

$ grep -r ! . | grep -e "let " | grep -v '\/\*\!' | wc -l
=> OSX 1120, iOS 336

Constants:

I noticed there are many constants that get translated to implicitly unwrapped optionals (e.g. in iOS CoreFoundation.swift). Would they be affected by the change in this proposal? Or are they "explicitly set to implicitly unwrapped" by the importer?

Most of the implicitly unwrapped optionals we currently get from (Objective-)C APIs can, in fact, never be nil. Forcing programmers to deal with all of these values as full optionals leads to a ton of optional-related boilerplate that doesn’t actual help users write better code. Rather, it dilutes the effectiveness of optionals as a language feature: optionals should mean “nil matters here, think about it carefully!”. But if some large number of optionals people deal with on a day-to-day basis are instead “the compiler didn’t know, so you have to write !”, the useful meaning of the “true” optionals gets lost in the noise.

I think exactly the opposite. If an API has not yet been audited, the client needs to consult the documentation if a return value can ever be nil. If not, she might explicitly and knowingly unwrap the value safely or non-safely. I deem this a good thing, as it prevents unknowingly unsafe use. I do not think that it would get in the way, as the syntactic sugar for Optionals is quite minimal. Also when the API gets audited for nullability later on, the compiler will hint which optional unwrappings have been rendered unnecessary.

···

On 10.12.2015, at 19:27, Douglas Gregor <dgregor@apple.com> wrote:

On Dec 10, 2015, at 3:42 AM, Fabian Ehrentraud via swift-evolution <swift-evolution@swift.org> wrote:

Are there any objections to variant 2 "Import un-annotated code as Optional"? If not, I would create the proposal.

I’m opposed to this.

First of all, this affects a lot of APIs. I went ahead and grepped through the API dumps over at

  GitHub - apple/swift-3-api-guidelines-review

to see how often implicitly-unwrapped optionals came through in the Cocoa APIs, and there are lots of them. For example, on OS X, grepping for !’s finds:

  4132 functions/methods
  2281 properties
  409 initializers

On iOS, which has a smaller number of overall APIs, the numbers are still pretty big:
  1071 functions/methods
  320 properties
  134 initializers

This may be a slight oversampling of the data—some of those might be explicitly annotated as _Null_unspecified in the (Objective-)C headers, some might be unavailable—but the scale of the impact here is very, very large.

Most of the implicitly unwrapped optionals we currently get from (Objective-)C APIs can, in fact, never be nil. Forcing programmers to deal with all of these values as full optionals leads to a ton of optional-related boilerplate that doesn’t actual help users write better code. Rather, it dilutes the effectiveness of optionals as a language feature: optionals should mean “nil matters here, think about it carefully!”. But if some large number of optionals people deal with on a day-to-day basis are instead “the compiler didn’t know, so you have to write !”, the useful meaning of the “true” optionals gets lost in the noise.

That said, I’d love for implicitly-unwrapped optionals to be used less often, and getting them further out of the type system would be beneficial for the Swift experience.

  - Doug

Am 09.12.2015 um 07:11 schrieb Chris Lattner <clattner@apple.com>:

On Dec 8, 2015, at 4:09 AM, Fabian Ehrentraud via swift-evolution <swift-evolution@swift.org> wrote:

Swift code accessing Objective-C methods can easily crash - if the Objective-C code does not include nullability attributes, the code is brought as implicitly unwrapped into Swift, where unsafe accesses do not produce compiler warnings.

I created issue SR-104 for this, but as suggested by Jordan Rose it should be discussed on this mailing list first.

Short example:

// Objective-C
- (NSString *)giveMeAString { return nil; }

// Swift
func thisWillCrash() {
let string = someObjectiveCObject.giveMeAString()
let length = string.length // crash
}

The ClangImporter could handle this issue in several safer ways:

1. Only import declarations that are nullability annotated (as proposed by Greg Parker)
Positive side effect would be that it would motivate to write nullability annotations.

This seems unfortunate to me, because it would severely hamper interoperating with Objective-C and *C* APIs that have not been audited. While Apple is keen to audit their APIs, I doubt that all the linux system headers will get updated in a timely manner and users don’t generally have a way to do anything about that.

2. Import un-annotated code as Optional
Values would need to be unwrapped every time, which does not hurt too much due to the easy use of the `?` syntactic sugar.

This is a very interesting idea, one I haven’t considered recently. I agree with you that this is worth considering, and I would love to see IUO just get summarily deleted :-)

-Chris

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Douglas_Gregor · December 10, 2015, 10:09pm

Right. If we accepted this proposal, how many additional force-unwrap operations would it take in a typical iOS or OS X application that “worked” in Swift 2 to compile again in Swift 3? Are you proposing to automatically migrate Swift 2 code to Swift 3 code by adding in those force-unwrap operations, or are you asking people to rethink and reimplement the code that breaks?

- Doug

···

On Dec 10, 2015, at 1:11 PM, Guillaume Lessard via swift-evolution <swift-evolution@swift.org> wrote:

On 10 déc. 2015, at 12:52, Fabian Ehrentraud <Fabian.Ehrentraud@willhaben.at> wrote:

Guillaume, could you elaborate on "discussing the status quo as an alternative", it sounds good, but I'm not sure I completely got what you mean :-)

I mean that the ‘alternatives’ section should explicitly compare the merits of leaving things as-is as compared to going ahead with the proposal. For 3rd-party libraries with limited API, importing unannotated functions as optional sounds good. However, with large frameworks (obviously including Cocoa), I think the community needs to be convinced that it will be an improvement.

glessard · December 10, 2015, 9:11pm

I mean that the ‘alternatives’ section should explicitly compare the merits of leaving things as-is as compared to going ahead with the proposal. For 3rd-party libraries with limited API, importing unannotated functions as optional sounds good. However, with large frameworks (obviously including Cocoa), I think the community needs to be convinced that it will be an improvement.

Cheers,
Guillaume Lessard

···

On 10 déc. 2015, at 12:52, Fabian Ehrentraud <Fabian.Ehrentraud@willhaben.at> wrote:

Guillaume, could you elaborate on "discussing the status quo as an alternative", it sounds good, but I'm not sure I completely got what you mean :-)

Douglas_Gregor · December 10, 2015, 10:28pm

Too bad to hear you are opposed to the change.
Thank you for the hint at the API dumps - this is really something that should be referenced in the proposal.

First of all, this affects a lot of APIs. I went ahead and grepped through the API dumps [...] to see how often implicitly-unwrapped optionals came through in the Cocoa APIs, and there are lots of them. For example, on OS X, grepping for !’s finds:

We should come to a common way to count the occurrences of affected APIs (despite that users would also need to adapt for un-annotated 3rd-party frameworks, and the own mixed&matched application code).
When I counted, it resulted in less than half the numbers for initializers and functions/methods.

There are many comment lines in the API dumps that contain /*! - we should not count those.

I didn’t, FWIW. Just “func”, “init”, or “var” lines that include !.

Also for functions/methods, input-parameters do not matter, due to automatic Optional wrapping on the Swift side. Maybe closure parameters that have implicitly unwrapped optional arguments themselves should be counted too. (anything else?)

Input parameters do matter, because their type communicates something: an optional input parameter says that “nil” has a meaning. An implicitly unwrapped optional input parameter says “check the docs!”. If we apply your proposal, you’re communicating meaning that “nil” is accepted in these input parameters when in fact the original (Objective-)C API never said such a thing, and very likely doesn’t permit “nil”. That actually makes it harder to use these APIs safely; at least when there are IUOs, we’re communicating “don’t know” to let the programmer decide what to do.

Initializers:

Should we only count "init!" instances?
$ grep init! . | wc -l
=> OSX 152, iOS 42

Other initializers containing ! in another place (I guess not relevant for the proposal)
$ grep -r ! . | grep init | grep -v '\/\*\!' | grep -v init!

Functions/Methods:

Only counting those that have a return type with an implicitly unwrapped optional. This does not take into account closure parameters that have implicitly unwrapped optional arguments themselves though.
$ grep -r ! . | grep func | grep '\->' | grep -v '\/\*\!' | grep -e '\!$' | wc -l
=> OSX 1577, iOS 452

Properties:

$ grep -r ! . | grep -e "let " | grep -v '\/\*\!' | wc -l
=> OSX 1120, iOS 336

This is still a large number of APIs that will require additional effort, where the common case is that you don’t care (because most things should be non-optional).

Constants:

I noticed there are many constants that get translated to implicitly unwrapped optionals (e.g. in iOS CoreFoundation.swift). Would they be affected by the change in this proposal? Or are they "explicitly set to implicitly unwrapped" by the importer?

You can grep for “ull_unspecified” in the SDK to find the cases where things are explicitly set to implicitly unwrapped optionals; I doubt there are many.

Most of the implicitly unwrapped optionals we currently get from (Objective-)C APIs can, in fact, never be nil. Forcing programmers to deal with all of these values as full optionals leads to a ton of optional-related boilerplate that doesn’t actual help users write better code. Rather, it dilutes the effectiveness of optionals as a language feature: optionals should mean “nil matters here, think about it carefully!”. But if some large number of optionals people deal with on a day-to-day basis are instead “the compiler didn’t know, so you have to write !”, the useful meaning of the “true” optionals gets lost in the noise.

I think exactly the opposite. If an API has not yet been audited, the client needs to consult the documentation if a return value can ever be nil. If not, she might explicitly and knowingly unwrap the value safely or non-safely. I deem this a good thing, as it prevents unknowingly unsafe use. I do not think that it would get in the way, as the syntactic sugar for Optionals is quite minimal. Also when the API gets audited for nullability later on, the compiler will hint which optional unwrappings have been rendered unnecessary.

You’re discounting my main argument, which is that the safety provided by optionals is contingent on the nil case being interesting most of the time. That’s generally true in “pure” Swift code—Swift programmers use optionals when nil matters—and it’s generally true in imported (Objective-)C code that has been audited, but your change would create a lot of optionals where “nil” isn’t actually interesting in practice.

- Doug

···

On Dec 10, 2015, at 1:22 PM, Fabian Ehrentraud <Fabian.Ehrentraud@willhaben.at> wrote:

On 10.12.2015, at 19:27, Douglas Gregor <dgregor@apple.com> wrote:

On Dec 10, 2015, at 3:42 AM, Fabian Ehrentraud via swift-evolution <swift-evolution@swift.org> wrote:

Are there any objections to variant 2 "Import un-annotated code as Optional"? If not, I would create the proposal.

I’m opposed to this.

First of all, this affects a lot of APIs. I went ahead and grepped through the API dumps over at

  GitHub - apple/swift-3-api-guidelines-review

to see how often implicitly-unwrapped optionals came through in the Cocoa APIs, and there are lots of them. For example, on OS X, grepping for !’s finds:

  4132 functions/methods
  2281 properties
  409 initializers

On iOS, which has a smaller number of overall APIs, the numbers are still pretty big:
  1071 functions/methods
  320 properties
  134 initializers

This may be a slight oversampling of the data—some of those might be explicitly annotated as _Null_unspecified in the (Objective-)C headers, some might be unavailable—but the scale of the impact here is very, very large.

Most of the implicitly unwrapped optionals we currently get from (Objective-)C APIs can, in fact, never be nil. Forcing programmers to deal with all of these values as full optionals leads to a ton of optional-related boilerplate that doesn’t actual help users write better code. Rather, it dilutes the effectiveness of optionals as a language feature: optionals should mean “nil matters here, think about it carefully!”. But if some large number of optionals people deal with on a day-to-day basis are instead “the compiler didn’t know, so you have to write !”, the useful meaning of the “true” optionals gets lost in the noise.

That said, I’d love for implicitly-unwrapped optionals to be used less often, and getting them further out of the type system would be beneficial for the Swift experience.

  - Doug

Am 09.12.2015 um 07:11 schrieb Chris Lattner <clattner@apple.com>:

On Dec 8, 2015, at 4:09 AM, Fabian Ehrentraud via swift-evolution <swift-evolution@swift.org> wrote:

Swift code accessing Objective-C methods can easily crash - if the Objective-C code does not include nullability attributes, the code is brought as implicitly unwrapped into Swift, where unsafe accesses do not produce compiler warnings.

I created issue SR-104 for this, but as suggested by Jordan Rose it should be discussed on this mailing list first.

Short example:

// Objective-C
- (NSString *)giveMeAString { return nil; }

// Swift
func thisWillCrash() {
let string = someObjectiveCObject.giveMeAString()
let length = string.length // crash
}

The ClangImporter could handle this issue in several safer ways:

1. Only import declarations that are nullability annotated (as proposed by Greg Parker)
Positive side effect would be that it would motivate to write nullability annotations.

This seems unfortunate to me, because it would severely hamper interoperating with Objective-C and *C* APIs that have not been audited. While Apple is keen to audit their APIs, I doubt that all the linux system headers will get updated in a timely manner and users don’t generally have a way to do anything about that.

2. Import un-annotated code as Optional
Values would need to be unwrapped every time, which does not hurt too much due to the easy use of the `?` syntactic sugar.

This is a very interesting idea, one I haven’t considered recently. I agree with you that this is worth considering, and I would love to see IUO just get summarily deleted :-)

-Chris

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Fabian_Ehrentraud · December 10, 2015, 9:29pm

Ok, I'll add this.

Impulse for this proposal was in fact the gradual transition of a project to Swift, so the newer Swift code of an app accessing the older Objective-C code of the same app. That's where this feature would shine and help a lot in the transition phase. But I guess it would not make sense to have different importing behavior for Cocoa.

···

On 10.12.2015, at 22:11, Guillaume Lessard <glessard@tffenterprises.com> wrote:

On 10 déc. 2015, at 12:52, Fabian Ehrentraud <Fabian.Ehrentraud@willhaben.at> wrote:

Guillaume, could you elaborate on "discussing the status quo as an alternative", it sounds good, but I'm not sure I completely got what you mean :-)

I mean that the ‘alternatives’ section should explicitly compare the merits of leaving things as-is as compared to going ahead with the proposal. For 3rd-party libraries with limited API, importing unannotated functions as optional sounds good. However, with large frameworks (obviously including Cocoa), I think the community needs to be convinced that it will be an improvement.

Cheers,
Guillaume Lessard

Maku · December 10, 2015, 9:33pm

I, on the other hand, totally agree with this proposal, even though it would break a lot of existing code. I believe that forcing the unwrap doesn’t cost much, especially with the new guard syntax introduced in Swift 2.0. This is obviously not a change that can be done in a minor release, but because we are in process of getting Swift 3.0 I believe we could introduce such behavior. This would somewhat force both Apple and 3rd party developers to make them use the nullability keywords even more in their code.

Regards,
M.

···

On Dec 10, 2015, at 10:22 PM, Fabian Ehrentraud via swift-evolution <swift-evolution@swift.org> wrote:

Too bad to hear you are opposed to the change.
Thank you for the hint at the API dumps - this is really something that should be referenced in the proposal.

First of all, this affects a lot of APIs. I went ahead and grepped through the API dumps [...] to see how often implicitly-unwrapped optionals came through in the Cocoa APIs, and there are lots of them. For example, on OS X, grepping for !’s finds:

We should come to a common way to count the occurrences of affected APIs (despite that users would also need to adapt for un-annotated 3rd-party frameworks, and the own mixed&matched application code).
When I counted, it resulted in less than half the numbers for initializers and functions/methods.

There are many comment lines in the API dumps that contain /*! - we should not count those.
Also for functions/methods, input-parameters do not matter, due to automatic Optional wrapping on the Swift side. Maybe closure parameters that have implicitly unwrapped optional arguments themselves should be counted too. (anything else?)

Initializers:

Should we only count "init!" instances?
$ grep init! . | wc -l
=> OSX 152, iOS 42

Other initializers containing ! in another place (I guess not relevant for the proposal)
$ grep -r ! . | grep init | grep -v '\/\*\!' | grep -v init!

Functions/Methods:

Only counting those that have a return type with an implicitly unwrapped optional. This does not take into account closure parameters that have implicitly unwrapped optional arguments themselves though.
$ grep -r ! . | grep func | grep '\->' | grep -v '\/\*\!' | grep -e '\!$' | wc -l
=> OSX 1577, iOS 452

Properties:

$ grep -r ! . | grep -e "let " | grep -v '\/\*\!' | wc -l
=> OSX 1120, iOS 336

Constants:

I noticed there are many constants that get translated to implicitly unwrapped optionals (e.g. in iOS CoreFoundation.swift). Would they be affected by the change in this proposal? Or are they "explicitly set to implicitly unwrapped" by the importer?

Most of the implicitly unwrapped optionals we currently get from (Objective-)C APIs can, in fact, never be nil. Forcing programmers to deal with all of these values as full optionals leads to a ton of optional-related boilerplate that doesn’t actual help users write better code. Rather, it dilutes the effectiveness of optionals as a language feature: optionals should mean “nil matters here, think about it carefully!”. But if some large number of optionals people deal with on a day-to-day basis are instead “the compiler didn’t know, so you have to write !”, the useful meaning of the “true” optionals gets lost in the noise.

I think exactly the opposite. If an API has not yet been audited, the client needs to consult the documentation if a return value can ever be nil. If not, she might explicitly and knowingly unwrap the value safely or non-safely. I deem this a good thing, as it prevents unknowingly unsafe use. I do not think that it would get in the way, as the syntactic sugar for Optionals is quite minimal. Also when the API gets audited for nullability later on, the compiler will hint which optional unwrappings have been rendered unnecessary.

On 10.12.2015, at 19:27, Douglas Gregor <dgregor@apple.com> wrote:

On Dec 10, 2015, at 3:42 AM, Fabian Ehrentraud via swift-evolution <swift-evolution@swift.org> wrote:

Are there any objections to variant 2 "Import un-annotated code as Optional"? If not, I would create the proposal.

I’m opposed to this.

First of all, this affects a lot of APIs. I went ahead and grepped through the API dumps over at

  GitHub - apple/swift-3-api-guidelines-review

to see how often implicitly-unwrapped optionals came through in the Cocoa APIs, and there are lots of them. For example, on OS X, grepping for !’s finds:

  4132 functions/methods
  2281 properties
  409 initializers

On iOS, which has a smaller number of overall APIs, the numbers are still pretty big:
  1071 functions/methods
  320 properties
  134 initializers

This may be a slight oversampling of the data—some of those might be explicitly annotated as _Null_unspecified in the (Objective-)C headers, some might be unavailable—but the scale of the impact here is very, very large.

Most of the implicitly unwrapped optionals we currently get from (Objective-)C APIs can, in fact, never be nil. Forcing programmers to deal with all of these values as full optionals leads to a ton of optional-related boilerplate that doesn’t actual help users write better code. Rather, it dilutes the effectiveness of optionals as a language feature: optionals should mean “nil matters here, think about it carefully!”. But if some large number of optionals people deal with on a day-to-day basis are instead “the compiler didn’t know, so you have to write !”, the useful meaning of the “true” optionals gets lost in the noise.

That said, I’d love for implicitly-unwrapped optionals to be used less often, and getting them further out of the type system would be beneficial for the Swift experience.

  - Doug

Am 09.12.2015 um 07:11 schrieb Chris Lattner <clattner@apple.com>:

On Dec 8, 2015, at 4:09 AM, Fabian Ehrentraud via swift-evolution <swift-evolution@swift.org> wrote:

Swift code accessing Objective-C methods can easily crash - if the Objective-C code does not include nullability attributes, the code is brought as implicitly unwrapped into Swift, where unsafe accesses do not produce compiler warnings.

I created issue SR-104 for this, but as suggested by Jordan Rose it should be discussed on this mailing list first.

Short example:

// Objective-C
- (NSString *)giveMeAString { return nil; }

// Swift
func thisWillCrash() {
let string = someObjectiveCObject.giveMeAString()
let length = string.length // crash
}

The ClangImporter could handle this issue in several safer ways:

1. Only import declarations that are nullability annotated (as proposed by Greg Parker)
Positive side effect would be that it would motivate to write nullability annotations.

This seems unfortunate to me, because it would severely hamper interoperating with Objective-C and *C* APIs that have not been audited. While Apple is keen to audit their APIs, I doubt that all the linux system headers will get updated in a timely manner and users don’t generally have a way to do anything about that.

2. Import un-annotated code as Optional
Values would need to be unwrapped every time, which does not hurt too much due to the easy use of the `?` syntactic sugar.

This is a very interesting idea, one I haven’t considered recently. I agree with you that this is worth considering, and I would love to see IUO just get summarily deleted :-)

-Chris

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution