Swift 4, stage 2 starts now

Back in July, we laid out a plan for Swift 4 which divided the release into two stages. Since then, we’ve been in Swift 4 stage 1, which is characterized by the following text in the swift-evolutionrepository's README.md <swift-evolution/README.md at master · apple/swift-evolution · GitHub; file as follows:

Stage 1 focuses on the essentials required for source and ABI stability. Features that don't fundamentally change the ABI of existing language features or imply an ABI-breaking change to the standard library will not be considered in this stage. Swift 4 is currently only considering proposals that fit in Stage 1.

Stage 2 will commence once the implementation work on the Stage 1 features is cresting, and can contain a few other large and small features. We expect that stage 2 will commence some time in Spring 2017.
Since July, we now have a much better understanding now of how to achieve ABI stability, with an ABI Manifesto <https://github.com/apple/swift/blob/master/docs/ABIStabilityManifesto.md&gt; detailing the list of all language/implementation work that is needed to achieve ABI stability. We have made substantial progress in that work during stage 1, but much remains to be done. Once Swift achieves ABI stability the ABI can be extended, but not changed. Thus the cost of locking down an ABI too early is quite high.

Deferring ABI Stability from Swift 4

Given the importance of getting the core ABI and the related fundamentals correct, we are going to defer the declaration of ABI stability out of Swift 4 while still focusing the majority of effort to get to the point where the ABI can be declared stable.

To allow the community to follow along with this effort, an ABI dashboard will get wired up from the swift-evolution <https://github.com/apple/swift-evolution&gt; home page that will present a table of main ABI tasks remaining and what Swift release they landed in. This dashboard will largely track open tasks in JIRA. I expect the dashboard to be up next week, and I'll send a follow up email when it is available.

Stage 2
With ABI stability well-understood and many of the stage 1 goals underway, it is time to open up Swift 4 stage 2 to expand the scope of proposals to be considered.

Timeline

Stage 2 starts right now. All design work and discussion for stage 2 extends to April 1, 2017. The intent is to timebox discussion to provide adequate time for the actual implementation of accepted proposals.

Scope

Swift 4 stage 2 builds on the goals of stage 1. It differs in that stage 2 proposals may include some additive changes and changes to existing features that don't affect the ABI. There are a few focus areas for Swift 4 stage 2:

Stage 1 proposals: Any proposal that would have been eligible for stage 1 is a priority for stage 2.

Source-breaking changes: The Swift 4 compiler will provide a source-compatibility mode to allow existing Swift 3 sources to compile, but source-breaking changes can manifest in "Swift 4" mode. That said, changes to fundamental parts of Swift's syntax or standard library APIs that breaks source code are better front-loaded into Swift 4 than delayed until later releases. Relative to Swift 3, the bar for such changes is significantly higher:

The existing syntax/API being changed must be actively harmful.
The new syntax/API must clearly be better and not conflict with existing Swift syntax.
There must be a reasonably automatable migration path for existing code.
Improvements to existing Standard Library facilities: Additive changes that improve existing standard library facilities can be considered. With standard library additions in particular, proposals that provide corresponding implementations are preferred. Potential focus areas for improvement include collections (e.g., new collection algorithms) and improvements to the ergonomics of Dictionary.

Foundation improvements: We anticipate proposing some targeted improvements to Foundation API to continue the goal of making the Cocoa SDK work seamlessly in Swift. Details on the specific goals will be provided as we get started on Swift 4 stage 2.

As part of initiating Swift 4 stage 2, the Core Team reviewed those proposals that were accepted into Swift 3 but were never implemented:

SE-0104 "Protocol-oriented Integers": This proposal requires revision and another round of review based on other changes that have been made to the language and standard library since acceptance.

SE-0075 "Adding a Build Configuration Import Test": This is (still) an additive proposal and no other changes affect this proposal. It will be carried forward and considered accepted for Swift 4.

SE-0042 "Flattening the function type of unapplied method references”: This proposal requires another round of review. This is a significant source-breaking change, and the bar for such source-breaking changes is considerably higher in Swift 4 than it was in Swift 3.

SE-0068 "Expanding Swift Self to class members and value types": This is (still) an additive proposal and no other changes affect this proposal. It will be carried forward and considered accepted for Swift 4.

The Core Team has also identified some starter proposals <[SR-3316] "Script mode" globals have internal visibility and can be accessed before initialization from other files · Issue #45904 · apple/swift · GitHub; that fit well with the goals of Swift 4. Taking one of those ideas and developing it into a complete proposal is a fantastic way to get involved in the Swift evolution process.

Ted

Given where we are in the yearly schedule, I think that this is a pragmatic decision. ABI stability is far more important to Apple than it is to most developers, so I’m happy to see that you’re prioritizing the needs of the community (improved compile time, compiler stability, etc), particularly given the importance of doing the right thing for the long term success of Swift.

Beyond that, I agree that it is prudent to continue work on the many ABI stability tasks despite it not being a goal for Swift 4. Given the significance of declaring ABI stability, it would be great to get these tasks really nailed down early in the Swift 5 schedule so you have time to bake it out.

Do you have any idea what the typical download size impact of the Swift 4 libraries will be on the ARM64 slice of a typical iOS app? I know that many of the ABI-related optimizations also contribute to a significant code size improvement, but don’t know how folks expect that to net out. Do you think that it is plausible to get the overlays coalesced with the stdlib into a single dylib, improving app launch times?

-Chris

···

On Feb 16, 2017, at 4:18 PM, Ted Kremenek via swift-evolution <swift-evolution@swift.org> wrote:

Deferring ABI Stability from Swift 4

Given the importance of getting the core ABI and the related fundamentals correct, we are going to defer the declaration of ABI stability out of Swift 4 while still focusing the majority of effort to get to the point where the ABI can be declared stable.

Because of the narrow scope and tight schedule, the core team will be looking at the various proposals coming in and actively deferring/closing those that are out of scope. A member of the core team will reply to the thread here on swift-evolution and/or annotate the pull request with our rationale when we do this.

  - Doug

···

On Feb 16, 2017, at 4:18 PM, Ted Kremenek via swift-evolution <swift-evolution@swift.org> wrote:

Stage 2

With ABI stability well-understood and many of the stage 1 goals underway, it is time to open up Swift 4 stage 2 to expand the scope of proposals to be considered.

Timeline

Stage 2 starts right now. All design work and discussion for stage 2 extends to April 1, 2017. The intent is to timebox discussion to provide adequate time for the actual implementation of accepted proposals.

Scope

Swift 4 stage 2 builds on the goals of stage 1. It differs in that stage 2 proposals may include some additive changes and changes to existing features that don't affect the ABI. There are a few focus areas for Swift 4 stage 2:

Stage 1 proposals: Any proposal that would have been eligible for stage 1 is a priority for stage 2.

Source-breaking changes: The Swift 4 compiler will provide a source-compatibility mode to allow existing Swift 3 sources to compile, but source-breaking changes can manifest in "Swift 4" mode. That said, changes to fundamental parts of Swift's syntax or standard library APIs that breaks source code are better front-loaded into Swift 4 than delayed until later releases. Relative to Swift 3, the bar for such changes is significantly higher:

The existing syntax/API being changed must be actively harmful.
The new syntax/API must clearly be better and not conflict with existing Swift syntax.
There must be a reasonably automatable migration path for existing code.
Improvements to existing Standard Library facilities: Additive changes that improve existing standard library facilities can be considered. With standard library additions in particular, proposals that provide corresponding implementations are preferred. Potential focus areas for improvement include collections (e.g., new collection algorithms) and improvements to the ergonomics of Dictionary.

Foundation improvements: We anticipate proposing some targeted improvements to Foundation API to continue the goal of making the Cocoa SDK work seamlessly in Swift. Details on the specific goals will be provided as we get started on Swift 4 stage 2.

Deferring ABI Stability from Swift 4

Given the importance of getting the core ABI and the related fundamentals correct, we are going to defer the declaration of ABI stability out of Swift 4 while still focusing the majority of effort to get to the point where the ABI can be declared stable.

Given where we are in the yearly schedule, I think that this is a pragmatic decision. ABI stability is far more important to Apple than it is to most developers, so I’m happy to see that you’re prioritizing the needs of the community (improved compile time, compiler stability, etc), particularly given the importance of doing the right thing for the long term success of Swift.

Beyond that, I agree that it is prudent to continue work on the many ABI stability tasks despite it not being a goal for Swift 4. Given the significance of declaring ABI stability, it would be great to get these tasks really nailed down early in the Swift 5 schedule so you have time to bake it out.

Absolutely — giving time for these changes to bake to make sure we did not miss anything is essential part of keeping the focus in Swift 4 on core ABI stability tasks (and other language fundamentals that impact ABI and source stability). That thinking went into the scoping of stage 2.

Do you have any idea what the typical download size impact of the Swift 4 libraries will be on the ARM64 slice of a typical iOS app? I know that many of the ABI-related optimizations also contribute to a significant code size improvement, but don’t know how folks expect that to net out. Do you think that it is plausible to get the overlays coalesced with the stdlib into a single dylib, improving app launch times?

This is an interesting idea, but off-topic for swift-evolution. A better place to discuss this is swift-dev.

That said, there are tradeoffs here: creating a single dylib slightly complicates the long-term goal of sinking the overlay APIs back into the OS, and the number of overlays is not guaranteed to remain fixed so the tradeoffs of creating a single dylib versus multiple are nuanced. The focus has been on the ABI-related optimizations, such as reducing the size of mangled symbols, which are tasks that absolutely need to be done before ABI stability.

···

On Feb 16, 2017, at 10:17 PM, Chris Lattner <clattner@nondot.org> wrote:
On Feb 16, 2017, at 4:18 PM, Ted Kremenek via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote: