Parameterized Extensions

Parameterized Extensions

Introduction

Hello Evolution, this is my first draft at a proposal for Parameterized Extensions. I would greatly appreciate any feedback, suggestions, and examples to use as well! If there's any areas you feel could use some clarification, let me know! I know I'm just pitching this, but hopefully I laid out the structure of this well enough to start discussion on.

This proposal aims to enable users to be able to supply extensions with generic parameters and to be able to extend specialized types.

Motivation

Currently in Swift, one cannot give extensions generic parameters to build expressive generic signatures for new members. Consider the example:

// You can't really express this extension where any array whose element is an optional type.
// This doesn't work.
extension Array where Element is Optional {
  var someValues: [T] {
    var result = [T]()
    for opt in self {
      if let value = opt { result.append(value) }
    }
   return result
  }
}

The above extension is almost impossible to currently express. You could get around it in a few ways by:

  1. Creating an OptionalProtocol and giving conformance to Optional:
protocol OptionalProtocol {}

extension Optional: OptionalProtocol {}

extension Array where Element: OptionalProtocol {}
  1. Using a function instead of a computed property and making the function generic:
extension Array {
  func someValues<T>() -> [T] where Element == T? {
    // ...
  }
}

Both workarounds are sub-optimal. With #1, you have to go through a level of indirection to ensure that the element type is Optional, but you aren't granted any of the methods on Optional for free. #2 is better, but what you really wanted was a computed property which is a constraint on the expressivity that Swift aims to provide with its Generics model. This also starts to become boilerplate when you want to have multiple members with the same generic signature. It would be nice to be able to define a common generic signature for all members, including computed properties.

Extending specialized types is also an awkward example of the current generics model for extensions.

struct Pair<Element> {
  let first: Element
  let second: Element
}

// error: constrained extension must be declared on the unspecialized generic
//        type 'Pair' with constraints specified by a 'where' clause
extension Pair<Int> {
  var sum: Int { first + second }
}

// Okay, but why not the more straight forward syntax?
extension Pair where Element == Int {
  // ...
}

Requiring users to use the second syntax is a little weird because now they have to remember the generic parameter names, and some types may not provide meaningful parameter names (extension SomeType where T == U, what is T and what is U?).

Proposed solution

Parameterized Extensions

Extensions can now be decorated with a generic parameter list to be used with constructing a generic signature when extending types. Using the array of optionals example above, we can now write:

extension<T> Array where Element == Optional<T> {
 // ...
}

to extend all arrays whose element type is an optional of any type. You can of course use the optional type sugar now to do:

extension<T> Array where Element == T? {}

With a generic parameter list, users can also define generic types that conform to protocols.

// Extend all arrays whose elements are optionals whose wrapped type conforms to FixedWidthInteger
extension<T: FixedWidthInteger> Array where Element == T? {
  var sum: T {
    // for all non nil elements, add em up
  }
}

Extending types with same type requirements

Throughout the language grammar, supplementing a generic type with types produces a generic signature with something called same type requirements. We saw them earlier with Array where Element == T? where the generic parameter Element has the same type as T?. We can simplify this syntax into what we're all comfortable writing, Array<T?>.

// Extend array whose element type is an optional T
extension<T> Array<T?> {}

Extending specialized types

This feature goes hand in hand with Extending types with same type requirements, but I propose that we finally allow extending specialized generic types (also known as concrete types). As mentioned in Motivation, we can now use this simpler syntax in extensions without worrying about what the extended type's generic parameter is named.

// We don't need to know that Array's only generic parameter is named Element
// This is especially useful for types that have not so great generic parameter
// names, such as T, U, V, etc.
extension Array<String> {}

Extending sugar types

With the above three new features, we can extend sugar types now. Whether it be generic or a concrete type, users can opt into a single mental model when working with types like [T], [T: U], and T? instead of switching between their canonical form when extending these types.

Examples:

extension [String] {
  func makeSentence() {
    // ...
  }
}

// Extend Array where the element type is an optional T
extension<T> [T?] {}

Detailed design

Swift's extension grammar changes ever so slightly to include a generic parameter clause after the extension keyword:

extension-declaration: attributes (opt) access-level-modifier (opt) extension generic-parameter-clause (opt)
                       type-identifier type-heritance-clause (opt) generic-where-clause (opt) extension-body

It's important to note that the extensions themselves are not technically generic, rather what's happening is that we're supplementing the extended type with new generic parameters. What this allows us to do is essentially copy this new generic signature for all new members within the extension rather than writing out a new generic function for each and every member. It also allows for those generic computed properties that I discussed earlier in Motivation.

Banning generic parameters extensions

When one extends the generic parameter that was declared in the extension, the compiler will diagnose that it's currently unable to do so.

extension<T> T {} // error: cannot extend non-nominal and non-structural type 'T'

I discuss more about this in Future Directions

Conditional Conformance

Parameterized extensions allow for some very neat generic signatures, including conditional conformance.

// If Array's Element type is Equatable, conform to Equatable
extension<T: Equatable> [T]: Equatable {}

Source compatibility

This change is additive, thus source compatibility is unchanged. All of the features discussed currently don't compile, so we aren't hurting source compatibility.

Effect on ABI stability

This feature does affect the ABI, but it doesn't break it. There are cases where one could accidently break their ABI. For example, moving the generic signature from an extended function to the extension is ABI breaking.

// Before
extension Array {
  func someValues<T>() -> [T] where Element == T? { /* ... */ }
}

// After (ABI broke)
extension<T> Array where Element == T? {
  func someValues() -> [T] { /* ... */ }
}

Another example would be using this new syntax for conditional conformance. Rewriting your conditional conformance to use parameterized extensions instead would break ABI.

// Before
extension Array: Equatable where Element: Equatable {}

// After (ABI broke)
extension<T: Equatable> [T]: Equatable {}

For simple cases like renaming extensions to use same type constraints or using the sugar types is ABI compatible.

// Before
extension Array where Element == Int {}

// After (ABI not broke)
extension [Int] {}

Effect on API resilience

This feature does not expose any new public API.

Alternatives considered

There were a couple of minor alternatives that I considered, one being to disallow sugar types. While it could make sense, many of us write properties and parameters using this syntax, so it makes sense to be able to extend them as well to be consistent.

Future Directions

Right now, extending generic parameters are banned. One could extend a generic parameter to add members to all types in the future.

// Extend every type to include a instance member named `abc`
extension<T> T {
  func abc() {}
}

let x = 3
x.abc() // ok

// Extend every type that conforms to ProtoA to conditionally conform to ProtoB as well.
extension<T: ProtoA> T: ProtoB {}

Parameterized extensions could work really well with new generic features such as extending non-nominal types and variadic generics. Using the infamous example from the Generics Manifesto:

// Extend tuple to conform to Equatable where all of its Elements conform to Equatable
extension<...Elements: Equatable> (Elements...): Equtable {}
58 Likes

Nice! One question, though:

Why would this break ABI? Both cases look fairly similar to me, in terms of what they are expressing.

What is considered dangerous is giving conformance of a protocol you don't own to a type you don't own. Giving conformance to a protocol you own to a type you don't is fine. As well, giving conformance to a protocol you don't own to a type you do own is also fine.

9 Likes

This is really an implementation detail at the moment, I'd expect it be ABI compatible myself. Theoretically it could be possible to make it ABI compatible, but I'm going to wait and see how the core team feels because maybe they don't want it to be compatible, or maybe they don't want to have another special case in the compiler.

1 Like

Thanks. I've updated that section.

Thanks to implement parameterized extensions.

I have been waiting for this feature for a long time since I read it in a manifesto.
I'm fed up with defining OptionalProtocol to extend compact method into Array. (compact is same with your someValues.)
No other conforming type for this protocol than Swift.Optional has ever been given in my project.

Direct writing specialized type in extension is also great.
This is obviously clear for both reader and writer of code.

1 Like

This looks great! I don’t have anything to add, but I hope it moves forward soon. Great work. Thank you!

8 Likes

Is the Swift 5.1 branch still taking any features in or only fixes? I‘d love to see this shipping with Swift 5.1.

6 Likes

This is very similar to how rust works. You can have a look here for inspiration in case you didn't knew this already ;)

https://doc.rust-lang.org/book/ch10-02-traits.html#using-trait-bounds-to-conditionally-implement-methods

1 Like

I've often wanted every generic type to implicitly conform to an identically named protocol with the generic parameters as associated types. When I'm thinking about it, I guess it's similar to this proposal in the sense that I want to express:

extension Collection where Element: Optional {
  func compacted() -> [Element.Wrapped] {
    return compactMap { $0 }
  }
}

extension Collection where Element: Promise {
  func all() -> Promise<Element.Value, Element.Error> {
    return Promise.all(Array(self))
  }
}

... etc, without having to explicitly wrap every type in a protocol.
But I guess (?) this solves the same problem, and perhaps (?) in a better way.

I just came across a wonderful motivating example for this proposal. Combine has a CurrentValueSubject type which exposes its current value. Unfortunately the ability to see the current value is lost as soon as an operator is applied. With parameterized extensions we would be able to flow the value up through some of the operators, most importantly map:

extension<UpstreamOutput, Failure: Error> Publishers.Map where Upstream == CurrentValueSubject<UpstreamOutput, Failure> {
    var value: Output { transform(upstream.value) }
}

This implementation could be generalized to support any Upstream type that can provide a value using a CurrentValuePublisher protocol, therefore supporting the ability to chain multiple map operations, etc.

10 Likes

+1 to this feature. It's a big hole in the generics model and it would be great to finally fill it!

I'll echo what others have said above about the ABI break. I don't see any obvious advantage to having each way of writing the same thing to have a separate mangled representation, and only potential pitfalls and annoyances.

1 Like

Would be nice finally provide reversed collection optimization for lazy collections in Foundation

extension<T> LazyCollectionProtocol 
  where Self: BidirectionalCollection, T: BidirectionalCollection, Elements == ReversedCollection<T> { ... }

So that doubly reversed lazy collections return the original lazy collection.

(approach with using additional protocol was declined)

3 Likes

I'll just say you have my vote if this moves onto review.

2 Likes

This is my #1 missing feature in Swift generics system, and I’d love to see it in Swift 5.1

2 Likes

@Alejandro Would you mind giving an update on this? What's the current status?

1 Like

Sure, there were some implementation details that popped up with nested types that I'm trying to fix. I've been wanting to look at ways to ensure that swapping conditional conformance syntaxes are ABI compatible, but I haven't looked too much into it yet. Needless to say, I'm still actively trying to push this forward and make sure we see this sometime soon.

8 Likes

I've been waiting this feature for years, literally. Thank you for working on it!

This looks great! Has anyone from the core team pitched in?

Sorry for taking so long! I wanted to provide an update and ask some questions about expected behavior. Since I've posted this I've had to keep my implementation up to date as I've been pulling in changes while implementing the runtime support for this. I'm getting really close to asking for another review, but I am still working to make sure this gets in soon!

However, I have been prompted to ask a few questions about some expected behaviors that I'd like to discuss. Consider the comparison below:

extension Array {}
// vs.
extension<T> Array<T> {} // or extension<T> [T] {}

Semantically speaking, these extensions do the same thing. However, swapping from one to the other is an ABI breaking change. Should it break when you go from one to the other? Should it not? This ties into the earlier discussion about rewriting conditional conformances to be expected to not break... With parameterized extensions, there's a lot of new ways to write the same extensions and I'm questioning whether they should all be specially treated to be ABI compatible (if possible) with existing extensions forms.

Note: swapping the following are still ABI compatible with each other

extension Array where Element == Int {}
// vs.
extension Array<Int> {}
// vs.
extension [Int] {}

Would love some feedback!

7 Likes