[Pitch 2] Light-weight same-type requirement syntax

Evolution Pitches
1

Hi all,

Here is the proposal allowing you to write Collection<String> and similar anywhere that a protocol conformance requirement can be written: https://github.com/apple/swift-evolution/pull/1538

This is based on the earlier pitch by @xedin and @hborla: [Pitch] Light-weight same-type constraint syntax.

The proposal isn't quite ready for review because the implementation is incomplete; the type representation syntax works, but the declaration side uses the @_primaryAssociatedType attribute and not the proposed generic parameter list syntax. It goes without saying that this is all behind a feature flag; use -Xfrontend -enable-parametrized-protocol-types if you want to experiment with it. I hope to have the new declaration-side syntax implemented in the coming days so that we can begin the formal review.

I have removed the material from the old pitch about the sugared syntax for extensions of concrete types, like extension Array<String>. We feel this is better pitched in a separate proposal.

38 Likes
2

:clap: :clap:

Can’t wait for this! I think the scope and the syntax at both declaration and the use site are just right.

To clarify, from the source/ABI/API compatibility portions of the proposal:

  1. It is safe from an ABI persepective to adopt primary associated type for standard library protocols, but
  2. The proposal doesn’t include that adoption, just the language feature.

Do I have that correct?

3

This is great! I have convinced myself that this is a very intuitive meaning for MyProtocol<AssociatedType>, and I expect this will do wonders for easing the generics learning curve. I wholeheartedly agree with aligning the protocol declaration syntax and the parameterization syntax rather than adding some sort of annotation on an associatedtype declaration.

I am also optimistic that we will eventually want to be able to provide explicit argument labels for generic parameterizations anyway (e.g. for types which want to have two variadic generic parameters), which would admit a further (intuitive, IMO) syntax here, e.g. Collection<String, Iterator: SomeConcreteIterator>.

IMO biggest potential concern here (given what features people have requested on the forums in the past) is that using this generics-like syntax for same-type constraints and primary associated types essentially cuts off any chance for generic protocols in the language. I'm not particularly concerned by this—in the event that in Swift X we do actually want such a feature, I believe it could be served by allowing protocols nested within (and inheriting) generic contexts, e.g.,

enum Generic<T> {
  protocol Interface {}
}

struct S {}
extension S: Generic<String>.Interface {}
extension S: Generic<Int>.Interface {}

AFAICT this would serve all the purposes that "true" generic protocols would, and IMO has clearer semantics about the identity of different versions of the Generic.Interface protocol.

+1000!

9 Likes
4

Correct. However we still have to work out a story for module interfaces, probably using an #if $feature to conditionally emit every protocol with a primary associated type twice, once using the new syntax and once without. It's a bit gross but not a huge deal.

That is also correct. Of course adopting it in the standard library is important, we just felt that this adoption could move in parallel with the compiler work.

1 Like
5

To be honest, as someone who has been working on the generics implementation for many years, it is still not clear to me what a hypothetical "generic protocols" feature actually means and whether such a generalization is desirable or even feasible to implement.

4 Likes
6

So I would expect that this syntax would preserve throwing semantics of the type. Specifically for this to be useful for AsyncSequence (which do not misconstrue my critique... this is VERY important for AsyncSequence... no one really wants to write .eraseToAnyPublisher() over and over...) it needs to preserve the throwyness or non-throwyness of the base type. As of current the implementation does not seem to do so.

Is that a consideration that is going to be accounted for in this proposal?

Here is an example I just ran on nearly top of tree and sadly it seems to fail:


@rethrows
protocol Foo {
  @_primaryAssociatedType associatedtype Element
  func bar() throws -> Element
}

struct NonthrowingFoo: Foo {
  func bar() -> String {
    return "bar"
  }
}

func testNonthrowing() -> some Foo<String> {
  NonthrowingFoo()
}

print(testNonthrowing().bar()) // call can throw but is not marked with 'try'

It is worth noting that this does not require the @rethrows to be there for the behavior to incorrectly propagate.

3 Likes
7

I'm still apprehensive about this. These are not generics and as such, I'm not convinced that they make the generics system easier to understand.

The problem of adding constraints to opaque types is real, but I think we can do better than this. One same-type constraint, half of which is defined by the protocol author rather than the user, is not an adequate solution.

I'm also not convinced that same-type constraints are worth blessing with special syntax. It makes generics less generic, and feels similar to the mistake we made with existentials. Some of that concern might be alleviated if we had the syntax some Collection<some StringProtocol> available to express a subtype constraint.

But even if it was, I'm still not sure this is the right call. The community has had very mixed feelings about this since it was pitched, and it seems like nothing has really changed. I'm also concerned that we're just "doing things" without a clear or ambitious-enough vision of where we want to go.

20 Likes
8

In the Detailed design section, are the examples with Element : String correct?

Should they be Element == String or Element : StringProtocol instead?

2 Likes
9

I think some Collection<some StringProtocol> as sugar for T : Collection where T.Element : StringProtocol falls out naturally from Doug's opaque parameters pitch and this. It might need some implementation work but I don't see any conceptual problems with this.

5 Likes
10

Apologies if this is specified and I've overlooked it. Is the following allowed, inferring the primary associatedtype in the conformance?

struct Lines: Sequence<String> { /* ... */ }
11

That is not allowed currently, since an inheritance clause entry on a concrete type does not desugar to a generic requirement on Self, so there is no base type to apply the same-type requirement to.

You could say that this is equivalent to

struct Lines: Sequence {
  typealias Element = String
  ...
}

But that's more of a stretch, in my opinion. However we could offer this as a fix it if the user writes struct Lines: Sequence<String>.

2 Likes
12

Looking forward to this! My only hope is that "multiple primary associated types" are prioritized sooner. Would be a shame to not be able to do things like:

-> some Publisher<Int, Never>
16 Likes
13

I’d actually go so far as to say this should be supported out the door. Is there any technical reason for this restriction?

6 Likes
14

Agreed—an example like this would be great to note in the proposal text.

In particular, banning struct Lines: Sequence<String> helps reinforce the notion that these are not generic protocols: a developer coming from e.g. Java can be guided into the realization that only one conformance to a protocol is allowed, by nature of only : Sequence being supported.

1 Like
15

What is the performance impact of using this? For example; would it be reasonable to have had the map operation return a some type? If a type is frozen and inlined for performance do we keep that performance or is it obscured by the some-ness?

1 Like
16

No, just conceptual simplicity at this point.

3 Likes
17

If the function is inlinable, the opaque type is effectively just sugar and callers will see the concrete underlying type at the SIL level. If the function is not inlinable, callers will manipulate it abstractly.

The underlying concrete type does not need to be frozen for the inlinable optimization to occur; it being frozen is orthogonal to the opaqueness of the function's result.

3 Likes
18

I was initially quite hesitant about the use of generics syntax to parameterize protocols by their associated types, but I am much more happy about this iteration.

I think my happiness is because of the alignment between how the parameterization is declared and how it’s used. To my mind, it’s basically saying that this is the way in which protocols are parameterized in Swift. Yes, it eliminates room for generic protocols but (as explained in the Generics Manifesto) that feature isn’t really what we’d want to support for very good reasons.

I think this iteration makes it plain, as pointed out by @Jumhyn, that the limitation to one parameter seems arbitrary. Indeed the proposal text offers an example off the bat of SetProtocol<Element>, which makes it pretty glaring that the corresponding hypothetical DictionaryProtocol<Key, Value> isn’t allowed. Not sure that an arbitrary restriction adds to rather than detracts from conceptual simplicity here.

Otherwise I think I’m growing to really like how this is shaping up.

Not to derail the main conversation, but is #if $feature ever going to be pitched for Swift Evolution, or will it remain an undocumented internal feature?

3 Likes
19

I'm still convinced that this feature is both bad syntax, as well as semantics-wise, due to conflating two orthogonal concepts (generic parameters and associated types) into one syntax:

-1

Previous comments

I can't comment on how feasible generic protocols would be to implement in Swift, but as an avid user of generic traits in Rust I can at least try to give some context as to why one would want them (or something equivalent) in Swift:

It's fairly long, so folding it:

Type-safe, compiler-checked value conversions

Type-safe, compiler-checked value conversions

Swift (like Rust) chose to make type conversions explicit. This is great from a correctness and safety perspective, but tends to be very hindering when trying to write abstract code that's generic over its types. The main reason for this is that Swift lacks a way to abstractly express type convertibility.

Say you have a function that is supposed to calculate the fitness of a given value in respect to a certain "ideal" value:

func fitness(actual: Double, ideal: Double) -> Double {
    (actual == ideal) ? 0.0 : abs(actual - ideal) / max(abs(actual), abs(ideal))
}

(This formula is taken from step 7 of the "fitness distance" algorithm defined by the WebRTC spec, which I had to implement recently and where I found myself—yet again—in need of generic protocols.)

Now say that the values you need to compute the fitness for have different types then Double, say Int:

func fitness(actual: Int, ideal: Int) -> Double {
    fitness(actual: Double(actual), ideal: Double(ideal))
}

or Float:

func fitness(actual: Float, ideal: Float) -> Double {
    fitness(actual: Double(actual), ideal: Double(ideal))
}

Now, these two latter functions work just on their own. But they are of no use when you need to abstract over the input type.

Sure, for this particular scenario you could write a protocol like this:

protocol DoubleConvertible {
    func asDouble() -> Double
}

and just have all necessary types conform to said protocol.

func fitness<Value>(actual: Value, ideal: Value) -> Double where Value: DoubleConvertible {
    fitness(actual: actual.asDouble(), ideal: ideal.asDouble())
}

But this approach has two major limitations:

  • it only "solves" the issue for Double. What about Float? Or Int? Or String? …

  • you can't make fitness(actual:ideal:) generic over multiple types:

    func fitness<Input, Output>(actual: Input, ideal: Input) -> Output where Input: ???Convertible {
    fitness(actual: actual.as???(), ideal: ideal.as???())
}

If we were to declare type-specific protocols for every type we possibly might want to convert to, we would be polluting our project's namespace with an immense amount of redundant garbage (and not be gaining much from it semantically, either).

Just do the math: Given N interchangeably convertible types we would need do define N individual protocols of the pattern protocol <…>Convertible { … } and come up with N unique, yet expressive method names to go with them?

And what if this N gets getting bigger and bigger? And what if instead of dealing with concrete types you were working on generic types or methods/functions, and hence no way to have the Swift compiler pick the right explicit protocol for a given generic type T?

This is where generic protocols safe the day!

protocol ConvertibleInto<T> {
  func into() -> T
}

But wait, can't we do this already with a protocol with associated types, like so? …

protocol ConvertibleInto {
    associatedtype T
    
    func into() -> T
}

Well, it depends. While you could easily implement Foo: CustomStringConvertible in terms of …

struct Foo {
    let bar: Int
}

extension Foo: ConvertibleInto {
    typealias T = String
    
    func into() -> T {
        return …
    }
}

… you would get in trouble as soon as you were to decide that it would be nifty to also be able to convert instances of Foo to Data:

// error: redundant conformance of 'Foo' to protocol 'ConvertibleInto':
extension Foo: ConvertibleInto {
    // error: invalid redeclaration of 'T':
    typealias T = Data
    
    // error: invalid redeclaration of 'into()':
    func into() -> T {
        return  …
    }
}

In today's Swift a protocol (regardless of whether it has associated types, or not) can only be conformed to once by any single type. In order to achieve polymorphic semantics of protocol ConvertibleInto we however would need to have a way to allow for multiple conformances per type.

Generic protocols would allow for this:

extension Foo: ConvertibleInto<String> {
    func into() -> String {
        return  …
    }
}

extension Foo: ConvertibleInto<Data> {    
    func into() -> Data {
        return  …
    }
}

In addition to a hypothetical ConvertibleInto<T> we would probably also want to have access to a corresponding ConstructibleFrom<T> protocol going the other way:

protocol ConstructibleFrom<T> {
    init(from value: T)
}

As such a hypothetical Real type might be constructible from both Float and Double, e.g.:

struct Real { /* ... */ }
extension Real: ConstructibleFrom<Float> { /* ... */ }
extension Real: ConstructibleFrom<Double> { /* ... */ }

(We will be using extension<…> as a hypothetical syntax for introducing generic arguments into an extension scope.)

Going even further one might want to have the Swift compiler automatically derive conformance of ConstructibleInto<U> for every type U: ConstructibleFrom` with a default implementation like so:

extension<T, U> T: ConvertibleInto<U> where U: ConstructibleFrom<T> {
    func into() -> U {
        return U(from: self)
    }
}

One might also want to have every type T auto-derive conformance to T: ConstructibleFrom<T> like so:

extension<T> T: ConvertibleInto<T> {
    func into() -> T {
        return self
    }
}

With generic protocols at one's disposal one could —for example— unify all of …

  • UIColor's var cgColor: CGColor
  • UIImage's var cgImage: CGImage?
  • UIColor's var ciImage: CIImage?

… into the single universal ConvertibleInto<T> protocol, and individually conform to it like so:

extension UIImage: ConvertibleInto<CGImage> {
    func into() -> CGImage {
        return …
    }
}

… which would then allow one to nicely write …

func draw(image: T) where T: ConvertibleInto<CGImage> {
    let cgImage: CGImage = image.into()
    // …
}

… having it accept images of any type that's convertible to CGImage (e.g. UIImage, NSImage, CGImage, …).

Generic overloading

Generic overloading

With all this talk about multiple conformances of protocols one might wonder "wait, isn't what what Swift has function overloading for? We already can implement variants of a function based in argument and/or return types, why need protocols for that?"

And of course some truth to this sentiment.

Let's assume we wanted to write an efficient and type-safe linear algebra framework for Swift. We would probably end up defining types for scalars, vectors and matrixes, like so:

struct Scalar<T> { /* ... */ }
struct Vector<T> { /* ... */ }
struct Matrix<T> { /* ... */ }

And it would not take long until one needed some way to perform arithmetic operations on them, such as multiplication:

extension Vector {
    // vector scaling:
    func *(lhs: Self, rhs: Scalar<T>) -> Vector<T> { /* ... */ }
    
    // dot product:
    func *(lhs: Self, rhs: Vector<T>) -> Scalar<T> { /* ... */ }

    // vector matrix product:
    func *(lhs: Self, rhs: Matrix<T>) -> Vector<T> { /* ... */ }
}

Notice how each method's return type directly depends on the type of rhs.

This works as long as one is dealing with explicit types, exclusively. But sooner or later one would want to be able to generalize over scalars, vectors and matrixes. (After all from the point of algebra they are just tensors of 0, 1, or 2 dimensions respectively.)

As it turns out there is no way to express overloading in today's Swift from the perspective of protocols. It's a dead spot in Swift's generics type system.

If one had generic protocol at one's disposal however one could express things like this:

protocol Multiplication<Rhs = Self> {
    associatedtype Output
    
    func *(lhs: Self, rhs: Rhs) -> Output
}

// vector scaling:
extension Vector: Multiplication<Scalar<T>> {
    typealias Output = Vector<T>
    func *(lhs: Self, rhs: Rhs) -> Output { /* ... */ }
}

// dot product:
extension Vector: Multiplication<Vector<T>> {
    typealias Output = Scalar<T>
    func *(lhs: Self, rhs: Rhs) -> Output { /* ... */ }
}

// vector matrix product:
extension Vector: Multiplication<Matrix<T>> {
    typealias Output = Vector<T>
    func *(lhs: Self, rhs: Rhs) -> Output { /* ... */ }
}

Now whenever one needs to be generic over a type U and require it to be multipliable with Vector<T>, one could express it through where Vector<T>: Multiplication<U>.

These are just two of the many use-cases for generic protocols off the top of my head. There are many more.

I don't see how avoiding having to write Collection<.Element == Int> in favor of Collection<Int> is worth effectively shutting the door for any possibility of proper generic protocols ever making it into Swift.

Not being able to express overloading (and abstracting of it in generic contexts) in protocols is one of one of the biggest gaps in Swift today and a daily annoyance for anybody writing generics-heavy code in Swift.

18 Likes
20
Details hidden because this is a bit of a tangent, but still slightly relevant insofar as it impacts the syntax here

I have always understood the generic protocols as having semantics similar to the (intuitive to myself) semantics of the Generic<T>.Interface example. Just as Generic<Int> and Generic<String> are distinct types related by being parameterizations of the same generic base, Generic<Int>.Interface and Generic<String>.Interface are distinct protocols related by being (indirect) instantiations of the same generic base.

For a hypothetical "true" generic protocol MyGenericProto<T>, types would be able to conform separately to MyGenericProto<Int>, MyGenericProto<String>, etc. etc.

Beyond my intuition about what "generic protocols" mean, though, I have no idea about the ultimate soundness or feasibility of such a system. I myself have not really encountered a spot where I wanted to use a generic protocol, so the using the "obvious" syntax for generic protocols as the syntax for same-type constraints is not that concerning for me.

4 Likes