[Pitch 2] Light-weight same-type requirement syntax

There's really no inherent reason for multi-parameter typeclasses to look like "generic protocols" with angle brackets after a protocol name. I still haven't seen anyone explain why instantiating protocol types with concrete arguments on the right hand side of a conformance requirement is a desirable feature, or what semantics it would have.

If you're looking for arbitrary where clauses on named opaque result types, please feel free to write up a pitch; the feature is 80% implemented behind an experimental flag on the main branch already.

4 Likes

I'm sorry I'm not a compiler developer, I wouldn't be able to finish even 1% of the remaining 20%. I find it unpleasant to be rolled over with a "if you want something else, do it / the rest yourself" counterargument. Have a great rest of your day.

6 Likes

This argument feels similarly structured to the argument that OOP-style methods shouldn’t exist because they awkwardly privilege one type in the defintion of equal(to:), and languages should use multimethods instead. I don’t find that form of argument to be very strong.

How did we get from a distinct interpretation of multiple-conformance back to a single conformance with an associated type constraint? Are you intending to imply that interpretation #3 is in fact equivalent to interpretation #1?

3 Likes

The "forcing" and the "sneaks" and the "takes away" come across as frantic and antagonistic. They are neither the tone we would like to set on these forums, nor are they helping your argument.

Fundamentally, your complaint is that this proposal is not syntactic sugar because, for opaque result types, it expresses something new: the ability to provide constraints on the associated types of the opaque result type.

As far as I can tell, you aren't disagreeing with that new feature, i.e., you agree that it is useful to be able to express a result type that is "some Sequence where the Element type is String". Your disagreement seems to come in two parts:

  1. You don't like taking the syntax Sequence<String> for this purpose, because it prevents us from using that syntax for something else in the future.
  2. You want the ability to express a complete where clause for the opaque type, rather than this restricted form.

Regarding (1), I don't actually think we want to use this syntax for the other things that "generic protocols" could mean. Here's my attempt at enumerating those things and why they should be spelled differently. It's not enough to say that generic protocols might need this syntax later; you actually need to make a strong case that this specific syntax is the best syntax for that future feature.

Regarding (2), the full "reverse generics" feature has an explicit type parameter list on the right-hand side of the ->. For example, let's write an "unzip" of sorts:

func unzip<C: Collection, T, U>(_ collection: C) -> <R1: Collection, R2: Collection> (R1, R2)
  where C.Element == (T, U), R1.Element == T, R2.Element == U { ... }

In other words, pass in a collection whose element type is (T, U) and get two collections back, one with the T's and one with the U's. With this proposal (and SE-0341), this can be expressed as:

func unzip<T, U>(_ collection: Collection<(T, U)>) -> (some Collection<T>, some Collection<U>)

That is so much clearer. The reverse-generics formulation isn't just more cluttered, it's forcing you to actively reason about both generic parameter lists and detangle the where clause to understand which bits affect the generic parameters left of the -> and which affect generic parameters to the right of the ->.

Reverse generics are a good conceptual underpinning for opaque result types that precisely matches the implementation model. Indeed, they are implemented in the compiler behind an experimental flag so we could test out all of the complicated combinations and internally desugar this pitch to that implementation. However, it is not at all clear to me that we ever want to surface the full reverse-generics models to users: you have to go very deep into the theory for the reverse-generics desugaring of this pitch to make more sense than other more-accessible ways of understanding opaque result types. This pitch covers the most common cases in a manner that we can teach.

If we did eventually get some other way to do more arbitrary where clauses, e.g., this suggestion:

then that would likely cover the expressivity gap. But I would say that this typealias solution by itself is not good enough to replace this pitch. Would we create Of-suffixed typealias versions of all of the collection protocols in the standard library? C++ did this with their type traits, from original class templates (is_same), to value forms (is_same_v) and finally concept forms (same_as), and the result is an awful mess, bloating the API with 3 names for each idea. We should not knowingly go down the same path.

If supporting an arbitrary where clause is important to be able to express in the language, then that feature needs supporting examples. And if the argument is that the need for an arbitrary where clause is so great that we should block progress on this particular pitch... then it needs to demonstrate that this pitch is going in the wrong direction, rather than just that this pitch isn't going far enough.

Doug

9 Likes

In Swift, we write equal(to:) as == in part because we don't want to privilege one particular type, so I'm not sure where your argument leads.

We didn't. If you allow multiple conformances, you're going to want a convenient way to talk about a specific one of those conformances, which is what this pitch does.

Doug

Spelling it == doesn’t avoid having to choose between func ==(lhs: A, rhs: B) and func ==(lhs: B, rhs: A). Swift still privileges the type of the left-hand side, so you have to write definitions of both functions. Some people consider this redundancy a damning indictment of languages without multimethods.

The argument that one “really” wants “multi-Self protocols” feels like the same argument. It’s also a straw man, because Rust doesn’t implement conversions using a trait with 2 generic parameters. It models conversions using two separate traits, From<T> and Into<T>. A single generic impl<T, U> Into<U> for T provides the Into that mirrors any user-defined impl From.

That, @Slava_Pestov, is the usefulness of generic traits—though perhaps you could call it generic impls, because the arity of the impl does not match that of the trait! But even though impl Into has two type parameters, it’s very clear which is Self, because the impl is for one of them.

Edit: Maybe conditional conformances provide the equivalent functionality to this use of generic impls?

1 Like

I don't want to waste time and energy by repeating arguments, but just to give a data point for those fighting for a cause that seems lost already: Nothing in the whole discussion has shifted my original evaluation.
The change may add some convenience for a small group, but the increase in complexity will unavoidably confuse and irritate novices ("should that associated type be primary?", "this looks like generics, why does it behave differently?"...).

The current iteration does more than allowing some minor syntactic sugar, but it still feels like a workaround for problems which have not been considered in the design of opaque result type or the "where"-clauses (if those are really such a burden, maybe they should be replaced completely?).

11 Likes

Let’s not discount the readability win here. Collection<Character> is easier to understand, especially if you don’t have an understanding of how associated types differ from generics, which is a fairly advanced topic that people commenting in this thread are more likely to have a pretty good handle on!

The concerns about taking away the most obvious syntax for generic protocols warrant more investigation into whether generic protocols are actually useful and whether their relationship to associated types makes spelling them similarly a source of potential confusion. From looking over the Rust examples I think the answer might in fact revolve around another feature we’ve punted on for a while: explicit specialization. I have to follow that mental trail later.

2 Likes

What happens when an associated type is inherited from another protocol?

The various collection protocols override their Element type, to support associated type inference (according to FIXME comments).

public protocol Sequence<Element> {
  associatedtype Element
}

public protocol Collection<Element>: Sequence {
  override associatedtype Element
}

public protocol BidirectionalCollection<Element>: Collection {
  override associatedtype Element
}

There's a similar example in the SIMD protocols, except there's no override redeclaration.

public protocol SIMDStorage<Scalar> {
  associatedtype Scalar: Codable, Hashable
}

public protocol SIMD<Scalar>: SIMDStorage {}

My suggestion is to keep the associatedtype declarations, and then reference them within the angle brackets (as shown above).


For source compatibility, could you also allow conditional compilation of the angle brackets?

public protocol Sequence
#if compiler(>=9999)
<Element>
#endif
{
  associatedtype Element
}

Re-stating an associated type with the same name is a no-op, except if the new associated type has additional requirements in its inheritance clause or where clause. So this is fine:

protocol Collection<Element> {}
protocol BidirectionalCollection<Element> : Collection {}

The way I'm imagining it is if the sub-protocol doesn't declare a primary associated type, then the primary associated type is not inherited; so if you instead write

protocol BidirectionalCollection : Collection {}

You wouldn't be able to say BidirectionalCollection<String>.

Hmm... I kind of like this idea; it feels more consistent in a way, and solves this problem where primary associated type declarations have a source range outside of the body of the protocol, which avoids a new special case for tooling to deal with.

My only concern is that then it's not clear if we want to allow writing an inheritance clause on the primary associated type name itself. Eg, is this valid?

protocol Foo<T : Equatable> {
  associatedtype T
}

Or should all requirements be stated on the associated type declaration itself then?

What do you think?

Unfortunately, this will itself be a new feature so it won't enable source compatibility with older compilers. It's really unfortunate that if is not allowed in more positions; in my opinion the C preprocessor model is too lax, but we could require that if can wrap any syntactically-valid element of the AST (so braces must match, etc).

3 Likes

I couldn’t help drafting up another example of how generic protocols could be useful. In this case, CollidesWith<T> is a generic protocol, and a generic extension implements weapon-collision logic for all entities that have hit points:

//MARK: Collision Detection

/// A generic protocol that describes how one kind of thing reacts to colliding with another kind of thing.
protocol CollidesWith<Other> {
    var position: Vec3 { get, private(set) }
    var velocity: Vec3 { get, private(set) }
    func handleCollision(with other: Other)
}

/// If T: CollidesWith<U>, then implicitly U: CollidesWith<T>.
extension<T, U> T: CollidesWith<U> where U: CollidesWith<T> {
    func handleCollision(with other: U) {
        // Nothing happens by default.
        // Can be refined by conformers.
    }
}

// Entry point for collision system.
extension CollidesWith<Other> {
    func collide(with other: Other) {
        handleCollision(with: other)
        Other.handleCollision(with: self)
    }
}

/// All the Entities that can collide with each other.
var objects: [any CollidesWith]

func physicsTick(deltaT: Int) {
    // Integrate object velocities and test for collisions.
    let collisions = gatherIntersections(among: objects, over: deltaT)
    
    for (object1, object2) in collisions {
        // This single call will handle both aspects of the collision, even if one object doesn’t react.
        object1.collide(with: object2)
    }
    
    // Apply (potentially modified) velocity.
    for object in objects {
        object.integratePosition(over: deltaT)
    }
}

//MARK: Game Specific Logic

// The world has a ground plane.
struct Ground {
    let elevation: Float
}

// Things can’t fall past the ground.
extension CollidesWith<Ground> {
    func handleCollision(with ground: Ground) {
        velocity.z = 0
    }
}

// A Weapon is anything that can cause damage.
struct Weapon {
    var damage: Int
}

// A protocol for anything that can be damaged or healed.
protocol HasHitPoints {
    var hp: Int { get, private(set) }
}

// When a weapon strikes, it causes damage.
// This is a generic extension, implemented on all types that conform to CollidesWith<Weapon>.
extension<T: HasHitPoints> T where T: CollidesWith<Weapon> {
    func handleCollision(with weapon: Weapon) {
        hp -= weapon.damage
    }
}

// A player loses when they run out of hit points.
// Notice how struct Player _only_ implements the game-over condition; weapon handling is handled by the generic extension above, but the hp setter is kept private.
struct Player: HasHitPoints, CollidesWith<Weapon>, CollidesWith<Ground> {
    private var _hp: Int
    var hp {
        get { _hp }
        private(set) {
            // No resurrecting dead players!
            if (_hp >= 0) {
                _hp = newValue
            }
        }
        didSet {
            if _hp <= 0 {
                gameOver()
            }
        }
    }
}

I agree in principle that Foo<Bar> etc. is probably the most readable sugar for what people want right now, but I have a few misgivings about the effects of how we get there. There's definitely a few things that rub me the wrong way of modifying the declaration site of a protocol to support a sugar.

What if the author of an API and its consumer differ in their imagination for the use cases of an opaque return type? (Or, more realistically, an author doesn't get around to it.) For example, if we decide not to pull Collection.Index into the type parameter list, what is a developer's recourse when they actually do want to use it that way? Needing to define an order of the PAT type parameters, and being limited with adding or removing ones, saddles PATs with the same things people dislike about the current syntax for generics.

I'm becoming less confident that we will be able to avoid one of:

  • being able to assign a name or sigil for the opaque return type to use with traditional where clauses
  • a variant of a where clause that binds to opaque types

If we come up with a satisfactory solution to that problem space, it's possible there may be no additional sugar needed, and the so-called lightweight syntaxes get to stay "lightweight". (There's also the idea that a solution to that problem space needs to happen, w.r.t. "sugar for something we don't have another syntax for" problem.)

3 Likes

I'm totally onboard with eventually introducing named opaque result types, eg

func foo() -> <T, U> (T, U) where T : Sequence, T : Sequence, T.Element == U.Element

However I think that should be a separate discussion. Even if named opaque result types are introduced I still believe that the syntax in this pitch will be the more common case by far.

1 Like

I think that should be allowed, if we want <T: Equatable> and <T> where T: Equatable to be interchangeable. The following syntax is already supported:

protocol Foo where T: Equatable {
  associatedtype T
}

The use of override for protocol requirements is a Swift-internal unofficial feature, which is not publicly documented and—having never even been pitched here—does not officially exist.

It was added in the run-up to ABI stability as the counterpart to @_nonoverride (double-check the spelling against the relevant compiler source), which allowed same-name requirements in more refined protocols to have their own ABI footprint.

If I recall, there is (or was) an internal compiler flag which can be used to require every redeclaration to be explicitly annotated either overriding or non-overriding, useful for ABI checking. Otherwise, outside this dialect of Swift for internal use only, redeclarations are implicitly one or the other (not confident as to which at this hour).

2 Likes

Thanks for pitching this, I'm so glad with all the improvements the type system is getting ^^

That said I'm a bit hesitant with this one, I'm in two minds.

  • I agree that we need a way to specify constraints on opaque return types. As others I would have loved to see that pitched independently of a "primary associated types" sugar.

  • As far as I understand this just works for associated types privileged by the author of the protocol. This feels quite awkward. Isn't a "primary type" something that the usage side should decide, depending on what algorithm or data structure you are writing you may need to give the privilege of a primary type to one type or another.

  • If the author didn't privilege the type you need you can't use this feature at all. That seems backwards too. I think we need an unsugared form of this.

  • I fully agree that this introduced syntax is very appealing, it looks a lot like something Swift users are used to and that's a win! But this also brings my main concern, how do we explain this now. For years every new Swift user has asked "how do I make a protocol generic" (even if that doesn't is not the correct name), and we had to reply with "that's not a thing in Swift, Swift has associated types in protocols that every conformance can specify...", to which you get weird looks and a eventually "ok whatever". Even last week I was in a podcast talking about Swift and its evolution and the host asked about generic protocols :laughing:. But now we have this syntax so ... how do we explain it? People will think this is what they were looking for al along, and maybe it is! But is it? I find it hard to evaluate something when there is no clear definitions or is not very clear what a newcomer to Swift will expect of this syntax, especially coming from other languages. What will a user coming from Java interpret from this? Kotlin? C++? Rust?... maybe we don't care about this but sill, how we explain this feature? Is it generic protocols? what is it if not? I've long thought that proposals should come with a section on teaching, it would also resolve most concerns people has every time something like this shows up.

Nothing is a deal breaker but I think there are some questions that are worth answering.

:hugs:

7 Likes

I want to point out again that there is another huge issue which is at least connected with the lack of generic protocols (at least I think so, and so far, nobody could prove the theory wrong):
You cannot nest a protocol declaration in another type (and as there are no namespaces either, there is no way to nest protocols at all).

class Delegator {
  protocol Delegate { // sorry, does not work
    func doSomething()
  }
}

Wait, there's not even a generic declaration — why should this be related??

Well, Delegator could be generic, and nested types inherit those parameters, so this construct would be a backdoor to generic protocols. Under the current limitations, there is a conflict, and probably the easiest solution is to disallow nesting.
There are other possibilities, but I don't think there is any option that does not introduce some other kind of inconsistency, so the most elegant choice would be to lift the restriction — and if we would allow declaration of protocols in other types, disallowing generic protocols would be a quite artificial restriction.

Maybe I'm living in a completely different reality, but many topics discussed in the forum have just theoretical benefit, whereas being forced to write SomeClassDelegate instead of SomeClass.Delegate bugs me quite a lot.

7 Likes

By contrast, I think this is the part of the pitched feature that is the most strong and convincing. The claim is that Collection has, semantically, a primary associated type (i.e., Element) by which it is most usefully parameterized, just like Array does.

Protocols aren't just bags of syntax but have semantic requirements, which exist to enable useful particular generic algorithms. As such, in the same way that functions and concrete types can be usefully parameterized, so too a protocol itself. And to your point, a conforming type can certainly be generic over another parameter separate from that the protocols to which it conforms.

The last time that the question about protocols being nested was brought up, as I recall, there was a tangled ball of gnarly issues that needed to be settled regarding capturing types from the outer scope even for the minimal viable product. The idea that, because it would be nice to write SomeClass.Delegate instead of SomeClassDelegate, therefore we should tackle nested protocols, and therefore we need generic protocols, and therefore we should not use angle brackets for another useful improvement in the language—I don't think this is reasonable to deem in scope for this discussion. Nor, mind you, would we likely want to contemplate a design for protocol namespacing which would cause users to stumble into the complexity of captured outer type aliases and generic nested protocols simply because they want to write SomeClass.Delegate.

1 Like

You make the thought about nested protocols sound like a complicated and stupid argument, but I don't think that is true:
You can nest classes in other types, you can nest structs in other types, you can nest enums in other types, and (I hope — never checked that ;-) you can do the same with actors. Protocols are the one big exception, and I fail to see why this ability is less useful or more complicated than in all the other cases.

So unless there is some fundamental problem (not just "it's some work to do it") with generic protocols, which really prevents us from removing the special case in the future, we should be very careful before we give a completely new meaning to a syntax which had a strong and direct connection to a core feature of the language for years.

6 Likes

Then you need to go and look at Rust, which has and does exactly this, and which is the "prior art" that a lot of the people in this thread (myself included) are looking at when they express distaste for the proposed syntax. I don't think you can have this discussion in good faith without understanding how Rust's generic protocols work.

Generics in Swift are something that beginners struggle a lot with, and I spend a lot of time explaining that types within angle brackets are "input types" under the control of the programmer, and associated types are "output types", under the control of the code. This syntax muddies these waters unnecessarily, conflating an associated type with a generic parameter in certain syntactic positions.

The obvious, Swift-y way to spell this seems to me to be some Collection where .Element == String, or similar. That gives lots of flexibility to write more complex where clauses, and doesn't unnecessarily constrain Swift from adopting generic protocols later. Indeed, it seems like this kind of syntax is a necessary prerequisite for the special case proposed in this pitch?

Why are we so keen on jumping two steps forward, to a place where we've irretrievably shut off a possibility for the language?

5 Likes