Pitch: Allow Protocols to be Nested in Non-Generic Contexts

taylorswift · 2023-06-01T00:55:09.274Z

bbrk24:

I would hate to reserve the keyword 'namespace' for a special kind of extension without also introducing proper namespacing. I feel like every Swift project I work on has at least one empty enum full of static helper functions.

let us be true swifties and spell it:

@namespace extension P
{
}

xwu · 2023-06-01T14:51:09.855Z

Joe_Groff:

It seems like there are a lot of circumstances where API designers would like to use a generic type or protocol as a namespace, for declarations such as types, static methods or static properties that are effectively global, and don't depend on the generic context or dynamic class type of the enclosing type.

Perhaps; but I think it'd be a separable proposal from this one, and it'd be nice to get some signal as to where that crops up versus scenarios where the generic context of the enclosing type is depended upon.

Which is to say, I guess, that I think @Karl's pitch as-is is appropriately scoped and a reasonable 'resting place.'

YoshihiroUnno · 2023-06-01T15:29:55.552Z

Excuse me for suddenly interrupting the conversation.
The main subject is that protocols should be defined inside some data type, right?
I think so too.
Furthermore, I think data types and protocols should be defined inside protocols.
Why is this currently not possible?
My guess is that protocols are treated quite differently than data types inside the compiler.
This is really unsubstantiated speculation as I don't know what's going on inside the compiler.
If protocols were treated as a kind of data type, like abstract classes in C++, this theme should be achieved from the beginning.
For all of you, I may have just said something that was obvious to you.

xwu · 2023-06-01T15:49:28.932Z

Hi @YoshihiroUnno, please see @Slava_Pestov’s reply above for an explanation to your question.

Slava_Pestov · 2023-06-01T15:59:05.859Z

ensan-hcl:

Is satisfying associatedtype requirement with nested protocol allowed?

This is an interesting point. In fact you can already write this today with a typealias:

protocol P {
  associatedtype A
}

struct S: P {
  typealias A = P
}

Notice the absence of any...

@hborla Should we disallow "constraint typealiases" from serving as type witnesses?

EDIT: Holly pointed out that this is actually in the proposal: " Once the any spelling is required under the Swift 6 language mode, a type alias to a plain protocol name is not a valid type witness for an associated type requirement; existential type witnesses must be explicit in the typealias with any :" So the nested protocol case should also be disallowed then.

YoshihiroUnno · 2023-06-01T16:23:34.997Z

thank you.
I caused you extra trouble by speaking without fully understanding the flow of the discussion.
I will carefully read the relevant part of the literature that you have taught me, and once I have gathered my opinion, I will make a statement.

ensan-hcl · 2023-06-02T02:25:49.005Z

Understood. But I also think we can disable satisfying associatedtype requirement with nested protocol in Swift 5.9 or later, since this is a new feature which will be introduced after SE-0335 is introduced.
I believe source break creates a bit difficult situation, consider the following case.

// Swift 5
public protocol P {
     associatedtype Q
}
public struct A: P {
    // this is now satisfying associatedtype requirement
    protocol Q {}
}

// usage as constraint
extension B: A.Q { /* ... */ }
// usage as type
let value: A.Q = /* ... */

In Swift 6, as Q is no longer usable as a type, you have to change the type into like this.

public struct A: P {
    public typealias Q = any _Q
    public protocol _Q {}
}

However, making such a change is API-breaking, because the usage of A.Q as constraint is no longer possible. While --enable-upcoming-feature ExistentialAny will expose such risks, I also think avoiding such scenario from the beginning of nested protocol introduction is worth considering.

JuneBash · 2023-06-02T03:10:11.215Z

Very excited to see this being worked on! I was planning to learn a lot more about the compiler and take it on last year, but alas, life got in the way, so I'm happy to see it be picked up!

anon9791410 · 2023-06-08T17:37:59.891Z

Karl:

func doSomething() {  
   func impl<T: Abstraction>(_ input: T) -> T.ResultType {
     // ...
   }
}

This pitch is good, but should at least come with a caveat about this spelling. Outside of a function, it would be a protocol extension.

fileprivate extension Abstraction {
  var impl: ResultType { // …
}

Without nested protocol extensions, switching to the function form will be necessary, trading good spelling for good scoping. So I don't think this pitch should be implemented without being paired with nested extensions.

[Pre-pitch] Allow extension of nested members within extension of parent Pitches

This is a tiny syntactic sugar pitch. Swift's ability to group code by use of extension is probably one of the best features of Swift and something Apple themselves use a lot, combined with Swift's ability to nest types these two language features make up the backbone of what we love about Swift. Today extensions are limited to global scope, I cannot extend a nested type within an extension of the enclosing parent type. This makes code hard to read, I would argue. Simple case study (sorry for…

Karl · 2023-06-09T09:29:03.245Z

I think it would be nice to allow nesting extensions within functions - it's one of the missing pieces before we're able to nest entire Swift programs within functions, which would be extremely cool (there are some things you can only do with an extension -- like adding a protocol conformance to a stdlib type).

anon9791410:

So I don't think this pitch should be implemented without being paired with nested extensions.

I think there is enough here even without support for nesting extensions. There's nothing here which prevents that being added to the language at a later date.

bbrk24 · 2023-06-09T11:31:06.154Z

Karl:

like adding a protocol conformance to a stdlib type

This should definitely not be possible at the function-level. That would mean that the vtable grabbed by as? SomeProtocol would differ depending on the current call stack, and it wouldn’t surprise me if that can’t be done.

Karl · 2023-06-09T11:39:57.855Z

Yeah I suppose - I was thinking more of conformances to function-local protocols:

func foo() {

  protocol Nested { ... }

  // How do I make 'Int' conform to 'Nested'?
  // I can't write an extension, so I need to wrap it in a local type,
  // which is not nice.
}

But yeah, conformances to other protocols could be problematic.

Karl · 2023-07-23T17:19:09.202Z

Apologies for not responding to this before. I agree that a nested protocol should not satisfy an associated type requirement.

Especially with the loss of bare protocol existential syntax, the language is moving towards putting a clearer distinction between protocols (i.e. constraints) and concrete types. associatedtype is currently used to refer to concrete types, and it would be problematic in a lot of senses for it to bind to a nested protocol.

There have been requests in the past for "associated protocols" or something broadly to that effect:

Protocols with associated protocols?
Support for protocols as parameters for constrained generics
Generalized supertype constraints

If we ever did add some kind of associated protocol feature, it might be reasonable to use nested protocols to define them, as we currently can with associated types and nested concrete types.

xwu · 2023-07-24T04:47:14.331Z

Karl:

Apologies for not responding to this before. I agree that a nested protocol should not satisfy an associated type requirement.

Especially with the loss of bare protocol existential syntax, the language is moving towards putting a clearer distinction between protocols (i.e. constraints) and concrete types. associatedtype is currently used to refer to concrete types, and it would be problematic in a lot of senses for it to bind to a nested protocol.

There have been requests in the past for "associated protocols" or something broadly to that effect:

Protocols with associated protocols?

Support for protocols as parameters for constrained generics

Generalized supertype constraints

If we ever did add some kind of associated protocol feature, it might be reasonable to use nested protocols to define them, as we currently can with associated types and nested concrete types.

I agree with this reasoning. However, it occurs to me that this could lead to a wee bit of a problem—well, let's not call it a problem but maybe a complexity or nuance for which we need to specify the behavior in this proposal.

Suppose we have the following type, which would be allowed under your proposal:

struct S {
  protocol P { }
}

I retroactively adopt conformance to the following protocol like so:

protocol Q {
  associatedtype P = Void
}
extension S: Q { }

If the protocol S.P does not satisfy the requirement Q.P (as you and I agree), then what is the behavior here?

[Dramatic pause to allow readers to consider their answer before continuing.]

As we do not have a partitioned namespace for protocols versus types, I can see two possibilities, one not exactly satisfying for its complexity and one seemingly straightforward but actually unacceptable:

Option A:
By analogy with the case where a protocol requires a property with a different type but same name as a property already on the concrete type, S.P shadows but not override the required associated type, which defaults to Void. Users really don't like this sort of behavior, but it's quite precedented when it comes to protocol adoption.

In the absence of a default specified by `Q`...

users would have to rely on associated type inference or a future formalization of the `@_implements` feature in order to fulfill the requirement. Otherwise, again by analogy with the case where a protocol requires a property with a different type but same name as a property already on the concrete type, users just wouldn't be able to conform.

Option B:
The compiler can just straight-up refuse to compile the retroactive conformance extension. Super straightforward.

Why even contemplate the complexities of Option A? Unless I'm mistaken, Option B breaks the following promise about what's supported under library evolution:

A new associatedtype requirement may be added (with the appropriate availability), as long as it has a default implementation. If the protocol did not have one or more associatedtype requirements before the change, then this is a binary-compatible source-breaking change.

By implication (and afaik it's in fact currently true), where a protocol already has associatedtype requirements, adding another one is a binary-compatible and source-compatible change. With lifting of the "Self or associated type" restriction on the use of existentials, I believe we're heading towards a state where the later addition of a first associatedtype requirement is source-compatible as well.

However, if the compiler would refuse to compile if a nested protocol takes the same name as an associated type requirement, then no protocol could adopt any new associatedtype requirements in a source-compatible way even if it supplies a default.

ksluder · 2023-07-24T05:50:17.011Z

xwu:

By analogy with the case where a protocol requires a property with a different type but same name as a property already on the concrete type, S.P shadows but not override the required associated type, which defaults to Void. Users really don't like this sort of behavior, but it's quite precedented when it comes to protocol adoption.

And the user would be able to work around this by adding typealias P = Self.P to their extension?

xwu · 2023-07-24T06:28:49.365Z

ksluder:

And the user would be able to work around this by adding typealias P = Self.P to their extension?

No: I wouldn't think we'd do anything to change the fact that typealias P and protocol P in the same lexical scope would be (as it currently is) a redeclaration error.

But that is actually immaterial—even if it were not a redeclaration error, the typealias declaration would have no effect under Swift 6 rules. The "this" you're asking about a "work around" for is SE-0355: i.e., you're asking if there's some incantation to opt into P (the protocol) and any P (the existential type) not being distinguished from each other.

I think if we wanted to do that we'd just revoke SE-0355 and allow nested protocols to satisfy associated type requirements as existentials—we wouldn't adopt the diametrically opposite rule into the language and then add additional special-case features to "work around" it on a case-by-case basis.

Karl · 2023-07-24T08:16:06.167Z

xwu:

By analogy with the case where a protocol requires a property with a different type but same name as a property already on the concrete type, S.P shadows but not override the required associated type, which defaults to Void. Users really don't like this sort of behavior, but it's quite precedented when it comes to protocol adoption.

It is interesting that we do this for properties but not associated types AFAICT:

Property:

struct S {
    var foo: String { "" }
}

protocol Q {
    var foo: Int { get }
}
extension Q {
    var foo: Int { 42 }
}

// OK
extension S: Q {}

Associated Type (with a constraint that means the existing P cannot be a witness):

struct S {
    typealias P = String
}

protocol Q {
    associatedtype P: Numeric = Int
}

// ❌ error: type 'S' does not conform to protocol 'Q'
extension S: Q {}

I think it should act like the latter. A nested protocol is not a valid witness to an associated type because it is not a concrete type, just as String cannot witness the Q.P associated type because it does not meet the Numeric constraint.

In fact, despite Q.P having a default, I couldn't find any way to implement this conformance. Neither by using inference nor by using @_implements. Granted, I was testing within a single file, but I would be surprised if this behaved differently were I to split the types across different modules.

protocol Q {
    associatedtype P: Numeric = Int
    func foo(_: P)
}

// ❌ error: type 'S' does not conform to protocol 'Q'
extension S: Q {

  @_implements(Q, P)
  typealias Foo = Int

  // None of these make a difference, either.
  // @_implements(Q, foo)
  // @_implements(Q, foo(_:))
  func foo(_: Int) {}
}

Godbolt if you'd like to try for yourself.

So it seems to me that we are already not meeting the goal that adding an associated type with default is a source-compatible change. It's definitely an issue worth solving, but if it's not an issue introduced by this feature I would say we can consider its solution to be out of scope for this proposal.

The solution would probably be to productise @_implements. It doesn't surprise me that it is unable to disambiguate associated types as currently implemented; it is an unofficial feature and appears to be limited in its capabilities.

taylorswift · 2023-07-24T15:40:08.565Z

xwu:

no protocol could adopt any new associatedtype requirements in a source-compatible way even if it supplies a default.

swift's extension-based API model and total lack of namespacing facilities means there are actually very few kinds of additions that can be made to a type's API in a way that is strictly "source-compatible".

even seemingly innocuous changes, like adding an initializer to a protocol as an extension member can cause client code to fail to typecheck or dramatically increase compilation times due to name collisions.

in my view, the fact that adding an associated type requirement risks a name collision with another protocol nested in a retroactively-conformed type is not particularly novel or even remarkable. we have been dealing with variations this problem for a very long time.

hborla · 2023-07-24T16:06:01.418Z

Hey folks, I've kicked this off for review here starting now through August 7th. Please move any further discussion about the witness matching behavior over to the review thread.

Thank you!

Holly Borla
Review Manager

John_McCall · 2023-07-24T17:00:17.258Z