Allow Member Lookup to Find Generic Parameters

Evolution Pitches
1

Hello dear Swift community, I waited with this pitch until the forum finally went live. Last year there were some topic regarding protocol nesting where this idea was born from.

Mainly this idea is meant to disambigute similar named types, but there are also other areas where this aligned behavior would come to a benifit.

Here a few snippets where I see this behavior as useful.

// Future direction:
class Generic<Element> {
	protocol P {
		// `Element` is not captured by P, but could allow the
		// default to be generic type parameter, therefore the
		// default for `Generic<Int>.P.Element` would be `Int`.
		associatedtype Element = Generic.Element
	}

	// In case of value types the constraint should be 
	// `== Generic.Element`. This is a perfect case for 
	// disambiguation between associated type `Element` 
	// and the other generic paramter type `Element`.
	typealias CapturedP = P where Element == Self.Element

	var p: CapturedP
}

// Constraint that forces a nested generic type to have the same
// generic type parameter as the outer generic type, but still
// allowing both the be named the same.
struct FirstDimension<Element> {
	// In Swift 4 inner `Element` is distinct from the outer `Element`.
	struct SecondDimension<Element> where Element == FirstDimension.Element {
		...
	}
}

// Preventing generic type parameter shadowing like the following one:
struct Generic<Element> {
	typealias Element = Int
	var element: Element
}
// error: cannot convert value of type 'String'
// to expected argument type 'Generic.Element' (aka 'Int')
Generic<String>(element: "Swift 5")

// Allowing new generic algorithms similar to associated types 
// directly on generic types by allowing ignoring some generic
// paramters.
struct Generic<A, B> {
	var a: A
	var b: B
}

func test<T>(_ t: T) where T : Generic, T.A == Int {
	print(t.a * 10)
}

test(Generic<Int, Int>(a: 1, b: 2))
test(Generic<Int, String>(a: 3, b: "String"))

I'm sure there are other cases where this behavior can be beneficial. I fully understand that this alignment would probably be a breaking change and I can name a scenario that comes to my mind:

class SuperClass<Element> {}
class SubClass<Element> : SuperClass<Int> {}

In this case SubClass.Element == SuperClass.Element (or Self.Element) == Int, at least I would assume that behavior.

Anyways, I'd like to see your feedback on this idea.

2

This pitch was floating around for quite long time. Please apologize me for directly pinging Swift contributors in this reply. I would really appreciate if we can discuss this idea or at least debunk it if it's complete nonsense from a view of a compiler engineer. :slight_smile:

cc. @Slava_Pestov @Douglas_Gregor

1 Like
3

As I understand it, the basic premise of this pitch is that generic parameters should be visible to qualified name lookup, i.e., you can refer to them after the name of a generic type:

struct X<Element> {
  var e1: Element   // okay, and has always been okay. this is "unqualified" lookup
  var e2: X<Int>.Element  // start allowing this; it's a "qualified" lookup (X<Int> is the qualifier) and it'll resolve to Int
}

typealias XD = X<Double>
typealias MyDouble = XD.Element // start allowing this qualified lookup; it'll resolve to Double

I think this is a good improvement to Swift. @jrose brought this up recently and mentioned that we should start allowing it. The only reason we didn't allow it was pre-historical: we made the decision not to allow qualified name lookup to find generic parameters before we were certain that generic parameter names would be API. The fact that all extensions use the same names means that generic parameter names are certainly API.

This suggests a shadowing rule, which basically says you can't redefine Element in a way that makes the generic parameter Element impossible to name. This sounds great to me.

I'm not so keen on this one, partly because it's not about the central point of your proposal--which is a change to name lookup---but is instead using "T : Generic" as a shorthand for introducing generic parameters. I think we should continue to write such a signature as:

func test<B>(_ t: Generic<Int, B>) { }

Both because I find it clearer (at least in the cases I've thought of) and because it makes the overall proposal a smaller change.

One suggestion for messaging: "Align Generic Type Parameters to Associated Types", to me, sounds really big and disruptive. I think titling your proposal/pitch something like "Allow Member Lookup to Find Generic Parameters" would be clearer and less scary.

Doug

5 Likes
4

Thank you Douglas for sharing your thoughts.

Since I had no idea about the correct terminology of this pitch I choose the former title and hoped it will describe the aligned behaviour. I edited the title like you suggested, but I'm not 100% sure it still fits perfectly.

I still have a few more questions that I would like to ask you.

(Click the spoilers to unfold.)

More options how we can write qualified lookups

I wonder if the qualified lookup can have a few more forms. For instance in the future when SE-0068 is finally implemented it would be great if we can allow this:

class Y<Element> {
  var e1: Element // nothing new
  var e2: Y<Element>.Element // resolves to `Element`
  var e3: Self.Element // resolves to `Element`
}

The main difference here is that conforming a super class to a protocol with an associated type will inherit the associated type and glue it to the rhs like Super.P and Sub.P.

protocol Q {
  associatedtype P
}

class Super : Q {
  typealias P = Int
}

class Sub : Super {
  var q: P?
}

Is there a technical reason why we don't do the same with subclasses of generic classes? (Here Self.Element would start to make sense, or in a non-generic subclass you could use the unqualified lookup to access the generic type parameter from the super class.)

class Generic<X> {
  var x: X?
}

class NonGenericSub : Generic<Int> {
  var xx: X? // this is not possible today, nor `Self.X`
}

class GenericSub<X, Y> : Generic<Y> {
  var xx: X? // That is a comletely different `X`
}

The second version I have in mind is a shorthand form of qualified lookup that can be used to disambiguate certain scenarios like for instance this made up example:

struct FirstDimension<Element> {
  // `FirstDimension.Element` is the shorthand form.
  // In this case we cannot write `FirstDimension<Element>.Element` because we would
  // then refer to the `Element` of the `SecondDimension` generic struct.
  struct SecondDimension<Element> where Element == FirstDimension.Element {
    ...
  }
}
Ignoring some generic type parameters when needed

It may reduce the complexity of the proposal but why shouldn't we be able to ignore certain generic parameters when needed. The same functionality does already exist for protocols with multiple associated types:

protocol TestProtocol {
  associatedtype A
  associatedtype B
  var a: A { get }
  var b: B { get }
}

func test<T>(_ t: T) where T : TestProtocol, T.A == Int {
  print(t.a * 10)
}

Sure this is a small example that works better if you do it the other way around, but it was completely made up. There might be better examples that could fit in here if for instance you have a few more generic type parameters but for your algorithm you only care about a smaller set of them.

Could you also explain what you mean if generic parameters are API? (I'm clearly not familiar with this way of expression it.)

5

It's possible that I haven't captured your intent well with my suggestion.

Yes, I believe this follows from SE-0068 and your proposal.

All of the above fits well as a consequence of your proposal.

The second version I have in mind is a shorthand form of qualified lookup that can be used to disambiguate certain scenarios like for instance this made up example:

struct FirstDimension<Element> {
  // `FirstDimension.Element` is the shorthand form.
  // In this case we cannot write `FirstDimension<Element>.Element` because we would
  // then refer to the `Element` of the `SecondDimension` generic struct.
  struct SecondDimension<Element> where Element == FirstDimension.Element {
    ...
  }
}

The reference to FirstDimension.Element does work, because FirstDimension.Element is interpreted as FirstElement<Element>.Element (using the outer "Element" generic parameter). Personally, I regret the rule that makes FirstDimension shorthand for FirstDimension<Element>; it causes confusion with type inference, and SE-0068 is a better answer here. Regardless, the example above will work with your proposal.

On "ignoring some generic type parameters when needed":

That works well for protocols because "T: TestProtocol" is a conformance constraint. That's the main concern I have with your original example:

Here, T: Generic isn't a conformance constraint, or a superclass constraint: it's a same-type constraint where the actual type arguments to Generic imply (hidden) type parameters. Personally, I find that to be too magical to reason about, and in the examples I've seen so far don't feel like improvements.

By API, I mean that they are part of the contract between the designer of the generic type and any user of the generic type. You can't change the name of a generic parameter without breaking client code, because the type parameter names are used by extensions.

Doug

6

If I understand this correctly then I'm a little confused why the above example with subclasses will work without introducing a breaking change.

class Generic<X> {
  var x: X?
}

class NonGenericSub : Generic<Int> {
  var xx: X? // Would this be allowed and is this also a qualified lookup like `Self.X` (which resolves to `Int`)?
}

class GenericSub<X, Y> : Generic<Y> {
  var xx: X? // `Self.X` should be the same as `Y` but here it is not?
}

This is the only case that I'm worried about, because for me it seem like it will introduce a breaking change as soon as qualified lookup would be introduced. If you have a little time for explanation, I'd really appreciate it. I think I could start writing a formal proposal soon (without the part of ignoring generic type parameters).

7

In your GenericSub example, Self.X would refer to the type parameter X in GenericSub. There is a source-breaking change in your proposal, which looks (e.g.) like this:

class Super {
  typealias A = Int
}
class Sub<A> : Super {
}

var foo: Sub<Float>.A      // currently Int, but would become Float with your proposal

Doug

8

I really need to start writing a formal proposal for this one. I just had a use case where I needed qualified lookup from a subclass to it's superclass to write nice and clean code, but had to workaround.

extension NotificationPipe {
  ///
  class Flow<Strategy> where Strategy : FlowStrategy { ... }
}

extension NotificationPipe {
  final class DriverFlow : Flow<DriverSharingStrategy> {
    let shouldBeArchived: Bool

    init(
      // This should be `Strategy.NotificationRelay` because 
      // 'Strategy' would be bound to the subclass after this
      // proposal
      relay: DriverSharingStrategy.NotificationRelay, 
      shouldBeArchived: Bool,
      discardsDuplicates: Bool
    ) {
      self.shouldBeArchived = shouldBeArchived
      super.init(relay: relay, discardsDuplicates: discardsDuplicates)
    }
  }
}
9

It’s a historical quirk that member lookup doesn’t find generic parameters. I’d love to see a proposal to clean this up.

6 Likes
10

This would be an extremely welcome change from me. It's somewhat frustrating that defining a generic class limits non-generic subclasses completely, and leads to avoiding generics and moving towards the use type-erasure which is really not a great solution to these types of problems.

1 Like
11

@Douglas_Gregor there is one idea that just came to my mind and I'm curious if it's total nonsense or possibly correct. Would you mind clarifying it to me please?

Today Swift still lacks parameterized extensions which would enable new sorts of API's. Therefore an extension like the following isn't possible yet.

protocol P {}
struct View<Base> {}
struct Generic<T: P> {
  var view: View<Generic> { ... }
}

// Not possible
extension View where Base == Generic { ... }

With parameterized extensions we could solve that issue and write:

extension<T: P> View where Base == Generic<T> { ... }

However in reality for these type of extensions it's not 'parameterized extensions' that is missing right? Instead it's the ability to lookup generic parameters inside the extension.

If my hypothesis is correct then if we were to allow qualified lookup then the above extension would become valid. Is that correct?

extension View where Base == Generic { 
  func foo() {
    print(Base.T.self) // with qualified lookup `T` becomes visible and accessible here
  }
}

To strengthen my thought path we can already achieve similar solution today if we add a protocol to the game:

protocol GenericTLookup {
  associatedtype T: P
}

extension Generic: GenericTLookup {}

extension View where Base: GenericTLookup { 
  func foo() {
    print(Base.T.self) // okay
  }
}

I could imagine that the example from the manifesto could be rewritten as following:

// before
extension<T> Array where Element == T? {
  var someValues: [T] {
    var result = [T]()
    for opt in self {
      if let value = opt { result.append(value) }
    }
   return result
  }
}

// after
extension Array where Element == Optional {
  var someValues: [Element.Wrapped] {
    var result = [Element.Wrapped]()
    for opt in self {
      if let value = opt { result.append(value) }
    }
   return result
  }
}
2 Likes
12

This is still a parameterized extension, but you've hidden the parameterization from the syntax of the surface language. Where you write Base == Generic, the right-hand side is shorthand for Generic<U> where U has to be something. The only thing that makes sense here is for U to be a parameter to the extension itself, because there's no concrete type there that would be well-formed.

This could perhaps be considered a more concise way of expressing parameterized extensions in the surface language, it's the same underlying implementation module + inference of the generic parameters of the extension from the same-type constraint. I think I'd argue that, overall, it's less clear and less general than the explicit parameterized syntax, and it wouldn't save on (e.g.) implementation time or complexity to do this feature rather than full parameterized extensions.

Doug

3 Likes