Generic type as generic constraint

Okay, I know the title is a little confusing, bear with me here.

You can write a generic constraint with either an equality or a subtype relation:

extension Collection where Element == Int {}

extension Collection where Element: BinaryInteger {}

But you cannot, today, write a constraint that specifies a particular generic type, leaving its generic parameters unspecified.

For example, suppose you want to extend “collections of optionals”. You cannot write:

extension Collection where Element: Optional {} 
// error: reference to generic type 'Optional' requires arguments in <...>

And the same error occurs if we replace the “:” with “==”.

• • •

First of all, why would one even want to do this?

Here’s a simple example:

Dictionary has a method called compactMapValues, which makes a new dictionary by transforming each value, and dropping the entries that got mapped to nil. That works for all dictionaries, and it requires passing in a transformation closure.

But suppose we have a dictionary of optionals (perhaps loaded from disk or fetched from a server) and we find ourselves calling dict.compactMapValues{ $0 } to remove the nils and unwrap the values.

We don’t actually transform the values, so we might decide to make a little convenience method that lets us simply write dict.compacted() instead:

func compacted() -> [Key: Value.Wrapped] {
  return self.compactMapValues{ $0 }
}

This is nice and clean, and puts things at the proper level of abstraction for us. The only problem is, where does that function go?

We want to put it in a constrained extension of Dictionary like so:

extension Dictionary where Value: Optional {
  func compacted() -> [Key: Value.Wrapped] {
    return self.compactMapValues{ $0 }
  }
}

But as mentioned above, the constraint Value: Optional results in a compile-time error.

• • •

There is a workaround, but it’s pure boilerplate which involves a new protocol that only one type conforms to, and pollutes the namespace of Optional with a superfluous property that just returns self:

protocol OptionalProtocol {
  associatedtype Wrapped
  var optionalValue: Wrapped? { get }
}

extension Optional: OptionalProtocol {
  var optionalValue: Self { self }
}

With that in place, we can write:

extension Dictionary where Value: OptionalProtocol {
  func compacted() -> [Key: Value.Wrapped] {
    return self.compactMapValues{ $0.optionalValue }
  }
}

And finally our method is available for use as dict.compacted().

• • •

If we wanted to do something similar for another generic type—say an extension on “collections of dictionaries”—then we’d need another boilerplate protocol and another self-returning property.

The workaround works, but it doesn’t scale well, and it isn’t necessarily obvious. So I propose that we allow generic types to be used as constraints directly, without any ceremony or boilerplate. This should just work:

extension Dictionary where Value: Optional {}
8 Likes

I’m not sure where things currently stand with this feature, but I think parameterized extensions should address this use case.

10 Likes

By itself, this:

doesn't give you any way to access the underlying wrapped type, because this:

isn't valid; Wrapped is a generic type parameter but not a member of Optional. So this change is more involved than it suggests, since it would require making all type arguments into members.

I think this would be better expressed as a parameterized extension, as already described in the generics manifesto, since those cover a superset of the functionality described here.

extension <T> Dictionary where Value: Optional<T>
16 Likes

Notably featured in @jrose’s Swift Regrets series. :slightly_smiling_face:

3 Likes

I don’t claim to know the best way to implement this, but the feature I am proposing does indeed allow accessing Value.Wrapped in that extension.

I strongly support parameterized extensions.

And even when we have them, I still feel strongly that extension Dictionary where Value: Optional is a much more natural spelling.

The version with angle brackets is fine for the full-fledged feature that allows multiple parameters with constraints between them, but in the simple case the simple syntax should work as well.

Also, I don’t think this simple version needs to wait for the full-fledged feature. It is useful on its own, the syntax is clear, and perhaps the work to implement it might help lay the groundwork for more complex parameterized extensions.

3 Likes

I'm certainly conflicted. On the one hand, I like that if you want them to be part of the API, you have to explicitly do so. On the other hand, a lot of folks would have motivation to use something other than T as a type parameter so that they don't have a bunch of .Ts stuck on their types...

Thanks for clarifying; since there would be broader implications to introducing that syntax, we should be more explicit about it instead of leaving it as something implied by the surrounding idea. Is your suggestion that you'd only be able to access the generic type using the Value.Wrapped syntax in an extension of this kind? What about contexts where someone would try to reference a generic type argument outside of one of these proposed extensions? For example,

// If this is allowed...
extension Array where Element: Optional {
  func foo() -> Element.Wrapped {}
}

// ...would this also be?
func foo() -> Optional<Int>.Wrapped {}

If the second one is allowed, then we have consistency, but then this is now a much larger change than making a certain kind of extension possible; it adds new members to all generic types.

If the second one isn't allowed, then we have an inconsistency where someone can access a member using a syntax and name inside an extension that they can't access using the same syntax and name outside of the extension, even when the actual types on the left-hand side are equivalent. That seems likely to cause confusion.

1 Like

It’s mildly surprising that this doesn’t already work today.

Within extension Optional you can write Self.Wrapped, so the inconsistency you describe already exists today. There’s no fundamental conceptual difference between that and Value.Wrapped as proposed: one is an extension where Self is optional, and the other is an extension where Self.Value is optional.

I don’t think that concern about such an inconsistency is any reason to wait on what I’m proposing, but rather it is simply another reason to make Wrapped available on Optional in every context, as a separate and orthogonal feature.

2 Likes

I think the discussion about making generic parameters into members is leading the discussion farther from the point that with the possible introduction of parametric extensions, the following extension context would not need to access Wrapped because it already has T:

2 Likes

Do you have an example where that works? Because this is what I see:

$ swiftc -version
swift-driver version: 1.26.9 Apple Swift version 5.5.1 (swiftlang-1300.0.31.4 clang-1300.0.29.6)
Target: arm64-apple-macosx12.0

$ swiftc -c - <<'EOF'
extension Optional { func wrappedValue() -> Self.Wrapped { fatalError() } }
EOF

<stdin>:1:50: error: 'Wrapped' is not a member type of generic enum 'Swift.Optional<Wrapped>'
extension Optional { func wrappedValue() -> Self.Wrapped { fatalError() } }
                                            ~~~~ ^
Swift.Optional:1:21: note: 'Optional' declared here
@frozen public enum Optional<Wrapped> : ExpressibleByNilLiteral {
                    ^

If you're trying it in a context where you've already applied the OptionalProtocol conformance from your original post, that would explain it since the associatedtype would automatically get a typealias from the matching generic argument.

1 Like

My mistake, you’re right. You can write Wrapped but not Self.Wrapped.

…which is itself surprising. Those seem like they ought to be synonyms there.

1 Like

It's a pretty fundamental difference between a generic parameter and a typealias member:

struct S<T> {
    typealias U = T
}
print(S<Int>.U.self) // Int
print(S<Int>.T.self) // Type 'S<Int>' has no member 'T'

As the author of the type, you could in fact design in such a way as to expose the generic parameter as a member:

struct Optional<Wrapped> {
    typealias Wrapped = Wrapped
}
print(Optional<Int>.Wrapped.self) // Int

But anyway, I think there's widespread consensus that parameterized extensions are a good thing, and that it would enable a spelling for your use case. If a "more natural" spelling would either be inconsistent with other parts of the language or require a separate proposal to fundamentally revamp the language, then it's an open question whether in fact it's more natural and to whom and in what circumstances.

It seems to be leading us further away from the point that (a) yes, it'd be nice to have support for the use case you describe; and (b) it'd be supported by parameterized extensions which we all seem to very much support (and, for which, incidentally, an implementation likely doesn't need further groundwork-laying as there's been an open PR since 2019).

8 Likes

I trust you can see how absurd that line looks.

An alias for a type, which has the exact same name as the type it is aliasing, and that somehow behaves differently than not having the alias?

Plus it doesn’t even have to be in the original type declaration. Anyone can add that alias in an extension:

extension Optional {
  typealias Wrapped = Wrapped
}

So really, all the discussion about Value.Wrapped needing extra complexity, is rather superficial.

1 Like

:man_shrugging: As you say, it's an orthogonal topic, not necessary to resolve for your use case here to be expressible.

2 Likes

Another workaround for your specific example is:

extension Dictionary where Value: ExpressibleByNilLiteral {
  func compacted<NewValue>() -> [Key: NewValue] where Value == NewValue? {
    compactMapValues { $0 }
  }
}
1 Like

Is that even needed? Can't you just do this:

extension Dictionary {
    func compacted<Wrapped>() -> [Key: Wrapped] where Value == Wrapped? {
        self.compactMapValues { $0 }
    }
}

?

For me that works as expected:

let dict: [Int: String?] = [
    0: "Hello",
    1: nil,
    2: "Foo"
]

print(dict.compacted()) // prints '[2: "Foo", 0: "Hello"]'

let dict2: [Int: String] = [:]

print(dict2.compacted()) // Error: Instance method 'compacted()' requires the types 'String' and 'Wrapped?' be equivalent
2 Likes

The outer where clause will filter the method in code completion suggestions.

The inner where clauses are equivalent. NewValue and Wrapped are both generic parameters of the method.

1 Like

Right, the fact that there is already a protocol to which only Optional conforms, means the boilerplate already exists for that particular type.

But there’s no existing analog to the nil-literal protocol for other types. Even for Dictionary, there are multiple types expressible by dictionary literals. Heck, technically there’s even another type that conforms to ExpressibleByNilLiteral: our good friend _OptionalNilComparisonType.

And it’s true that making the method generic avoids needing the associated type in the protocol. But the method is not really generic. It’s fully constrained. Contorting the generic constraint system to make a fully constrained generic method is not ideal.

1 Like

Okay, I’ve gone back through the Parameterized Extensions proposal, and based on the examples shown there I am led to the conclusion that it amounts to, essentially, syntactic salt for what I propose here.

The motivation section of that proposal contains exactly 2 examples, one of which is essentially identical to my motivation here, and the other is pure sugar independent of both proposals and could be implemented as such today.

Going through that proposal section by section:

Motivation

This is the same motivation as for my thread here. The syntax they use to convey the idea is very similar to what I propose, except they have “is” in place of “:”.

This is pure sugar, not a parameterized extension. We could make this valid today by simply teaching the compiler to interpret it as extension Pair where Element == Int.

• • •

Proposed solution

This is the same functionality as I propose, but with a more awkward spelling. It is equivalent to my “extension Array where Element: Optional”.

As before, this is isomorphic to what I propose. There is no functional difference between the above and its equivalent in non-parameterized syntax:

extension Array where
  Element: Optional,
  Element.Wrapped: FixedWidthInteger
{

It is true that the <T> form is shorter, but at the same time the second version reads more naturally and better matches what we already write for protocol extensions with multiple constraints.

This is just sugar for the first example. Again it is shorter than “extension Array where Element: Optional”, but not necessarily as intuitive.

This is the pure-sugar example again. It is independent of parameterized extensions.

This is just a sugared form of the ongoing example, which is still equivalent to my proposed “extension Array where Element: Optional”.

• • •

Detailed design

The parameterized extensions proposal needs a special-case to prohibit this, whereas with my proposal it simply follows from the existing restriction on extending Any.

This is isomorphic to what is already done today: extension Array: Equatable where Element: Equatable. I do not see any benefit to introducing a new, punctuation-dense syntax for what can already be written clearly with words.

• • •

ABI stability

…and here we see an actual drawback to introducing the second parallel syntax. No such problem exists with my proposal, because there would remain only one way of writing this.

• • •

Future Directions

If we were to decide to allow extensions of all types, the natural spelling would be extension Any. Once again, parameterized extensions would create a parallel syntax with no benefit.

This is a new and powerful feature. I think it is worth considering this functionality on its own merits. Notably, this is not actually part of the parameterized extensions proposal: it is a potential future direction.

If we decide we want this, then we can come up with a good spelling for it at that time. I do not think we should salt the syntax for using generic types as generic constraints, based on the mere possibility of a future feature.

This is covered by the separate Variadic Generics proposal.

• • •

My ultimate takeaway is that the actual motivating problem to be solved, is that generic types cannot currently be used as generic constraints.

If we could write

extension Dictionary where Value: Optional

and have access to Value.Wrapped within that extension, there would be no need for the parameterized extensions proposal. Its motivating use-case would be solved.

5 Likes

By the same token, then, parameterized extensions would solve your motivating use case, and then there would be no need to make a fundamental change in the language that makes generic parameters available as typealiases.

Regarding the future directions:

One sign of a good design is that it makes future directions natural rather than awkward warts.

In the ideal case, parameterized extensions would in even their first iteration cover the use case “extend every type where …” and not just “extend every X where …”. However, that they are divisible features with a natural resting place in the language that supports the latter and not the former does not mean that an ideal design has two different spellings that treats these use cases as unrelated. Indeed, as you point out, the syntax of parameterized extensions makes the former feature expressible without anything else added to the language—it’s just that the proposal (as it was then pitched) doesn’t encompass the implementation work to make it actually compile, so has to ban it.

It’s a powerful argument—certainly at least persuasive for me—in favor of tackling your use case as a specific use of parameterized extensions rather than as its own topic.

5 Likes

Conversely, we should make generic parameters available anyway, for the reasons Jordan Rose outlined, and we should make the syntax I propose work for consistency and simplicity.

Once we have those two things, then parameterized extensions as proposed only serve to introduce a second, redundant. punctuation-heavy spelling for the same thing, and that redundant syntax creates an attractive-nuisance for unintentionally breaking ABI.