Placeholder types

Evolution Pitches
1

I've fleshed out a few more of the details for placeholder types (formerly "partial type annotations") and have a more complete pitch that I would love the community's feedback on!

Introduction

When Swift's type inference is unable to work out the type of a particular expression, it requires the programmer to provide the necessary type context explicitly. However, all mechanisms for doing this require the user to write out the entire type signature, even if only one portion of that type is actually needed by the compiler. E.g.,

func foo(_ x: String) -> Double { return Double(x.count) / 2.0 }
func foo(_ x: Int) -> Double { return Double(x) }

let stringTransform = foo as (String) -> Int

In the above example, we only really need to clarify the argument type—once that's determined, the return type is a given. This proposal allows the user to provide type hints which use placeholder types in such circumstances, so that the initialization of stringTransform could be written as:

let stringTransform = foo as (String) -> _

Swift-evolution thread: Partial type annotations

Motivation

Swift's type inference system is quite powerful, but there are many situations where it is impossible (or simply infeasible) for the compiler to work out the type of an expression, or where the user needs to override the default types worked out by the compiler. The example above of an overloaded function foo is one such example.

Fortunately, Swift provides several ways for the user to provide type information explicitly. Common forms are:

  • Variable type annotations:
let stringTransform: (String) -> Double = foo
  • Type coercion via as (seen above):
let stringTransform = foo as (String) -> Double
  • Passing type parameters explicitly (e.g., JSONDecoder):
let dict = JSONDecoder().decode([String: Int].self, from: data)

The downside of all of these approaches is that they require the user to write out the entire type, even when the compiler only needs guidance on some sub-component of that type. This can become particularly problematic in cases where a complex type that would normally be inferred has to be written out explicitly because some unrelated portion of the type signature is required. E.g.,

enum Either<Left, Right> {
  case left(Left)
  case right(Right)

  init(left: Left) { self = .left(left) }
  init(right: Right) { self = .right(right) }
}

func makePublisher() -> Some<Complex<Nested<Publisher<Chain<Int>>>>> { ... }

Attempting to initialize an Either from makePublisher isn't as easy as one might like:

let publisherOrValue = Either(left: makePublisher()) // Error: generic parameter 'Right' could not be inferred

Instead, we have to write out the full generic type:

let publisherOrValue = Either<Some<Complex<Nested<Publisher<Chain<Int>>>>>, Int>(left: makePublisher())

The resulting expression is more difficult to write and read. If Left were the result of a long chain of Combine operators, the author may not even know the correct type to write and would have to glean it from several pages of documentation or compiler error messages.

Proposed solution

Allow users to write types with designated placeholder types (spelled "_") which indicate that the corresponding type should be filled in during type checking. For the above publisherOrValue example, this would look like:

let publisherOrValue = Either<_, Int>(left: makePublisher())

Because the generic argument to the Left parameter can be inferred from the return type of makePublisher, we do not need to write it out. Instead, during type checking, the compiler will see that the first generic argument to Either is a placeholder and leave it unresolved until other type information can be used to fill it in.

Detailed design

Grammar

This proposal introduces the concept of a user-specified "placeholder type," which, in terms of the grammar, can be written anywhere a type can be written. In particular, the following productions will be introduced:

type → placeholder-type
placeholder-type → '_'

Examples of types containing placeholders are:

Array<_> // array with placeholder element type
[Int: _] // dictionary with placeholder value type
(_) -> Int // function type accepting a single placeholder type argument and returning 'Int'
@escaping _ // attributed placeholder type
(_, Double) // tuple type of placeholder and 'Double'
_? // optional wrapping a placeholder type
_ // a bare placeholder type is also ok!

Type inference

When the type checker encounters a type containing a placeholder type, it will fill in all of the non-placeholder context exactly as before. Placeholder types will be treated as providing no context for that portion of the type, requiring the rest of the expression to be solvable given the partial context. Effectively, placeholder types act as user-specified anonymous type variables that the type checker will attempt to solve using other contextual information.

Let's examine a concrete example:

import Combine

func makeValue() -> String { "" }
func makeValue() -> Int { 0 }

let publisher = Just(makeValue()).setFailureType(to: Error.self).eraseToAnyPublisher()

As written, this code is invalid. The compiler complains about the "ambiguous use of makeValue()" because it is unable to determine which makeValue overload should be called. We could solve this by providing a full type annotation:

let publisher: AnyPublisher<Int, Error> = Just(makeValue()).setFailureType(to: Error.self).eraseToAnyPublisher()

Really, though, this is overkill. The generic argument to AnyPublisher's Failure parameter is clearly Error, since the result of setFailureType(to:) has no ambiguity. Thus, we can substitute in a placeholder type for the Failure parameter, and still successfully typecheck this expression:

let publisher: AnyPublisher<Int, _> = Just(makeValue()).setFailureType(to: Error.self).eraseToAnyPublisher()

Now, the type checker has all the information it needs to resolve the reference to makeValue: the ultimately resulting AnyPublisher must have Output == Int, so the result of setFailureType(to:) must have Output == Int, so the instance of Just must have Output == Int, so the argument to Just.init must have type Int, so makeValue must refer to the Int-returning overload!

Note: it's technically legal to specify a bare placeholder type, which may act as a more explicit way to call out "this type is inferred":

let percent: _ = 100.0

Generic constraints

In some cases, placeholders may be expected to conform to certain protocols. E.g., it is perfectly legal to write:

let dict: [_: String] = [0: "zero", 1: "one", 2: "two"]

When examining the storage type for dict, the compiler will expect the placeholder key type to conform to Hashable. Conservatively, placeholder types are assumed to satisfy all necessary constraints, deferring the verification of these constraints until the checking of the intialization expression.

Generic parameter inference

A limited version of this feature is already present in the language via generic parameter inference. When the generic arguments to a generic type can be inferred from context, you are permitted to omit them, like so:

import Combine

let publisher = Just(0) // Just<Int> is inferred!

With placeholder types, writing the bare name of a generic type (in most cases, see note below) becomes equivalent to writing the generic signature with placeholder types for the generic arguments. E.g., the initialization of publisher above is the same as:

let publisher = Just<_>(0)

Note: there is an existing rule that inside the body of a generic type S<T1, ..., Tn>, the bare name S is equivalent to S<T1, ..., Tn>. This proposal does not augment this rule nor attempt to express this rule in terms of placeholder types.

Function signatures

As is the case today, function signatures under this proposal are required to have their argument and return types fully specified. Generic parameters cannot be inferred and placeholder types are not permitted to appear within the signature, even if the type could ostensibly be inferred from e.g., a protocol requirement or default argument expression.

Thus, it is an error under this proposal to write something like:

func doSomething(_ count: _? = 0) { ... }

just as it would be an error to write:

func doSomething(_ count: Optional = 0) { ... }

even though the type checker could infer the Wrapped type in an expression like:

let count: _? = 0

Source compatibility

This is an additive change with no effect on source compatibility.

Effect on ABI stability

This feature does not have any effect on the ABI.

Effect on API resilience

Placeholder types are not exposed as API. In a compiled interface, they are replaced by whatever type the type checker fills in for the placeholder. While the introduction or removal of a placeholder on its own is not necessarily an API or ABI break, authors should be careful that the introduction/removal of the additional type context does not ultimately change the inferred type of the variable.

Alternatives considered

Alternative spellings

Several potential spellings of the placeholder type were suggested, with most users preferring either "_" or "?". The question mark version was rejected primarily for the fact that the existing usage of ? in the type grammar for optionals would be confusing and or ambiguous if it were overloaded to also stand for a placeholder type.

Some users also worried that the underscore spelling would preclude the same spelling from being used for automatically type-erased containers, e.g.,

var anyArray: Array<_> = [0]
anyArray = ["string"]
let stringArray = anyArray as? Array<String>

This objection to the _ is compelling, but it was noted during discussion that usage of an explicit existential marker keyword (a la any Array<_>) could allow the usage of an underscore for both placeholder types and erased types.

At the pitch phase, the author remains open to alternative spellings for this feature. In particular, the "any Array<_>" resolution does not address circumstances where an author may want to both erase some components of a type but allow inference to fill in others.

Future directions

Placeholders for generic bases

In some examples, we're still technically providing more information than the compiler strictly needs to determine the type of an expression. E.g., in the example from the Type inference section, we could have conceivably written the type annotation as:

let publisher: _<Int, _> = Just(makeValue()).setFailureType(to: Error.self).eraseToAnyPublisher()

Since the type of the generic AnyPublisher base is fully determined from the result type of eraseToAnyPublisher(). The author is skeptical that this ultimately results in clearer code, and so opts to defer consideration of such a feature until there is further discussion about potential uses/tradeoffs.

47 Likes
2

Great pitch, and I think it makes a lot of sense for the partial generic case. For this one I'm not as sure about the utility though:

let stringTransform = foo as (String) -> _
3

Yeah, I could see a case for restricting this feature initially to just generic arguments and seeing if users really end up wanting to expand the supported locations for placeholder types. OTOH, I also can't think of a great reason that placeholders shouldn't be available for, e.g., the return type in function types, even if the utility is more limited than for generic arguments.

6 Likes
4

I really like this bikeshed. Now about the color. I think _ should be more explicit. My initial thought is that we are throwing away the type. What about about saying

var anyArray: Array<typealias _> = [0]

This would then allows us to capture these types for later use if we wanted.

alternative: autotype, inferredtype

5

A similar idea was proposed for C# 8 years ago. I don't think it went anywhere. (Which is probably because of the unpronounceable name.)

6

I could get behind an explicit marking, though I think it should be as lightweight as possible. Maybe something like infer _? This could also be made to apply recursively, so that infer [_: _] allows you to have placeholders for the Key and Value types for a Dictionary.

This would also resolve the issue mentioned in alternatives considered where you have a type where you want _ to sometimes refer to existential erasure and sometimes refer to a placeholder: this type would be written as, e.g., any [infer _: _], which is a dictionary of unknown value type with the Key type inferred from context.

Interested in other's thoughts on this.

1 Like
8

This looks totally reasonable to me. Nicely-written proposal

10 Likes
9

I like it, and I like the plain _ syntax.

I think it’s worth mentioning that this:

can be solved as follows:

func makeEither<L, R>(left: L, rightType: R.Type) -> Either<L, R> { Either(left: left) }
let publisherOrValue = makeEither(left: makePublisher(), rightType: Int.self)

Either<_, Int> is the same, only there’s no function.

I see this as an argument in favour of the pitch; it’s not adding “more magic” to the language, just expressing something that the compiler can already do in a vastly clearer way.

5 Likes
10

I'm worried that this proposal would make code too brittle. If I understand correctly, use of placeholder types would prevent the definition of overloads whose types would be ambiguous; new overloads would become breaking changes. For instance, let stringTransform = foo as (String) -> _ would prevent the future definition of func foo(_ x: String) -> String. And allowing Either<_, Int>(left: makePublisher()) would make future overloads of makePublisher cause existing code to stop compiling. I think it's a bad idea to lock in code like this.

11

Thanks, Doug!

This is a good point. The "pass the type" pattern is basically an ad-hoc version of this feature (although that pattern has the added benefit of allowing you to explicitly specialize functions).

This already the case—if we adopted @jayton's suggested makePublisher factory today (which compiles just fine without placeholder types!) introducing makePublisher() overloads would cause ambiguity issue.

Even if you just restrict your view to functions and methods that already have overloads and worry about issues with introducing additional overloads, this proposal doesn't introduce any new problems. For instance, consider the two foo overloads from the top of the pitch:

func foo(_ x: String) -> Double { return Double(x.count) / 2.0 }
func foo(_ x: Int) -> Double { return Double(x) }

Say we had a function:

func transformEmoji<T: ExpressibleByUnicodeScalarLiteral>(_ f: (T) -> Double) -> Double {
    return f("😄")
}

Then, today, the following line compiles just fine without any additional type context:

transformEmoji(foo)

However, if we add another foo overload:

func foo(_ x: Character) -> Double { return Double(String(x).count) / 2.0 }

Then the above line becomes ambiguous, even though there were already other overloads of foo which were overloaded on the argument type.

1 Like
12

+1

I just encountered a situation where this would have been helpful:

struct ManagedArrayBuffer<Header, Element> {
  init(minimumCapacity: Int, initialHeader: Header)
}

struct MyStorage<Header> {
  var codeUnits: ManagedArrayBuffer<Header, UInt8>

  init(...) {
    // Element == UInt8 inferred here
    self.codeUnits = ManagedArrayBuffer(minimumCapacity: c, initialHeader: header)
  }
}

Refactoring this code, I was exploring making the MAB initializer failable and propagating that through MyStorage's initializer. This meant it needed to be stored in a local variable, which lost the inference of the Element type, and meant that I had to write out the entire generic signature:

struct MyStorage<Header> {
  var codeUnits: ManagedArrayBuffer<Header, UInt8>

  init?(...) {
    // Error: Cannot infer type of 'Element'.
    guard let storage = ManagedArrayBuffer(minimumCapacity: c, initialHeader: header) else {
      return nil
    }
    self.codeUnits = storage
  }
}

I actually had to split the new version up across multiple lines, it was so long:

guard let storage = ManagedArrayBuffer<Header, UInt8>(
  minimumCapacity: c,
  initialHeader: header
) else {
  return nil
}

This feature would have allowed me to write:

guard let storage = ManagedArrayBuffer<_, UInt8>(minimumCapacity: c, initialHeader: header) else {
  return nil
}

It's kind of annoying to have to go through all of these steps for quite a simple thing. Swift has type inference because it's convenient and makes code easier to read (which comes with lots of positive side-effects), but right now it's binary - either it can infer everything, or it infers nothing. So I like the idea of being able to fill in the gaps and enabling inference to work in more situations.

The benefits also scale nicely if you have longer, more descriptive type names (which Swift encourages) that can now be omitted, or if you have more generic parameters.

2 Likes
13

Since Swift now have a very high bar for source breakages, wouldn't it be better to require the infer keyword now, then remove it later when we are sure that it's not needed?

14

If I understand correctly, the placeholder is not permitted in where clauses, it would be good to specify that.

Is this intended to work for more complicated situations, say something like:

MyType1<_, MyType2<_>>

where there are some constraints which force computation of one after the other? For example, it's possible that MyType2's generic argument is needed to infer MyType1's first generic argument. Are these all guaranteed to be part of the same constraint system/will it just work regardless of what structure the type has and what the order of placeholder type instantiation needs to be?

2 Likes
15

I’m not sure I follow. How would planning to make a source-breaking change later fulfill the very high bar for source-breaking changes over a proposal that makes an additive change?

3 Likes
16

I'm probably thinking it wrong then.

I was thinking that removing the requirement of infer keyword will not be source-breaking, but adding the requirement later on will be source-breaking.

17

I understood the line of argument to be:

Let's conservatively require the infer keyword to introduce this feature so that we don't risk removing the possibility of using _ in type signatures to mean something else. Later, once we've seen where those hypothetical features actually fit in the language, we could drop the required infer keyword if it still makes sense (but still keep it around as a legacy syntax).

If we use a bare underscore now, we're committing to the meaning of "infer this type" for a long time to come.

Given the general sentiment in this thread, though, it seems that meaning matches most people's intuitive expectations anyway.

4 Likes
18

Yep, good point, will do!

From a feature-level perspective, yeah, I see nothing wrong with the type as written nor with the order-dependent resolution of placeholder types. As long as there's not a cyclical dependency (e.g., the compiler can figure out the generic argument of MyType2 with the rest of the context), I'd expect there to be no issues inferring the full type.

Somewhat off-topic implementation-level discussion

I can't think of a situation where a type like the one you've written wouldn't end up with all of the placeholders injected in the same constraint system, since it would either appear as the contextual type for an expression or as a type expression itself (e.g., in an as cast). FWIW I have a prototype implementation locally which just resolves placeholders to fresh type variables and for the most part it does "just work" with simpler systems, but if you have any more pathological test cases you'd like me to throw at it I'm more than happy to give it a whirl.

Anyway, further discussion along this line (which I'm happy to indulge in—don't want to seem like I'm shutting you down) probably deserves to move to a new thread in Compiler.

19

The way I see it, I don't have to use any extra keywords to declare my intent when I want to match any value in the payload of a case, nor when ignoring the return value of a function:

switch someValue {
case .someCase(let first, _, let third):
}

_ = ignoredReturnValue()

So I agree—I think adding a keyword here would just be noise.

I do like the idea of being able in the future to capture the types for later use and typealias seems like a great keyword for that (there's no need to invent a new one), but that's an entirely additive change that can be deferred and nothing in this proposal prevents it.

9 Likes
20

Thanks for the explanation. This is exactly what I meant, and I think it comes across much more clearly than my wording.

21

This proposal looks nice to me. It makes perfect sense and dovetails with existing functionality well.

13 Likes