Unlock Existential Types for All Protocols

I don't think Encodable is a good example for self-conformance. I would expect self-conformance of Encodable to encode as a container - serialising the type of the value together with the serialised value.

I think that's more of an example for the need of opening existentials:

extension URLRequest {
  mutating func encoded(encodable: Encodable, encoder: JSONEncoder = JSONEncoder()) throws -> URLRequest {
    let <T: Encodable> value: T = encodable
    httpBody = try encoder.encode(value)
  }
}

Why is even that necessary? You can just make the function generic, no need to erase or open anything. AnyCodable is useful for heterogenous collections, but this isn't an example of that requirement.

Unfortunately, it can not be generic.

/// Represents an HTTP task.
public enum Task {
  case requestPlain
  
  case requestData(Data)
  
  /// A request body set with `Encodable` type
  case requestJSONEncodable(Encodable)
}

Task is used in different network providers, it is impossible to make it generic, such as Task<T>.
In case we use requestJSONEncodable(Encodable), we use type erased Encodable. But try encoder.encode(value) function needs a concrete Type, so type erasure struct is needed.

For more details, see Moya: GitHub - Moya/Moya: Network abstraction layer written in Swift.

Yeah, Moya has some unfortunate abstractions that prevent it from working well in these situations. A router that goes directly to the URLRequest can handle this much more easily, as you can assign a specific type per route, or use other abstractions. Raw use of Encodable should be avoided for that reason. I do wonder if this case will be helped by this proposal though.

It will be off-topic, but maybe you can use extension of Encodable.

private extension Encodable {
    func encoded(encoder: JSONEncoder = JSONEncoder()) throws -> Data {
        return try encoder.encode(self)
    }
}

With this, you can use Encodable.encoded(encoder:), so you can write this. As far as I checked your repository, you don't have to use AnyEncodable with this extension.

extension URLRequest {
  mutating func encoded(encodable: Encodable, encoder: JSONEncoder = JSONEncoder()) throws -> URLRequest {
    httpBody = try encodable.encoded(encoder: encoder)
  }
}

Can you please clarify, why raw use of Encodable should be avoided?

Here we talk about different approaches for making network layer abstractions. Each one has its own trade-offs.
My point here is that more expressivity gives us a choice in making such abstractions.

Thanks for this great example. Yes, this is a good workaround.
Interesting thing here is that really concrete Type isn't known to the compiler, and Encodable behaves like a self conforming protocol:

extension Encodable {
    func encoded(encoder: JSONEncoder = JSONEncoder()) throws -> Data {
        return try encoder.encode(self) // encodable is used here as self conforming protocol, concrete type isn't known
    }
}

So why it is not possible to pass Encodable existential directly to encoder.encode(_:) function now? This is the question. Seems like an unnecessary limitation, because writing a function in extension do the thing.

1 Like

Also there are situations, where this trick is impossible.

let result: Result<String, LocalizedError> // compiler error

extension LocalizedError {
  // compiled, can be called inside extension, but can not be called outside 
  func makeResult<T>(_ t: T.Type) -> Result<T, Self> {
    .failure(self)
  }
    
  func performFailureCompletion<T>(_ t: T.Type, completion: @escaping (Result<T, Self>) -> Void) {
    let result = makeResult0(t.self, error: self)
    completion(result)
  }
  
  func process() {
    performFailureCompletion(Int.self) { result in
      // result here is of Type Result<String, LocalizedError>, which can not be created directly. But here it exists
      print(result)
    }
  }
}

Having Result with Failure of type LocalizedError or any other Error Type is very convenient and often necessary. In real project many of Error-type protocols can exist. Creating type erasure struct for each one is unpleasant.

Pretty sure that's incorrect. The type would be Result<String, Self>, where Self is the type conforming to LocalizedError. It's no different than creating the type directly: Result<String, SomeConformingType>. So it's not clear what you're getting at here.

struct InteractorError: LocalizedError {}

let localizedError: LocalizedError = InteractorError()

localizedError.process() // So, calling this function, we get Result<Int, LocalizedError> in completion

In fact, if we print type(of: result) inside this function, Result<Int, InteractorError> will be printed.

I want to pay attention, that Result<Int, LocalizedError> exists in mental model. We call this function on existential instance of LocalizedError, and it creates result of its own type – Result<Int, LocalizedError>.

I think your example is a good motivation example for Existential subtyping as generic constraint.

For the sake of this discussion, I'm gonna use syntax any P vs P to disambiguate between protocol-as-existential vs protocol-as-contraint, but exact syntax is irrelevant.

Image that there is no self-conformance (any Error): Error. But in the generic signature of the Result we still want to be able to use any Error. Signature of enum Result<Success, Failure> where Failure : Error would not work here. But we can fix this if we introduce another kind of constraint - existential subtyping. I was suggesting syntax where Failure : any Error. This constraint means that Failure can be type that conforms to Error, but also any Error itself, and other existentials like any LocalizedError.

Such constraint would allow Failure to be any type which can be implicitly converted to any Error.

let concrete: Failure = ....
// If this compiles
let existential: any Error = concrete
// then this will compile as well.
let result: Result<Never, Failure> = .failure(concrete)

Would such language mechanism cover your needs?

Thanks for detailed response. Yes, this concept covers them.
Declaration Result<Success, Failure> where Failure : any Error makes it possible. It is also possible if we could make SomeCustomErrorProtocol self conforming.

In addition I want to say, that your example with WeakRef<T: any AnyObject> is very revealing. We have the same problem in our project.

I'm not against this per se though I don't like that there is any semantic difference between Failure: any Error and Failure: Error.
If Error doesn't self conform to itself, then the inner type of the existential should be implicitly unboxed to satisfy the bound.

1 Like

Not sure we understood each other. I'm suggesting to introduce a new kind of bound, which is semantically different from the existing "conforms to protocol" bound. And for that new kind of bound, existential satisfies the bound without unboxing.

Semantic difference between the two cases is essentially what I'm suggesting. Syntax though could be tweaked. You can make one of the cases where Failure <: Error, where Failure as Error, where Failure castableTo Error or anything else.

The reason why I'm suggesting to use the colon, is because I think this kind of bound is not so new actually:

class Base {}
class Derived: Base {}
struct Foo<C: Base> {}
let y = Foo<Derived>() // OK
let x = Foo<Base>() // OK

Superclass constraint is already this kind of constraint - base class itself satisfies the constraint. So the colon is already overloaded to mean two kinds of constraints, and overload resolution is driven by the RHS. If RHS a type - it is a subtyping constraint and bound is included. If RHS is a protocol - it is a protocol conformance constraint and bound is not included.

So I'm suggesting the syntax which preserved this overloading, and continues driving it by allowing to switch the role of the protocol in the RHS.

I understand you and I don't like it. I kinda want conformance imply subtyping and vice versa (for protocols), which is currently not that correct. But it may be closed some day with automatic existential unboxing, adding your feature would enlarge the gap between conformance and (existential) subtyping.

Merging subtyping and conformance for protocols is not possible in general case, because of static requirements (including initialisers) and associated types used in non-covariant positions.

One of the challenges for automatic unboxing is ambiguity created by self-conforming protocols:

func handleError<E: Error>(_ error: E) {}
let e: Error = MyError()
handleError(e) // Unbox or not?

My feature allows to eliminate self-conformance, thus avoiding ambiguity and making automatic unboxing one step closer.

Don't know what you mean with static requirements, but Self, associated types create hidden constraints.
We may have a chance to handle all that with path dependent typing.

Only unbox if self conformance isn't given. I think the shadowing type has precedence, and ideally implicit unboxing shouldn't change semantic considering self conformances.

I mean protocol requirements which are static properties or functions.

protocol P {
    static var foo: Int { get }
}
struct S: P { static var foo = 42 }
struct R: P { static var foo = 37 }
print(S.foo) // 42
print(R.foo) // 37
print(P.foo) // ???

// Similar problems with initialisers.
protocol P2 {
    init(string: String)
}

Where can I read more about this? Is it the same as unboxing existentials?

Meh, I wasn't aware of this. I would reject this intuitively as we have extensions for using statics, but it seems to relate to classes/structs here.

A bit of it was mentioned in the future directions of this proposal.

I found a nice but older post to that topic.

After reading the links, I think path dependent typing can be summarised as "implicit unboxing of existentials". Looks nice and I see myself using it, though I would feel safer with explicit unboxing available as well, as a sort of fallback to troubleshoot complicated cases, or optimise compilation speed.

Thinking about how to apply implicit or explicit unboxing to @Dmitriy_Ignatyev's example, I see a couple of challenges:

// How result type should look like?
// We cannot use `le` in the result type, because it is out of scope.
// We need to erase the type, while preserving the fact that error type conforms to LocalizedError
func getResult<Success>() -> ??? {
    let le: LocalizedError = getError() // Existential
    // We need to specify `Success` while using unboxed error type of `le` as failure.
    // What would the syntax look like?
    let result: Result<???>  = .failure(le) // Result<Success, type(of: le)>
    return result
}

Applying Syntax for existential type as a box for generic type and Placeholder types to address those, solution may look like this:

func getResult<Success>() -> any<Failure: LocalizedError> Result<Success, Failure> {
    let le: LocalizedError = getError() // Existential
    let result: Result<Success, _>  = .failure(le) // Result<Success, type(of: le)>
    return result
}

This also avoids the need for self-conformance, instead "existentialness" bubbles up. And cases where we currently use Result<S, any Error> become any<F: Error> Result<S, F>. Since that's a pretty common case, this probably deserves a typealias in the standard library:

typealias AnyErrorResult<S> = any<F: Error> Result<S, F>