Associated Conformances? Similar to Higher Kinded Types?

I encountered a need for something I don’t think Swift can handle. In my mind it vaguely resembles higher kinded types, but it operates on different parts of the type system. I’m wondering what it really is, has it been named, proposed before, etc.

Basically I want to be able to define a sort of “associated conformance” in a protocol. This is a parameter that conforming types will specify, that needs to be something that can be used in the conformance declaration of a type (placed after the :). This includes protocols and non-final classes. I then want to be able to access that parameter in generics to constrain other generic parameters.

The example in which I encountered this is roughly as follows:

I want to define a formatter for HTTP requests and responses. Right now it looks like this:

protocol EncodingFormat {
  associatedtype Input

  var contentType: String { get } // To determine what the “Content-Type” header should be

  func format(_ input: Input) -> Data
}

protocol DecodingFormat {
    associatedtype Output

    var contentType: String { get } // To determine what the “Accept” header should be

    func parse(_ input: Data) throws -> Output
}

The canonical example of a formatter is for JSON:

struct JSONEncodingFormat<Input: Encodable>: EncodingFormat {
  let contentType = "application/json"

  func format(_ input: Input) -> Data {
    try! JSONEncoder().encode(input) 
  }
}

struct JSONDecodingFormat<Output: Decodable>: DecodingFormat {
    let contentType = "application/json"

    func parse(_ input: Data) throws -> Output {
        try JSONDecoder().decode(Output.self, from: input)
    }
}

With this, the encoding or decoding formats for two different types are different types (due to the generic parameter). For example, DecodingFormat<Int> is a different type than DecodingFormat<String>. But I want to express that they’re actually the same. For example, a response from a server might return one type of object if successful, and a different type of object for an error, but they should both be in the same format, specified by one “Accept-Encoding” header. As is, generic code must take two type parameters for the success and error decoding formats, but really they should take one, which is then capable of decoding a category of response objects.

This means the type parameter should move down to the functions:

protocol EncodingFormat {
  var contentType: String { get }

  func format<Input>(_ input: Input) -> Data
}

protocol DecodingFormat {
    var contentType: String { get }

    func parse<Output>(_ input: Data, to type: Output.Type) throws -> Output
}

But now for something to be an Encoding/Decoding format, it must be capable of encoding/decoding anything. JSON no longer satisfies this, nor I imagine would any other useful codec. JSON requires Encodable/Decodable types. Another codec might require some other conformance for what it can encode/decode. So I want to be able to say something like this:

protocol EncodingFormat {
  associatedconformance Formattable

  var contentType: String { get }

  func format<Input: Formattable>(_ input: Input) -> Data
}

protocol DecodingFormat {
    associatedconformance Parseable

    var contentType: String { get }

    func parse<Output: Parseable>(_ input: Data, to type: Output.Type) throws -> Output
}

Then the JSON codec would be:

struct JSONEncodingFormat: EncodingFormat {
  conformancealias Formattable = Encodable // Inferrable from function below?

  let contentType = "application/json"

  func format<Input: Encodable>(_ input: Input) -> Data {
    try! JSONEncoder().encode(input) 
  }
}

struct JSONDecodingFormat: DecodingFormat {
    conformancealias Parseable = Decodable // Inferrable from function below?

    let contentType = "application/json"

    func parse<Output: Decodable>(_ input: Data, to: Output.Type) throws -> Output {
        try JSONDecoder().decode(Output.self, from: input)
    }
}

In fact, even better, we can now merge the encoding and decoding side together and eliminate the duplicate contentType:

struct JSONFormat: EncodingFormat, DecodingFormat {
  conformancealias Formattable = Encodable // Inferrable from function below?
  conformancealias Parseable = Decodable // Inferrable from function below?

  let contentType = "application/json"

  func format<Input: Encodable>(_ input: Input) -> Data {
    try! JSONEncoder().encode(input) 
  }

  func parse<Output: Decodable>(_ input: Data, to: Output.Type) throws -> Output {
    try JSONDecoder().decode(Output.self, from: input)
  }
}

I would then want to be able to access the conformance through a generic parameter and use it to constrain another generic parameter:

struct Client {
  func post<
    RequestFormat: EncodingFormat, 
    ResponseFormat: DecodingFormat, 
    Body: RequestFormat.Formattable, 
    Success: ResponseFormat.Parseable, 
    Failure: Error & ResponseFormat.Parseable
  >(
      _ body: Body,
      to url: URL,
      requestFormat: RequestFormat,
      responseFormat: ResponseFormat,
      failureType: Failure.Type
   ) async throws -> Success {
      let bodyData = requestFormat.format(body)

      let urlRequest = /* build URLRequest with httpBody built from bodyData,
        and appropriate headers added from the request and response parsers */

      let (data, response) = try await urlSession.data(for: urlRequest)

      guard let httpResponse = response as? HTTPURLResponse else {
        throw UnexpectedResponse(response)
      }

      if httpResponse.statusCode >= 400 {
        throw try responseFormat.parse(data, to: Failure.self))
      } else {
        return try responseFormat.parse(data, to: Success.self)
      }
   }
}

So this way, I can post only Encodable bodies if I specify JSON for the request format, but if I have another protocol like XMLEncodable for encoding XML bodies (that provides additional context for attributes, how to represent arrays, etc.), I can post only XMLEncodable bodies if I specify XML for the request format. And similarly for the response types, and both response types must be identically constrained because they are tied to a single DecodingFormatter type parameter.

This strikes me as vaguely related to higher-kinded types because both seem to involve categories of types and eventually composing a concrete type out of the interaction between such a category and another type parameter. With higher kinded types we want to deal with a generic concrete type without the type parameter as a sort of supertype of types (super-metatype, or meta-supertype?), from which specific examples may be generated by supplying a specific type parameter (i.e. for a struct MyStruct<T>, MyStruct.self is an abstract supertype of MyStruct<Int>.self, MyStruct<String>.self, etc.,). Here I also want to use something that acts only as a meta-supertype containing concrete types (MyProtocol.Protocol acting as a supertype of MyConformingStruct.self). Would these two capabilities look similar at all to the compiler or type checker?

1 Like

It looks like you’re looking for something we’ve called “generalized supertype constraints” in the past. There was a pitch thread on it several years ago. I don’t know if it’s been discussed more recently or not.

2 Likes

That's totally what it is! I've read that proposal before, not sure why I didn't make the connection. I think I was imagining I couldn't assign an associatedtype to a protocol and we'd need some new concept for that, but I've typealiased protocols plenty of times. Maybe there's some ambiguity here between a protocol and its existential that Swift 6 will fix.

The above seems like a solid use case for this that doesn't involve some type of reinvented casting (where I've also usually encountered the need for it). Really it's an enhancement to TopLevel(En|De)coder:

protocol TopLevelDecoder {
    associatedtype Input
    associatedtype Decodable = Swift.Decodable

    func decode<T: Decodable>(_ type: T.Type, from: Self.Input) throws -> T
}

protocol TopLevelEncoder {
    associatedtype Output
    associatedtype Encodable = Swift.Encodable

    func encode<T: Encodable>(_ value: T) throws -> Self.Output 
}

Then you could define an XML codec that includes both the (en|de)coder and the protocol the types must conform to:

protocol XMLDecodable {
  associatedtype Attributes: ...
  associatedtype Children: ...
}

protocol XMLEncodable {
  associatedtype Attributes: ...
  associatedtype Children: ...
}

struct XMLDecoder: TopLevelDecoder {
    func decode<T: XMLDecodable>(_ type: T.Type, from: Data) throws -> T { ... }
}

struct XMLEncoder: TopLevelEncoder {
    func encode<T: XMLEncodable>(_ value: T) throws -> Data { ... }
}

You can also unify some other existing concepts under the "codec" umbrella:

struct RawValueDecoder<Input>: TopLevelDecoder {
    associatedtype Decodable = RawRepresentable where Decodable.RawValue = Input

    func decode<T: Decodable>(_ type: T.Type, from rawValue: Input) throws -> T {
        guard let result = type.init(rawValue: rawValue) else { throw DecodeError(...) }
        return result 
    }
}

struct RawValueEncoder<Output>: TopLevelEncoder {
    associatedtype Encodable = RawRepresentable where Encodable.RawValue = Output

    func encode<T: Encodable>(_ value: T) throws -> Output {
        value.rawValue
    }
}

(Not sure if where clauses go beyond the scope of the existing proposal).

Then write generic code to, for example, transform Sequences or Publishers by (en|de)coding their outputs using only codecs that work with their output types.