Challenges with optional associated types

Hi,

I hit an interesting problem recently that I think bumps against a couple of issues with Swift 4. I’m curious to hear what others think and if anything has been proposed to tackle this in future.

The tl;dr: associated types can be specified to be optional types, and this seems to break the ability to use generic constraints that use the associated types. This affects Swift-y API design.

Here’s a concrete example:

// ---- Primitives needed to demonstrate the issue

protocol Action {
   associatedtype StateType

   var name: String { get }
}

protocol QueryParameterCodable {
   func encodeAsQueryItems() -> [URLQueryItem]
}

/// Here we constrain on the associated type
func encodeState<T>(for action: T, with state: T.StateType) where T: Action, T.StateType: QueryParameterCodable {
   print("Encoded state for \(action.name): \(state.encodeAsQueryItems())")
}

// ---- Use-cases

class FancyAction: Action {
   /// This works. Change it to `FanceState?` and we’re in trouble...
   typealias StateType = FancyState

   let name: String = "Fancy"

   struct FancyState: QueryParameterCodable {
       var title: String

       func encodeAsQueryItems() -> [URLQueryItem] {
           return [URLQueryItem(name: "title", value: title)]
       }
   }
}

let action = FancyAction()
let state = FancyAction.StateType(title: "Haken - Affinity")
encodeState(for: action, with: state)

So the above works, but as shown in the comments if you change StateType to be aliased to the optional FancyType?, you hit problems which are entirely reasonable given current Swift:

  1. The encodeState method is not available for the optional StateType. This is expected as Optional does not directly conform to QueryParameterCodable. However because you cannot constrain on optional types because they are enums, you cannot overload the function to have a version that is also generic over Optional - so you cannot workaround it this way
  2. You cannot seemingly change the requirements for the associatedtype StateType to express the fact that it can be any type except Optional<_>. It does not seem possible to specify that it should be any non-Optional or non-Enum type either.
  3. This means you apparently cannot use such an associatedtype safely for any generic constraints. If you are creating a “public” API, the developer cannot be prevented from specifying an optional that will break your generically constrained function declarations, and unfortunately this results in rather oblique compiler errors for the developer that do not make it clear that the problem is related to the Optionality they have specified.

Ultimately it is the enum-ness (enumbness?!) that is the actual problem for constraining on the associated type, but even if there were some kind of AnyNonEnum you could constrain the associatedtype with, you’ve now lost the ability for the API user to support optional values, and likely have to make all the function calls accept T.StateType?, which unnecessarily weakens your typing and means any caller could pass nil and you’d have to precondition or similar on this for cases where it really made no sense (for certain kinds of StateType).

Essentially, and I haven’t read this anywhere so I’m probably misunderstanding, but it seems that generally speaking associatedtype cannot be used for Optional types unless you guarantee you never use that type for generic constraints. This seems like quite an opaque but important problem.

It looks like Swift 4.1 conditional conformances can supply a workaround:

extension Optional: QueryParameterCodable where Wrapped: QueryParameterCodable {
   func encodeAsQueryItems() -> [URLQueryItem] {
       switch self {
           case .some(let value): return value.encodeAsQueryItems()
           case .none: return []
       }
   }
}

…but it looks like this would become pretty onerous having to add myriad extensions on Optional for every type that has an associatedtype that might be optional.

Is there any proposal or discussion elsewhere e.g. on swift-evolution, about this problem?

Cheers

The thing is whenever you specify an Optional type you need to define the fallback behaviour somewhere.
You wrote an example on how to support it with conditional conformance (though you could write return self?.encodeAsQueryItems() ?? [] I believe instead of switching).
I can also think of overloading encodeState to accept the case where T.StateType: Optional, T.StateType.Wrapped: QueryParameterCodable, have a base _encodeState(for:queryItems:) and call that from both, but I see as it is cumbersome.
It seems you want something like automatic promotion for generic constraints, so if declaring encodeState only for Optional StateTypes, it would catch it for non-Optional ones as well, and there you would be required to specify the fallback behaviour in place. i.e. \(state?.encodeAsQueryItems() ?? []).

I'm not sure this is a case common enough to warrant language changes, as decent workarounds (conditional conformance) exist...

There are plenty of occasions where this is desirable.

In my example, an action can specify the type that represents its state, and if you consider a perform function on Action that takes state, that state in many cases could be optional.

That's where it would be dealt with. The protocol extensions should be able to facilitate this, but due to the enum limitation cannot.

BTW I'm not arguing that we should have auto-promotion for the sake of generic constraints, although that might solve it.

The wider problem I am getting at is a very practical one: associatedtypes and optionals do not mix well at all as it immediately precludes using the associatedtype in generic constraints. It's like a little trap waiting to trip you up after you've designed your API. That's not good is it?

There's one thing I don't really get in what you're saying though: what is it about enums, that you say are the problem, that is specific about this issue? Maybe you mean (generic) wrapper types?

I think focusing on Optional or enums specifically is a bit misleading. You would get the same problem if you wrote typealias StateType = [FancyState], except then, instead of needing logic to handle nil, you need logic to handle multiple values. But it amounts to the same limitation.

As @defrenz points out, you have to specify the handling of the nil somewhere. The usual choice for a default behavior is different from the one you use. Normally if you wanted to lift encodeState to handle optional arguments, you would also change it's return type to be optional as well. This is similar to how optional map or chaining works. Then, if an empty array was a suitable result, you'd use ?? [] to convert the nil to that. But the choice is very domain specific, it can't be provided at the outermost point.

Note that you can force StateType to conform to QueryParameterCodable:

typealias StateType: QueryParameterCodable = FancyState

That way you are banning Optional (and Array, and MyWrapperType<T>) unless they also conform to QueryParameterCodable.

Hi Ben -

Yes you are correct, however I would contend that the use case / mental model is quite different. As an optional value is still a single value, and therefore you would expect "somethig or nil" to work out of the box more than "Something or a collection of things".

I'm afraid my example is perhaps clouding the discussion of this. Your reference to returning a nil from encodeState makes me think this.

Instead of encodeState imagine another generic function that has no return value.

The specific use case I have is about invoking actions with their own associated state type - and possibly accepting nil - and calling a perform method. I was just trying to present a simpler example of a similar problem.

I can't constraint the associated type because conforming to e.g. QueryParameterCodable is optional in the API. The StateType can and should be anything, and it should be possible to make it optional IMO.

As soon as I introduce this associatedtype however, the user of the API is likely to see confusing compiler messages about missing functions if they make the type optional.

I don't see any solutions for this:

  1. The associatedtype cannot be constrained, it should essentially be Any, with non-required conformance to other protocols used to provide protocol extension functions conditionally
  2. The associatedtype should support optionals, or forcefully prevent their use if supporting them is not possible

It seems the possible workaround using conditional conformances in 4.1 may work but doesn't scale very well.

At this point I have another question then, still relative to your example: is it extremely desirable to have encodeState available only for types that have a StateType: QueryParameterCodable? Otherwise a solution might be:

func encodeState<T>(for action: T, with state: T.StateType) where T: Action {
   let queryItems = (state as? QueryParameterCodable)?.encodeAsQueryItems() ?? []
   print("Encoded state for \(action.name): \(queryItems)")
}
1 Like

IMO it is desirable because when the programmer is using this API they will not be able to autocomplete a function that they cannot validly call.

We want it so they cannot compile code that e.g. tries to encode state for actions that have not declared a state type that is capable of this, it is a programmer error.

Trying to do the "Swift thing" :)