Optional Protocol Members

Evolution Discussion
1

Optional Protocol Requirements Members

This post tries to summarise the challenges, the solutions to the challenges, and the problems of the solutions to the challenges brought here ~8 years ago. Many things have changed in Swift since then and I believe it's worth having a fresh look at the issues.

Consider Swift framework. No Objective-C, no NSObject, and, sadly, no optional protocol members. (I'm deliberately using "protocol members" name firstly to account for not just functions but variables, and secondly to avoid the "optional requirements" oxymoron).

API protocols do evolve. Take URLSessionDataTaskDelegate protocol for example – it was changed at least 5 times in different OS versions. It's great to have optional methods: old code that doesn't know about them would still compile, the old binary would still run. Often times protocol default implementation (done in a protocol extension) is brought up as the way to go. There are a few gotchas with that approach in practice.

  1. Default implementation is not enough sometimes.
    Sometimes API needs to know if the client implemented method or not to act differently. Example:
CBCentralManagerDelegate.centralManager(CBCentralManager, willRestoreState: [String : Any])).

Depending upon whether this method is implemented or not the API acts differently. Compared to Objective-C in Swift there is no way to say if a type provided protocol method implementation, other than resorting to hacks:

protocol P {
	func foo()
}
extension P {
	func foo() {
		someTaskLocalVariable += 1
	}
}
let old = someTaskLocalVariable
delegate.foo()
let methodImplemented = someTaskLocalVariable == old
  1. Default implementation could be expensive:
protocol P {
	func foo() -> ExpensiveToCreateType
}
extension P {
	func foo() -> ExpensiveToCreateType {
		.init() // expensive
	} 
}

If the client doesn't implement this method and there was a way to query if the method is implemented by the client or not – the system could skip calling it's default implementation and do things more efficiently.

  1. Splitting a protocol into several (as a means to support backward compatibility):
    Would result in a serious protocols proliferation.
public protocol URLSessionTaskDelegate : URLSessionDelegate {
    func urlSession(_ session: URLSession, task: URLSessionTask, willPerformHTTPRedirection response: HTTPURLResponse, newRequest request: URLRequest, completionHandler: @escaping @Sendable (URLRequest?) -> Void)
    func urlSession(_ session: URLSession, task: URLSessionTask, willPerformHTTPRedirection response: HTTPURLResponse, newRequest request: URLRequest) async -> URLRequest?
    func urlSession(_ session: URLSession, task: URLSessionTask, didReceive challenge: URLAuthenticationChallenge, completionHandler: @escaping @Sendable (URLSession.AuthChallengeDisposition, URLCredential?) -> Void)
    func urlSession(_ session: URLSession, task: URLSessionTask, didReceive challenge: URLAuthenticationChallenge) async -> (URLSession.AuthChallengeDisposition, URLCredential?)
    func urlSession(_ session: URLSession, task: URLSessionTask, needNewBodyStream completionHandler: @escaping @Sendable (InputStream?) -> Void)
    func urlSession(_ session: URLSession, needNewBodyStreamForTask task: URLSessionTask) async -> InputStream?
    func urlSession(_ session: URLSession, task: URLSessionTask, didSendBodyData bytesSent: Int64, totalBytesSent: Int64, totalBytesExpectedToSend: Int64)
    func urlSession(_ session: URLSession, task: URLSessionTask, didCompleteWithError error: Error?)
}

protocol URLSessionTaskDelegate_macOS_10_12 {
    func urlSession(_ session: URLSession, task: URLSessionTask, didFinishCollecting metrics: URLSessionTaskMetrics)
}

protocol URLSessionTaskDelegate_macOS_10_13 {
    func urlSession(_ session: URLSession, task: URLSessionTask, willBeginDelayedRequest request: URLRequest, completionHandler: @escaping @Sendable (URLSession.DelayedRequestDisposition, URLRequest?) -> Void)
    func urlSession(_ session: URLSession, task: URLSessionTask, willBeginDelayedRequest request: URLRequest) async -> (URLSession.DelayedRequestDisposition, URLRequest?)
    func urlSession(_ session: URLSession, taskIsWaitingForConnectivity task: URLSessionTask)
}

protocol URLSessionTaskDelegate_macOS_13_0 {
    func urlSession(_ session: URLSession, didCreateTask task: URLSessionTask)
}

protocol URLSessionTaskDelegate_macOS_14_0 {
    func urlSession(_ session: URLSession, task: URLSessionTask, needNewBodyStreamFrom offset: Int64, completionHandler: @escaping @Sendable (InputStream?) -> Void)
    func urlSession(_ session: URLSession, needNewBodyStreamForTask task: URLSessionTask, from offset: Int64) async -> InputStream?
    func urlSession(_ session: URLSession, task: URLSessionTask, didReceiveInformationalResponse response: HTTPURLResponse)
}

// usage from within the API:
if let delegate = delegate as? URLSessionTaskDelegate_macOS_13_0 {
    delegate.urlSession(session, didCreateTask: task)
}
  1. Splitting a protocol into multiple protocols (each optional member getting it's own protocol):
    Would result into even more severe protocol proliferation and would be beyond ridiculous:
public protocol URLSessionTaskDelegate_didCreateTask {
    func urlSession(_ session: URLSession, didCreateTask task: URLSessionTask)
}
public protocol URLSessionTaskDelegate_willBeginDelayedRequest {
    func urlSession(_ session: URLSession, task: URLSessionTask, willBeginDelayedRequest request: URLRequest, completionHandler: @escaping @Sendable (URLSession.DelayedRequestDisposition, URLRequest?) -> Void)
}
public protocol URLSessionTaskDelegate_willBeginDelayedRequest_async {
    func urlSession(_ session: URLSession, task: URLSessionTask, willBeginDelayedRequest request: URLRequest) async -> (URLSession.DelayedRequestDisposition, URLRequest?)
}
public protocol URLSessionTaskDelegate_taskIsWaitingForConnectivity {
    func urlSession(_ session: URLSession, taskIsWaitingForConnectivity task: URLSessionTask)
}
public protocol URLSessionTaskDelegate_willPerformHTTPRedirection {
    func urlSession(_ session: URLSession, task: URLSessionTask, willPerformHTTPRedirection response: HTTPURLResponse, newRequest request: URLRequest, completionHandler: @escaping @Sendable (URLRequest?) -> Void)
}
public protocol URLSessionTaskDelegate_willPerformHTTPRedirection {
    func urlSession(_ session: URLSession, task: URLSessionTask, willPerformHTTPRedirection response: HTTPURLResponse, newRequest request: URLRequest) async -> URLRequest?
}
public protocol URLSessionTaskDelegate_didReceiveChallenge {
    func urlSession(_ session: URLSession, task: URLSessionTask, didReceive challenge: URLAuthenticationChallenge, completionHandler: @escaping @Sendable (URLSession.AuthChallengeDisposition, URLCredential?) -> Void)
}
public protocol URLSessionTaskDelegate_didReceiveChallenge_async {
    func urlSession(_ session: URLSession, task: URLSessionTask, didReceive challenge: URLAuthenticationChallenge) async -> (URLSession.AuthChallengeDisposition, URLCredential?)
}
public protocol URLSessionTaskDelegate_needNewBodyStream {
    func urlSession(_ session: URLSession, task: URLSessionTask, needNewBodyStream completionHandler: @escaping @Sendable (InputStream?) -> Void)
}
public protocol URLSessionTaskDelegate_needNewBodyStreamForTask {
    func urlSession(_ session: URLSession, needNewBodyStreamForTask task: URLSessionTask) async -> InputStream?
}
public protocol URLSessionTaskDelegate_needNewBodyStreamFromOffset {
    func urlSession(_ session: URLSession, task: URLSessionTask, needNewBodyStreamFrom offset: Int64, completionHandler: @escaping @Sendable (InputStream?) -> Void)
}
public protocol URLSessionTaskDelegate_needNewBodyStreamForTask {
    func urlSession(_ session: URLSession, needNewBodyStreamForTask task: URLSessionTask, from offset: Int64) async -> InputStream?
}
public protocol URLSessionTaskDelegate_didSendBodyData {
    func urlSession(_ session: URLSession, task: URLSessionTask, didSendBodyData bytesSent: Int64, totalBytesSent: Int64, totalBytesExpectedToSend: Int64)
}
public protocol URLSessionTaskDelegate_didReceiveInformationalResponse {
    func urlSession(_ session: URLSession, task: URLSessionTask, didReceiveInformationalResponse response: HTTPURLResponse)
}
public protocol URLSessionTaskDelegate_didFinishCollecting_metrics {
    func urlSession(_ session: URLSession, task: URLSessionTask, didFinishCollecting metrics: URLSessionTaskMetrics)
}
public protocol URLSessionTaskDelegate_didCompleteWithError {
    func urlSession(_ session: URLSession, task: URLSessionTask, didCompleteWithError error: Error?)
}

// usage from within the API:
if let delegate = delegate as? URLSessionTaskDelegate_taskIsWaitingForConnectivity {
    delegate.urlSession(session, taskIsWaitingForConnectivity: task)
}

  1. Lost argument labels.
    When you switch from func foo(x: Int, y: Int) to var foo: ((x: Int, y: Int) -> Void)? you are missing argument labels: var foo: ((Int, Int) -> Void)?. The best you could do at the declaration site is to keep the internal names: var foo: ((_ x: Int, _ y: Int) -> Void)? but those names would be invisible at the call site: delegate.foo(1, 2)

  2. Variable names clash.
    In many cases you'd have to rename the variable to avoid the clash:

    var tableView: (_ tableView: UITableView, _ titleForHeaderInSection: Int) -> String?
    var tableView: (_ tableView: UITableView, _ titleForFooterInSection: Int) -> String? :stop_sign:

    var tableView_titleForHeaderInSection: (_ tableView: UITableView, _ section: Int) -> String?
    var tableView_titleForFooterInSection: (_ tableView: UITableView, _ section: Int) -> String?

  3. Does not compile.
    Consider the following example. Not only it introduces an extra nesting level and quite ugly, it doesn't even compile.

protocol P {
    var foo: ((_ int: Int, _ string: String) -> Void)? { mutating get }
}

extension P {
    var foo: ((_ int: Int, _ string: String) -> Void)? { nil }
}

struct S: P {
    var x = 1
    var foo: ((_ int: Int, _ string: String) -> Void)? {
        mutating get {
            { int, string in
                self.x = 1 // 🛑 Escaping closure captures mutating 'self' parameter
            }
        }
    }
}

Thoughts?
What is your view on this? Does Swift need optional protocol members going forward or could it do without?

Previous threads (8 years old).
If you know other threads on the matter please shout, and I'll update this list:

4 Likes
2

We just need the ability to give closures named parameters.

var foo(x:y:): ((Int, String) -> Void)? { mutating get }

Would solve the problem just fine. The only question is how much sugar we want for this.

2 Likes
3

Yes, but that will solve only bullet points #4 and #5 from the above list.

4

Yeah, point six is going to need some advanced ownership voodoo. That's probably the most compelling use-case for optional protocol requirements, although it'd be more compelling with an actual example instead of a placeholder.

5

I’ll note the recommended replacement for optional protocol methods today is a default implementation that returns nil. If your method already needed to return an optional value, either use a nested optional or don’t distinguish those cases. For properties this is all that the current optional handling even does (though it’s sometimes a problem).

You’re free to say this is insufficient; I just want to clarify that it’s recommended over a computed property that’s a function.

1 Like
6

i have shot countless feet doing this:

protocol P { var name:String? { get } }

extension P { var name:String? { nil } }

struct S:P { let name:String }
5 Likes
7

IMO we ought to warn on that kind of near-miss (or in a case where the subtyping conversion works like this, just allow name to be the witness), I agree it is quite insidious.

6 Likes
8

There's an idiom I've been playing with to encode conditional capabilities directly into protocol conformances, using uninhabited types. You can have a base protocol capture the type's conformance to a derived protocol in an associated type, which is either constrained to be equal to Self when the type conforms to the derived protocol, or defaults to an uninhabited type such as Never:


protocol Base {
  associatedtype AsDerived: Derived = Never

  func baseRequirement()
}

protocol Derived: Base where AsDerived == Self {
  func derivedRequirement()
}

extension Never: Derived {
  func baseRequirement() {}
  func derivedRequirement() {}
}

This allows for types that don't have an implementation of derivedRequirement not to provide one. Generic code that wants to require an implementation can do so by requiring T: Derived, but with only T: Base, we can check whether a type also conforms to Derived without paying for a full dynamic cast:

extension Base {
  var asDerived: Self.AsDerived? {
    if Self.self == AsDerived.self {
      return unsafeBitCast(self, to: AsDerived.self)
    } else {
      return nil
    }
  }
}

If you have many independently optional requirements, then I can see why you don't want to have a protocol for each. Within one protocol, you could still use a possibly-uninhabited associated type as a token to indicate whether each requirement is present:

protocol Base {
  func baseRequirement()

  associatedtype OptionToken = Never
  // If `OptionToken` is uninhabited, then this function can't be called
  func optionalRequirement(_: OptionToken)
}

// default implementation for the unreachable case
extension Base where OptionToken == Never { func optionalRequirement(_: Never) {} }

struct FulfillsOnlyBaseRequirement: Base {
  func baseRequirement() {}

  // doesn't need to implement optionalRequirement since it defaults to unreachable
}

struct FulfillsBothRequirements: Base {
  func baseRequirement() {}

  // implements the requirement for real, with a real inhabited type
  func optionalRequirement(_: Void) {}
}

And you could require or dynamically check where OptionToken == Void in code that could go either way to get access to the derived requirement. The associated type requirement also helps somewhat mitigate the "near miss" issue taylorswift raised above when using value-level Optional requirements, since the types won't line up if you expect the associated type to be inhabited but try to use the uninhabited default implementation. These approaches still require a lot of boilerplate in the language as it stands today, but could be looked at as a desugaring for a more fully-cooked optional requirement mechanism (or could possibly be used to implement a macro for an optional-like requirement).

10 Likes
9

Right, a warning alone doesn't help because there's no way to resolve the warning today.* We need to either allow the implicit conversion (a breaking change, but one that's probably worth doing based on swift-version, and one that would align with method overrides), and/or allow explicitly satisfying a requirement with a different name (implemented, but using an underscored attribute that never went through the Evolution process).

* this is technically untrue, you can bounce through a secondary protocol and a computed property, but it's brittle and verbose:

protocol P { var name:String? { get } }

extension P { var name:String? { nil } }

struct S:P { let name:String }

// ---

protocol NonOptionalNameBridge {
  var name: String { get }
}
extension NonOptionalNameBridge where Self: P {
  var name: String? { self.name as String }
}

extension S: NonOptionalNameBridge {}

print((S(name: "foo") as any P).name ?? "nil")
1 Like
10

Yeah I absolutely think the larger problem is worth solving too, but even without a new language feature the warning solves the problem when there’s no real reason why name is non-optional and it was typed that way only inadvertently, or because the protocol definition was refactored and the conforming type definition was missed, or because the user expected the subtype conversion to produce a valid witness. In the cases I’ve encountered in the wild where this sort of near miss was actually causing a bug the fix has almost always been “make the types match”.

We’d still need to have a way to silence the warning in the case where you really don’t want the near miss to be a witness, though.

1 Like