Introduction
The Swift language could benefit from allowing KeyPaths to represent funcs. The infrastructure for this is already supported by the existing implementation of KeyPaths; all non-final properties of classes, any property with a didSet observer, and all subscripts are essentially functions. The compiler simply does not yet have a way to transform a statement like \Foo.bar() into a KeyPath instance yet.
This pitch also includes a second half suggesting several potential paths to changing the KeyPath class in the standard library, the necessity of which will be demonstrated throughout the preceding sections. These could be incorporated into a future SE alongside this change, or a separate SE all their own. I'm hoping to get feedback on these from folks who understand the options available to design such an API, as the Swift language features available at the Standard Library level like class and protocol seem poorly suited to implementing such an API.
I have implemented a proof of concept of the first part of this change in the swift compiler and swift-syntax. I have to give the disclaimer that I am not happy with the quality of this code and accept that if this pitch ultimately turns into an accepted SE I will probably need to make significant revisions.
To give a full example, code like the following would be accepted by the compiler:
class S {
func f() {
print(“called f”)
}
}
let kp = \S.f()
print(S()[keyPath: kp]) // prints “called f”
More importantly, code like this would also be accepted:
@dynamicMemberLookup
class WeakReference<Referred> where Referred: AnyObject {
weak var referred: Referred?
subscript<T>(dynamicMember member: KeyPath<Referred, T>) -> T? {
referred?[keyPath: member]
}
}
let strongLocalRef = S()
let weakRef = WeakReference(strongLocalRef)
weakRef.foo()
Supporting code like let kp: KeyPath<S, () -> ()> = \S.f is not in scope for this pitch, nor is adding the ability for KeyPaths to represent throwing, async, or mutating functions, though avenues for supporting those features will be discussed in the second section of this pitch.
Motivation
“Prefer composition over inheritance” is a commonly accepted practice in modern software engineering. However, language support for composition is often lacking. Inheriting from a class gives users an easy way to take on all its behaviors, but adding an instance of another type as a member variable requires users to expose all the features of that type they want to use by hand. DynamicMemberLookup is a rare exception to this bias towards inheritance; not only does it allow users to expose features of other types as though they were declared on the “parent” type, but it also allows them to transform these APIs in interesting ways by returning a different type from the invocation of a property or subscript than in the original declaration.
However, the current design of KeyPaths limits the APIs that can be built using DynamicMemberLookup.
This pitch could be the first step in resolving this problem; by allowing KeyPaths to represent functions, users can begin to build true proxies for other types. Subsequent sections of this pitch explain my thinking around how to fully achieve this goal by making potentially breaking changes to KeyPath which allow users to represent functions with non-hashable arguments, async, and throwing functions with KeyPaths, as well as opening the door for future enhancements, such as opt-in Encodable conformance for individual KeyPaths that is visible to the type system.
This pitch also resolves a strange consequence of features added to KeyPaths and vars in recent years: func is arguably the least versatile member declaration available to Swift programmers today. A var or subscript can be substituted directly for an unapplied function by using its keypath, while KeyPaths themselves offer metadata such as equality and hashability. Vars can even throw or be async, and subscripts are allowed to be overloaded. The only feature funcs have that subscripts and vars don’t is ergonomics; we’d all like to write object.foo() instead of object[foo:()].
A final motivation for this pitch is to improve Swift’s ability to express messaging concepts to a level that surpasses Objective-C. Objective-C has long had support for forwarding through runtime features like -forwardInvocation: and -forwardingTargetForSelector:. Despite being far more expressive than Objective-C in many respects, Swift falls well short of it in this area; if this pitch was ultimately implemented, Swift would surpass Objective-C by exposing these features with the added type safety that existing Swift features provide.
Specific Examples of Designs that this feature will enable
- A
Mockable<T>type could forward every member function or variable to an underlying type, or to handlers in a unit test situation. Needing to wrap every type you want to mock at the usage sites, is a bad API, but this could be combined with the seemingly abandoned type wrapper pitch of a few months ago, to eliminate even the need for aMockablewrapper - A
WeakRef<T>type could ensure that an object was never stored with anything other than a strong reference within a particular scope - A
TransitionParticipant<T: UIView>type could record all properties and functions called on a UIView (not including blocklisted functions likeremoveFromSuperview()), then construct a replica of it to safely mutate as part of an animated transition to another screen where that view morphed into another view, allowing users to easily implement SwiftUI’sMatchedGeometryEffectin UIKit - An
actor Isolated<T>type could seamlessly transform any existing type into anactor
What does this feature provide that can't be done with Macros?
Macros can be a good implementation of many of the use cases above; in fact, they can be better for certain applications. A macro that generates metadata for each declaration in a type, or a pass-through to a backing object, might be far more performant than the dynamicMemberLookup based designs described above.
However, such a design is not comprehensive; it will only work on declarations decorated by the macro. This means that only types that the author of the program owns, that have no superclasses and are completely free of any extensions, can ever be fully covered by macro based solutions.
You can see an example of engineers correctly choosing the right trade-off between performance and comprehensiveness, correctness, and ease of use in SwiftUI; while model objects have transitioned to use Macros, because only their stored properties are relevant to updating a view hierarchy, Bindings remain dynamicMemberLookup based.
Of the specific examples above, only Mockable can be practically implemented with a Macro, and, again, only for declarations on the originally declaration of a type.
Design & Implementation
I’ve added a new type of keypath segment at the parsing, typechecking, and SIL generation levels of the compiler. This segment type encapsulates both getting a reference to the appropriate function and calling it. This seems like the most ergonomic design for users in the context of DynamicMemberLookup, I think the overwhelming majority of users would prefer to deal in the results of functions than in getting references to the functions themselves.
That said, an alternative implementation would add a type of segment: a "call" segment that contained all the arguments of the function.
Below the SIL level, no changes seem to be necessary; since so many properties and subscripts turn out to be functions anyway, everything this feature needs already seems to be implemented in my testing.
This design decision to combine the dot expression and call expression into one segment currently leads to some very odd code in the PoC, particularly in ExprRewriter; in several places, we need to manually tear apart types to convert them from KeyPath<Root, (Args) -> (Result)> to KeyPath<Root, Result>, and we need to return from visitDotExpr without having done any rewriting to “approach” the same expression from visitCallExpr, which gives us the arguments and the dot expression. This seems wrong to me, and I’d appreciate feedback on what the right approach is.
Role of the expanded feature set for KeyPath in the Swift language
A reasonable question in response to this pitch might be “what’s the point of function types like (Button) -> () -> () if KeyPath<Button, ()> also represents the same thing?”
KeyPath is distinct from pure function types, because it contains additional metadata about the members being represented. Function types should continue to be used in places where this metadata isn’t necessary, while KeyPath can be used in cases where it is necessary.
That said, in my opinion, the closest existing analog for KeyPath will be NSInvocation; they both package everything you need to call a function other than the callee, along with metadata allowing other systems to modify, record, or otherwise interact with those invocations.
Breaking changes to the KeyPath type hierarchy
Unfortunately, KeyPaths today have several restrictions that make it impossible to forward all functions, or even subscripts and properties, of a particular type. Unfortunately, addressing these limitations may be impossible in the current design of KeyPath’s class hierarchy without making breaking changes.
The most important one is the decision to make KeyPath Hashable. This means that all arguments must also be Hashable. This restriction is enforced today on KeyPaths representing subscript.
The decision to use a nominal type, specifically a class, means that every attribute of a var or function must be expressed in the type name. This will lead to an explosion of types as more features are added to KeyPaths.
For example, to remove all the restrictions currently on KeyPaths, they need support for:
mutatingthrowingasync
Supporting these would require the following types to be added to the Swift Standard Library:
MutatingKeyPath
MutatingPartialKeyPath
MutatingAnyKeyPath
ThrowingKeyPath
ThrowingPartialKeyPath
ThrowingAnyKeyPath
MutatingThrowingKeyPath
MutatingThrowingPartialKeyPath
MutatingThrowingAnyKeyPath
AsyncKeyPath
AsyncPartialKeyPath
AsyncAnyKeyPath
AsyncMutatingThrowingKeyPath
AsyncMutatingThrowingPartialKeyPath
AsyncMutatingThrowingAnyKeyPath
AsyncThrowingKeyPath
AsyncThrowingPartialKeyPath
AsyncThrowingAnyKeyPath
However, this is just a start. Here are a few more potential features that might be useful to add
- The ability to restrict the type of
KeyPaththat a function accepts to only be a property, subscript or function - Opt-in Encodable conformance
- Opt-in Sendable conformance (my understanding is that KeyPaths are currently supposed to always be Sendable, but that this has not yet been implemented)
- Opt-in Hashable conformance (meaning that KeyPaths can represent subscripts and functions with non-hashable arguments)
The last feature is critical to comprehensive forwarding becoming a reality, without it, only functions with all Hashable arguments can be forwarded. And while users could work around this restriction with a propertyWrapper for parameters that fakes a Hashable conformance, this is an ugly hack that we should not inflict on users.
Here, then, are a few options for resolving this problem. Frankly, I'm not particularly happy with any of them. I'm hoping that there's folks with experience working on the Swift type system who can suggest some realistic alternatives to explore.
Expose KeyPaths as existentials
Because KeyPath has already committed to being Hashable in the ABI, one option would be to not publicly expose these types, and instead, to express them as existentials, like the following: some (KeyPathProtocol<T, U> & NonThrowingKeyPath & NonAsyncKeyPath & Hashable).
The existing subscript that accepts KeyPath would remain for backwards compatibility purposes only. New subscripts would be added to accept every possible combination of properties.
The Swift compiler would create existential KeyPaths in contexts where they were available, and in other contexts would create the old KeyPath class type. Here’s a few concrete examples of what that would look like, assuming that this feature was added in Swift 6, and that Swift 6’s stdlib was shipped in iOS 18, and we were authoring a program for iOS devices:
- In a program with a minimum deployment target of iOS 18, existential based KeyPaths would always be created from an expression like
\Foo.barexcept if the type was explicitly annotated to be the old type, or type inference otherwise determined that that was the only solution to the constraints - In a program with a minimum deployment target of iOS 17, class based KeyPaths would be created unless explicitly specified. Inside of a
#available(iOS 18)scope, the default would be the new existential KeyPaths
Naturally, this would be source breaking, but would be a simple change to automatically migrate in most cases.
This solution has several drawbacks, the first is ergonomics. A KeyPath that doesn’t throw is actually a subtype of those that do throw, but expressing that means that the top-level KeyPathProtocol must be assumed to throw and be async, and writing a KeyPath that works as it does today looks like this: some (KeyPathProtocol<T, U> & NonThrowingKeyPath & SynchronousKeyPath & Hashable & NonMutatingKeyPath.
Existentials also are known to have poor performance at runtime; there isn’t much point in using efficient access methods if they’re buried by the overhead of the runtime calling functions on KeyPath from the runtime.
Also, the explosion of types would still exist in the standard library, because every combination of existentials would still need a matching implementation. One solution to prevent this might be to make the various protocols “Markers” like the current implementation of Sendable, whose only impact would be in type-checking which subscript was chosen to access the value in the KeyPath. This works well with the current implementation of KeyPath, where almost all the features are implemented on AnyKeyPath anyway, so there is no need for conformance records to be kept. I’m not certain if this path is actually available, but it seems like it may be worth exploring. It could also solve the issue of existential performance raised above.
Here's what the new APIs might roughly look like, assuming that we chose to support Async and Throws in future.
protocol AnyKeyPathProtocol {
// A substitute for Hashable conformances, this would be the equivalent UUID that's generated for KeyPaths, but without
// the addition of the parameters' hash values.
// There are cases where a user might want to store a KeyPath in a Dictionary, but not actually care about what parameters
// it was called with; this can be used as a key.
var uniqueIdentifier: Int { get }
}
protocol PartialKeyPathProtocol<Root>: AnyKeyPathProtocol {
associatedType Root
}
protocol KeyPathProtocol<Root, Leaf>: PartialKeyPathProtocol<Leaf> {
associatedType Leaf
}
protocol HashableKeyPath: Hashable {
}
protocol NonAsyncKeyPath { // marker
}
protocol NonThrowingKeyPath { // marker
}
extension Any { // this is effectively how the KeyPath subscript works today, though I don't think it's actually declared anywhere.
// base subscripts
subscript (keyPath: some AnyKeyPathProtocol) throws async -> Any { get }
subscript <T>(keyPath: some PartialKeyPathProtocol<T>) throws async -> T { get }
subscript <T>(keyPath: some PartialKeyPathProtocol<T>) throws async -> T { get }
// non-throwing
subscript (keyPath: some AnyKeyPathProtocol & NonThrowingKeyPath) async -> Any { get }
subscript <T>(keyPath: some PartialKeyPathProtocl<T> & NonThrowingKeyPath) throws async -> T { get }
subscript <T>(keyPath: some KeyPathProtocol<Self, T> & NonThrowingKeyPath) throws async -> T { get }
// non-async
subscript (keyPath: some AnyKeyPathProtocol & NonAsyncKeyPath) throws -> Any { get }
subscript <T>(keyPath: some PartialKeyPathProtocol<T> & NonAsyncKeyPath) throws -> T { get }
subscript <T>(keyPath: some KeyPathProtocol<Self, T> & NonAsyncKeyPath) throws -> T { get }
// non-async & non-throwing
subscript (keyPath: some AnyKeyPathProtocol & NonThrowingKeyPath & NonAsyncKeyPath) -> Any { get }
subscript <T>(keyPath: some PartialKeyPathProtcol<T> & NonThrowingKeyPath & NonAsyncKeyPath) -> T { get }
subscript <T>(keyPath: some KeyPathProtocol<Self, T> & NonThrowingKeyPath & NonAsyncKeyPath) -> T { get }
}
We don't need any extra subscripts for Hashable or Encodable conformance, those are just for usage by consumers.
One minor syntactic improvement might be to spell the Non family of protocols using the ~ from non-copyable types, if that option is available.
Option 2: make KeyPath more like func
The most superficial implementation of this idea is purely syntactic, we might add some special sugar for KeyPaths like \T throws -> U instead of the nominal type name. PartialKeyPath would be written as \Any -> T, and AnyKeyPath would become \Any -> Any.
This could allow us to decouple the capabilities of the KeyPath from implementation choices like a class hierarchy versus a collection of protocols. It is also clearly the correct way to model the capabilities of a type like KeyPath, which is more akin to a function than any nominal type.
Of course, unless we retain some nominal KeyPath types, this could end up being a much more significant source breaking change than a new spelling. Existing extensions on KeyPath types will break, and while there isn’t much reason to write such an extension today, it’s quite possible that this would be a major inconvenience for many codebases.
I'm far less comfortable with the implementation details of this option than the first, particularly in how it would affect the implementation of type checking in Swift, I'd hope that a lot of the code that already exists for functions could be reused, but I suspect that there would end up being a lot of very odd special cases for KeyPaths in the type checker, which seems like an outcome to be avoided, if possible.
Option 3: a purely additive change, perhaps not using KeyPath
We could also abandon the premise of this pitch and create a new way of referring to invocations.
Such an approach would have to fulfill the following design goals:
- Provide a comprehensive solution for mocking
- Provide a universal way of referring to property accesses and function calls
- Provide metadata for the contents of those calls
- Provide a way to make statements about the whole invocation, for example, "this invocation is hashable," "this invocation is encodable"
- Be compatible with
DynamicMemberLookup, or otherwise provide a mechanism for forwarding/transforming invocations - Ideally, avoid the type-explosiion
I briefly considered this, however it seems very challenging to do, and it would likely render KeyPath wholly unnecessary, leading to source-breaking changes or confusion from users regardless.
Future directions
Hopefully the first part of this pitch was convincing, and a consensus can be reached on the best path forward to improving KeyPath is. If an implementation of this pitch ultimately lands, there will be more work to do:
- A logical next step will be to support all flavors of function, from
throwstoasync - The expanded design of KeyPath should open the door to different conformances for
KeyPathto be implemented with compiler support, such asEncodable - If enhanced reflection APIs moves beyond the pitch stage, it might also be useful to allow users to define conformances conditional on the arguments to all functions/subscripts in a
KeyPath
