Hi all,
I ran into some blockers implementing SE-0216 for user-defined dynamic "callable" types and need help.
- Sorry for the very long post, click here to jump to a description of the blockers. The key points are in bold.
- Here is the WIP PR.
Introduction
In this proposal, types marked with @dynamicCallable
must implement one of the following methods:
func dynamicallyCall(withArguments: [T1]) -> T2
func dynamicallyCall(withKeywordArguments: T3) -> T4
// `T1` can be any type that conforms to `ExpressibleByArrayLiteral`.
// `T3` can be any type that conforms to `ExpressibleByDictionaryLiteral` where
// `T3.Key` conforms to `ExpressibleByStringLiteral`.
Such types then become callable:
@dynamicCallable
struct Callable {
func dynamicallyCall(withArguments: [Int]) -> Int
func dynamicallyCall(withKeywordArguments: KeyValuePairs<String, Int>) -> Int
}
func foo(x: Callable) {
x(1, 2, 3) // sugar for `x.dynamicallyCall(withArguments: [1, 2, 3])`
x(a: 1, 2, b: 3) // sugar for `x.dynamicallyCall(withKeywordArguments: ["a": 1, "": 2, "b": 3])`
}
As proposed, @dynamicCallable
can be applied to any nominal type: structs, classes, enums, and protocols.
I believe @dynamicCallable
method resolution should behave just like protocol requirement resolution: it should be possible to have overloaded dynamicallyCall
methods and dynamic calls should behave exactly like direct calls (e.g. x(a: 1, 2)
should resolve in the exact same way as x.dynamicallyCall(withKeywordArguments: ["a": 1, "": 2])
.
Implementation approach
Originally, I tried to implement @dynamicCallable
using a lookup-based approach outside of the constraint system. This was problematic for many reasons, the biggest one was that overloaded method resolution didn't work well.
I've now moved on to a constraint system based approach for resolving dynamic calls. The general algorithm is this:
- In
CSSimplify.cpp
, check if the "called" nominal type has or somehow "inherits" the@dynamicCallable
attribute. Then, look up and cache alldynamicallyCall(withArguments:)
anddynamicallyCall(withKeywordArguments:)
methods within the nominal type. - Then, attempt to resolve the correct method by creating a disjunction constraint over all candidate methods (this is done via
CS.addOverloadSet(tv, candidates, ...)
wheretv
is a type variable). I then add various constraints ontv
.
Let's work through an example program:
@dynamicCallable
struct Callable {
func dynamicallyCall(withArguments: [Int]) {}
func dynamicallyCall(withArguments: [Float]) {}
}
func test(x: Callable) {
x(1, 2) // We're handling this in `simplifyApplicableFnConstraint` in CSSimplify now.
}
Here are the type variables:
Type Variables:
# The constraint system created the first 5 type variables.
$T0 [lvalue allowed] as Callable @ locator@0x7ff6ac037800 [DeclRef@test.swift:7:3]
$T1 fully_bound literal=3 involves_type_vars bindings={(subtypes of) (default from ExpressibleByIntegerLiteral) Int} @ locator@0x7ff6ac037880 [IntegerLiteral@test.swift:7:5]
$T2 fully_bound literal=3 involves_type_vars bindings={(subtypes of) (default from ExpressibleByIntegerLiteral) Int} @ locator@0x7ff6ac037920 [IntegerLiteral@test.swift:7:8]
$T3 fully_bound subtype_of_existential involves_type_vars bindings={} @ locator@0x7ff6ac037a08 [Call@test.swift:7:3 -> function result]
$T4 [lvalue allowed] subtype_of_existential involves_type_vars bindings={} @ locator@0x7ff6ac037aa0 [Call@test.swift:7:3 -> apply function]
# I create $T5 to represent the type of the argument to the
# `dynamicallyCall(withArguments:)` method.
$T5 fully_bound subtype_of_existential involves_type_vars bindings={} @ locator@0x7ff6ac037aa0 [Call@test.swift:7:3 -> apply function] # `apply function` is probably wrong here
And here are the constraints:
Inactive Constraints:
# These first two constraints were generated by the constraint system.
$T1 literal conforms to ExpressibleByIntegerLiteral [[locator@0x7ff6ac037880 [IntegerLiteral@test.swift:7:5]]];
$T2 literal conforms to ExpressibleByIntegerLiteral [[locator@0x7ff6ac037920 [IntegerLiteral@test.swift:7:8]]];
# The rest below are mine.
disjunction [[locator@0x7ff6ac037aa0 [Call@test.swift:7:3 -> apply function]]]:$T4 bound to decl test.(file).Callable.dynamicallyCall(withArguments:)@test.swift:3:8 : (Callable) -> ([Int]) -> () at test.swift:3:8 [[locator@0x7ff6ac037aa0 [Call@test.swift:7:3 -> apply function]]]; or $T4 bound to decl test.(file).Callable.dynamicallyCall(withArguments:)@test.swift:4:8 : (Callable) -> ([Float]) -> () at test.swift:4:8 [[locator@0x7ff6ac037aa0 [Call@test.swift:7:3 -> apply function]]];
$T5 conforms to ExpressibleByArrayLiteral [[locator@0x7ff6ac037aa0 [Call@test.swift:7:3 -> apply function]]];
$T5.ArrayLiteralElement can default to Any [[locator@0x7ff6ac037aa0 [Call@test.swift:7:3 -> apply function]]];
$T1 arg conv $T5.ArrayLiteralElement [[locator@0x7ff6ac037d48 [Call@test.swift:7:3 -> apply function -> tuple element #0]]];
$T2 arg conv $T5.ArrayLiteralElement [[locator@0x7ff6ac037dd8 [Call@test.swift:7:3 -> apply function -> tuple element #1]]];
($T5) -> $T3 applicable fn $T4 [[locator@0x7ff6ac037aa0 [Call@test.swift:7:3 -> apply function]]];
Resolved overloads:
selected overload set choice x: $T0 == Callable
The disjunction constraint binds $T4
to each of the candidate dynamicallyCall
methods.
I create $T5
representing the type of the argument to $T5
. I add conversion constraints from $T1
and $T2
to $T5.ArrayLiteralElement
.
I add an applicable function constraint ($T5) -> $T3 applicable fn $T4
.
These constraints seem fine to me, please reply if anything looks wrong at this point.
- The hard work is done. If the disjunction constraint is unsolved, a diagnostic will naturally be emitted. If it is solved, access the resolved
dynamicallyCall
method inCSApply.cpp
and do expression rewriting.
Implementation blockers
Dynamic calls aren't an exact sugar
A tricky detail about dynamic calls are that they're not an exact syntactic sugar for direct calls to dynamicallyCall
.
Dynamic calls like x(1, 2)
have multiple arguments in the AST (and thus multiple type variables for each argument) while direct calls only have one array-like argument: x.dynamicallyCall(withArguments: [1, 2])
.
This causes simplification of the ($T5) -> $T3 applicable fn $T4
constraint to fail, because it's expected that the number of arguments is consistent:
Assertion failed: (params.size() == labels.size()), function relabelParams, file /Users/dan/swift-build/swift/lib/AST/ASTContext.cpp, line 3623.
Stack dump:
...
3. While type-checking expression at [test.swift:7:3 - line:7:9] RangeText="x(1, 2"
...
swift::AnyFunctionType::relabelParams(llvm::MutableArrayRef<swift::AnyFunctionType::Param>, llvm::ArrayRef<swift::Identifier>) + 119
swift::constraints::matchCallArguments(swift::constraints::ConstraintSystem&, bool, llvm::ArrayRef<swift::AnyFunctionType::Param>, llvm::ArrayRef<swift::AnyFunctionType::Param>, swift::constraints::ConstraintLocatorBuilder) + 1616
swift::constraints::ConstraintSystem::simplifyApplicableFnConstraint(swift::Type, swift::Type, swift::OptionSet<swift::constraints::ConstraintSystem::TypeMatchFlags, unsigned int>, swift::constraints::ConstraintLocatorBuilder) + 1574
swift::constraints::ConstraintSystem::simplifyConstraint(swift::constraints::Constraint const&) + 1150
swift::constraints::ConstraintSystem::simplify(bool) + 210
swift::constraints::ConstraintSystem::solveRec(llvm::SmallVectorImpl<swift::constraints::Solution>&) + 83
swift::constraints::DisjunctionChoice::solve(llvm::SmallVectorImpl<swift::constraints::Solution>&) + 113
swift::constraints::ConstraintSystem::solveForDisjunctionChoices(swift::constraints::Disjunction&, llvm::SmallVectorImpl<swift::constraints::Solution>&) + 1338
I need help solving this problem.
An idea that probably won't work is adding a flag to matchCallArguments
to specifically handle @dynamicCallable
calls. However, I see no way to actually set that flag because matchCallArguments
is not being invoked directly. This is super ad-hoc anyways.
An idea that may work is adding a new DynamicCallableApplicableFn
constraint, and using that instead: ($T5) -> $T3 dynamic callable applicable fn $T4
. With a custom constraint, I can do whatever simplification logic I want, including calling matchCallArguments
with a flag. Is it reasonable to add a new constraint kind just for @dynamicCallable
? One unknown is how to detect whether the argument type ($T5 here
) has a constraint conforming it to ExpressibleByArrayLiteral
or ExpressibleByDictionaryLiteral
. If detection isn't possible, then two variants of the DynamicCallableApplicableFn
constraint may be necessary.
Label-less dynamic calls can desugar to either withArguments
or withKeywordArguments
method calls
Consider the following program:
@dynamicCallable
struct S {
func dynamicallyCall(withArguments: [String]) -> Int { return 1 }
func dynamicallyCall(withKeywordArguments: [String : String]) -> Float { return 3.14 }
}
let s = S()
let _: Float = s("hi")
// Should resolve to `s.dynamicallyCall(withKeywordArguments: ["": "hi"]
With the current proposal, the above program is valid. This means, when resolving label-less dynamic calls, I need to create disjunction constraints over both withArguments
and withKeywordArguments
methods.
However, the set of constraints associated with the withArguments
disjunction constraint and withKeywordArgument
disjunction constraint are different. I don't see how to reconcile this. Conceptually, I need something like a union constraint: (disjunctionOverArgsMethods($T1) /\ isValidArgsMethod($T1)) \/ (disjunctionOverKwargsMethods($T1) /\ isValidKwargsMethod($T1))
. I'm not sure it's possible to express this using constraints.
However, this amendment to SE-0216 makes the problem go away. If @dynamicCallable
types can only define either the withArguments
method or the withKeywordArguments
one, then I never need to check both sets of constraints. I strongly support the amendment (rationale here) and it would simplify the implementation effort if accepted.