SE-0216: User-defined dynamically callable types

Or should "_" be the way to represent no label?

I assume you're speaking in terms of the current design, where argument labels have a non-optional type (e.g. String).
In this design, I strongly feel that empty string should be used.

It's more obvious that "" indicates no label. One can simply check label.isEmpty rather than label == "_".
It's theoretically possible for some dynamic language to support _ as an argument label, so I believe "" is more sound.

3 Likes

Can you use placeholders for the parameter and return types?

func dynamicallyCall(
  withArguments arguments: <#T##ExpressibleByArrayLiteral#>
) -> <#T#> {
  <#code#>
}

func dynamicallyCall(
  withKeywordArguments arguments: <#T##ExpressibleByDictionaryLiteral#>
) -> <#T#> {
  <#code#>
}

ASTPrinter has swift::getCodePlaceholder() for the <#code#> placeholder. I'm not sure about the other placeholders.

1 Like

Yep, I believe so. AFAIK, the fix-it API just prints arbitrary strings (including placeholders like <#type#>) and Xcode is responsible for rendering the placeholders.

Edit: I like your templates, by the way!

Good idea. +1

Demo
import Foundation
import JavaScriptCore

//@dynamicCallable
@dynamicMemberLookup
public protocol JSDynamicValue: AnyObject {}

extension JSDynamicValue where Self: JSValue {

  @discardableResult
  func call(
    dynamicMember name: String,
    withArguments args: [Any]
  ) -> JSValue {
    if name.isEmpty {
      return call(withArguments: args)
    } else if name == "init" {
      return construct(withArguments: args)
    } else {
      return invokeMethod(name, withArguments: args)
    }
  }

  public subscript(dynamicMember name: String) -> JSValue {
    get {
      return objectForKeyedSubscript(name as NSString)
    }
    set {
      setObject(newValue, forKeyedSubscript: name as NSString)
    }
  }
}

extension JSValue: JSDynamicValue {}

//@dynamicCallable
@dynamicMemberLookup
public protocol JSDynamicContext: AnyObject {}

extension JSDynamicContext where Self: JSContext {

  @discardableResult
  func call(
    dynamicMember name: String,
    withArguments args: [Any]
  ) -> JSValue {
    return globalObject!.call(dynamicMember: name, withArguments: args)
  }

  public subscript(dynamicMember name: String) -> JSValue {
    get {
      return globalObject![dynamicMember: name]
    }
    set {
      globalObject![dynamicMember: name] = newValue
    }
  }
}

extension JSContext: JSDynamicContext {}

let context = JSContext()!

#if false
context.parseInt("0x2A", 16).toInt32()
#else
context.call(dynamicMember: "parseInt", withArguments: ["0x2A", 16]).toInt32()
#endif

#if true
let parseInt = context.parseInt
#else
let parseInt = context[dynamicMember: "parseInt"]
#endif

#if false
parseInt("0x2A", 16).toInt32()
#else
parseInt.call(dynamicMember: "", withArguments: ["0x2A", 16]).toInt32()
#endif

My concern here is that semantically, it doesn't quite make sense that "calling a method with no name" actually becomes "call self".

Also, what about Python interop, where dynamicallyCallMethod isn't necessary?

FYI, this is an easy policy decision to change, but this was discussed extensively in the @dynamicMemberLookup review cycle. I am not opposed to considering opening these up more, but I pretty strongly feel that that should be an independent review cycle because it should apply to dynamicMemberLookup too.

-Chris

2 Likes

I managed to retroactively add @dynamicMemberLookup in the demo, but maybe that shouldn't be allowed?

As John said up-thread, it would be preferable to move open design discussions to another thread so this thread can remain a proposal review thread :slight_smile:

-Chris

2 Likes

Imagine if "regular callable" types simply had a method with no name, similar to how subscripts are declared.

enum BinaryOperation<T: Numeric> {
  case add, subtract, multiply

  // BinaryOperation is "regular callable".
  func(lhs x: T, rhs y: T) -> T { ... }

  // BinaryOperation is "subscriptable".
  subscript(lhs x: T, rhs y: T) -> T { ... }
}

let multiply: BinaryOperation<Int> = .multiply

// Call or subscript, with argument labels.
multiply(lhs: 6, rhs: 7)
multiply[lhs: 6, rhs: 7]

There would only be two possible templates, for all dynamic languages.

  • func call(dynamicMember:withArguments:)
  • func call(dynamicMember:withKeywordArguments:)

Python could implement them as follows.

public struct ThrowingPythonObject {

  @discardableResult
  func call(
    dynamicMember name: String,
    withArguments args: [PythonConvertible]
  ) throws -> PythonObject {
    guard name.isEmpty else {
      return try self[dynamicMember: name]
        .call(dynamicMember: "", withArguments: args)
    }
    // Paste the `dynamicallyCall(withArguments:)` body here.
  }

  @discardableResult
  func call(
    dynamicMember name: String,
    withKeywordArguments args: KeyValuePairs<String, PythonConvertible>
  ) throws -> PythonObject {
    guard name.isEmpty else {
      return try self[dynamicMember: name]
        .call(dynamicMember: "", withKeywordArguments: args)
    }
    // Paste the `dynamicallyCall(withKeywordArguments:)` body here.
  }
}
1 Like

I was thinking something like this:

func _(...) -> ... {...}
2 Likes

Why not just call? Regardless of the name, I strongly support a reworking of this proposal on top of “regular callable” if that is at all viable.

2 Likes

A non-dynamically callable type would presumably use Swift's ordinary, static argument-passing rules, i.e. you'd declare the call operation something like this:

func __call(labels: Int, are: Int, statically: Bool, checked: Bool) -> Int

And calls to that would be checked exactly as if you'd written

foo.__call(labels: 0, are: 1, statically: false, checked: true)

instead of

foo(labels: 0, are: 1, statically: false, checked: true)

That is, the features seem basically different.

2 Likes

For the record, I don't think treating dynamic freestanding calls as a dynamic special case of dynamic member calls is actually a good idea; it was just an example. Doing this would force clients looking to just support dynamic freestanding calls to also support this other construction which may not have a real analogue in what they're trying to allow.

I'll strongly second this. This syntax:

foo(x,y,z)

…and this one:

foo.bar(x, y, z)

…have a very different flavor, and carry different intents. It is not at all given that some particular foo will want to support both syntaxes — and unlike “missing name” and “wrong arguments” errors, that's something the compiler can actually catch.

Using a dummy method name to fold both syntaxes into a single construct prevents the compiler from telling us that freestanding calls are an error for things that only support dynamic member lookup, and vice versa. It unnecessarily turns a compile-time error into a runtime error.

For example, if this is legal code bridged to Ruby:

User.find(id)

…then this necessarily has to compile under the proposed merging of method dispatch and freestanding calls:

User.find(id)(7)("fish")

…even though there is no such thing as a freestanding call in Ruby and it would thus never make sense.

I’m usually the one in the “fewer features with more generality” camp, but in this case keeping these features separate still seems like the right thing to do.

For all this back and forth, I’m not convinced there’s any clear problem with this proposal as it stands. Still listening, but I don’t see it … at least not yet.

7 Likes

I suppose that's true. It might take more spelunking to construct the call, but the information will still be available if you take the time to find it.

I'll withdraw the argument that it preserves flexibility for the AST. However, I still think it's a more parsimonious way to expose dynamic lookup in Swift. It means that dynamic lookup of both properties and methods is only concerned with a single Swift feature—declaration references—which ought to give us a simpler Sema and a cleaner mental model.

I'm not sure why this check is necessary. Suppose there is a setter on the subscript—what harm does that cause? There's no way to write an lvalue which will be transformed into a reference to the setter that includes argument labels. This is more important with proposals where the subscript performs the call, but when it merely looks up a callable object and returns it—and especially when many types using the feature will be handling both properties and methods through the same subscript—I don't think it really matters.

Now, let's talk about these three objections together:

I think you may be imagining that users will often call these interfaces directly. I do not imagine that users would ever write a direct call to either dynamicCall(…) or subscript(dynamicMember:argumentLabels:). Using these APIs directly will never be as nice as using purpose-built, language-specific APIs made by the bridge designer.

In particular, the "direct calls" you mention are not something that native Swift has any support for at all—you can't even splat an array into a variadic parameter today. But a Python bridge could mimic Python's syntax. Here's a rough pseudocode cut at some bridging code which would let you do direct calls with Python-style splatting:

let randint = np.random.randint
randint(-10, 10, **["dtype": np.float])

The Python bridge could also handle this by exposing a more "raw" API, like a call(positional:keywords:) method. Or you could add a special case to your subscript(dynamicMember:argumentLabels:) method so that you could write, say, randint._call(-10, 10, dtype: np.float). Depending on how far you want to go, there are plenty of options which get you pretty close to randint(-10, 10, dtype: np.float).

My point is, I don't think we should design SE-0216 and its ilk to make desugared code pretty so that it can be directly called. I think we should design these features to cover as many use cases with as little surface area as we can.


I'm going to talk a little bit about Ruby now, but before I do that:

I apologize that so much of the discussion has turned into designing other features, but I think this is unfortunately necessary, because one of the cores of my argument is that there are better designs than this one. I'll try to avoid going too deep into designing those alternatives, but I can't really make the case against this design without discussing other options at least a little.


I did a little bit of experimenting, and it turns out that a Ruby bridge would only need to extend dynamicMember: with a variadic argumentLabels: list—it wouldn't need a callable feature at all. Consider a bridge like this:

@dynamicMemberLookup
struct RbObject {
    ...
    // Call this the "property subscript"
    subscript(dynamicMember name: String) -> RbObject {
        get {...}
        set {...}
    }
    // Call this the "method subscript"
    subscript(dynamicMember name: String, argumentLabels labels: String...) -> (RbConvertible...) -> RbObject {
        get {...}
    }
}

The compiler prefers to use the property subscript, apparently because it disfavors overloads with variadic parameter lists, but it will readily switch to the method-lookup subscript if you use argument labels or you directly call the result (or otherwise indicate through context that you want a closure). In a playground, these expressions all select the appropriate overload:

base[dynamicMember: "hello"]                          // uses property subscript
base[dynamicMember: "hello", argumentLabels: "world"] // uses method subscript
base[dynamicMember: "hello"]()                        // uses method subscript

I suspect the third case partly works because RbObject is not directly callable. But Ruby function objects aren't directly callable either—you have to use a method or a subscript to call a function/closure assigned to a variable—so that isn't something a Ruby developer would expect anyway.


So the subscript(dynamicMember:argumentLabels:) design would handle Ruby methods (and other languages with the same method/property ambiguity problem, like Perl) and Smalltalk-style methods (which can't be looked up without argument labels). If you add a static-callable feature, it would then handle Python methods too (and other languages which return callable method objects, like Javascript).

It still wouldn't support keyword arguments on freestanding Python functions, but again, that's not something we support on Swift closures, and you can get some decent syntaxes which go most of the way.


In summary: In my opinion, the reasons to prefer the SE-0216 design to labeled dynamic members + regular callable are:

  1. Desugared code looks nicer, and hand-written uses aren't terrible.
  2. Call sequences are more pleasant for the bridge implementor.
  3. It supports labeled parameters on bridged closures, which are not supported on native closures, but are natural in many foreign languages where labeled parameters are handled differently.
  4. It has already been designed and implemented. (This is, to be fair, not a small advantage.)

The reasons to prefer the opposite are:

  1. Both of these features serve additional use cases. Labeled dynamic members cover the semantics needed to support languages SE-0216 does not; regular callable supports non-bridging use cases.
  2. Both of these features override very specific Swift semantics. They affect smaller and simpler sub-trees within the AST, and I suspect their implementations would be smaller.
  3. They do not include an arbitrary set of variants to support slightly different use cases, the way this proposal has both dynamicCall(withArguments:) and dynamicCall(withKeywordArguments:), with the former mostly serving to avoid the clumsiness of the latter.
  4. To the extent possible, they make sure bridged types have the same semantics as native types.

I find the case for a different solution more compelling, but the core team might feel differently.

2 Likes

@beccadax FWIW, I agree that supporting direct "desugared" calls nicely is not a strong goal.

I think that the future directions proposal covers the Ruby use case. This was extensively discussed in the pitch phase, and I believe that it is a good plan. It is completely compatible with this proposal as written, it is just split out because there is no client for it at this point.

I personally want the best thing for Swift, I'm not too tied to the design we have above. That said, I don't really understand your counterproposal here. Maybe I'm missing it, but I don't see a description of the proposed design anywhere. Can you please split it out to a gist so we can understand what you're suggesting? If it is a better design, then I'm all for it. Please keep in mind that we need to support direct calls fp(x) not just method calls like obj.method(x), and that SE-0216 has a future direction specifically designed to solve the Ruby/Squeak/etc requirements.

-Chris

1 Like

+1 but I feel that DynamicMemberCallable should've not been dropped from this iteration. I personally do not see why this would have to be split up. Is there not going to be a Swift 4.3? Do we need to get this into Swift 4.2? The pitch phase focused on both features so I would assume that they should also be revived together. On the other hand, as long as we are still getting the DynamicMemberCallable then its all good.

Yes. Dynamic languages are very important.

Yes.

none

Participated in the original discussion.

Here you go. If there are still some points that are unclear, please let me know.

The design does support direct calls, but with the caveat that direct calls can't have argument labels. (If they were Swift closures, they would not be able to have argument labels, either.)

I appreciate that, but if we can cover both use cases adequately with the same design, I think that would be better than having two very similar but separate features.

1 Like

Brent, unless I'm missing something, your proposal doesn’t handle Ruby properties. I may be missing it.

In Ruby, everything is a method call, including property accesses; the parens are optional. This this:

post.author.name.length

…is equivalent to this:

post.author().name().length()  # unidiomatic but also correct

Any reasonable Ruby bridge needs to support both these syntaxes. (I explained this at length on another thread long ago; if there's doubt about it, I can dig that back up.) The existing family of proposals distinguishes does handle this.

It seems, however, that the @dynamicMemberLookup you propose doesn’t know whether the thing it returns is about to be called like a function or used like a value, and thus can only support one of the two options above. Am I missing something?