I’d like to formally propose the inclusion of user-defined dynamic member lookup types.
Here is my latest draft of the proposal:
https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438
An implementation of this design is available here:
The implementation is straight-forward and (IMO) non-invasive in the compiler.
-Chris
I like the proposal. Having a “magic” protocol isn’t ideal in some ways, but it seems like the best option for the moment, considering the other restrictions.
The name “DynamicMemberLookupProtocol” is a bit unwieldy, though. What about something like “SupportsDynamicMemberLookup“ or "SupportsDynamicMembers"? I know it doesn’t follow the usual protocol naming pattern, but for an empty “magic” protocol like this, it reads fairly well as a description of the capabilities of the type:
enum JSON: SupportsDynamicMemberLookup {
// ...
}
Anyway, I like the proposal. The “Supports-“ prefix on the name may be a good idea, or may not. Either way, the “JSON” example in the proposal shows how this is useful for more than just bridging to dynamic languages, and is quite compelling to me.
-BJ
On Nov 26, 2017, at 11:04 PM, Chris Lattner via swift-evolution <swift-evolution@swift.org> wrote:
I’d like to formally propose the inclusion of user-defined dynamic member lookup types.
Here is my latest draft of the proposal:
https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438
DynamicMemberLookup proposal by lattner · Pull Request #768 · apple/swift-evolution · GitHubAn implementation of this design is available here:
Implement the DynamicLookupProtocol proposal. by lattner · Pull Request #13076 · apple/swift · GitHubThe implementation is straight-forward and (IMO) non-invasive in the compiler.
-Chris
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution
I like the proposal. I think it’s simple and straight forward and provides some really nice ergonomics for working with dynamic data.
I appreciate the option to change the names too.
If I understand it correctly, you could also call “add_trick” via:
dog.addTrick("Roll over");
And simply update the subscript implementation to provide additional name look-ups. Cool!
And I like that the DynamicCallable provides the next level that I want to make the API more “Swift-like” and make use of named parameters:
dog.add(trick: "Roll over");
Or maybe more appropriate:
dog.addTrick("Roll over”, favorite: true);
I didn’t look at the implementation details, but I think the overall approach. I’d like it better if we have provisions to create these types of tools outside of the compiler, but that’s another topic.
-David
On Nov 26, 2017, at 10:04 PM, Chris Lattner via swift-evolution <swift-evolution@swift.org> wrote:
I’d like to formally propose the inclusion of user-defined dynamic member lookup types.
Here is my latest draft of the proposal:
https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438
DynamicMemberLookup proposal by lattner · Pull Request #768 · apple/swift-evolution · GitHubAn implementation of this design is available here:
Implement the DynamicLookupProtocol proposal. by lattner · Pull Request #13076 · apple/swift · GitHubThe implementation is straight-forward and (IMO) non-invasive in the compiler.
-Chris
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution
The implementation is straight-forward and (IMO) non-invasive in the compiler.
At least it’s smaller than I would have expected — but I don’t have the overview to judge the consequences to the compiler.
However, the impact those two proposals have on Swift is imho huge, and as the whole story is quite special for several reasons, there’s an elephant in the room that shouldn’t be ignored:
- What would you do if the proposal isn’t accepted?
- What does core think you would do in this case?
- What impact will that speculation have on the decision?
I can’t see what’s happening behind the scenes (basically, everything beyond the ML), but those non-technical questions might have a big impact if people construct their own narratives.
Besides that general concern, I think the whole „dynamic Swift“ topic shouldn’t be rushed:
How about keeping it in a public branch, so that users of script languages (not only Python) can play with it, and use that experience to decide the question in the Swift 6 timeframe?
Hi Chris,
Some questions:
How does the subscript approach work for dynamically passing in multiple parameters? For example, what would “dog.addTrick(“Roll Over”, favorite: true)” be when expressed as a subscript method? The subscript syntax only really seems to suit property-style invocations, unless I’m missing something obvious.
It seems like “dog.add_trick(…)” would have a dynamic lookup name of “add_trick”. What would the lookup name of “dog.add(trick: …)” be? Or in other words, how do named parameters affect the lookup name?
Can I implement the subscript method multiple times, or is only a single implementation allowed? For example, in the JSON example, there’s currently only a dynamic member subscript implementation that returns “JSON?”. Could I add another one that returns “String?” and have the type checker choose the appropriate one based on type inference?
As far as naming goes, I’d propose “DynamicMemberForwarding”. In my mind, “Lookup” and “Resolve” are inappropriate, because you’re not actually “looking up” or “resolving” to a particular implementation. In Objective-C, the “+resolveInstanceMethod” stuff expects you to *provide* a method implementation (an IMP). On the other hand, if there isn’t a method implementation (which is the case here), you end up in the “-forwardInvocation:” path, where you can just introspect the NSInvocation and do whatever it is you’re doing to do. So, it seems like this proposal sounds a lot more like ObjC's “invocation forwarding” rather than the method resolution; hence my preference for using “Forwarding” in the name for naming consistency.
Dave
On Nov 26, 2017, at 11:04 PM, Chris Lattner via swift-evolution <swift-evolution@swift.org> wrote:
I’d like to formally propose the inclusion of user-defined dynamic member lookup types.
Here is my latest draft of the proposal:
https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438
DynamicMemberLookup proposal by lattner · Pull Request #768 · apple/swift-evolution · GitHubAn implementation of this design is available here:
Implement the DynamicLookupProtocol proposal. by lattner · Pull Request #13076 · apple/swift · GitHubThe implementation is straight-forward and (IMO) non-invasive in the compiler.
-Chris
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution
I tuned out the initial discussions of this proposal because there seemed
to be a lot of noise centered around implementation/maintainability. I'm
curious if the actual premise of the syntactic/sugar conversion has been
discussed/debated yet? i.e. making dynamic/stringly calls look like normal
calls is very clean, but it's also very misleading (by definition; they're
not normal/safe/checked calls) with a potential net reduction in ergonomics.
Cleanliness is a double edged sword, which Swift already suffers at the
hand of. Type inference and overloading are fantastic. Literal convertibles
are magic. Omitting "redundant" types/descriptors from signatures is so
right. But they (especially when combined) lead to a lot of Swift code
ranging from unclear to meaningless at a glance, with a couple
option-clicks per line necessary to extract exactly what's going on from
the all knowing compiler. This is a hindrance in Xcode, and not possible if
you've only got the text (like on GitHub).
But whereas these features only really effect code comprehension, the
proposed syntactic conversion effects code comprehension and writing safe
code. Usually in Swift you have to opt in at the call site to unchecked
behaviour, but with these changes, working in a mixed pure Swift/Pythonic
Swift environment, if you don't pay a good deal of attention to each object
you're writing calls on (whether it conforms to the new protocol), you can
write totally unchecked code by accident. Not to say the code comprehension
itself isn't a problem — with type inferred/descriptor-cleaned code it's
evident the information isn't there, so if something isn't clear there's an
impetus to option-click/check it. With this dynamic-posing-as-checked code,
the truth and the evidence is disguised.
Anyway, there's my primary concern. Has this been addressed yet? Has any
thought been given to requiring '?'/'!' on these calls (or some other
symbol, but this would retain some cleanliness..) to opt into the inherent
fallibility and distinguish them visually?
Thanks,
Mathew
On Mon, Nov 27, 2017 at 6:04 AM, Chris Lattner via swift-evolution < swift-evolution@swift.org> wrote:
I’d like to formally propose the inclusion of user-defined dynamic member
lookup types.Here is my latest draft of the proposal:
https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438
DynamicMemberLookup proposal by lattner · Pull Request #768 · apple/swift-evolution · GitHubAn implementation of this design is available here:
Implement the DynamicLookupProtocol proposal. by lattner · Pull Request #13076 · apple/swift · GitHubThe implementation is straight-forward and (IMO) non-invasive in the
compiler.-Chris
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution
I’m not sure this solves the Ruby ergonomics problem. Picking up from an earlier thread:
Chris wrote:
Paul wrote:
An “always use parens” bridge to Ruby has bad ergonomics
Zero-arg Ruby methods are a mixture of property-like things that would certainly not use parens in Swift, and function-like things that certainly would:// Idiomatic Swift:
post.author.name.reversed()// Swift bridging to Ruby…
// …if no-args methods •must• use parens:
post.author().name().reverse()// …if no-args methods •can’t• use parens:
post.author.name.reverseIf the goal is to make Swift mostly feel like Swift even when bridging to Ruby, then the bridge needs to support both access forms.
Ok, that can certainly be implemented by the two proposals I have in flight. No obvious problem there.
Chris, can you elaborate? I think the proposal precludes this; I must be missing something!
As I read the proposal, the dynamic member subscript for post.author returns either a property value or a DynamicCallable, depending on whether the thing is a property or a method — but post.author and post.author() would both look identical to that subscript implementation, and there’s no distinction on the Ruby side, so the subscript can’t know which one to return.
The getter could return something that is both a dynamic callable and has dynamic members that lazily make the implicit method call. But this has two serious problems:
1. The necessarily lazy evaluation of the property value leads to nightmare scenarios:
post.author = sally
let oldAuthor = post.author()
post.author = fred
oldAuthor.name // "Sally"
// but
post.author = sally
let oldAuthor = post.author
post.author = fred
oldAuthor.name // "Fred"
2. This precludes bridging to Swift types, e.g. Ruby strings → Swift strings.
Therefore it seems the proposal forces post.author().name().reverse(). Something I'm missing?
Cheers,
Paul
On Nov 27, 2017, at 12:04 AM, Chris Lattner via swift-evolution <swift-evolution@swift.org> wrote:
I’d like to formally propose the inclusion of user-defined dynamic member lookup types.
Here is my latest draft of the proposal:
https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438
DynamicMemberLookup proposal by lattner · Pull Request #768 · apple/swift-evolution · GitHubAn implementation of this design is available here:
Implement the DynamicLookupProtocol proposal. by lattner · Pull Request #13076 · apple/swift · GitHubThe implementation is straight-forward and (IMO) non-invasive in the compiler.
-Chris
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution
I suggest using different subscripts for function/method calls and properties:
* `value(...)` ==> `value[dynamicFunction: ""](...)`
* `value.name(...)` ==> `value[dynamicFunction: "name"](...)`
* `value.name` ==> `value[dynamicProperty: "name"]`
Dynamic callable types have an unnamed method.
Dynamic argument labels are always allowed (but can be ignored).
For example, the labels of `Date.UTC` below are not passed to JavaScript.
//===----------------------------------------------------------------------===//
// JavaScriptCore.playground
//===----------------------------------------------------------------------===//
import Foundation
import JavaScriptCore
typealias Arguments = DictionaryLiteral<String, Any>
extension JSContext {
subscript(dynamicFunction name: String) -> (_ arguments: Arguments) -> JSValue {
return globalObject[dynamicFunction: name]
}
subscript(dynamicProperty name: String) -> JSValue {
get {
return globalObject[dynamicProperty: name]
}
set {
globalObject[dynamicProperty: name] = newValue
}
}
}
extension JSValue {
subscript(dynamicFunction name: String) -> (_ arguments: Arguments) -> JSValue {
return { arguments in
let argumentValues = arguments.map({ $0.value })
if name.isEmpty {
return self.call(withArguments: argumentValues)
} else if name == "new" {
return self.construct(withArguments: argumentValues)
} else {
return self.invokeMethod(name, withArguments: argumentValues)
}
}
}
subscript(dynamicProperty name: String) -> JSValue {
get {
return forProperty(name)
}
set {
setValue(newValue, forProperty: name)
}
}
}
//===----------------------------------------------------------------------===//
// Examples
//===----------------------------------------------------------------------===//
let context = JSContext()!
//: ```
//: context.isNaN(Double.nan)
//: ```
context[dynamicFunction: "isNaN"](["": Double.nan])
//: ```
//: context.Math.PI
//: ```
context[dynamicProperty: "Math"][dynamicProperty: "PI"]
//: ```
//: context.Math.PI.toFixed(5)
//: ```
context[dynamicProperty: "Math"][dynamicProperty: "PI"][dynamicFunction: "toFixed"](["": 5])
//: ```
//: context.Math.pow(2, 53)
//: ```
context[dynamicProperty: "Math"][dynamicFunction: "pow"](["": 2, "": 53])
//: ```
//: let time = context.Date.UTC(
//: year: 9999, month: 11, day: 31,
//: hour: 23, minute: 59, second: 59, millisecond: 999)
//: ```
let time = context[dynamicProperty: "Date"][dynamicFunction: "UTC"]([
"year": 9999, "month": 11, "day": 31,
"hour": 23, "minute": 59, "second": 59, "millisecond": 999])
//: ```
//: let date = context.Date.new(time)
//: ```
let date = context[dynamicProperty: "Date"][dynamicFunction: "new"](["": time])
//: ```
//: date.toISOString()
//: ```
date[dynamicFunction: "toISOString"]([:])
On 27 Nov 2017, at 06:04, Chris Lattner wrote:
I’d like to formally propose the inclusion of user-defined dynamic member lookup types.
Here is my latest draft of the proposal:
https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438
DynamicMemberLookup proposal by lattner · Pull Request #768 · apple/swift-evolution · GitHubAn implementation of this design is available here:
Implement the DynamicLookupProtocol proposal. by lattner · Pull Request #13076 · apple/swift · GitHubThe implementation is straight-forward and (IMO) non-invasive in the compiler.
-Chris
I think better interoperability with Python (and other OO languages in widespread use) is a good goal, and I agree that the implementation of the feature described is straight-forward and not terribly invasive in the compiler.
However, I do not think this proposal is going in the right direction for Swift. I have objections on several different grounds.
Philosophy
Swift is, unabashedly, a strong statically-typed language. We don’t allow implicit down casting, we require “as?” so you have to cope with the possibility of failure (or use “as!” and think hard about the “!”). Even the gaping hole that is AnyObject dispatch still requires the existence of an @objc declaration and produces an optional lookup result, so the user must contend with the potential for dynamic failure. Whenever we discuss adding more dynamic features to Swift, there’s a strong focus on maintaining that strong static type system.
IMO, this proposal is a significant departure from the fundamental character of Swift, because it allows access to possibly-nonexistent members (as well as calls with incorrect arguments, in the related proposal) without any indication that the operation might fail. It’s easy to fall through these cracks for any type that supports DynamicMemberLookupProtocol—a single-character typo when using a DynamicMemberLookupProtocol-capable type means you’ve fallen out of the safety that Swift provides. I think that’s a poor experience for the Python interoperability case, but more on that in the Tooling section below.
While we shouldn’t necessarily avoid a feature simply because it can be used distastefully, consider something like this:
public extension NSObject : DynamicMemberLookupProtocol, DynamicCallableProtocol { … }
that goes directly to the Objective-C runtime to resolve member lookups and calls—avoiding @objc, bridging headers, and so on. It’s almost frighteningly convenient, and one could imagine some mixed Objective-C/Swift code bases where this would save a lot of typing (of code)… at the cost of losing static typing in the language. The presence of that one extension means I can no longer rely on the safety guarantees Swift normally provides, for any project that imports that extension and uses a subclass of NSObject. At best, we as a community decide “don’t do that”; at worse, some nontrivial fraction of the community decides that the benefits outweigh the costs (for this type or some other), and we can no longer say that Swift is a strong statically-typed language without adding “unless you’re using something that adopts DynamicMemberLookupProtocol”.
Tooling
The Python interoperability enabled by this proposal *does* look fairly nice when you look at a small, correctly-written example. However, absolutely none of the tooling assistance we rely on when writing such code will work for Python interoperability. Examples:
* As noted earlier, if you typo’d a name of a Python entity or passed the wrong number of arguments to it, the compiler will not tell you: it’ll be a runtime failure in the Python interpreter. I guess that’s what you’d get if you were writing the code in Python, but Swift is supposed to be *better* than Python if we’re to convince a community to use Swift instead.
* Code completion won’t work, because Swift has no visibility into declarations written in Python
* Indexing/jump-to-definition/lookup documentation/generated interface won’t ever work. None of the IDE features supported by SourceKit will work, which will be a significant regression for users coming from a Python-capable IDE.
Statically-typed languages should be a boon for tooling, but if a user coming from Python to Swift *because* it’s supposed to be a better development experience actually sees a significantly worse development experience, we’re not going to win them over. It’ll just feel inconsistent.
Dynamic Typing Features
It’s possible that the right evolutionary path for Swift involves some notion of dynamic typing, which would have a lot of the properties sought by this proposal (and the DynamicCallableProtocol one). If that is true—and I’m not at all convinced that it is—we shouldn’t accidentally fall into a suboptimal design by taking small, easy, steps. If we’re to include dynamic-typing facilities, we should look at more existing practice—C# ‘dynamic' is one such approach, but more promising would be some form of gradual typing a la TypeScript that let’s one more smoothly (and probably explicitly) shift between strong and weak typing.
How Should Python Interoperability Work?
Going back to the central motivator for this proposal, I think that providing something akin to the Clang Importer provides the best interoperability experience: it would turn Python declarations into *real* Swift declarations, so that we get the various tooling benefits of having a strong statically-typed language. Sure, the argument types will all by PyObject or PyVal, but the names are there for code completion (and indexing, etc.) to work, and one could certainly imagine growing the importer to support Python’s typing annotations <https://docs.python.org/3/library/typing.html>\. But the important part here is that it doesn’t change the language model at all—it’s a compiler feature, separate from the language. Yes, the Clang importer is a big gnarly beast—but if the goal is to support N such importers, we can refactor and share common infrastructure to make them similar, perhaps introducing some kind of type provider infrastructure to allow one to write new importers as Swift modules.
In truth, you don’t even need the compiler to be involved. The dynamic “subscript” operation could be implemented in a Swift library, and one could write a Python program to process a Python module and emit Swift wrappers that call into that subscript operation. You’ll get all of the tooling benefits with no compiler changes, and can tweak the wrapper generation however much you want, using typing annotations or other Python-specific information to create better wrappers over time.
- Doug
On Nov 26, 2017, at 10:04 PM, Chris Lattner via swift-evolution <swift-evolution@swift.org> wrote:
I’d like to formally propose the inclusion of user-defined dynamic member lookup types.
Here is my latest draft of the proposal:
https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438
DynamicMemberLookup proposal by lattner · Pull Request #768 · apple/swift-evolution · GitHubAn implementation of this design is available here:
Implement the DynamicLookupProtocol proposal. by lattner · Pull Request #13076 · apple/swift · GitHubThe implementation is straight-forward and (IMO) non-invasive in the compiler.
I think this is a bad idea.
I’ve worked with APIs like this in Perl, Ruby, and Obj-C. Even in a dynamic language, IMO it’s almost always a mistake to use them. It makes code harder to reason about when reading and debugging.
This seems especially true with Swift because you can no longer autocomplete or introspect these sort of things. (Not that Swift has that much introspection at the moment. Aside: how would these work, if at all, with `KeyPath`s?)
I personally don’t want to work with APIs like this. And while I can understand the appeal of easier interop—especially if you’ve come into a large library of Python code—IMO this would inevitably lead to other misuses of dynamic lookups in 1st- or popular 3rd-party libraries.
But apart from my general philosophical complaints, I have 2 specific criticisms of the proposal:
1. I don’t think this would actually be good for writing Python
The examples in the proposal are very shallow, highlighting single lookups. But what are the types of the returned values?
Since all the lookups resolve to a single type, it must either (1) require casting or (2) only use dynamic lookups.
If you’re casting, then the dynamic lookups serve no real purpose. Any serious development project would quickly add a typed interface on top of the dynamic one. It would be better to build this typing into the bridging instead of performing dynamic lookups.
If you’re only using dynamic lookups, then you’re no longer really writing Swift. Any real typing has gone out the window and you’re left writing a strange variant of Python. When you do need to get a value out, I’d imagine you’d need to cast.
I think the proposal should include a more complete example that demonstrates how meaningful interop code could be written.
2. The JavaScript example will be popular, but isn’t actually that useful
Swift already has `Codable`, which provides more automatic and more type-safe (in that it encourages all-or-nothing decoding) JSON decoding. On that basis alone, I don’t think it’d be useful.
But this also ignores the ugly reality of JSON: the difficulty isn’t in accessing fields. The real trouble comes when you try to map a real, evolved, flawed API into a well-typed, thought-out data model. (This criticism applies to `Coding` too. Mapping JSON to an `enum` gets ugly quick.)
Arguably, this proposal’s JSON interface would be better for throwaway scripting, but IMO that case isn’t worth optimizing for.
-Matt
On Nov 27, 2017, at 1:04 AM, Chris Lattner via swift-evolution <swift-evolution@swift.org> wrote:
I’d like to formally propose the inclusion of user-defined dynamic member lookup types.
Here is my latest draft of the proposal:
https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438
DynamicMemberLookup proposal by lattner · Pull Request #768 · apple/swift-evolution · GitHubAn implementation of this design is available here:
Implement the DynamicLookupProtocol proposal. by lattner · Pull Request #13076 · apple/swift · GitHubThe implementation is straight-forward and (IMO) non-invasive in the compiler.
-Chris
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution
Paul Cantrell wrote:
It seems like this fundamentally alters Swift’s aesthetic of “either an
operation is type-safe, or it’s clear at the point of use that it’s not.”
Should this use an operator other than a period, e.g. `pickle->loads(blob)`?
This is a major concern of mine as well. What you end up by camouflaging
dynamic, unchecked APIs in what looks like normal (i.e. static, checked)
Swift is a wolf in sheeps’ clothing. One would not be able to tell one from
the other, yet would have to juggle two completely orthogonal language
models in one’s mind when reading/writing Swift code.
For me for such a proposal to even be remotely acceptable such dynamic
calls would have to have a clearly distinguishable syntax (`foo.bar?(…)`
would have been a great fit for this, but that ship has sailed).
In order to prevent such dynamic code from getting sprinkled all over the
place (dirt gets everywhere if given enough time) I’d further more expect
entire code blocks to be marked with a label that leans more towards
discouragement, than encouragement, such as:
unchecked {
foo.bar?!(…)
}
Brent Royal-Gordon wrote:
* Let's step away from bridging entirely and just think about Swift for a
moment. There are cases where we'd like to make *semi*-dynamic proxies
which wrap another type and allow operations based on what's statically
known about that type. Think, for example, of the appearance proxy in
UIKit: This is an object attached to UIView subclasses which lets you (in
essence) set default values for all instances. We currently just pretend
it's an instance of `Self`, which mostly works because of Objective-C, but
a Swift-native version would probably prefer to return a
`UIAppearance<Self>` object which used its knowledge of `Self` to expose
`Self`'s properties on itself. Is there a way we could design this feature,
or a related feature, to cover that kind of use case? That is, to allow a
limited set of keys—perhaps even key-path-based when you want static
control—with a different type for each key, *or* to allow any key with some
common type, depending on your type's needs?
This particular use-case (along with NSProxy) looks to me like a perfect us
of a new-type/derive as known from Haskell or Rust, rather than dynamic.
(Albeit there are problems with new-type and types making use of `Self` as
associated types.)
Doug Gregor wrote:
Swift is, unabashedly, a strong statically-typed language.
And thankfully so, if I may say so. There a few places in the language
(other than objc-bridging) where one needs to make us of dynamic features.
But most of these are caused by the fact that Swift’s type system is still
rather limited and certain type relations simply cannot be expressed yet,
forcing one to make use of the type-safety escape hook that is
Any/AnyObject.
IMO, this proposal is a significant departure from the fundamental
character of Swift, because it allows access to possibly-nonexistent
members […] without any indication that the operation might fail.
Unless there would be a corresponding compiler flag for marking use of such
dynamic calls as warnings if not errors, the addition of this proposal
would eliminate many of the safety guarantees that Swift gives me for code
written in it for safety critical programs. If one cannot judge about the
safety of a piece of code in Swift by merely looking at it, then what is
the use of a remaining half-baked safety promise to begin with?
Matt Diephouse wrote:
I personally don’t want to work with APIs like this. And while I can
understand the appeal of easier interop—especially if you’ve come into a
large library of Python code—IMO this would inevitably lead to other
misuses of dynamic lookups in 1st- or popular 3rd-party libraries.
Neither do I. The still unfinished nature of Swift’s type checking/system
and lack of meta-programming capabilities will drive users of the language,
who don’t know any better (especially when coming from dynamic languages),
to a wide-spread misuse of such features.
There also is another aspect to it:
The main driving force behind this proposal according to this discussion is
“interop with dynamic languages”.
The majority of language users however work in an environment where such
inter is prohibited: iOS.
Releasing a feature that’s basically shouting “MISUSE ME, MISUSE ME!” at
hundreds of thousands of users who’s pretty much only possible use _is_ a
misuse, and of which many are coming from ObjC or Java and still at odds
with, if not outright denial of Swift’s more strict type rules, is a
terrible foundation, I think.
I also think that we should rather have a syntax sugar freeze until Swift 6
(more on that: Syntactic Diabetes. When learning a new language (be it a… | by Vincent Esche | DefiniteLoops
<Medium), rather
than adding yet another to the mix without really thinking about the
consequences (lack of coherence, e.g.).
Benjamin G wrote:
A few examples of the kind of things that i would think about :
- use different file names for code using dynamic features.
- have every dynamic code included in a dynamic{} block
- have a different syntax for dynamic function calls, using ! or !!!
- Make the dynamic calls work only of subclasses of a base "PythonObject".
That would make the goal clear.
etc.
Now, i think those are all bad ideas (and that's why i don't like the
proposal in the first place). But i really think the concern should be
addressed.
Two of these (!, !!! and dynamic {}) are similar in spirit to what I
proposed above (?, ?! and unchecked {}).
And I too think that those are all bad ideas (and that's why i don't like
the proposal in the first place).
Karl Wagner wrote:
My worry is that in eagerness to try and scoop up developers of other
languages, that we diminish the Swift language to the point where it
doesn’t really provide a benefit for them. Python developers will be
writing the same unsafe, non-checked code using Swift that they do in
Python; where’s the win?
I agree with this 100%.
A language should never have growth (beyond what is necessary for
sustainable as a language!) be its driving force (unlike the often quoted
“world dominance” of Swift). Instead a language should strive to provide
the best tools for its use-case. Quality over quantity. Otherwise you’re
gaining the world and losing your soul.
Especially when it comes to Python and JS I’m not sure I’d want their
ecosystems' practices to get carried over to Swift.
Python is infamous for its liberal use of “putting everything and the
kitchen sink into stringly typed dicts”.
Please, please let’s not have this in Swift (which DynamicMemberLookup
would enable all too conveniently).
I’m also rather skeptical about the influences of the Javascript ecosystem
on Swift. The JS ecosystem is "the best we could have from nothing”, but
still utterly broken by design, with abominations such as leftpad being
more the norm, than the exception.
Before opening the floodgates to external influences (from communities that
dwarf the Swift community in sheer size) shall we please first wait long
enough until we as a community have come to a consensus about what actually
is idiomatic Swift? (Case in point: Apple claimed to have invented [sic!]
Protocol-oriented Programming with Swift and touted it as the driving force
behind its design. Yet a brief look at the stdlib reveals little to no use
of it. It’s concrete types all over the place. Arrays all over the place.)
When we consider interoperability with other languages, it should be from
the perspective of mapping their features to Swift’s philosophies.
This proposal takes the reverse approach, and makes Swift like Python, so
I’m against it.
This. I am further more of the opinion that in a pair of interoperable
languages it should always be the dynamic one that’s abstracting over the
static one, not the other way round. Statically typed languages are—by
nature—more concrete than dynamic ones. They have to be.
Jon Gilbert wrote:
However, I believe that you bring up an extremely valid concern. To
extrapolate from your points, this proposal seems to have a high potential
to erode what makes Swift special, and it provides a way to get around
Swift’s safe-guards, avoid writing Swift versions of libraries, and
ultimately, avoid hiring real Swift developers.
This is another valid and often overlooked point. Management will find the
path of least (initial) financial resistance. Every single time.
It's the very same reasons that we have more and more apps build on the
technically inferior (as in much more memory & power-hungry, non-native
feel, etc.) Electron apps these days. And while at the current time they
might look like the right solution they will be looked upon in similar ways
as one looks at Flash these days. How did we manage to obsolete Flash? By
extending the capabilities and expressiveness of HTML and our browsers.
Let’s extend the capabilities and expressiveness of Swift, instead of
dividing the ecosystem as Flash did with the web.
Why compromise Swift’s static-ness and roll out the red carpet for the
dynamic, unwashed masses?
Swift’s static-ness is the unique selling point of the language. As is for
Kotlin, Rust, Haskell, and the like (all with varying degrees of
static-ness). Overall we’re seeing type-systems spread through languages
(in many instances in form of gradual typing). There hardly is a language
out there that did not gain some kind of typing recently. Python, Ruby,
Javascript, Racket, … you name it.
Everything in life and engineering is a trade-off. Swift’s trademark is
that it consistently sacrifices the convenience of dynamic-ness for the
safety and performance of static-ness. This proposal does seem to do the
opposite, and we’d be naive not to fear the effect this will have.
ThisI can’t help but read this proposal as a bit of a “Wait, I actually
_did_ mean Swift to be Objective-C without the C, please fix!” So does this
mean Craig was right after all? ;)
Or why it would be bad for a Swift project to depend upon a stack of
third-language dependencies that CocoaPods/Carthage can’t manage, XCode
can’t analyze, stack traces can’t delve into (or can it?), lldb can’t step
through, etc.
This is another big one. Which is especially worrying given the state of
Swift’s tooling today: there literally is none.
We have a compiler that still provides utterly unhelpful diagnostics every
other day with no proper official swift-env handling, no proper
cross-compilation tooling, no proper unit testing, no proper formatting, no
proper lints, no proper plugins, … nothing.
All we have is a compiler that has much potential but is still very much
unfinished. Embedded in an IDE that completely breaks down when asked to
provide auto-completion and debugging support leaving a lot to be desired,
too. Even in Xcode 9. Regularly. With a package manager that up until
recently featured a fatal remote code execution exploit (which afaik is
still there on platforms other than macOS).
I think we have more urgent things to work on at this point, than interop
with other languages—which, as Chris pointed out don’t give a damn about
Swift to begin with.
Chris Lattner wrote:
3) Many Python APIs are wrappers for C APIs. “Swiftizing” a Python API in
this case means writing a new Swift wrapper for the API, not adding type
annotations.
Especially in the context of interoperation with machine learning /
scientific computing libraries from Python I don’t get why one would want
to bridge to Python.
Tensorflow, Scipy, Numpy, … they all boil down to being shallow wrappers
around native language extensions written in C or C++, just as you wrote
yourself.
What I would want is to have these be ported to/wrapped with Swift, instead
of Python.
In general cannot see a single scenario where I would want to directly
interact (as not just “please evaluate the entire file Foo.py”) with a
scripting language from Swift, where it wouldn’t have been a better idea to
implement either side in the given other language to begin with.
You want to make use of Tensorflow from Swift? Simple, use the C-API. It
exists for _exactly_ this reason.
If I am coding in a low-level language to begin with, then why would I want
to interface with 3rd-party languages/library on a layer that’s any higher
than necessary?
I’m not opposed to going further over time, but I’d like to get started at
some point :-). I’m not in a super urgent hurry to get this in in the next
week or month or anything like that, but I also don’t want to wait until
Swift 10.
From my point of view Swift 10 would be a perfect time to revisit the
prospect of extending Swift to such dynamic use-cases. At a point where its
strengths, weaknesses and idiomatic patterns have had plenty of time to be
investigated.
- Vincent
Hi Chris,
We do the Swift integration with Objective-C via modules. Why wouldn’t that approach work with Python? Or if it would, why would we favor this dynamic member lookup over importing a Python API via a module?
Dave
On Nov 26, 2017, at 11:04 PM, Chris Lattner via swift-evolution <swift-evolution@swift.org> wrote:
I’d like to formally propose the inclusion of user-defined dynamic member lookup types.
Here is my latest draft of the proposal:
https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438
DynamicMemberLookup proposal by lattner · Pull Request #768 · apple/swift-evolution · GitHubAn implementation of this design is available here:
https://github.com/apple/swift/pull/13076The implementation is straight-forward and (IMO) non-invasive in the compiler.
-Chris
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution
The implementation is straight-forward and (IMO) non-invasive in the compiler.
At least it’s smaller than I would have expected — but I don’t have the overview to judge the consequences to the compiler.
However, the impact those two proposals have on Swift is imho huge, and as the whole story is quite special for several reasons, there’s an elephant in the room that shouldn’t be ignored:
- What would you do if the proposal isn’t accepted?
Circle back and try to solve the problem another way? It depends on why it would be rejected.
- What does core think you would do in this case?
The core team officially weighs in after the review process, not before.
- What impact will that speculation have on the decision?
It should be zero. The point of the review process is to weigh the merits of a proposal.
In any case, FWIW, I don’t find your email very helpful, since you’re not actually talking about technical merits of the proposal: you’re encouraging speculation about politics :-)
-Chris
On Nov 27, 2017, at 2:56 AM, Tino Heth <2th@gmx.de> wrote:
I tuned out the initial discussions of this proposal because there seemed to be a lot of noise centered around implementation/maintainability.
There was some noise, but as I believe the patch shows, it was misplaced.
I'm curious if the actual premise of the syntactic/sugar conversion has been discussed/debated yet? i.e. making dynamic/stringly calls look like normal calls is very clean, but it's also very misleading (by definition; they're not normal/safe/checked calls) with a potential net reduction in ergonomics.
In fact, the calls are fully type safe and checked. The JSON example shows that.
I think what you’re getting at here is that there is opportunity to misuse this feature. It is certainly not something that every one should use commonly on their types: it’s a power feature that is extremely important for some narrow cases.
I’ll observe that this is exactly the sort of feature that Swift traditionally includes, and there are many examples of this: generalized operator overloading, the ability to syntax extend almost everything in the language (e.g. literals), user defined pattern matching in switches, etc.
Anyway, there's my primary concern. Has this been addressed yet? Has any thought been given to requiring '?'/'!' on these calls (or some other symbol, but this would retain some cleanliness..) to opt into the inherent fallibility and distinguish them visually?
This feature isn’t unsafe at all, I’m not sure why that would be appropriate.
-Chris
On Nov 27, 2017, at 6:37 AM, Mathew Huusko V <mhuusko5@gmail.com> wrote:
Hi Chris,
Some questions:
Hi Dave,
Both of these questions are really about the other proposal, the DynamicCallable proposal:
https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d
It isn’t as refined as the DynamicMemberLookup proposal, but the two do work together.
DynamicMemberLookup is only about changing how “x.y” resolves.
Multiple implementations are allowed and are resolved through standard overload resolution. Here’s an extract from the test case in the patch:
// References to overloads are resolved just like normal subscript lookup:
+// they are either contextually disambiguated or are invalid.
+struct Ambiguity : DynamicMemberLookupProtocol {
+ subscript(dynamicMember member: String) -> Int {
+ return 42
+ }
+ subscript(dynamicMember member: String) -> Float {
+ return 42
+ }
+}
On Nov 27, 2017, at 12:54 PM, Dave DeLong <swift@davedelong.com> wrote:
+
+func testAmbiguity(a : Ambiguity) {
+ let _ : Int = a.flexibility
+ let _ : Float = a.dynamism
+ _ = a.dynamism // expected-error {{ambiguous use of 'subscript(dynamicMember:)'}}
+}
+
For example, in the JSON example, there’s currently only a dynamic member subscript implementation that returns “JSON?”. Could I add another one that returns “String?” and have the type checker choose the appropriate one based on type inference?
Yes. It is tricky to actually use such APIs, but they are definitely possible and work exactly like current overloads in Swift do.
As far as naming goes, I’d propose “DynamicMemberForwarding”. In my mind, “Lookup” and “Resolve” are inappropriate, because you’re not actually “looking up” or “resolving” to a particular implementation. In Objective-C, the “+resolveInstanceMethod” stuff expects you to *provide* a method implementation (an IMP). On the other hand, if there isn’t a method implementation (which is the case here), you end up in the “-forwardInvocation:” path, where you can just introspect the NSInvocation and do whatever it is you’re doing to do. So, it seems like this proposal sounds a lot more like ObjC's “invocation forwarding” rather than the method resolution; hence my preference for using “Forwarding” in the name for naming consistency.
Sure, I’m not strongly attached to any of the names in the proposal.
-Chris
Thanks David,
-Chris
On Nov 27, 2017, at 11:43 AM, David Owens <owensd@apple.com> wrote:
I like the proposal. I think it’s simple and straight forward and provides some really nice ergonomics for working with dynamic data.
I appreciate the option to change the names too.
If I understand it correctly, you could also call “add_trick” via:
dog.addTrick("Roll over");
And simply update the subscript implementation to provide additional name look-ups. Cool!
And I like that the DynamicCallable provides the next level that I want to make the API more “Swift-like” and make use of named parameters:
dog.add(trick: "Roll over");
Or maybe more appropriate:
dog.addTrick("Roll over”, favorite: true);
I didn’t look at the implementation details, but I think the overall approach. I’d like it better if we have provisions to create these types of tools outside of the compiler, but that’s another topic.
-David
On Nov 26, 2017, at 10:04 PM, Chris Lattner via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
I’d like to formally propose the inclusion of user-defined dynamic member lookup types.
Here is my latest draft of the proposal:
https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438
https://github.com/apple/swift-evolution/pull/768An implementation of this design is available here:
https://github.com/apple/swift/pull/13076The implementation is straight-forward and (IMO) non-invasive in the compiler.
-Chris
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution
I'm sure the implementation is fine, and I'm not criticising the proposal's
intent/concept, least of all simply because it might be misused (because
everything can and is).
Instead, I firmly believe — not to say I necessarily *deserve* a contrary
explanation/argument if no one thinks this is a big deal, but just to be
clear.. — that it is clean but fundamentally misleading (and thus
potentially non-ergonomic) to transparently convert `Python.open(filename)`
to `Python.get(member: "open")(filename)`.
The former line reads as "this lines calls this Swift function the compiler
knows exists, as usual", and the latter reads as "this line looks for/tries
to call something that may exist" — two very/importantly different things,
and you will only be able to tell the difference by option-clicking the
call site.
When we want to deal with a value that may not exist at runtime (an
optional), there's extra syntax ('?') for that. It seems like the same
thing for a function/call that may not exist *might* be appropriate.
Equally happy to hear why not.
On Mon, Nov 27, 2017 at 6:30 PM, Chris Lattner <clattner@nondot.org> wrote:
On Nov 27, 2017, at 6:37 AM, Mathew Huusko V <mhuusko5@gmail.com> wrote:
> I tuned out the initial discussions of this proposal because there
seemed to be a lot of noise centered around implementation/
maintainability.There was some noise, but as I believe the patch shows, it was misplaced.
> I'm curious if the actual premise of the syntactic/sugar conversion has
been discussed/debated yet? i.e. making dynamic/stringly calls look like
normal calls is very clean, but it's also very misleading (by definition;
they're not normal/safe/checked calls) with a potential net reduction in
ergonomics.In fact, the calls are fully type safe and checked. The JSON example
shows that.I think what you’re getting at here is that there is opportunity to misuse
this feature. It is certainly not something that every one should use
commonly on their types: it’s a power feature that is extremely important
for some narrow cases.I’ll observe that this is exactly the sort of feature that Swift
traditionally includes, and there are many examples of this: generalized
operator overloading, the ability to syntax extend almost everything in the
language (e.g. literals), user defined pattern matching in switches, etc.> Anyway, there's my primary concern. Has this been addressed yet? Has any
thought been given to requiring '?'/'!' on these calls (or some other
symbol, but this would retain some cleanliness..) to opt into the inherent
fallibility and distinguish them visually?This feature isn’t unsafe at all, I’m not sure why that would be
appropriate.-Chris
There is nothing that is inherently non-fallible with "normal" Swift calls. As far as the caller can tell, any function or method you call can fail. There is no difference here; the implementation of the "user-defined dynamic member lookup" object will determine if you have called it in a proper way or not, in the same way as "+" will determine if you have called it in a way that overflows or not.
/Magnus
27 Nov. 2017 22:38 Mathew Huusko V via swift-evolution <swift-evolution@swift.org> wrote:
I tuned out the initial discussions of this proposal because there seemed to be a lot of noise centered around implementation/maintainability. I'm curious if the actual premise of the syntactic/sugar conversion has been discussed/debated yet? i.e. making dynamic/stringly calls look like normal calls is very clean, but it's also very misleading (by definition; they're not normal/safe/checked calls) with a potential net reduction in ergonomics.
Hi Ben,
This proposal has nothing to do with “callables”, just dynamics member lookup. The other proposal: https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d (which I need to revise a bit) relates to dynamic callable stuff, and it does in fact have a “DynamicCallableWithKeywordsToo” form that provides what you are asking for IIUC.
There is no need to model calls as subscripts though, because syntactic calls in swift (intentionally) do not produce lvalues.
-Chris
On Nov 28, 2017, at 1:55 PM, Ben Rimmington <me@benrimmington.com> wrote:
I suggest using different subscripts for function/method calls and properties:
* `value(...)` ==> `value[dynamicFunction: ""](...)`
* `value.name(...)` ==> `value[dynamicFunction: "name"](...)`
* `value.name` ==> `value[dynamicProperty: "name"]`Dynamic callable types have an unnamed method.
Dynamic argument labels are always allowed (but can be ignored).
For example, the labels of `Date.UTC` below are not passed to JavaScript.
Chris wrote:
Paul wrote:
An “always use parens” bridge to Ruby has bad ergonomics
Zero-arg Ruby methods are a mixture of property-like things that would certainly not use parens in Swift, and function-like things that certainly would:// Idiomatic Swift:
post.author.name.reversed()// Swift bridging to Ruby…
// …if no-args methods •must• use parens:
post.author().name().reverse()// …if no-args methods •can’t• use parens:
post.author.name.reverseIf the goal is to make Swift mostly feel like Swift even when bridging to Ruby, then the bridge needs to support both access forms.
Ok, that can certainly be implemented by the two proposals I have in flight. No obvious problem there.
Chris, can you elaborate? I think the proposal precludes this; I must be missing something!
There are two proposals in flight: one that allows specifying runtime behavior for “a.b” syntax and one that allows specifying runtime behavior for “a(x)” and “a.b(x)” syntax (“DynamicCallableWithKeywordsToo”, which needs to be renamed obviously).
For your use case, you’d implement both of them. If someone wrote
a.b().c
Then you’d look up and call a zero-argument method named “b” on the a object. However if someone wrote:
a.b.c
Then you’d do the same thing (in the implementation of the DynamicMemberLookupProtocol). This approach allows the human writing code at your bridge to use whatever syntax “feels best” to them given the specific API in question that they are using.
You could even have “a.b” syntax fall back to finding Ruby ivars as well, though I don’t know if that is actually a good idea in practice for a Ruby bridge.
As I read the proposal, the dynamic member subscript for post.author returns either a property value or a DynamicCallable, depending on whether the thing is a property or a method — but post.author and post.author() would both look identical to that subscript implementation, and there’s no distinction on the Ruby side, so the subscript can’t know which one to return.
“post.author" always invokes the DynamicMemberLookupProtocol proposal.
“post.author()” would invoke the “DynamicCallableWithKeywordsToo” hook in the dynamic callable proposal:
https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d
-Chris
On Nov 28, 2017, at 10:12 AM, Paul Cantrell <cantrell@pobox.com> wrote: