Actually all this talk about existentials vs. generics or protocols vs. classes has had me confused somewhat and I think there are still some misconceptions present on this list sometimes, so I’ll try to clear them up:
(1) misconception: protocols with associated types are somehow very different from generics
I don’t think they are and I will explain why. The only difference is the way the type parameters are bound: generics use explicit parameter lists whereas protocols use inheritance. That has some advantages (think long parameter lists of generics) and some disadvantages.
These ways are dual in a notation sense: generic types have to have all parameters bound whereas protocols cannot bind any of them.
The „existential“ notation `Any<>` being discussed on this list is nothing more than adding the ability to protocols to bind the parameters to be used just like Java’s wildcards are adding the opposite feature to generics, namely not having to bind all parameters.
Essentially `Any<Collection>` in Swift is just the same as `Collection<?>` in Java (assuming for comparability’s sake that Swift’s Collection had no additional associated types; otherwise I would just have to introduce a Collection<Element, Index> in Java).
Likewise `Any<Collection where .Element: Number>` is just the same as `Collection<? extends Number>` in Java.
And just like Collection<?> does not conform to a type parameter `T extends Collection<?>` because Collection<?> is the type `forall E. Collection<E>` whereas `T extends Collection<?>` is the type `T. Collection<T>` for a given T.
In essence protocols with associated types are like generics with wildcards.
Coming back to the questions whether (a) allowing existentials to be used as types is useful and (b) whether sacrificing type safety would somehow be necessary for that, I think we can safely answer
(a) yes, it *is* useful to be able to use existentials like Any<Collection> as types, because wildcards are quite often needed and very useful in Java (they haven’t been added without a reason)
(b) no, sacrificing type safety does not make sense, as the experience with Java’s wildcards shows that this is not needed. Especially if something like path dependent types is used like proposed and some notation to open an existential’s type is added, which is both something that Java does not have.
(2) misconception: POP is different from OOP
It is not. Protocols are just interfaces using subtyping like OOP has always done. They just use associated types instead of explicit type parameters for generics (see above). The more important distinction of Swift is emphasizing value types and making mutation safely available by enforcing copy semantics for value types.
But protocols are not really different from interfaces in Java. I would have preferred a unified model using just classes with real multiple inheritance like Eiffel has and value types just being a part of that similar to Eiffel’s `expanded` classes. But that ship has probably sailed a long time ago :-/
So be it. But at least there should be no reasons for POP vs OOP wars ;-)
(I’d like to add that I liked Dave’s talks at last WWDC very much, it’s just that I don’t think that POP is something new or different.)
-Thorsten
···
Am 09.06.2016 um 19:50 schrieb Thorsten Seitz via swift-evolution <swift-evolution@swift.org>:
Am 09.06.2016 um 18:49 schrieb Dave Abrahams via swift-evolution <swift-evolution@swift.org>:
on Wed Jun 08 2016, Jordan Rose <swift-evolution@swift.org> wrote:
On Jun 8, 2016, at 13:16, Dave Abrahams via swift-evolution >>>> <swift-evolution@swift.org> wrote:
on Wed Jun 08 2016, Thorsten Seitz >>> >>>> <swift-evolution@swift.org >>>> <mailto:swift-evolution@swift.org>> >>>> wrote:
Ah, thanks, I forgot! I still consider this a bug, though (will have
to read up again what the reasons are for that behavior).
Yes, but in the case of the issue we're discussing, the choices are:
1. Omit from the existential's API any protocol requirements that depend
on Self or associated types, in which case it *can't* conform to
itself because it doesn't fulfill the requirements.
2. Erase type relationships and trap at runtime when they don't line up.
Matthew has been arguing against #2, but you can't “fix the bug” without
it.
#1 has been my preference for a while as well, at least as a starting
point.
I should point out that with the resyntaxing of existentials to
Any<Protocols...>, the idea that Collection's existential doesn't
conform to Collection becomes far less absurd than it was, so maybe this
is not so bad.
I think the problem is more that Any<Collection> does not conform to a specific value for a type parameter T: Collection
What I mean by this is that `Collection` denotes a type family, a generic parameter `T: Collection` denotes a specific (though unknown) member of that type family and `Any<Collection>` denotes the type family again, so there is really no point in writing Any<Collection> IMO.
The type family cannot conform to T because T is just one fixed member of it.
It conforms to itself, though, as I can write
let c1: Any<Collection> = …
let c2: Any<Collection> = c1
That’s why I think that we could just drop Any<Collection> and simply write Collection.
, but haven't realized
that if you step around the type relationships encoded in Self
requirements and associated types you end up with types that appear to
interoperate but in fact trap at runtime unless used in exactly the
right way.
Trap at runtime? How so? Generalized existentials should still be
type-safe.
There are two choices when you erase static type relationships:
1. Acheive type-safety by trapping at runtime
FloatingPoint(3.0 as Float) + FloatingPoint(3.0 as Double) // trap
2. Don't expose protocol requirements that involve these relationships,
which would prevent the code above from compiling and prevent
FloatingPoint from conforming to itself.
Or are you talking about the hypothetical types / behaviors people
think they want when they don’t fully understand what is happening...
I don't know what you mean here. I think generalized existentials will
be nice to have, but I think most people will want them to do something
they can't possibly do.
Exactly. What I meant is that people think they want that expression
to compile because they don’t understand that the only thing it can do
is trap. I said “hypothetical” because producing a compile time error
rather than a runtime trap is the only sane thing to do. Your comment
surprised me because I can’t imagine we would move forward in Swift
with the approach of trapping.
I would very much like to be able to create instances of “Collection
where Element == Int” so we can throw away the wrappers in the stdlib.
That will require some type mismatches to be caught at runtime via
trapping.
For invalid index because the existential accepts a type erased index?
Exactly.
How do you decide where to draw the line here? It feels like a very
slippery slope for a language where safety is a stated priority to
start adopting a strategy of runtime trapping for something as
fundamental as how you expose members on an existential.
If you don't do this, the alternative is that “Collection where Element
== Int” does not conform to Collection.
This isn’t directly related to having self or associated type
requirements. It is true of all existentials.
That is just an implementation limitation today, IIUC. What I'm talking
about here would make it impossible for some to do that.
If it is just an implementation limitation I am happy to hear that.
If that changes for simple existentials and generalized existentials
expose all members (as in the latest draft of the proposal) maybe it
will be possible for all existentials to conform to their protocol.
Not without introducing runtime traps. See my “subscript function”
example.
That's weird and not very
useful. You could expose all the methods that were on protocol
extensions of Collection on this existential, unless they used
associated types other than the element type. But you couldn't pass the
existential to a generic function like
func scrambled<C: Collection>(_ c: C) -> [C.Element]
IMO you should *have* to introduce unsafe behavior like that manually.
Collection where Element == Int & Index == *
?
I didn’t mean directly through the type of the existential.
My question is, why not? That is still explicit.
It’s not explicit in the sense that nobody wrote `fatalError` or
similar in their code. It’s too easy to write something like that
without realizing that it introduces the possibility of a crash. If
we adopt syntax like that to introduce an existential that introduces
traps we should at least require members that can be trap to be
invoked using a `!` suffix or something like that to make it clear to
users that a trap will happen if they are not extremely careful when
using that member.
More generally though, I don’t want the rules of the language to be
written in a way that causes the compiler to synthesize traps in such
a general way.
The existential should not introduce a precondition that isn’t already
present in the semantics of the protocol itself. If the semantics of
the protocol do not place preconditions on arguments beyond their type
(such as “must be a valid index into this specific instance”) the
compiler should not allow the existential to conform if a trap is
required in some circumstances. That is a new precondition and
therefore the existential does not actually fulfill the requirements
of the protocol.
I could *maybe* live with a solution where protocol requirements are
marked as trapping, etc depending on the specific argument received at
runtime. This is a total straw man syntax, but maybe `IndexableBase`
would declare the subscript `@trapping` (probably something different
but I hope this communicates the idea). This alerts users to the fact
that they need to be extra careful - not any value of `Self.Index` is
valid and you can get a crash if you’re not careful.
Having this semantic explicit in the definition of the protocol opens
the door to maybe considering an existential synthesized by the
compiler that traps because it doesn’t introduce a new precondition
that wasn’t already present in the protocol.
I would want to give consideration to specific details of a proposal
along these lines before deciding how I feel about it, but I have a
more open mind to this approach than introducing traps not present in
the preconditions of the protocol.
/// You can subscript a collection with any valid index other than the
/// collection's end index. The end index refers to the position one past
/// the last element of a collection, so it doesn't correspond with an
/// element.
///
/// - Parameter position: The position of the element to access. `position`
/// must be a valid index of the collection that is not equal to the
/// `endIndex` property. @trapping public subscript(position: Self.Index) -> Self._Element { get }
One obvious mechanism for introducing unsafe behavior is to write
manual type erasure wrappers like we do today.
Another possibility would be to allow extending the existential type
(not the protocol). This would allow you to write overloads on the
Collection existential that takes some kind of type erased index if
that is what you want and either trap if you receive an invalid index
or better (IMO) return an `Element?`. I’m not sure how extensions on
existentials might be implemented, but this is an example of the kind
of operation you might want available on it that you wouldn’t want
available on all Collection types.
Collection indices are already something that isn’t fully statically
safe so I understand why you might want to allow this.
By the same measure, so are Ints :-)
The fact that a type's methods have preconditions does *not* make it
“statically unsafe.”
That depends on what you mean by safe. Sure, those methods aren’t
going corrupt memory, but they *are* going to explicitly and
intentionally crash for some inputs. That doesn’t qualify as “fully
safe” IMO.
Please pick a term other than “unsafe” here; it's not unsafe in the
sense we mean the word in Swift. It's safe in exactly the same way that
array indexes and integers are. When you violate a precondition, it
traps.
Safe. The most obvious way to write code should also behave in a safe
manner. Undefined behavior is the enemy of safety, and developer
mistakes should be caught before software is in production. Opting for
safety sometimes means Swift will feel strict, but we believe that
clarity saves time in the long run.
Safety
Swift was designed from the outset to be safer than C-based languages,
and eliminates entire classes of unsafe code. Variables are always
initialized before use, arrays and integers are checked for overflow,
and memory is managed automatically. Syntax is tuned to make it easy
to define your intent — for example, simple three-character keywords
define a variable (var) or constant (let).
Another safety feature is that by default Swift objects can never be
nil, and trying to make or use a nil object will results in a
compile-time error. This makes writing code much cleaner and safer,
and prevents a common cause of runtime crashes. However, there are
cases where nil is appropriate, and for these situations Swift has an
innovative feature known as optionals. An optional may contain nil,
but Swift syntax forces you to safely deal with it using ? to indicate
to the compiler you understand the behavior and will handle it safely.
This positioning statement makes it appear as if preventing common
causes of crashes falls within the meaning of safe that Swift is
using. Having existentials introduce new preconditions and traps when
they are not met does not seem aligned with that goal IMO.
Static typing “increases safety,” in the casual sense. That doesn't
mean that an operation that traps on a failed precondition check is
“unsafe.”
The user doesn't do anything “manual” to introduce that trapping
behavior for integers. Preconditions are a natural part of most types.
The user doesn’t, but isn’t the overflow trap implemented in the
standard library?
Whether it is or is not is an implementation detail.
Regardless, this is a specific case that has been given explicit
design attention by humans. The precondition is designed, not
introduced by compiler rules that haven’t considered the specific case
in question.
But I don’t think having the language's existentials do this
automatically is the right approach. Maybe there is another
approach that could be used in targeted use cases where the less
safe behavior makes sense and is carefully designed.
Whether it makes sense or not really depends on the use-cases. There's
little point in generalizing existentials if the result isn't very useful.
Usefulness depends on your perspective.
Of course. As I've said, let's look at the use cases.
Agree. We can consider those in depth when the time comes to ramp up
discussion of Austin’s proposal.
I have run into several scenarios where they would be very useful
without needing to be prone to crashes when used incorrectly. One
obvious basic use case is storing things in a heterogenous collection
where you bind .
bind what?
Sorry, I must have gotten distracted and not finished that paragraph.
I meant to say bind the associated types that are necessary for your
use case. Sometimes you bind *all* of the associated types to
concrete types and the protocol has no `Self` requirements. In that
case there is no trouble at all in conforming the type-erased
“existential" to the protocol itself. Austin’s proposal would
eliminate the need to manually write these “existentials” manually.
The way to find out is to take a look at the examples we currently have
of protocols with associated types or Self requirements and consider
what you'd be able to do with their existentials if type relationships
couldn't be erased.
We have known use-cases, currently emulated in the standard library, for
existentials with erased type relationships. *If* these represent the
predominant use cases for something like generalized existentials, it
seems to me that the language feature should support that. Note: I have
not seen anyone build an emulation of the other kind of generalized
existential. My theory: there's a good reason for that :-).
AFAIK (and I could be wrong) the only rules in the language that
require the compiler to synthesize a trap except using a nil IUO, `!`
on a nil Optional, and an invalid `as` cast . These are all
syntactically explicit unsafe / dangerous operations. All other traps
are in the standard library (array index, overflow, etc). Most
important about all of these cases is that they have received direct
human consideration.
There is no distinction in the user model between what might be
synthesized by the language and what appears on standard library types.
Maybe I shouldn’t have made that distinction.
The point I am trying to emphasize is that each of these are special
cases that have received direct human consideration. The potential
for a trap is not introduced by language rules that apply to
user-defined constructs in without consideration of the specific
details of that construct.
Introducing a language (not library) mechanism that exposes members on
generalized existentials in a way that relies on runtime traps for
type safety feels to me like a pretty dramatic turn agains the stated
priority of safety. It will mean you must understand exactly what is
going on and be extremely careful to use generalized existentials
without causing crashes. This will either make Swift code much more
crashy or will scare people away from using generalized existentials
(and maybe both).
I don't accept either of those statements without seeing some analysis
of the use-cases. For example, I don't believe that AnyCollection et al
are particularly crash-prone. The likelihood that you'll use the wrong
index type with a collection is very, very low. I'm less certain of
what happens with Self requirements in real cases.
But again, I believe this is an exceptional case as the precondition
is explicitly stated in the semantics of the protocol.
IIUC, it has been cited by Doug as the exemplar of the
predominantly-requested case by a 10:1 ratio!
In terms of forming the existential, storing it in variables,
accepting arguments of that type, etc yes. I don’t know how many of
those requests expect it to conform to the protocol and expect to be
able to use it in generic code constrained to the protocol.
IMO the burden of proof should be on the side that proposes a
mechanism to introduce traps, not the side that proposes avoiding
them.
If you really want to make this about sides and burdens, the burden of
proof always rests with the side proposing to extend the language. We
shouldn't be making changes without understanding how they will play out
in real use-cases.
I agree with this. But if we are discussing two different options for
extending the language I think the option that doesn’t introduce
crashes should be preferred without pretty compelling reasons to
choose the option that can introduce crashes.
Neither of those outcomes is good.
Collection indices are a somewhat special case as there is already a
strong precondition that people are familiar with because it would be
too costly to performance and arguably too annoying to deal with an
Optional result in every array lookup. IMO that is why the library is
able to get away with it in the current type erased AnyCollection.
But this is not a good model for exposing any members on an
existential that do not already have a strong precondition that causes
a trap when violated.
I think a big reason why you maybe haven’t seen a lot of examples of
people writing type erased “existentials" is because it is a huge pain
in the neck to write this stuff manually today. People may be
designing around the need for them. I haven’t seen a huge sampling of
type erased “existentials" other people are writing but I haven’t
written any that introduce a trap like this. The only traps are in
the “abstract" base class whose methods will never be called (and
wouldn’t even be implemented if they could be marked abstract).
What specific things do you think we need to be able to do that rely
on the compiler synthesizing a trap in the way it exposes the members
of the existential?
I don't know. I'm saying, I don't think we understand the use-cases
well enough to make a determination.
That’s fair. I agree that use cases should be carefully considered.
Here are a few examples from Austin’s proposal that safely use
existential collections. I don’t understand why you think this
approach is insufficient. Maybe you could supply a concrete example
of a use case that can’t be written with the mechanism in Austin’s
proposal.
// A variable whose type is the Index associated type of the underlying
// concrete type of 'a'.
let theIndex : a.Index = ...
// A variable whose type is the Element associated type of the underlying
// concrete type of 'a'.
let theElement : a.Element = ...
// Given a mutable collection, swap its first and last items.
// Not a generic function.
func swapFirstAndLast(inout collection: Any<BidirectionalMutableCollection>) {
// firstIndex and lastIndex both have type "collection.Index"
guard let firstIndex = collection.startIndex,
lastIndex = collection.endIndex?.predecessor(collection) where lastIndex != firstIndex else {
print("Nothing to do")
return
}
// oldFirstItem has type "collection.Element"
let oldFirstItem = collection[firstIndex]
var a : Any<BidirectionalMutableCollection where .Element == String> = ...
let input = "West Meoley"
// Not actually necessary, since the compiler knows "a.Element" is String.
// A fully constrained anonymous associated type is synonymous with the concrete
// type it's forced to take on, and the two are interchangeable.
// However, 'as' casting is still available if desired.
let anonymousInput = input as a.Element
a[a.startIndex] = anonymousInput
// as mentioned, this also works:
a[a.startIndex] = input
// If the collection allows it, set the first element in the collection to a given string.
func setFirstElementIn(inout collection: Any<Collection> toString string: String) {
if let element = string as? collection.Element {
// At this point, 'element' is of type "collection.Element"
collection[collection.startIndex] = element
}
}
Neither of these look like they actually make *use* of the fact that
there's type erasure involved (and therefore should probably be written
as generics?). The interesting cases with Any<Collection...>, for the
purposes of this discussion, arise when you have multiple instances of
the same existential type that wrap different concrete types.
One use case I have found is to work around the lack of higher-kinder
types.
Really, now: a use-case for feature A that is a workaround for the lack
of feature B hardly justifies adding feature A! We do want to add
higher-kinded types eventually.
Good to know. I thought higher-kinder types were on the “maybe if
someone shows a compelling enough use case” list. AFAIK this is the
first time a member of the core team has stated the intent to add
them.
Well, please don't take this as a formal statement on behalf of the
team. IIUC, the team is generally interested in having this feature.
Of course not. The only official plan of record for Swift is approved proposals. But it’s always interesting to know the opinions of the core team about potential features. Previously my impression had been that the general leaning was towards skepticism that the practical benefits of HKT would pay for the complexity. Maybe that was (or is now) not correct and the opinion is generally favorable. That is great news.
If that is the case I agree that this use case isn’t relevant. The
workaround isn’t great because it loses type information that is
critical to the optimizer (but it’s all we have available today).
If you have a protocol where specific implementations will return
different types, but all conform to a second protocol you can define
the protocol in terms of a generic type-erased wrapper which conforms
to the second protocol and accepts type arguments that match the
associated types (thus binding the associated types to concrete
types). I have found this to be a useful technique (granted it is a
workaround and I’m not sure how useful it would continue to be if
Swift eventually gets higher-kinder types).
Another problem I see: in this new world, what is the model for choosing
whether to write a function as a protocol extension/generic, or as a
regular function taking existential parameters? Given that either of
the above could have been written either way, we need to be able to
answer that question. When existentials don't conform to their
protocols, it seems to me that the most general thing to do is use
existentials whenever you can, and only resort to using generics when
forced by the type system. This does not seem like a particularly good
programming model to me, but I might be convinced otherwise.
That doesn’t seem like a particularly good programming model to me either.
The rule of thumb I am operating with for protocols with Self or
associated type requirements is to prefer generics and use type
erasure / existentials when that isn’t possible. For example, when
heterogeneity is required or when you can’t form the necessary type in
a protocol requirement (as in the preceding example).
This heuristic has been working out pretty well for me thus far.
I do worry a bit that people will choose the opposite heuristic.
It would be somewhat reassuring to me if we could prove to ourselves
that, using your heuristic, one is never forced to copy/paste a generic
function implementation into a corresponding function that uses
existentials.
The primary impact of introducing a language mechanism for generalized
existentials in my code would be to eliminate a lot of manual type
erasing boilerplate.
If your code has many manual type erasing wrappers corresponding to
protocols with associated types and/or Self requirements that also never
have to trap type mismatches, that would certainly be instructive
empirical data. Would you care to share the protocols and wrappers you
are talking about?
I put together a sample implementation of a Cocoa-like responder chain
in Swift a while ago when the “Swift dynamism” debate was raging.
It isn't intended to be a Swifty design. It is intended to be similar
to Cocoa and show techniques that can be used to do things similar to
Cocoa’s responder chain and targer-action in Swift. It uses a type
erased wrapper for actions that binds `Sender` while hiding the
concrete `Action` type and also the `Handler` associated type. It
cannot and should not conform to the protocol it is derived from and
could be replaced with the generalized existentials in Austin’s
proposal.
This is a good example to start with because it is related to a topic
that has been hotly debated and is clearly something a lot of people
want to be able to do.
It's great to start with, but if this is just one example, we still
technically have more examples in the standard library of the other kind
of wrapper. You said it would eliminate a lot of manual boilerplate. I
presume that means you have lots more examples like this one?
There are variations on the command / responder chain pattern that are useful in various contexts. Another related example would be using this technique to do something similar to the Elm architecture (http://www.elm-lang.org <http://www.elm-lang.org/>\), but without requiring switch statements to do the dispatching.
Another common pattern where existentials can be useful is when you want to hide implementation details of an API. This can be just because you want the concrete type to be private, but it can also be similar to class clusters without the inheritance where the concrete type actually differs for one reason or another.
public protocol Foo {
associatedtype Bar
}
public struct AnyFoo<T> : Foo {
typealias Bar = T
private var base: AnyFooBase<T>
init<F: Foo where Bar == T>(foo: F) { … }
}
// assume the rest of the type erasure machinery
public func myFunc<T>(t: T) -> AnyFoo<T> {
// create an instance of of a private type
// which implements Foo and return
// it wrapped in AnyFoo
}
As has been mentioned already, another common case where you need existentials like this is if you need a heterogeneous collection.
There have been several blog posts and conference talks on type erasure in Swift. I don’t have time to review all of them right now but they may also be fruitful sources of additional good examples.
But really what I mean by a lot of manual boilerplate is three types each with an implementation of each member you want to expose every time you want to form an existential with types that conform to a protocol with associated types. This adds up quickly and is annoying to have to write. In terms of percentage of total lines of code it isn’t huge, but in terms of percentage of lines of boilerplate code it can be a lot. It is one of the bigger sources of boilerplate in Swift and probably steers people away from designs they might prefer because they just aren’t willing to tolerate the boilerplate.
-Matthew
···
On Jun 9, 2016, at 3:05 PM, Dave Abrahams <dabrahams@apple.com> wrote:
on Thu Jun 09 2016, Matthew Johnson <matthew-AT-anandabits.com> wrote:
On Jun 9, 2016, at 11:42 AM, Dave Abrahams <dabrahams@apple.com> wrote:
on Thu Jun 09 2016, Matthew Johnson <matthew-AT-anandabits.com <http://matthew-at-anandabits.com/>> wrote:
On Jun 8, 2016, at 1:33 PM, Dave Abrahams <dabrahams@apple.com <mailto:dabrahams@apple.com>> wrote:
on Tue Jun 07 2016, Matthew Johnson <matthew-AT-anandabits.com <http://matthew-at-anandabits.com/>> wrote:
On Jun 7, 2016, at 4:13 PM, Dave Abrahams via >>>>>>>>>>> swift-evolution <swift-evolution@swift.org >>>>>>>>>>> <mailto:swift-evolution@swift.org>> wrote:
on Tue Jun 07 2016, Matthew Johnson <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
That's certainly a valid question. I have been very vocally skeptical
that HKTs like Monad belong in the standard library, but I am not
opposed to having the language feature if it supports important use
cases.
···
on Thu Jun 09 2016, Matthew Johnson <swift-evolution-m3FHrko0VLzYtjvyW6yDsg@public.gmane.org> wrote:
On Jun 9, 2016, at 3:05 PM, Dave Abrahams <dabrahams-2kanFRK1NckAvxtiuMwx3w@public.gmane.org> wrote:
on Thu Jun 09 2016, Matthew Johnson <matthew-AT-anandabits.com> wrote:
On Jun 9, 2016, at 11:42 AM, Dave Abrahams <dabrahams-2kanFRK1NckAvxtiuMwx3w@public.gmane.org> wrote:
On Jun 7, 2016, at 4:13 PM, Dave Abrahams via >>>>>>>>>>>> swift-evolution <swift-evolution-m3FHrko0VLzYtjvyW6yDsg@public.gmane.org >>>>>>>>>>>> <mailto:swift-evolution-m3FHrko0VLzYtjvyW6yDsg@public.gmane.org>> wrote:
on Tue Jun 07 2016, Matthew Johnson >>>>>>>>>>>> <swift-evolution@swift.org >>>>>>>>>>>> <mailto:swift-evolution-m3FHrko0VLzYtjvyW6yDsg@public.gmane.org>> >>>>>>>>>>>> wrote:
, but haven't realized
that if you step around the type relationships encoded in Self
requirements and associated types you end up with types that appear to
interoperate but in fact trap at runtime unless used in exactly the
right way.
Trap at runtime? How so? Generalized existentials should still be
type-safe.
There are two choices when you erase static type relationships:
1. Acheive type-safety by trapping at runtime
FloatingPoint(3.0 as Float) + FloatingPoint(3.0 as Double) // trap
2. Don't expose protocol requirements that involve these relationships,
which would prevent the code above from compiling and prevent
FloatingPoint from conforming to itself.
Or are you talking about the hypothetical types / behaviors people
think they want when they don’t fully understand what is happening...
I don't know what you mean here. I think generalized existentials will
be nice to have, but I think most people will want them to do something
they can't possibly do.
Exactly. What I meant is that people think they want that expression
to compile because they don’t understand that the only thing it can do
is trap. I said “hypothetical” because producing a compile time error
rather than a runtime trap is the only sane thing to do. Your comment
surprised me because I can’t imagine we would move forward in Swift
with the approach of trapping.
I would very much like to be able to create instances of “Collection
where Element == Int” so we can throw away the wrappers in the stdlib.
That will require some type mismatches to be caught at runtime via
trapping.
For invalid index because the existential accepts a type erased index?
Exactly.
How do you decide where to draw the line here? It feels like a very
slippery slope for a language where safety is a stated priority to
start adopting a strategy of runtime trapping for something as
fundamental as how you expose members on an existential.
If you don't do this, the alternative is that “Collection where Element
== Int” does not conform to Collection.
This isn’t directly related to having self or associated type
requirements. It is true of all existentials.
That is just an implementation limitation today, IIUC. What I'm talking
about here would make it impossible for some to do that.
If it is just an implementation limitation I am happy to hear that.
If that changes for simple existentials and generalized existentials
expose all members (as in the latest draft of the proposal) maybe it
will be possible for all existentials to conform to their protocol.
Not without introducing runtime traps. See my “subscript function”
example.
That's weird and not very
useful. You could expose all the methods that were on protocol
extensions of Collection on this existential, unless they used
associated types other than the element type. But you couldn't pass the
existential to a generic function like
func scrambled<C: Collection>(_ c: C) -> [C.Element]
IMO you should *have* to introduce unsafe behavior like that manually.
Collection where Element == Int & Index == *
?
I didn’t mean directly through the type of the existential.
My question is, why not? That is still explicit.
It’s not explicit in the sense that nobody wrote `fatalError` or
similar in their code. It’s too easy to write something like that
without realizing that it introduces the possibility of a crash. If
we adopt syntax like that to introduce an existential that introduces
traps we should at least require members that can be trap to be
invoked using a `!` suffix or something like that to make it clear to
users that a trap will happen if they are not extremely careful when
using that member.
More generally though, I don’t want the rules of the language to be
written in a way that causes the compiler to synthesize traps in such
a general way.
The existential should not introduce a precondition that isn’t already
present in the semantics of the protocol itself. If the semantics of
the protocol do not place preconditions on arguments beyond their type
(such as “must be a valid index into this specific instance”) the
compiler should not allow the existential to conform if a trap is
required in some circumstances. That is a new precondition and
therefore the existential does not actually fulfill the requirements
of the protocol.
I could *maybe* live with a solution where protocol requirements are
marked as trapping, etc depending on the specific argument received at
runtime. This is a total straw man syntax, but maybe `IndexableBase`
would declare the subscript `@trapping` (probably something different
but I hope this communicates the idea). This alerts users to the fact
that they need to be extra careful - not any value of `Self.Index` is
valid and you can get a crash if you’re not careful.
Having this semantic explicit in the definition of the protocol opens
the door to maybe considering an existential synthesized by the
compiler that traps because it doesn’t introduce a new precondition
that wasn’t already present in the protocol.
I would want to give consideration to specific details of a proposal
along these lines before deciding how I feel about it, but I have a
more open mind to this approach than introducing traps not present in
the preconditions of the protocol.
/// You can subscript a collection with any valid index other than the
/// collection's end index. The end index refers to the position one past
/// the last element of a collection, so it doesn't correspond with an
/// element.
///
/// - Parameter position: The position of the element to access. `position`
/// must be a valid index of the collection that is not equal to the
/// `endIndex` property. @trapping public subscript(position: Self.Index) -> Self._Element { get }
One obvious mechanism for introducing unsafe behavior is to write
manual type erasure wrappers like we do today.
Another possibility would be to allow extending the existential type
(not the protocol). This would allow you to write overloads on the
Collection existential that takes some kind of type erased index if
that is what you want and either trap if you receive an invalid index
or better (IMO) return an `Element?`. I’m not sure how extensions on
existentials might be implemented, but this is an example of the kind
of operation you might want available on it that you wouldn’t want
available on all Collection types.
Collection indices are already something that isn’t fully statically
safe so I understand why you might want to allow this.
By the same measure, so are Ints :-)
The fact that a type's methods have preconditions does *not* make it
“statically unsafe.”
That depends on what you mean by safe. Sure, those methods aren’t
going corrupt memory, but they *are* going to explicitly and
intentionally crash for some inputs. That doesn’t qualify as “fully
safe” IMO.
Please pick a term other than “unsafe” here; it's not unsafe in the
sense we mean the word in Swift. It's safe in exactly the same way that
array indexes and integers are. When you violate a precondition, it
traps.
Safe. The most obvious way to write code should also behave in a safe
manner. Undefined behavior is the enemy of safety, and developer
mistakes should be caught before software is in production. Opting for
safety sometimes means Swift will feel strict, but we believe that
clarity saves time in the long run.
Safety
Swift was designed from the outset to be safer than C-based languages,
and eliminates entire classes of unsafe code. Variables are always
initialized before use, arrays and integers are checked for overflow,
and memory is managed automatically. Syntax is tuned to make it easy
to define your intent — for example, simple three-character keywords
define a variable (var) or constant (let).
Another safety feature is that by default Swift objects can never be
nil, and trying to make or use a nil object will results in a
compile-time error. This makes writing code much cleaner and safer,
and prevents a common cause of runtime crashes. However, there are
cases where nil is appropriate, and for these situations Swift has an
innovative feature known as optionals. An optional may contain nil,
but Swift syntax forces you to safely deal with it using ? to indicate
to the compiler you understand the behavior and will handle it safely.
This positioning statement makes it appear as if preventing common
causes of crashes falls within the meaning of safe that Swift is
using. Having existentials introduce new preconditions and traps when
they are not met does not seem aligned with that goal IMO.
Static typing “increases safety,” in the casual sense. That doesn't
mean that an operation that traps on a failed precondition check is
“unsafe.”
The user doesn't do anything “manual” to introduce that trapping
behavior for integers. Preconditions are a natural part of most types.
The user doesn’t, but isn’t the overflow trap implemented in the
standard library?
Whether it is or is not is an implementation detail.
Regardless, this is a specific case that has been given explicit
design attention by humans. The precondition is designed, not
introduced by compiler rules that haven’t considered the specific case
in question.
But I don’t think having the language's existentials do this
automatically is the right approach. Maybe there is another
approach that could be used in targeted use cases where the less
safe behavior makes sense and is carefully designed.
Whether it makes sense or not really depends on the use-cases. There's
little point in generalizing existentials if the result isn't very useful.
Usefulness depends on your perspective.
Of course. As I've said, let's look at the use cases.
Agree. We can consider those in depth when the time comes to ramp up
discussion of Austin’s proposal.
I have run into several scenarios where they would be very useful
without needing to be prone to crashes when used incorrectly. One
obvious basic use case is storing things in a heterogenous collection
where you bind .
bind what?
Sorry, I must have gotten distracted and not finished that paragraph.
I meant to say bind the associated types that are necessary for your
use case. Sometimes you bind *all* of the associated types to
concrete types and the protocol has no `Self` requirements. In that
case there is no trouble at all in conforming the type-erased
“existential" to the protocol itself. Austin’s proposal would
eliminate the need to manually write these “existentials” manually.
The way to find out is to take a look at the examples we currently have
of protocols with associated types or Self requirements and consider
what you'd be able to do with their existentials if type relationships
couldn't be erased.
We have known use-cases, currently emulated in the standard library, for
existentials with erased type relationships. *If* these represent the
predominant use cases for something like generalized existentials, it
seems to me that the language feature should support that. Note: I have
not seen anyone build an emulation of the other kind of generalized
existential. My theory: there's a good reason for that :-).
AFAIK (and I could be wrong) the only rules in the language that
require the compiler to synthesize a trap except using a nil IUO, `!`
on a nil Optional, and an invalid `as` cast . These are all
syntactically explicit unsafe / dangerous operations. All other traps
are in the standard library (array index, overflow, etc). Most
important about all of these cases is that they have received direct
human consideration.
There is no distinction in the user model between what might be
synthesized by the language and what appears on standard library types.
Maybe I shouldn’t have made that distinction.
The point I am trying to emphasize is that each of these are special
cases that have received direct human consideration. The potential
for a trap is not introduced by language rules that apply to
user-defined constructs in without consideration of the specific
details of that construct.
Introducing a language (not library) mechanism that exposes members on
generalized existentials in a way that relies on runtime traps for
type safety feels to me like a pretty dramatic turn agains the stated
priority of safety. It will mean you must understand exactly what is
going on and be extremely careful to use generalized existentials
without causing crashes. This will either make Swift code much more
crashy or will scare people away from using generalized existentials
(and maybe both).
I don't accept either of those statements without seeing some analysis
of the use-cases. For example, I don't believe that AnyCollection et al
are particularly crash-prone. The likelihood that you'll use the wrong
index type with a collection is very, very low. I'm less certain of
what happens with Self requirements in real cases.
But again, I believe this is an exceptional case as the precondition
is explicitly stated in the semantics of the protocol.
IIUC, it has been cited by Doug as the exemplar of the
predominantly-requested case by a 10:1 ratio!
In terms of forming the existential, storing it in variables,
accepting arguments of that type, etc yes. I don’t know how many of
those requests expect it to conform to the protocol and expect to be
able to use it in generic code constrained to the protocol.
IMO the burden of proof should be on the side that proposes a
mechanism to introduce traps, not the side that proposes avoiding
them.
If you really want to make this about sides and burdens, the burden of
proof always rests with the side proposing to extend the language. We
shouldn't be making changes without understanding how they will play out
in real use-cases.
I agree with this. But if we are discussing two different options for
extending the language I think the option that doesn’t introduce
crashes should be preferred without pretty compelling reasons to
choose the option that can introduce crashes.
Neither of those outcomes is good.
Collection indices are a somewhat special case as there is already a
strong precondition that people are familiar with because it would be
too costly to performance and arguably too annoying to deal with an
Optional result in every array lookup. IMO that is why the library is
able to get away with it in the current type erased AnyCollection.
But this is not a good model for exposing any members on an
existential that do not already have a strong precondition that causes
a trap when violated.
I think a big reason why you maybe haven’t seen a lot of examples of
people writing type erased “existentials" is because it is a huge pain
in the neck to write this stuff manually today. People may be
designing around the need for them. I haven’t seen a huge sampling of
type erased “existentials" other people are writing but I haven’t
written any that introduce a trap like this. The only traps are in
the “abstract" base class whose methods will never be called (and
wouldn’t even be implemented if they could be marked abstract).
What specific things do you think we need to be able to do that rely
on the compiler synthesizing a trap in the way it exposes the members
of the existential?
I don't know. I'm saying, I don't think we understand the use-cases
well enough to make a determination.
That’s fair. I agree that use cases should be carefully considered.
Here are a few examples from Austin’s proposal that safely use
existential collections. I don’t understand why you think this
approach is insufficient. Maybe you could supply a concrete example
of a use case that can’t be written with the mechanism in Austin’s
proposal.
// A variable whose type is the Index associated type of the underlying
// concrete type of 'a'.
let theIndex : a.Index = ...
// A variable whose type is the Element associated type of the underlying
// concrete type of 'a'.
let theElement : a.Element = ...
// Given a mutable collection, swap its first and last items.
// Not a generic function.
func swapFirstAndLast(inout collection: Any<BidirectionalMutableCollection>) {
// firstIndex and lastIndex both have type "collection.Index"
guard let firstIndex = collection.startIndex,
lastIndex = collection.endIndex?.predecessor(collection) where lastIndex != firstIndex else {
print("Nothing to do")
return
}
// oldFirstItem has type "collection.Element"
let oldFirstItem = collection[firstIndex]
var a : Any<BidirectionalMutableCollection where .Element == String> = ...
let input = "West Meoley"
// Not actually necessary, since the compiler knows "a.Element" is String.
// A fully constrained anonymous associated type is synonymous with the concrete
// type it's forced to take on, and the two are interchangeable.
// However, 'as' casting is still available if desired.
let anonymousInput = input as a.Element
a[a.startIndex] = anonymousInput
// as mentioned, this also works:
a[a.startIndex] = input
// If the collection allows it, set the first element in the collection to a given string.
func setFirstElementIn(inout collection: Any<Collection> toString string: String) {
if let element = string as? collection.Element {
// At this point, 'element' is of type "collection.Element"
collection[collection.startIndex] = element
}
}
Neither of these look like they actually make *use* of the fact that
there's type erasure involved (and therefore should probably be written
as generics?). The interesting cases with Any<Collection...>, for the
purposes of this discussion, arise when you have multiple instances of
the same existential type that wrap different concrete types.
One use case I have found is to work around the lack of higher-kinder
types.
Really, now: a use-case for feature A that is a workaround for the lack
of feature B hardly justifies adding feature A! We do want to add
higher-kinded types eventually.
Good to know. I thought higher-kinder types were on the “maybe if
someone shows a compelling enough use case” list. AFAIK this is the
first time a member of the core team has stated the intent to add
them.
Well, please don't take this as a formal statement on behalf of the
team. IIUC, the team is generally interested in having this feature.
Of course not. The only official plan of record for Swift is approved
proposals. But it’s always interesting to know the opinions of the
core team about potential features. Previously my impression had been
that the general leaning was towards skepticism that the practical
benefits of HKT would pay for the complexity.
Ah, thanks, I forgot! I still consider this a bug, though (will have
to read up again what the reasons are for that behavior).
Yes, but in the case of the issue we're discussing, the choices are:
1. Omit from the existential's API any protocol requirements that depend
on Self or associated types, in which case it *can't* conform to
itself because it doesn't fulfill the requirements.
2. Erase type relationships and trap at runtime when they don't line up.
Matthew has been arguing against #2, but you can't “fix the bug” without
it.
#1 has been my preference for a while as well, at least as a starting
point.
I should point out that with the resyntaxing of existentials to
Any<Protocols...>, the idea that Collection's existential doesn't
conform to Collection becomes far less absurd than it was, so maybe this
is not so bad.
I think the problem is more that Any<Collection> does not conform to a specific value for a type parameter T: Collection
What I mean by this is that `Collection` denotes a type family, a generic parameter `T: Collection` denotes a specific (though unknown) member of that type family and `Any<Collection>` denotes the type family again, so there is really no point in writing Any<Collection> IMO.
The type family cannot conform to T because T is just one fixed member of it.
It conforms to itself, though, as I can write
let c1: Any<Collection> = …
let c2: Any<Collection> = c1
That’s why I think that we could just drop Any<Collection> and simply write Collection.
Let me expand that a bit:
Actually all this talk about existentials vs. generics or protocols vs. classes has had me confused somewhat and I think there are still some misconceptions present on this list sometimes, so I’ll try to clear them up:
(1) misconception: protocols with associated types are somehow very different from generics
I don’t think they are and I will explain why. The only difference is the way the type parameters are bound: generics use explicit parameter lists whereas protocols use inheritance. That has some advantages (think long parameter lists of generics) and some disadvantages.
These ways are dual in a notation sense: generic types have to have all parameters bound whereas protocols cannot bind any of them.
The „existential“ notation `Any<>` being discussed on this list is nothing more than adding the ability to protocols to bind the parameters to be used just like Java’s wildcards are adding the opposite feature to generics, namely not having to bind all parameters.
btw, i tried to to see if Any<> could have a simpler alternative
Essentially `Any<Collection>` in Swift is just the same as `Collection<?>` in Java (assuming for comparability’s sake that Swift’s Collection had no additional associated types; otherwise I would just have to introduce a Collection<Element, Index> in Java).
Likewise `Any<Collection where .Element: Number>` is just the same as `Collection<? extends Number>` in Java.
Java supports co/contra variant params
And just like Collection<?> does not conform to a type parameter `T extends Collection<?>` because Collection<?> is the type `forall E. Collection<E>` whereas `T extends Collection<?>` is the type `T. Collection<T>` for a given T.
This picture is accurate today, but there are going to be more serious differences after 10 no date is currently geven for when it will come)
In essence protocols with associated types are like generics with wildcards.
Yes, java has kept everything within its generics system rather than split parts out. Something people may not immediately think about iwith respect to the 2 generic systems is that when u call a func<T>capture(T t){} in java with a wildcard you are doing a compile time capture only (to avoid the dreaded unsafe casts), whereas it is really nice to do the same in swift and subsequently be able to access T.Type and see that it is not Any. The closest u ever get to that type at runtime in java is via generics introspection, but u still can't do everything ( like no new T() ). But today the bridging between existential types and generics is definitely a work in progress.
Coming back to the questions whether (a) allowing existentials to be used as types is useful and (b) whether sacrificing type safety would somehow be necessary for that, I think we can safely answer
(a) yes, it *is* useful to be able to use existentials like Any<Collection> as types, because wildcards are quite often needed and very useful in Java (they haven’t been added without a reason)
IMO they made java 8 (referring to streams). And even though the syntax for co/contra variance is pretty heavy, it is the foundation for all modern java code. The any-fication of the generics is going to open new doors as some of it will translate into A partial reification in the jvm. It seems the decision for now the decision is to not use the extra info in java to retain binary compatibility with all the erased code out there, this is something scala might use in areas where it won't mind loosing java compatibility.
(b) no, sacrificing type safety does not make sense, as the experience with Java’s wildcards shows that this is not needed. Especially if something like path dependent types is used like proposed and some notation to open an existential’s type is added, which is both something that Java does not have.
I hope sypesafe opening inside the " if let " syntax gets added. I know that chris is against sugaring, but I played if an implementation of
... x is String?
If let! x {}
That runs as
if let x = x {}
something equally short could be done here.
(2) misconception: POP is different from OOP
It is not. Protocols are just interfaces using subtyping like OOP has always done. They just use associated types instead of explicit type parameters for generics (see above). The more important distinction of Swift is emphasizing value types and making mutation safely available by enforcing copy semantics for value types.
Values are coming to the jvm, which will narrow this gap (like their view identity for value vs ref and the whole deep ==) . I also really like the cow approach of the swift runtime.
But protocols are not really different from interfaces in Java.
There is one big difference: default methods, but it seems swift will add that soon. I also really like how extensions and conformance mix together in swift to bake retro-modelling in and the adapter pattern (spent enough years deep diving inside eclipse to appreciate it).
I would have preferred a unified model using just classes with real multiple inheritance like Eiffel has and value types just being a part of that similar to Eiffel’s `expanded` classes. But that ship has probably sailed a long time ago :-/
-1 i am old school c/c++... i really like protocol, struct, class, extensions, enums. It is a really nice mix that gives objc people room to grow, but I do miss how they are an integral part of generics (i protocols as a replacement and look forward to when they interact better) and namespaces+scoped-imports (c#)... Looking forward to where things go next
So be it. But at least there should be no reasons for POP vs OOP wars ;-)
(I’d like to add that I liked Dave’s talks at last WWDC very much, it’s just that I don’t think that POP is something new or different.)
I used to thin that way. But today I think that although in broad brush strokes the similarities and bigger than the differences, there is room for making a bigger difference in the how.
···
On Jun 10, 2016, at 9:35 PM, Thorsten Seitz via swift-evolution <swift-evolution@swift.org> wrote:
Am 09.06.2016 um 19:50 schrieb Thorsten Seitz via swift-evolution <swift-evolution@swift.org>:
Am 09.06.2016 um 18:49 schrieb Dave Abrahams via swift-evolution <swift-evolution@swift.org>:
on Wed Jun 08 2016, Jordan Rose <swift-evolution@swift.org> wrote:
On Jun 8, 2016, at 13:16, Dave Abrahams via swift-evolution >>>>> <swift-evolution@swift.org> wrote:
on Wed Jun 08 2016, Thorsten Seitz >>>> >>>>> <swift-evolution@swift.org >>>>> <mailto:swift-evolution@swift.org>> >>>>> wrote:
Ah, thanks, I forgot! I still consider this a bug, though (will have
to read up again what the reasons are for that behavior).
Yes, but in the case of the issue we're discussing, the choices are:
1. Omit from the existential's API any protocol requirements that depend
on Self or associated types, in which case it *can't* conform to
itself because it doesn't fulfill the requirements.
2. Erase type relationships and trap at runtime when they don't line up.
Matthew has been arguing against #2, but you can't “fix the bug” without
it.
#1 has been my preference for a while as well, at least as a starting
point.
I should point out that with the resyntaxing of existentials to
Any<Protocols...>, the idea that Collection's existential doesn't
conform to Collection becomes far less absurd than it was, so maybe this
is not so bad.
I think the problem is more that Any<Collection> does not conform to
a specific value for a type parameter T: Collection
What I mean by this is that `Collection` denotes a type family, a
generic parameter `T: Collection` denotes a specific (though
unknown) member of that type family and `Any<Collection>` denotes
the type family again, so there is really no point in writing
Any<Collection> IMO.
The type family cannot conform to T because T is just one fixed member of it.
It conforms to itself, though, as I can write
let c1: Any<Collection> = …
let c2: Any<Collection> = c1
That’s why I think that we could just drop Any<Collection> and simply write Collection.
Let me expand that a bit:
Actually all this talk about existentials vs. generics or protocols
vs. classes has had me confused somewhat and I think there are still
some misconceptions present on this list sometimes, so I’ll try to
clear them up:
There are several objectively incorrect statements here, and several
others with which I disagree. I was hoping someone else would write
this for me, but since the post has such a tone of authority I feel I
must respond.
(1) misconception: protocols with associated types are somehow very
different from generics
I don’t think they are and I will explain why. The only difference is
the way the type parameters are bound: generics use explicit parameter
lists whereas protocols use inheritance. That has some advantages
(think long parameter lists of generics) and some disadvantages.
These ways are dual in a notation sense: generic types have to have
all parameters bound whereas protocols cannot bind any of them.
The „existential“ notation `Any<>` being discussed on this list is
nothing more than adding the ability to protocols to bind the
parameters to be used just like Java’s wildcards are adding the
opposite feature to generics, namely not having to bind all
parameters.
Protocols and generics fulfill completely different roles in Swift, and
so, **especially in a language design context like the one we're in
here**, must be thought of differently. The former are an abstraction
mechanism for APIs, and the latter a mechanism for generalizing
implementations. The only place you could argue that they intersect is
in generic non-final classes, because a class fills the dual role of
abstraction and implementation mechanism (and some might say that's a
weakness). But even accounting for generic classes, protocols with
associated types are very different from generics. Two utterly
different types (an enum and a struct, for example) can conform to any
given protocol P, but generic types always share a common basis
implementation. There is no way to produce distinct instances of a
generic type with all its type parameters bound, but for any protocol P
I can make infinitely many instances of P with P.AssociatedType == Int.
Back to the my original point: while protocols and generic types have
some similarities, the idea that they are fundamentally the same thing
(I know you didn't say *exactly* that, but I think it will be read that
way) would be wrong and a very unproductive way to approach language
evolution.
Essentially `Any<Collection>` in Swift is just the same as
`Collection<?>` in Java (assuming for comparability’s sake that
Swift’s Collection had no additional associated types; otherwise I
would just have to introduce a Collection<Element, Index> in Java).
I don't see how you can use an example that requires *assuming away*
assoociated types to justify an argument that protocols *with associated
types* are the same as generics.
Likewise `Any<Collection where .Element: Number>` is just the same as
`Collection<? extends Number>` in Java.
And just like Collection<?> does not conform to a type parameter `T
extends Collection<?>` because Collection<?> is the type `forall
E. Collection<E>` whereas `T extends Collection<?>` is the type
`T. Collection<T>` for a given T.
In essence protocols with associated types are like generics with
wildcards.
It is true that generics with wildcards in Java *are* (not just “like”)
existential types but I don't agree with the statement above. Because
Java tries to create an “everything is a class” world, generic classes
with bound type parameters end up playing the role of existential type.
But protocols in Swift are not, fundamentally, just existential types,
and the resyntaxing of ProtocolName to Any<ProtocolName> for use in type
context is a huge leap forward in making that distinction clear... when
that's done (unless we leave Array<ProtocolName> around as a synonym for
Array<Any<ProtocolName>>—I really hope we won't!) protocols indeed
*won't* be types at all, existential or otherwise.
Coming back to the questions whether (a) allowing existentials to be
used as types is useful
That's the only use existentials have. They *are* types. Of course
they're useful, and I don't think anyone was arguing otherwise.
and (b) whether sacrificing type safety would somehow be necessary for
that, I think we can safely answer (a) yes, it *is* useful to be able
to use existentials like Any<Collection> as types, because wildcards
are quite often needed and very useful in Java (they haven’t been
added without a reason) (b) no, sacrificing type safety does not make
sense, as the experience with Java’s wildcards shows that this is not
needed.
I would call this “interesting information,” but hardly conclusive.
Java's generics are almost exactly the same thing as Objective-C
lightweight generics, which are less capable and less expressive in
many ways than Swift's generics.
Especially if something like path dependent types is used like
proposed and some notation to open an existential’s type is added,
which is both something that Java does not have.
(2) misconception: POP is different from OOP
It is not. Protocols are just interfaces using subtyping like OOP has
always done. They just use associated types instead of explicit type
parameters for generics (see above).
They are not the same thing at all (see above ;->). To add to the list
above, protocols can express fundamental relationships—like Self
requirements—that OOP simply can't handle. There's a reason Java can't
express Comparable without losing static type-safety. Finally, in a
language with first-class value types, taking a protocol-oriented
approach to abstraction leads to *fundamentally* different designs from
what you get using OOP.
The more important distinction of Swift is emphasizing value types and
making mutation safely available by enforcing copy semantics for value
types.
We don't, in fact, enforce copy semantics for value types. That's
something I'd like to change. But regardless, value types would be a
*lot* less useful if they couldn't conform to protocols, and so they
would be a lot less used. Heck, before we got protocol extensions in
Swift 2, there was basically *no way* to share implementation among
value types. So you can't take protocols out of the picture without
making value types, and the argument for value semantics, far weaker.
But protocols are not really different from interfaces in Java.
I would have preferred a unified model using just classes with real
multiple inheritance like Eiffel has and value types just being a part
of that similar to Eiffel’s `expanded` classes. But that ship has
probably sailed a long time ago :-/ So be it. But at least there
should be no reasons for POP vs OOP wars ;-) (I’d like to add that I
liked Dave’s talks at last WWDC very much, it’s just that I don’t
think that POP is something new or different.)
Protocol-oriented programming is about the synergy of features and ideas
most of which not *individually* new, but that together create a new
world of possibilities. I've already discussed the synergy of protocols
and first-class value semantics. There's also the fact that in
protocols we have one construct with which to express dynamic
polymorphism (existentials) and static polymorphism (generic
constraints), both of which have important roles to play but that I
maintain are very different indeed. One result is that you can “start
with a protocol” as your abstraction mechanism and know that you're not
going to design yourself into a corner where it becomes impossibly
awkward to express what you need. Finally—and I'm certain this *is*
new—in protocol extensions we have a means to express both post-hoc
conformance and generic functions that is much more accessible to users
than in any previous language, to the point where generic programming
can become a natural part of everyday work.
···
on Fri Jun 10 2016, Thorsten Seitz <tseitz42-AT-icloud.com> wrote:
Am 09.06.2016 um 19:50 schrieb Thorsten Seitz via swift-evolution <swift-evolution@swift.org>:
Am 09.06.2016 um 18:49 schrieb Dave Abrahams via swift-evolution <swift-evolution@swift.org>:
on Wed Jun 08 2016, Jordan Rose <swift-evolution@swift.org> wrote:
On Jun 8, 2016, at 13:16, Dave Abrahams via swift-evolution >>>>> <swift-evolution@swift.org> wrote:
on Wed Jun 08 2016, Thorsten Seitz >>>> >>>>> <swift-evolution@swift.org >>>>> <mailto:swift-evolution@swift.org>> >>>>> wrote:
, but haven't realized
that if you step around the type relationships encoded in Self
requirements and associated types you end up with types that appear to
interoperate but in fact trap at runtime unless used in exactly the
right way.
Trap at runtime? How so? Generalized existentials should still be
type-safe.
There are two choices when you erase static type relationships:
1. Acheive type-safety by trapping at runtime
FloatingPoint(3.0 as Float) + FloatingPoint(3.0 as Double) // trap
2. Don't expose protocol requirements that involve these relationships,
which would prevent the code above from compiling and prevent
FloatingPoint from conforming to itself.
Or are you talking about the hypothetical types / behaviors people
think they want when they don’t fully understand what is happening...
I don't know what you mean here. I think generalized existentials will
be nice to have, but I think most people will want them to do something
they can't possibly do.
Exactly. What I meant is that people think they want that expression
to compile because they don’t understand that the only thing it can do
is trap. I said “hypothetical” because producing a compile time error
rather than a runtime trap is the only sane thing to do. Your comment
surprised me because I can’t imagine we would move forward in Swift
with the approach of trapping.
I would very much like to be able to create instances of “Collection
where Element == Int” so we can throw away the wrappers in the stdlib.
That will require some type mismatches to be caught at runtime via
trapping.
For invalid index because the existential accepts a type erased index?
Exactly.
How do you decide where to draw the line here? It feels like a very
slippery slope for a language where safety is a stated priority to
start adopting a strategy of runtime trapping for something as
fundamental as how you expose members on an existential.
If you don't do this, the alternative is that “Collection where Element
== Int” does not conform to Collection.
This isn’t directly related to having self or associated type
requirements. It is true of all existentials.
That is just an implementation limitation today, IIUC. What I'm talking
about here would make it impossible for some to do that.
If it is just an implementation limitation I am happy to hear that.
If that changes for simple existentials and generalized existentials
expose all members (as in the latest draft of the proposal) maybe it
will be possible for all existentials to conform to their protocol.
Not without introducing runtime traps. See my “subscript function”
example.
That's weird and not very
useful. You could expose all the methods that were on protocol
extensions of Collection on this existential, unless they used
associated types other than the element type. But you couldn't pass the
existential to a generic function like
func scrambled<C: Collection>(_ c: C) -> [C.Element]
IMO you should *have* to introduce unsafe behavior like that manually.
Collection where Element == Int & Index == *
?
I didn’t mean directly through the type of the existential.
My question is, why not? That is still explicit.
It’s not explicit in the sense that nobody wrote `fatalError` or
similar in their code. It’s too easy to write something like that
without realizing that it introduces the possibility of a crash. If
we adopt syntax like that to introduce an existential that introduces
traps we should at least require members that can be trap to be
invoked using a `!` suffix or something like that to make it clear to
users that a trap will happen if they are not extremely careful when
using that member.
More generally though, I don’t want the rules of the language to be
written in a way that causes the compiler to synthesize traps in such
a general way.
The existential should not introduce a precondition that isn’t already
present in the semantics of the protocol itself. If the semantics of
the protocol do not place preconditions on arguments beyond their type
(such as “must be a valid index into this specific instance”) the
compiler should not allow the existential to conform if a trap is
required in some circumstances. That is a new precondition and
therefore the existential does not actually fulfill the requirements
of the protocol.
I could *maybe* live with a solution where protocol requirements are
marked as trapping, etc depending on the specific argument received at
runtime. This is a total straw man syntax, but maybe `IndexableBase`
would declare the subscript `@trapping` (probably something different
but I hope this communicates the idea). This alerts users to the fact
that they need to be extra careful - not any value of `Self.Index` is
valid and you can get a crash if you’re not careful.
Having this semantic explicit in the definition of the protocol opens
the door to maybe considering an existential synthesized by the
compiler that traps because it doesn’t introduce a new precondition
that wasn’t already present in the protocol.
I would want to give consideration to specific details of a proposal
along these lines before deciding how I feel about it, but I have a
more open mind to this approach than introducing traps not present in
the preconditions of the protocol.
/// You can subscript a collection with any valid index other than the
/// collection's end index. The end index refers to the position one past
/// the last element of a collection, so it doesn't correspond with an
/// element.
///
/// - Parameter position: The position of the element to access. `position`
/// must be a valid index of the collection that is not equal to the
/// `endIndex` property. @trapping public subscript(position: Self.Index) -> Self._Element { get }
One obvious mechanism for introducing unsafe behavior is to write
manual type erasure wrappers like we do today.
Another possibility would be to allow extending the existential type
(not the protocol). This would allow you to write overloads on the
Collection existential that takes some kind of type erased index if
that is what you want and either trap if you receive an invalid index
or better (IMO) return an `Element?`. I’m not sure how extensions on
existentials might be implemented, but this is an example of the kind
of operation you might want available on it that you wouldn’t want
available on all Collection types.
Collection indices are already something that isn’t fully statically
safe so I understand why you might want to allow this.
By the same measure, so are Ints :-)
The fact that a type's methods have preconditions does *not* make it
“statically unsafe.”
That depends on what you mean by safe. Sure, those methods aren’t
going corrupt memory, but they *are* going to explicitly and
intentionally crash for some inputs. That doesn’t qualify as “fully
safe” IMO.
Please pick a term other than “unsafe” here; it's not unsafe in the
sense we mean the word in Swift. It's safe in exactly the same way that
array indexes and integers are. When you violate a precondition, it
traps.
Safe. The most obvious way to write code should also behave in a safe
manner. Undefined behavior is the enemy of safety, and developer
mistakes should be caught before software is in production. Opting for
safety sometimes means Swift will feel strict, but we believe that
clarity saves time in the long run.
Safety
Swift was designed from the outset to be safer than C-based languages,
and eliminates entire classes of unsafe code. Variables are always
initialized before use, arrays and integers are checked for overflow,
and memory is managed automatically. Syntax is tuned to make it easy
to define your intent — for example, simple three-character keywords
define a variable (var) or constant (let).
Another safety feature is that by default Swift objects can never be
nil, and trying to make or use a nil object will results in a
compile-time error. This makes writing code much cleaner and safer,
and prevents a common cause of runtime crashes. However, there are
cases where nil is appropriate, and for these situations Swift has an
innovative feature known as optionals. An optional may contain nil,
but Swift syntax forces you to safely deal with it using ? to indicate
to the compiler you understand the behavior and will handle it safely.
This positioning statement makes it appear as if preventing common
causes of crashes falls within the meaning of safe that Swift is
using. Having existentials introduce new preconditions and traps when
they are not met does not seem aligned with that goal IMO.
Static typing “increases safety,” in the casual sense. That doesn't
mean that an operation that traps on a failed precondition check is
“unsafe.”
The user doesn't do anything “manual” to introduce that trapping
behavior for integers. Preconditions are a natural part of most types.
The user doesn’t, but isn’t the overflow trap implemented in the
standard library?
Whether it is or is not is an implementation detail.
Regardless, this is a specific case that has been given explicit
design attention by humans. The precondition is designed, not
introduced by compiler rules that haven’t considered the specific case
in question.
But I don’t think having the language's existentials do this
automatically is the right approach. Maybe there is another
approach that could be used in targeted use cases where the less
safe behavior makes sense and is carefully designed.
Whether it makes sense or not really depends on the use-cases. There's
little point in generalizing existentials if the result isn't very useful.
Usefulness depends on your perspective.
Of course. As I've said, let's look at the use cases.
Agree. We can consider those in depth when the time comes to ramp up
discussion of Austin’s proposal.
I have run into several scenarios where they would be very useful
without needing to be prone to crashes when used incorrectly. One
obvious basic use case is storing things in a heterogenous collection
where you bind .
bind what?
Sorry, I must have gotten distracted and not finished that paragraph.
I meant to say bind the associated types that are necessary for your
use case. Sometimes you bind *all* of the associated types to
concrete types and the protocol has no `Self` requirements. In that
case there is no trouble at all in conforming the type-erased
“existential" to the protocol itself. Austin’s proposal would
eliminate the need to manually write these “existentials” manually.
The way to find out is to take a look at the examples we currently have
of protocols with associated types or Self requirements and consider
what you'd be able to do with their existentials if type relationships
couldn't be erased.
We have known use-cases, currently emulated in the standard library, for
existentials with erased type relationships. *If* these represent the
predominant use cases for something like generalized existentials, it
seems to me that the language feature should support that. Note: I have
not seen anyone build an emulation of the other kind of generalized
existential. My theory: there's a good reason for that :-).
AFAIK (and I could be wrong) the only rules in the language that
require the compiler to synthesize a trap except using a nil IUO, `!`
on a nil Optional, and an invalid `as` cast . These are all
syntactically explicit unsafe / dangerous operations. All other traps
are in the standard library (array index, overflow, etc). Most
important about all of these cases is that they have received direct
human consideration.
There is no distinction in the user model between what might be
synthesized by the language and what appears on standard library types.
Maybe I shouldn’t have made that distinction.
The point I am trying to emphasize is that each of these are special
cases that have received direct human consideration. The potential
for a trap is not introduced by language rules that apply to
user-defined constructs in without consideration of the specific
details of that construct.
Introducing a language (not library) mechanism that exposes members on
generalized existentials in a way that relies on runtime traps for
type safety feels to me like a pretty dramatic turn agains the stated
priority of safety. It will mean you must understand exactly what is
going on and be extremely careful to use generalized existentials
without causing crashes. This will either make Swift code much more
crashy or will scare people away from using generalized existentials
(and maybe both).
I don't accept either of those statements without seeing some analysis
of the use-cases. For example, I don't believe that AnyCollection et al
are particularly crash-prone. The likelihood that you'll use the wrong
index type with a collection is very, very low. I'm less certain of
what happens with Self requirements in real cases.
But again, I believe this is an exceptional case as the precondition
is explicitly stated in the semantics of the protocol.
IIUC, it has been cited by Doug as the exemplar of the
predominantly-requested case by a 10:1 ratio!
In terms of forming the existential, storing it in variables,
accepting arguments of that type, etc yes. I don’t know how many of
those requests expect it to conform to the protocol and expect to be
able to use it in generic code constrained to the protocol.
IMO the burden of proof should be on the side that proposes a
mechanism to introduce traps, not the side that proposes avoiding
them.
If you really want to make this about sides and burdens, the burden of
proof always rests with the side proposing to extend the language. We
shouldn't be making changes without understanding how they will play out
in real use-cases.
I agree with this. But if we are discussing two different options for
extending the language I think the option that doesn’t introduce
crashes should be preferred without pretty compelling reasons to
choose the option that can introduce crashes.
Neither of those outcomes is good.
Collection indices are a somewhat special case as there is already a
strong precondition that people are familiar with because it would be
too costly to performance and arguably too annoying to deal with an
Optional result in every array lookup. IMO that is why the library is
able to get away with it in the current type erased AnyCollection.
But this is not a good model for exposing any members on an
existential that do not already have a strong precondition that causes
a trap when violated.
I think a big reason why you maybe haven’t seen a lot of examples of
people writing type erased “existentials" is because it is a huge pain
in the neck to write this stuff manually today. People may be
designing around the need for them. I haven’t seen a huge sampling of
type erased “existentials" other people are writing but I haven’t
written any that introduce a trap like this. The only traps are in
the “abstract" base class whose methods will never be called (and
wouldn’t even be implemented if they could be marked abstract).
What specific things do you think we need to be able to do that rely
on the compiler synthesizing a trap in the way it exposes the members
of the existential?
I don't know. I'm saying, I don't think we understand the use-cases
well enough to make a determination.
That’s fair. I agree that use cases should be carefully considered.
Here are a few examples from Austin’s proposal that safely use
existential collections. I don’t understand why you think this
approach is insufficient. Maybe you could supply a concrete example
of a use case that can’t be written with the mechanism in Austin’s
proposal.
// A variable whose type is the Index associated type of the underlying
// concrete type of 'a'.
let theIndex : a.Index = ...
// A variable whose type is the Element associated type of the underlying
// concrete type of 'a'.
let theElement : a.Element = ...
// Given a mutable collection, swap its first and last items.
// Not a generic function.
func swapFirstAndLast(inout collection: Any<BidirectionalMutableCollection>) {
// firstIndex and lastIndex both have type "collection.Index"
guard let firstIndex = collection.startIndex,
lastIndex = collection.endIndex?.predecessor(collection) where lastIndex != firstIndex else {
print("Nothing to do")
return
}
// oldFirstItem has type "collection.Element"
let oldFirstItem = collection[firstIndex]
var a : Any<BidirectionalMutableCollection where .Element == String> = ...
let input = "West Meoley"
// Not actually necessary, since the compiler knows "a.Element" is String.
// A fully constrained anonymous associated type is synonymous with the concrete
// type it's forced to take on, and the two are interchangeable.
// However, 'as' casting is still available if desired.
let anonymousInput = input as a.Element
a[a.startIndex] = anonymousInput
// as mentioned, this also works:
a[a.startIndex] = input
// If the collection allows it, set the first element in the collection to a given string.
func setFirstElementIn(inout collection: Any<Collection> toString string: String) {
if let element = string as? collection.Element {
// At this point, 'element' is of type "collection.Element"
collection[collection.startIndex] = element
}
}
Neither of these look like they actually make *use* of the fact that
there's type erasure involved (and therefore should probably be written
as generics?). The interesting cases with Any<Collection...>, for the
purposes of this discussion, arise when you have multiple instances of
the same existential type that wrap different concrete types.
One use case I have found is to work around the lack of higher-kinder
types.
Really, now: a use-case for feature A that is a workaround for the lack
of feature B hardly justifies adding feature A! We do want to add
higher-kinded types eventually.
Good to know. I thought higher-kinder types were on the “maybe if
someone shows a compelling enough use case” list. AFAIK this is the
first time a member of the core team has stated the intent to add
them.
Well, please don't take this as a formal statement on behalf of the
team. IIUC, the team is generally interested in having this feature.
Of course not. The only official plan of record for Swift is approved
proposals. But it’s always interesting to know the opinions of the
core team about potential features. Previously my impression had been
that the general leaning was towards skepticism that the practical
benefits of HKT would pay for the complexity.
That's certainly a valid question. I have been very vocally skeptical
that HKTs like Monad belong in the standard library, but I am not
opposed to having the language feature if it supports important use
cases.
That makes sense. Language support is where we get expressive power. I think you are right to challenge placing specific protocols in the standard library.
In the case of Monad in particular, Swift has a much different design direction and feature set than languages where it plays a central role. I have an open mind on this but the case for putting it in the standard library definitely isn't obvious. Concrete, practical and useful examples of generic code that would be written in terms of Monad are required. And they should make it clear why this is a good approach in Swift. It is not hard to show this for concrete monads, but pretty difficult to do so for generic code that only depends on the Monad abstraction.
I haven't come across any good examples yet so I share your skepticism on this until I see them.
···
Sent from my iPad
On Jun 10, 2016, at 7:46 PM, Dave Abrahams via swift-evolution <swift-evolution@swift.org> wrote:
on Thu Jun 09 2016, Matthew Johnson <swift-evolution-m3FHrko0VLzYtjvyW6yDsg@public.gmane.org> wrote:
On Jun 9, 2016, at 3:05 PM, Dave Abrahams <dabrahams-2kanFRK1NckAvxtiuMwx3w@public.gmane.org> wrote:
on Thu Jun 09 2016, Matthew Johnson <matthew-AT-anandabits.com> wrote:
On Jun 9, 2016, at 11:42 AM, Dave Abrahams <dabrahams-2kanFRK1NckAvxtiuMwx3w@public.gmane.org> wrote:
on Thu Jun 09 2016, Matthew Johnson <matthew-AT-anandabits.com <http://matthew-at-anandabits.com/>> wrote:
On Jun 8, 2016, at 1:33 PM, Dave Abrahams >>>>>>>>> <dabrahams@apple.com >>>>>>>>> <mailto:dabrahams@apple.com>> >>>>>>>>> wrote:
on Tue Jun 07 2016, Matthew Johnson <matthew-AT-anandabits.com <http://matthew-at-anandabits.com/>> wrote:
On Jun 7, 2016, at 4:13 PM, Dave Abrahams via >>>>>>>>>>>>> swift-evolution <swift-evolution-m3FHrko0VLzYtjvyW6yDsg@public.gmane.org >>>>>>>>>>>>> <mailto:swift-evolution-m3FHrko0VLzYtjvyW6yDsg@public.gmane.org>> wrote:
on Tue Jun 07 2016, Matthew Johnson >>>>>>>>>>>>> <swift-evolution@swift.org >>>>>>>>>>>>> <mailto:swift-evolution-m3FHrko0VLzYtjvyW6yDsg@public.gmane.org>> >>>>>>>>>>>>> wrote:
Ah, thanks, I forgot! I still consider this a bug, though (will have
to read up again what the reasons are for that behavior).
Yes, but in the case of the issue we're discussing, the choices are:
1. Omit from the existential's API any protocol requirements that depend
on Self or associated types, in which case it *can't* conform to
itself because it doesn't fulfill the requirements.
2. Erase type relationships and trap at runtime when they don't line up.
Matthew has been arguing against #2, but you can't “fix the bug” without
it.
#1 has been my preference for a while as well, at least as a starting
point.
I should point out that with the resyntaxing of existentials to
Any<Protocols...>, the idea that Collection's existential doesn't
conform to Collection becomes far less absurd than it was, so maybe this
is not so bad.
I think the problem is more that Any<Collection> does not conform to a specific value for a type parameter T: Collection
What I mean by this is that `Collection` denotes a type family, a generic parameter `T: Collection` denotes a specific (though unknown) member of that type family and `Any<Collection>` denotes the type family again, so there is really no point in writing Any<Collection> IMO.
The type family cannot conform to T because T is just one fixed member of it.
It conforms to itself, though, as I can write
let c1: Any<Collection> = …
let c2: Any<Collection> = c1
That’s why I think that we could just drop Any<Collection> and simply write Collection.
Let me expand that a bit:
Actually all this talk about existentials vs. generics or protocols vs. classes has had me confused somewhat and I think there are still some misconceptions present on this list sometimes, so I’ll try to clear them up:
(1) misconception: protocols with associated types are somehow very different from generics
I don’t think they are and I will explain why. The only difference is the way the type parameters are bound: generics use explicit parameter lists whereas protocols use inheritance. That has some advantages (think long parameter lists of generics) and some disadvantages.
These ways are dual in a notation sense: generic types have to have all parameters bound whereas protocols cannot bind any of them.
The „existential“ notation `Any<>` being discussed on this list is nothing more than adding the ability to protocols to bind the parameters to be used just like Java’s wildcards are adding the opposite feature to generics, namely not having to bind all parameters.
As you know I like using `&` as type intersection operator. But you write "The syntax leave a void when it comes to expressing the so called Top type:“
Why? Just give the top type a name, e.g. `Any` and you are done. Why should the top type have special *syntax*? It is just the type all other types conform to. No need to do something special here and therefore no need to invent an alternative syntax like `Any<>` or the alternative from your gist which is rather confusing IMO (and I don’t like the special case given to classes in the syntax).
Essentially `Any<Collection>` in Swift is just the same as `Collection<?>` in Java (assuming for comparability’s sake that Swift’s Collection had no additional associated types; otherwise I would just have to introduce a Collection<Element, Index> in Java).
Likewise `Any<Collection where .Element: Number>` is just the same as `Collection<? extends Number>` in Java.
Java’s wildcards are a way to express use-site variance. The proposed `Any<>` does just the same.
And just like Collection<?> does not conform to a type parameter `T extends Collection<?>` because Collection<?> is the type `forall E. Collection<E>` whereas `T extends Collection<?>` is the type `T. Collection<T>` for a given T.
This picture is accurate today, but there are going to be more serious differences after 10 no date is currently geven for when it will come)
You mean Java 10?
In essence protocols with associated types are like generics with wildcards.
Yes, java has kept everything within its generics system rather than split parts out. Something people may not immediately think about iwith respect to the 2 generic systems is that when u call a func<T>capture(T t){} in java with a wildcard you are doing a compile time capture only (to avoid the dreaded unsafe casts), whereas it is really nice to do the same in swift and subsequently be able to access T.Type and see that it is not Any. The closest u ever get to that type at runtime in java is via generics introspection, but u still can't do everything ( like no new T() ). But today the bridging between existential types and generics is definitely a work in progress.
Coming back to the questions whether (a) allowing existentials to be used as types is useful and (b) whether sacrificing type safety would somehow be necessary for that, I think we can safely answer
(a) yes, it *is* useful to be able to use existentials like Any<Collection> as types, because wildcards are quite often needed and very useful in Java (they haven’t been added without a reason)
IMO they made java 8 (referring to streams). And even though the syntax for co/contra variance is pretty heavy, it is the foundation for all modern java code. The any-fication of the generics is going to open new doors as some of it will translate into A partial reification in the jvm. It seems the decision for now the decision is to not use the extra info in java to retain binary compatibility with all the erased code out there, this is something scala might use in areas where it won't mind loosing java compatibility.
(b) no, sacrificing type safety does not make sense, as the experience with Java’s wildcards shows that this is not needed. Especially if something like path dependent types is used like proposed and some notation to open an existential’s type is added, which is both something that Java does not have.
I hope sypesafe opening inside the " if let " syntax gets added. I know that chris is against sugaring, but I played if an implementation of
... x is String?
If let! x {}
That runs as
if let x = x {}
something equally short could be done here.
(2) misconception: POP is different from OOP
It is not. Protocols are just interfaces using subtyping like OOP has always done. They just use associated types instead of explicit type parameters for generics (see above). The more important distinction of Swift is emphasizing value types and making mutation safely available by enforcing copy semantics for value types.
Values are coming to the jvm, which will narrow this gap (like their view identity for value vs ref and the whole deep ==) . I also really like the cow approach of the swift runtime.
But protocols are not really different from interfaces in Java.
There is one big difference: default methods, but it seems swift will add that soon.
Swift already has default extension methods, doesn’t it?
I also really like how extensions and conformance mix together in swift to bake retro-modelling in and the adapter pattern (spent enough years deep diving inside eclipse to appreciate it).
I would have preferred a unified model using just classes with real multiple inheritance like Eiffel has and value types just being a part of that similar to Eiffel’s `expanded` classes. But that ship has probably sailed a long time ago :-/
I like extensions very much (having used Smalltalk for a long time). I like enums for the pattern matching and structs as value types. But having to split protocols off instead of using abstract classes makes things more complicated IMO.
-1 i am old school c/c++... i really like protocol, struct, class, extensions, enums. It is a really nice mix that gives objc people room to grow, but I do miss how they are an integral part of generics (i protocols as a replacement and look forward to when they interact better) and namespaces+scoped-imports (c#)... Looking forward to where things go next
Yeah, namespacing/submodules/conflict resolution (when doing imports but also when conforming to multiple protocols which has just the same problems) are still missing. But I’m optimistic :-) Let’s complete generics first, then tackle existentials/type intersections.
-THorsten
···
Am 11.06.2016 um 08:00 schrieb L. Mihalkovic <laurent.mihalkovic@gmail.com>:
On Jun 10, 2016, at 9:35 PM, Thorsten Seitz via swift-evolution <swift-evolution@swift.org> wrote:
Am 09.06.2016 um 19:50 schrieb Thorsten Seitz via swift-evolution <swift-evolution@swift.org>:
Am 09.06.2016 um 18:49 schrieb Dave Abrahams via swift-evolution <swift-evolution@swift.org>:
on Wed Jun 08 2016, Jordan Rose <swift-evolution@swift.org> wrote:
On Jun 8, 2016, at 13:16, Dave Abrahams via swift-evolution >>>>>> <swift-evolution@swift.org> wrote:
on Wed Jun 08 2016, Thorsten Seitz >>>>> >>>>>> <swift-evolution@swift.org >>>>>> <mailto:swift-evolution@swift.org>> >>>>>> wrote:
So be it. But at least there should be no reasons for POP vs OOP wars ;-)
(I’d like to add that I liked Dave’s talks at last WWDC very much, it’s just that I don’t think that POP is something new or different.)
I used to thin that way. But today I think that although in broad brush strokes the similarities and bigger than the differences, there is room for making a bigger difference in the how.
Of course not. The only official plan of record for Swift is approved
proposals. But it’s always interesting to know the opinions of the
core team about potential features. Previously my impression had been
that the general leaning was towards skepticism that the practical
benefits of HKT would pay for the complexity.
That's certainly a valid question. I have been very vocally skeptical
that HKTs like Monad belong in the standard library, but I am not
opposed to having the language feature if it supports important use
cases.
It is surprising to see people debating the place of monads in Swift when nobody seems to speak about the elephant in the middle of the room: the language has a real engineering defisciency if it is possible for people to think that there is no better way to use it than to do "we have around 50+ packages. This list grows very fast" (http://docs.zewo.io - many more swift github projects are equally struggling to get the code organized).
At the moment the only thing preventing the situation from getting more out of hands is that the import mechanism currently gets in the way, but the only opinions I read so far all focussed on making it even easier to slap things together ala zewo rather than address the engineering issue.
Ah, thanks, I forgot! I still consider this a bug, though (will have
to read up again what the reasons are for that behavior).
Yes, but in the case of the issue we're discussing, the choices are:
1. Omit from the existential's API any protocol requirements that depend
on Self or associated types, in which case it *can't* conform to
itself because it doesn't fulfill the requirements.
2. Erase type relationships and trap at runtime when they don't line up.
Matthew has been arguing against #2, but you can't “fix the bug” without
it.
#1 has been my preference for a while as well, at least as a starting
point.
I should point out that with the resyntaxing of existentials to
Any<Protocols...>, the idea that Collection's existential doesn't
conform to Collection becomes far less absurd than it was, so maybe this
is not so bad.
I think the problem is more that Any<Collection> does not conform to
a specific value for a type parameter T: Collection
What I mean by this is that `Collection` denotes a type family, a
generic parameter `T: Collection` denotes a specific (though
unknown) member of that type family and `Any<Collection>` denotes
the type family again, so there is really no point in writing
Any<Collection> IMO.
The type family cannot conform to T because T is just one fixed member of it.
It conforms to itself, though, as I can write
let c1: Any<Collection> = …
let c2: Any<Collection> = c1
That’s why I think that we could just drop Any<Collection> and simply write Collection.
Let me expand that a bit:
Actually all this talk about existentials vs. generics or protocols
vs. classes has had me confused somewhat and I think there are still
some misconceptions present on this list sometimes, so I’ll try to
clear them up:
There are several objectively incorrect statements here, and several
others with which I disagree. I was hoping someone else would write
this for me, but since the post has such a tone of authority I feel I
must respond.
(1) misconception: protocols with associated types are somehow very
different from generics
I don’t think they are and I will explain why. The only difference is
the way the type parameters are bound: generics use explicit parameter
lists whereas protocols use inheritance. That has some advantages
(think long parameter lists of generics) and some disadvantages.
These ways are dual in a notation sense: generic types have to have
all parameters bound whereas protocols cannot bind any of them.
The „existential“ notation `Any<>` being discussed on this list is
nothing more than adding the ability to protocols to bind the
parameters to be used just like Java’s wildcards are adding the
opposite feature to generics, namely not having to bind all
parameters.
Protocols and generics fulfill completely different roles in Swift, and
so, **especially in a language design context like the one we're in
here**, must be thought of differently. The former are an abstraction
mechanism for APIs, and the latter a mechanism for generalizing
implementations. The only place you could argue that they intersect is
in generic non-final classes, because a class fills the dual role of
abstraction and implementation mechanism (and some might say that's a
weakness). But even accounting for generic classes, protocols with
associated types are very different from generics. Two utterly
different types (an enum and a struct, for example) can conform to any
given protocol P, but generic types always share a common basis
implementation. There is no way to produce distinct instances of a
generic type with all its type parameters bound, but for any protocol P
I can make infinitely many instances of P with P.AssociatedType == Int.
Back to the my original point: while protocols and generic types have
some similarities, the idea that they are fundamentally the same thing
(I know you didn't say *exactly* that, but I think it will be read that
way) would be wrong and a very unproductive way to approach language
evolution.
Essentially `Any<Collection>` in Swift is just the same as
`Collection<?>` in Java (assuming for comparability’s sake that
Swift’s Collection had no additional associated types; otherwise I
would just have to introduce a Collection<Element, Index> in Java).
I don't see how you can use an example that requires *assuming away*
assoociated types to justify an argument that protocols *with associated
types* are the same as generics.
Likewise `Any<Collection where .Element: Number>` is just the same as
`Collection<? extends Number>` in Java.
And just like Collection<?> does not conform to a type parameter `T
extends Collection<?>` because Collection<?> is the type `forall
E. Collection<E>` whereas `T extends Collection<?>` is the type
`T. Collection<T>` for a given T.
In essence protocols with associated types are like generics with
wildcards.
It is true that generics with wildcards in Java *are* (not just “like”)
existential types but I don't agree with the statement above. Because
Java tries to create an “everything is a class” world, generic classes
with bound type parameters end up playing the role of existential type.
But protocols in Swift are not, fundamentally, just existential types,
and the resyntaxing of ProtocolName to Any<ProtocolName> for use in type
context is a huge leap forward in making that distinction clear... when
that's done (unless we leave Array<ProtocolName> around as a synonym for
Array<Any<ProtocolName>>—I really hope we won't!) protocols indeed
*won't* be types at all, existential or otherwise.
Coming back to the questions whether (a) allowing existentials to be
used as types is useful
That's the only use existentials have. They *are* types. Of course
they're useful, and I don't think anyone was arguing otherwise.
and (b) whether sacrificing type safety would somehow be necessary for
that, I think we can safely answer (a) yes, it *is* useful to be able
to use existentials like Any<Collection> as types, because wildcards
are quite often needed and very useful in Java (they haven’t been
added without a reason) (b) no, sacrificing type safety does not make
sense, as the experience with Java’s wildcards shows that this is not
needed.
I would call this “interesting information,” but hardly conclusive.
Java's generics are almost exactly the same thing as Objective-C
lightweight generics, which are less capable and less expressive in
many ways than Swift's generics.
Especially if something like path dependent types is used like
proposed and some notation to open an existential’s type is added,
which is both something that Java does not have.
(2) misconception: POP is different from OOP
It is not. Protocols are just interfaces using subtyping like OOP has
always done. They just use associated types instead of explicit type
parameters for generics (see above).
They are not the same thing at all (see above ;->). To add to the list
above, protocols can express fundamental relationships—like Self
requirements—that OOP simply can't handle. There's a reason Java can't
express Comparable without losing static type-safety. Finally, in a
language with first-class value types, taking a protocol-oriented
approach to abstraction leads to *fundamentally* different designs from
what you get using OOP.
The more important distinction of Swift is emphasizing value types and
making mutation safely available by enforcing copy semantics for value
types.
We don't, in fact, enforce copy semantics for value types.
It was again clarified that Copy on Write was only automatically added for Strings and Arrays, and subclasses I suppose, but how else are structs passed around if not by copy? If they are passed around by reference and people are starting, being encouraged to do so, to massively make use of structs... will enforcing copy semantics (even with opt-in CoW behaviour) be a massive cause of performance regression in games and similar kinds of tight processing loops?
That's
something I'd like to change. But regardless, value types would be a
*lot* less useful if they couldn't conform to protocols, and so they
would be a lot less used. Heck, before we got protocol extensions in
Swift 2, there was basically *no way* to share implementation among
value types.
Ah so it is their (value types) fault our beautiful abstract API contracts meant to decouple behaviour from implementation now contain code and this implementation ;)?
The other day I was revisiting the rationale by Java engineers for default methods in Java interfaces, which I always used to compare Objective-C protocols to, and beside from the ease of adding code to a lot of pre-existing objects without forcing developers using a library to have to do much to adapt to a new update of said library... Well, I did not actually find much beyond convenience.
In Swift, they are already enabling structs to do something you could have only done with classes before (sharing implementation): in a way it is kind of forcing people to use composition over subclassing without as much effort as before.
Still, I appreciated how clear dispatching/overriding rules were designed (although they may be limiting for some use cases people here might conceive):
When you extend an interface that contains a default method, you can do the following:
* Not mention the default method at all, which lets your extended interface inherit the default method.
* Redeclare the default method, which makes it abstract.
* Redefine the default method, which overrides it.
I find that quite intuitive to adapt to. What I would like to ask you is this: don't you feel it is dangerous or at least unintuitive that casting generally does not allow to change how the object behaves (it does not interfere with the object's implementation of its API contract) while if you are using default methods in protocol extensions you are suddenly allowing casting to change the class implementation?
Are extensions and in particular default methods meant more for value types, implementation sharing too as you were saying, than classes?
···
Sent from my iPhone
On 13 Jun 2016, at 03:04, Dave Abrahams via swift-evolution <swift-evolution@swift.org> wrote:
on Fri Jun 10 2016, Thorsten Seitz <tseitz42-AT-icloud.com> wrote:
Am 09.06.2016 um 19:50 schrieb Thorsten Seitz via swift-evolution <swift-evolution@swift.org>:
Am 09.06.2016 um 18:49 schrieb Dave Abrahams via swift-evolution <swift-evolution@swift.org>:
on Wed Jun 08 2016, Jordan Rose <swift-evolution@swift.org> wrote:
On Jun 8, 2016, at 13:16, Dave Abrahams via swift-evolution >>>>>> <swift-evolution@swift.org> wrote:
on Wed Jun 08 2016, Thorsten Seitz >>>>> >>>>>> <swift-evolution@swift.org >>>>>> <mailto:swift-evolution@swift.org>> >>>>>> wrote:
So you can't take protocols out of the picture without
making value types, and the argument for value semantics, far weaker.
But protocols are not really different from interfaces in Java.
I would have preferred a unified model using just classes with real
multiple inheritance like Eiffel has and value types just being a part
of that similar to Eiffel’s `expanded` classes. But that ship has
probably sailed a long time ago :-/ So be it. But at least there
should be no reasons for POP vs OOP wars ;-) (I’d like to add that I
liked Dave’s talks at last WWDC very much, it’s just that I don’t
think that POP is something new or different.)
Protocol-oriented programming is about the synergy of features and ideas
most of which not *individually* new, but that together create a new
world of possibilities. I've already discussed the synergy of protocols
and first-class value semantics. There's also the fact that in
protocols we have one construct with which to express dynamic
polymorphism (existentials) and static polymorphism (generic
constraints), both of which have important roles to play but that I
maintain are very different indeed. One result is that you can “start
with a protocol” as your abstraction mechanism and know that you're not
going to design yourself into a corner where it becomes impossibly
awkward to express what you need. Finally—and I'm certain this *is*
new—in protocol extensions we have a means to express both post-hoc
conformance and generic functions that is much more accessible to users
than in any previous language, to the point where generic programming
can become a natural part of everyday work.
I had decided to watch from the sideline, but a couple things push me to say something
Ah, thanks, I forgot! I still consider this a bug, though (will have
to read up again what the reasons are for that behavior).
Yes, but in the case of the issue we're discussing, the choices are:
1. Omit from the existential's API any protocol requirements that depend
on Self or associated types, in which case it *can't* conform to
itself because it doesn't fulfill the requirements.
2. Erase type relationships and trap at runtime when they don't line up.
Matthew has been arguing against #2, but you can't “fix the bug” without
it.
#1 has been my preference for a while as well, at least as a starting
point.
I should point out that with the resyntaxing of existentials to
Any<Protocols...>, the idea that Collection's existential doesn't
conform to Collection becomes far less absurd than it was, so maybe this
is not so bad.
I think the problem is more that Any<Collection> does not conform to
a specific value for a type parameter T: Collection
What I mean by this is that `Collection` denotes a type family, a
generic parameter `T: Collection` denotes a specific (though
unknown) member of that type family and `Any<Collection>` denotes
the type family again, so there is really no point in writing
Any<Collection> IMO.
The type family cannot conform to T because T is just one fixed member of it.
It conforms to itself, though, as I can write
let c1: Any<Collection> = …
let c2: Any<Collection> = c1
That’s why I think that we could just drop Any<Collection> and simply write Collection.
Let me expand that a bit:
Actually all this talk about existentials vs. generics or protocols
vs. classes has had me confused somewhat and I think there are still
some misconceptions present on this list sometimes, so I’ll try to
clear them up:
There are several objectively incorrect statements here, and several
others with which I disagree. I was hoping someone else would write
this for me, but since the post has such a tone of authority I feel I
must respond.
(1) misconception: protocols with associated types are somehow very
different from generics
I don’t think they are and I will explain why. The only difference is
the way the type parameters are bound: generics use explicit parameter
lists whereas protocols use inheritance. That has some advantages
(think long parameter lists of generics) and some disadvantages.
These ways are dual in a notation sense: generic types have to have
all parameters bound whereas protocols cannot bind any of them.
The „existential“ notation `Any<>` being discussed on this list is
nothing more than adding the ability to protocols to bind the
parameters to be used just like Java’s wildcards are adding the
opposite feature to generics, namely not having to bind all
parameters.
Protocols and generics fulfill completely different roles in Swift, and
so, **especially in a language design context like the one we're in
here**, must be thought of differently.
The temptation is great to mention your own words from 2+ years ago regarding the choice of the name Protocol instead of Interface, and the ‘hope’ (your word) that the differentiation would wind-up being big enough to justify the new name. Wanting to view them in a different light is one thing, succeeding in establishing a gap wide enough that people with superficial or intimate knowledge of both would come to the unavoidable conclusion that they are fundamentally different, is an entirely different exercise.
At this point I think it is fair to say that the differences are notable but, perhaps out of lack of familiarity with the details, not quite strong enough to make all developers reach the conclusion you’d like them to reach.
The former are an abstraction
mechanism for APIs, and the latter a mechanism for generalizing
implementations. The only place you could argue that they intersect is
in generic non-final classes, because a class fills the dual role of
abstraction and implementation mechanism (and some might say that's a
weakness). But even accounting for generic classes, protocols with
associated types are very different from generics. Two utterly
different types (an enum and a struct, for example) can conform to any
given protocol P, but generic types always share a common basis
implementation. There is no way to produce distinct instances of a
generic type with all its type parameters bound, but for any protocol P
I can make infinitely many instances of P with P.AssociatedType == Int.
somehow I think both your arguments contain some truth. You are pointing out the differentiating fact that a Protocol is abstracting across multiple concrete types, while he is showing that the mechanism in which associated types in a Protocol is reminiscent of the way a type T operates on a generic type. The two are not mutually exclusive.
In all cases, "A is to B what 1 is to 2” has never meant that 1 and A are the same, it simply means that the relationship inside each pair operates in a similar fashion to the relationship within the other pair; which again is very different from saying that the relationships are the same. In this context, I think that there is a strong case for wanting to further bridge the gap, as was even pointed out by Doug in his Generics Manifesto. Not to mention that Java and a few others are there to demonstrate how far the similarities can be pushed.
Back to the my original point: while protocols and generic types have
some similarities, the idea that they are fundamentally the same thing
(I know you didn't say *exactly* that, but I think it will be read that
way) would be wrong and a very unproductive way to approach language
evolution.
Yes, similarities are just that, and I am absolutely with you to claim that they are NOT THE SAME, and it is unfortunate that analogies get so often confused with alike-ness. Although it is undeniably helpful for people to learn new concepts via their alike-ness to other concepts, it can also create the wrong mental picture. Unfortunately I think that 10s of thousands of years of evolution are conspiring to make our brains likely to jump from analogous to alike quite readily, wiping out critical differences in the process.
Essentially `Any<Collection>` in Swift is just the same as
`Collection<?>` in Java (assuming for comparability’s sake that
Swift’s Collection had no additional associated types; otherwise I
would just have to introduce a Collection<Element, Index> in Java).
I don't see how you can use an example that requires *assuming away*
assoociated types to justify an argument that protocols *with associated
types* are the same as generics.
You might have to write a lot of educational material on the language to cast the “the relationship between protocols and their associated types is analogous to the relationship of a generic type and its parameters” in stone, and dispel the “protocols are the same as generics” views. I do hope this is a topic you will dwell-on in your presentation at WWDC this week, openly presenting the similarities to better point out the critical differences !!!! :)
Likewise `Any<Collection where .Element: Number>` is just the same as
`Collection<? extends Number>` in Java.
And just like Collection<?> does not conform to a type parameter `T
extends Collection<?>` because Collection<?> is the type `forall
E. Collection<E>` whereas `T extends Collection<?>` is the type
`T. Collection<T>` for a given T.
In essence protocols with associated types are like generics with
wildcards.
It is true that generics with wildcards in Java *are* (not just “like”)
existential types but I don't agree with the statement above. Because
Java tries to create an “everything is a class” world,generic classes
with bound type parameters end up playing the role of existential type.
But protocols in Swift are not, fundamentally, just existential types,
and the resyntaxing of ProtocolName to Any<ProtocolName> for use in type
for as much as I would prefer the "var p : ProtocolName” and its extension to “var t : UITableView[UITableViewDataSource & UITableViewDelegate], I do understand wanting to surrender to Any<UITableView & UITableViewDelegate> to carve the difference with the Generics in the hardest marble. But I do think that over time people will forget the meaning and just get into the habit of writing code that will just work.
context is a huge leap forward in making that distinction clear... when
that's done (unless we leave Array<ProtocolName> around as a synonym for
Array<Any<ProtocolName>>—I really hope we won't!) protocols indeed
*won't* be types at all, existential or otherwise.
Coming back to the questions whether (a) allowing existentials to be
used as types is useful
That's the only use existentials have. They *are* types. Of course
they're useful, and I don't think anyone was arguing otherwise.
:) I sure hope not…
and (b) whether sacrificing type safety would somehow be necessary for
that, I think we can safely answer (a) yes, it *is* useful to be able
to use existentials like Any<Collection> as types, because wildcards
are quite often needed and very useful in Java (they haven’t been
added without a reason) (b) no, sacrificing type safety does not make
sense, as the experience with Java’s wildcards shows that this is not
needed.
I would call this “interesting information,” but hardly conclusive.
Java's generics are almost exactly the same thing as Objective-C
lightweight generics, which are less capable and less expressive in
many ways than Swift's generics.
This is the part that had me jump in my seat… With all due respect, I find this a very reductionist view. As things stand, Java generics covers both classes and interfaces equally, which the swift generics+protocol&assocType is far from doing. It has to be the case or this thread would not exist and Austin and myself would not have bothered to write long proposals to attempt to fill the gap.
Furthermore, Java & JVM are currently undergoing an effort to any-fy the generics system that will result in a number of new doors opening: Collection<any> that includes int, reification of generic parameters inside the .class, … when coupled with the other ongoing effort to add value types, then the difference with swift would be in the direction of swift having some serious catchup to do. I do hope that the current status-quo between Generics and Protocols will not remain what it is and that some form of the extended existential proposals (ok… I can’t resists: https://gist.github.com/lmihalkovic/68c321ea7ffe27e553e37b794309b051\) will have made it into swift by then.
I place existentials/dynamic polymorphism high on my TODO because looking at the evolution of Java in the last 3 years, the greatest leaps have come from this part of the language. But I do understand that when looking at where Apple developers come from with Objc, it may seem that that this form of abstraction may seem far away on their priority list. However IMHO designing swift to be a simple upgrade for objc might be missing the point that today, even Typescript has a very powerful way of abstracting over generic interfaces.
Especially if something like path dependent types is used like
proposed and some notation to open an existential’s type is added,
which is both something that Java does not have.
(2) misconception: POP is different from OOP
It is not. Protocols are just interfaces using subtyping like OOP has
always done. They just use associated types instead of explicit type
parameters for generics (see above).
They are not the same thing at all (see above ;->). To add to the list
above, protocols can express fundamental relationships—like Self
requirements—that OOP simply can't handle. There's a reason Java can't
express Comparable without losing static type-safety.
self requirements on interfaces would be a very nice addition to java…
Finally, in a
language with first-class value types, taking a protocol-oriented
approach to abstraction leads to *fundamentally* different designs from
what you get using OOP.
Looking at the code people actually write, I’m afraid this may be a tough battle to win…
The more important distinction of Swift is emphasizing value types and
making mutation safely available by enforcing copy semantics for value
types.
We don't, in fact, enforce copy semantics for value types. That's
something I'd like to change. But regardless, value types would be a
*lot* less useful if they couldn't conform to protocols, and so they
would be a lot less used. Heck, before we got protocol extensions in
Swift 2, there was basically *no way* to share implementation among
value types. So you can't take protocols out of the picture without
making value types, and the argument for value semantics, far weaker.
But protocols are not really different from interfaces in Java.
I would have preferred a unified model using just classes with real
multiple inheritance like Eiffel has and value types just being a part
of that similar to Eiffel’s `expanded` classes. But that ship has
probably sailed a long time ago :-/ So be it. But at least there
should be no reasons for POP vs OOP wars ;-) (I’d like to add that I
liked Dave’s talks at last WWDC very much, it’s just that I don’t
think that POP is something new or different.)
Protocol-oriented programming is about the synergy of features and ideas
most of which not *individually* new, but that together create a new
world of possibilities. I've already discussed the synergy of protocols
and first-class value semantics. There's also the fact that in
protocols we have one construct with which to express dynamic
polymorphism (existentials) and static polymorphism (generic
constraints), both of which have important roles to play but that I
maintain are very different indeed. One result is that you can “start
with a protocol” as your abstraction mechanism and know that you're not
going to design yourself into a corner where it becomes impossibly
awkward to express what you need. Finally—and I'm certain this *is*
new—in protocol extensions we have a means to express both post-hoc
conformance and generic functions that is much more accessible to users
than in any previous language, to the point where generic programming
can become a natural part of everyday work.
As a long term Erich Gama fan and Eclipse cave-diver, I do wholeheartedly agree that protocol extensions is an incredible feature!!! It would have done wonders for all the adapters I have had to write, to be able to do that in java directly. I hope you manage to make people see how powerful they are today, and make them even more powerful in the future. As you bring the topic, I had gathered some thoughts on methods dispatching in protocol extensions : [swift-evolution] [summary] Protocol extension method dispatch - was [Static Dispatch Pitfalls]
Looking forward to a presentation on POP/Extensions/Generics this week.
···
On Jun 13, 2016, at 4:04 AM, Dave Abrahams via swift-evolution <swift-evolution@swift.org> wrote:
on Fri Jun 10 2016, Thorsten Seitz <tseitz42-AT-icloud.com <http://tseitz42-at-icloud.com/>> wrote:
Am 09.06.2016 um 19:50 schrieb Thorsten Seitz via swift-evolution <swift-evolution@swift.org>:
Am 09.06.2016 um 18:49 schrieb Dave Abrahams via swift-evolution <swift-evolution@swift.org>:
on Wed Jun 08 2016, Jordan Rose <swift-evolution@swift.org> wrote:
On Jun 8, 2016, at 13:16, Dave Abrahams via swift-evolution >>>>>> <swift-evolution@swift.org> wrote:
on Wed Jun 08 2016, Thorsten Seitz >>>>> >>>>>> <swift-evolution@swift.org >>>>>> <mailto:swift-evolution@swift.org>> >>>>>> wrote:
Ah, thanks, I forgot! I still consider this a bug, though (will have
to read up again what the reasons are for that behavior).
Yes, but in the case of the issue we're discussing, the choices are:
1. Omit from the existential's API any protocol requirements that depend
on Self or associated types, in which case it *can't* conform to
itself because it doesn't fulfill the requirements.
2. Erase type relationships and trap at runtime when they don't line up.
Matthew has been arguing against #2, but you can't “fix the bug” without
it.
#1 has been my preference for a while as well, at least as a starting
point.
I should point out that with the resyntaxing of existentials to
Any<Protocols...>, the idea that Collection's existential doesn't
conform to Collection becomes far less absurd than it was, so maybe this
is not so bad.
I think the problem is more that Any<Collection> does not conform to
a specific value for a type parameter T: Collection
What I mean by this is that `Collection` denotes a type family, a
generic parameter `T: Collection` denotes a specific (though
unknown) member of that type family and `Any<Collection>` denotes
the type family again, so there is really no point in writing
Any<Collection> IMO.
The type family cannot conform to T because T is just one fixed member of it.
It conforms to itself, though, as I can write
let c1: Any<Collection> = …
let c2: Any<Collection> = c1
That’s why I think that we could just drop Any<Collection> and simply write Collection.
Let me expand that a bit:
Actually all this talk about existentials vs. generics or protocols
vs. classes has had me confused somewhat and I think there are still
some misconceptions present on this list sometimes, so I’ll try to
clear them up:
There are several objectively incorrect statements here, and several
others with which I disagree. I was hoping someone else would write
this for me, but since the post has such a tone of authority I feel I
must respond.
You are right, the tone of my post was not appropriate, for which I want to apologize sincerely.
I still believe my statements to be valid, though, and will respond to your arguments inline. Please don't get me wrong, I'm not trying to have an argument for the argument's sake. All I want is to contribute maybe a tiny bit to make Swift even better than it already is, by sharing ideas and thoughts not only from me but from the designs of other perhaps more obscure programming languages which I happen to have stumbled upon in the past (often with much delight).
(1) misconception: protocols with associated types are somehow very
different from generics
I don’t think they are and I will explain why. The only difference is
the way the type parameters are bound: generics use explicit parameter
lists whereas protocols use inheritance. That has some advantages
(think long parameter lists of generics) and some disadvantages.
These ways are dual in a notation sense: generic types have to have
all parameters bound whereas protocols cannot bind any of them.
The „existential“ notation `Any<>` being discussed on this list is
nothing more than adding the ability to protocols to bind the
parameters to be used just like Java’s wildcards are adding the
opposite feature to generics, namely not having to bind all
parameters.
Protocols and generics fulfill completely different roles in Swift, and
so, **especially in a language design context like the one we're in
here**, must be thought of differently. The former are an abstraction
mechanism for APIs, and the latter a mechanism for generalizing
implementations.
That's not what I was talking about. Of course, protocols are a mechanism for deriving types from each other whereas generics are a way to parameterize types. My point was that Swift's other way to parameterize types, namely by associated types, is very similar to generics with wildcards when looking a the existentials of such protocols. In addition I was talking about generics in general, not just about generics in Swift which restricts them to implementations and does not support wildcards.
Other languages like Java offer generics for interfaces as well and support wildcards (adding generic types parameters to protocols in Swift is currently discussed on the mailing list as well).
FWIW my arguments were not about whether we should have wildcards in Swift or not, but simply to relate one parametrization feature (associated types) to a more well known parametrization feature (generics with wildcards) in order to understand them better.
The only place you could argue that they intersect is
in generic non-final classes, because a class fills the dual role of
abstraction and implementation mechanism (and some might say that's a
weakness). But even accounting for generic classes, protocols with
associated types are very different from generics. Two utterly
different types (an enum and a struct, for example) can conform to any
given protocol P, but generic types always share a common basis
implementation.
The latter is not the case for generic interfaces in Java, for example, so it is just an artificial restriction present in Swift.
There is no way to produce distinct instances of a
generic type with all its type parameters bound,
That is true in Swift (except for generic classes) due to the restriction just mentioned.
but for any protocol P
I can make infinitely many instances of P with P.AssociatedType == Int.
This likewise applies to generic interfaces and for generic types in general if taking inheritance into account - just like you do here for protocols.
Back to the my original point: while protocols and generic types have
some similarities, the idea that they are fundamentally the same thing
(I know you didn't say *exactly* that, but I think it will be read that
way) would be wrong and a very unproductive way to approach language
evolution.
I said that protocols *with associated types* are much like generics *with wildcards* and tried to show why.
Essentially `Any<Collection>` in Swift is just the same as
`Collection<?>` in Java (assuming for comparability’s sake that
Swift’s Collection had no additional associated types; otherwise I
would just have to introduce a Collection<Element, Index> in Java).
I don't see how you can use an example that requires *assuming away*
assoociated types to justify an argument that protocols *with associated
types* are the same as generics.
Note, that I said *additional* associated types, i.e. in addition to .Element, even giving an example how the Java interface had to be extended by a type parameter `Index` if this assumption was not applied (still simplifying because Generator would have been more correct which would have to be added as type parameter in addition to `Index`).
So, in essence the comparison is between the following (I'm using Foo now instead of Collection to avoid the differences mentioned. Note that this has no impact on the argument at all):
protocol Foo {
associatedtype T
...
}
interface Foo<T> {
...
}
My argument is that existentials of protocols with associated types are just like generic types with wildcards, i.e. `Any<Foo>` in Swift is just the same as `Foo<?>` in Java.
Likewise `Any<Foo where .T: Number>` is just the same as `Foo<? extends Number>` in Java. For me that was an insight I wanted to share.
And just like Collection<?> does not conform to a type parameter `T
extends Collection<?>` because Collection<?> is the type `forall
E. Collection<E>` whereas `T extends Collection<?>` is the type
`T. Collection<T>` for a given T.
In essence protocols with associated types are like generics with
wildcards.
It is true that generics with wildcards in Java *are* (not just “like”)
existential types but I don't agree with the statement above. Because
Java tries to create an “everything is a class” world, generic classes
with bound type parameters end up playing the role of existential type.
But protocols in Swift are not, fundamentally, just existential types,
and the resyntaxing of ProtocolName to Any<ProtocolName> for use in type
context is a huge leap forward in making that distinction clear... when
that's done (unless we leave Array<ProtocolName> around as a synonym for
Array<Any<ProtocolName>>—I really hope we won't!) protocols indeed
*won't* be types at all, existential or otherwise.
I fully agree that protocols are not types, their existentials are. But I haven't seen yet what we really *gain* from making that distinction explicit (except an ugly type syntax :-).
And like I already wrote in this or another thread we would have to apply the same logic to non-final classes, which are existentials, too.
Coming back to the questions whether (a) allowing existentials to be
used as types is useful
That's the only use existentials have. They *are* types. Of course
they're useful, and I don't think anyone was arguing otherwise.
I'm pretty sure that there was a discussion about whether being able to write something like Any<Collection> is useful. My wording was certainly imprecise, though, and didn't make sense as written. I should have said something like "whether adding the ability to use existential types of protocols with unbound associated types is useful".
and (b) whether sacrificing type safety would somehow be necessary for
that, I think we can safely answer (a) yes, it *is* useful to be able
to use existentials like Any<Collection> as types, because wildcards
are quite often needed and very useful in Java (they haven’t been
added without a reason) (b) no, sacrificing type safety does not make
sense, as the experience with Java’s wildcards shows that this is not
needed.
I would call this “interesting information,” but hardly conclusive.
Java's generics are almost exactly the same thing as Objective-C
lightweight generics, which are less capable and less expressive in
many ways than Swift's generics.
I agree that Java does not have something like `Self` or associated types (which are really useful for not having to bind all type parameters explicitly, especially when binding type parameters to other generics which makes for long type parameter lists in Java where I have to repeat everything over and over again), but do you mean something else here?
Especially in the context of sacrificing type safety?
Especially if something like path dependent types is used like
proposed and some notation to open an existential’s type is added,
which is both something that Java does not have.
(2) misconception: POP is different from OOP
It is not. Protocols are just interfaces using subtyping like OOP has
always done. They just use associated types instead of explicit type
parameters for generics (see above).
They are not the same thing at all (see above ;->). To add to the list
above, protocols can express fundamental relationships—like Self
requirements—that OOP simply can't handle.
Eiffel has something like Self, it is called anchoring and allows binding the type of a variable to that of another one or self (which is called `Current` in Eiffel). And Eiffel does model everything with classes which may be abstract and allow for real multiple inheritance with abilities to resolve all conflicts including those concerning state (which is what other languages introduce interfaces for to avoid conflicts concerning state while still failing to solve *semantic* conflicts with the same diamond pattern).
No protocols or interfaces needed. Why do you say this is not OOP? The book which describes Eiffel is called "Object-Oriented Software Construction" (and is now about 20 years old).
There's a reason Java can't
express Comparable without losing static type-safety.
You are certainly right that Java is not the best language out there especially when talking about type systems (I often enough rant about it :-) but I'm not sure what you mean here. Java's Comparable<T> seems quite typesafe to me. Or do you mean that one could write `class A implements Comparable<B>` by mistake? That's certainly a weak point but doesn't compromise type safety, does it?
Ceylon has an elegant solution for that without using Self types:
interface Comparable<in Other> of Other given Other satisfies Comparable<Other> {...}
Note the variance annotation (which Swift currently has not) and the `of` which ensures that the only subtype of Comparable<T> is T. This is a nice feature that I haven't seen often in programming languages (only Cecil comes to mind IIRC) and which is used for enumerations as well in Ceylon. In Swift I cannot do this but can use Self which solves this problem differently, albeit with some drawbacks compared to Ceylon's solution (having to redefine the compare method in all subtypes, which has lead to lengthy discussion threads about Self, StaticSelf, self etc.).
Finally, in a
language with first-class value types, taking a protocol-oriented
approach to abstraction leads to *fundamentally* different designs from
what you get using OOP.
Eiffel has expanded types which are value types with copy semantics quite like structs in Swift. These expanded types are pretty much integrated into Eiffel's class-only type system. Just define a class as `expanded` and you are done. Eiffel seems to have no need to introduce interfaces or protocols to the language to support value types.
You can even derive from expanded classes which is currently not possible in Swift but has already been discussed several times on this mailing list.
Polymorphic usage is only possible for non expanded super types, which means as far as I understood that a reference is used in that case. Variables with an expanded type do not use refences and therefore may not be used polymorphically in Eiffel.
This should be similar in Swift, at least as far as I did understand it. The question whether variables with a value type can be used polymorphically currently does not arise in Swift as structs cannot inherit from each other (yet?).
The more important distinction of Swift is emphasizing value types and
making mutation safely available by enforcing copy semantics for value
types.
We don't, in fact, enforce copy semantics for value types. That's
something I'd like to change. But regardless, value types would be a
*lot* less useful if they couldn't conform to protocols, and so they
would be a lot less used. Heck, before we got protocol extensions in
Swift 2, there was basically *no way* to share implementation among
value types. So you can't take protocols out of the picture without
making value types, and the argument for value semantics, far weaker.
Why? Like I said, Eiffel *has* value types without needing protocols. They just have a unified mechanism built around classes.
But protocols are not really different from interfaces in Java.
I would have preferred a unified model using just classes with real
multiple inheritance like Eiffel has and value types just being a part
of that similar to Eiffel’s `expanded` classes. But that ship has
probably sailed a long time ago :-/ So be it. But at least there
should be no reasons for POP vs OOP wars ;-) (I’d like to add that I
liked Dave’s talks at last WWDC very much, it’s just that I don’t
think that POP is something new or different.)
Protocol-oriented programming is about the synergy of features and ideas
most of which not *individually* new, but that together create a new
world of possibilities. I've already discussed the synergy of protocols
and first-class value semantics.
Again, I think that Eiffel demonstrates that protocols are not needed for first-class value semantics.
There's also the fact that in
protocols we have one construct with which to express dynamic
polymorphism (existentials) and static polymorphism (generic
constraints), both of which have important roles to play but that I
maintain are very different indeed.
I don't argue with the fact that protocols can be used for both of these roles (existentials and constraints) and that these roles are something different. But I didn't say anything like that, at least not intentionally.
One result is that you can “start
with a protocol” as your abstraction mechanism and know that you're not
going to design yourself into a corner where it becomes impossibly
awkward to express what you need.
Not sure what problems you are hinting at here specifically? Is it the single inheritance restriction most languages impose on classes? In that case I agree with you that that is a problem. I would not agree with interfaces being the only or even best solution, though, because I would prefer classes with real support for multiple inheritance like Eiffel shows is possible (especially as interfaces do not solve all problems with MI).
Finally—and I'm certain this *is*
new—in protocol extensions we have a means to express both post-hoc
conformance and generic functions that is much more accessible to users
than in any previous language, to the point where generic programming
can become a natural part of everyday work.
Protocol extensions are a great feature that I was very very happy to see after having used extensions in Smalltalk for many years and missing them in Java. Especially being able to use post-hoc conformance is a really nice feature!
Please don't get me wrong, I think Swift is a fantastic language and I love it.
I'm not arguing for removal of protocols in Swift or something like that. That ship has sailed. But I don't think that classes should be second class citizens in Swift, like several posts on this mailing list implied, e.g. in the discussion about abstract classes.
Best regards
-Thorsten
···
Am 13.06.2016 um 04:04 schrieb Dave Abrahams <dabrahams@apple.com>:
on Fri Jun 10 2016, Thorsten Seitz <tseitz42-AT-icloud.com> wrote:
Am 09.06.2016 um 19:50 schrieb Thorsten Seitz via swift-evolution <swift-evolution@swift.org>:
Am 09.06.2016 um 18:49 schrieb Dave Abrahams via swift-evolution <swift-evolution@swift.org>:
on Wed Jun 08 2016, Jordan Rose <swift-evolution@swift.org> wrote:
On Jun 8, 2016, at 13:16, Dave Abrahams via swift-evolution >>>>>> <swift-evolution@swift.org> wrote:
on Wed Jun 08 2016, Thorsten Seitz >>>>> >>>>>> <swift-evolution@swift.org >>>>>> <mailto:swift-evolution@swift.org>> >>>>>> wrote:
That's certainly a valid question. I have been very vocally skeptical
that HKTs like Monad belong in the standard library, but I am not
opposed to having the language feature if it supports important use
cases.
It is surprising to see people debating the place of monads in Swift when nobody seems to speak about the elephant in the middle of the room: the language has a real engineering defisciency if it is possible for people to think that there is no better way to use it than to do "we have around 50+ packages. This list grows very fast" (http://docs.zewo.io - many more swift github projects are equally struggling to get the code organized).
At the moment the only thing preventing the situation from getting more out of hands is that the import mechanism currently gets in the way, but the only opinions I read so far all focussed on making it even easier to slap things together ala zewo rather than address the engineering issue.
It would be helpful if you could do two things:
1. Be clear about exactly what you think is wrong and, preferably, how you think we might be able to solve it, or at least what sorts of problems you want the solution to address. Reading this post, I'm not sure what you find objectionable about this project containing 50 packages or what "situation" you think is getting out of hand. Even if you have a good point, you're not making it very well.
2. Post about your issue in a new thread rather than trying to derail something *entirely* unrelated to it. I'm not sure exactly what your complaint is, but I gather it has something to do with package management, modules, or imports. None of these have anything to do with higher-kinded types, the type system, or (as far as I can tell) anything that has ever been discussed in this thread. The result is rather like interrupting a charity drive for the local children's hospital to tell people that actually, they really *ought* to be donating to your mosquito-net charity instead. Even if you're correct that your cause is more important, it's inappropriate, unwelcome, and frankly just rude.
In short: Please don't do this. It does not help your case.
Ah, thanks, I forgot! I still consider this a bug, though (will have
to read up again what the reasons are for that behavior).
Yes, but in the case of the issue we're discussing, the choices are:
1. Omit from the existential's API any protocol requirements that depend
on Self or associated types, in which case it *can't* conform to
itself because it doesn't fulfill the requirements.
2. Erase type relationships and trap at runtime when they don't line up.
Matthew has been arguing against #2, but you can't “fix the bug” without
it.
#1 has been my preference for a while as well, at least as a starting
point.
I should point out that with the resyntaxing of existentials to
Any<Protocols...>, the idea that Collection's existential doesn't
conform to Collection becomes far less absurd than it was, so maybe this
is not so bad.
I think the problem is more that Any<Collection> does not conform to a specific value for a type parameter T: Collection
What I mean by this is that `Collection` denotes a type family, a generic parameter `T: Collection` denotes a specific (though unknown) member of that type family and `Any<Collection>` denotes the type family again, so there is really no point in writing Any<Collection> IMO.
The type family cannot conform to T because T is just one fixed member of it.
It conforms to itself, though, as I can write
let c1: Any<Collection> = …
let c2: Any<Collection> = c1
That’s why I think that we could just drop Any<Collection> and simply write Collection.
Let me expand that a bit:
Actually all this talk about existentials vs. generics or protocols vs. classes has had me confused somewhat and I think there are still some misconceptions present on this list sometimes, so I’ll try to clear them up:
(1) misconception: protocols with associated types are somehow very different from generics
I don’t think they are and I will explain why. The only difference is the way the type parameters are bound: generics use explicit parameter lists whereas protocols use inheritance. That has some advantages (think long parameter lists of generics) and some disadvantages.
These ways are dual in a notation sense: generic types have to have all parameters bound whereas protocols cannot bind any of them.
The „existential“ notation `Any<>` being discussed on this list is nothing more than adding the ability to protocols to bind the parameters to be used just like Java’s wildcards are adding the opposite feature to generics, namely not having to bind all parameters.
As you know I like using `&` as type intersection operator. But you write "The syntax leave a void when it comes to expressing the so called Top type:“
Why? Just give the top type a name, e.g. `Any` and you are done.
Yes.. I just don't like magic. I think that all types should be expressable with a syntax and that you can then decide to alias one parsing case with a specific name. Not the other way around.
Why should the top type have special *syntax*? It is just the type all other types conform to. No need to do something special here and therefore no need to invent an alternative syntax like `Any<>` or the alternative from your gist which is rather confusing IMO (and I don’t like the special case given to classes in the syntax).
The _ case is just a degenerate case of the syntax, showing that it is expressible inside, as opposoed to have to define a contextual keyword (read the SourceKit code). And then it is aliasable. Part of the problems in swift today to me is that some things are no doable in swift, so the compiler must contain that semantic. This here is just one example, but there are others. These are notions that the standard library has to defer to the compiler. Some of them have been tabled for 4.0 it seems. My point here was that it is not fate, and choosing the syntax carefully for existentials (whichever it is, i don't really care inthe end) would prevent having to add a magic keyword for to top type to the compiler and to SourceKit again (they are trying to remove them).
As for the current Any<...> proposal for generalizing existentials it is IMHO cluncky and magic. There is nothing stopping us mechanically from entering Any<UITableView, UIButton>. To me that is the holemark of bad design. In what I tested you just can't do it and the reason is clear. Of course when I say you can't I do not mean that your keyboard will zap u if you try. But the way to verify correctness is different... and in a world where xcode would be a good ide, then code completion would even show you why and actually truly assist (because the grammar i tested makes really makes it possible). I know that xcode is only as good as SourceKit lets it be.
Essentially `Any<Collection>` in Swift is just the same as `Collection<?>` in Java (assuming for comparability’s sake that Swift’s Collection had no additional associated types; otherwise I would just have to introduce a Collection<Element, Index> in Java).
Likewise `Any<Collection where .Element: Number>` is just the same as `Collection<? extends Number>` in Java.
Java’s wildcards are a way to express use-site variance. The proposed `Any<>` does just the same.
And just like Collection<?> does not conform to a type parameter `T extends Collection<?>` because Collection<?> is the type `forall E. Collection<E>` whereas `T extends Collection<?>` is the type `T. Collection<T>` for a given T.
This picture is accurate today, but there are going to be more serious differences after 10 no date is currently geven for when it will come)
You mean Java 10?
Yes. After 10. Than is their only date hint.
In essence protocols with associated types are like generics with wildcards.
Yes, java has kept everything within its generics system rather than split parts out. Something people may not immediately think about with respect to the 2 generic systems is that when u call a func<T>capture(T t){} in java with a wildcard you are doing a compile time capture only (to avoid the dreaded unsafe casts), whereas it is really nice to do the same in swift and subsequently be able to access T.Type and see that it is not Any. The closest u ever get to that type at runtime in java is via generics introspection, but u still can't do everything ( like no new T() ). But today the bridging between existential types and generics is definitely a work in progress.
Coming back to the questions whether (a) allowing existentials to be used as types is useful and (b) whether sacrificing type safety would somehow be necessary for that, I think we can safely answer
(a) yes, it *is* useful to be able to use existentials like Any<Collection> as types, because wildcards are quite often needed and very useful in Java (they haven’t been added without a reason)
IMO they made java 8 (referring to streams). And even though the syntax for co/contra variance is pretty heavy, it is the foundation for all modern java code. The any-fication of the generics is going to open new doors as some of it will translate into a partial reification in the jvm. It seems the decision for now is to not use the extra info in java to retain binary compatibility with all the erased code out there, this is something scala might use in areas where it won't mind loosing java compatibility.
(b) no, sacrificing type safety does not make sense, as the experience with Java’s wildcards shows that this is not needed. Especially if something like path dependent types is used like proposed and some notation to open an existential’s type is added, which is both something that Java does not have.
I hope typesafe opening inside the " if let " syntax gets added. I know that chris is against sugaring, but I played if an implementation of
... x is String?
If let! x {}
That runs as
if let x = x {}
something equally short could be done here.
(2) misconception: POP is different from OOP
It is not. Protocols are just interfaces using subtyping like OOP has always done. They just use associated types instead of explicit type parameters for generics (see above). The more important distinction of Swift is emphasizing value types and making mutation safely available by enforcing copy semantics for value types.
Values are coming to the jvm, which will narrow this gap (like their view identity for value vs ref and the whole deep ==) . I also really like the cow approach of the swift runtime.
But protocols are not really different from interfaces in Java.
There is one big difference: default methods, but it seems swift will add that soon.
Swift already has default extension methods, doesn’t it?
Yes, and no. It has differently dispatched code that can be found to fill in the gap of the conformance req, yes. But Joe Grof (?) said that there are no reasons why these could not be added. Having true defaults could be one way to deal with optional comformance...
I also really like how extensions and conformance mix together in swift to bake retro-modelling in and the adapter pattern (spent enough years deep diving inside eclipse to appreciate it).
I would have preferred a unified model using just classes with real multiple inheritance like Eiffel has and value types just being a part of that similar to Eiffel’s `expanded` classes. But that ship has probably sailed a long time ago :-/
I like extensions very much (having used Smalltalk for a long time). I like enums for the pattern matching and structs as value types. But having to split protocols off instead of using abstract classes makes things more complicated IMO.
-1 i am old school c/c++... i really like protocol, struct, class, extensions, enums. It is a really nice mix that gives objc people room to grow, but I do miss how they are an integral part of generics (i protocols as a replacement and look forward to when they interact better) and namespaces+scoped-imports (c#)... Looking forward to where things go next
Yeah, namespacing/submodules/conflict resolution (when doing imports but also when conforming to multiple protocols which has just the same problems) are still missing. But I’m optimistic :-) Let’s complete generics first, then tackle existentials/type intersections.
I think I care more about existentials, but only because of the kind of java i wrote for a living. I just looked at a bunch of opensource swift libs (particularly for server side swift)... some of it is a real engineering disaster: 20+ folders, each with 2 source files... or 3 folders "extensions" "utils" "classes". Swift is currently not equiped for people to write big things with... I wish the team would address it sooner than later (look at their own c++ code to see the difference). IMHO import conflicts are more often the symptom of bad code than a real issue.
Cheers
···
On Jun 11, 2016, at 11:30 AM, Thorsten Seitz <tseitz42@icloud.com> wrote:
Am 11.06.2016 um 08:00 schrieb L. Mihalkovic <laurent.mihalkovic@gmail.com>:
On Jun 10, 2016, at 9:35 PM, Thorsten Seitz via swift-evolution <swift-evolution@swift.org> wrote:
Am 09.06.2016 um 19:50 schrieb Thorsten Seitz via swift-evolution <swift-evolution@swift.org>:
Am 09.06.2016 um 18:49 schrieb Dave Abrahams via swift-evolution <swift-evolution@swift.org>:
on Wed Jun 08 2016, Jordan Rose <swift-evolution@swift.org> wrote:
On Jun 8, 2016, at 13:16, Dave Abrahams via swift-evolution >>>>>>> <swift-evolution@swift.org> wrote:
on Wed Jun 08 2016, Thorsten Seitz >>>>>> >>>>>>> <swift-evolution@swift.org >>>>>>> <mailto:swift-evolution@swift.org>> >>>>>>> wrote:
-THorsten
So be it. But at least there should be no reasons for POP vs OOP wars ;-)
(I’d like to add that I liked Dave’s talks at last WWDC very much, it’s just that I don’t think that POP is something new or different.)
I used to thin that way. But today I think that although in broad brush strokes the similarities and bigger than the differences, there is room for making a bigger difference in the how.
, but haven't realized
that if you step around the type relationships encoded in Self
requirements and associated types you end up with types that appear to
interoperate but in fact trap at runtime unless used in exactly the
right way.
Trap at runtime? How so? Generalized existentials should still be
type-safe.
There are two choices when you erase static type relationships:
1. Acheive type-safety by trapping at runtime
FloatingPoint(3.0 as Float) + FloatingPoint(3.0 as Double) // trap
2. Don't expose protocol requirements that involve these relationships,
which would prevent the code above from compiling and prevent
FloatingPoint from conforming to itself.
Or are you talking about the hypothetical types / behaviors people
think they want when they don’t fully understand what is happening...
I don't know what you mean here. I think generalized existentials will
be nice to have, but I think most people will want them to do something
they can't possibly do.
Exactly. What I meant is that people think they want that expression
to compile because they don’t understand that the only thing it can do
is trap. I said “hypothetical” because producing a compile time error
rather than a runtime trap is the only sane thing to do. Your comment
surprised me because I can’t imagine we would move forward in Swift
with the approach of trapping.
I would very much like to be able to create instances of “Collection
where Element == Int” so we can throw away the wrappers in the stdlib.
That will require some type mismatches to be caught at runtime via
trapping.
For invalid index because the existential accepts a type erased index?
Exactly.
How do you decide where to draw the line here? It feels like a very
slippery slope for a language where safety is a stated priority to
start adopting a strategy of runtime trapping for something as
fundamental as how you expose members on an existential.
If you don't do this, the alternative is that “Collection where Element
== Int” does not conform to Collection.
This isn’t directly related to having self or associated type
requirements. It is true of all existentials.
That is just an implementation limitation today, IIUC. What I'm talking
about here would make it impossible for some to do that.
If it is just an implementation limitation I am happy to hear that.
If that changes for simple existentials and generalized existentials
expose all members (as in the latest draft of the proposal) maybe it
will be possible for all existentials to conform to their protocol.
Not without introducing runtime traps. See my “subscript function”
example.
That's weird and not very
useful. You could expose all the methods that were on protocol
extensions of Collection on this existential, unless they used
associated types other than the element type. But you couldn't pass the
existential to a generic function like
func scrambled<C: Collection>(_ c: C) -> [C.Element]
IMO you should *have* to introduce unsafe behavior like that manually.
Collection where Element == Int & Index == *
?
I didn’t mean directly through the type of the existential.
My question is, why not? That is still explicit.
It’s not explicit in the sense that nobody wrote `fatalError` or
similar in their code. It’s too easy to write something like that
without realizing that it introduces the possibility of a crash. If
we adopt syntax like that to introduce an existential that introduces
traps we should at least require members that can be trap to be
invoked using a `!` suffix or something like that to make it clear to
users that a trap will happen if they are not extremely careful when
using that member.
More generally though, I don’t want the rules of the language to be
written in a way that causes the compiler to synthesize traps in such
a general way.
The existential should not introduce a precondition that isn’t already
present in the semantics of the protocol itself. If the semantics of
the protocol do not place preconditions on arguments beyond their type
(such as “must be a valid index into this specific instance”) the
compiler should not allow the existential to conform if a trap is
required in some circumstances. That is a new precondition and
therefore the existential does not actually fulfill the requirements
of the protocol.
I could *maybe* live with a solution where protocol requirements are
marked as trapping, etc depending on the specific argument received at
runtime. This is a total straw man syntax, but maybe `IndexableBase`
would declare the subscript `@trapping` (probably something different
but I hope this communicates the idea). This alerts users to the fact
that they need to be extra careful - not any value of `Self.Index` is
valid and you can get a crash if you’re not careful.
Having this semantic explicit in the definition of the protocol opens
the door to maybe considering an existential synthesized by the
compiler that traps because it doesn’t introduce a new precondition
that wasn’t already present in the protocol.
I would want to give consideration to specific details of a proposal
along these lines before deciding how I feel about it, but I have a
more open mind to this approach than introducing traps not present in
the preconditions of the protocol.
/// You can subscript a collection with any valid index other than the
/// collection's end index. The end index refers to the position one past
/// the last element of a collection, so it doesn't correspond with an
/// element.
///
/// - Parameter position: The position of the element to access. `position`
/// must be a valid index of the collection that is not equal to the
/// `endIndex` property. @trapping public subscript(position: Self.Index) -> Self._Element { get }
One obvious mechanism for introducing unsafe behavior is to write
manual type erasure wrappers like we do today.
Another possibility would be to allow extending the existential type
(not the protocol). This would allow you to write overloads on the
Collection existential that takes some kind of type erased index if
that is what you want and either trap if you receive an invalid index
or better (IMO) return an `Element?`. I’m not sure how extensions on
existentials might be implemented, but this is an example of the kind
of operation you might want available on it that you wouldn’t want
available on all Collection types.
Collection indices are already something that isn’t fully statically
safe so I understand why you might want to allow this.
By the same measure, so are Ints :-)
The fact that a type's methods have preconditions does *not* make it
“statically unsafe.”
That depends on what you mean by safe. Sure, those methods aren’t
going corrupt memory, but they *are* going to explicitly and
intentionally crash for some inputs. That doesn’t qualify as “fully
safe” IMO.
Please pick a term other than “unsafe” here; it's not unsafe in the
sense we mean the word in Swift. It's safe in exactly the same way that
array indexes and integers are. When you violate a precondition, it
traps.
Safe. The most obvious way to write code should also behave in a safe
manner. Undefined behavior is the enemy of safety, and developer
mistakes should be caught before software is in production. Opting for
safety sometimes means Swift will feel strict, but we believe that
clarity saves time in the long run.
Safety
Swift was designed from the outset to be safer than C-based languages,
and eliminates entire classes of unsafe code. Variables are always
initialized before use, arrays and integers are checked for overflow,
and memory is managed automatically. Syntax is tuned to make it easy
to define your intent — for example, simple three-character keywords
define a variable (var) or constant (let).
Another safety feature is that by default Swift objects can never be
nil, and trying to make or use a nil object will results in a
compile-time error. This makes writing code much cleaner and safer,
and prevents a common cause of runtime crashes. However, there are
cases where nil is appropriate, and for these situations Swift has an
innovative feature known as optionals. An optional may contain nil,
but Swift syntax forces you to safely deal with it using ? to indicate
to the compiler you understand the behavior and will handle it safely.
This positioning statement makes it appear as if preventing common
causes of crashes falls within the meaning of safe that Swift is
using. Having existentials introduce new preconditions and traps when
they are not met does not seem aligned with that goal IMO.
Static typing “increases safety,” in the casual sense. That doesn't
mean that an operation that traps on a failed precondition check is
“unsafe.”
The user doesn't do anything “manual” to introduce that trapping
behavior for integers. Preconditions are a natural part of most types.
The user doesn’t, but isn’t the overflow trap implemented in the
standard library?
Whether it is or is not is an implementation detail.
Regardless, this is a specific case that has been given explicit
design attention by humans. The precondition is designed, not
introduced by compiler rules that haven’t considered the specific case
in question.
But I don’t think having the language's existentials do this
automatically is the right approach. Maybe there is another
approach that could be used in targeted use cases where the less
safe behavior makes sense and is carefully designed.
Whether it makes sense or not really depends on the use-cases. There's
little point in generalizing existentials if the result isn't very useful.
Usefulness depends on your perspective.
Of course. As I've said, let's look at the use cases.
Agree. We can consider those in depth when the time comes to ramp up
discussion of Austin’s proposal.
I have run into several scenarios where they would be very useful
without needing to be prone to crashes when used incorrectly. One
obvious basic use case is storing things in a heterogenous collection
where you bind .
bind what?
Sorry, I must have gotten distracted and not finished that paragraph.
I meant to say bind the associated types that are necessary for your
use case. Sometimes you bind *all* of the associated types to
concrete types and the protocol has no `Self` requirements. In that
case there is no trouble at all in conforming the type-erased
“existential" to the protocol itself. Austin’s proposal would
eliminate the need to manually write these “existentials” manually.
The way to find out is to take a look at the examples we currently have
of protocols with associated types or Self requirements and consider
what you'd be able to do with their existentials if type relationships
couldn't be erased.
We have known use-cases, currently emulated in the standard library, for
existentials with erased type relationships. *If* these represent the
predominant use cases for something like generalized existentials, it
seems to me that the language feature should support that. Note: I have
not seen anyone build an emulation of the other kind of generalized
existential. My theory: there's a good reason for that :-).
AFAIK (and I could be wrong) the only rules in the language that
require the compiler to synthesize a trap except using a nil IUO, `!`
on a nil Optional, and an invalid `as` cast . These are all
syntactically explicit unsafe / dangerous operations. All other traps
are in the standard library (array index, overflow, etc). Most
important about all of these cases is that they have received direct
human consideration.
There is no distinction in the user model between what might be
synthesized by the language and what appears on standard library types.
Maybe I shouldn’t have made that distinction.
The point I am trying to emphasize is that each of these are special
cases that have received direct human consideration. The potential
for a trap is not introduced by language rules that apply to
user-defined constructs in without consideration of the specific
details of that construct.
Introducing a language (not library) mechanism that exposes members on
generalized existentials in a way that relies on runtime traps for
type safety feels to me like a pretty dramatic turn agains the stated
priority of safety. It will mean you must understand exactly what is
going on and be extremely careful to use generalized existentials
without causing crashes. This will either make Swift code much more
crashy or will scare people away from using generalized existentials
(and maybe both).
I don't accept either of those statements without seeing some analysis
of the use-cases. For example, I don't believe that AnyCollection et al
are particularly crash-prone. The likelihood that you'll use the wrong
index type with a collection is very, very low. I'm less certain of
what happens with Self requirements in real cases.
But again, I believe this is an exceptional case as the precondition
is explicitly stated in the semantics of the protocol.
IIUC, it has been cited by Doug as the exemplar of the
predominantly-requested case by a 10:1 ratio!
In terms of forming the existential, storing it in variables,
accepting arguments of that type, etc yes. I don’t know how many of
those requests expect it to conform to the protocol and expect to be
able to use it in generic code constrained to the protocol.
IMO the burden of proof should be on the side that proposes a
mechanism to introduce traps, not the side that proposes avoiding
them.
If you really want to make this about sides and burdens, the burden of
proof always rests with the side proposing to extend the language. We
shouldn't be making changes without understanding how they will play out
in real use-cases.
I agree with this. But if we are discussing two different options for
extending the language I think the option that doesn’t introduce
crashes should be preferred without pretty compelling reasons to
choose the option that can introduce crashes.
Neither of those outcomes is good.
Collection indices are a somewhat special case as there is already a
strong precondition that people are familiar with because it would be
too costly to performance and arguably too annoying to deal with an
Optional result in every array lookup. IMO that is why the library is
able to get away with it in the current type erased AnyCollection.
But this is not a good model for exposing any members on an
existential that do not already have a strong precondition that causes
a trap when violated.
I think a big reason why you maybe haven’t seen a lot of examples of
people writing type erased “existentials" is because it is a huge pain
in the neck to write this stuff manually today. People may be
designing around the need for them. I haven’t seen a huge sampling of
type erased “existentials" other people are writing but I haven’t
written any that introduce a trap like this. The only traps are in
the “abstract" base class whose methods will never be called (and
wouldn’t even be implemented if they could be marked abstract).
What specific things do you think we need to be able to do that rely
on the compiler synthesizing a trap in the way it exposes the members
of the existential?
I don't know. I'm saying, I don't think we understand the use-cases
well enough to make a determination.
That’s fair. I agree that use cases should be carefully considered.
Here are a few examples from Austin’s proposal that safely use
existential collections. I don’t understand why you think this
approach is insufficient. Maybe you could supply a concrete example
of a use case that can’t be written with the mechanism in Austin’s
proposal.
// A variable whose type is the Index associated type of the underlying
// concrete type of 'a'.
let theIndex : a.Index = ...
// A variable whose type is the Element associated type of the underlying
// concrete type of 'a'.
let theElement : a.Element = ...
// Given a mutable collection, swap its first and last items.
// Not a generic function.
func swapFirstAndLast(inout collection: Any<BidirectionalMutableCollection>) {
// firstIndex and lastIndex both have type "collection.Index"
guard let firstIndex = collection.startIndex,
lastIndex = collection.endIndex?.predecessor(collection) where lastIndex != firstIndex else {
print("Nothing to do")
return
}
// oldFirstItem has type "collection.Element"
let oldFirstItem = collection[firstIndex]
var a : Any<BidirectionalMutableCollection where .Element == String> = ...
let input = "West Meoley"
// Not actually necessary, since the compiler knows "a.Element" is String.
// A fully constrained anonymous associated type is synonymous with the concrete
// type it's forced to take on, and the two are interchangeable.
// However, 'as' casting is still available if desired.
let anonymousInput = input as a.Element
a[a.startIndex] = anonymousInput
// as mentioned, this also works:
a[a.startIndex] = input
// If the collection allows it, set the first element in the collection to a given string.
func setFirstElementIn(inout collection: Any<Collection> toString string: String) {
if let element = string as? collection.Element {
// At this point, 'element' is of type "collection.Element"
collection[collection.startIndex] = element
}
}
Neither of these look like they actually make *use* of the fact that
there's type erasure involved (and therefore should probably be written
as generics?). The interesting cases with Any<Collection...>, for the
purposes of this discussion, arise when you have multiple instances of
the same existential type that wrap different concrete types.
One use case I have found is to work around the lack of higher-kinder
types.
Really, now: a use-case for feature A that is a workaround for the lack
of feature B hardly justifies adding feature A! We do want to add
higher-kinded types eventually.
Good to know. I thought higher-kinder types were on the “maybe if
someone shows a compelling enough use case” list. AFAIK this is the
first time a member of the core team has stated the intent to add
them.
Well, please don't take this as a formal statement on behalf of the
team. IIUC, the team is generally interested in having this feature.
Of course not. The only official plan of record for Swift is approved
proposals. But it’s always interesting to know the opinions of the
core team about potential features. Previously my impression had been
that the general leaning was towards skepticism that the practical
benefits of HKT would pay for the complexity.
That's certainly a valid question. I have been very vocally skeptical
that HKTs like Monad belong in the standard library, but I am not
opposed to having the language feature if it supports important use
cases.
That makes sense. Language support is where we get expressive power. I think you are right to challenge placing specific protocols in the standard library.
In the case of Monad in particular, Swift has a much different design direction and feature set than languages where it plays a central role. I have an open mind on this but the case for putting it in the standard library definitely isn't obvious. Concrete, practical and useful examples of generic code that would be written in terms of Monad are required. And they should make it clear why this is a good approach in Swift. It is not hard to show this for concrete monads, but pretty difficult to do so for generic code that only depends on the Monad abstraction.
I haven't come across any good examples yet so I share your skepticism on this until I see them.
Before getting to monads in the std lib, it would be nice if the generics and existential system were powerfull and complementary enough that the stdlib would not have to contain so much generated code for collections. I am not saying that generated code should be eliminated, but it should be the result of a performance optimization choice, not the only way to do it.
···
On Jun 11, 2016, at 3:03 PM, Matthew Johnson via swift-evolution <swift-evolution@swift.org> wrote:
On Jun 10, 2016, at 7:46 PM, Dave Abrahams via swift-evolution <swift-evolution@swift.org> wrote:
on Thu Jun 09 2016, Matthew Johnson <swift-evolution-m3FHrko0VLzYtjvyW6yDsg@public.gmane.org> wrote:
On Jun 9, 2016, at 3:05 PM, Dave Abrahams <dabrahams-2kanFRK1NckAvxtiuMwx3w@public.gmane.org> wrote:
on Thu Jun 09 2016, Matthew Johnson <matthew-AT-anandabits.com> wrote:
On Jun 9, 2016, at 11:42 AM, Dave Abrahams <dabrahams-2kanFRK1NckAvxtiuMwx3w@public.gmane.org> wrote:
on Thu Jun 09 2016, Matthew Johnson <matthew-AT-anandabits.com <http://matthew-at-anandabits.com/>> wrote:
On Jun 8, 2016, at 1:33 PM, Dave Abrahams >>>>>>>>>> <dabrahams@apple.com >>>>>>>>>> <mailto:dabrahams@apple.com>> >>>>>>>>>> wrote:
on Tue Jun 07 2016, Matthew Johnson <matthew-AT-anandabits.com <http://matthew-at-anandabits.com/>> wrote:
On Jun 7, 2016, at 4:13 PM, Dave Abrahams via >>>>>>>>>>>>>> swift-evolution <swift-evolution-m3FHrko0VLzYtjvyW6yDsg@public.gmane.org >>>>>>>>>>>>>> <mailto:swift-evolution-m3FHrko0VLzYtjvyW6yDsg@public.gmane.org>> wrote:
on Tue Jun 07 2016, Matthew Johnson >>>>>>>>>>>>>> <swift-evolution@swift.org >>>>>>>>>>>>>> <mailto:swift-evolution-m3FHrko0VLzYtjvyW6yDsg@public.gmane.org>> >>>>>>>>>>>>>> wrote:
This has been a point of confusion for me as well. I keep hearing that associated types are different from generic protocols, but this seems like a distinction without a difference.
Suppose Swift allowed generic protocols. How would a hypothetical Collection<Foo> be different in practice from the proposed existential Any<Collection where .Element == Foo>?
Yes, in the realm of type theory and compiler internals they might represented differently, sure. But in practice, in terms of what code can actually do? I know of only two differences:
1. A type can only conform to any given protocol with one set of type parameters. (Nothing can be both Collection<Foo> and Collection<Bar>.)
2. When a type conforms to Collection, it declares “associatedtype Foo” instead of “: Collection<Foo>”, and Foo can be inferred by the compiler in some circumstances. That’s handy, but it’s a syntactic difference.
Is there a deeper difference I’m missing?
Cheers, P
···
On Jun 16, 2016, at 8:29 AM, Thorsten Seitz via swift-evolution <swift-evolution@swift.org> wrote:
Protocols are a mechanism for deriving types from each other whereas generics are a way to parameterize types. My point was that Swift's other way to parameterize types, namely by associated types, is very similar to generics with wildcards when looking a the existentials of such protocols.
Ah, thanks, I forgot! I still consider this a bug, though (will have
to read up again what the reasons are for that behavior).
Yes, but in the case of the issue we're discussing, the choices are:
1. Omit from the existential's API any protocol requirements that depend
on Self or associated types, in which case it *can't* conform to
itself because it doesn't fulfill the requirements.
2. Erase type relationships and trap at runtime when they don't line up.
Matthew has been arguing against #2, but you can't “fix the bug” without
it.
#1 has been my preference for a while as well, at least as a starting
point.
I should point out that with the resyntaxing of existentials to
Any<Protocols...>, the idea that Collection's existential doesn't
conform to Collection becomes far less absurd than it was, so maybe this
is not so bad.
I think the problem is more that Any<Collection> does not conform to
a specific value for a type parameter T: Collection
What I mean by this is that `Collection` denotes a type family, a
generic parameter `T: Collection` denotes a specific (though
unknown) member of that type family and `Any<Collection>` denotes
the type family again, so there is really no point in writing
Any<Collection> IMO.
The type family cannot conform to T because T is just one fixed member of it.
It conforms to itself, though, as I can write
let c1: Any<Collection> = …
let c2: Any<Collection> = c1
That’s why I think that we could just drop Any<Collection> and simply write Collection.
Let me expand that a bit:
Actually all this talk about existentials vs. generics or protocols
vs. classes has had me confused somewhat and I think there are still
some misconceptions present on this list sometimes, so I’ll try to
clear them up:
There are several objectively incorrect statements here, and several
others with which I disagree. I was hoping someone else would write
this for me, but since the post has such a tone of authority I feel I
must respond.
You are right, the tone of my post was not appropriate, for which I want to apologize sincerely.
I still believe my statements to be valid, though, and will respond to your arguments inline. Please don't get me wrong, I'm not trying to have an argument for the argument's sake. All I want is to contribute maybe a tiny bit to make Swift even better than it already is, by sharing ideas and thoughts not only from me but from the designs of other perhaps more obscure programming languages which I happen to have stumbled upon in the past (often with much delight).
(1) misconception: protocols with associated types are somehow very
different from generics
I don’t think they are and I will explain why. The only difference is
the way the type parameters are bound: generics use explicit parameter
lists whereas protocols use inheritance. That has some advantages
(think long parameter lists of generics) and some disadvantages.
These ways are dual in a notation sense: generic types have to have
all parameters bound whereas protocols cannot bind any of them.
The „existential“ notation `Any<>` being discussed on this list is
nothing more than adding the ability to protocols to bind the
parameters to be used just like Java’s wildcards are adding the
opposite feature to generics, namely not having to bind all
parameters.
Protocols and generics fulfill completely different roles in Swift, and
so, **especially in a language design context like the one we're in
here**, must be thought of differently. The former are an abstraction
mechanism for APIs, and the latter a mechanism for generalizing
implementations.
That's not what I was talking about. Of course, protocols are a mechanism for deriving types from each other whereas generics are a way to parameterize types. My point was that Swift's other way to parameterize types, namely by associated types, is very similar to generics with wildcards when looking a the existentials of such protocols. In addition I was talking about generics in general, not just about generics in Swift which restricts them to implementations and does not support wildcards.
Other languages like Java offer generics for interfaces as well and support wildcards (adding generic types parameters to protocols in Swift is currently discussed on the mailing list as well).
FWIW my arguments were not about whether we should have wildcards in Swift or not, but simply to relate one parametrization feature (associated types) to a more well known parametrization feature (generics with wildcards) in order to understand them better.
The only place you could argue that they intersect is
in generic non-final classes, because a class fills the dual role of
abstraction and implementation mechanism (and some might say that's a
weakness). But even accounting for generic classes, protocols with
associated types are very different from generics. Two utterly
different types (an enum and a struct, for example) can conform to any
given protocol P, but generic types always share a common basis
implementation.
The latter is not the case for generic interfaces in Java, for example, so it is just an artificial restriction present in Swift.
There is no way to produce distinct instances of a
generic type with all its type parameters bound,
That is true in Swift (except for generic classes) due to the restriction just mentioned.
but for any protocol P
I can make infinitely many instances of P with P.AssociatedType == Int.
This likewise applies to generic interfaces and for generic types in general if taking inheritance into account - just like you do here for protocols.
Back to the my original point: while protocols and generic types have
some similarities, the idea that they are fundamentally the same thing
(I know you didn't say *exactly* that, but I think it will be read that
way) would be wrong and a very unproductive way to approach language
evolution.
I said that protocols *with associated types* are much like generics *with wildcards* and tried to show why.
Essentially `Any<Collection>` in Swift is just the same as
`Collection<?>` in Java (assuming for comparability’s sake that
Swift’s Collection had no additional associated types; otherwise I
would just have to introduce a Collection<Element, Index> in Java).
I don't see how you can use an example that requires *assuming away*
assoociated types to justify an argument that protocols *with associated
types* are the same as generics.
Note, that I said *additional* associated types, i.e. in addition to .Element, even giving an example how the Java interface had to be extended by a type parameter `Index` if this assumption was not applied (still simplifying because Generator would have been more correct which would have to be added as type parameter in addition to `Index`).
So, in essence the comparison is between the following (I'm using Foo now instead of Collection to avoid the differences mentioned. Note that this has no impact on the argument at all):
protocol Foo {
associatedtype T
...
}
interface Foo<T> {
...
}
My argument is that existentials of protocols with associated types are just like generic types with wildcards, i.e. `Any<Foo>` in Swift is just the same as `Foo<?>` in Java.
Likewise `Any<Foo where .T: Number>` is just the same as `Foo<? extends Number>` in Java. For me that was an insight I wanted to share.
The like-ness applies only to a subset of the java type system, and this is the area that will drastically change within the next 2 years.
And just like Collection<?> does not conform to a type parameter `T
extends Collection<?>` because Collection<?> is the type `forall
E. Collection<E>` whereas `T extends Collection<?>` is the type
`T. Collection<T>` for a given T.
In essence protocols with associated types are like generics with
wildcards.
It is true that generics with wildcards in Java *are* (not just “like”)
existential types but I don't agree with the statement above. Because
Java tries to create an “everything is a class” world,
Never did, never will...
generic classes
with bound type parameters end up playing the role of existential type.
But protocols in Swift are not, fundamentally, just existential types,
and the resyntaxing of ProtocolName to Any<ProtocolName> for use in type
context is a huge leap forward in making that distinction clear... when
that's done (unless we leave Array<ProtocolName> around as a synonym for
Array<Any<ProtocolName>>—I really hope we won't!) protocols indeed
Hmmm starting to see the ugly Any<> burgeoning all over swift soon... :(
*won't* be types at all, existential or otherwise.
I fully agree that protocols are not types, their existentials are. But I haven't seen yet what we really *gain* from making that distinction explicit (except an ugly type syntax :-).
And like I already wrote in this or another thread we would have to apply the same logic to non-final classes, which are existentials, too.
Coming back to the questions whether (a) allowing existentials to be
used as types is useful
That's the only use existentials have. They *are* types. Of course
they're useful, and I don't think anyone was arguing otherwise.
I'm pretty sure that there was a discussion about whether being able to write something like Any<Collection> is useful. My wording was certainly imprecise, though, and didn't make sense as written. I should have said something like "whether adding the ability to use existential types of protocols with unbound associated types is useful".
and (b) whether sacrificing type safety would somehow be necessary for
that, I think we can safely answer (a) yes, it *is* useful to be able
to use existentials like Any<Collection> as types, because wildcards
are quite often needed and very useful in Java (they haven’t been
added without a reason) (b) no, sacrificing type safety does not make
sense, as the experience with Java’s wildcards shows that this is not
needed.
I would call this “interesting information,” but hardly conclusive.
Java's generics are almost exactly the same thing as Objective-C
lightweight generics, which are less capable and less expressive in
many ways than Swift's generics.
I agree that Java does not have something like `Self` or associated types (which are really useful for not having to bind all type parameters explicitly, especially when binding type parameters to other generics which makes for long type parameter lists in Java where I have to repeat everything over and over again), but do you mean something else here?
Especially in the context of sacrificing type safety?
Especially if something like path dependent types is used like
proposed and some notation to open an existential’s type is added,
which is both something that Java does not have.
(2) misconception: POP is different from OOP
It is not. Protocols are just interfaces using subtyping like OOP has
always done. They just use associated types instead of explicit type
parameters for generics (see above).
They are not the same thing at all (see above ;->). To add to the list
above, protocols can express fundamental relationships—like Self
requirements—that OOP simply can't handle.
Eiffel has something like Self, it is called anchoring and allows binding the type of a variable to that of another one or self (which is called `Current` in Eiffel). And Eiffel does model everything with classes which may be abstract and allow for real multiple inheritance with abilities to resolve all conflicts including those concerning state (which is what other languages introduce interfaces for to avoid conflicts concerning state while still failing to solve *semantic* conflicts with the same diamond pattern).
No protocols or interfaces needed. Why do you say this is not OOP? The book which describes Eiffel is called "Object-Oriented Software Construction" (and is now about 20 years old).
There's a reason Java can't
express Comparable without losing static type-safety.
You are certainly right that Java is not the best language out there especially when talking about type systems (I often enough rant about it :-) but I'm not sure what you mean here. Java's Comparable<T> seems quite typesafe to me. Or do you mean that one could write `class A implements Comparable<B>` by mistake? That's certainly a weak point but doesn't compromise type safety, does it?
Ceylon has an elegant solution for that without using Self types:
interface Comparable<in Other> of Other given Other satisfies Comparable<Other> {...}
Note the variance annotation (which Swift currently has not) and the `of` which ensures that the only subtype of Comparable<T> is T. This is a nice feature that I haven't seen often in programming languages (only Cecil comes to mind IIRC) and which is used for enumerations as well in Ceylon. In Swift I cannot do this but can use Self which solves this problem differently, albeit with some drawbacks compared to Ceylon's solution (having to redefine the compare method in all subtypes, which has lead to lengthy discussion threads about Self, StaticSelf, self etc.).
Finally, in a
language with first-class value types, taking a protocol-oriented
approach to abstraction leads to *fundamentally* different designs from
what you get using OOP.
Eiffel has expanded types which are value types with copy semantics quite like structs in Swift. These expanded types are pretty much integrated into Eiffel's class-only type system. Just define a class as `expanded` and you are done. Eiffel seems to have no need to introduce interfaces or protocols to the language to support value types.
You can even derive from expanded classes which is currently not possible in Swift but has already been discussed several times on this mailing list.
Polymorphic usage is only possible for non expanded super types, which means as far as I understood that a reference is used in that case. Variables with an expanded type do not use refences and therefore may not be used polymorphically in Eiffel.
This should be similar in Swift, at least as far as I did understand it. The question whether variables with a value type can be used polymorphically currently does not arise in Swift as structs cannot inherit from each other (yet?).
The more important distinction of Swift is emphasizing value types and
making mutation safely available by enforcing copy semantics for value
types.
We don't, in fact, enforce copy semantics for value types. That's
something I'd like to change. But regardless, value types would be a
*lot* less useful if they couldn't conform to protocols, and so they
would be a lot less used. Heck, before we got protocol extensions in
Swift 2, there was basically *no way* to share implementation among
value types. So you can't take protocols out of the picture without
making value types, and the argument for value semantics, far weaker.
Value types are coming to the jvm without protocols.
Why? Like I said, Eiffel *has* value types without needing protocols. They just have a unified mechanism built around classes.
But protocols are not really different from interfaces in Java.
I would have preferred a unified model using just classes with real
multiple inheritance like Eiffel has and value types just being a part
of that similar to Eiffel’s `expanded` classes. But that ship has
probably sailed a long time ago :-/ So be it. But at least there
should be no reasons for POP vs OOP wars ;-) (I’d like to add that I
liked Dave’s talks at last WWDC very much, it’s just that I don’t
think that POP is something new or different.)
Protocol-oriented programming is about the synergy of features and ideas
most of which not *individually* new, but that together create a new
world of possibilities. I've already discussed the synergy of protocols
and first-class value semantics.
Again, I think that Eiffel demonstrates that protocols are not needed for first-class value semantics.
So does the clr today and soon the jvm.
There's also the fact that in
protocols we have one construct with which to express dynamic
polymorphism (existentials) and static polymorphism (generic
constraints), both of which have important roles to play but that I
maintain are very different indeed.
I don't argue with the fact that protocols can be used for both of these roles (existentials and constraints) and that these roles are something different. But I didn't say anything like that, at least not intentionally.
One result is that you can “start
with a protocol” as your abstraction mechanism and know that you're not
going to design yourself into a corner where it becomes impossibly
awkward to express what you need.
Not sure what problems you are hinting at here specifically? Is it the single inheritance restriction most languages impose on classes? In that case I agree with you that that is a problem. I would not agree with interfaces being the only or even best solution, though, because I would prefer classes with real support for multiple inheritance like Eiffel shows is possible (especially as interfaces do not solve all problems with MI).
Finally—and I'm certain this *is*
new—in protocol extensions we have a means to express both post-hoc
conformance and generic functions that is much more accessible to users
than in any previous language, to the point where generic programming
can become a natural part of everyday work.
Protocol extensions are a great feature that I was very very happy to see after having used extensions in Smalltalk for many years and missing them in Java. Especially being able to use post-hoc conformance is a really nice feature!
Please don't get me wrong, I think Swift is a fantastic language and I love it.
I'm not arguing for removal of protocols in Swift or something like that. That ship has sailed. But I don't think that classes should be second class citizens in Swift, like several posts on this mailing list implied, e.g. in the discussion about abstract classes.
I don't think the core team views them as second-class considering the restructuring that went in jn Foundation, i.e pairing Data with NSData ... val+ref is the arghment Dave was making a few weeks ago in the discussion around PureValue
···
On Jun 16, 2016, at 3:29 PM, Thorsten Seitz via swift-evolution <swift-evolution@swift.org> wrote:
Am 13.06.2016 um 04:04 schrieb Dave Abrahams <dabrahams@apple.com>:
on Fri Jun 10 2016, Thorsten Seitz <tseitz42-AT-icloud.com> wrote:
Am 09.06.2016 um 19:50 schrieb Thorsten Seitz via swift-evolution <swift-evolution@swift.org>:
Am 09.06.2016 um 18:49 schrieb Dave Abrahams via swift-evolution <swift-evolution@swift.org>:
on Wed Jun 08 2016, Jordan Rose <swift-evolution@swift.org> wrote:
On Jun 8, 2016, at 13:16, Dave Abrahams via swift-evolution >>>>>>> <swift-evolution@swift.org> wrote:
on Wed Jun 08 2016, Thorsten Seitz >>>>>> >>>>>>> <swift-evolution@swift.org >>>>>>> <mailto:swift-evolution@swift.org>> >>>>>>> wrote:
Ah, thanks, I forgot! I still consider this a bug, though (will have
to read up again what the reasons are for that behavior).
Yes, but in the case of the issue we're discussing, the choices are:
1. Omit from the existential's API any protocol requirements that depend
on Self or associated types, in which case it *can't* conform to
itself because it doesn't fulfill the requirements.
2. Erase type relationships and trap at runtime when they don't line up.
Matthew has been arguing against #2, but you can't “fix the bug” without
it.
#1 has been my preference for a while as well, at least as a starting
point.
I should point out that with the resyntaxing of existentials to
Any<Protocols...>, the idea that Collection's existential doesn't
conform to Collection becomes far less absurd than it was, so maybe this
is not so bad.
I think the problem is more that Any<Collection> does not conform to
a specific value for a type parameter T: Collection
What I mean by this is that `Collection` denotes a type family, a
generic parameter `T: Collection` denotes a specific (though
unknown) member of that type family and `Any<Collection>` denotes
the type family again, so there is really no point in writing
Any<Collection> IMO.
The type family cannot conform to T because T is just one fixed member of it.
It conforms to itself, though, as I can write
let c1: Any<Collection> = …
let c2: Any<Collection> = c1
That’s why I think that we could just drop Any<Collection> and simply write Collection.
Let me expand that a bit:
Actually all this talk about existentials vs. generics or protocols
vs. classes has had me confused somewhat and I think there are still
some misconceptions present on this list sometimes, so I’ll try to
clear them up:
There are several objectively incorrect statements here, and several
others with which I disagree. I was hoping someone else would write
this for me, but since the post has such a tone of authority I feel I
must respond.
You are right, the tone of my post was not appropriate, for which I
want to apologize sincerely.
My fundamental disagreement is with the content, not the tone.
I still believe my statements to be valid, though, and will respond to
your arguments inline. Please don't get me wrong, I'm not trying to
have an argument for the argument's sake. All I want is to contribute
maybe a tiny bit to make Swift even better than it already is, by
sharing ideas and thoughts not only from me but from the designs of
other perhaps more obscure programming languages which I happen to
have stumbled upon in the past (often with much delight).
And I want you to know, even though I disagree with what you've written,
that I very much appreciate the contribution you're making.
(1) misconception: protocols with associated types are somehow very
different from generics
I don’t think they are and I will explain why. The only difference is
the way the type parameters are bound: generics use explicit parameter
lists whereas protocols use inheritance. That has some advantages
(think long parameter lists of generics) and some disadvantages.
These ways are dual in a notation sense: generic types have to have
all parameters bound whereas protocols cannot bind any of them.
The „existential“ notation `Any<>` being discussed on this list is
nothing more than adding the ability to protocols to bind the
parameters to be used just like Java’s wildcards are adding the
opposite feature to generics, namely not having to bind all
parameters.
Protocols and generics fulfill completely different roles in Swift, and
so, **especially in a language design context like the one we're in
here**, must be thought of differently. The former are an abstraction
mechanism for APIs, and the latter a mechanism for generalizing
implementations.
That's not what I was talking about. Of course, protocols are a
mechanism for deriving types from each other whereas generics are a
way to parameterize types. My point was that Swift's other way to
parameterize types, namely by associated types, is very similar to
generics with wildcards when looking a the existentials of such
protocols. In addition I was talking about generics in general, not
just about generics in Swift which restricts them to implementations
and does not support wildcards.
I'm aware of these other systems. One of the problems with the way
you're writing about this is that we're speaking in the context of Swift
and you're assuming a completely open design space, as though Swift's
choice to sharply distinguish classes from protocols was not a conscious
one... but it was. Yes, Swift could have been designed differently, so
that a single language construct, a kind of generic class, was stretched
so it could express almost everything. Personally, I don't believe that
results in a better language.
Other languages like Java offer generics for interfaces as well and
support wildcards (adding generic types parameters to protocols in
Swift is currently discussed on the mailing list as well). FWIW my
arguments were not about whether we should have wildcards in Swift or
not, but simply to relate one parametrization feature (associated
types) to a more well known parametrization feature (generics with
wildcards) in order to understand them better.
The only place you could argue that they intersect is
in generic non-final classes, because a class fills the dual role of
abstraction and implementation mechanism (and some might say that's a
weakness). But even accounting for generic classes, protocols with
associated types are very different from generics. Two utterly
different types (an enum and a struct, for example) can conform to any
given protocol P, but generic types always share a common basis
implementation.
The latter is not the case for generic interfaces in Java, for
example, so it is just an artificial restriction present in Swift.
It's not an artificial restriction, it's a design choice. Sure, if by
“generic type” you just mean anything that encodes a static type
relationship, lots of things fall into that bucket.
There is no way to produce distinct instances of a generic type with
all its type parameters bound,
That is true in Swift (except for generic classes) due to the
restriction just mentioned.
but for any protocol P I can make infinitely many instances of P with
P.AssociatedType == Int.
This likewise applies to generic interfaces and for generic types in
general if taking inheritance into account - just like you do here for
protocols.
Back to the my original point: while protocols and generic types have
some similarities, the idea that they are fundamentally the same thing
(I know you didn't say *exactly* that, but I think it will be read that
way) would be wrong and a very unproductive way to approach language
evolution.
I said that protocols *with associated types* are much like generics
*with wildcards* and tried to show why.
If all you're trying to do is say that there's an analogy there, then we
have no argument.
Essentially `Any<Collection>` in Swift is just the same as
`Collection<?>` in Java (assuming for comparability’s sake that
Swift’s Collection had no additional associated types; otherwise I
would just have to introduce a Collection<Element, Index> in Java).
I don't see how you can use an example that requires *assuming away*
assoociated types to justify an argument that protocols *with associated
types* are the same as generics.
Note, that I said *additional* associated types, i.e. in addition to
.Element, even giving an example how the Java interface had to be
extended by a type parameter `Index` if this assumption was not
applied (still simplifying because Generator would have been more
correct which would have to be added as type parameter in addition to
`Index`).
So, in essence the comparison is between the following (I'm using Foo
now instead of Collection to avoid the differences mentioned. Note
that this has no impact on the argument at all):
protocol Foo {
associatedtype T
...
}
interface Foo<T> {
...
}
Yes, those correspond.
My argument is that existentials of protocols with associated types
are just like generic types with wildcards, i.e. `Any<Foo>` in Swift
is just the same as `Foo<?>` in Java.
Likewise `Any<Foo where .T: Number>` is just the same as `Foo<?
extends Number>` in Java. For me that was an insight I wanted to
share.
It's a good one.
And just like Collection<?> does not conform to a type parameter `T
extends Collection<?>` because Collection<?> is the type `forall
E. Collection<E>` whereas `T extends Collection<?>` is the type
`T. Collection<T>` for a given T.
In essence protocols with associated types are like generics with
wildcards.
It is true that generics with wildcards in Java *are* (not just “like”)
existential types but I don't agree with the statement above. Because
Java tries to create an “everything is a class” world, generic classes
with bound type parameters end up playing the role of existential type.
But protocols in Swift are not, fundamentally, just existential types,
and the resyntaxing of ProtocolName to Any<ProtocolName> for use in type
context is a huge leap forward in making that distinction clear... when
that's done (unless we leave Array<ProtocolName> around as a synonym for
Array<Any<ProtocolName>>—I really hope we won't!) protocols indeed
*won't* be types at all, existential or otherwise.
I fully agree that protocols are not types, their existentials
are. But I haven't seen yet what we really *gain* from making that
distinction explicit (except an ugly type syntax :-).
For me, it helps distinguish static from dynamic polymorphism.
And like I already wrote in this or another thread we would have to
apply the same logic to non-final classes, which are existentials,
too.
Coming back to the questions whether (a) allowing existentials to be
used as types is useful
That's the only use existentials have. They *are* types. Of course
they're useful, and I don't think anyone was arguing otherwise.
I'm pretty sure that there was a discussion about whether being able
to write something like Any<Collection> is useful. My wording was
certainly imprecise, though, and didn't make sense as written. I
should have said something like "whether adding the ability to use
existential types of protocols with unbound associated types is
useful".
and (b) whether sacrificing type safety would somehow be necessary for
that, I think we can safely answer (a) yes, it *is* useful to be able
to use existentials like Any<Collection> as types, because wildcards
are quite often needed and very useful in Java (they haven’t been
added without a reason) (b) no, sacrificing type safety does not make
sense, as the experience with Java’s wildcards shows that this is not
needed.
I would call this “interesting information,” but hardly conclusive.
Java's generics are almost exactly the same thing as Objective-C
lightweight generics, which are less capable and less expressive in
many ways than Swift's generics.
I agree that Java does not have something like `Self` or associated
types (which are really useful for not having to bind all type
parameters explicitly, especially when binding type parameters to
other generics which makes for long type parameter lists in Java where
I have to repeat everything over and over again), but do you mean
something else here?
Especially in the context of sacrificing type safety?
I do, but it will take some research for me to recover my memory of
where the holes are. It has been years since I thought about Java
generics. It's also possible that I'm wrong ;-)
Especially if something like path dependent types is used like
proposed and some notation to open an existential’s type is added,
which is both something that Java does not have.
(2) misconception: POP is different from OOP
It is not. Protocols are just interfaces using subtyping like OOP has
always done. They just use associated types instead of explicit type
parameters for generics (see above).
They are not the same thing at all (see above ;->). To add to the list
above, protocols can express fundamental relationships—like Self
requirements—that OOP simply can't handle.
Eiffel has something like Self, it is called anchoring and allows
binding the type of a variable to that of another one or self (which
is called `Current` in Eiffel). And Eiffel does model everything with
classes which may be abstract and allow for real multiple inheritance
with abilities to resolve all conflicts including those concerning
state (which is what other languages introduce interfaces for to avoid
conflicts concerning state while still failing to solve *semantic*
conflicts with the same diamond pattern).
No protocols or interfaces needed. Why do you say this is not OOP? The
book which describes Eiffel is called "Object-Oriented Software
Construction" (and is now about 20 years old).
It's not *incompatible* with OOP, but it is not part of the essence of
OOP either. If you survey object-oriented languages, what you find in
common is inheritance-based dynamic polymorphism and reference
semantics. Those are the defining characteristics of OOP, and taking an
object-oriented approach to a given problem means reaching for those
features.
There's a reason Java can't
express Comparable without losing static type-safety.
You are certainly right that Java is not the best language out there
especially when talking about type systems (I often enough rant about
it :-) but I'm not sure what you mean here. Java's Comparable<T> seems
quite typesafe to me. Or do you mean that one could write `class A
implements Comparable<B>` by mistake? That's certainly a weak point
but doesn't compromise type safety, does it?
Java has cleverly avoided compromising type safety here by failing to
express the constraint that comparable conformance means a type can be
compared to itself ;-)
Ceylon has an elegant solution for that without using Self types:
interface Comparable<in Other> of Other given Other satisfies Comparable<Other> {...}
Note the variance annotation (which Swift currently has not) and the
`of` which ensures that the only subtype of Comparable<T> is T. This
is a nice feature that I haven't seen often in programming languages
(only Cecil comes to mind IIRC) and which is used for enumerations as
well in Ceylon. In Swift I cannot do this but can use Self which
solves this problem differently, albeit with some drawbacks compared
to Ceylon's solution (having to redefine the compare method in all
subtypes,
That sounds interesting but is a bit vague. A concise example of how
this plays out in Swift and in Ceylon would be instructive here.
which has lead to lengthy discussion threads about Self, StaticSelf, self etc.).
Finally, in a
language with first-class value types, taking a protocol-oriented
approach to abstraction leads to *fundamentally* different designs from
what you get using OOP.
Eiffel has expanded types which are value types with copy semantics
quite like structs in Swift. These expanded types are pretty much
integrated into Eiffel's class-only type system. Just define a class
as `expanded` and you are done.
Unless this part of the language has changed since 1996, or unless I've
misread https://www.cs.kent.ac.uk/pubs/1996/798/content.pdf, you can't
make an efficient array with value semantics in Eiffel. That, IMO,
cannot be considered a language with first-class value types.
Eiffel seems to have no need to introduce interfaces or protocols to
the language to support value types.
No, of course not. By saying that everything from abstract interfaces
to static constraints and even value types is to be expressed a kind of
possibly-generic class, you can eliminate distinctions in the language
that IMO help to clarify design intent. This is a language design
choice one could make, but not one I'd want to. In LISP, everything is
an S-expression. That has certain upsides, but for me it fails the
expressivity test.
You can even derive from expanded classes which is currently not
possible in Swift but has already been discussed several times on this
mailing list. Polymorphic usage is only possible for non expanded
super types, which means as far as I understood that a reference is
used in that case. Variables with an expanded type do not use refences
and therefore may not be used polymorphically in Eiffel. This should
be similar in Swift, at least as far as I did understand it. The
question whether variables with a value type can be used
polymorphically currently does not arise in Swift as structs cannot
inherit from each other (yet?).
The more important distinction of Swift is emphasizing value types and
making mutation safely available by enforcing copy semantics for value
types.
We don't, in fact, enforce copy semantics for value types. That's
something I'd like to change. But regardless, value types would be a
*lot* less useful if they couldn't conform to protocols, and so they
would be a lot less used. Heck, before we got protocol extensions in
Swift 2, there was basically *no way* to share implementation among
value types. So you can't take protocols out of the picture without
making value types, and the argument for value semantics, far weaker.
Why? Like I said, Eiffel *has* value types without needing
protocols. They just have a unified mechanism built around classes.
Because I'm speaking about Swift, not some other world where Protocol ==
Generic Class ;-)
···
on Thu Jun 16 2016, Thorsten Seitz <tseitz42-AT-icloud.com> wrote:
Am 13.06.2016 um 04:04 schrieb Dave Abrahams <dabrahams@apple.com>:
on Fri Jun 10 2016, Thorsten Seitz <tseitz42-AT-icloud.com> wrote:
Am 09.06.2016 um 19:50 schrieb Thorsten Seitz via swift-evolution <swift-evolution@swift.org>:
Am 09.06.2016 um 18:49 schrieb Dave Abrahams via swift-evolution <swift-evolution@swift.org>:
on Wed Jun 08 2016, Jordan Rose <swift-evolution@swift.org> wrote:
On Jun 8, 2016, at 13:16, Dave Abrahams via swift-evolution >>>>>>> <swift-evolution@swift.org> wrote:
on Wed Jun 08 2016, Thorsten Seitz >>>>>> >>>>>>> <swift-evolution@swift.org >>>>>>> <mailto:swift-evolution@swift.org>> >>>>>>> wrote:
Ah, thanks, I forgot! I still consider this a bug, though (will have
to read up again what the reasons are for that behavior).
Yes, but in the case of the issue we're discussing, the choices are:
1. Omit from the existential's API any protocol requirements that depend
on Self or associated types, in which case it *can't* conform to
itself because it doesn't fulfill the requirements.
2. Erase type relationships and trap at runtime when they don't line up.
Matthew has been arguing against #2, but you can't “fix the bug” without
it.
#1 has been my preference for a while as well, at least as a starting
point.
I should point out that with the resyntaxing of existentials to
Any<Protocols...>, the idea that Collection's existential doesn't
conform to Collection becomes far less absurd than it was, so maybe this
is not so bad.
I think the problem is more that Any<Collection> does not conform to a specific value for a type parameter T: Collection
What I mean by this is that `Collection` denotes a type family, a generic parameter `T: Collection` denotes a specific (though unknown) member of that type family and `Any<Collection>` denotes the type family again, so there is really no point in writing Any<Collection> IMO.
The type family cannot conform to T because T is just one fixed member of it.
It conforms to itself, though, as I can write
let c1: Any<Collection> = …
let c2: Any<Collection> = c1
That’s why I think that we could just drop Any<Collection> and simply write Collection.
Let me expand that a bit:
Actually all this talk about existentials vs. generics or protocols vs. classes has had me confused somewhat and I think there are still some misconceptions present on this list sometimes, so I’ll try to clear them up:
(1) misconception: protocols with associated types are somehow very different from generics
I don’t think they are and I will explain why. The only difference is the way the type parameters are bound: generics use explicit parameter lists whereas protocols use inheritance. That has some advantages (think long parameter lists of generics) and some disadvantages.
These ways are dual in a notation sense: generic types have to have all parameters bound whereas protocols cannot bind any of them.
The „existential“ notation `Any<>` being discussed on this list is nothing more than adding the ability to protocols to bind the parameters to be used just like Java’s wildcards are adding the opposite feature to generics, namely not having to bind all parameters.
As you know I like using `&` as type intersection operator. But you write "The syntax leave a void when it comes to expressing the so called Top type:“
Why? Just give the top type a name, e.g. `Any` and you are done.
Yes.. I just don't like magic. I think that all types should be expressable with a syntax and that you can then decide to alias one parsing case with a specific name. Not the other way around.
Well, I think that would be backwards, because we have a nominal type system. That means that the syntax for a type is just its name.
`&` is a type operator which creates a new type from its operands.
`where` clauses add constraints to types.
But it all starts with a type's name.
The only magic would be that all type definitions (`protocol` etc.) which do not give a supertype they conform to, will implicitly conform to `Any`, i.e.
protocol Foo { … }
means
protocol Foo : Any { … }
That’s less magic than creating a special syntax just to express the top type.
Why should the top type have special *syntax*? It is just the type all other types conform to. No need to do something special here and therefore no need to invent an alternative syntax like `Any<>` or the alternative from your gist which is rather confusing IMO (and I don’t like the special case given to classes in the syntax).
The _ case is just a degenerate case of the syntax, showing that it is expressible inside, as opposoed to have to define a contextual keyword (read the SourceKit code). And then it is aliasable. Part of the problems in swift today to me is that some things are no doable in swift, so the compiler must contain that semantic. This here is just one example, but there are others. These are notions that the standard library has to defer to the compiler. Some of them have been tabled for 4.0 it seems. My point here was that it is not fate, and choosing the syntax carefully for existentials (whichever it is, i don't really care inthe end) would prevent having to add a magic keyword for to top type to the compiler and to SourceKit again (they are trying to remove them).
`Any` would *not* be a keyword. It is just a type name like `Collection` or `Int`. Like I said, the only magic would be in adding `: Any` to type definitions without a conforming clause.
As for the current Any<...> proposal for generalizing existentials it is IMHO cluncky and magic. There is nothing stopping us mechanically from entering Any<UITableView, UIButton>. To me that is the holemark of bad design. In what I tested you just can't do it and the reason is clear. Of course when I
`Any<UITableView, UIButton>` is just an intersection type and would be written UITableView & UIButton. And, yes, that would be ok, because it would be just the empty set, i.e. the bottom type (which has no members). There is no magic involved, it is just the normal result of an intersection: each type is a set containing all instances of this type (instances conforming to it). Intersecting two sets might result in an empty set. The type denoting the empty set is the bottom type which is the subtype of all types and might be called `Nothing` or `Never` or `Bottom`.
Ceylon makes very nice use of type intersections and type unions and the beautiful thing is that these type operations really just work like you would expect if you think of types as sets (which is the standard definition for a type AFAIK). No surprises and no magic there!
So it is *not* a sign of bad design, quite to the contrary! What did you test it with? Probably not Ceylon, because otherwise you would have seen that it just works.
say you can't I do not mean that your keyboard will zap u if you try. But the way to verify correctness is different... and in a world where xcode would be a good ide, then code completion would even show you why and actually truly assist (because the grammar i tested makes really makes it possible). I know that xcode is only as good as SourceKit lets it be.
Essentially `Any<Collection>` in Swift is just the same as `Collection<?>` in Java (assuming for comparability’s sake that Swift’s Collection had no additional associated types; otherwise I would just have to introduce a Collection<Element, Index> in Java).
Likewise `Any<Collection where .Element: Number>` is just the same as `Collection<? extends Number>` in Java.
Java’s wildcards are a way to express use-site variance. The proposed `Any<>` does just the same.
And just like Collection<?> does not conform to a type parameter `T extends Collection<?>` because Collection<?> is the type `forall E. Collection<E>` whereas `T extends Collection<?>` is the type `T. Collection<T>` for a given T.
This picture is accurate today, but there are going to be more serious differences after 10 no date is currently geven for when it will come)
You mean Java 10?
Yes. After 10. Than is their only date hint.
In essence protocols with associated types are like generics with wildcards.
Yes, java has kept everything within its generics system rather than split parts out. Something people may not immediately think about with respect to the 2 generic systems is that when u call a func<T>capture(T t){} in java with a wildcard you are doing a compile time capture only (to avoid the dreaded unsafe casts), whereas it is really nice to do the same in swift and subsequently be able to access T.Type and see that it is not Any. The closest u ever get to that type at runtime in java is via generics introspection, but u still can't do everything ( like no new T() ). But today the bridging between existential types and generics is definitely a work in progress.
Coming back to the questions whether (a) allowing existentials to be used as types is useful and (b) whether sacrificing type safety would somehow be necessary for that, I think we can safely answer
(a) yes, it *is* useful to be able to use existentials like Any<Collection> as types, because wildcards are quite often needed and very useful in Java (they haven’t been added without a reason)
IMO they made java 8 (referring to streams). And even though the syntax for co/contra variance is pretty heavy, it is the foundation for all modern java code. The any-fication of the generics is going to open new doors as some of it will translate into a partial reification in the jvm. It seems the decision for now is to not use the extra info in java to retain binary compatibility with all the erased code out there, this is something scala might use in areas where it won't mind loosing java compatibility.
(b) no, sacrificing type safety does not make sense, as the experience with Java’s wildcards shows that this is not needed. Especially if something like path dependent types is used like proposed and some notation to open an existential’s type is added, which is both something that Java does not have.
I hope typesafe opening inside the " if let " syntax gets added. I know that chris is against sugaring, but I played if an implementation of
... x is String?
If let! x {}
That runs as
if let x = x {}
something equally short could be done here.
(2) misconception: POP is different from OOP
It is not. Protocols are just interfaces using subtyping like OOP has always done. They just use associated types instead of explicit type parameters for generics (see above). The more important distinction of Swift is emphasizing value types and making mutation safely available by enforcing copy semantics for value types.
Values are coming to the jvm, which will narrow this gap (like their view identity for value vs ref and the whole deep ==) . I also really like the cow approach of the swift runtime.
But protocols are not really different from interfaces in Java.
There is one big difference: default methods, but it seems swift will add that soon.
Swift already has default extension methods, doesn’t it?
Yes, and no. It has differently dispatched code that can be found to fill in the gap of the conformance req, yes. But Joe Grof (?) said that there are no reasons why these could not be added. Having true defaults could be one way to deal with optional comformance…
The dispatch issue only arises for extensions introducing a method that is not declared in a protocol. That is something you *cannot* do in Java. Java’s default methods are implementations for methods declared in interfaces. Swift’s extension methods providing defaults for methods declared in a protocol are dynamically dispatched and should work like Java’s default methods.
(One caveat exists with subclasses where the superclasses didn’t implement the method because I then am not allowed to `override` the default method but that is a bug IMO).
I also really like how extensions and conformance mix together in swift to bake retro-modelling in and the adapter pattern (spent enough years deep diving inside eclipse to appreciate it).
I would have preferred a unified model using just classes with real multiple inheritance like Eiffel has and value types just being a part of that similar to Eiffel’s `expanded` classes. But that ship has probably sailed a long time ago :-/
I like extensions very much (having used Smalltalk for a long time). I like enums for the pattern matching and structs as value types. But having to split protocols off instead of using abstract classes makes things more complicated IMO.
-1 i am old school c/c++... i really like protocol, struct, class, extensions, enums. It is a really nice mix that gives objc people room to grow, but I do miss how they are an integral part of generics (i protocols as a replacement and look forward to when they interact better) and namespaces+scoped-imports (c#)... Looking forward to where things go next
Yeah, namespacing/submodules/conflict resolution (when doing imports but also when conforming to multiple protocols which has just the same problems) are still missing. But I’m optimistic :-) Let’s complete generics first, then tackle existentials/type intersections.
I think I care more about existentials, but only because of the kind of java i wrote for a living. I just looked at a bunch of opensource swift libs (particularly for server side swift)... some of it is a real engineering disaster: 20+ folders, each with 2 source files... or 3 folders "extensions" "utils" "classes". Swift is currently not equiped for people to write big things with... I wish the team would address it sooner than later (look at their own c++ code to see the difference). IMHO import conflicts are more often the symptom of bad code than a real issue.
Alas, most languages suffer from poor module systems. A good module system should not only allow resolving conflicts between modules and making it possible to structure code well, but solve the issue around versioned modules so that is is possibly to safely use different versions of the same module in the same application.
-Thorsten
···
Am 11.06.2016 um 12:38 schrieb L. Mihalkovic <laurent.mihalkovic@gmail.com>:
On Jun 11, 2016, at 11:30 AM, Thorsten Seitz <tseitz42@icloud.com> wrote:
Am 11.06.2016 um 08:00 schrieb L. Mihalkovic <laurent.mihalkovic@gmail.com>:
On Jun 10, 2016, at 9:35 PM, Thorsten Seitz via swift-evolution <swift-evolution@swift.org> wrote:
Am 09.06.2016 um 19:50 schrieb Thorsten Seitz via swift-evolution <swift-evolution@swift.org>:
Am 09.06.2016 um 18:49 schrieb Dave Abrahams via swift-evolution <swift-evolution@swift.org>:
on Wed Jun 08 2016, Jordan Rose <swift-evolution@swift.org> wrote:
On Jun 8, 2016, at 13:16, Dave Abrahams via swift-evolution >>>>>>>> <swift-evolution@swift.org> wrote:
on Wed Jun 08 2016, Thorsten Seitz >>>>>>> >>>>>>>> <swift-evolution@swift.org >>>>>>>> <mailto:swift-evolution@swift.org>> >>>>>>>> wrote:
Cheers
-THorsten
So be it. But at least there should be no reasons for POP vs OOP wars ;-)
(I’d like to add that I liked Dave’s talks at last WWDC very much, it’s just that I don’t think that POP is something new or different.)
I used to thin that way. But today I think that although in broad brush strokes the similarities and bigger than the differences, there is room for making a bigger difference in the how.
That's certainly a valid question. I have been very vocally skeptical
that HKTs like Monad belong in the standard library, but I am not
opposed to having the language feature if it supports important use
cases.
It is surprising to see people debating the place of monads in Swift when nobody seems to speak about the elephant in the middle of the room: the language has a real engineering defisciency if it is possible for people to think that there is no better way to use it than to do "we have around 50+ packages. This list grows very fast" (http://docs.zewo.io - many more swift github projects are equally struggling to get the code organized).
At the moment the only thing preventing the situation from getting more out of hands is that the import mechanism currently gets in the way, but the only opinions I read so far all focussed on making it even easier to slap things together ala zewo rather than address the engineering issue.
It would be helpful if you could do two things:
1. Be clear about exactly what you think is wrong and, preferably, how you think we might be able to solve it, or at least what sorts of problems you want the solution to address. Reading this post, I'm not sure what you find objectionable about this project containing 50 packages or what "situation" you think is getting out of hand. Even if you have a good point, you're not making it very well.
I am assuming that you have been doing professional software development for a long time, worked with 10s of developers in dozens of teams, using objc, swift, and many more languages. I think it is safe to assume you spent a great deal of time reading the compiler (and llvm's) source code since it became open source, and considering how regularly you refer to what others need or use, I also took for granted that you have read the code for a great deal of open-source projects in all sorts of languages. Having assumed this as your background, you might appreciate why I thought we might be able to skip of few steps.
2. Post about your issue in a new thread rather than trying to derail something *entirely* unrelated to it. I'm not sure exactly what your complaint is, but I gather it has something to do with package management, modules, or imports. None of these have anything to do with higher-kinded types, the type system, or (as far as I can tell) anything that has ever been discussed in this thread. The result is rather like interrupting a charity drive for the local children's hospital to tell people that actually, they really *ought* to be donating to your mosquito-net charity instead. Even if you're correct that your cause is more important, it's inappropriate, unwelcome, and frankly just rude.
I can appreciate your view. And maybe you can equally appreciate how plain surreal parts of this thread and a few others might appear when put in perspective with some of what I, undoubtedly
wrongly, perceive as elephants in the room.
···
On Jun 11, 2016, at 8:46 PM, Brent Royal-Gordon <brent@architechies.com> wrote:
Protocols are a mechanism for deriving types from each other whereas generics are a way to parameterize types. My point was that Swift's other way to parameterize types, namely by associated types, is very similar to generics with wildcards when looking a the existentials of such protocols.
This has been a point of confusion for me as well. I keep hearing that associated types are different from generic protocols, but this seems like a distinction without a difference.
Suppose Swift allowed generic protocols. How would a hypothetical Collection<Foo> be different in practice from the proposed existential Any<Collection where .Element == Foo>?
If Collection were a Java-like generic protocol you would have 5 generic parameters, all of which must be explicitly provided with arguments when forming an existential, although the arguments could be wildcards. This is a bit unwieldy. You would probably end up using a typealias in either case.
Yes, in the realm of type theory and compiler internals they might represented differently, sure. But in practice, in terms of what code can actually do? I know of only two differences:
1. A type can only conform to any given protocol with one set of type parameters. (Nothing can be both Collection<Foo> and Collection<Bar>.)
This is a pretty huge difference. Multiple conformances are on the unlikely list for Swift and generic protocol syntax implies multiple conformances are possible (as is the case in at least some languages).
···
On Jun 16, 2016, at 10:36 AM, Paul Cantrell via swift-evolution <swift-evolution@swift.org> wrote:
On Jun 16, 2016, at 8:29 AM, Thorsten Seitz via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
2. When a type conforms to Collection, it declares “associatedtype Foo” instead of “: Collection<Foo>”, and Foo can be inferred by the compiler in some circumstances. That’s handy, but it’s a syntactic difference.