Make generics covariant and add generics to protocols

I should be clear, I am not proposing the Java way. I haven't seen a
language that uses the system I am proposing where the instance carries a
field that says what type it accepts and that is checked at runtime. From
using this pattern in my own code it seems to work better than either the
current associated types and also the Haskel/Java/Scala type erasure
systems.

However since it is only tested in the sort of code I write it could be
that I haven't turned up all the problems. Do you have a *short* example of
where a Java system failed so that I can see how it would go using this
approach?

Hope you can find a suitable example,

-- Howard.

¡¡¡

On 13 January 2016 at 13:07, David Waite <david@alkaline-solutions.com> wrote:

Java only has invariance at the generic type definition level. They allow
you to use wildcards as another option to generic constraints for instance
references. In effect, variance is faked by the user of a type, not defined
by the author of the type.

The dynamic and unreified nature of java generics is such that several
valid systems I have encountered are impossible to properly express.
Likewise, several systems I have tried to debug using generics have been
found to have unsound generic type systems due to the language not
enforcing proper static typing.

IMHO Java is not a good language to use as an example when trying to make
a case for changing the behavior of generics.

-DW

On Jan 12, 2016, at 6:47 PM, Howard Lovatt via swift-evolution < > swift-evolution@swift.org> wrote:

Yes you can annotate for covariance, invariance, and contravariance, both
Java and Scala, allow all three. The problem is that the code becomes
splattered with variance annotations. The Java people themselves have
publicly regretted this and wished that they had made covariance the
default. If you look at generic code invariance and covariance are by far
the most common requirements; this proposal would address these common use
case without burdening the programmer.

--
  -- Howard.

@Thorsten,

It doesn't make the type system unsound; the types are mostly checked at
compile time but some are runtime checked, either way the types are checked
and therefore the type system is sound. I have posted an example of array
runtime type checking in a previous response.

You can annotate variance, but this generally adds a lot of clutter to the
code (see some typical Java code that uses generics you will see stuff like
? extends Type everywhere). In other languages like Scala the annotation is
less visually intrusive, because they use shorter syntax and because it is
annotated at declaration site rather than use site, but it is still there.

I think Swift arrays are a good example of partially static and partially
runtime checked. The content is statically typed but the size is runtime
typed. Other languages do type both the content and the size (see Dependent
Typing on Wiki), however at some considerable burden on the programmer.

Hope this explains the thought process,

-- Howard.

¡¡¡

On 13 January 2016 at 16:50, Thorsten Seitz <tseitz42@icloud.com> wrote:

Strong -1 from me as well for making the type system unsound.

> Am 13.01.2016 um 02:47 schrieb Howard Lovatt via swift-evolution < > swift-evolution@swift.org>:
>
> Yes you can annotate for covariance, invariance, and contravariance,
both Java and Scala, allow all three. The problem is that the code becomes
splattered with variance annotations

Ceylon uses a different approach which is variance annotations at the
definition site.
This restricts the possible usage of the type parameters to appropriately
variant positions.

This migt be a better way to deal with variance.

-Thorsten

--
  -- Howard.

Actually, only the equality comparison part will be invariant. Generally, the APIs which call the IComparable and IEquatable methods are disjunct, so it’s not a problem in practice at all (comparable is almost only for sorting, equatable for putting in dictionaries and many other things). The reasoning for making Equatable invariant is explained on MSDN, but it has to do with what being equals means. It’s always a choice, of course.

At any rate, contravariance is definitely sometimes useful, and, I’d argue, just as often as covariance. Most of the time you don’t need either.

I personally regret that C# arrays are covariant (and so do the designers of C#), because it means more errors at runtime, maybe when the code is in production.

-Sune

¡¡¡

On 14 Jan 2016, at 00:02, Howard Lovatt <howard.lovatt@gmail.com> wrote:

Furthermore Microsoft chose to make Equatable<T> invariant. A strange choice heh. How come comparing using IComparable behaves differently than using Equatable? If you implement both, which Microsoft recommends, then you get invariance, because Equatable is invariant.

What uses do you see for covariance defined by the author of a type?

Just chiming in although I'm not David: you could have producer and consumer ointerfaces e.g. for collection types.

Also I agree with you about Java; in particular for me the combination of covariance, Raw Types and Type Erasure create a very confusing system which sometimes seems to be of little benefit.

Yes, for most questions about the type system Java is a bad example...

Scala uses definition site variance annotations instead of use site annotations (like Java's wildcards). Ceylon actually does both but has a strong preference on definition site variance annotations, see Eclipse Ceylon™ | projects.eclipse.org

-Thorsten

¡¡¡

Am 13.01.2016 um 04:08 schrieb Simon Pilkington via swift-evolution <swift-evolution@swift.org>:

It does indeed make the type system unsound in some cases. You have chosen one particular variance direction because it is convenient. A classic counterexample is Java making Arrays covariant by default. So this works

Integer arr = [2, 4, 6, 8, 10];
Object orr = arr;
orr[0] = "crash bang";

And crashes at runtime. For that, I must give this part of the proposal a strong -1. Any amount of type safety I have to give up in the name of convenience is far too much.

I am, however, a fan of generic protocols. They seem like an orthogonal concept given the first part here though.

~Robert Widmann

2016/01/13 17:03、Howard Lovatt via swift-evolution <swift-evolution@swift.org> のメッセージ:

¡¡¡

@Thorsten,

It doesn't make the type system unsound; the types are mostly checked at compile time but some are runtime checked, either way the types are checked and therefore the type system is sound. I have posted an example of array runtime type checking in a previous response.

You can annotate variance, but this generally adds a lot of clutter to the code (see some typical Java code that uses generics you will see stuff like ? extends Type everywhere). In other languages like Scala the annotation is less visually intrusive, because they use shorter syntax and because it is annotated at declaration site rather than use site, but it is still there.

I think Swift arrays are a good example of partially static and partially runtime checked. The content is statically typed but the size is runtime typed. Other languages do type both the content and the size (see Dependent Typing on Wiki), however at some considerable burden on the programmer.

Hope this explains the thought process,

-- Howard.

On 13 January 2016 at 16:50, Thorsten Seitz <tseitz42@icloud.com> wrote:
Strong -1 from me as well for making the type system unsound.

> Am 13.01.2016 um 02:47 schrieb Howard Lovatt via swift-evolution <swift-evolution@swift.org>:
>
> Yes you can annotate for covariance, invariance, and contravariance, both Java and Scala, allow all three. The problem is that the code becomes splattered with variance annotations

Ceylon uses a different approach which is variance annotations at the definition site.
This restricts the possible usage of the type parameters to appropriately variant positions.

This migt be a better way to deal with variance.

-Thorsten

--
  -- Howard.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

It doesn't make the type system unsound; the types are mostly checked at compile time but some are runtime checked, either way the types are checked and therefore the type system is sound.

Runtime type checking is not an adequate replacement for compile-time type checking. Runtime type checking ensures your code does not continue to run if it encounters a type error; compile-time type checking ensures your code is *incapable of encountering* a type error. This is a bit like saying that it's okay to drive drunk because seatbelts and airbags will mitigate the injuries caused if you crash.

Swift has a lovely property where, if your code is typed strongly enough and compiles without errors, it's probably pretty close to working. I would hate to see that watered down.

¡¡¡

--
Brent Royal-Gordon
Architechies

It doesn't make the type system unsound; the types are mostly checked at compile time but some are runtime checked, either way the types are checked and therefore the type system is sound.

Runtime type checking is not an adequate replacement for compile-time type checking. Runtime type checking ensures your code does not continue to run if it encounters a type error; compile-time type checking ensures your code is *incapable of encountering* a type error. This is a bit like saying that it's okay to drive drunk because seatbelts and airbags will mitigate the injuries caused if you crash.

Swift has a lovely property where, if your code is typed strongly enough and compiles without errors, it's probably pretty close to working. I would hate to see that watered down.

+1. Swift's type system is very well thought out. It is going to get even better with the generics enhancements in Swift 3, removing many of the current limitations. We are heading down the right path. Changing course and introducing runtime checks where static checks are possible is a bad idea.

¡¡¡

Sent from my iPad
On Jan 14, 2016, at 2:39 AM, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> wrote:

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

@Simon,

In the Swift code below:

    let array = [1]

    array[0] // 1, OK

    array[1] // Error not detected by compiler but detected at runtime

you have an example of an error detected at runtime that terminates program
execution, it doesn't return an optional.

In other languages this would be a type error, these languages are usually
described as dependent type (https://en.wikipedia.org/wiki/Dependent_type\).
An example of the advantage of this would be vector (or matrix
multiplication), e.g. imagine that Swift had dependent types:

    let row = RowVec(1, 2) // Type: matrix of int with 1 row and 0 columns
(note size is part of the type)

    let col = ColVec(3) // Type: matrix of int with 0 rows and 1 column

    let scaler = row * col // Compile time error because both vectors
should be the same length

At the moment if you wrote a matrix package in Swift the above example
would be a runtime error and not a compile time error, but with dependent
typing it would be a compile time error.

There is another discussion of Swift Evolution on calculable types that are
closely related to dependent typing.

To me you just strike the balance, sometimes static checking is best
sometimes runtime. You strike the balance by how practical it is to do the
static checking, if the burden that the static checking adds to the users
then it isn't worth it. This is the case with annotated variance in
languages like Java and Scala, the annotations do not add much. Hence I am
suggesting system that is simple to use, much like Swift arrays are easy to
use but not totally, but largely, statically typed.

Hope that explains my reasoning for making most type error compile time
checked but a small subset runtime checked,

-- Howard.

¡¡¡

On 13 January 2016 at 14:01, Simon Pilkington <simonmpilkington@icloud.com> wrote:

I’d be interested in reading up on what the Oracle response was if you
have links.

As you mentioned even Swift doesn’t get away from runtime type checking
but in the two examples you mentioned - array out of bounds and casts -
Swift makes use of the Optionals system to highlight that an operation may
fail and allows the user to handle that failure. Covariance should have
similar syntax support (for example use of optional chaining similar to
optional protocol requirements to indicate that a call may fail due to
incorrect types). For the compiler to understand when such failure is
possible, some kind of covariance syntax would be required.

As a related question, do you see covariance syntax as such a burden?

-Simon

On 13 Jan 2016, at 12:47 PM, Howard Lovatt <howard.lovatt@gmail.com> > wrote:

Yes you can annotate for covariance, invariance, and contravariance, both
Java and Scala, allow all three. The problem is that the code becomes
splattered with variance annotations. The Java people themselves have
publicly regretted this and wished that they had made covariance the
default. If you look at generic code invariance and covariance are by far
the most common requirements; this proposal would address these common use
case without burdening the programmer.

Swift, and no usable language, is completely statically typed. Examples in
Swift of runtime type checking are array out of bounds and casts. There are
other examples of non-type related runtime checks is Swift: numerical
overflow, throws, using optionals to signal errors, and using enums to
signal errors. I say use what is appropriate, static type checking if it is
easy to do, otherwise runtime type checking. Note I am not proposing an
unsafe language like C, it is still type checked.

There is a strong positive precedence for a type check on write, Java
arrays (not Java `List`s). Arrays in Java may be passed covariantly, and
this is extensively used. However if you attempt to write the wrong type
into the array you will get an `ArrayStoreException`. In practice you don't
get many `ArrayStoreException`, non of my code has ever had one. Its just
not something you do in practice, as noted before contravariance is rare.

Thanks for you comments and I hope this eases your concerns,

  -- Howard.

On 13 January 2016 at 11:33, Simon Pilkington <simonmpilkington@icloud.com > > wrote:

The problem is that conceptually and behaviourally Box<Bottom> *is indeed
not* a Box<Top> and cannot be treated the same as one. The proposal
attempts to get around this difference with a runtime failure but this
would result in very fragile code - you get passed a Box<Top> and want to
pass it a subclass of Top, will it succeed, who knows. You probably would
be able to check the types but the complier wouldn’t highlight that this is
an operation that could potentially fail.

This seems to be very much against Swift’s goal of safety being enforced
by the compiler as much as possible.

Java uses the wildcard syntax to highlight this conceptual and
behavioural difference - Box<Bottom> is not covariant with Box<Top> but
rather with Box<? extends Top>. The compiler can then enforce that a
programmer doesn’t try to pass an incompatible type to a variable of such
type. Even though this is a complication to the language (many Java
programmers struggle with correctly using the wildcard syntax) I don’t see
covariance for generics being added to Swift in a robust manner without
some kind of similar syntax.

-Simon

On 12 Jan 2016, at 8:45 PM, Howard Lovatt via swift-evolution < >> swift-evolution@swift.org> wrote:

Currently you generics are invariant whereas function arguments etc. are
covariant. I am suggesting that if the way generics are implemented is
changed then they can be made covariant and that this will add considerable
utility to Swift generics.

1st a demonstration of the current situation of invariant generics:

    // Current system
    class Top {}
    class Bottom: Top {}

    struct Box<T: AnyObject> {
        var value: T
        init(_ initialValue: T) {
            value = initialValue;
        }
    }

    let boxB = Box(Bottom())
    // let boxT: Box<Top> = boxB // Covariance currently not allowed

The key point is although `Bottom` 'is a’ `Top`, `Box<Bottom>` *is not* a
`Box<Top>`.

I am suggesting:

1. That `Box<Bottom>` should be a `Box<Top>` (covariance).
2. An implementation that allows the above covariance.
3. That protocols are made generic, i.e. `protocol Box<T> { var value: T
{ get set } }` and that this mechanism replaces associated types for
protocols.

    // Proposal:
    // 1. No change to Box, i.e. programmer would just write Box as
before
    // 2. Code transformed by comiler with write check for each
specific, generic type instance
    // Best approximation of resulting code in current Swift to
demonstrate spirit of idea:

    // Compiler writes a universal form using the upper bound (it writes
the underlyting representation).
    // In practice this would be called `Box` but used `BoxAnyObject` to
indicate that it has a generic argument bounded by `AnyObject`.
    struct BoxAnyObject {
        // Generated from generic argument `<T: AnyObject>`.
        let T: AnyObject.Type // Store the actual type.

        // Generated from stored property `var value: T` and noting that
`T`'s upper bound is `AnyObject`.
        private var _value: AnyObject // Access the stored property
through a setter so that type can be checked
        var value: AnyObject {
            get {
                return _value
            }
            set {
                // In all functions check that args declared as `T` are
actually a `T` or a sub-type.
                // Note: `is` only works with type literal and there is
no `>=` operator for types :(.
                // `is` would need changing or `>=` for types adding,
nearest at moment `==`.
                precondition(T == /* >= */ newValue.dynamicType, "Type
of newValue, \(newValue.dynamicType), is not a sub-type of generic type
T, \(T)")
                _value = newValue
            }
        }

        // Generated from `init(_ initialValue: T)` and noting that
`T`'s upper bound is `AnyObject`.
        init(_ lowestCommonDeclaredT: AnyObject.Type, _ initialValue:
AnyObject) {
            T = lowestCommonDeclaredT
            _value = initialValue
        }
    }

    // Demonstrate that all `Box`es are the same size and therefore can
be bitwise copied
    // Compiler supplies lowest-common, declared, generic type for all
the `T`s in the `init` call.
    var bT = BoxAnyObject(Top.self, Top()) // In practice user would
write `let bT = Box(Top())`.
    bT.T // Top.Type
    sizeofValue(bT) // 16

    var bB = BoxAnyObject(Bottom.self, Bottom()) // In practice user
would write `let bB = Box(Bottom())`.
    bB.T // Bottom.Type
    sizeofValue(bB) // 16

    // Demonstration covariance.
    bT = bB // Compiler would check covariance of declared generic types.
    bT.T // Bottom.Type

    // Demonstrate generic returned type
    // Compiler would add cast to declared, generic type.
    bB.value as! Bottom // In practice user would write `bB.value`.

    // Demonstrate type safety
    bT = BoxAnyObject(Top.self, Top()) // In practice user would write
`bT = Box(Top())`.
    bT.value = Top() // OK
    // bT.value = Bottom() // Doesn't work at present because need `>=`
for types, but would work in practice
    // bB.value = Top() // Runtime error - wrong type

The implications of this proposal are:

1. The compiler can statically type check a read from a stored property.
2. A write to a stored property is type checked at runtime.
3. Protocols can be made generic instead of having an associated type and
then they become a proper type with dynamic dispatch.
4. Generic protocols can be a type just like non-generic protocols,
structs, and classes and unlike associated type protocols that can only be
a generic constraint.
5. The awkwardness of dealing with associated type generics is replaced
by a more powerful and easier to understand semantic of a type, just like
the other types.
6. There is a lot of ‘non-obvoius’, long code, for example `inits`, that
use a `where` clause to constrain an associated type protocol, this would
be unnecessary.
7. There are whole types, `AnySequence`, `AnyGenerator`, etc., that would
be replaced by a generic protocols, `Sequence`, `Generator`, etc.

Advantages:

1. Covariant generics are a powerful addition to the language.
2. Generics’ invariance are inconsistent with the rest of the language.
3. Generic protocols would become a ‘proper’ type and you could have
arrays and fields of a generic protocol.
4. There are many threads on swift-evolution looking at how protocols can
be made into a ‘proper’ type or at least a concept that is easier to
understand.

Compatibility:

1. This would be a major change since associated types in protocols would
be replaced by generics.
2. The new implementation of generics might break some existing `struct`
and `class` code, for example if it is dependent on the exact size of an
object because the class will have extra fields, one for each generic type,
and therefore will be larger.

Disadvantages:

1. Major change.
2. Object size increases.

Thanks in advance for any comments,

  â€” Howard.

PS This is part of a collection of proposals previously presented as
“Protocols on Steroids”.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
  -- Howard.

--
  -- Howard.

Hello Brent,

To be fair this is not the first nor the last time people wonder if static typing gives an overvalued sense of security while making prototyping and gluing different part of the app together more difficult (Shows | Microsoft Learn), if the language/compiler should protect you from yourself and if so just how much. Also, still in the interest of fairness, type safety is one of the many things you need when trying to deterministically ensure program safety (Midori post mortem): Joe Duffy - A Tale of Three Safeties

Still, the ship with Swift has clearly sailed towards static typing with Swift, I think we should work on the other two categories of safety. Waiting next year to deal with concurrency makes sense as we do not want to rush things, but it probably w much hotter topic than static typing in a lot of fields today and if you compare what C++ has available from developers (see Intel Threading Building Blocks for example) shows that there is plenty of exciting stuff ahead :).

¡¡¡

Sent from my iPhone

On 14 Jan 2016, at 08:39, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> wrote:

It doesn't make the type system unsound; the types are mostly checked at compile time but some are runtime checked, either way the types are checked and therefore the type system is sound.

Runtime type checking is not an adequate replacement for compile-time type checking. Runtime type checking ensures your code does not continue to run if it encounters a type error; compile-time type checking ensures your code is *incapable of encountering* a type error. This is a bit like saying that it's okay to drive drunk because seatbelts and airbags will mitigate the injuries caused if you crash.

Swift has a lovely property where, if your code is typed strongly enough and compiles without errors, it's probably pretty close to working. I would hate to see that watered down.

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

@Robert,

The behaviour you show for Java arrays is what I am proposing, but that
isn't type unsafe. The type error is detected at runtime and is flagged
with an ArrayStoreException. This is unlike C for example that would allow
the store, at least with a cast, and would put into the array the address
of the string. Therefore neither the proposal or Java is type unsafe, both
are type safe and both detect type errors at runtime.

The question is whether protecting against this is worthwhile, not whether
it can be protected against or not (it can be). Arrays are a good example,
Swift takes the approach (like most languages) of checking the array size
at runtime. But languages with dependent typing check the array size at
compile time, i.e. a dependently typed language would detect this error:

let array = [1]

array[0] // 1, OK

array[1] // Error not detected by compiler but detected at runtime

because in such a language array would be typed as int array of size one,
not int array of any size.

So the real question is in a language like Java, how many times do you get
ArrayStoreException? In my own code I have never seen an array store
exception! Why? Because contravariant writes are very rare. If you were to
protect against contravariant writes you would be complicating the language
for no practical gain, which I think is a bad trade off.

Hope this explains the reasoning,

-- Howard.

¡¡¡

On 14 January 2016 at 09:21, Developer <devteam.codafi@gmail.com> wrote:

It does indeed make the type system unsound in some cases. You have
chosen one particular variance direction because it is convenient. A
classic counterexample is Java making Arrays covariant by default. So this
works

Integer arr = [2, 4, 6, 8, 10];
Object orr = arr;
orr[0] = "crash bang";

And crashes at runtime. For that, I must give this part of the proposal a
strong -1. Any amount of type safety I have to give up in the name of
convenience is far too much.

I am, however, a fan of generic protocols. They seem like an orthogonal
concept given the first part here though.

~Robert Widmann

2016/01/13 17:03、Howard Lovatt via swift-evolution <
swift-evolution@swift.org> のメッセージ:

@Thorsten,

It doesn't make the type system unsound; the types are mostly checked at
compile time but some are runtime checked, either way the types are checked
and therefore the type system is sound. I have posted an example of array
runtime type checking in a previous response.

You can annotate variance, but this generally adds a lot of clutter to the
code (see some typical Java code that uses generics you will see stuff like
? extends Type everywhere). In other languages like Scala the annotation is
less visually intrusive, because they use shorter syntax and because it is
annotated at declaration site rather than use site, but it is still there.

I think Swift arrays are a good example of partially static and partially
runtime checked. The content is statically typed but the size is runtime
typed. Other languages do type both the content and the size (see Dependent
Typing on Wiki), however at some considerable burden on the programmer.

Hope this explains the thought process,

-- Howard.

On 13 January 2016 at 16:50, Thorsten Seitz <tseitz42@icloud.com> wrote:

Strong -1 from me as well for making the type system unsound.

> Am 13.01.2016 um 02:47 schrieb Howard Lovatt via swift-evolution < >> swift-evolution@swift.org>:
>
> Yes you can annotate for covariance, invariance, and contravariance,
both Java and Scala, allow all three. The problem is that the code becomes
splattered with variance annotations

Ceylon uses a different approach which is variance annotations at the
definition site.
This restricts the possible usage of the type parameters to appropriately
variant positions.

This migt be a better way to deal with variance.

-Thorsten

--
  -- Howard.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
  -- Howard.

Big +1 from me!

-Thorsten

¡¡¡

Am 14.01.2016 um 14:57 schrieb Matthew Johnson via swift-evolution <swift-evolution@swift.org>:

On Jan 14, 2016, at 2:39 AM, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> wrote:

It doesn't make the type system unsound; the types are mostly checked at compile time but some are runtime checked, either way the types are checked and therefore the type system is sound.

Runtime type checking is not an adequate replacement for compile-time type checking. Runtime type checking ensures your code does not continue to run if it encounters a type error; compile-time type checking ensures your code is *incapable of encountering* a type error. This is a bit like saying that it's okay to drive drunk because seatbelts and airbags will mitigate the injuries caused if you crash.

Swift has a lovely property where, if your code is typed strongly enough and compiles without errors, it's probably pretty close to working. I would hate to see that watered down.

+1. Swift's type system is very well thought out. It is going to get even better with the generics enhancements in Swift 3, removing many of the current limitations. We are heading down the right path. Changing course and introducing runtime checks where static checks are possible is a bad idea.

Hi Howard,

Making generics covariant by default would add even more of a burden to
users. They would need to check the type property of a generic object any
time they wanted to mutate that generic object or risk their program
terminating.

I don't think the fact that array accesses are checked at runtime is a good
example:

1. The Swift team has stated that subscripting into an array returns a
non-optional at least in part because of performance issues.
(Unfortunately, I think this was mentioned on the old Apple developer
forums, which are now inaccessible, so don't take my word for it until
someone with more insight says so one way or another :).
2. The fact that this specific aspect of Swift is checked at runtime
doesn't provide insight as to whether or not another aspect of Swift should
be compile-time or runtime-checked; it just indicates that there exists the
possibility of some checks being done at runtime (and every mainstream
statically typed language performs runtime checks to some extent, this
isn't a novel conclusion).
3. Checking that an array access is in bounds is trivial from a conceptual
standpoint. The index needs to be at least 0 and at most the length of the
array - 1.
4. The length of a Swift array is not part of the type contract, whereas
the type enclosed within a generic type is. Swift doesn't have fixed-length
arrays.

I think a proposal to get rid of optionals and non-nullable types would be
a better analogy. Here is a comparison to that hypothetical proposal:

1. The most popular objection to Swift's optional system seems to be either
clutter (from the ?, ! sigils) and ease of use. (Your proposal cites the
ugliness of variance annotations in other languages as a primary
motivation.)
2. Removing the optional system would result in moving a compile-time check
to a run-time check. (Your proposal states that the burden of ensuring the
access is valid lies at the use site, which introduces the possibility of
runtime failures that cannot be currently expressed.)
3. Removing optionals would keep the language type-safe, as an exception
would occur whenever calling a method on nil occurred at runtime (your
proposal also keeps the language type-safe, in the formal sense).
4. The possibility of a run-time check failing and terminating the program
with an NPE can be mitigated by user code performing an explicit check for
nil at the use site. (Your proposal would require user code to check
against the metatype property whenever a generic type is mutated to ensure
that no preconditions can fail.)
5. Removing the optional system would cause the type system to be more
imprecise, since a variable of type T would contain either an instance of T
or nil. (Your proposal would cause the type system to be more imprecise,
since a T<U> might actually be a T<V>, where V is a subtype of U, in a
context where V cannot substitute for U.)

I hope this explains my objections (and those expressed elsewhere within
this thread) more clearly.

Best,
Austin

¡¡¡

On Wed, Jan 13, 2016 at 1:54 PM, Howard Lovatt via swift-evolution < swift-evolution@swift.org> wrote:

@Simon,

In the Swift code below:

    let array = [1]

    array[0] // 1, OK

    array[1] // Error not detected by compiler but detected at runtime

you have an example of an error detected at runtime that terminates
program execution, it doesn't return an optional.

In other languages this would be a type error, these languages are usually
described as dependent type (https://en.wikipedia.org/wiki/Dependent_type\).
An example of the advantage of this would be vector (or matrix
multiplication), e.g. imagine that Swift had dependent types:

    let row = RowVec(1, 2) // Type: matrix of int with 1 row and 0
columns (note size is part of the type)

    let col = ColVec(3) // Type: matrix of int with 0 rows and 1 column

    let scaler = row * col // Compile time error because both vectors
should be the same length

At the moment if you wrote a matrix package in Swift the above example
would be a runtime error and not a compile time error, but with dependent
typing it would be a compile time error.

There is another discussion of Swift Evolution on calculable types that
are closely related to dependent typing.

To me you just strike the balance, sometimes static checking is best
sometimes runtime. You strike the balance by how practical it is to do the
static checking, if the burden that the static checking adds to the users
then it isn't worth it. This is the case with annotated variance in
languages like Java and Scala, the annotations do not add much. Hence I am
suggesting system that is simple to use, much like Swift arrays are easy to
use but not totally, but largely, statically typed.

Hope that explains my reasoning for making most type error compile time
checked but a small subset runtime checked,

-- Howard.

On 13 January 2016 at 14:01, Simon Pilkington <simonmpilkington@icloud.com > > wrote:

I’d be interested in reading up on what the Oracle response was if you
have links.

As you mentioned even Swift doesn’t get away from runtime type checking
but in the two examples you mentioned - array out of bounds and casts -
Swift makes use of the Optionals system to highlight that an operation may
fail and allows the user to handle that failure. Covariance should have
similar syntax support (for example use of optional chaining similar to
optional protocol requirements to indicate that a call may fail due to
incorrect types). For the compiler to understand when such failure is
possible, some kind of covariance syntax would be required.

As a related question, do you see covariance syntax as such a burden?

-Simon

On 13 Jan 2016, at 12:47 PM, Howard Lovatt <howard.lovatt@gmail.com> >> wrote:

Yes you can annotate for covariance, invariance, and contravariance, both
Java and Scala, allow all three. The problem is that the code becomes
splattered with variance annotations. The Java people themselves have
publicly regretted this and wished that they had made covariance the
default. If you look at generic code invariance and covariance are by far
the most common requirements; this proposal would address these common use
case without burdening the programmer.

Swift, and no usable language, is completely statically typed. Examples
in Swift of runtime type checking are array out of bounds and casts. There
are other examples of non-type related runtime checks is Swift: numerical
overflow, throws, using optionals to signal errors, and using enums to
signal errors. I say use what is appropriate, static type checking if it is
easy to do, otherwise runtime type checking. Note I am not proposing an
unsafe language like C, it is still type checked.

There is a strong positive precedence for a type check on write, Java
arrays (not Java `List`s). Arrays in Java may be passed covariantly, and
this is extensively used. However if you attempt to write the wrong type
into the array you will get an `ArrayStoreException`. In practice you don't
get many `ArrayStoreException`, non of my code has ever had one. Its just
not something you do in practice, as noted before contravariance is rare.

Thanks for you comments and I hope this eases your concerns,

  -- Howard.

On 13 January 2016 at 11:33, Simon Pilkington < >> simonmpilkington@icloud.com> wrote:

The problem is that conceptually and behaviourally Box<Bottom> *is
indeed not* a Box<Top> and cannot be treated the same as one. The proposal
attempts to get around this difference with a runtime failure but this
would result in very fragile code - you get passed a Box<Top> and want to
pass it a subclass of Top, will it succeed, who knows. You probably would
be able to check the types but the complier wouldn’t highlight that this is
an operation that could potentially fail.

This seems to be very much against Swift’s goal of safety being enforced
by the compiler as much as possible.

Java uses the wildcard syntax to highlight this conceptual and
behavioural difference - Box<Bottom> is not covariant with Box<Top> but
rather with Box<? extends Top>. The compiler can then enforce that a
programmer doesn’t try to pass an incompatible type to a variable of such
type. Even though this is a complication to the language (many Java
programmers struggle with correctly using the wildcard syntax) I don’t see
covariance for generics being added to Swift in a robust manner without
some kind of similar syntax.

-Simon

On 12 Jan 2016, at 8:45 PM, Howard Lovatt via swift-evolution < >>> swift-evolution@swift.org> wrote:

Currently you generics are invariant whereas function arguments etc. are
covariant. I am suggesting that if the way generics are implemented is
changed then they can be made covariant and that this will add considerable
utility to Swift generics.

1st a demonstration of the current situation of invariant generics:

    // Current system
    class Top {}
    class Bottom: Top {}

    struct Box<T: AnyObject> {
        var value: T
        init(_ initialValue: T) {
            value = initialValue;
        }
    }

    let boxB = Box(Bottom())
    // let boxT: Box<Top> = boxB // Covariance currently not allowed

The key point is although `Bottom` 'is a’ `Top`, `Box<Bottom>` *is not*
a `Box<Top>`.

I am suggesting:

1. That `Box<Bottom>` should be a `Box<Top>` (covariance).
2. An implementation that allows the above covariance.
3. That protocols are made generic, i.e. `protocol Box<T> { var value: T
{ get set } }` and that this mechanism replaces associated types for
protocols.

    // Proposal:
    // 1. No change to Box, i.e. programmer would just write Box as
before
    // 2. Code transformed by comiler with write check for each
specific, generic type instance
    // Best approximation of resulting code in current Swift to
demonstrate spirit of idea:

    // Compiler writes a universal form using the upper bound (it
writes the underlyting representation).
    // In practice this would be called `Box` but used `BoxAnyObject`
to indicate that it has a generic argument bounded by `AnyObject`.
    struct BoxAnyObject {
        // Generated from generic argument `<T: AnyObject>`.
        let T: AnyObject.Type // Store the actual type.

        // Generated from stored property `var value: T` and noting
that `T`'s upper bound is `AnyObject`.
        private var _value: AnyObject // Access the stored property
through a setter so that type can be checked
        var value: AnyObject {
            get {
                return _value
            }
            set {
                // In all functions check that args declared as `T` are
actually a `T` or a sub-type.
                // Note: `is` only works with type literal and there is
no `>=` operator for types :(.
                // `is` would need changing or `>=` for types adding,
nearest at moment `==`.
                precondition(T == /* >= */ newValue.dynamicType, "Type
of newValue, \(newValue.dynamicType), is not a sub-type of generic type
T, \(T)")
                _value = newValue
            }
        }

        // Generated from `init(_ initialValue: T)` and noting that
`T`'s upper bound is `AnyObject`.
        init(_ lowestCommonDeclaredT: AnyObject.Type, _ initialValue:
AnyObject) {
            T = lowestCommonDeclaredT
            _value = initialValue
        }
    }

    // Demonstrate that all `Box`es are the same size and therefore can
be bitwise copied
    // Compiler supplies lowest-common, declared, generic type for all
the `T`s in the `init` call.
    var bT = BoxAnyObject(Top.self, Top()) // In practice user would
write `let bT = Box(Top())`.
    bT.T // Top.Type
    sizeofValue(bT) // 16

    var bB = BoxAnyObject(Bottom.self, Bottom()) // In practice user
would write `let bB = Box(Bottom())`.
    bB.T // Bottom.Type
    sizeofValue(bB) // 16

    // Demonstration covariance.
    bT = bB // Compiler would check covariance of declared generic
types.
    bT.T // Bottom.Type

    // Demonstrate generic returned type
    // Compiler would add cast to declared, generic type.
    bB.value as! Bottom // In practice user would write `bB.value`.

    // Demonstrate type safety
    bT = BoxAnyObject(Top.self, Top()) // In practice user would write
`bT = Box(Top())`.
    bT.value = Top() // OK
    // bT.value = Bottom() // Doesn't work at present because need `>=`
for types, but would work in practice
    // bB.value = Top() // Runtime error - wrong type

The implications of this proposal are:

1. The compiler can statically type check a read from a stored property.
2. A write to a stored property is type checked at runtime.
3. Protocols can be made generic instead of having an associated type
and then they become a proper type with dynamic dispatch.
4. Generic protocols can be a type just like non-generic protocols,
structs, and classes and unlike associated type protocols that can only be
a generic constraint.
5. The awkwardness of dealing with associated type generics is replaced
by a more powerful and easier to understand semantic of a type, just like
the other types.
6. There is a lot of ‘non-obvoius’, long code, for example `inits`, that
use a `where` clause to constrain an associated type protocol, this would
be unnecessary.
7. There are whole types, `AnySequence`, `AnyGenerator`, etc., that
would be replaced by a generic protocols, `Sequence`, `Generator`, etc.

Advantages:

1. Covariant generics are a powerful addition to the language.
2. Generics’ invariance are inconsistent with the rest of the language.
3. Generic protocols would become a ‘proper’ type and you could have
arrays and fields of a generic protocol.
4. There are many threads on swift-evolution looking at how protocols
can be made into a ‘proper’ type or at least a concept that is easier to
understand.

Compatibility:

1. This would be a major change since associated types in protocols
would be replaced by generics.
2. The new implementation of generics might break some existing `struct`
and `class` code, for example if it is dependent on the exact size of an
object because the class will have extra fields, one for each generic type,
and therefore will be larger.

Disadvantages:

1. Major change.
2. Object size increases.

Thanks in advance for any comments,

  â€” Howard.

PS This is part of a collection of proposals previously presented as
“Protocols on Steroids”.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
  -- Howard.

--
  -- Howard.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

One might even argue the array length check is a precondition rather than a dynamic behavior. A developer is forced not to rely upon the bounds checking behavior, because an out-of-bounds error crashes the application. “Code crashes” is not a useful dynamic behavior and probably should not be used as an example for other behavior except in limited “developer messed up” scenarios.

-DW

¡¡¡

On Jan 13, 2016, at 4:02 PM, Austin Zheng via swift-evolution <swift-evolution@swift.org> wrote:

Hi Howard,

Making generics covariant by default would add even more of a burden to users. They would need to check the type property of a generic object any time they wanted to mutate that generic object or risk their program terminating.

I don't think the fact that array accesses are checked at runtime is a good example:

1. The Swift team has stated that subscripting into an array returns a non-optional at least in part because of performance issues. (Unfortunately, I think this was mentioned on the old Apple developer forums, which are now inaccessible, so don't take my word for it until someone with more insight says so one way or another :).
2. The fact that this specific aspect of Swift is checked at runtime doesn't provide insight as to whether or not another aspect of Swift should be compile-time or runtime-checked; it just indicates that there exists the possibility of some checks being done at runtime (and every mainstream statically typed language performs runtime checks to some extent, this isn't a novel conclusion).
3. Checking that an array access is in bounds is trivial from a conceptual standpoint. The index needs to be at least 0 and at most the length of the array - 1.
4. The length of a Swift array is not part of the type contract, whereas the type enclosed within a generic type is. Swift doesn't have fixed-length arrays.

To clarify, the reason that array indexes don’t take and produce an optional is a bit more subtle than that.

1) subscripts have to be uniform for their getting and setter. It would be weird & very confusing for "A[i] = nil” to work, given that you can’t remove an element from an array.

2) Integer array indexes are almost always derived from some other value that is in range. Having to deal with the optional produced by a getter would almost always be noise. That noise would force a lot of pointless optional manipulation, and would end up as a lot of !’s. This would water down the optional model by making “!” a common thing, and be seen as less scary.

-Chris

¡¡¡

On Jan 13, 2016, at 3:02 PM, Austin Zheng via swift-evolution <swift-evolution@swift.org> wrote:

Hi Howard,

Making generics covariant by default would add even more of a burden to users. They would need to check the type property of a generic object any time they wanted to mutate that generic object or risk their program terminating.

I don't think the fact that array accesses are checked at runtime is a good example:

1. The Swift team has stated that subscripting into an array returns a non-optional at least in part because of performance issues. (Unfortunately, I think this was mentioned on the old Apple developer forums, which are now inaccessible, so don't take my word for it until someone with more insight says so one way or another :).

I think this is missing the point of type safety. Whether checking occurs at runtime or compile time is irrelevant. What I want is a programming language whose well-typed programs “don’t go wrong” (in the Robin Milner sense), not one that allows me to discover its own definition for “wrong”. What you call type safety is really just deferring errors until runtime, which defeats the purpose of even having one in the first place in my opinion. Java made a horrendous mistake in this case, regardless of how common is may be in actual programs, and I don’t think Swift needs to make the same.

I’ll put it simply: I would rather have a language that allows me to write a smaller class of safer programs than one that allows a larger class of flat-out wrong ones.

¡¡¡

On Jan 13, 2016, at 8:05 PM, Howard Lovatt <howard.lovatt@gmail.com> wrote:

@Robert,

The behaviour you show for Java arrays is what I am proposing, but that isn't type unsafe. The type error is detected at runtime and is flagged with an ArrayStoreException. This is unlike C for example that would allow the store, at least with a cast, and would put into the array the address of the string. Therefore neither the proposal or Java is type unsafe, both are type safe and both detect type errors at runtime.

The question is whether protecting against this is worthwhile, not whether it can be protected against or not (it can be). Arrays are a good example, Swift takes the approach (like most languages) of checking the array size at runtime. But languages with dependent typing check the array size at compile time, i.e. a dependently typed language would detect this error:

let array = [1]
array[0] // 1, OK
array[1] // Error not detected by compiler but detected at runtime

because in such a language array would be typed as int array of size one, not int array of any size.

So the real question is in a language like Java, how many times do you get ArrayStoreException? In my own code I have never seen an array store exception! Why? Because contravariant writes are very rare. If you were to protect against contravariant writes you would be complicating the language for no practical gain, which I think is a bad trade off.

Hope this explains the reasoning,

-- Howard.

On 14 January 2016 at 09:21, Developer <devteam.codafi@gmail.com <mailto:devteam.codafi@gmail.com>> wrote:
It does indeed make the type system unsound in some cases. You have chosen one particular variance direction because it is convenient. A classic counterexample is Java making Arrays covariant by default. So this works

Integer arr = [2, 4, 6, 8, 10];
Object orr = arr;
orr[0] = "crash bang";

And crashes at runtime. For that, I must give this part of the proposal a strong -1. Any amount of type safety I have to give up in the name of convenience is far too much.

I am, however, a fan of generic protocols. They seem like an orthogonal concept given the first part here though.

~Robert Widmann

2016/01/13 17:03、Howard Lovatt via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> のメッセージ:

@Thorsten,

It doesn't make the type system unsound; the types are mostly checked at compile time but some are runtime checked, either way the types are checked and therefore the type system is sound. I have posted an example of array runtime type checking in a previous response.

You can annotate variance, but this generally adds a lot of clutter to the code (see some typical Java code that uses generics you will see stuff like ? extends Type everywhere). In other languages like Scala the annotation is less visually intrusive, because they use shorter syntax and because it is annotated at declaration site rather than use site, but it is still there.

I think Swift arrays are a good example of partially static and partially runtime checked. The content is statically typed but the size is runtime typed. Other languages do type both the content and the size (see Dependent Typing on Wiki), however at some considerable burden on the programmer.

Hope this explains the thought process,

-- Howard.

On 13 January 2016 at 16:50, Thorsten Seitz <tseitz42@icloud.com <mailto:tseitz42@icloud.com>> wrote:
Strong -1 from me as well for making the type system unsound.

> Am 13.01.2016 um 02:47 schrieb Howard Lovatt via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>>:
>
> Yes you can annotate for covariance, invariance, and contravariance, both Java and Scala, allow all three. The problem is that the code becomes splattered with variance annotations

Ceylon uses a different approach which is variance annotations at the definition site.
This restricts the possible usage of the type parameters to appropriately variant positions.

This migt be a better way to deal with variance.

-Thorsten

--
  -- Howard.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

--
  -- Howard.

@Robert,

The behaviour you show for Java arrays is what I am proposing, but that isn't type unsafe. The type error is detected at runtime and is flagged with an ArrayStoreException. This is unlike C for example that would allow the store, at least with a cast, and would put into the array the address of the string. Therefore neither the proposal or Java is type unsafe, both are type safe and both detect type errors at runtime.

In case you hadn't noticed, Swift provides a basic form of covariance for the collections in the standard library... And it does so without the need for runtime checking, because value semantics allow safe mutation and collection covariance.

  - Doug

¡¡¡

Sent from my iPhone

On Jan 13, 2016, at 5:05 PM, Howard Lovatt via swift-evolution <swift-evolution@swift.org> wrote:

The question is whether protecting against this is worthwhile, not whether it can be protected against or not (it can be). Arrays are a good example, Swift takes the approach (like most languages) of checking the array size at runtime. But languages with dependent typing check the array size at compile time, i.e. a dependently typed language would detect this error:

let array = [1]
array[0] // 1, OK
array[1] // Error not detected by compiler but detected at runtime

because in such a language array would be typed as int array of size one, not int array of any size.

So the real question is in a language like Java, how many times do you get ArrayStoreException? In my own code I have never seen an array store exception! Why? Because contravariant writes are very rare. If you were to protect against contravariant writes you would be complicating the language for no practical gain, which I think is a bad trade off.

Hope this explains the reasoning,

-- Howard.

On 14 January 2016 at 09:21, Developer <devteam.codafi@gmail.com> wrote:
It does indeed make the type system unsound in some cases. You have chosen one particular variance direction because it is convenient. A classic counterexample is Java making Arrays covariant by default. So this works

Integer arr = [2, 4, 6, 8, 10];
Object orr = arr;
orr[0] = "crash bang";

And crashes at runtime. For that, I must give this part of the proposal a strong -1. Any amount of type safety I have to give up in the name of convenience is far too much.

I am, however, a fan of generic protocols. They seem like an orthogonal concept given the first part here though.

~Robert Widmann

2016/01/13 17:03、Howard Lovatt via swift-evolution <swift-evolution@swift.org> のメッセージ:

@Thorsten,

It doesn't make the type system unsound; the types are mostly checked at compile time but some are runtime checked, either way the types are checked and therefore the type system is sound. I have posted an example of array runtime type checking in a previous response.

You can annotate variance, but this generally adds a lot of clutter to the code (see some typical Java code that uses generics you will see stuff like ? extends Type everywhere). In other languages like Scala the annotation is less visually intrusive, because they use shorter syntax and because it is annotated at declaration site rather than use site, but it is still there.

I think Swift arrays are a good example of partially static and partially runtime checked. The content is statically typed but the size is runtime typed. Other languages do type both the content and the size (see Dependent Typing on Wiki), however at some considerable burden on the programmer.

Hope this explains the thought process,

-- Howard.

On 13 January 2016 at 16:50, Thorsten Seitz <tseitz42@icloud.com> wrote:
Strong -1 from me as well for making the type system unsound.

> Am 13.01.2016 um 02:47 schrieb Howard Lovatt via swift-evolution <swift-evolution@swift.org>:
>
> Yes you can annotate for covariance, invariance, and contravariance, both Java and Scala, allow all three. The problem is that the code becomes splattered with variance annotations

Ceylon uses a different approach which is variance annotations at the definition site.
This restricts the possible usage of the type parameters to appropriately variant positions.

This migt be a better way to deal with variance.

-Thorsten

--
  -- Howard.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
  -- Howard.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

@David,

You said "developer messed up" for runtime crashes. The developer messed up
when the compiler finds a problem also. Its the same thing; if the compiler
statically finds a problem or the runtime finds a problem, in both cases
the "developer messed up". The only difference is that one is found at
compile time and the other is found at runtime.

My own guide in order of importance is:

1. Ideally it should be type safe, i.e. the error is found either by the
runtime of the compiler. Obviously some languages, C, Obj-C, C++, are not
type safe, and you can write great programs in all 3 and therefore you
can't say that type safe is mandatory. By extension calls in Swift to C
etc. are not type safe and there are many such calls in any realistic Swift
program, is this a problem? Not really.
2. Ideally the compiler will catch errors. Ideally, because this saves on
testing. However if this burdens the programmer greatly you have to
consider is it worth while. Many people prefer dynamically typed languages
for exactly this reason, they find declaring types more trouble than it is
worth. IE they feel the extra time spent annotating with type information
is not paid back in terms of reduced testing. This 'explicitly typing is a
pain' meme is common these days and has resulted in languages like Swift
that infer the type.

Combining these 2 points leads me to conclude that you should stick with
type safety were you can, infer type were you can, and only annotate extra
type information if that annotation increases programmer productivity.

Hence the proposal: its type safe, it infers type when it can, and for the
stuff that isn't worth annotating for it runtime checks.

Does that explain my reasoning?

-- Howard.

¡¡¡

On 14 January 2016 at 10:13, David Waite <david@alkaline-solutions.com> wrote:

On Jan 13, 2016, at 4:02 PM, Austin Zheng via swift-evolution < > swift-evolution@swift.org> wrote:

Hi Howard,

Making generics covariant by default would add even more of a burden to
users. They would need to check the type property of a generic object any
time they wanted to mutate that generic object or risk their program
terminating.

I don't think the fact that array accesses are checked at runtime is a
good example:

1. The Swift team has stated that subscripting into an array returns a
non-optional at least in part because of performance issues.
(Unfortunately, I think this was mentioned on the old Apple developer
forums, which are now inaccessible, so don't take my word for it until
someone with more insight says so one way or another :).
2. The fact that this specific aspect of Swift is checked at runtime
doesn't provide insight as to whether or not another aspect of Swift should
be compile-time or runtime-checked; it just indicates that there exists the
possibility of some checks being done at runtime (and every mainstream
statically typed language performs runtime checks to some extent, this
isn't a novel conclusion).
3. Checking that an array access is in bounds is trivial from a conceptual
standpoint. The index needs to be at least 0 and at most the length of the
array - 1.
4. The length of a Swift array is not part of the type contract, whereas
the type enclosed within a generic type is. Swift doesn't have fixed-length
arrays.

One might even argue the array length check is a precondition rather than
a dynamic behavior. A developer is forced not to rely upon the bounds
checking behavior, because an out-of-bounds error crashes the application.
“Code crashes” is not a useful dynamic behavior and probably should not be
used as an example for other behavior except in limited “developer messed
up” scenarios.

-DW

--
  -- Howard.

@ Austin,

Comments inline below.

Hi Howard,

Making generics covariant by default would add even more of a burden to
users. They would need to check the type property of a generic object any
time they wanted to mutate that generic object or risk their program
terminating.

Its a matter of balance, is that checking necessary. You certainly don't
see it with Java arrays and this isn't a problem in practice. If you burden
everyone with an annotation and in real programs that imperceptibly reduces
the number of error then it was a bad call. The Java use case indicates
that it is a bad call.

I don't think the fact that array accesses are checked at runtime is a
good example:

1. The Swift team has stated that subscripting into an array returns a
non-optional at least in part because of performance issues.
(Unfortunately, I think this was mentioned on the old Apple developer
forums, which are now inaccessible, so don't take my word for it until
someone with more insight says so one way or another :).

What about write to an array, you still have to check bounds. An optional
won't help.

2. The fact that this specific aspect of Swift is checked at runtime
doesn't provide insight as to whether or not another aspect of Swift should
be compile-time or runtime-checked; it just indicates that there exists the
possibility of some checks being done at runtime (and every mainstream
statically typed language performs runtime checks to some extent, this
isn't a novel conclusion).

Yes it says that they are making pragmatic choices, the designers will be
well aware of type systems that check array bounds but chose, in my opinion
correctly, to go down the path of a runtime check for reasons of
practicality. The dependant type system or similar does not pay for itself
in terms of improved programmer productivity and application reliability.

3. Checking that an array access is in bounds is trivial from a conceptual
standpoint. The index needs to be at least 0 and at most the length of the
array - 1.

Type checking is equally trivial and can often be optimised away.

4. The length of a Swift array is not part of the type contract, whereas
the type enclosed within a generic type is. Swift doesn't have fixed-length
arrays.

Well it isn't in Swift, but in some languages it is. Proponents of these
languages would point to a matrix math API and note how the compiler can
check matrix size for conformance at compile time. Whereas in Swift that is
a runtime check. The proponents would point to the inefficiency of runtime
checking and the fact that you now need to test the code. I think the Swift
team have taken the right approach and are saying typing the size of an
array is not worth it.

I think a proposal to get rid of optionals and non-nullable types would be
a better analogy. Here is a comparison to that hypothetical proposal:

1. The most popular objection to Swift's optional system seems to be
either clutter (from the ?, ! sigils) and ease of use. (Your proposal cites
the ugliness of variance annotations in other languages as a primary
motivation.)

Not really, you are not comparing like with like. If Swift didn't have
explicit optionals it would most likely have implicit optionals like Obj-C
and Java. IE everything is an optional. I like Swift's solution since most
things aren't actually optional and therefore overall it reduces the burden
on the programmer (in other systems you are continuously testing for nil).
In terms of variance for generics if there were a method of giving static
type checking without burdening the programmer with annotations then I
would be in favour (unfortunately I do not know of such a system).

2. Removing the optional system would result in moving a compile-time
check to a run-time check. (Your proposal states that the burden of
ensuring the access is valid lies at the use site, which introduces the
possibility of runtime failures that cannot be currently expressed.)

No it would most likely mean that there were nil checks everywhere, just
like Java and to some extent Obj-C. IE an overall increase in programmer
burden.

3. Removing optionals would keep the language type-safe, as an exception
would occur whenever calling a method on nil occurred at runtime (your
proposal also keeps the language type-safe, in the formal sense).

Yes. Whether you have explicit optionals or implicit optionals it is still
type safe.

4. The possibility of a run-time check failing and terminating the program
with an NPE can be mitigated by user code performing an explicit check for
nil at the use site. (Your proposal would require user code to check
against the metatype property whenever a generic type is mutated to ensure
that no preconditions can fail.)

Yes, you see a lot of that for implicit optionals. But tellingly not for
Java arrays. Therefore the decision as to whether a type check should be
static or dynamic should be based on the use case. You cannot simply say
that everything should be static, it places enormous burden on the
programmer. People do say that everything should be dynamic though! Myself,
I think static is good if it is easy to do.

In practice you don't see runtime type checking in Java array code,
therefore I don't think there is any evidence to suggest you would in
Swift. Why would the two be different? Therefore I think runtime type
errors due to incorrect writes to Java generics would not be a problem in
practice.

5. Removing the optional system would cause the type system to be more
imprecise, since a variable of type T would contain either an instance of T
or nil. (Your proposal would cause the type system to be more imprecise,
since a T<U> might actually be a T<V>, where V is a subtype of U, in a
context where V cannot substitute for U.)

Sure, but for optionals that is a trade off well worth taking. The
alternative is nil checks everywhere.

The trade off made at the moment of using associated types is a great
burden on the programmer. Take a look at all the complexity in the
collections library at present. To give an idea of how complicated this is
take a look at the playground you can download from
https://github.com/rnapier/MyAnySequence\. Its mightily complicated compared
to my proposal! Therefore I don't think the current trade off is optimum.
Its always a trade off, the trick is to make the best.

Is that giving you an idea of where I am coming from?

  -- Howard.

¡¡¡

On 14 January 2016 at 10:02, Austin Zheng <austinzheng@gmail.com> wrote:

I hope this explains my objections (and those expressed elsewhere within
this thread) more clearly.

Best,
Austin

On Wed, Jan 13, 2016 at 1:54 PM, Howard Lovatt via swift-evolution < > swift-evolution@swift.org> wrote:

@Simon,

In the Swift code below:

    let array = [1]

    array[0] // 1, OK

    array[1] // Error not detected by compiler but detected at runtime

you have an example of an error detected at runtime that terminates
program execution, it doesn't return an optional.

In other languages this would be a type error, these languages are
usually described as dependent type (
https://en.wikipedia.org/wiki/Dependent_type\). An example of the
advantage of this would be vector (or matrix multiplication), e.g. imagine
that Swift had dependent types:

    let row = RowVec(1, 2) // Type: matrix of int with 1 row and 0
columns (note size is part of the type)

    let col = ColVec(3) // Type: matrix of int with 0 rows and 1 column

    let scaler = row * col // Compile time error because both vectors
should be the same length

At the moment if you wrote a matrix package in Swift the above example
would be a runtime error and not a compile time error, but with dependent
typing it would be a compile time error.

There is another discussion of Swift Evolution on calculable types that
are closely related to dependent typing.

To me you just strike the balance, sometimes static checking is best
sometimes runtime. You strike the balance by how practical it is to do the
static checking, if the burden that the static checking adds to the users
then it isn't worth it. This is the case with annotated variance in
languages like Java and Scala, the annotations do not add much. Hence I am
suggesting system that is simple to use, much like Swift arrays are easy to
use but not totally, but largely, statically typed.

Hope that explains my reasoning for making most type error compile time
checked but a small subset runtime checked,

-- Howard.

On 13 January 2016 at 14:01, Simon Pilkington < >> simonmpilkington@icloud.com> wrote:

I’d be interested in reading up on what the Oracle response was if you
have links.

As you mentioned even Swift doesn’t get away from runtime type checking
but in the two examples you mentioned - array out of bounds and casts -
Swift makes use of the Optionals system to highlight that an operation may
fail and allows the user to handle that failure. Covariance should have
similar syntax support (for example use of optional chaining similar to
optional protocol requirements to indicate that a call may fail due to
incorrect types). For the compiler to understand when such failure is
possible, some kind of covariance syntax would be required.

As a related question, do you see covariance syntax as such a burden?

-Simon

On 13 Jan 2016, at 12:47 PM, Howard Lovatt <howard.lovatt@gmail.com> >>> wrote:

Yes you can annotate for covariance, invariance, and contravariance,
both Java and Scala, allow all three. The problem is that the code becomes
splattered with variance annotations. The Java people themselves have
publicly regretted this and wished that they had made covariance the
default. If you look at generic code invariance and covariance are by far
the most common requirements; this proposal would address these common use
case without burdening the programmer.

Swift, and no usable language, is completely statically typed. Examples
in Swift of runtime type checking are array out of bounds and casts. There
are other examples of non-type related runtime checks is Swift: numerical
overflow, throws, using optionals to signal errors, and using enums to
signal errors. I say use what is appropriate, static type checking if it is
easy to do, otherwise runtime type checking. Note I am not proposing an
unsafe language like C, it is still type checked.

There is a strong positive precedence for a type check on write, Java
arrays (not Java `List`s). Arrays in Java may be passed covariantly, and
this is extensively used. However if you attempt to write the wrong type
into the array you will get an `ArrayStoreException`. In practice you don't
get many `ArrayStoreException`, non of my code has ever had one. Its just
not something you do in practice, as noted before contravariance is rare.

Thanks for you comments and I hope this eases your concerns,

  -- Howard.

On 13 January 2016 at 11:33, Simon Pilkington < >>> simonmpilkington@icloud.com> wrote:

The problem is that conceptually and behaviourally Box<Bottom> *is
indeed not* a Box<Top> and cannot be treated the same as one. The proposal
attempts to get around this difference with a runtime failure but this
would result in very fragile code - you get passed a Box<Top> and want to
pass it a subclass of Top, will it succeed, who knows. You probably would
be able to check the types but the complier wouldn’t highlight that this is
an operation that could potentially fail.

This seems to be very much against Swift’s goal of safety being
enforced by the compiler as much as possible.

Java uses the wildcard syntax to highlight this conceptual and
behavioural difference - Box<Bottom> is not covariant with Box<Top> but
rather with Box<? extends Top>. The compiler can then enforce that a
programmer doesn’t try to pass an incompatible type to a variable of such
type. Even though this is a complication to the language (many Java
programmers struggle with correctly using the wildcard syntax) I don’t see
covariance for generics being added to Swift in a robust manner without
some kind of similar syntax.

-Simon

On 12 Jan 2016, at 8:45 PM, Howard Lovatt via swift-evolution < >>>> swift-evolution@swift.org> wrote:

Currently you generics are invariant whereas function arguments etc.
are covariant. I am suggesting that if the way generics are implemented is
changed then they can be made covariant and that this will add considerable
utility to Swift generics.

1st a demonstration of the current situation of invariant generics:

    // Current system
    class Top {}
    class Bottom: Top {}

    struct Box<T: AnyObject> {
        var value: T
        init(_ initialValue: T) {
            value = initialValue;
        }
    }

    let boxB = Box(Bottom())
    // let boxT: Box<Top> = boxB // Covariance currently not allowed

The key point is although `Bottom` 'is a’ `Top`, `Box<Bottom>` *is not*
a `Box<Top>`.

I am suggesting:

1. That `Box<Bottom>` should be a `Box<Top>` (covariance).
2. An implementation that allows the above covariance.
3. That protocols are made generic, i.e. `protocol Box<T> { var value:
T { get set } }` and that this mechanism replaces associated types for
protocols.

    // Proposal:
    // 1. No change to Box, i.e. programmer would just write Box as
before
    // 2. Code transformed by comiler with write check for each
specific, generic type instance
    // Best approximation of resulting code in current Swift to
demonstrate spirit of idea:

    // Compiler writes a universal form using the upper bound (it
writes the underlyting representation).
    // In practice this would be called `Box` but used `BoxAnyObject`
to indicate that it has a generic argument bounded by `AnyObject`.
    struct BoxAnyObject {
        // Generated from generic argument `<T: AnyObject>`.
        let T: AnyObject.Type // Store the actual type.

        // Generated from stored property `var value: T` and noting
that `T`'s upper bound is `AnyObject`.
        private var _value: AnyObject // Access the stored property
through a setter so that type can be checked
        var value: AnyObject {
            get {
                return _value
            }
            set {
                // In all functions check that args declared as `T`
are actually a `T` or a sub-type.
                // Note: `is` only works with type literal and there
is no `>=` operator for types :(.
                // `is` would need changing or `>=` for types adding,
nearest at moment `==`.
                precondition(T == /* >= */ newValue.dynamicType, "Type
of newValue, \(newValue.dynamicType), is not a sub-type of generic
type T, \(T)")
                _value = newValue
            }
        }

        // Generated from `init(_ initialValue: T)` and noting that
`T`'s upper bound is `AnyObject`.
        init(_ lowestCommonDeclaredT: AnyObject.Type, _ initialValue:
AnyObject) {
            T = lowestCommonDeclaredT
            _value = initialValue
        }
    }

    // Demonstrate that all `Box`es are the same size and therefore
can be bitwise copied
    // Compiler supplies lowest-common, declared, generic type for all
the `T`s in the `init` call.
    var bT = BoxAnyObject(Top.self, Top()) // In practice user would
write `let bT = Box(Top())`.
    bT.T // Top.Type
    sizeofValue(bT) // 16

    var bB = BoxAnyObject(Bottom.self, Bottom()) // In practice user
would write `let bB = Box(Bottom())`.
    bB.T // Bottom.Type
    sizeofValue(bB) // 16

    // Demonstration covariance.
    bT = bB // Compiler would check covariance of declared generic
types.
    bT.T // Bottom.Type

    // Demonstrate generic returned type
    // Compiler would add cast to declared, generic type.
    bB.value as! Bottom // In practice user would write `bB.value`.

    // Demonstrate type safety
    bT = BoxAnyObject(Top.self, Top()) // In practice user would write
`bT = Box(Top())`.
    bT.value = Top() // OK
    // bT.value = Bottom() // Doesn't work at present because need
`>=` for types, but would work in practice
    // bB.value = Top() // Runtime error - wrong type

The implications of this proposal are:

1. The compiler can statically type check a read from a stored property.
2. A write to a stored property is type checked at runtime.
3. Protocols can be made generic instead of having an associated type
and then they become a proper type with dynamic dispatch.
4. Generic protocols can be a type just like non-generic protocols,
structs, and classes and unlike associated type protocols that can only be
a generic constraint.
5. The awkwardness of dealing with associated type generics is replaced
by a more powerful and easier to understand semantic of a type, just like
the other types.
6. There is a lot of ‘non-obvoius’, long code, for example `inits`,
that use a `where` clause to constrain an associated type protocol, this
would be unnecessary.
7. There are whole types, `AnySequence`, `AnyGenerator`, etc., that
would be replaced by a generic protocols, `Sequence`, `Generator`, etc.

Advantages:

1. Covariant generics are a powerful addition to the language.
2. Generics’ invariance are inconsistent with the rest of the language.
3. Generic protocols would become a ‘proper’ type and you could have
arrays and fields of a generic protocol.
4. There are many threads on swift-evolution looking at how protocols
can be made into a ‘proper’ type or at least a concept that is easier to
understand.

Compatibility:

1. This would be a major change since associated types in protocols
would be replaced by generics.
2. The new implementation of generics might break some existing
`struct` and `class` code, for example if it is dependent on the exact size
of an object because the class will have extra fields, one for each generic
type, and therefore will be larger.

Disadvantages:

1. Major change.
2. Object size increases.

Thanks in advance for any comments,

  â€” Howard.

PS This is part of a collection of proposals previously presented as
“Protocols on Steroids”.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
  -- Howard.

--
  -- Howard.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
  -- Howard.

@Developer,

I doubt you really mean:

"I’ll put it simply: I would rather have a language that allows me to write
a smaller class of safer programs than one that allows a larger class of
flat-out wrong ones."

I doubt what you say because if that were the case in Swift you would
manually check for overflow and use the &+ operator instead of the +
operator, because the + operator overflows with a runtime exception.

All practical programming languages and all programmers even when given the
choice, e.g. + or &+ in Swift, will make a pragmatic choice:

1. Is that compile time check worth it?
2. Is that runtime check worth it?
3. Or should I just cover this with a test case?

Its all a matter of balance,

-- Howard.

¡¡¡

On 14 January 2016 at 12:09, Developer <devteam.codafi@gmail.com> wrote:

I think this is missing the point of type safety. Whether checking occurs
at runtime or compile time is irrelevant. What I want is a programming
language whose well-typed programs “don’t go wrong” (in the Robin Milner
sense), not one that allows me to discover its own definition for “wrong”.
What you call type safety is really just deferring errors until runtime,
which defeats the purpose of even having one in the first place in my
opinion. Java made a horrendous mistake in this case, regardless of how
common is may be in actual programs, and I don’t think Swift needs to make
the same.

I’ll put it simply: I would rather have a language that allows me to write
a smaller class of safer programs than one that allows a larger class of
flat-out wrong ones.

On Jan 13, 2016, at 8:05 PM, Howard Lovatt <howard.lovatt@gmail.com> > wrote:

@Robert,

The behaviour you show for Java arrays is what I am proposing, but that
isn't type unsafe. The type error is detected at runtime and is flagged
with an ArrayStoreException. This is unlike C for example that would allow
the store, at least with a cast, and would put into the array the address
of the string. Therefore neither the proposal or Java is type unsafe, both
are type safe and both detect type errors at runtime.

The question is whether protecting against this is worthwhile, not whether
it can be protected against or not (it can be). Arrays are a good example,
Swift takes the approach (like most languages) of checking the array size
at runtime. But languages with dependent typing check the array size at
compile time, i.e. a dependently typed language would detect this error:

let array = [1]
array[0] // 1, OK
array[1] // Error not detected by compiler but detected at runtime

because in such a language array would be typed as int array of size one,
not int array of any size.

So the real question is in a language like Java, how many times do you get
ArrayStoreException? In my own code I have never seen an array store
exception! Why? Because contravariant writes are very rare. If you were to
protect against contravariant writes you would be complicating the language
for no practical gain, which I think is a bad trade off.

Hope this explains the reasoning,

-- Howard.

On 14 January 2016 at 09:21, Developer <devteam.codafi@gmail.com> wrote:

It does indeed make the type system unsound in some cases. You have
chosen one particular variance direction because it is convenient. A
classic counterexample is Java making Arrays covariant by default. So this
works

Integer arr = [2, 4, 6, 8, 10];
Object orr = arr;
orr[0] = "crash bang";

And crashes at runtime. For that, I must give this part of the proposal
a strong -1. Any amount of type safety I have to give up in the name of
convenience is far too much.

I am, however, a fan of generic protocols. They seem like an orthogonal
concept given the first part here though.

~Robert Widmann

2016/01/13 17:03、Howard Lovatt via swift-evolution <
swift-evolution@swift.org> のメッセージ:

@Thorsten,

It doesn't make the type system unsound; the types are mostly checked at
compile time but some are runtime checked, either way the types are checked
and therefore the type system is sound. I have posted an example of array
runtime type checking in a previous response.

You can annotate variance, but this generally adds a lot of clutter to
the code (see some typical Java code that uses generics you will see stuff
like ? extends Type everywhere). In other languages like Scala the
annotation is less visually intrusive, because they use shorter syntax and
because it is annotated at declaration site rather than use site, but it is
still there.

I think Swift arrays are a good example of partially static and partially
runtime checked. The content is statically typed but the size is runtime
typed. Other languages do type both the content and the size (see Dependent
Typing on Wiki), however at some considerable burden on the programmer.

Hope this explains the thought process,

-- Howard.

On 13 January 2016 at 16:50, Thorsten Seitz <tseitz42@icloud.com> wrote:

Strong -1 from me as well for making the type system unsound.

> Am 13.01.2016 um 02:47 schrieb Howard Lovatt via swift-evolution < >>> swift-evolution@swift.org>:
>
> Yes you can annotate for covariance, invariance, and contravariance,
both Java and Scala, allow all three. The problem is that the code becomes
splattered with variance annotations

Ceylon uses a different approach which is variance annotations at the
definition site.
This restricts the possible usage of the type parameters to
appropriately variant positions.

This migt be a better way to deal with variance.

-Thorsten

--
  -- Howard.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
  -- Howard.

--
  -- Howard.

@Douglas,

Sure you can use SequenceType and constrain its associated types in where
clauses. But it is hard work :(. When you write your own stuff you end up
writing the equivalent of AnySequence and AnyGenerator as well, also hard
work.

I am proposing something that reduces programmer burden and strikes a
different balance,

-- Howard.

¡¡¡

On 14 January 2016 at 15:38, Douglas Gregor <dgregor@apple.com> wrote:

Sent from my iPhone

On Jan 13, 2016, at 5:05 PM, Howard Lovatt via swift-evolution < > swift-evolution@swift.org> wrote:

@Robert,

The behaviour you show for Java arrays is what I am proposing, but that
isn't type unsafe. The type error is detected at runtime and is flagged
with an ArrayStoreException. This is unlike C for example that would allow
the store, at least with a cast, and would put into the array the address
of the string. Therefore neither the proposal or Java is type unsafe, both
are type safe and both detect type errors at runtime.

In case you hadn't noticed, Swift provides a basic form of covariance for
the collections in the standard library... And it does so without the need
for runtime checking, because value semantics allow safe mutation and
collection covariance.

  - Doug

The question is whether protecting against this is worthwhile, not whether
it can be protected against or not (it can be). Arrays are a good example,
Swift takes the approach (like most languages) of checking the array size
at runtime. But languages with dependent typing check the array size at
compile time, i.e. a dependently typed language would detect this error:

let array = [1]

array[0] // 1, OK

array[1] // Error not detected by compiler but detected at runtime

because in such a language array would be typed as int array of size one,
not int array of any size.

So the real question is in a language like Java, how many times do you get
ArrayStoreException? In my own code I have never seen an array store
exception! Why? Because contravariant writes are very rare. If you were to
protect against contravariant writes you would be complicating the language
for no practical gain, which I think is a bad trade off.

Hope this explains the reasoning,

-- Howard.

On 14 January 2016 at 09:21, Developer <devteam.codafi@gmail.com> wrote:

It does indeed make the type system unsound in some cases. You have
chosen one particular variance direction because it is convenient. A
classic counterexample is Java making Arrays covariant by default. So this
works

Integer arr = [2, 4, 6, 8, 10];
Object orr = arr;
orr[0] = "crash bang";

And crashes at runtime. For that, I must give this part of the proposal
a strong -1. Any amount of type safety I have to give up in the name of
convenience is far too much.

I am, however, a fan of generic protocols. They seem like an orthogonal
concept given the first part here though.

~Robert Widmann

2016/01/13 17:03、Howard Lovatt via swift-evolution <
swift-evolution@swift.org> のメッセージ:

@Thorsten,

It doesn't make the type system unsound; the types are mostly checked at
compile time but some are runtime checked, either way the types are checked
and therefore the type system is sound. I have posted an example of array
runtime type checking in a previous response.

You can annotate variance, but this generally adds a lot of clutter to
the code (see some typical Java code that uses generics you will see stuff
like ? extends Type everywhere). In other languages like Scala the
annotation is less visually intrusive, because they use shorter syntax and
because it is annotated at declaration site rather than use site, but it is
still there.

I think Swift arrays are a good example of partially static and partially
runtime checked. The content is statically typed but the size is runtime
typed. Other languages do type both the content and the size (see Dependent
Typing on Wiki), however at some considerable burden on the programmer.

Hope this explains the thought process,

-- Howard.

On 13 January 2016 at 16:50, Thorsten Seitz <tseitz42@icloud.com> wrote:

Strong -1 from me as well for making the type system unsound.

> Am 13.01.2016 um 02:47 schrieb Howard Lovatt via swift-evolution < >>> swift-evolution@swift.org>:
>
> Yes you can annotate for covariance, invariance, and contravariance,
both Java and Scala, allow all three. The problem is that the code becomes
splattered with variance annotations

Ceylon uses a different approach which is variance annotations at the
definition site.
This restricts the possible usage of the type parameters to
appropriately variant positions.

This migt be a better way to deal with variance.

-Thorsten

--
  -- Howard.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
  -- Howard.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
  -- Howard.