Support for newtype feature/typesafe calculations

Grzegorz_Adam_Hankie · 2016-01-05T16:11:16.000Z

Hello, newcomer here.

The other day I was reading about https://www.reddit.com/r/swift/comments/3zbk64/type_math/ and given the verbose solutions decided to pitch a possible language improvement. But after reading through the archives it seems that this idea has already been pitched before:

* Proposal: newtype feature for creating brand new types from existing types

- [swift-evolution] Proposal: newtype feature for creating brand new types from existing types

Some time later:

* Epic: Typesafe calculations

- [swift-evolution] Epic: Typesafe calculations

Since this is already known I would like to talk about this potential feature in order to address the perceived lack of community interest.

Recently I’ve been bitten by a lack of this feature for code dealing with database serialisation. If a database table is modelled as an object it will most likely use integers or longs to represent primary and foreign keys. An example would be:

struct SomeTable {
    var primaryKey: Int64
    var otherTableForeignKey: Int64
    var yeatAnotherForeginKey: Int64
}

Unfortunately the types can be mixed, one can assign the value from otherTableForeignKey to primaryKey without impunity. Using the reddit proposed struct as separate type would help here:

    public struct RefTablePk {
        private let value: Int64
    }

    public struct SomeTablePk {
        private let value: Int64
    }

    struct RefTable {
        var primaryKey: RefTablePk
    }

    struct SomeTable {
        var primaryKey = SomeTablePk(value: 0)
        var otherTableForeignKey = RefTablePk(value: 0)
    }

    var a = SomeTable()
    a.primaryKey = SomeTablePk(value: 3)
    a.otherTableForeignKey = a.primaryKey // Fails, good!
    print(a.primaryKey.value)

So far so good. The solution gets more hairy when one attempts to use such fake types for mathematical operations because the new type doesn’t inherit possible operations done on the parent type it is based of:

    public struct Euros {
        private let value: Double
    }

    public struct Dollars {
        private let value: Double
    }

    var account = Euros(value: 100)
    var bill = Euros(value: 42)
    account = account - bill

The last line won’t compile: Binary operator ‘-‘ cannot be applied to two ‘Euros’ operands. The immediate solution would be to add variants of all the usual operators for every type, which quickly gets tedious and verbose. On top of this, each *new* type is generating code, which is actually not desirable. The ideal would be for the compiler to pretend Euros or RefTablePk are different types, yet use their parent type at the binary level. This needs a specific syntax to teach the compiler which existing methods/operations are allowed on the new fake types and which aren’t. These new distinct types would *borrow* previous implementations.

Other examples of possible use for such fake types (note again: these exist only at compilation time and generate no extra code) would be user input validation. Strings coming from *the exterior* (whatever that might be) could use a distinct type of TaintedString instead of the usual String, and would require a validation or converter function to produce a correct String, or maybe a ValidatedString for better distinction. Similarly for a security library it would be useful to mark Strings as encrypted. The EncryptedString would be tracked by the compiler to prevent mistakes or uses where plaintext strings are expected. One bug I recently had the pleasure to solve was mixing time units, some APIs use seconds, other use milliseconds, both store the values as longs. The *fix* was to wrap all primitive type access inside objects with explicit methods having the unit in their signature. Effective, but tedious and verbose.

The typealias jumped out of the swift language documentation while I was reading it, but it is actually the opposite of the desired behaviour. The current type alias doesn’t prevent one from mixing these new fake types so it has effectively the same value as a user comment. Also, having seen swift-evolution/0011-replace-typealias-associated.md at master · apple/swift-evolution · GitHub it seems that the keyword won’t be used any more, plus it would be convenient to invent a new one to avoid migration mistakes.

Examples I know of this feature in other languages and/or implementations which can be used for reference/inspiration:

- C++. I asked c++ - How can I create a new primitive type using C++11 style strong typedefs? - Stack Overflow while following a talk by Bjarne Stroustrup and some of the answers include using boost units (http://www.boost.org/doc/libs/1_60_0/doc/html/boost_units/Units.html\) to alleviate writing the required boilerplate code. The library was already mentioned here as reference in previous threads.

- Nim (http://nim-lang.org). Defined as distinct types (Nim Manual) they can be used with the borrow pragma and templates to avoid boilerplate code yet customise which existing procs can be applied to the new distinct type.

- Java. The language doesn’t allow this kind of fake types, but it has been patched to various degrees through annotation support. Recently Android Studio added resource type annotations (Support Annotations - Android Studio Project Site) for mobile developers which allow marking integers with @StringRes, @IntegerRes and similar. The linter will look for these annotations and issue warnings when the assigned literals don’t satisfy the requirements. A more generic approach can be found in the Checker Framework in the Subtyping checker (The Checker Framework Manual: Custom pluggable types for Java). The provided example shows how this hypothetical encrypted/plaintext string distinction would work. They also support the typical units annotations (The Checker Framework Manual: Custom pluggable types for Java).

Precisely this last reference is an argument against implementing this feature directly at the language level. I’ve seen that there is an attributes proposal (https://github.com/rsmogura/swift/wiki/Attributes-Proposal\) which would provide the equivalent of java’s annotations. Given that Java has a checker framework it would surely make sense to implement this similarly in swift? I don’t think so. Essentially nobody I’ve worked with knows about such annotations or external tools (sad). While anecdotal, it’s easy for me to think that annotations are easy to brush off and forget.

And that’s why I’ve written this email, hopefully I can spur interest in others for this feature so that it becomes a first class citizen of the language at some point. I still haven’t got enough experience with swift to make a more concrete proposal. If nobody picks this up you may hear from me in the future.

anandabits · 2016-01-05T16:24:03.000Z

Thanks for doing the work to collect links related to this topic! I am planning to give them a read.

You may be interested in the proposal I am working on for automatic protocol forwarding.

The first draft was posted to the list last week: https://github.com/anandabits/swift-evolution/blob/automatic-protocol-forwarding/proposals/NNNN-automatic-protocol-forwarding.md\. It will give you an idea of where the proposal is going, although it is somewhat out of date at this point. The motivation section is also not fully fleshed out. Several good suggestions came up during the discussion leading to some important design changes.

I am working on a significantly enhanced second draft right now. It will include examples showing how it can be used to achieve a similar effect to `newtype` (as well as several other examples). It won’t be quite as concise syntactically as it is designed to support many use cases, not just the `newtype` use case. A future enhancement to that proposal might allow more concise syntax to better support the `newtype` use case.

I am hoping to have the second draft ready to share soon.

Matthew

···

On Jan 5, 2016, at 10:11 AM, Grzegorz Adam Hankiewicz via swift-evolution <swift-evolution@swift.org> wrote:

Hello, newcomer here.

The other day I was reading about Reddit - Dive into anything and given the verbose solutions decided to pitch a possible language improvement. But after reading through the archives it seems that this idea has already been pitched before:

* Proposal: newtype feature for creating brand new types from existing types

- [swift-evolution] Proposal: newtype feature for creating brand new types from existing types

- [swift-evolution] Proposal: newtype feature for creating brand new types from existing types

Some time later:

* Epic: Typesafe calculations

- [swift-evolution] Epic: Typesafe calculations

Since this is already known I would like to talk about this potential feature in order to address the perceived lack of community interest.

Recently I’ve been bitten by a lack of this feature for code dealing with database serialisation. If a database table is modelled as an object it will most likely use integers or longs to represent primary and foreign keys. An example would be:

struct SomeTable {
   var primaryKey: Int64
   var otherTableForeignKey: Int64
   var yeatAnotherForeginKey: Int64
}

Unfortunately the types can be mixed, one can assign the value from otherTableForeignKey to primaryKey without impunity. Using the reddit proposed struct as separate type would help here:

   public struct RefTablePk {
       private let value: Int64
   }

   public struct SomeTablePk {
       private let value: Int64
   }

   struct RefTable {
       var primaryKey: RefTablePk
   }

   struct SomeTable {
       var primaryKey = SomeTablePk(value: 0)
       var otherTableForeignKey = RefTablePk(value: 0)
   }

   var a = SomeTable()
   a.primaryKey = SomeTablePk(value: 3)
   a.otherTableForeignKey = a.primaryKey // Fails, good!
   print(a.primaryKey.value)

So far so good. The solution gets more hairy when one attempts to use such fake types for mathematical operations because the new type doesn’t inherit possible operations done on the parent type it is based of:

   public struct Euros {
       private let value: Double
   }

   public struct Dollars {
       private let value: Double
   }

   var account = Euros(value: 100)
   var bill = Euros(value: 42)
   account = account - bill

The last line won’t compile: Binary operator ‘-‘ cannot be applied to two ‘Euros’ operands. The immediate solution would be to add variants of all the usual operators for every type, which quickly gets tedious and verbose. On top of this, each *new* type is generating code, which is actually not desirable. The ideal would be for the compiler to pretend Euros or RefTablePk are different types, yet use their parent type at the binary level. This needs a specific syntax to teach the compiler which existing methods/operations are allowed on the new fake types and which aren’t. These new distinct types would *borrow* previous implementations.

Other examples of possible use for such fake types (note again: these exist only at compilation time and generate no extra code) would be user input validation. Strings coming from *the exterior* (whatever that might be) could use a distinct type of TaintedString instead of the usual String, and would require a validation or converter function to produce a correct String, or maybe a ValidatedString for better distinction. Similarly for a security library it would be useful to mark Strings as encrypted. The EncryptedString would be tracked by the compiler to prevent mistakes or uses where plaintext strings are expected. One bug I recently had the pleasure to solve was mixing time units, some APIs use seconds, other use milliseconds, both store the values as longs. The *fix* was to wrap all primitive type access inside objects with explicit methods having the unit in their signature. Effective, but tedious and verbose.

The typealias jumped out of the swift language documentation while I was reading it, but it is actually the opposite of the desired behaviour. The current type alias doesn’t prevent one from mixing these new fake types so it has effectively the same value as a user comment. Also, having seen https://github.com/apple/swift-evolution/blob/master/proposals/0011-replace-typealias-associated.md it seems that the keyword won’t be used any more, plus it would be convenient to invent a new one to avoid migration mistakes.

Examples I know of this feature in other languages and/or implementations which can be used for reference/inspiration:

- C++. I asked c++ - How can I create a new primitive type using C++11 style strong typedefs? - Stack Overflow while following a talk by Bjarne Stroustrup and some of the answers include using boost units (http://www.boost.org/doc/libs/1_60_0/doc/html/boost_units/Units.html\) to alleviate writing the required boilerplate code. The library was already mentioned here as reference in previous threads.

- Nim (http://nim-lang.org). Defined as distinct types (Nim Manual) they can be used with the borrow pragma and templates to avoid boilerplate code yet customise which existing procs can be applied to the new distinct type.

- Java. The language doesn’t allow this kind of fake types, but it has been patched to various degrees through annotation support. Recently Android Studio added resource type annotations (Support Annotations - Android Studio Project Site) for mobile developers which allow marking integers with @StringRes, @IntegerRes and similar. The linter will look for these annotations and issue warnings when the assigned literals don’t satisfy the requirements. A more generic approach can be found in the Checker Framework in the Subtyping checker (The Checker Framework Manual: Custom pluggable types for Java). The provided example shows how this hypothetical encrypted/plaintext string distinction would work. They also support the typical units annotations (The Checker Framework Manual: Custom pluggable types for Java).

Precisely this last reference is an argument against implementing this feature directly at the language level. I’ve seen that there is an attributes proposal (https://github.com/rsmogura/swift/wiki/Attributes-Proposal\) which would provide the equivalent of java’s annotations. Given that Java has a checker framework it would surely make sense to implement this similarly in swift? I don’t think so. Essentially nobody I’ve worked with knows about such annotations or external tools (sad). While anecdotal, it’s easy for me to think that annotations are easy to brush off and forget.

And that’s why I’ve written this email, hopefully I can spur interest in others for this feature so that it becomes a first class citizen of the language at some point. I still haven’t got enough experience with swift to make a more concrete proposal. If nobody picks this up you may hear from me in the future.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

jeremyp · 2016-01-05T17:19:24.000Z

I’m not saying your proposal would not be useful, but in the case of all of your examples, there are solutions that can be implemented now that might be better than just creating a new-type-but-exactly-the-same.

The other day I was reading about Reddit - Dive into anything

The idea of having a different type for each unit of distance is just wrong. You have one type for Distance with a value in a canonical unit and properties that extract the value in different units e.g

struct Distance
{
    let m: Double // SI unit of distance
    var km: Double { return m / 1000 }
    var cm: Double { return m * 100 }
    var miles: Double { return km * 0.62 }
}

Then you only need one set of addition and subtraction operators. You would also have multiplication by a dimensionless constant yielding a Distance and multiplication of two Distances yielding an Area.

struct SomeTable {
   var primaryKey: Int64
   var otherTableForeignKey: Int64
   var yeatAnotherForeginKey: Int64
}

Unfortunately the types can be mixed, one can assign the value from otherTableForeignKey to primaryKey without impunity.
Using the reddit proposed struct as separate type would help here:

   public struct RefTablePk {
       private let value: Int64
   }

   public struct SomeTablePk {
       private let value: Int64
   }

   struct RefTable {
       var primaryKey: RefTablePk
   }

   struct SomeTable {
       var primaryKey = SomeTablePk(value: 0)
       var otherTableForeignKey = RefTablePk(value: 0)
   }

   var a = SomeTable()
   a.primaryKey = SomeTablePk(value: 3)
   a.otherTableForeignKey = a.primaryKey // Fails, good!
   print(a.primaryKey.value)

That all looks fine except, why not take advantage of nested types:

struct SomeTable
{
struct PrimaryKey { let value: Int }

var primaryKey: PrimaryKey
var refTableId: RefTable.PrimaryKey
}

So far so good. The solution gets more hairy when one attempts to use such fake types for mathematical operations

Why would you want to do mathematical operations on the primary key of a table? Primary keys obviously need to be Equatable and possibly Comparable (except what if the database uses a GUID as its key) but that is about it.

   public struct Euros {
       private let value: Double
   }

   public struct Dollars {
       private let value: Double
   }

   var account = Euros(value: 100)
   var bill = Euros(value: 42)
   account = account - bill

The last line won’t compile: Binary operator ‘-‘ cannot be applied to two ‘Euros’ operands.

How about a Currency protocol?

protocol Currency
{
    // value in smallest unit of currency e.g. EUR = cents, GBP = pence
    var rawValue: Int64 { get }
    init(rawValue: Int64)
}

func -<T: Currency>(a: T, b: T) -> T
{
return T(rawValue: a.rawValue - b.rawValue)
}

struct EUR: Currency
{
var rawValue: Int64
}

struct USD: Currency
{
var rawValue: Int64
}

let foo = EUR(rawValue: 2000)
let bar = EUR(rawValue: 1000)

let baz = foo - bar // compiles

let whizz = USD(rawValue: 4000)

let biz = baz - whizz // error: binary operator '-' cannot be applied to operands of type 'EUR' and 'USD'

Only one - operator needs to be defined for all currencies and the fact that EUR and USD are intended to model currencies is self documenting.

So I don’t think any of your examples are compelling evidence for your proposal. That’s not to say there are not good reasons for it, I just haven’t seen them yet.

···

On 5 Jan 2016, at 16:11, Grzegorz Adam Hankiewicz via swift-evolution <swift-evolution@swift.org> wrote:

trs · 2016-01-05T17:16:12.000Z

What about citing the relevant protocols in the newtype definition? This should include the ability to use my own protocols to which I have made the underlying type conform to by an extension.

Throwing some syntax into the discussion:

newtype Euro = Double : Addable, Subtractable

where I have defined the protocols Addable and Subtractable somewhere and made Double conform to them if all this is not provided by the standard library.
The implementation of Euro then borrows the implementation of Double for these protocols.

-Thorsten

···

Am 05.01.2016 um 17:11 schrieb Grzegorz Adam Hankiewicz via swift-evolution <swift-evolution@swift.org>:

The ideal would be for the compiler to pretend Euros or RefTablePk are different types, yet use their parent type at the binary level. This needs a specific syntax to teach the compiler which existing methods/operations are allowed on the new fake types and which aren’t. These new distinct types would *borrow* previous implementations.

anandabits · 2016-01-05T17:24:18.000Z

The ideal would be for the compiler to pretend Euros or RefTablePk are different types, yet use their parent type at the binary level. This needs a specific syntax to teach the compiler which existing methods/operations are allowed on the new fake types and which aren’t. These new distinct types would *borrow* previous implementations.

What about citing the relevant protocols in the newtype definition? This should include the ability to use my own protocols to which I have made the underlying type conform to by an extension.

This is how my forwarding proposal works. The newtype syntax I suggested as a possible extension looks like this:

newtype Euro = Double forwarding Addable, Subtractable

The keyword could be different, but I think `forwarding` is not bad. When I complete the second draft I think it will make even more sense. The forwarding facility has features to handle non-trivial cases (Self and associated type requirements, etc).

···

On Jan 5, 2016, at 11:16 AM, Thorsten Seitz via swift-evolution <swift-evolution@swift.org> wrote:

Am 05.01.2016 um 17:11 schrieb Grzegorz Adam Hankiewicz via swift-evolution <swift-evolution@swift.org>:

Throwing some syntax into the discussion:

newtype Euro = Double : Addable, Subtractable

where I have defined the protocols Addable and Subtractable somewhere and made Double conform to them if all this is not provided by the standard library.
The implementation of Euro then borrows the implementation of Double for these protocols.

-Thorsten
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

James_Campbell · 2016-01-06T17:15:52.000Z

I've managed to implement this already in the language with a few ugly
corners due to the lack of generic protocols.

I created a protocol based on Box (https://github.com/robrix/Box/\) which
works really well. I have extended this to handle certain special protocols
like Equatable so you can do SpecialType == SpecialType, and even
literalConversion.

There is however a lot of boilerplate:

- You have to declare all of your Convertible protocols for converting from
one type to another
- You have to define an empty init so the protocol extensions have
something to chain to.
- You need to write the value property with type.

Due to the lack of protocol generics, you also need to have a protocol for
every type you wish to box which sets the associated type. Of course I
could have done this with classes but I wanted to keep this as a value type
:).

With member-wise initializations and generic protocols this could be
achievable just by adding a Box protocol to the standard library.

Here is my implementation of Box as a protocol:

*protocol Box: CustomStringConvertible, CustomDebugStringConvertible {*

* typealias FloatLiteralType = Double*

* typealias IntegerLiteralType = Int*

* typealias BoxType = Any*

* var value: BoxType { get set }*

* init()*

* init(_ value: BoxType)*

*}*

*extension Box where BoxType: CustomStringConvertible {*

* var description: String {*

* return self.value.description*

* }*

* var debugDescription: String {*

* return "\(self.value.description)㎭"*

* }*

*}*

*//MARK: FloatingPointBox*

*protocol FloatingPointBox: Box, FloatLiteralConvertible,
IntegerLiteralConvertible {*

* typealias BoxType = Double*

* typealias FloatLiteralConvertible = Double*

* typealias IntegerLiteralConvertible = Int*

*}*

*extension Box where Self.BoxType == Double {*

* init(_ value: Double) {*

* self.init()*

* self.value = value*

* }*

* init(_ value: Int) {*

* self.init()*

* self.value = Double(value)*

* }*

*}*

*extension FloatLiteralType {*

* init<T: Box where T.BoxType == Double >(_ box: T) {*

* self.init(box.value)*

* }*

* init<T: Box where T.BoxType == Int >(_ box: T) {*

* self.init(box.value)*

* }*

*}*

*extension CGFloat {*

* init<T: Box where T.BoxType == Double >(_ box: T) {*

* self.init(box.value)*

* }*

* init<T: Box where T.BoxType == Int >(_ box: T) {*

* self.init(box.value)*

* }*

*}*

*//Adding FloatLiteralConvertible, IntegerLiteralConvertible*

*extension FloatingPointBox where Self.BoxType == Double,
Self.FloatLiteralConvertible == Double {*

* init(floatLiteral value: Double) {*

* self.init(value)*

* }*

* init(integerLiteral value: Int) {*

* self.init(value)*

* }*

* init<T: IntegerType>(_ value: T) {*

* self.init(value)*

* }*

*}*

Here is my example of using the Box protocol:

*struct Degree: FloatingPointBox {*

* var value: Double = 0*

* init()*

* {*

* }*

*}*

*protocol DegreeConvertiable {*

* init(degreeLiteral value: Degree)*

*}*

*extension Degree: RadianConvertiable {*

* init(radianLiteral value: Radian) {*

* self.value = Double(value) * 180.0 / M_PI*

* }*

* init(_ value: Radian) {*

* self.init(radianLiteral: value)*

* }*

*}*

···

On Tue, Jan 5, 2016 at 5:24 PM, Matthew Johnson via swift-evolution < swift-evolution@swift.org> wrote:

> On Jan 5, 2016, at 11:16 AM, Thorsten Seitz via swift-evolution < > swift-evolution@swift.org> wrote:
>
>
>> Am 05.01.2016 um 17:11 schrieb Grzegorz Adam Hankiewicz via > swift-evolution <swift-evolution@swift.org>:
>>
>> The ideal would be for the compiler to pretend Euros or RefTablePk are
different types, yet use their parent type at the binary level. This needs
a specific syntax to teach the compiler which existing methods/operations
are allowed on the new fake types and which aren’t. These new distinct
types would *borrow* previous implementations.
>
> What about citing the relevant protocols in the newtype definition? This
should include the ability to use my own protocols to which I have made the
underlying type conform to by an extension.

This is how my forwarding proposal works. The newtype syntax I suggested
as a possible extension looks like this:

newtype Euro = Double forwarding Addable, Subtractable

The keyword could be different, but I think `forwarding` is not bad. When
I complete the second draft I think it will make even more sense. The
forwarding facility has features to handle non-trivial cases (Self and
associated type requirements, etc).

>
> Throwing some syntax into the discussion:
>
> newtype Euro = Double : Addable, Subtractable
>
> where I have defined the protocols Addable and Subtractable somewhere
and made Double conform to them if all this is not provided by the standard
library.
> The implementation of Euro then borrows the implementation of Double for
these protocols.
>
> -Thorsten
> _______________________________________________
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
 Wizard
james@supmenow.com
+44 7523 279 698