Newtype without automatic protocol forwarding

Yes, please!

Definitely a +1 from me too. A feature similar to this has been brought up before but no official pitch/proposal has ever been made. This feature could solve a large number of issues where reusing types for things like identifiers or file handles leads to hard to track bugs.

Any reason not to use the existing RawRepresentable protocol automatically? That would also give you a default implementation for Equatable already (though not an implicitly-added one).

3 Likes

How would using RawRepresentable work with the "Address" example in the OP?

I just mean making the default synthesized struct be

struct Recipient: RawRepresentable {
  var rawValue: Address
  init(rawValue: Address) { … }
}

instead of

struct Recipient {
  var value: Address
}

But I can potentially answer my own question here with another question: what's the access level of value/rawValue? Does it match the access of the newtype, or is it always internal or private? If the latter, it's not suitable for RawRepresentable.

Given the prevalence of protocols for expressing common functionality in Swift, I consider a newtype-like facility without some form of (opt-in) forwarding of protocol conformances to be too small to be useful. Indeed, I think the right feature for Swift is to add opt-in forwarding of protocol conformances without adding newtype, per my comments in Opaque result types - #177 by Douglas_Gregor.

Doug

7 Likes

The OP did state it would be 'semantically' similar to a struct with a value, but it doesn't have to be exactly equivalent to that. I'm no expert, but perhaps it would be possible for the compiler to fudge the access level so that we could have the RawRepresentable conformance while still hiding the internally stored value?

Of course, part of the pitch does state Int is not a RecordId and having RawRepresentable with the copied type would break that wouldn't it?

It also may be against the goals/design of this proposal to do too much compiler magic.

It's not a compiler restriction. Conforming to a protocol means exposing the members that satisfy the requirement because anyone could just write a generic function that does the same thing. In this case, conforming to RawRepresentable on a public newtype means that the newtype has a publicly-accessible rawValue method.

2 Likes

This seems like a huge surprise and yet has no justifying explanation. What's the point of basing one type on another without forwarding conformances and functionality? What use is newtype Identifier = String when I can't do anything with it?

6 Likes

If you just want a newtype without the forwarding mechanism, then couldn't you just use Tagged? I don't see how this rates syntactic sugar if there's no forwarding mechanism.

If Tagged's pre-existing conditional conformances bother you then you can always roll your own, as the overwhelming majority of that (single source file) library is the conditional conformances.

public struct Tagged<Wrapped, Tag> {
    var wrapped: Wrapped
    public init(_ wrapped: Wrapped) { self.wrapped = wrapped }
}

// For when you don't need a tag.
public typealias Newtype<T> = Tagged<T, Void>

Edit: Sorry, I meant to reply directly to the OP, not to @jrose

5 Likes

I think this would be a good idea. We can recommend that types provide conditional extensions to RawRepresentable which forward to the underlying types, and these will be available to both enums and structs. We might even provide sugar on the protocol to provide the appropriate defaults, forwarding any Self parameter to the rawValue:

@rawForwarding public protocol Equatable {
  static func == (lhs: Self, rhs: Self) -> Bool
}
// Compiler synthesizes:
// 
//   extension RawRepresentable where Self: Equatable, RawValue: Equatable {
//     public static func == (lhs: Self, rhs: Self) -> Bool {
//       return lhs.rawValue == rhs.rawValue
//     }
//   }
// 
// So you can write something like this and automatically get a conformance:
// 
//   newtype Identifier: Equatable = Int

If we do go with RawRepresentable, we might also replace the newtype keyword with a syntax similar to the existing enum feature:

struct RecordID: Int, Equatable {}

You would, of course, be free to provide your own init?(rawValue: RawValue) if you needed to validate.

I'm not perturbed by having to make the rawValue public. Enums are already forced to provide a public rawValue if they want the convenient sugar; this is no different. In both cases, you can always forgo the convenience if you really need to control public use.

(But this does make me wonder if we could support some sort of "opaque associated type"—that would make it more difficult to use the rawValue in a way the type didn't want you to. For instance—to assume one of the proposed opaque syntaxes—if you made the raw type Int as Equatable, most code would only know that the raw value was Equatable unless it started fishing for the actual type with as?.)

3 Likes

+1 with opt-in protocol forwarding

I would also like to suggest something for the "Future Directions" section. Allow newtype to make thunks for PATs. For example:

protocol A {
    associatedtype Element
    var element:Element
}

newtype GenericA<T> = A where Element == T 

would generate at compile time:

struct GenericA<T>:A {
    typealias A = T
    private var box:GenericABox<T>
    init<P:A>(_ boxed:P) {
        self.box = GenericAConcrete(boxed)
    }
    var element:A {
        return self.box.element
    }
}

private class GenericABox<T>:A {
     typealias A = T
     var element:A {fatalError()}
}

final private class GenericAConcrete<P:A>:GenericABox<P.A> {
    var concrete:P
    init(_ concrete:P){
        self.concrete = concrete
    }
    override var element:A {
        return self.concrete.element
    }
}

You could also specialize without generics:

newtype IntA = A where Element == Int

or completely erase a type:

newtype AnyA = A where Element == _

The last one would omit any setters of the Element type, and all getters/returns of Element would return Any instead. For example:

newtype AnyCollection = Collection where Element == _

would allow calls to things like count, and would return Any from things like first and the subscript getter. Setting values via subscript, etc... would not be possible in the wrapper.

Two optional (but super nice to have) pieces of compiler magic:

  1. The ability to unbox using as? or as!
  2. The ability to automatically box when assigning a conforming type to a variable of one of these newtyped thunks
let c:AnyCollection = myCollection //Auto Box
//vs
let c:AnyCollection = AnyCollection(myCollection)

guard let a = c as? MyCollection else {...}  //Unbox using as
//vs
let a:MyCollection = try c.unbox()

My initial thought seeing this was, +1! It is something I came across recently in a project I am working on.

I mean, this proposal would simplify this code:

internal struct Device: Encodable {
    internal let id: UUID

    internal init(id: UUID) {
        self.id = id
    }
}

to something like this:

newtype Device = UUID

My only concern with this is that it indirectly competes with the existing syntax (not a deal breaker for me personally).

typealias Something = UUID

What I would be more interested in seeing is what @jrose suggested above - having synthesized structs seems like a very reasonable approach.

“Considerably harder” seems like an exaggeration to me. I understand the temptation to pare this pitch down to something uncontroversial, but I don't think it clears the bar if all it does is remove a few lines of very straightforward code (most of which only exist because of acknowledged deficiencies in automatic initialiser generation and will often not be required). In the common case I'm left comparing

struct Identifier { var value: String }
newtype Identifier = String

which is a hard sell.

2 Likes

I'm moderately +1 on this and I can see how this proposal could come in handy. I think the name is really confusing, though, and might collide with a fuller featured "new type" in the future.

With the name "newType", I'd expect it to behave like a subclass, if a subclass were passed by value instead of reference. Like I'd assume the compiler would let me pass a MyInt to myfunc(foo:Int), but not an Int to myfunc(foo:MyInt)

This seems more like it should be "WrappedType" or "UniqueType" or something.

The failable initialiser seems problematic to me. How would forwarding init() of e.g. the SetAlgebra protocol work if the newtype's initialiser can fail?

I made a last-minute change to the title and forgot to include one important word: "automatic". I've updated the title: "Newtype without automatic protocol forwarding".

Protocol forwarding is a great feature, and newtype could definitly use it in some form. But I think automatic protocol forwarding would go against the principle of a new type, if it isn't truly distinct.

Like any other normal type, the new type should only implement protocols on a opt-in basis. The implementation of a protocol could also differ from the implementation of the underlying type.

Perhaps with one exception: Automatically implementing of Equatable and Hashable (if the underlying type implements those) would be very useful. Those two protocols in particular for the same reasons they can already be generated by the Swift compiler.

With Equatable and Hashable implemented, all three of our original use cases are handled: Identifiers can be compared and used as keys in dictionary. Sender and Receiver addresses are separated by their types, until the underlying Address value is actually needed.

Newtype does become more useful with optional protocol forwarding. But that's also true for normal structs.

2 Likes

Not really, I wasn't sure if it's semantically correct. The optional initializer for RawRepresentable is also bit weird in this context.

In one of our possible future enhancements, we use a Newtypable protocol, that could also be the existing RawRepresentable.

As for the access of the newtype. I would imagine to use the access modifier of the newtype for the value as well:

// A public newtype
public newtype Identifier = String

// Becomes something like:
public struct Identifier {
  public let value: String
  public init(value: String) { self.value = value }
}

Although the private version should probabibly use fileprivate for the members:

// A private newtype
private newtype Identifier = String

// Becomes something like:
private struct Identifier {
  fileprivate let value: String
  fileprivate init(value: String) { self.value = value }
}

In our projects we often use separate frameworks, so we need types to be public. We also need Equatable and Hashable, to store the identifiers as keys in dictionaries.

So this:

public newtype Identifier = String

Would be:

public struct Identifier: Equatable, Hashable {
  public let value: String
  public init(value: String) { self.value = value }
}

This four line version is a lot more work to initially write, and more importantly also harder to read back.
So instead of adding type-safety and create an Identifier, I currently often chose the quick-and-dirty solution and use a String instead.

I would be quite perturbed if this solution were adopted. Access control should be completely orthogonal to any official forwarding mechanism intended to support composition (including the simple case of newtype). In fact, any such forwarding mechanism should be completely orthogonal to every aspect of the forwarder's interface aside from the part that is being forwarded. Using any protocol to drive forwarding would mean that newtype is translucent rather than opaque, thus violating encapsulation.

The fact that the enum sugar requires exposing rawValue and init(rawValue:) makes it unusable by a library which does not wish to expose rawValue or especially init(rawValue:). This has already caused me to grumble more than once, wishing I could saw public enum Foo: internal Int (or something like that) to derive the rawValue and init(rawValue:) without having to expose them outside a library. It obviously wouldn't be possible to synthesize a RawRepresentable conformance in this case, but synthesis of the members rawValue and init(rawValue:) members themselves would still be useful.

I also believe a forwarding mechanism should be completely orthogonal to every aspect of the forwardee's interface as well, except what is absolutely necessary to perform forwarding (as if it were written manually). This is why [Proposal Draft] automatic protocol forwarding did not require actual protocol conformance by the forwardee. It only required potential protocol conformance. This point was somewhat controversial but I believe is an important one. There are any number of reasons one may not wish to provide an actual conformance for the forwardee (mostly related to access control and dynamic casts).

This would not be an adequate solution IMO. The fact that a type is implemented as a newtype should be an implementation detail of a type, not part of its interface. What if I need to change the implementation later? Under this design that would be a breaking change making newtype a feature that is unavailable to library authors who care about preserving implementation flexibility.

Frankly, I don't see this feature adding much value over what we can already do unless it includes a forwarding mechanism that is able to properly preserve encapsulation of implementation details. If we're going to have such a mechanism, it should be much more general than newtype, supporting encapsulation-preserving composition in general. Of course newtype could be added as syntactic sugar on top of such a mechanism as it would be a very common use case.

In summary, I agree with @Douglas_Gregor that we should be focused on forwarding rather than rushing into adding a half-baked newtype feature that inherently leaks implementation details.

8 Likes