Newtype for Swift

newtype has come up again and since I don't have a specific pitch I thought I'd start some discussion outside of the older thread. Before diving into specific syntax or implementation details I think it would be good to gather ideas about what people want out of such a feature and what they feel is most important.

Personally, I'm a fan of starting with simpler newtype behavior: newtypes inherit all of the functionality and conformances of the original type, but are considered separate types. Developers could add additional functionality and protocol conformances but couldn't override any. It would also be useful if these simple newtypes were covariant with their original types, but I'm not sure what the overall impact would be there. To my mind, this is a useful feature on its own, even given its limited nature.

If we wanted to go further, perhaps extensions could be used to override protocol conformances, as those are usually self contained. However, there may be other issues around some of the original type's conformances depending on overridden conformances. So perhaps if you override a dependent conformance you have to override whatever depends on it? (e.g. If you want custom Sequence behavior, you'll need to override the Collection implementation as well to use your new conformance.) I think these overrides would still be covariant with the original type since no API surface has been removed.

If we cared more about customizing which protocols the newtypes conform to and don't care about losing covariance, it seems likely that explicitly listing the conformances we want, either as part of the declaration or as extensions that don't implement anything, would be an easy way to accomplish that. This would allow developers to pick and choose which conformances from a type they want. I'm not sure what a great motivating example would be for this aspect of the feature, anyone have ideas?

Two aspects of newtypes I'm most unsure about are the overriding or removal of non-conformance API. Given the internal dependence on many of these APIs it seems likely it would be very difficult or dangerous to allow arbitrary overrides (though that may also be true of protocol conformances as well). Similarly, I'm also not sure the probable verbosity of any feature controlling the visiblity of particular APIs would hold its weight.

21 Likes

I much rather have a way to declare a custom subtype in a similar fashion Optional<T> is subtype of T implicitly. Somewhere in between typealias and newtype

4 Likes

We already have typealias, generic Wrapper<T> and inverse generic newtype newname { associatedtype value:Value } which can be auto synthesized by compiler for this feature, furthermore Swift has already become a very complex type system and a super grab bag of syntaxes from other languages, it's better to focus on some more important features rather than adding syntax sugar, sugar of sugar, or very limited usage scenarios type aliasing newtype = oldtype with custom protocol conformance but type(of:newtype)!=type(of:oldtype) type...

It maybe has some merits in specific edge cases, nevertheless let's make swift simple to understand and more swifty to write.

12 Likes

As the author of the original pitch I think I should reiterate why we didn’t want to automatically inherit all conformances of the original type.

One of our motivating examples was Identifiers:

struct Person {
  newtype Identifier = String

  let id: Identifier
  let name: String
}

Here, we don’t want Identifier to inherit almost any of the many features String has. We don’t want to concat two Identifiers, we don’t want to create a substring from an Identifier. None of those features make any sense.
In fact, the only thing we want from Identifier is Equatable, Hashable en perhaps CustomStringConvertable (but only for debugging).

So if newtype automatically inherits all the behaviours of the original type, it would make it less useful for me, because now I can no longer use it for Identifiers.

Additionally, it would be really nice to optionally request to inherit (forward) existing behaviour. In cases where it is needed, like in a lot of Int/Float examples.

9 Likes

maybe
newtype newname(copy) = oldtype copy everything/behaviors of oldtype
newtype newname:P... = oldtype customized protocols conformance newtype
newtype newname = oldtype by default copy nothing behaviors just pure repackage of oldtype value
is what you want.

But newtype decays into a value wrapper without value operations.

Maybe you also don't want all the functions implemented by String and yet not coming from protocols. So it would be cherry picking and make the type very hard to understand. And it wouldn't be a String anymore. In these case I think it would make more sense to have a real separate type.

When I find myself wishing for a "newtype" it is just to make sure I'm not inverting/mixing two attributes with same types yet with different meanings. Basically I'd like to be able to call a function with Person.Identifier when it expect a String yet not being able to call a function expecting Person.Identifier with any String. So some sort of enforced typealias.

4 Likes

I'll second this. I realise Swift is not Haskell but the original point of newtypes is to give a type of publicly known shape a different meaning, which the type system guards for as long as the newtype wrapping isn't stripped by the developer. In that sense, I think the preferred features for a similar feature in Swift would be:

  1. The syntax for declaring newtypes should be light.
  2. There should be a clear way to explicitly convert both ways between the wrapped type and the newtype.
  3. Conformances should be explicit but trivial to add alongside or separate from the newtype declaration.

But point 3 might be hard to achieve for arbitrary protocols, so a lower bar might be to just list a set of protocols for which the explicit auto-conformance were allowed (e.g. Equatable, Hashable, Comparable, Custom*StringConvertible, ExpressibleBy*Literal, {En,De}codable, RawRepresentable, ...).

I think the original pitch caught much of this spirit already, but I forget why it didn't progress.

2 Likes

I’d go further that the conversion is very limited to a few places. I don’t want users to create gibberish ID willy-nilly.

I agree. Most of the discussion so far is focused on mechanics and not the problem to be solved.

Single property structs are trivial to write. Properties can easily be forwarded using @dynamicMemberLookup. Protocol conformances can easily be forwarded using a protocol.

There are some things that are not possible today such as:

  • forwarding methods that are not part of a protocol
  • encapsulating the wrapped value while still forwarding protocols
  • forwarding some protocols but not others (this is possibly, but only with a parallel set of forwarding protocols)

I think we need to be really clear about the use cases that are not well supported by the language today and would therefore make good motivation for a new language feature.

I dug up an old thread I had written about protocol-based forwarding. I was working on a second draft of that pitch when the Swift 3 proposal pause happened and never picked it up again. Some of the details should look different than the ones in that thread, but I still think a more general protocol-based forwarding feature is the right direction. It would be useful in many cases where newtype alone would not. newtype would still be possible to add as additional sugar.

This is somewhat representative of the kind of example that might make good motivation. However, I personally think there is benefit in handing identifiers using a single generic ID type parameterized by a “scope” rather than a bunch distinct newtypes.

When it's String but I don't want to mix it up with other Strings, I do:

struct Person {
    struct Name: RawRepresentable { var rawValue: String }
    var name: Name
}

var person = Person(name: Person.Name(rawValue: "John"))
print(person.name.rawValue)  // Use as String when necessary.
person.name = "Steve"  // But you can't do this. Good.

And this is exact what Notification.Name does. So, there's legit use case here. But I'm not sure if new syntax and mechanism is really better than a simple implementation like above.

1 Like

If newtype doesn't strip functionality it seems like we'd get that for free, or something like it. I don't know that the conversion would go both ways, but I certainly think it would very useful for newtypes to accept the original type as well.

Then why do you want a newtype at all? You can accomplish what you want now with a simple wrapping type, you don't need newtype to erase a type's functionality. To justify newtype's existence we need motivating examples that aren't currently possible in the language, which is why I start with the "everything" case, since it's not really possible to create a separate type that inherits another type's functionality automatically.

This is my thought as well. Once this sort of functionality is in place, graceful degradation of capability through hiding or selective declarations of conformance could happen next.

1 and 2 make sense, but I don't see why 3 would be. To my mind, wanting all of the functionality would be the common case, and is where newtype is most valuable, as wrapper types are easiest when you don't need most of the conformances or functionality of the wrapped type.

Properties could be, but methods can't. Perhaps if keypaths were extended, but we're far away from that, and wouldn't help in the protocol case. Perhaps if protocols could see through @dynamicMemberLookup for conformances. And while the gist could be a possible underlying implementation, it seems seems limiting to force the implementation to manually declare wrappers for every possible conformance, and tedious for every newtype to have to manually declare them. It also seems less maintainable to have to redeclare every new conformances the original type gains.

It's better when you want the capabilities of the original type. It wouldn't really replace the use case you've mentioned.

3 Likes

I find myself agreeing with @Jon_Shier here. We can already make a new type that doesn’t forward to its base type. So if there’s any for reason newtype to exist, it should be to make a type that does forward everything.

Protocol forwarding in general is the powerhouse in this space. For example, let’s consider LazyMapCollection. If we had the ability to forward protocols and selectively implement only the requirements which are different from the base type, it could look more like this:

extension LazyMapCollection: Collection via _base {
  public typealias SubSequence = LazyMapCollection<Base.SubSequence, Element>

  public subscript(position: Base.Index) -> Element {
    return _transform(_base[position])
  }

  public subscript(bounds: Range<Base.Index>) -> SubSequence {
    return SubSequence(_base: _base[bounds], transform: _transform)
  }
}

All the other members that simply forward to _base would be synthesized by the compiler.

If we had this sort of protocol forwarding, then the scenarios which people have described of wanting to forward only a few protocols, could easily be achieved without newtype.

The only remaining use-case for newtype is when you want to forward everything. But maybe we could work that into standard declaration syntax form as well, eg:

struct Foo via _base { var _base: String }

That’s a bit more verbose than newtype Foo = String, but it’s also a lot easier to work with if we want to implement some members directly on Foo which don’t get forwarded.

Seems to me that the real problem is not declaring the type, but implementing protocol conformances and non-protocol members. While I don’t have a solution for the latter, I did just come up with a pretty general solution for the former:

struct ID: Hashable by \.index {
   private init(index: Int) { self.index = index }
   private let index: Int
   private let someOtherPropertyToThrowOffHashableSynthesis: Int
}

The gist of what I’m proposing is to allow, when adding a conformance to protocol P on type R, to specify a KeyPath<R, V> (where V conforms to P).

The type of the specified KeyPath should be enough to synthesize the static requirements (by forwarding them to V.self).

The value of KeyPath is used to access a member of type V (which is required to conform to P) from an instance of R, which should be enough to synthesize all the non-static requirements (by forwarding them to v).

In order to satisfy all init requirements, R must have the initializer init(v: V) (v is an external label and is replaced by the name of the property referred to by the KeyPath), which will be used to initialize R after an instance of V has been initialized and passed to init(v:).

All Self requirements should be translatable to either a call to that init (if Self is in a return-value position) or by applying the KeyPath (if Self is in a parameter position).

DISCLAIMER: Any new syntax described above is purely for illustration purposes and is highly bikesheddable.

Wow... This sounded a lot simpler in my head... :man_shrugging:t2::grin:

I see purge-by-default as simpler than include-everything-by-default. Remember that the list of protocol conformances is open-ended! Besides the protocols in the main documentation, authors can add implementation-helper protocols, and application developers can add conformances to local protocols. Making a finite white list is far more feasible than an open-ended black list.

Include-everything-by-default forbids using newtype as a way to either reset conformances or implement them differently.

Although we call Swift a protocol-oriented language, types handle their members as members, not as part of a protocol; protocol conformance is consolidated afterwards. (That's why we can mix up protocol conformance declarations and various required members among a type's primary definition and its extensions. Even using extensions that are declared after the type's protocol conformance) Forwarding should be done on a per-member basis, not per-protocol. We should have per-protocol forwarding only as a shortcut (if at all), not as a primary means.

I feel that forwarding is a nice-to-have, not necessarily a requirement. Actually, not necessarily an initial requirement. It should be there in some form by Beta 1, but it does not have to be in Alpha 1, because users can always manually write the trampoline members. The early alphas should focus on the parts of newtype that are core but have nothing to do with forwarding.

3 Likes

We already have a way to do that:

struct MyType { var x: Int }

If that’s all you want, then you already have it.

• • •

Also, I disagree with your statement that automatic forwarding forbids implementing conformances differently. I described earlier how it could allow one to write only the requirements that are different from the wrapped implementation, and forward all the rest.

Not really, this still lacks an easy way to do protocol forwarding to the inner value. Something like

newtype Foo: Equatable, Hashable = Int

would remove a lot of the friction.

3 Likes

To be fair, you can write:

struct MyType: Hashable { var x: Int }

with the same result.

Yeah, Hashable was a poor example due to the synthesis :slight_smile: Something like Collection or Codable would have been better.

5 Likes

I think we’re in agreement then: forwarding (especially protocol forwarding) is the important goal.

If the collective vision for implementation forwarding is restricted to protocols, then that still leaves us short when one wishes to make an exact and complete copy of an existing type, but distinct from it in the type system.

That’s the only real motivation I see for newtype.

On the other hand, if we design implementation forwarding to allow “forward everything”, then there’s no need for newtype at all.

3 Likes

I'm really keen on seeing this discussion happen and moving forward. I probably sent one of the earlier emails about the topic on the old mailing list! ^^

I think this is key. We often get focused on newtype but what we really want is systems for easily forwarding things to wrapped properties. The trick of dynamicMemberLookup and keypath are great, we should maybe keep expanding the language in that direction and we eventually would have all we need from a hypothetical newtype.

3 Likes