Pitch: Static and class subscripts

Evolution Pitches
,
1

Over the last few days I've put together a prototype of static subscripts, which allow you to add subscripts that are applied to a type, rather than an instance. There's a lot of work to be done, but so far, it's looking pretty good; I've even gotten dynamic member lookup and key paths working (although they're a stretch goal and I might defer them before the final proposal). I think they would be a good addition to the language.

Prototype toolchains are available macOS and Linux. I've only tried very basic things so far, so if you run into trouble using key paths to generic static subscripts of generic subclasses of resilient Objective-C framework classes or something, please let me know.

(And yes, I think these features could work with key path member lookup, although I haven't tried it.)

Draft proposal follows:

Static and class subscripts

Introduction

We propose allowing static subscript and, in classes, class subscript declarations. These could be used through either TypeName[index] or TypeName.self[index] and would have all of the capabilities you would expect of a subscript. We also propose extending dynamic member lookup and key paths to static properties by using static subscripts.

Swift-evolution thread: Static Subscript (2016), This thread you're reading right now

Motivation

Subscripts have a unique and useful combination of features. Like functions, they can take arguments to change their behavior and generic parameters to support many types; like properties, they are permitted as lvalues so their results can be set, modified, and passed as inout. This is a powerful feature set, which is why they are used for features like key paths and @dynamicMemberLookup.

Unfortunately, unlike functions and properties, Swift only supports subscripts on regular types, not metatypes. This prevents us from supporting key paths, dynamic members, or any other design requiring a subscript's feature set on metatypes.

Wait, what the heck is a "metatype"?

A type like Int has many instances, like 0 and -42. But Swift also creates a special instance > representing the Int type itself, as opposed to any specific instance of the Int type. This special instance can be directly accessed by writing Int.self; it is also returned by type(of:) and used in various other places. In fact, static members of Int are instance members of Int.self, so you use it any time you call one of those.

Since Int.self is an instance, it must have a type, but the type of Int.self is not Int; after all, Int.self cannot do the things an Int can do, like arithmetic and comparison. Instead, Int.self is an instance of the type Int.Type. Because Int.Type is the type of a type, it is called a "metatype".

The lack of static subscripts also limits the expressivity of Swift code that uses metatypes. For instance, Ruby's equivalent feature, the self.[] method, is used by types such as WEBrick::HTTPStatus (taking a numeric HTTP status code) and RDoc::I18n::Locale (taking a locale identifier) to represent lookup-or-create operations. It would be very plausible for equivalent Swift types to adopt similar interfaces:

let posixLocale = Locale["en_US_POSIX"]
let status = HTTPStatus[404]

To emulate this today, you would need to create a helper type with an instance subscript, add a static property of that type, and reference the static property on every use. But nobody does this; instead, they use an initializer or, if they can't, a static method.

let posixLocale = Locale(identifier: "en_US_POSIX")    // Not lookup-y.
let status = HTTPStatus.forCode(404)                   // Gross.

Swift originally omitted static subscripts for a good reason: They conflicted with an early sugar syntax for arrays, Element[]. But we have long since changed that syntax to [Element] and we aren't going back. There is no longer a technical obstacle to supporting them, and there never was a philosophical one. The only obstacle to this feature is inertia.

It's time we gave it a little push.

Proposed solution

In any place where it was previously legal to declare a subscript, it will now be legal to declare a static subscript as well. In classes it will also be legal to declare a class subscript.

extension Identifier {
  public static subscript(_ value: String) -> Identifier {
    return Identifier(value)
  }
}

The static and class subscripts on a type T can be used on any expression of type T.Type, including T.self[...] and plain T[...].

let identType = Identifier.self
let readIdent = identType["read"]          // OK

let openIdent = Identifier.self["open"]    // OK
let closeIdent = Identifier["close"]       // OK

A static subscript with the parameter label dynamicMember can also be used to look up static properties on types marked with @dynamicMemberLookup.


@dynamicMemberLookup struct Dyn {
  static subscript(dynamicMember name: String) -> String {
    return "type \(name)"
  }
  subscript(dynamicMember name: String) -> String {
    return "instance \(name)"
  }
}

print(Dyn.foo)      // => "type foo"
print(Dyn().foo)    // => "instance foo"

Finally, it will be possible to form key paths rooted in or passing through metatypes, and to look up metatype-rooted key paths using a static subscript with the label keyPath.

let tableNameProperty = \Record.Type.tableName
Person[keyPath: tableNameProperty]

Detailed design

Static subscripts

Static subscripts can be declared in classes, enums, structs, and protocols, as well as extensions thereof; class subscripts can be declared in classes and extensions of classes. In classes, static subscripts are implicitly final, but class subscripts are not.

Static and class subscripts will support all relevant features that instance subscripts do, including accessors, multiple parameters, labeled parameters, overloads, and generics. Since metatypes are reference types, static subscript accessors will not support the mutating and nonmutating keywords; this is the same behavior seen in classes and static property accessors.

Objective-C class methods with the same names as its subscript methods (like +objectAtIndexedSubscript:) will not be imported to Swift as class subscripts; Objective-C technically allows them but doesn't make them usable in practice, so this is no worse than the native experience. Likewise, it will be an error to mark a static or class subscript with @objc.

Dynamic member lookup

@dynamicMemberLookup can be applied to any type with an appropriate subscript(dynamicMember:) or static subscript(dynamicMember:) (or class subscript(dynamicMember:), of course). If subscript(dynamicMember:) is present, it will be used to find instance members; if static subscript(dynamicMember:) is present, it will be used to find static members. A type can provide both or either one.

Key paths

It will be possible to form and use key paths involving static properties and subscripts. If a key path starts in a metatype, its Root type will be Foo.Type, where Foo is the type it's on. Settable key paths will always be ReferenceWritableKeyPaths, since metatypes are reference types. We don't believe this feature will require any runtime changes.

Implementation notes

This feature worked remarkably well from the beginning; we expect the bulk of the work to be writing tests to shake out subtle assumptions throughout the compiler.

Source compatibility

This proposal is purely additive; it does not change any prevously existing behavior. All syntax it will add support for was previously illegal.

ABI compatibility and backwards deployment

Static subscripts are an additive change to the ABI. They do not require any runtime support; the Swift 5.0 runtime should even demangle their names correctly. Dynamic member lookup is implemented in the type checker and has no backwards deployment concerns.

Metatype key paths will always use accessors; doing this will ensure that they backwards deploy to Swift 5.0 correctly. Swift 5.0 code should also be able to access values through metatype key paths and otherwise use them without noticing the difference.

We anticipate one fly in the ointment: The Equatable and Hashable conformances for key paths may return incorrect results for static properties in resilient libraries built with the Swift 5.0 compiler. That is, if an app or library compiled with Swift 5.1 forms a key path to a static property in a system framework, the == operator may return incorrect results on machines running macOS 10.14.4/iOS 12.2/etc. or older. (This is because Swift 5.0 didn't emit resilient descriptors for static properties, so there is no agreed-upon stable identifier for == to use.) We think the impact of this issue will be tolerable and it can simply be documented.

Effect on API resilience

The rules for the resilience of static and class subscripts will be the same as the rules of their instance subscript equivalents.

Alternatives considered

Leave our options open

The main alternative is to defer this feature again, leaving this syntax unused and potentially available for some other purpose.

The most compelling suggestion we've seen is using Element[n] as type sugar for fixed-size arrays, but we don't think we would want to do that anyway. If fixed-size arrays need a sugar at all, we would want one that looked like an extension of the existing Array sugar, like [Element * n]. We can't really think of any other possibilities, so we feel confident that we won't want the syntax back in a future version of Swift.

46 Likes
2

Very cool! Nice to see this long-standing "hole" being filled, and I'm thrilled that the implementation side went so smoothly.

Doug

5 Likes
3

I agree, this would be a great addition. It fills an obvious gap in the language that I've hit a few times now.

1 Like
4

Very well written pitch. It seems like a natural addition.

(Minor note: you have a typo in "… since metatypes are referecne types …")

5

+1 :grin:

2 Likes
6

@Douglas_Gregor @beccadax
I always wanted to write static subscripts here and there but then I always said to myself "the core team decided to not include static subscripts and reserve these for potential other language features that require special syntax on types instead of allowing them to be used the same way as non-static subscripts". Is this no longer the case?

7

Is this part of the current proposal or an implication of it? IIRC no-one has implemented the keypath extension to allow them to walk on metatypes.

8

Forming metatype key paths is part of the proposal. (But key paths are not the primary goal of this proposal; if they end up being too much work to finish while getting static subscripts working, I'll defer them.)

Once you remove the error preventing the compiler from trying to form metatype key paths, you basically just need to force SILGen to always generate accessor-based key path components for them, and then you need to shake out bugs in the existing code and fix them. There isn't really any fundamental obstacle—the work just hasn't been done, largely because you couldn't have used them anyway without static subscripts.

3 Likes
9

I don't think that was ever the case. My recollection of the history is as @beccadax wrote it in the pitch: when we introduced subscripts, array types were still written Int[], so a zero-argument static subscript would have been ambiguous with an array type within expression context. Array syntax changed in an early beta of Swift 1.0, but we never got back to static subscripts.

Doug

10 Likes
10

Great, thank you both for clarifying that to me. Huge +1 nevertheless.

11

I think the inclusion of static subscripts could motivate static static callables or even static dynamic callables, except that those syntax sugars will collide with the mental model of the initializer syntax.

3 Likes
12

Nit regarding this specific example, I would not include it in a proposal:

  1. This is not just a lookup, so it better not look like one. The identifier is normalised so that also the standard BCP47 format, which uses a dash instead of an underscore, results in the same locale: Locale(identifier: "en_US") vs Locale(identifier: "en-US"). https://github.com/apple/swift-corelibs-foundation/blob/master/CoreFoundation/Locale.subproj/CFLocale.c for the curious.
  2. It is not a great API in the first place, it would be better if there was a locale identifier type

I do not like this example much either, because since HTTP allows custom status codes, this should be a construction rather than a lookup, eg:

HTTPStatus(code: 404)
HTTPStatus(code: 418, message: "I'm a teapot")
11 Likes
13

I agree with the post above 100%. I'm supportive of the feature overall because it makes metatypes work like any other types, removing an inconsistency in the language. But the specific examples here feel a bit artificial and I don't think they "sell" the pitch effectively.

One thing to keep in mind is that subscript APIs are very undiscoverable in terms of things like autocomplete right now. We naturally use them for collections because of decades of programming experience, but my gut feeling is that nobody would expect to subscript a Locale or an HTTPStatus to get a specific instance of them. Instead, they'd start typing TypeName followed by a dot to pop up the list of static members, or a ( to look at the initializer overloads. It would take some mental retraining to have Swift users consider adding [ to that list. That's by no means impossible, but it's something that should be factored in when considering specific APIs that would use this feature. Tooling improvements would help here as well.

Off the top of my head, a use case that might better convey how subscripts would apply to metatypes would be some kind of "instance cache" on the type, where you can store and retrieve instances of the type given some key. That can be done today with static methods but maybe the subscript feels more natural in that case? It's a lookup but it's not as heavyweight as the Locale example. That sort of usage could open up some potentially interesting dependency injection patterns, as well.

6 Likes
14

This looks really great, thank you for tackling this @beccadax!

There are reasons to include static static callable independently of aligning with static subscript, but I agree that this pitch strengthens the case (in terms of maintaining consistency).

In this model, from the perspective of a caller, an initializer behaves exactly like a static call that returns Self. I don't see how this is a collision at all. Initializers just have special declaration syntax associated with special DI rules for the implementation of the body. I think we could just say init implies static call with a return type of Self (or Self? for failable initializers).

1 Like
15

I'd say this suggests we should add subscripts to code completion even when you've typed a dot. We know how to do code completions that require changing existing code already. I think.

13 Likes
16

Agreed—especially since initializers already appear there even though explicit .init isn't the "preferred" way of calling them.

We should do likewise for call (or whatever the final name ends up being), assuming the static callables proposal is accepted, and I'd argue that we might even want to extend this to operators?

1 Like
17

Operators are tricky because it's harder to tell what code needs to change to call them successfully (because of precedence). For init and call, I also think it makes sense for code completion to leave the name in there (maybe you're not invoking them right away), whereas for subscripts the only thing you can use is the [] syntax.

2 Likes
18

I agree with this too. Most of the motivating examples look like they should be initializers. An example of something like a DateFormatter cache would be more useful.

1 Like
19

Hmm. Maybe I should bring back the Identifier example alluded to in "proposed solution"—it was basically a table of interned strings, based on a (C++) type in the Swift compiler.

20

It may also be a matter of unlabeled subscripts looking a lot like unlabeled initializers.