[Pitch] `@OptionSet` macro

[taylorswift]

@itaiferber

The name CodingKeys was intentionally chosen as a plural name; I won't re-hash the explanation here since it's available on the forums, but the enum doesn't follow what you might consider "regular" naming rules because it exists primarily as a namespace.

Whether or not the same logic applies here, I'll leave for others to have strong opinions on. (Though I personally believe it could.)

Option is different from CodingKeys because you almost never want to bind a variable with type CodingKeys, but it is quite common to want to work with singular Option flags. when i see Options i have a hard time remembering that it is modeling one single option flag and not an entire option set. i personally follow a convention where i use plural type names for things that are "collections" or "sequences", and Options is not a collection of its cases.

(also, let us not hold up Codable as a paragon of great API design.)

[Jon_Shier]

Does the macro need to handle the case where the user has provided more cases than can fit in the bits of the indicated raw type? It also seems that the macro only checks for a raw type but not that the type is one of the integer types, shouldn't it check? (I could also be missing that part.)

[Douglas_Gregor]

@Joe_Groff

An approach like this also lets developers control access to individual options, as in:

@OptionSet public struct ShippingOptions {
  @Option public static var nextDay, secondDay, priority, standard

  @Option private static var topSecretTeleportation
}

which doesn't seem possible using a separate type to derive the options.

Going down this path, you could have @OptionSet infer the @Option for any static var that has the appropriate type (Self or ShippingOptions) and neither an initializer nor accessors. So the minimum is:

@OptionSet public struct ShippingOptions {
  public static var nextDay, secondDay, priority, standard: ShippingOptions

  private static var topSecretTeleportation: ShippingOptions  
}

I like that it's a smaller "diff" for the macro expansion, which is filling in specific details (e.g., adding getters, adding rawValue) rather than creating redundant-feeling static variables.

I think it's a little harder to implement the macro itself, because we would want to do some basic analysis of the various properties that have initializers to make sure they don't conflict. But I think that's doable with the available information.

Doug

[rauhul (Rauhul Varma)]

@Joe_Groff

where the @Option annotation takes care of doling out raw values for the options, and choosing the appropriate underlying bits type? That seems to me like it gets fairly close to your "first-class optionset declaration", while still also being less indirect than the proposed behavior, since the author of the code is still directly declaring the static vars that make up the API of the type. An approach like this also lets developers control access to individual options, as in:

Without commenting on the idea or pitch directly, @Option would collide with Swift-Argument-Parser's @Option property wrapper in an unfortunate way. Do macros from the Swift module have lower overload resolution precedence than wrappers/macros from other modules?

[Joe_Groff]

@rauhul

Without commenting on the idea or pitch directly, @Option would collide with Swift-Argument-Parser's @Option property wrapper in an unfortunate way. Do macros from the Swift module have lower overload resolution precedence than wrappers/macros from other modules?

I don't have a particular opinion about the names. The standard library does have lower precedence than user-defined modules, but you can also write @Foo.Options to qualify a reference.

[mickeyl (Dr. Mickey Lauer)]

I‘m afraid, I‘m -1 on that. It doesn‘t feel „Swifty“ at all. Readers will be left pretty clueless about that until they’re peeking in the macro source. (Offtopic: I hope there are more convincing use cases for macros.)

[taylorswift]

there are essentially no functioning refactoring/expansion features implemented in VSCode for swift today, even the symbol indexing does not work very reliably right now. the compiler diagnostic tooltips do not work either; they always redirect you back to the file containing the error instead of the file referenced by the diagnostic.

i still pretty much just use the extension as a semantic syntax highlighter, on a good day maybe the Ctrl-click to definition will work as well.

i do not know what things are like in XCode as i do not use XCode. but in the linux domain, i do not find the "tooling will solve the readability issue" argument convincing.

[Douglas_Gregor]

@8675309

Compared to the alternative: imminently readable, no indirection, no "magic" public types.

optionset ShippingOptions {
  case nextDay, secondDay, priority, standard
}

What is the criteria for choosing a macro here in the std library? Why not implement enums as structs with a macro too?

Enums represent a fundamentally different thing than structs---they are a choice among possibilities ("sum" type in programming language parlance) vs. an aggregation of other values ("product" type).

Option sets are not fundamentally different from structs, and there is a clear pattern one can use to describe them as a structs. Adding new kinds of nominal types is exceedingly rare, and should only be done when there is a deep semantic difference that must be captured in the type system. We have added one new nominal type kind (actor) since the creation of Swift, and that was justified because their isolation semantics are deeply entwined in the language. Option sets will never cross that threshold, and a macro is the best (probably only) way to improve on them.

Doug

[8675309]

Thanks for the explanation. I find that this perspective comes as a language designer rather than a language user. Are the details of sum type vs product type particularly salient to the user?

[Douglas_Gregor]

@8675309

Thanks for the explanation. I find that this perspective comes as a language designer rather than a language user. Are the details of sum type vs product type particularly salient to the user?

It's the fundamental difference between an enum and a struct. Yes, that's salient to everyone.

Doug

[mayoff (Rob Mayoff)]

@8675309

Are the details of sum type vs product type particularly salient to the user?

The details are critical if you try to follow the ‘make illegal states unrepresentable’ philosophy for reducing bugs. Here are some helpful links if you're not familiar with it and want to learn:

Making illegal states unrepresentable

Designing with types: Making illegal states unrepresentable | F# for fun and...

"Making Impossible States Impossible" by Richard Feldman

Making Illegal States Unrepresentable

[8675309]

@Douglas_Gregor

It's the fundamental difference between an enum and a struct. Yes, that's salient to everyone.

Would an end user ever say to themselves, "let's reach for a product type here"?

[tera]

@Douglas_Gregor

Option sets are not fundamentally different from structs, and there is a clear pattern one can use to describe them as a structs. Adding new kinds of nominal types is exceedingly rare, and should only be done when there is a deep semantic difference that must be captured in the type system.

There could've been this nice symmetry though if we went down the route of introducing a new type:

Dictionary ← → struct
Array ← → C-style arrays (int x[100])
Set ← → option set

On the lefthand side those are dynamic normally heap allocated objects, on the righthand side - more restricted typically fixed sized and faster "stack" or "in place" allocated objects.

[thomasvl (Thomas Van Lenten)]

This is likely some what of a tangent, but…

With the original proposal, how would doing DocC comments end up working? Would comments from the private enum get relayed along by the macro so the generated api that developers will use for the OptionSet would appear in documentation?

How about for @OptionSet/@Option idea where individual visibility could be done?

[allevato (Tony Allevato)]

@thomasvl

With the original proposal, how would doing DocC comments end up working? Would comments from the private enum get relayed along by the macro so the generated api that developers will use for the OptionSet would appear in documentation?

DocC uses symbol graphs extracted from the module, which (I believe) would have any doc comments that are provided by the macro implementation after expansion. So, that's another reason that @Douglas_Gregor's suggestion of basing this off original declarations would be a benefit, because you could write this:

@Douglas_Gregor

@OptionSet public struct ShippingOptions {
  /// Doc about nextDay
  public static var nextDay: ShippingOptions

  /// Doc about secondDay
  public static var secondDay: ShippingOptions

  /// Doc about priority
  public static var priority: ShippingOptions

  /// Doc about standard
  public static var standard: ShippingOptions

  private static var topSecretTeleportation: ShippingOptions  
}

and the doc comments of the original declarations would be preserved, even after the macro expands initializers into them. This would be a lot easier for readers to understand vs. having a separate enum that needs to have the comments copied over onto the new declarations.

[tgrapperon (Thomas Grapperon)]

What happens if the number of cases in the enum exceeds the number of bits available for the OptionSet's rawValue? Would it overflow at runtime?
(I'm able to build the example with more cases than bits, but I'm not able to launch it for some reason, so this is an honest question).

[Ben_Cohen]

@thomasvl

Would comments from the private enum get relayed along by the macro so the generated api that developers will use for the OptionSet would appear in documentation?

Good catch. Yes, that is how it should work. Will add this to the proposal doc.

[bbrk24]

I just tried this:

let i = 1 << 65
print(i)

I expected to get some kind of diagnostic about this, akin to how Int.max + 1 does, but instead it just accepted it and printed 0. That's probably not what we want here.

[bzamayo (Benjamin Mayo)]

Great use of macros! I’m all for this being shipped in stdlib.

@Douglas_Gregor

I like that it's a smaller "diff" for the macro expansion, which is filling in specific details (e.g., adding getters, adding rawValue) rather than creating redundant-feeling static variables.

FWIW, when I saw the thread title on the forum homepage, I assumed this was how it would work. It feels like a more intuitive construction. So it’d be great if you can make the implementation work for that.

[Ben_Cohen]

@tgrapperon

What happens if the number of cases in the enum exceeds the number of bits available for the OptionSet's rawValue? Would it overflow at runtime?

This is an interesting question. Today, this is another source of bugs:

struct SmolSet: OptionSet {
  var rawValue: Int8
  // much later
  // oops, this has rawValue of 0
  static var big = Self(rawValue: 1 << 8)
}

The macro could potentially do better by counting the number of options. This could work great for small sets like with rawValue: Int8. However, this runs afoul in the case of rawValue: Int, because Int's size is dependent on the target, and the macro will not be running on the target, so can't know if it's 32-bit or 64-bit. Though it could at least cover the case of more than 64 options.