[Pitch] `@OptionSet` macro

Evolution Pitches
1

Hi Swift Evolution,

Here's a proposal for a macro to facilitate declaring option sets. Note this relies on SE-0389, currently in review. Proposal PR can be found here.

Option Set Declaration Macro

Introduction

This proposal introduces an attached macro @OptionSet which takes over the boilerplate required when declaring a bitfield-based option set.

Motivation

Implementing the standard library's OptionSet protocol, while not overly burdensome, does involve some fiddly boilerplate:

struct ShippingOptions: OptionSet {
  let rawValue: Int
  
  static let nextDay    = ShippingOptions(rawValue: 1 << 0)
  static let secondDay  = ShippingOptions(rawValue: 1 << 1)
  static let priority   = ShippingOptions(rawValue: 1 << 2)
  static let standard   = ShippingOptions(rawValue: 1 << 3)
}

Aside from the repetition, there is also a risk that a bit location be accidentally repeated:

  // oops, this should have been 1 << 4
  static let expedited   = ShippingOptions(rawValue: 1 << 3)

While this looks unlikely from the above code, in real code options often have long comment blocks above them describing their meaning, separating each case. This proposal was inspired by a bug reported to the authors caused by exactly this.

Here is a technique for preventing such errors:

struct ShippingOptions: OptionSet {
  let rawValue: Int
  
  private enum Options: Int {
    case nextDay, secondDay, priority, standard
  }
  
  static let nextDay    = ShippingOptions(rawValue: 1 << Options.nextDay.rawValue)
  static let secondDay  = ShippingOptions(rawValue: 1 << Options.secondDay.rawValue)
  static let priority   = ShippingOptions(rawValue: 1 << Options.priority.rawValue)
  static let standard   = ShippingOptions(rawValue: 1 << Options.standard.rawValue)
}

By using the compiler to generate the source of the raw values, the possibility of accidentally repeating a field number is eliminated. But it comes at the cost of even more repetitive boilerplate – which is also still a potential source of errors i.e.

// copy paste error – forgot to update the second occurrence
static let expedited = ShippingOptions(rawValue: 1 << Options.standard.rawValue))

Proposed solution

We propose the addition of an @OptionSet attached macro to the standard library:

@OptionSet
struct ShippingOptions {
    private enum Options: Int {
        case nextDay, secondDay, priority, standard
    }
}

The macro would then generate the remaining option set implementation, similar to the code seen in the example.

Detailed design

The above declaration would expand out to the following code:

struct ShippingOptions {
  typealias RawValue = Int

  var rawValue: RawValue

  init() { self.rawValue = 0 }

  init(rawValue: RawValue) { self.rawValue = rawValue }

  static let nextDay: Self =
    Self(rawValue: 1 << Options.nextDay.rawValue)

  static let secondDay: Self =
    Self(rawValue: 1 << Options.secondDay.rawValue)

  static let priority: Self =
    Self(rawValue: 1 << Options.priority.rawValue)

  static let standard: Self =
    Self(rawValue: 1 << Options.standard.rawValue)

  private enum Options: Int {
    case nextDay, secondDay, priority, standard
  }
}

extension ShippingOptions: OptionSet { }

This will require the addition of the @OptionSet macro declaration to the standard library:

@attached(member)
@attached(conformance)
public macro OptionSet<RawType>() = #externalMacro(module: "MacroExamplesPlugin", type: "OptionSetMacro")

If control over the raw value type is desired, the user can explicitly declare it:

  typealias RawValue = Int8

If the user wishes to explicitly choose the fields used, the enum's raw value can be specified explicitly:

  private enum Options: Int {
    case nextDay = 0
    case secondDay = 1
    // this bit is cursed, don't use it
    case priority = 3
    case standard = 4
  }

Existing language functionality prevents field numbers from being re-used in this case.

Users of the macro are free to include additional computed properties such as:

  static let express: ShippingOptions = [.nextDay, .secondDay]

If the user fails to declare an inner enum named Options, or fails to give it a raw type of Int, they will receive an error from the macro asking for it to be declared. An argument can also be supplied to the macro if the enum has a different name i.e. @OptionSet("MyOptions").

The static option properties will have the same access modifier as the option set struct.

This macro may only be applied to structs, and must be applied to the struct's declaration.

Source compatibility

None.

ABI compatibility

None. Authors of ABI-stable libraries should take care to ensure, if simplifying existing option sets, that the resulting type has the same ABI as before.

Implications on adoption

This macro will only be available when compiling with a new enough toolchain. Source code that needs to compile with earlier toolchains must continue to conform explicitly.

Alternatives considered

Other than doing nothing, an alternative to this proposal could be to add true native support for option sets to the language:

optionset ShippingOptions {
  case nextDay, secondDay, priority, standard
  
  static let express: ShippingOptions = [.nextDay, .secondDay]
}

Other than a slight further decrease in verbosity, there is no meaningful benefit to this approach over the use of a macro.

38 Likes
2

I think it has already been reported elsewhere, but IIRC there is no #if that lets us check for the version of the standard library (not to be mismatched with the version of the compiler). This makes it difficult to use such features in code intended to compile on multiple toolchains (i.e. libraries).

Edit: I found a previous thread on the topic: Introduce `#if stdlib` or clarify `#if swift`

6 Likes
3

At the same time, if it’s missing, your only option is to write the long form. If you’re going to conditionally compile either the use of the macro or a manual implementation of the option set, you might as well keep only the manual implementation.

3 Likes
4

The trouble is there is no good way to detect if the newer standard library is in the SDK yet as it isn’t the same as the compiler. But while there aren’t good ways, there are alternatives (including just waiting). Let’s not turn this into a discussion of that aspect though, there is already a thread on this.

2 Likes
5

Isn't for this particular pitch only the version of the stdlib with the compiler the deciding factor because it ships the precompiled macro implementation?

6

Macros still require a declaration in the library.

3 Likes
7

Thank you for this section. It helps us understand the kind of ergonomics we can expect from macros. They look good in this particular case.

There is one caveat:

This macro [...] must be applied to the struct's declaration.

I think I can understand it: the macro needs to inspect the content of the decorated type - or rather, the syntax enclosed between { and }.

Please correct me if I'm wrong! I guess this would not work:

// Does not work, or does it?
@OptionSet
struct ShippingOptions { }

extension ShippingOptions {
  private enum Options: Int {
    case nextDay, secondDay, priority, standard
  }
}

extension ShippingOptions {
  // Other @OptionSet configuration

  typealias RawValue = ...
}

If this is correct, if the type has to be fully defined "in one stroke", then the pitch assumes that users of @OptionSet will never want to split the definition of the decorated type in several extensions.

I think this should be written black on white in the pitch text, because it could ring some bells in some reviewers' minds.

What I mean is that compiler-based features were able to process a whole file, when macros are not. This "regression" in ergonomics may surprise some users. And by "ergonomics", I do not mean the freedom to write code as one wants. I mean the ability to run Xcode plugins, code generators, guard some features or configure them with #if or other conditionals, etc.

// The kind of ergonomics to which
// compiler-based features have accustomed us.
import Foundation

struct Player {
  var name: String
  var score: Int
}

// Separate addition of Encodable
extension Player: Encodable { }

// Separate configuration of Encodable
extension Player {
  enum CodingKeys: String, CodingKey {
    case name = "player_name"
  }
}

// Still works as expected:
// Prints {"player_name":"Alice"}
let player = Player(name: "Alice", score: 1000)
try print(String(data: JSONEncoder().encode(player), encoding: .utf8)!)
7 Likes
8

Interesting. I haven't thought about this for the stdlib but that kinda goes into the same directions as @backDeployed where the declaration is almost trivial so if the stdlib provides the implementation already couldn't we back deploy the declaration.

1 Like
9

I like the alternative form - that would make option sets first class citizen. These shorter names also worth considering "options", "flags".

I assume it would be possible to specify the raw type for optionset and there'll be some sanity checking on the explicit constant values (if specified) – those can't be negative or too big for the raw type or repeat.

Please note however, that both the pitched and alternative versions do not support everything that could be done using today's form. Example:

struct SomeOptions: OptionSet {
    let rawValue: Int
    
    static let first = SomeOptions(rawValue: 0b00000001)
    static let second = SomeOptions(rawValue: 0b00000010)
    static let third = SomeOptions(rawValue: 0b01000001)
    static let fourth = SomeOptions(rawValue: 0b01000100)
}

in this example the only options you could specify are 00000001, 00000010, 01000001 and 01000100, and you are not supposed to specify 01000000 or 00000100 options on their own.

10

You would be able to support that just fine, you just need to set the rawValue of the relevant cases in the private Options enum that's declared to support the macro.

Speaking of which, what's the UX when we don't meet the macro requires menus. Say, perhaps we spell Options as Option (which seems correct :slight_smile:), what output do we get?

1 Like
11

I do not find the syntax particularly pleasing (the three occurrences of the word Option, the enum within the struct).

I take it the following could not be made to work?

enum ShippingOptions: Int, OptionSet {
    case nextDay, secondDay, priority, standard
}
4 Likes
12

I think the alternate form is better, as then there's more symmetry with enums. For a beginner to Swift they might wonder why there is a difference for two fairly similar concepts.

Also the macro form, as I understand it, doesn't leave room to allow for associated values to be added to OptionSets.

13

Huge nit: The Options enum should be called just Option (singular) because its value only ever represents a single option not several of them. The type that gets the OptionSet conformance can have a plural naming, that's totally fine. I find that we should not use the precedence of the plural CodingKeys enum from Codable structures. Each Option has a rawValue explicit or implicit, and it's not rawValues (plural) like an Options type should theoretically have.

21 Likes
14

The original implementation has the OptionSet parts filled in by the macro. Yours would need the enum declaration transformed into one that works for an OptionSet, since you'd need to change the Int value synthesis and remove the conflict between the cases and options that are desired. So while your example could be clearer initially, it actually seems like a larger transform.

To make the connection between the declaration and the synthesized options more clear, and allow for flexibility, could we add a parameter to @OptionSet? @OptionSet(from: Options) or something like that would give the user flexibility to provide the name they way and allow an obvious place to attach a name diagnostic.

1 Like
15

Why is making option sets "a first class citizen" a goal?

Normally when that term is used, it is because a library implementation is deficient in some way. For example, it would be infeasible to implement concurrency or closures as well in a Swift library than they are supported in the language. But you need to show examples of this. Personally, I don't see the slight syntactic improvement as clearing that bar.

I don't think that's quite what @tera was describing. In their example, third also sets first. You could not do that with this proposal.

However, I am a bit skeptical about whether that is a good practice. More normally, you would create a computed property for that, in addition to the real third option which has a value of 0b01000000.

Of course, the alternative if you believe the code should appear exactly as written is to implement it in full by hand. A macro solution remains better than a built-in language feature, because macros can be expanded into the code via a refactoring action, after which you can then take that expanded code and tweak it (same with synthesized conformances, which in this future world would probably be implemented as macros).

1 Like
16

That might be a rationale, but it looks pretty unpleasant as a singular noun, which far outweighs a technical argument for making it so in my mind.

2 Likes
17

Interestingly, the macro implementation linked actually does parse the name from the macro arguments list, so this does appear to work, there was just no example.

2 Likes
18

Is it however the only logically correct answer to this problem. CodingKeys (plural) only exists because we wanted to avoid the collision with the Swift.CodingKey (singular) protocol and no one wanted the regular user to type enum CodingKey: Swift.CodingKey to satisfy the compiler. We don't seem to have this issue here, so logically this should be Option and not Options here.

enum Fruit { // this is clearly not `Fruits`
  case 🍎, 🍐
}
8 Likes
19

Would it be appropriate to suggest different diagnostic text here or on the PR?

20

More than that. It also adds a stored property, something that cannot be done in an extension (being able to do so in the same file has been pitched... personally I'm against such a change, see that thread for details).

Now, macros could potentially support going back to the definition if it's in the same file and adding things. But it still doesn't seem like a good idea to add properties from an extension, even via a macro.

3 Likes