[Pitch] `@OptionSet` macro

The OptionSet macro could detect the presence of this stored property and produce an error. It's a couple of lines of code in the macro implementation. Even if that were not true, the possibility that someone could write such a thing and get confused would not motivate the addition of a new feature.

I don't know how to be anything but blunt here: there is no path where Swift gets a new kind of nominal type for option sets. They do not, and will not ever, meet the criteria for addition into the language. If you want to continue this discussion, please do so in a separate thread, where I will not be participating.

Please let this thread focus on how best to use macros to eliminate boilerplate for such a case. If the end result is not good enough, not compelling enough, then we should reject it then. But it won't be in favor of a language feature.

Doug

6 Likes

+1 on using macros to synthesise options. As many have pointed out OptionSet brings pitfalls and synthesis removes much of the subtlety.

I am drawn to using static var for the options over a nested enum Options: Int because as mentioned the macro expansion diff is smaller which is a usability win and should not be understated.

It may take some time for me to get used to, but I much prefer when the code I am reading explicitly states the protocol conformance — generic constraints and protocol conformance already have a steep learning curve in Swift and using macros like @OptionSet makes this even steeper.

struct ShippingOptions: OptionSet { } // explicit and clear

@OptionSet
struct ShippingOptions { } // yes tooling can expand but this is an additional abstraction

I would be hesitant to support @OptionSet so quickly in this form to the standard library without some discussion on these forums about the feasibility of witness macros mentioned in the possible vision.

2 Likes

For a project of mine, instead of using OptionSet I built a BitwiseSet type that can be used in this way:

enum Side: Int {
    case left, right, bottom, top
}
typealias Sides = BitwiseSet<Side>

// in usage:
let sides: Sides = [.left, .right]

I've been pretty happy with this.

In my opinion, this OptionSet macro feels more heavyweight than necessary (both in conceptual complexity and declaration syntax), and has an uglier declaration syntax. The equivalent option-set to the above would look like this:

@OptionSet
struct Sides {
    enum Options: Int {
        case left, right, bottom, top
    }
}

// in usage:
let sides: Sides = [.left, .right]

The only advantage of OptionSet I can see is that the type of the set is the same type as its value, allowing you to omit the [] when there's a single value:

let sides: Sides = .left // no [] only works with OptionSet

Although honestly I'm not sure if this is an advantage or a fault. (I wonder now if this could be imitated for BitwiseSet with some key path forwarding trickery.)

On the negative, with @OptionSet the enum has to be nested in the @OptionSet type and has a rigid name. Most of the time I'd rather use an external enum. I suppose an external enum could be "typealised" inside of the @OptionSet type (will the macro allow that?), but that's just more boilerplate.


BitwiseSet implementation

/// BitwiseSet is a set of values stored as a bit mask. Elements in the set
/// must be RawRepresentable with an Int as a RawValue. Typically, the element 
/// type is an enum based on Int:
///
///     enum State: Int {
///         case closed
///         case open
///         case mixed
///     }
///     var validStates: BitwiseSet<State> = [.open, .closed]
///
/// Since the storage for BitwiseSet is an Int, raw values of its element
/// must not exceed the number of bits in an Int. So on a 64-bit environment,
/// the valid range of raw values for its elements are `0 ..< 64`, and with 
/// 32-bit it is `0 ..< 32`.
public struct BitwiseSet<Element>: SetAlgebra, Sequence, RawRepresentable, Hashable where Element: RawRepresentable, Element: Equatable, Element.RawValue == Int {
	public var rawValue: Int
	public init() {
		rawValue = 0
	}
	public init(rawValue: Int) {
		self.rawValue = rawValue
	}

	/// Initialize the set with one optional element, resulting either in a
	/// 
	/// single-member set or the empty set (if `nil`).
	public init(_ member: Element?) {
		self.rawValue = BitwiseSet.rawValue(for: member)
	}

	public var first: Element? {
		return first(where: { _ in true })
	}

	public func intersects(_ other: BitwiseSet) -> Bool {
		return rawValue & other.rawValue != 0
	}
	public func union(_ other: BitwiseSet) -> BitwiseSet {
		return BitwiseSet(rawValue: rawValue | other.rawValue)
	}
	public func intersection(_ other: BitwiseSet) -> BitwiseSet {
		return BitwiseSet(rawValue: rawValue & other.rawValue)
	}
	public func symmetricDifference(_ other: BitwiseSet) -> BitwiseSet {
		return BitwiseSet(rawValue: rawValue ^ other.rawValue)
	}
	public mutating func formUnion(_ other: BitwiseSet) {
		rawValue |= other.rawValue
	}
	public mutating func formIntersection(_ other: BitwiseSet) {
		rawValue &= other.rawValue
	}
	public mutating func formSymmetricDifference(_ other: BitwiseSet) {
		rawValue ^= other.rawValue
	}

	public func contains(_ member: Element) -> Bool {
		return rawValue & BitwiseSet.rawValue(for: member) != 0
	}
	@discardableResult
	public mutating func insert(_ newMember: Element) -> (inserted: Bool, memberAfterInsert: Element) {
		let oldRawValue = rawValue
		rawValue |= BitwiseSet.rawValue(for: newMember)
		return (oldRawValue == rawValue, newMember)
	}
	@discardableResult
	public mutating func remove(_ member: Element) -> Element? {
		let oldRawValue = rawValue
		rawValue &= ~BitwiseSet.rawValue(for: member)
		return oldRawValue == rawValue ? nil : member
	}
	@discardableResult
	public mutating func update(with newMember: Element) -> Element? {
		let oldRawValue = rawValue
		rawValue |= BitwiseSet.rawValue(for: newMember)
		return oldRawValue == rawValue ? newMember : nil
	}

	/// The range of valid raw values for elements. This is determined by the 
	/// bit size of Int. On a 64-bit architecture, the range is `0 ..< 64`.
	public static var supportedRangeOfMemberRawValues: CountableRange<Int> {
		return 0 ..< MemoryLayout<RawValue>.size * 8
	}
	/// The raw value to use for storing the given element. This is
	/// simply `1 << member.rawValue`. `member.rawValue` must be inside of
	/// `supportedRangeOfMemberRawValues`.
	public static func rawValue(for member: Element?) -> Int {
		guard let member = member else { return 0 }
		assert(supportedRangeOfMemberRawValues.contains(member.rawValue), "BitwiseSet is limited to elements having a raw value in the range \(supportedRangeOfMemberRawValues) (dependent on the current architecture). Value \(member.rawValue) for \(member) is out of range.")
		return 1 << member.rawValue
	}

	/// The set that contains all possible elements values.
	/// - Note: Implemented by attempting to create an element with
	///         `init?(rawValue:)` from all supported raw values in
	///         `supportedRangeOfMemberRawValues`, adding non-nil elements into
	///         the set. This is not efficient if `init?(rawValue:)` is not.
	///         Also, there is no cache.
	public static var all: BitwiseSet {
		var all = BitwiseSet()
		for elementRawValue in BitwiseSet.supportedRangeOfMemberRawValues {
			if let element = Element(rawValue: elementRawValue) {
				all.insert(element)
			}
		}
		return all
	}

	// MARK: Sequence

	public struct Iterator: IteratorProtocol {
		fileprivate var index = 0
		fileprivate var remainingSet: BitwiseSet
		fileprivate init(_ set: BitwiseSet) { remainingSet = set }
		mutating public func next() -> Element? {
			while !remainingSet.isEmpty {
				defer { index += 1 }
				if let element = Element(rawValue: index), remainingSet.contains(element) {
					remainingSet.remove(element)
					return element
				}
			}
			return nil
		}
	}

	public func makeIterator() -> Iterator {
		return Iterator(self)
	}

}

extension BitwiseSet: CustomStringConvertible {

	public var description: String {
		return "[\(map { "\($0)" }.joined(separator: ", "))]"
	}

}

6 Likes

I like this very much. Combining this with the idea of @ricketson above we may have:

public protocol HasOptionSet: Equatable, CaseIterable, RawRepresentable where RawValue == Int {
    typealias OptionSet = BitwiseSet<Self>
}

that'll get to this final usage:

enum Side: Int, HasOptionSet {
    case left, right, bottom, top
}

let sides: Side.OptionSet = [.left, .right]

but frankly that's not much better than the explicit:

let sides: BitwiseSet<Side> = [.left, .right]

The fact that I can do:

let sides: SideOptionSet = .left

but not:

let sides: Set<Side> = .left // 🛑 Type 'Set<Side>' has no member 'left'

does feel like a bug. IMHO the two should behave the same way (whatever it is).

1 Like

Is there an affordance for adding availability to new options as the option set evolves?

5 Likes

I went ahead and implemented something similar to your idea. The result is this:

@OptionSet<UInt8>
struct ShippingOptions {
  static var nextDay: ShippingOptions
  static var secondDay: ShippingOptions
  static var priority: ShippingOptions
  static var standard: ShippingOptions

  static let express: ShippingOptions = [.nextDay, .secondDay]
  static let all: ShippingOptions = [.express, .priority, .standard]
}

The implementation wasn't hard, and turns each of those non-initialized static properties into computed properties, e.g.,

static var nextDay: ShippingOptions {
  get {
    Self(rawValue: 1 << 0)
  }
}

One of the nice things about @Joe_Groff 's formulation here is that you can go ahead and put availability on the static variables, along with comments, access control, and anything else. This is one of the advantages of this "fill in the details" approach vs. what was originally proposed.

Doug

20 Likes

I think it would be really neat if the conformance-synthesising macros could be used at the call site as if they were just @-prefixed protocol names:

struct ShippingOptions: @OptionSet<Int> {
  static let nextDay: Self
  static let secondDay: Self
}
4 Likes

That's not an OptionSet though. If you have to specify the backing storage type anyway, why not macro-generate values for a property wrapper instead? (This compiles. I like the lack of explicit typing.)

struct ShippingOptions: OptionSet {
  @Option(0) static var nextDay
  @Option(after: _nextDay) static var secondDay
  @Option(3) static var priority
  @Option(after: _priority) static var standard

  let rawValue: UInt8
}
/// An "option", represented by a single bit flag.
@propertyWrapper public struct OptionSetOption<Options: OptionSet>
where Options.RawValue: BinaryInteger {
  public init(_ bitFlagIndex: Options.RawValue) {
    wrappedValue = .init(rawValue: 1 << bitFlagIndex)
  }

  public let wrappedValue: Options
}

public extension OptionSetOption {
  /// An option with the next higher bit flag.
  init(after previous: Self) {
    wrappedValue = .init(rawValue: previous.wrappedValue.rawValue << 1)
  }
}

public extension OptionSet where RawValue: BinaryInteger {
  typealias Option = OptionSetOption<Self>
}

Macro-fication would maybe be like

@Option static var nextDay
public extension OptionSetOption {
  public init(_: MacroMagic = HoweverMacrosWork) {
    self.init(🪄 as Options.RawValue)
  }
}

(I still think this is all uglier than the tuple solution. It's probably a better use of time to switch to making it so that property wrappers can apply to multiple variables, integrating with variadic generics.)

For whatever it's worth: I would, and I have never written a line of compiler code in my life, and I consider myself an app developer. ¯\_(ツ)_/¯

6 Likes

I don't think that there are any technical arguments for either singular or plural spelling. Both works from a technical point of view, non? I think the only argument here is whether one or the other "looks unpleasant", and that is a matter of opinion, not fact.

Personally, I think the suggested plural spelling "looks pretty unpleasant", but that's just my opinion.

However, I really like this pitch. Would use! Would recommend to others! <3

1 Like

What about improving the compiler to check all options have unique bit masks as It does with RawRepresentable enums?

It will also be convenient if automatic conformance of CaseIterable or some respectful analog (like OptionsIterable) will appear for OptionSet.

In my opinion, the best syntax for this would be something like:

@OptionSet<UInt8>
struct ShippingOptions {
  case nextDay
  case secondDay
  case priority
  case standard = 0b1000_0000

  static let express: ShippingOptions = [.nextDay, .secondDay]
  static let all: ShippingOptions = [.express, .priority, .standard]
}

Would it be possible to use a macro to enable something like this? If not, then should we consider making it possible?

1 Like

I really like this approach. Using an enumeration as a schema sounds good on the surface, but it doesn’t capture .express in this example.

This might be cool if a macro could double as a protocol with a synthesized implementation. It would need to go further then just synthesizing protocol conformance in this case, but maybe that is ok. Particularly if it is otherwise just synthesizing unimplemented declarations. It would help with documentation of synthesized methods too.

Another idea is this could be sugar for a macro and a protocol with the same name.

1 Like

Remember, key paths can’t reference static properties, so things like \.none (or \Optional<Int>.Type.none) won’t be legal even if this transformation is applied to enums. I have yet to see a good justification for why static key paths are illegal, though.

4 Likes
4 Likes

Note: I accidentally deleted this, so reposting. I revised my take a bit. I'm overall neutral on this, but I think it could be interesting and would work nice with this macro variation suggested by user @1-877-547-7272.

I think this has some interesting implications beyond option sets to anywhere literals are defined.

OptionSet Macro Example
@OptionSet<UInt8>
struct ShippingOptions {
  case nextDay
  case secondDay
  case priority
  case standard = 0b1000_0000

  static let express: ShippingOptions = [.nextDay, .secondDay]
  static let all: ShippingOptions = [.express, .priority, .standard]
}

case as a singleton for constant value types

I think case could be used as a singleton for constant value types that conform to Equatable. It basically already is for enums. It shouldn't be allowed for reference types like classes/actors or for values with nested reference types. I think it might improve readability– particularly for those new to Swift since it is easy to correlate case with singleton values you test against. It might be great metadata for documentation organized by cases or generating case name based coding implementations.

If this were done, I’d say:

case nextDay
case secondDay

roughly transforms to: (the values just can't be calculated at run time)

static let nextDay: Self
static let secondDay: Self

And it should work that way for enums too. Including for use in Key Paths. If this fits in to unifying enum cases with key paths then it might be a great idea.

If this is true, no reason this shouldn’t be allowed too:
(static let might be more appropriate depending on the details)

 // #expression represents whatever the compile time eval macro is
case express = #expression([.nextDay, .secondDay])

There is implied a limited set of options to cases. The compiler uses this to check for bugs in your switch statements. I think this is the biggest reason not to do this since option sets are sets and not enums. It doesn't make sense to match an option set exhaustively.

If exhaustivity wasn't an issue, then it should be extended to other singletons like within NSNotification.Name extensions.

Since case would take its type from the value-type it is contained in, it would not be possible to use top-level or within a function. The same as case in an enum.
​ ​

roughly:

static let nextDay: Self = unsafeBitCast(0 as UInt8, to: Self)
static let secondDay: Self = unsafeBitCast(1 as UInt8, to: Self)

There's also rawValue issues but that's separate.

It actually does make sense... just hasn't been done in swift so far...

Pseudo code:

enum ShippingOptions = { fast, reliable }
let val: OptionSet<ShippingOptions> = ...

switch val {
    case []: print("nope")
    case [.fast]: print("fast")
    case [.reliable]: print("reliable")
    case [.fast, .reliable]: print("fast and reliable")
    // and there is no default case as it can't possibly be anything else
}
3 Likes

I like your pseudo code version of an option set. It might be worth considering as another idea for an option set macro. I like it because it feels more like a set of elements.

OptionSets should support set-based pattern matching in the future

// The literals are specified separately
enum ShippingOptions: Int { case fast, reliable }

// Conforms to FixedSetProtocol for exhaustive pattern matching on sets.
// An associated type for literals is autogenerated in this case.
@OptionSet<ShippingOptions> struct ShippingOptionsSet: FixedSetProtocol

extension ShippingOptionsSet {
  static let all = [.fast, .reliable]
}

Alternatively, I like the idea of this as a type, but that would be yet another kind of macro. Although the implementation would probably be almost the same as declaration macros.

typealias ShippingOptionsSet = @OptionSet<ShippingOptions>

I think it might be important to make sure switch exhaustivity checks could be performed in the future with pattern matching. Option sets would conform to a protocol (FixedSetProtocol above) that turns on exhaustivity checks.

// exhaustive checks with hypothetical set based pattern matching
switch options {
case []: print(“none”)
case .fast: print(“fast”)
case .reliable: print(“reliable”)
case [.fast, .reliable]: print(“fast and reliable”) // or `case .fast & .reliable:`
case .all: print("the kitchen sink") // redundant, just an example
// no default needed since every literal is specified so it will always exit
}

I'm actually starting to see the potential rationale behind creating a new type for option sets.

Option sets sit in an awkward space between enums and structs, as evidenced by the fact that the proposal would require two separate types to perform its goal. Currently, we have a set of values that have a small memory footprint and statically-allocated size. There's awkward boilerplate when it comes to constructing the bitmask part of it, however. In a lot of cases it would also be nice to be able to switch over individual options within that set.

This proposal addresses the awkward boilerplate, but adds an awkward extra type for all the individual cases. This could be used for exhaustive switch statements, but it would be preferable, IMO, if the individual options and the bitmask were unified in a single type.

Although I didn't at first, I actually find myself agreeing with @davedelong that we should instead investigate some alternative. I wouldn't be opposed to a new value type alongside struct and enum.

optionset MyOptions: UInt64 {
  case fast = 0 // refers to bit offset
  case reliable = 1
}

/*
synthesized:
extension MyOptions {
  // both of these could return nil with invalid values
  init?(rawValue: UInt64) { ... }
  init?(bitOffset: some BinaryInteger) { ... }
}
*/

let emptyOptions: MyOptions = []
var bitOffsetOption = MyOptions(bitOffset: 1) // ?[.reliable] 
var multipleOptions = MyOptions(rawValue: 3) // ?[.reliable, .fast]
bitOffsetOption = MyOptions(bitOffset: 42) // nil
multipleOptions = MyOptions(rawValue: 42) // nil

optionset MyNonCompilingOptions: UInt8 {
  case a, b, c, d, e, f, g, h, i // Error: too many options for memory layout
  case j = 9 // Error: bit offset too large
}

There would be a lot to work out with specifics (eg, semantics of switching through an optionset that contains multiple values), but I think this would be preferable to the band-aid solution of a macro.

8 Likes