[Pitch] `@OptionSet` macro

Option 3 seems to tacitly admit that set algebra is separable from the more primitive “flags” concept, which would be very useful on its own. I can envision a @Flags macro that optionally supported conformance to OptionSet:

struct MyView {
  @Flags
  struct ViewFlags {
    // Set algebra doesn’t make sense for these flags.
    var inWindow: Bool
    var hasSuperview: Bool
    var needsLayout: Bool
  }
  var flags: ViewFlags

  @Flags
  enum Edge : OptionSet {
    // Set algebra _does_ make sense for these flags.
    case leading
    case trailing
    case top
    case bottom
  }
}

It might even be the case that all @Flags enums are implicitly option sets, while all @Flags structs are not.

1 Like

I think this too. It might be a good thing because you are less likely to abuse OptionSets if it gets enum baggage. It would still allow OptionSets to work the traditional way for interop.

I think it makes sense not to combine them in to one macro though. The "primitive flags" just needs memberwise initializers. If it uses a macro at all, it would just be for bit packing.

I'm very supportive of Swift adding memberwise initializers. Hopefully a way to do memberwise pattern matching and memberwise destructuring too. This has almost happened a number of times I think. Memberwise structs could allow a lot of cool stuff beyond just FlagSets:

let Point(x, y, z) = somePoint // destructures a point to x, y, z variables

I like that that very much!

Yes! Perhaps not making @State default though, but:

view Map {
    state var region: MKCoordinateRegion = ...
}

where state is this new "meta-macro" getting translated into (or treated as) "@State"

With Doug's recent examples we could have a nice looking, as if it was a first class language construction on par with struct, enum, etc:

optionset ShippingOptions: UInt8 {

getting translated / treated as one of the following forms (TBD which):

// Form #1
@OptionSet<UInt8>
struct ShippingOptions {

// Form #2
@OptionSet<UInt8>
struct ShippingOptions {
  enum Options {

// Form #3
@OptionSet<UInt8>
enum ShippingOptions: UInt8 {

Could macro system be extended to support that? @Karl would you be interested making a pitch for that?

Sure. It should work already.

It's going to turn into a computed property, so it can't be a let.

For forms 2 and 3 where we have raw-valued enums, yes, you can customize the value and the compiler will pick the "next" value for subsequent cases.

We could certainly allow it. If we wanted customization for form #1, that's how it would have to be anyway.

Right now, it'll trap at runtime when you use that 9th element, but that's just me being lazy: so long as we know how many bits are in the underlying type (easy with sized integer types, problematic for Int/UInt), we could diagnose this in the macro. I intend to do this, but haven't gotten to it yet.

That'd be easy to add.

This seems doable, although the ExpressibleByOptionSetLiteral bit is stepping beyond "macro automating boilerplate for existing patterns" and toward "creating new ways to define option sets"; it might not be the best form for the latter.

FWIW, putting the @OptionSet on the nested Set type won't work here, because the macro doesn't have visibility into the cases of the outer enum.

I posted a similar @FlagSet notion earlier. I think it's a cool direction, and useful, and... not option sets.

Doug

2 Likes

It seems possible to support larger option sets:

struct Options256: OptionSet {
  let rawValue: SIMD4<UInt64>
  static var lower: Self { Self(bitIndex: 0) }
  static var other: Self { Self(bitIndex: 123) }
  static var upper: Self { Self(bitIndex: 255) }
}

The init(bitIndex:) is custom API. Only four default implementations are needed:

extension OptionSet where RawValue: SIMD, RawValue.Scalar: FixedWidthInteger {

  public init(bitIndex: Int) {
    precondition(bitIndex >= 0)
    precondition(bitIndex < RawValue.scalarCount * RawValue.Scalar.bitWidth)
    let vectorIndex = bitIndex >> RawValue.Scalar.bitWidth.trailingZeroBitCount
    let scalarValue = RawValue.Scalar(1) &<< bitIndex
    var rawValue = RawValue()
    rawValue[vectorIndex] = scalarValue
    self.init(rawValue: rawValue)
  }

  public init() {
    self.init(rawValue: RawValue())
  }

  public mutating func formUnion(_ other: Self) {
    self = Self(rawValue: self.rawValue | other.rawValue)
  }

  public mutating func formIntersection(_ other: Self) {
    self = Self(rawValue: self.rawValue & other.rawValue)
  }

  public mutating func formSymmetricDifference(_ other: Self) {
    self = Self(rawValue: self.rawValue ^ other.rawValue)
  }
}
3 Likes

I think 64 options is already on the extreme side. Anything that wide is mainly doing it to align with pointer size. It is very rare for me to find OptionSets that need more than 8 options. The sets that get anywhere close are doing it for C-interop and those cases are generally restricted to 32 or 64 bit to match pointer size.

FlagSets are of course another story, but those should grow arbitrarily large. I could see multiple OptionSets packed in to a FlagSet. In fact, I’m pretty sure the FlagSet macro design that Douglas_Gregor presented earlier would allow clipping OptionSets down, For example, you could have a 7-bit option set and use the extra bit for a flag. You would just need to make sure you don't have 8 options or the last one will get chopped off!

1 Like

I think one could also use tuples of UInt64s to get a predictably-laid-out bag of bits that could be arbitrarily packed.

Doug

3 Likes

The macro could avoid this scenario by checking the number of cases at compile-time.

public struct Options: MemberMacro {
    public static func expansion<Declaration, Context>(of node: AttributeSyntax, providingMembersOf declaration: Declaration, in context: Context) throws -> [DeclSyntax] where Declaration : DeclGroupSyntax, Context : MacroExpansionContext {
        guard let declaration = declaration.as(EnumDeclSyntax.self) else {
            fatalError("Cannot apply OptionSet macro to a non-enum type")
        }
    
        func makeAssociatedType() -> InheritedTypeSyntax {
            switch declaration.memberBlock.members.filter { $0.as(EnumCaseElementSyntax.self) != nil }.count {
            case 0...8:
                return InheritedTypeSyntax(typeName: TypeSyntax("UInt8"))
            case 9...16:
                return InheritedTypeSyntax(typeName: TypeSyntax("UInt16"))
            case 17...32:
                return InheritedTypeSyntax(typeName: TypeSyntax("UInt32"))
            case 33...64:
                return InheritedTypeSyntax(typeName: TypeSyntax("UInt64"))
            default:
                fatalError("Too many enum cases")
            }
            
        }
        let associatedType = declaration.inheritanceClause?.inheritedTypeCollection.first ?? makeAssociatedType()
        
        var index = 0
        var members: [String: String] = [:]
        
        for member in declaration.memberBlock.members {
            if let member = member.decl.as(EnumCaseDeclSyntax.self) {
                if let element = member.elements.first?.as(EnumCaseElementSyntax.self) {
                    if let value = element.rawValue?.value.as(IntegerLiteralExprSyntax.self) {
                        if let i = Int(value.description) {
                            index = i
                        }
                        
                        members[element.identifier.description] = value.description
                    } else {
                        members[element.identifier.description] = String(describing: index)
                    }
                    
                    index += 1
                }
            }
        }
        
        let string = members.map {
            "\tpublic static var \($0.key.trimmingCharacters(in: .whitespaces)) = Self(rawValue: 1 << \($0.value))"
        }.joined(separator: "\n\t")
        
        return ["""
        \n\n\tpublic struct Set: OptionSet {
            \tpublic var rawValue: \(associatedType)
            \tpublic init(rawValue: \(raw: associatedType.description.trimmingCharacters(in: .whitespaces))) {
               \t\tself.rawValue = rawValue
            \t}
            \t
            \(raw: string)
        \t}
        """]
    }
}

The @OptionSet macro appears to be implemented in the Swift 5.9 snapshots. Should it be removed until this proposal has been accepted?

https://github.com/apple/swift/blob/swift-5.9-DEVELOPMENT-SNAPSHOT-2023-05-26-a/stdlib/public/core/Macros.swift#L66-L94

6 Likes

Why did this landed without a review?

5 Likes

It is quite common nowadays that features are integrated without review, isn't it? This might be more prominent than some new capability that is hidden behind a compiler flag, but I think the official evolution documents just don't reflect reality (anymore).

Examples

This topic seems to have become big enough for a separate thread...

1 Like

This landed inadvertently as part of the macros deployment and the plan is either to run a review or back it out (or keep it under an experimental flag – unlikely in this case due to its nature).

I don't think it's at all true that it's "quite common" to land changes without a review. Some minor changes are considered bug fixes, and some changes land with underscores or flags to ensure they can develop in-tree (especially if they are more complex in terms of their integration with the compiler).

The exception here is C++ interoperability which is fairly different in nature and so is taking a path outlined in John's post here.

11 Likes

I think the underlying confusion stems from wondering how a person goes about introducing something behind a feature flag.

Can anyone do it? Like, can literally anyone just create their own flag and implement their own feature into Swift behind that flag, without going through Evolution?

Or is there some process for making that happen, and if so where is that documented and how does it work?

3 Likes

Swift's compiler development process tries to avoid longstanding complex feature branches; instead, we merge unreviewed features but gate them behind experimental-feature flags. I don't think we have a firm policy about this, but there's a general expectation that features being developed this way are simultaneously going through at least the pitch phase of the evolution process. (We do have a firm policy against adding unreviewed features that aren't gated behind experimental-feature flags — a policy which unfortunately was violated here, albeit inadvertently and (I think) harmlessly.)

Now, these features are definitely in the compiler, and they have not gone through evolution; but note that you can get the same effect by just forking the compiler source code to add a feature yourself. In either case, the feature is not really in the language because the project is not promising to support it in any way. Experimental features which no longer have a recognizable path to approval can and do simply get removed from the compiler, flag and all.

7 Likes

This doesn’t really answer the question.

If I, personally, have an unreviewed feature, and I open a PR to add it to Swift, will the compiler development team accept that PR, gating the feature behind an experimental flag?

Or is there some other process that I would have to go through to request permission to create such a flag and feature?

Or is it only possible for individuals at Apple, or on the Swift Core Team, or in some other group, to take that route?

2 Likes

It’s not reserved only for certain people. As code owners, we make a judgment call based on an informal assessment of the feature’s prospects and how much work it would take to maintain it as a branch for a few months. Sometimes the Language Working Group gets asked for their opinion on the first point.

If the feature is merged but doesn’t seem to be making progress, it may be removed. For what it’s worth, that has happened to work by Apple engineers.

Again, this is all development process, not evolution process. Is there a reason you’re interested in this?

5 Likes

Just trying to understand the process.

Thanks for the explanation.

1 Like

Looks like this was reverted as of the latest Xcode beta, as the macro seems to no longer be available.

3 Likes

@John_McCall hmm… an out-of-the-box c++ style aggregate initialization support for simple swift structs? is there anyplace i can follow along for any more developments on that? thanks!

I'm not aware of anyone working on that.