I haven't followed this closely... Will the modularized Foundation make it easier to use for server-side?
i imagine if the goal is to split Foundation into many smaller modules, then that is a reason to keep the hypothetical OptionSet module separate from the existing Foundation functionality.
1 Like
And yet, it's part of the standard library as the OptionSet protocol, and baked into the C interoperability story. The proposed macro makes it easier to define new types that fit in with the standard library's notion of an OptionSet, something that is currently error-prone and tedious. The alternative you are proposing doesn't exist, and (without further extension) delegates control over layout to the Swift compiler.
Sometimes you do want low-level control over bit placements. I think of how extensively we use the C++ FlagSet in the runtime to provide an abstraction over a specific layout of flags in a fixed-sized integer, and I want to be able to do that in Swift (with less boilerplate). A FlagSet-style macro could follow the same general pattern of the OptionSet macro (especially with the revised formulation that ditches the nested enum), but with annotations on properties to provide bit positions and widths:
@FlagSet<UInt16>
struct TypeContextDescriptorFlags {
@Bitfield(width: 2)
var metadataInitialization: MetadataInitialization
var hasImportInfo: Bool
var hasCanonicalMetadataPrespecializations: Bool
var hasLayoutString: Bool
@Bitfield(bit: 7)
var isActor: Bool
var isDefaultActor: Bool
@Bitfield(width: 3)
var resilientSuperclassReferenceKind: TypeReferenceKind
var ImmediateMembersNegative: Bool
var hasResilientSuperclass: Bool
var hasOverrideTable: Bool
var hasVTable: Bool
}
The corresponding C++ definition of this ABI type is here. We can provide better checking within the macro to ensure that bitfields aren't overlapping, don't run over the end of the raw type, etc.
How many language extensions would we need to be able to express the TypeContextDescriptorFlags above as a simple struct? I can write that today with macros (might be fun), and I feel like that's the whole point of macros---that we don't need to wait for a future language feature to make things automatic, because we can build it now out of existing pieces.
Doug
8 Likes
The OptionSet protocol is part of the standard library. It cannot be moved to Foundation without breaking existing clients.
The proposed OptionSet macro could go anywhere, I suppose, but given that it's providing behavior specifically to make conformance to the OptionSet protocol easier, it should go in the standard library.
Doug
4 Likes
Intriguingly, this could also open up an opportunity for making swift structs layout compatible with C (something we don't have today):
@frozen(packed)
// some packed layout, fields could be reordered
@frozen(canReorder: false, alignment: natural)
// struct order is honoured
@frozen(canReorder: false, alignment: 4)
// struct order is honoured, fields aligned at 4-byte boundaries
@frozen(canReorder: false, alignment: no)
// struct order is honoured, there's no padding in this structure
4 Likes
FTM, OptionSets are also used massively used in modern Swift-only Apple frameworks (e.g. swiftUI), so that's not just C/NS_Options compatibility thing. Example:
/// An enumeration to indicate one edge of a rectangle.
@available(iOS 13.0, macOS 10.15, tvOS 13.0, watchOS 6.0, *)
@frozen public enum Edge : Int8, CaseIterable {
case top
case leading
case bottom
case trailing
/// An efficient set of `Edge`s.
@frozen public struct Set : OptionSet {
/// The element type of the option set.
///
/// To inherit all the default implementations from the `OptionSet` protocol,
/// the `Element` type must be `Self`, the default.
public typealias Element = Edge.Set
/// The corresponding value of the raw type.
///
/// A new instance initialized with `rawValue` will be equivalent to this
/// instance. For example:
///
/// enum PaperSize: String {
/// case A4, A5, Letter, Legal
/// }
///
/// let selectedSize = PaperSize.Letter
/// print(selectedSize.rawValue)
/// // Prints "Letter"
///
/// print(selectedSize == PaperSize(rawValue: selectedSize.rawValue)!)
/// // Prints "true"
public let rawValue: Int8
/// Creates a new option set from the given raw value.
///
/// This initializer always succeeds, even if the value passed as `rawValue`
/// exceeds the static properties declared as part of the option set. This
/// example creates an instance of `ShippingOptions` with a raw value beyond
/// the highest element, with a bit mask that effectively contains all the
/// declared static members.
///
/// let extraOptions = ShippingOptions(rawValue: 255)
/// print(extraOptions.isStrictSuperset(of: .all))
/// // Prints "true"
///
/// - Parameter rawValue: The raw value of the option set to create. Each bit
/// of `rawValue` potentially represents an element of the option set,
/// though raw values may include bits that are not defined as distinct
/// values of the `OptionSet` type.
public init(rawValue: Int8)
public static let top: Edge.Set
public static let leading: Edge.Set
public static let bottom: Edge.Set
public static let trailing: Edge.Set
public static let all: Edge.Set
public static let horizontal: Edge.Set
public static let vertical: Edge.Set
/// Creates an instance containing just `e`
public init(_ e: Edge)
/// The type of the elements of an array literal.
public typealias ArrayLiteralElement = Edge.Set.Element
/// The raw type that can be used to represent all values of the conforming
/// type.
///
/// Every distinct value of the conforming type has a corresponding unique
/// value of the `RawValue` type, but there may be values of the `RawValue`
/// type that don't have a corresponding value of the conforming type.
public typealias RawValue = Int8
}
/// Creates a new instance with the specified raw value.
///
/// If there is no value of the type that corresponds with the specified raw
/// value, this initializer returns `nil`. For example:
///
/// enum PaperSize: String {
/// case A4, A5, Letter, Legal
/// }
///
/// print(PaperSize(rawValue: "Legal"))
/// // Prints "Optional("PaperSize.Legal")"
///
/// print(PaperSize(rawValue: "Tabloid"))
/// // Prints "nil"
///
/// - Parameter rawValue: The raw value to use for the new instance.
public init?(rawValue: Int8)
/// A type that can represent a collection of all values of this type.
public typealias AllCases = [Edge]
/// The raw type that can be used to represent all values of the conforming
/// type.
///
/// Every distinct value of the conforming type has a corresponding unique
/// value of the `RawValue` type, but there may be values of the `RawValue`
/// type that don't have a corresponding value of the conforming type.
public typealias RawValue = Int8
/// A collection of all values of this type.
public static var allCases: [Edge] { get }
/// The corresponding value of the raw type.
///
/// A new instance initialized with `rawValue` will be equivalent to this
/// instance. For example:
///
/// enum PaperSize: String {
/// case A4, A5, Letter, Legal
/// }
///
/// let selectedSize = PaperSize.Letter
/// print(selectedSize.rawValue)
/// // Prints "Letter"
///
/// print(selectedSize == PaperSize(rawValue: selectedSize.rawValue)!)
/// // Prints "true"
public var rawValue: Int8 { get }
}
Dozens of those.
Interestingly, it is reversed compared to an example in the motivation section of the pitch (not an enum inside a struct but a struct inside an enum).
1 Like
I'm not trying to argue that this is an inappropriate use of macros. I'm saying that I don't like OptionSet and don't want to encourage people to use that as a design pattern. The right language model for these types is a struct with a property for each option, not a collection of separate option values. I do not think people use these types in practice in any way that benefits from thinking of them as collections.
I do think there’s a missing language feature — one that generalizes memberwise initializers — that would make it a lot easier to create values of these types. But that would be much more broadly usable than just one set of types and would be reasonable to embrace in the core language.
9 Likes
EDIT:
I agree with this based on what I see in my use of OptionSet. It isn't that bad in some non-collection cases, but Swift doesn't have enough packing options and occasionally I use option sets when it is a really bad design to do so. If I literally just cared about packing 8 discrete flags in an OptionSet, then I feel it isn't horrible as much as it is inflexible. I'd like to see it easier to use structs because it is often the better abstraction and thinking forward I'd rather match on structs then sets for bit-field options if pattern matching is added later.
Although useful to configuration- I'm often using these abstractions to compact large data structures, implement a wire format, or pack data for a GPU kernel.
I would find versions of these used in the compiler (OptionSet, FlagSet, a hypothetical bit-field version of FlagSet) all useful. I think there probably should be a whole suite of these macros that all compose well.
I'd love to see an automatic struct packing option where the compiler makes reasonable decisions for size/performance. Sometimes you just want something reasonably performant that doesn't need to interop or use a predefined ABI.
Couldn't this be a macro too or is there a different syntax desired for simple configuration structs? Rust-like syntax?
I wonder if this pitch should change so it could be used in SwiftUI?
This small change would allow it to work both like in the pitch or like in SwiftUI.
// Similar to Edge.Set
enum ShippingOptions: Int {
case nextDay, secondDay, priority, standard
@OptionSet<ShippingOptions>
struct Set { }
}
// Similar to example in pitch
@OptionSet<ShippingOptions.Options>
struct ShippingOptions {
private enum Options: Int {
case nextDay, secondDay, priority, standard
}
}
EDIT: Updated to work with enum on outside or inside.
I do not understand how you have come to this point of view, but it doesn't seem in line with Swift practice. Nor our own: we have the same abstraction in the Swift compiler (also called OptionSet) and we use it in many places. We do set containment checks and difference operations because they have useful semantic meaning. And this is in C++, where we can (and do) use bitfields when they make sense, so we repeatedly chose to use this pattern because it is the right one many cases.
Doug
1 Like
If you look at how we use the C++ OptionSet in the compiler, it follows what I said pretty closely: most of the applications only involve concrete reads and writes of single, specific flags, and the arbitrary set-algebra aspects are not significantly used. There are a few exceptions where we do use OptionSet as a fixed-size bit vector and perform arbitrary set-algebra with it. Those places are generally using an existing enum type, so the current suggestions in this thread would not work for that pattern because they expect the enum to be defined within the macro. They are also definitely exceptions and could easily be expected to use a different macro / abstraction.
FlagSet, which you brought up before, is basically what I am describing as the right design: the external interface of a FlagSet type is a struct with a bunch of getters and setters that abstract over bitmasking. FlagSet does not support set algebra, and it wouldn't be semantically sensible for most of the types: they generally also pack non-boolean fields into the underlying bit-field. If this macro just did dense packing of a struct into a bit-field of some given underlying type, with optional ways to take control of the layout of that bit-field, plus a few operations for extracting and constructing with the raw underlying value, I think that would be great. That would eliminate a ton of annoying boilerplate in these cases where an exact layout is required for interop or whatever other reasons. I would love to have that macro in C++ instead of what we have to do with things like FlagSet. (I do think that we should also have built-in struct packing for people who don't need this control and are comfortable with the compiler's layout decisions, but the way I see it, the macro would mostly be rewriting implementations instead of adding new declarations to the interface, so switching between this builtin packing and the macro would be fairly lightweight. And as you point out, the macro can be available right now.)
Constructing a value of a FlagSet type is annoying in C++ because you typically initialize it with one or two mandatory arguments (unlabeled, of course), and then the rest has to be filled in with calls to setters. Similar types in Swift have similar problems, and I think that would be great to sugar as a form of memberwise initialization.
9 Likes
How does that reconcile with the fact that modern swift-only framework from Apple (SwiftUI) uses OptionSets massively? Being swift-only there's no Obj-C compatibility concerns in this case, just these:
-
use bits for flags instead of full bytes to save space (compiler packing bools to bits is a pipe dream yet).
-
keep the structs that hold option sets
POD(if they werePODalready) -
use compact syntax to represent set values, like so:
SubView() .padding([.top, .trailing], 10) .padding(.all.subtracting(.top), 20) // vs somewhat mildly heavier: SubView() .padding(.init(top: true, trailing: true), 10) .padding(.init(top: true, trailing: true, bottom: true), 20)
5 Likes
The problem we have now is that everybody works with OptionSet as if these constraints were required:
extension OptionSet where Self == Self.Element {
extension OptionSet where Self.RawValue : FixedWidthInteger {
They're not. But that's where all of the default extensions are defined, so they might as well be. And this giant thread illustrates that making a type a "collection" of itself is too hard to work with. OptionSet's Element should not be itself; it should be another type which represents a single option.
Without macros, we can still all stop conflating Options and Sets thereof.
var shippingOptions = ShippingOption.Set.secondDayPriority
shippingOptions.formUnion(.init(.standard))
[ShippingOption.secondDay, .priority, .standard] == .init(shippingOptions) // true
enum ShippingOption: BitFlag<Int> {
case nextDay, secondDay
// this bit is cursed, don't use it
case priority = 3, standard
}
extension BitFlagRepresentableOptionSet<ShippingOption> {
// A static property should always return the enclosing type, unless named with a different type.
// The type of this is ShippingOption.Set. It belongs here, not in `ShippingOption`.
static let secondDayPriority = Self([ShippingOption.secondDay, .priority])
}
It's slightly uglier with more rawValues and .init()s sprinkled about, but consistent with sets of other enumerations. Some methods would be improved by copying the defaults from where Self == Self.Element to:
extension OptionSet where
Element: RawRepresentable,
Element.RawValue: RawRepresentable<RawValue>
BitFlagRepresentable.Set
public extension RawRepresentable
where RawValue: RawRepresentable, RawValue.RawValue: FixedWidthInteger {
typealias Set = BitFlagRepresentableOptionSet<Self>
}
public struct BitFlagRepresentableOptionSet<BitFlagRepresentable: RawRepresentable>: OptionSet
where
BitFlagRepresentable.RawValue: RawRepresentable,
BitFlagRepresentable.RawValue.RawValue: FixedWidthInteger {
public typealias RawValue = BitFlagRepresentable.RawValue.RawValue
public init(rawValue: RawValue) {
self.rawValue = rawValue
}
public let rawValue: RawValue
}
// MARK: - public
public extension BitFlagRepresentableOptionSet {
init(_ bitFlagRepresentable: BitFlagRepresentable) {
self.init(rawValue: bitFlagRepresentable.rawValue.rawValue)
}
}
// MARK: - Array
public extension Array where
Element: RawRepresentable,
Element.RawValue: FixedWidthInteger & _ExpressibleByBuiltinIntegerLiteral {
init<OptionSet: Swift.OptionSet>(_ optionSet: OptionSet) where
OptionSet.RawValue == Element.RawValue {
self =
optionSet.rawValue == 0 ? []
: (
optionSet.rawValue.trailingZeroBitCount
..<
OptionSet.RawValue.bitWidth - optionSet.rawValue.leadingZeroBitCount
).compactMap {
.init(rawValue: optionSet.rawValue & 1 << $0)
}
}
}
public extension Array where
Element: RawRepresentable,
Element.RawValue: RawRepresentable,
Element.RawValue.RawValue: FixedWidthInteger & _ExpressibleByBuiltinIntegerLiteral {
init<OptionSet: Swift.OptionSet>(_ optionSet: OptionSet)
where OptionSet.RawValue == Element.RawValue.RawValue {
self = [BitFlag](optionSet).compactMap {
Element.RawValue(rawValue: $0.rawValue).flatMap(Element.init)
}
}
}
@michelf @davedelong Those spellings are confusing, but the language provides no safe way to do better. The relevant missing feature is the ability to use method syntax for setters, and its near-equivalent, named subscripts.
var style: NSWindow.StyleMask = []
style.contains[.borderless] = true
style.contains[.closable].toggle()
style.contains[.resizable] = style.contains(.closable)
public extension SetAlgebra {
var contains: ValueSubscript<Self, Element, Bool> {
mutating get {
.init(
&self,
get: Self.contains,
set: { set, element, newValue in
if newValue {
set.insert(element)
} else {
set.remove(element)
}
}
)
}
}
}
ValueSubscript is probably unsafe but I don't know how to make it fail.
/// An emulation of the missing Swift feature of named subscripts.
/// - Note: Argument labels are not supported.
public struct ValueSubscript<Root, Index, Value> {
public typealias Pointer = UnsafeMutablePointer<Root>
public typealias Get = (Root) -> (Index) -> Value
public typealias Set = (inout Root, Index, Value) -> Void
public var pointer: Pointer
public var get: Get
public var set: Set
}
public extension ValueSubscript {
init(
_ pointer: Pointer,
get: @escaping Get,
set: @escaping Set
) {
self.pointer = pointer
self.get = get
self.set = set
}
subscript(index: Index) -> Value {
get { get(pointer.pointee)(index) }
nonmutating set { set(&pointer.pointee, index, newValue) }
}
}
3 Likes
I don't have an opinion on the proposal for now.
Regarding the struct argument, my main use case for OptionSet is parsing network header flags as a single integer, use set operations on those flags and be able to store and retrieve set containing unknown values. In this use case, I need to be specific about the raw values so I don't gain much from the macro.
For very simple set of options which rawValues should all be generated a simple Set where Enum: Int, RawValue should do je job ? For the syntax, if we had the ability to use a single value when a set is expected we could avoid OptionSet altogether.
1 Like
Well, these uses are significant to me :). The effects of a declaration are captured in an option set (with throws and async), where set-algebra operations are useful to check sub typing. Macro roles are captured in an option set and we use set-algebra for separating out attached from freestanding macros.
This all feels moot, because, as others have pointed out, OptionSet is used in both new APIs and existing ones in the Swift ecosystem.
At this point, I'm not actually sure what you are arguing for, beyond you not liking the design of the existing OptionSet. I can imagine several possibilities...
- Are you arguing that we shouldn't provide a macro to help folks define new
OptionSet-conforming types, because the design ofOptionSetis so bad that it would be better to wait for some as-yet-undesigned language feature than to promote more usage? - Are you arguing that if we were to build a language feature to make this area better (instead of doing a macro), it wouldn't be anything like a new nominal
optionsettype so those language-based solutions are a dead end? - Are you arguing that we should provide a macro here, but that the pitched form of the macro isn't good enough and should be revised?
... or is it something else entirely?
Doug
to be honest, this has also been my experience. the amount of associated functionality i want to provide with these bitfields eclipses the size of the definitions themselves.
while i always welcome features that save me typing, i do feel like 99% of my pain would be solved by an "copilot"-like sourcekit-lsp feature that just makes it faster for me to type out these definitions.
I think what I would like the most is for the macro to be almost invisible. Instead of writing this:
struct ShippingOptions: OptionSet {
let rawValue: Int
static let nextDay = ShippingOptions(rawValue: 1 << 0)
static let secondDay = ShippingOptions(rawValue: 1 << 1)
static let priority = ShippingOptions(rawValue: 1 << 2)
static let standard = ShippingOptions(rawValue: 1 << 3)
}
Ideally I'd just write this:
@AutoFill // would be nicer if this was unnecessary
struct ShippingOptions: OptionSet {
static let nextDay
static let secondDay
static let priority
static let standard
}
Also see the shorter version
@AutoFill
struct ShippingOptions: OptionSet {
static let nextDay, secondDay, priority, standard
}
This syntax is pretty self-explanatory and familiar because it does not try to reinvent the option set.
At the opposite end, the @OptionSet macro syntax proposed here amounts to a full redesign of the declaration language for an option set. This is why this discussion devolved into a myriad of new concepts: people saw a redesign and came up with their own better redesign ideas.
An by "redesign" I mean there is almost nothing in common between this and a normal option set declaration:
@OptionSet
struct ShippingOptions {
private enum Options: Int {
case nextDay, secondDay, priority, standard
}
}
There's no conformance declaration, and semantically you aren't even using the enum cases: they're mostly just there as a template to generate static variables with corresponding names in the parent type. It's an entirely new language that produces the same thing as before with a syntax that is now far removed from the actual type you'll be using in the end.
I think this redesign is for the most part unnecessary. What we need is only a bit of convenience on top of the current design.
2 Likes
To reiterate my previous post with a simpler example, there's nothing that breaks the current "design" by separating the conflated type. You just have to copy over an implementation for contains, insert, remove, and update.
enum ShippingOption: Int {
case nextDay, secondDay, priority, standard
struct Set: OptionSet {
typealias Element = ShippingOption
let rawValue: Element.RawValue
}
}
I like this, but I feel like Swift should adopt a macro policy that says if you forget to add the macro, the code either fails to compile in a way that tells you use the macro, or is valid on its own. Unfortunately there's no way to do that with static values without initial values, which rather defeats the purpose of this macro. I also rather dislike any of the transforms that veer too far from what the user has written. The original proposal meets the first rule easy, but fully synthesizing the static members of the options may be too far.
I do like the idea of replacing the OptionSet conformance with @OptionSet if you want the auto synthesis. It sets up a good pattern going forward. Another option is something like @synthesize(OptionSet) or something like that, but that doesn't seem to scale well for macros.
1 Like
Is this really reasonable to expect this from macros when we already don't expect this from property wrappers?
import SwiftUI
struct ContentView: View {
@Environment(\.locale) var locale // look, no value and no type!
... // also, no init()
}
But I suppose we could use static var if it's the let you find objectionable.
1 Like