Enumerating a parameter pack of metatype instances

Using Swift
1

I’m trying to build a generic ManagedBuffer subclass to store densely-packed data (think of a table in a database). Each element of my buffer is a tuple of fields, with the fields determined by the client.

I’m having trouble enumerating over the field types themselves. Let’s say I want to store the names of the fields in my table’s header. Here’s where I’m stuck:

protocol Field: BitwiseCopyable {
  static var name: String { get }
}

class Table<each F: Field>: ManagedBuffer<TableHeader, (repeat each F)> {
  static func create() { Table.create(minimumCapacity: 1) { TableHeader<repeat each F>() } }
}

struct TableHeader {
  var fieldNames = [String]()
  init<each F: Field>(fieldTypes: repeat each F) {
    for fieldType in repeat each F {
//                   ^ error: pack expansion 'repeat each F' can only appear in a function parameter list, tuple element, or generic argument of a variadic type
//                   ^ error: for-in loop requires '_.Type' to conform to 'Sequence'
      fieldNames.append(fieldType.name) 
  } 
}

Adding parentheses around repeat each F doesn’t help… the compiler seems to think I'm trying to enumerate the type of the tuple itself, which is obviously wrong:

struct TableHeader { 
  var fieldNames = [String]() 
  init<each F: Field>(fieldTypes: repeat each F) { 
    for fieldType in (repeat each F) {
//                   ^ error: for-in loop requires '(repeat each F).Type' to conform to 'Sequence'
      fieldNames.append(fieldType.name) 
    } 
  } 
}

I tried adding .self and shuffling parentheses around but could not come up with a spelling that works. Does anyone know how to achieve this in Swift 6.2?

2

for fieldType in repeat (each F).self {

3 
struct TableHeader {
  var fieldNames = [String]()
  init<each F: Field>(fieldTypes: repeat each F.self)
    for fieldType in repeat (each F).self {
//                                   ^ error: 'self' is not a member type of type 'each F'
//                           ^ error: 'each' cannot be applied to non-pack type '(each F).`self`
      fieldNames.append(fieldType.name)
    }
  }
}
4

This change is wrong, my original edit was for your for loop only. Remember that in type context, the spelling for a metatype is T.Type, while in expression context, a metatype value is T.self.

Here is a self-contained example that compiles:

protocol Field {
  static var name: String { get }
}

struct TableHeader {
  var fieldNames = [String]()
  init<each F: Field>(fieldTypes: repeat (each F).Type) {
    for fieldType in repeat (each F).self {
      fieldNames.append(fieldType.name)
    }
  }
}
5

D'oh, the extra .self must have been left over from my attempt to use the parameter directly. (Honestly I don’t want that parameter at all; I only need it in order to satisfy the compiler’s demand that the generic parameter is used in the function signature.)

Thanks.

1 Like
6

Since F is a generic parameter of the init, it has to appear in the parameter list, otherwise there is no way to specify it at all. But if making TableHeader generic over F is an option, like this:

struct TableHeader<each F> {
  init() {
    ...
  }
}

Then you can simply write TableHeader<...>() to call your init.

Also, if you keep the fieldTypes parameter, you can also write:

for fieldType in repeat each fieldTypes {

which is perhaps more readable than

for fieldType in repeat (each F).self {
1 Like
7

…with the fun caveat that these can behave differently when subclasses or erasure are involved >:-)

2 Likes
8

That was my original design, and I will likely go back to it. Though I could avoid it if variadic parameters were allowed to have default values:

struct TableHeader {
  var fieldNames = [String]()
  init<each F: Field>(fieldTypes: repeat (each F).Type = repeat (each F).self) {
//                                                     ^ error: variadic parameter cannot have a default value
    for fieldType in repeat (each F).self {
      fieldNames.append(fieldType.name)
    }
  }
}

Thankfully, my ManagedBuffer subclass is final. But I do need a type-erased AnyTable wrapper…

9

I think just subclasses, no? Are there any other metatypes which have more than one inhabitant?

10

Protocols?

11

Let's think about that for a second. Suppose you could write that, and you then did:

let header = TableHeader()

What is F?

(In fact, you can almost achieve this if you change the parameter into a tuple containing a pack expansion, instead of a bare pack expansion:

struct TableHeader {
  var fieldNames = [String]()
  init<each F: Field>(fieldTypes: (repeat (each F).Type) = (repeat (each F).self)) {

This type checks just fine, but the trouble is, since F is not mentioned anywhere else in the signature of the init, you're back to not being able to call it without providing a value for the argument, so the default is in fact useless. You can demonstrate the same problem without variadic generics:

struct TableHeader {
  init<F>(fieldType: F.Type = F.self) {}
}

TableHeader()  // error: cannot infer generic parameter F

)

12

Protocol metatypes are singletons, so P.Type has exactly one value P.self no matter what (even if protocol Q inherits from P for example, then Q.self is not actually an instance of P.Type.)

13

I was thinking existentials, like Kyle said, but the inference doesn't work there: Any.Type is apparently not a valid T.Type. (You can make it work with values, but then the difference is more obvious because there's an explicit type(of:) invocation.) Here's the example with subclasses, for anyone curious what we're talking about:

func printStaticAndDynamicTypes<each T>(_ types: repeat (each T).Type) {
    for ty in repeat (each T).self {
        print("static: \(ty)")
    }
    for ty in repeat each types {
        print("dynamic: \(ty)")
    }
}

class Base {}
class SubA: Base {}
class SubB: Base {}

printStaticAndDynamicTypes(SubA.self, SubB.self)

print("---")

let types: [Base.Type] = [SubA.self, SubB.self]
printStaticAndDynamicTypes(types[0], types[1])
2 Likes
14

Yep, they're different kinds of types. (You can open an existential metatype to get a concrete metatype out, but that's not the same thing as passing it directly.) The confusing overloaded spelling of existential vs. concrete metatypes strikes again...

1 Like
15

Speaking of opening existentials, I would like to remove duplicates from the parameter pack before passing it to the Table constructor. I thought maybe implicitly opened existentials would let me turn an array of type values back into a parameter pack, but alas:

struct AnyTable {
    let storage: AnyObject
    
    init<each F: Field>(fieldTypes: repeat (each F).Type) {
        var canonicalTypes = [Any.Type]()
        for fieldType in repeat each fieldTypes {
            if !canonicalTypes.contains(where: { ObjectIdentifier($0) == ObjectIdentifier(fieldType) }) {
                fieldTypes.append(fieldTypes)
            }
        }
        
        storage = _makeStorage(fieldTypes)
//                             ^ error: Cannot convert value of type '[any Any.Type]' to expected argument type '_.Type'
//                ^ error: Could not infer pack element #0 from context
    }
    
    func _makeStorage<each F: Field>(_ fieldTypes: repeat (each F).Type) -> ManagedBuffer<(), (repeat each F)> {
        return ManagedBuffer<(), (repeat each F)>.create(minimumCapacity: 1) { _ in () }
    }
}

This obviously shouldn’t compile as written, but ideally I could put a ... somewhere and it would work.

16

Welp, it looks like ManagedBuffer isn’t going to work out for my needs, since I can’t figure out how to alter the parameter pack I pass to the Elements type parameter. And since ManagedRawBuffer doesn’t exist yet, that means I am left rolling my own storage based on UnsafeRawBufferPointer.

Since the layout can be determined dynamically, and I want to offer field-level access, I will need to store the offset of each field. @Slava_Pestov’s comment implied this might require writing a type-erased wrapper around Field.Type, since metatypes conforming to Field are not subtypes of the Field.Type existential. But the introduction of explicit any has given us the ability to spell a different type which seems to fit the bill:

protocol Field: BitwiseCopyable { }
struct FirstName: Field { }
struct LastName: Field { }

struct Table {
  let storage: UnsafeRawBufferPointer
  let fields: [any (Field.Type)]
  let offsets: [Int]

struct Table {
  // TODO: move this to a class so it can be refcounted (and ideally tail-allocated)
  var storage: UnsafeMutableRawBufferPointer
  var fields: [any (Field.Type)] = .init()
  var offsets: [Int]

  func _stride<F: Field>(of field: F.Type) -> (stride: Int) { MemoryLayout<F>.stride }

  init<each F: Field>(fields fieldTypes: repeat (each F).Type) {
    var lastFieldEnd = 0
    for fieldType in repeat each fieldTypes {
        fields.append(fieldType)
        let offset = lastFieldEnd // TODO: round up to respect alignment of this field type
        offsets.append(lastFieldEnd)
        lastFieldEnd += _stride(of: fieldType)
    }
    // TODO: compute offset of and initialize storage buffer
  }
}

One surprise was the need to explicitly specify the type of fields. If I try to use type inference from the initialization expression, the compiler interprets any as referring to the SIMD.any function:

var fields = [any (Field.Type)]()
//                 ^ error: cannot convert value of type '(any Field.Type).Type' to expected argument type 'SIMDMask<Storage>'
//            ^ error: generic parameter 'Storage' could not be inferred
//            ^ error: cannot call value of non-function type '[Bool]'

Edit: one more weird thing… the _stride(of:) function doesn’t work. But I can fall back to the old approach of declaring static properties an extension on Field and calling them on an instance of any (Field.Type):

extension Field {
  var stride: Int { MemoryLayout<Self>.stride }
}

/* ... */

    init<each F: Field>(fields fieldTypes: repeat (each F).Type) {
        for fieldType in repeat each fieldTypes {
            lastFieldEnd += fieldType.stride // this works!
    }

I guess this is a bug in implicitly opened existentials?

17

Metatypes conforming to Field are subtypes of Field.Type. This type is also spelled as any (Field.Type), with the parens optional.

The lack of subtyping is with protocol metatypes, which are useless anyway because they can’t express anything interesting. So (any Collection).Type (note the parens) is not a subtype of (any Sequence).Type.

18

I was thoroughly confused by this error, especially in light of your explanation:

protocol Field { }
struct S1: Field { }
struct FieldDescription { 
    var type: any (Field.Type) 
    private static func _stride<F: Field>(of fieldType: F.Type) -> Int { MemoryLayout<F>.stride } 
    var stride: Int { FieldDescription._stride(of: type) } 
    // error: cannot convert value of type 'any Field.Type' to expected argument type 'F.Type'
    // error: generic parameter 'F' could not be inferred
}

I thought this had to be a sign that any (Field.Type) was not an existential. But it turns out that if I rename type to ty, everything works! type must be resolving to the unapplied function type(of:) in this expression. Oddly, this doesn’t happen in a standalone function or in top-level code. I don‘t know why the lookup rules inside of a property getter would match Swift.type(of:) before self.type.

19

The lookup rules are fine, but they produce an overload set with two overloads, your type property and the type(of:) function. There is a longstanding bug in the SE-0352 implementation where existential opening doesn’t always work in combination with overloading. The workaround is to refactor the expression to eliminate the overloading, which renaming the property achieves.

2 Likes