Preparing metadata records for ABI stability

I’m trying to plan out changes to our type metadata record formats for ABI stability. I’ll start by looking at the current situation and then make suggestions about things we ought to change. We want to settle on a design that leaves room for future expansion and runtime changes, and still allows efficient access to the most frequently-accessed parts of metadata. I’ll be looking exclusively at metadata records themselves for this message, leaving other data structures for separate scrutiny. I'd appreciate all your feedback.

ABI concerns for type metadata records

These are the primary ways in which the compiler and runtime are exposed to direct binary layout of metadata records:

The compiler generates metadata records for some types, either as static data that’s expected to be directly usable as a type metadata record, or a pattern for a metadata record that’s fed as input to an instantiation process.
The compiler generates code that interacts with metadata records. It generates metadata accesses to form the metadata pointer for a specific type. It also projects information out of metadata records of a known kind. This can take the form either of runtime calls or of direct projection into known offsets inside the metadata record, making a tradeoff between abstracting binary layout details and performance of generated code.
1. Compiler-generated metadata records

For concrete struct and enum types, the compiler generates a metadata record as static data. If the type has known layout (no resilient fields), then the metadata record is expected to be usable as valid metadata as-is. If the type has unknown layout, the metadata record requires one-time initialization to become valid metadata.
For concrete class types, the compiler generates a metadata record as static data. If the class is @objc, a pointer into this metadata record is also exported as the Objective-C class symbol for Objective-C binaries to link against. Class metadata records always require one-time initialization to become valid metadata because of, at minimum, Objective-C class realization. (On platforms without Objective-C interop, a concrete class with a fully concrete, non-resilient ancestry and no resilient fields could be made usable as-is as valid metadata.)
For generic types, the compiler generates a generic metadata pattern. The swift_getGenericMetadata runtime function takes a pointer to a generic metadata pattern and a list of generic arguments and produces a valid metadata record for the generic type instantiated with those arguments.
For Clang-imported types, the compiler generates a metadata candidate. This looks like a metadata record, with an added uniquing prefix. The swift_getForeignMetadata runtime function picks a metadata record to be the canonical metadata pointer, and performs one-time initialization if required.
2. Metadata code generation

ABI concerns for all metadata

The compiler makes the following assumptions about all type metadata records:

Every formal Swift type has a corresponding type metadata record at a unique address, so pointer identity can be used to test type identity. metadataPointerA == metadataPointerB if and only if metadataPointerA and metadataPointerB represent the exact same formal type.
The value witness table can be loaded at fixed offset -1*sizeof(Int) from the address point.
Classes

The compiler accesses concrete class metadata by calling swift_getInitializedObjCClass to perform one-time initialization of the metadata. Generic class metadata is instantiated usingswift_getGenericMetadata, and the template is responsible for initialization of the generated class object.

A subclass’s metadata record serves as a physical subtype of its parent class metadata record. Any projection pattern that works on the parent class metadata should also work equivalently on the subclass metadata, allowing for overrides of methods and other entries where the subclass customizes behavior.

For all classes, the compiler expects access to the following fields:

Information Projection strategy
Destructor fixed offset -2*sizeof(Int)
ObjC matter fixed offset (0...4)*sizeof(Int)
Nominal type descriptor [1]
Generic arguments [1]
Stored property offsets [1]
[1] For classes with fully-fragile ancestry, generic arguments and stored property offsets can be loaded at a fixed offset determined by the compiler. If the class has any resilient ancestry, such as a base class from another module, then the base offset into the subclass’s own entries must be determined at instantiation time and loaded indirectly.

For classes with fully-fragile ancestry, the compiler can also emit fixed offset accesses for vtable entries.

The metadata currently includes field offsets for all stored properties, but this should be changed to include only the fields with offsets that are dependent on the struct’s generic arguments.

Structs

The compiler accesses concrete, known-layout struct metadata by direct reference to the global symbol. If the struct contains resilient fields, the metadata is accessed through a runtime call for one-time initialization. Generic struct metadata is instantiatied using swift_getGenericMetadata.

For all structs, the compiler expects access to the following fields:

Information Projection strategy
Nominal type descriptor fixed offset, compiler-determined
Generic arguments fixed offset, compiler-determined
Stored property offsets fixed offset, compiler-determined
The metadata currently includes field offsets for all stored properties, but this should be changed to include only the fields with offsets that are dependent on the struct’s generic arguments.

Enums

The compiler accesses concrete, known-layout enum metadata by direct reference to the global symbol. If the enum contains resilient payloads, the metadata is accessed through a runtime call for one-time initialization. Generic enum metadata is instantiatied using swift_getGenericMetadata.

For all enums, the compiler expects access to the following fields:

Information Projection strategy
Nominal type descriptor fixed offset, compiler-determined
Generic arguments fixed offset, compiler-determined
Tuples

Tuple metadata records are accessed by calling the swift_getTupleTypeMetadata* runtime functions.

The compiler expects access to the following fields:

Information Projection strategy
Element offsets fixed offset
If we wanted to be able to satisfy type metadata requirements for element types from tuple metadata, we could also provide access to:

Information Projection strategy
Element types TBD
Functions

Function metadata records are accessed by calling the swift_get*FunctionTypeMetadata* runtime functions.

The compiler does not currently emit any projections into function type metadata.

If we wanted to be able to satisfy type metadata requirements for input or output types from function metadata, we could also provide access to:

Information Projection strategy
Return type TBD
Argument types TBD
Existentials, existential metatypes

Existential metadata records are accessed by calling the swift_getExistentialTypeMetadata* runtime functions.

The compiler does not currently generate any code that reaches into existential metadata records.

If we wanted to be able to satisfy type metadata requirements for input or output types from function metadata, we could also provide access to:

Information Projection strategy
Generic signature TBD
Metatypes

Metatype metadata records are accessed by calling the swift_getMetatypeMetadata function.

The compiler does not currently generate any code that reaches into metatype metadata records.

If we wanted to be able to satisfy type metadata requirements for the instance type from metatype metadata, we could also provide access to:

Information Projection strategy
Instance type TBD
Objective-C wrappers

Class objects for natively-Objective-C classes are not Swift metadata by themselves, so need a wrapper metadata record. This wrapper metadata is only produced by the Swift runtime (and could be produced more efficiently by future cooperation with the ObjC runtime). The compiler accesses Objective-C class metadata by calling the swift_getObjCClassMetadata function, which may return either a wrapper or pass through the input class object if it is already valid metadata.

The compiler generates code against the following:

Information Projection strategy
Class object runtime call swift_getObjCClassFromMetadata
Core Foundation class

Foreign class metadata records are generated by Swift’s Clang importer for imported Core Foundation class types. The records require runtime canonicalization to determine which metadata record uniquely identifies the type, since there is no home Swift module for the CF type. The compiler does not currently generate code against any fields of the metadata.

Recommended changes

Abstract away fixed offsets across modules and for structural metadata

Anything we lock down fixed offsets for in the ABI should justify itself by being necessary for performance or code size reasons. Unspecialized code relies heavily on:

the value witness table
stored property offsets
generic arguments
and these are the things that nominal type metadata currently makes directly available, for the most part. Anything else we emit direct access to (such as nominal type descriptors) should definitely be abstracted behind a runtime call. For structural type metadata (tuples, functions, existentials, metatypes), we already do so for most of the layout details, with the one exception of tuple element offsets, which are exposed for performance of unspecialized code.

For concrete nominal types, the compiler also currently generates metadata records that are expected to be directly usable as metadata. This is a nice code size optimization that’d be unfortunate to give up. At least for internal or private types, the emitted metadata format and all of the code that ought to be reaching into it are together in one resilience boundary, so one could say that the precise metadata record format for internal value types is a private contract of the compiler. However, even for public fragile types, it would be beneficial still to be able to directly export the symbol as usable metadata. For the performance-sensitive field offset and generic argument vectors, we could nonetheless avoid hard-coding offsets in the compiler. We could still guarantee that the generic arguments followed by (fragile) stored property offsets are stored contiguously, and abstract the base offset to the contiguous data structure. Possibilities for accessing the base offset include:

Require a runtime call to get it
Place the base offset somewhere else in the metadata
Export the base offset as a separate symbol (possibly an absolute symbol, if it’s knowable at compile time for the home module, or a global variable, if it requires runtime computation).
For classes with resilient ancestry, we already must have an approach that allows the offset to be computed at instantiation time and efficiently accessed in unspecialized code afterward. Technique (3) is proven by the Objective-C runtime and should be sufficient for our needs. It would impose a two- or three-instruction cost per type to load the base offset, which would sit on the dependency chain for any generic type or field offset loads based on that base offset, but would give us the flexibility to add new information to metadata records in future compilers.

Make most metadata kinds private to the runtime

The compiler does not emit any code that looks at metadata kinds for projection purposes. It does, however, use metadata kinds when it builds metadata records for nominal types. However, the metadata kind is already redundantly coded in the nominal type descriptor, so we could conceivably reduce the exposure of metadata kinds to a single “value type” kind (in addition to the “isa-pointer-in-the-kind-field-means-class” kind, imposed by ObjC compatibility).

-Joe

1 Like

I’m trying to plan out changes to our type metadata record formats for ABI stability. I’ll start by looking at the current situation and then make suggestions about things we ought to change. We want to settle on a design that leaves room for future expansion and runtime changes, and still allows efficient access to the most frequently-accessed parts of metadata. I’ll be looking exclusively at metadata records themselves for this message, leaving other data structures for separate scrutiny. I'd appreciate all your feedback.

Overall, I think this is a nice description of the problem. A few comments:

  - One notable omission here is that you don't discuss nominal type descriptors at all.

  - For the structural types (tuples, functions, existentials, metatypes), I don't think there's any particular reason to abstract access to the component types. If there are future changes to these types that require additional data to be stored, I don't see any reason that that couldn't be stored compatibly with the current layout. We should make sure that all of these metadata have some space to encode flags and that reads of those flags are appropriately delimited, as opposed to e.g. assuming that an entire word is used to store a single enum.

  - I think having a single "value type" metadata kind for enums / structs makes sense, assuming we can discriminate them via the NTD.

  - I feel like we're not boxing ourselves in too much to assume a layout of the generic requirements. Having a base offset already imposes a pretty steep compatibility requirement — e.g. even if we significantly generalized a type's generic signature, we would still need to store old requirements (when actually obeyed) in the appropriate places for the old signature, or else the old pattern just doesn't work at all. As long as we can extend that with more information later, we don't really suffer from making layout assumptions today.

John.

···

On Nov 7, 2017, at 8:58 PM, Joe Groff via swift-dev <swift-dev@swift.org> wrote:
Tuple metadata records are accessed by calling the swift_getTupleTypeMetadata* runtime functions.

The compiler expects access to the following fields:

Information Projection strategy
Element offsets fixed offset
If we wanted to be able to satisfy type metadata requirements for element types from tuple metadata, we could also provide access to:

Information Projection strategy
Element types TBD
Functions

Function metadata records are accessed by calling the swift_get*FunctionTypeMetadata* runtime functions.

The compiler does not currently emit any projections into function type metadata.

If we wanted to be able to satisfy type metadata requirements for input or output types from function metadata, we could also provide access to:

Information Projection strategy
Return type TBD
Argument types TBD
Existentials, existential metatypes

Existential metadata records are accessed by calling the swift_getExistentialTypeMetadata* runtime functions.

The compiler does not currently generate any code that reaches into existential metadata records.

If we wanted to be able to satisfy type metadata requirements for input or output types from function metadata, we could also provide access to:

Information Projection strategy
Generic signature TBD
Metatypes

Metatype metadata records are accessed by calling the swift_getMetatypeMetadata function.

The compiler does not currently generate any code that reaches into metatype metadata records.

If we wanted to be able to satisfy type metadata requirements for the instance type from metatype metadata, we could also provide access to:

Information Projection strategy
Instance type TBD
Objective-C wrappers

Class objects for natively-Objective-C classes are not Swift metadata by themselves, so need a wrapper metadata record. This wrapper metadata is only produced by the Swift runtime (and could be produced more efficiently by future cooperation with the ObjC runtime). The compiler accesses Objective-C class metadata by calling the swift_getObjCClassMetadata function, which may return either a wrapper or pass through the input class object if it is already valid metadata.

The compiler generates code against the following:

Information Projection strategy
Class object runtime call swift_getObjCClassFromMetadata
Core Foundation class

Foreign class metadata records are generated by Swift’s Clang importer for imported Core Foundation class types. The records require runtime canonicalization to determine which metadata record uniquely identifies the type, since there is no home Swift module for the CF type. The compiler does not currently generate code against any fields of the metadata.

Recommended changes

Abstract away fixed offsets across modules and for structural metadata

Anything we lock down fixed offsets for in the ABI should justify itself by being necessary for performance or code size reasons. Unspecialized code relies heavily on:

the value witness table
stored property offsets
generic arguments
and these are the things that nominal type metadata currently makes directly available, for the most part. Anything else we emit direct access to (such as nominal type descriptors) should definitely be abstracted behind a runtime call. For structural type metadata (tuples, functions, existentials, metatypes), we already do so for most of the layout details, with the one exception of tuple element offsets, which are exposed for performance of unspecialized code.

For concrete nominal types, the compiler also currently generates metadata records that are expected to be directly usable as metadata. This is a nice code size optimization that’d be unfortunate to give up. At least for internal or private types, the emitted metadata format and all of the code that ought to be reaching into it are together in one resilience boundary, so one could say that the precise metadata record format for internal value types is a private contract of the compiler. However, even for public fragile types, it would be beneficial still to be able to directly export the symbol as usable metadata. For the performance-sensitive field offset and generic argument vectors, we could nonetheless avoid hard-coding offsets in the compiler. We could still guarantee that the generic arguments followed by (fragile) stored property offsets are stored contiguously, and abstract the base offset to the contiguous data structure. Possibilities for accessing the base offset include:

Require a runtime call to get it
Place the base offset somewhere else in the metadata
Export the base offset as a separate symbol (possibly an absolute symbol, if it’s knowable at compile time for the home module, or a global variable, if it requires runtime computation).
For classes with resilient ancestry, we already must have an approach that allows the offset to be computed at instantiation time and efficiently accessed in unspecialized code afterward. Technique (3) is proven by the Objective-C runtime and should be sufficient for our needs. It would impose a two- or three-instruction cost per type to load the base offset, which would sit on the dependency chain for any generic type or field offset loads based on that base offset, but would give us the flexibility to add new information to metadata records in future compilers.

Make most metadata kinds private to the runtime

The compiler does not emit any code that looks at metadata kinds for projection purposes. It does, however, use metadata kinds when it builds metadata records for nominal types. However, the metadata kind is already redundantly coded in the nominal type descriptor, so we could conceivably reduce the exposure of metadata kinds to a single “value type” kind (in addition to the “isa-pointer-in-the-kind-field-means-class” kind, imposed by ObjC compatibility).

-Joe
_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Do you mean if we significantly generalized the schema for generic requirements, or if we changed the definition of a single type to be more general? I would rather the latter did not happen.

Accessing generic arguments efficiently is important for extension methods, I think. In fact right now even thin functions recover metadata from a thick metatype argument if its the last parameter, although perhaps this is not intentional.

Slava

···

On Nov 13, 2017, at 11:40 AM, John McCall via swift-dev <swift-dev@swift.org> wrote:

  - I feel like we're not boxing ourselves in too much to assume a layout of the generic requirements. Having a base offset already imposes a pretty steep compatibility requirement — e.g. even if we significantly generalized a type's generic signature, we would still need to store old requirements (when actually obeyed) in the appropriate places for the old signature, or else the old pattern just doesn't work at all. As long as we can extend that with more information later, we don't really suffer from making layout assumptions today.

I’m trying to plan out changes to our type metadata record formats for ABI stability. I’ll start by looking at the current situation and then make suggestions about things we ought to change. We want to settle on a design that leaves room for future expansion and runtime changes, and still allows efficient access to the most frequently-accessed parts of metadata. I’ll be looking exclusively at metadata records themselves for this message, leaving other data structures for separate scrutiny. I'd appreciate all your feedback.

Overall, I think this is a nice description of the problem. A few comments:

  - One notable omission here is that you don't discuss nominal type descriptors at all.

Yeah, sorry if I didn't make that clear that that was intentional. I'm working on a plan for renovating nominal type descriptors in a separate document.

  - For the structural types (tuples, functions, existentials, metatypes), I don't think there's any particular reason to abstract access to the component types. If there are future changes to these types that require additional data to be stored, I don't see any reason that that couldn't be stored compatibly with the current layout. We should make sure that all of these metadata have some space to encode flags and that reads of those flags are appropriately delimited, as opposed to e.g. assuming that an entire word is used to store a single enum.

Maybe not. I can think of a few possible evolution avenues to keep room for:

- Some of these could conceivably become nominal types at some point. Tuple<T...> and Metatype<T> structs seem conceivable at some point in the future, and it might be nice if we can move the structural metadata formats over to nominal metadata at some point in the future. It may be sufficient to ensure that tuple metadata lays out its element type and field offset vectors in a way that's compatible with how a generic struct would.
- The shape of existential type metadata could change quite a bit to support generalized existentials. (This isn't too much of an ABI problem yet, maybe, since there isn't really much type information to project from existential metadata in its current form.)
- Functions have tended to grow lots of new top-level and per-argument bits. We'll want to ensure there's room for those (and it looks like Pavel's latest patch does).

  - I think having a single "value type" metadata kind for enums / structs makes sense, assuming we can discriminate them via the NTD.

Agreed.

  - I feel like we're not boxing ourselves in too much to assume a layout of the generic requirements. Having a base offset already imposes a pretty steep compatibility requirement — e.g. even if we significantly generalized a type's generic signature, we would still need to store old requirements (when actually obeyed) in the appropriate places for the old signature, or else the old pattern just doesn't work at all. As long as we can extend that with more information later, we don't really suffer from making layout assumptions today.

Sounds good. If, for some reason we needed to move the base offset of the generic argument structure to make room for new fields, we could conceivably ABI-version-gate the presence of those fields. Resilient types compiled with an older ABI would just have to either forgo those fields or fall into slower access paths to extract them.

-Joe

···

On Nov 13, 2017, at 11:40 AM, John McCall <rjmccall@apple.com> wrote:

On Nov 7, 2017, at 8:58 PM, Joe Groff via swift-dev <swift-dev@swift.org <mailto:swift-dev@swift.org>> wrote:

  - I feel like we're not boxing ourselves in too much to assume a layout of the generic requirements. Having a base offset already imposes a pretty steep compatibility requirement — e.g. even if we significantly generalized a type's generic signature, we would still need to store old requirements (when actually obeyed) in the appropriate places for the old signature, or else the old pattern just doesn't work at all. As long as we can extend that with more information later, we don't really suffer from making layout assumptions today.

Do you mean if we significantly generalized the schema for generic requirements, or if we changed the definition of a single type to be more general? I would rather the latter did not happen.

I mean changing the definition of a type to be more general, and I think it's probably inevitable. I mean, we're already anticipating at least generalizing things to accept move-only types, although that at least has the advantage of not corresponding to a change in runtime protocol conformances.

Accessing generic arguments efficiently is important for extension methods, I think. In fact right now even thin functions recover metadata from a thick metatype argument if its the last parameter, although perhaps this is not intentional.

We recover metadata from all arguments, I think, unless it's a protocol witness.

John.

···

On Nov 13, 2017, at 2:42 PM, Slava Pestov <spestov@apple.com> wrote:

On Nov 13, 2017, at 11:40 AM, John McCall via swift-dev <swift-dev@swift.org <mailto:swift-dev@swift.org>> wrote:

I’m trying to plan out changes to our type metadata record formats for ABI stability. I’ll start by looking at the current situation and then make suggestions about things we ought to change. We want to settle on a design that leaves room for future expansion and runtime changes, and still allows efficient access to the most frequently-accessed parts of metadata. I’ll be looking exclusively at metadata records themselves for this message, leaving other data structures for separate scrutiny. I'd appreciate all your feedback.

Overall, I think this is a nice description of the problem. A few comments:

  - One notable omission here is that you don't discuss nominal type descriptors at all.

Yeah, sorry if I didn't make that clear that that was intentional. I'm working on a plan for renovating nominal type descriptors in a separate document.

Okay, great. I guess that's what you meant by "other data structures" above, sorry.

  - For the structural types (tuples, functions, existentials, metatypes), I don't think there's any particular reason to abstract access to the component types. If there are future changes to these types that require additional data to be stored, I don't see any reason that that couldn't be stored compatibly with the current layout. We should make sure that all of these metadata have some space to encode flags and that reads of those flags are appropriately delimited, as opposed to e.g. assuming that an entire word is used to store a single enum.

Maybe not. I can think of a few possible evolution avenues to keep room for:

- Some of these could conceivably become nominal types at some point. Tuple<T...> and Metatype<T> structs seem conceivable at some point in the future, and it might be nice if we can move the structural metadata formats over to nominal metadata at some point in the future. It may be sufficient to ensure that tuple metadata lays out its element type and field offset vectors in a way that's compatible with how a generic struct would.

Much like Optional has a special kind, I don't think it's unreasonable to say that special structural types could always have a special layout.

- The shape of existential type metadata could change quite a bit to support generalized existentials. (This isn't too much of an ABI problem yet, maybe, since there isn't really much type information to project from existential metadata in its current form.)

We should at least leave space for this, I agree. But again, if there are cases we can't express with the current format, we could certainly just create a new kind + format for them. We don't have compatibility requirements for types that aren't expressible in the current type system.

- Functions have tended to grow lots of new top-level and per-argument bits. We'll want to ensure there's room for those (and it looks like Pavel's latest patch does).

Yeah, Pavel and I have been working on that.

  - I think having a single "value type" metadata kind for enums / structs makes sense, assuming we can discriminate them via the NTD.

Agreed.

  - I feel like we're not boxing ourselves in too much to assume a layout of the generic requirements. Having a base offset already imposes a pretty steep compatibility requirement — e.g. even if we significantly generalized a type's generic signature, we would still need to store old requirements (when actually obeyed) in the appropriate places for the old signature, or else the old pattern just doesn't work at all. As long as we can extend that with more information later, we don't really suffer from making layout assumptions today.

Sounds good. If, for some reason we needed to move the base offset of the generic argument structure to make room for new fields, we could conceivably ABI-version-gate the presence of those fields. Resilient types compiled with an older ABI would just have to either forgo those fields or fall into slower access paths to extract them.

Agreed.

John.

···

On Nov 13, 2017, at 4:30 PM, Joe Groff <jgroff@apple.com> wrote:
On Nov 13, 2017, at 11:40 AM, John McCall <rjmccall@apple.com <mailto:rjmccall@apple.com>> wrote:

On Nov 7, 2017, at 8:58 PM, Joe Groff via swift-dev <swift-dev@swift.org <mailto:swift-dev@swift.org>> wrote:

Sure. Optional's special kind does however still act as a physical subtype of general enum metadata layout.

-Joe

···

On Nov 13, 2017, at 2:16 PM, John McCall <rjmccall@apple.com> wrote:

On Nov 13, 2017, at 4:30 PM, Joe Groff <jgroff@apple.com <mailto:jgroff@apple.com>> wrote:

On Nov 13, 2017, at 11:40 AM, John McCall <rjmccall@apple.com <mailto:rjmccall@apple.com>> wrote:

On Nov 7, 2017, at 8:58 PM, Joe Groff via swift-dev <swift-dev@swift.org <mailto:swift-dev@swift.org>> wrote:

I’m trying to plan out changes to our type metadata record formats for ABI stability. I’ll start by looking at the current situation and then make suggestions about things we ought to change. We want to settle on a design that leaves room for future expansion and runtime changes, and still allows efficient access to the most frequently-accessed parts of metadata. I’ll be looking exclusively at metadata records themselves for this message, leaving other data structures for separate scrutiny. I'd appreciate all your feedback.

Overall, I think this is a nice description of the problem. A few comments:

  - One notable omission here is that you don't discuss nominal type descriptors at all.

Yeah, sorry if I didn't make that clear that that was intentional. I'm working on a plan for renovating nominal type descriptors in a separate document.

Okay, great. I guess that's what you meant by "other data structures" above, sorry.

  - For the structural types (tuples, functions, existentials, metatypes), I don't think there's any particular reason to abstract access to the component types. If there are future changes to these types that require additional data to be stored, I don't see any reason that that couldn't be stored compatibly with the current layout. We should make sure that all of these metadata have some space to encode flags and that reads of those flags are appropriately delimited, as opposed to e.g. assuming that an entire word is used to store a single enum.

Maybe not. I can think of a few possible evolution avenues to keep room for:

- Some of these could conceivably become nominal types at some point. Tuple<T...> and Metatype<T> structs seem conceivable at some point in the future, and it might be nice if we can move the structural metadata formats over to nominal metadata at some point in the future. It may be sufficient to ensure that tuple metadata lays out its element type and field offset vectors in a way that's compatible with how a generic struct would.

Much like Optional has a special kind, I don't think it's unreasonable to say that special structural types could always have a special layout.

True. We could change that, of course.

John.

···

On Nov 13, 2017, at 5:19 PM, Joe Groff <jgroff@apple.com> wrote:

On Nov 13, 2017, at 2:16 PM, John McCall <rjmccall@apple.com <mailto:rjmccall@apple.com>> wrote:

On Nov 13, 2017, at 4:30 PM, Joe Groff <jgroff@apple.com <mailto:jgroff@apple.com>> wrote:

On Nov 13, 2017, at 11:40 AM, John McCall <rjmccall@apple.com <mailto:rjmccall@apple.com>> wrote:

On Nov 7, 2017, at 8:58 PM, Joe Groff via swift-dev <swift-dev@swift.org <mailto:swift-dev@swift.org>> wrote:

I’m trying to plan out changes to our type metadata record formats for ABI stability. I’ll start by looking at the current situation and then make suggestions about things we ought to change. We want to settle on a design that leaves room for future expansion and runtime changes, and still allows efficient access to the most frequently-accessed parts of metadata. I’ll be looking exclusively at metadata records themselves for this message, leaving other data structures for separate scrutiny. I'd appreciate all your feedback.

Overall, I think this is a nice description of the problem. A few comments:

  - One notable omission here is that you don't discuss nominal type descriptors at all.

Yeah, sorry if I didn't make that clear that that was intentional. I'm working on a plan for renovating nominal type descriptors in a separate document.

Okay, great. I guess that's what you meant by "other data structures" above, sorry.

  - For the structural types (tuples, functions, existentials, metatypes), I don't think there's any particular reason to abstract access to the component types. If there are future changes to these types that require additional data to be stored, I don't see any reason that that couldn't be stored compatibly with the current layout. We should make sure that all of these metadata have some space to encode flags and that reads of those flags are appropriately delimited, as opposed to e.g. assuming that an entire word is used to store a single enum.

Maybe not. I can think of a few possible evolution avenues to keep room for:

- Some of these could conceivably become nominal types at some point. Tuple<T...> and Metatype<T> structs seem conceivable at some point in the future, and it might be nice if we can move the structural metadata formats over to nominal metadata at some point in the future. It may be sufficient to ensure that tuple metadata lays out its element type and field offset vectors in a way that's compatible with how a generic struct would.

Much like Optional has a special kind, I don't think it's unreasonable to say that special structural types could always have a special layout.

Sure. Optional's special kind does however still act as a physical subtype of general enum metadata layout.