[Returned for revision] SE-0379: Opt-in Reflection Metadata

Thanks everybody for participating in the SE-0379 review. The Language Workgroup has decided to return this proposal for revision. We accept in principle that it should be possible for developers with code size and/or secrecy concerns to minimize the amount of reflection metadata they ship with their Swift binaries. We like the proposed design of having a Reflectable generic constraint, since it allows both for individual types to declare themselves as always providing full reflection information and for libraries to vend protocols which require all conforming types to provide full reflection information. Changing the compiler's behavior to choose between emitting all reflection metadata and only opted-in metadata, instead of the current all-or-nothing switch, is also a clear improvement over the status quo, as the current "disable reflection metadata" behavior is becoming less tenable over time as libraries start to take more advantage of Swift's reflection facilities. We'd like to see additional design discussion about the following aspects of the proposal:

Changing the default behavior of reflection generation

The proposal specifies that, under Swift 6 language mode, the default behavior for reflection ought to become opt-in. Community response was mixed toward this aspect of the proposal, with many developers raising concerns about the effect this would have on pervasive use of print, Mirror, and other reflection-based APIs in existing code. There was good philosophical discussion about the tension between favoring a "pay-for-what-you-use" model and providing rich runtime facilities. The proposal suggests resolving this tension by making APIs that rely on reflection add a Reflectable constraint to their arguments. However, this is a problem for many APIs, particularly standard library facilities like print and related stringification features, that are intended to make a best effort to work on any value. Requiring generic constraints to be threaded through potentially many layers of API just to add some logging or printf debugging would be a serious imposition. Eliminating the need for these constraints, however, leaves the compiler with no breadcrumbs to assist developers in making all types Reflectable or Custom*StringConvertible that need to be; developers would have to discover and repair the runtime impact with a lot of dedicated testing.

The Language Workgroup is also mindful of prior attempts by the Swift project to retroactively change runtime behavior, particularly SE-0083, which would have simplified dynamic casting behavior by removing Foundation type bridging from the runtime. The Core Team at the time ultimately rejected that proposal because, given the dynamic nature of the proposal, they could not come up with a method of assessing the impact on the Swift ecosystem with enough certainty to feel that the change would not be overly disruptive. The Language Workgroup has similar concerns with changing the default reflection behavior; even if the default change is gated on language version, we want developers to adopt Swift 6 to get the benefits of static concurrency safety and other changes, and expecting them to also audit the dynamic behavior of their projects for unexpected reflection dependencies would be a roadblock to updating their language version mode. Without a plan to assess the real-world impact of changing the default, we are not comfortable with accepting that change.

The nature of Reflectable, and dynamic casting support

The proposal specifies that Reflectable is a marker protocol, like Sendable; however, unlike a true marker protocol, the presence of the constraint has a real runtime impact on what type metadata is available to the running program. Also, unlike conforming to a protocol, reflection metadata cannot be retroactively added to a type by an extension outside of its defining module. So although having it as a generic constraint is a good language design, calling it a marker protocol, or a protocol at all, doesn't seem quite right.

The proposal also specifies that the programmer can query whether a type carries full reflection metadata by dynamic casting as? Reflectable. If Reflectable is a marker protocol, then it isn't clear how this might work in full generality, since although we might statically recognize the expression x as? Reflectable, it wouldn't be possible to handle x as? T for a generic type parameter T bound to the Reflectable existential type at runtime. However, the Language Workgroup also feels that this functionality might be better expressed as a standalone query function in the new Reflection library, which reduces the complexity of it having to interact with the entire dynamic cast infrastructure.

There is an existing non-protocol generic constraint in the language today—the AnyObject constraint, which only classes and class existentials with no witness tables can satisfy. The compiler classifies this internally as a "layout constraint", since it doesn't require explicit conformance but is implicitly satisfied by types whose layout intrinsically satisfies the requirements of the constraint. There are some other layout constraints currently only implemented for the optimizer to allow for partial specialization, but some of these, particularly a constraint for bitwise-copyable types which would be called "trivial" or "POD" in C++ jargon, could also be surfaced in the language. Possibly Reflectable could fit into this family as well, since it doesn't require explicit "conformance" when the compiler is set to emit all reflection metadata; although it isn't a constraint specifically on the layout of values of the type, it does impose a constraint on the layout of the metadata for the type.

Debugger support

One aspect of the implementation that needs further consideration is its interactions with the debugger. LLDB's Swift support currently relies heavily on full reflection metadata to provide full functionality, and the Language Workgroup would like to see a plan to ensure that debug support is available for binaries that aren't deployed with full reflection metadata, when that information can be recovered by the developers independent of the binary using something like Apple's dSYM bundles or Windows pdb files. The proposal specifies that the -g flag changes compiler behavior to emit full reflection metadata, but that effect would disrupt some developers' workflows, who rely on -g having no effect on the resulting binary except for the presence of debug sections, and could also confuse developers who might test with debug builds, observe behavior that relies on full reflection metadata, and then see the behavior break when a release build is performed. Providing full reflection metadata on behalf of the debugger is the right track, but we'd like to see the compiler and lldb work on developing a way for the debugger to import that full reflection metadata from outside of the main binary.

26 Likes