Pitch #3: Opt-in Reflection metadata

It's been a while since the last iteration on the Opt-In Reflection metadata proposal, but we've been working with @Joe_Groff, @ctp and @mren on the updated version recently and while finishing the last bits, we would like to get the community's feedback on the new pieces in the proposal.

TL;DR

Reflectable is a marker protocol now and developers can express the dependency on Reflection metadata as a generic requirement.
We decided not to pursue Reflection symbols stripability and instead focus on conformance to Reflectable protocol based on which the compiler will emit Reflection metadata.

Because of the introduction of generic requirements, the type-checker will ensure that a type conforms to Reflectable if it is used on an API that consumes reflection.

Swift Opt-In Reflection Metadata

Introduction

This proposal seeks to increase the safety and efficiency of Swift Reflection Metadata by improving the existing mechanism and providing the opportunity to express a requirement on Reflection Metadata in APIs that consume it.

Motivation

APIs can use Reflection Metadata differently. Some like print, and dump will still work with disabled reflection, but the output will be limited. Others, like SwiftUI rely on it and won't work correctly if the reflection metadata is missing.
While the former can potentially benefit as well, the main focus of this proposal is on the latter.

A developer can mistakenly turn off Reflection Metadata for a Swift module and won't be warned at compile-time if APIs that consume reflection are used by that module. An app with such a module won't behave as expected at runtime which may be challenging to notice and track down such bugs back to Reflection. For instance, SwiftUI implementation uses reflection metadata from user modules to trigger re-rendering of the view hierarchy when a state has changed. If for some reason a user module was compiled with metadata generation disabled, changing the state won't trigger that behaviour and will cause inconsistency between state and representation which will make such API less safe since it becomes a runtime issue rather than a compile-time one.

On the other hand, excessive Reflection metadata may be preserved in a binary even if not used, because there is currently no way to statically determine its usage. There was an attempt to limit the amount of unused reflection metadata by improving its stripability by the Dead Code Elimination LLVM pass, but in many cases, it’s still preserved in the binary because it’s referenced by Full Type Metadata which prevents Reflection Metadata from stripping.

Introducing a static compilation check potentially can help to solve both of mentioned issues by adding to the language a way to express the requirement to have Reflection metadata at runtime.

Proposed solution

Teaching the Type-checker to ensure Reflection metadata is preserved in a binary if reflection-consuming APIs are used, will help to move the issue from runtime to compile time.

To achieve that, a new marker protocol Reflectable will be introduced. Firstly, APIs developers will gain an opportunity to express a dependency on Reflection Metadata through a generic requirement of their functions, which will make such APIs safer. Secondly, during IRGen, the compiler will be able to selectively emit Reflection symbols for the types that explicitly conform to the Reflectable protocol, which will reduce the overhead from reflection symbols for cases when reflection is emitted but not consumed.

Case Study 1:

SwiftUI Framework:

protocol SwiftUI.View: Reflectable {}  
class NSHostingView<Content> where Content : View {  
    init(rootView: Content) { ... }  
}

User module:

import SwiftUI  
  
struct SomeModel {}  
  
struct SomeView: SwiftUI.View {  
    var body: some View {          
        Text("Hello, World!")  
            .frame(...)      
    }  
}  
  
window.contentView = NSHostingView(rootView: SomeView())

Reflection metadata for SomeView will be emitted because it implicitly conforms to Reflectable protocol, while for SomeModel Reflection metadata won't be emitted. If the user module gets compiled with the reflection metadata disabled, the compiler will emit an error.

Case Study 2:

Framework:

public func foo<T: Reflectable>(_ t: T) { ... }

User module:

struct Bar: Reflectable {}  
foo(Bar())

Reflection metadata for Bar will be emitted because it explicitly conforms to Reflectable protocol. Without conformance to Reflectable, an instance of type Bar can't be used on function foo. If the user module gets compiled with the reflection metadata disabled, the compiler will emit an error.

Conditional cast (as? Reflectable)

We also propose to allow a conditional cast to the Reflectable marker protocol, which would succeed only if Reflection Metadata related to a type is available at runtime. This would allow developers to explicitly check if reflection metadata is available and based on that fact branch the code accordingly.

public func consume(_ t:  Any) {
    if  let _t = t as? Reflectable {
        // Use Mirror API to extract Reflection Metadata
    }  else  {
        // Back to default implementation
    }
}

Behaviour change for Swift 6

For Swift 6, we propose to enable Opt-in behaviour by default, to make the user experience consistent and safe. To achieve that we will need to deprecate the compiler's options that can lead to missing reflection - -reflection-metadata-for-debugger-only and -disable-reflection-metadata. Starting with Swift 6, these arguments will be ignored in favour of the default opt-in mode.

Detailed design

Since Reflection symbols might be used by LLDB, there will be difference in emitted Reflection symbols across Debug and Release modes.
Release mode: if -O, -Osize, -Ospeed passed.
Debug: - if -Onone passed or if not set.

One more level of reflection metadata will be introduced in addition to the existing ones:

  1. Reflection Disabled (-disable-reflection-metadata)
  • Do not emit reflection in Release and Debug modes.
  • If there is a type in a module conforming to Reflectable, the compiler will emit an error.
  1. Enabled for the debugger support (-reflection-metadata-for-debugger-only)
  • Emit Reflection metadata for all types in Debug mode while emitting nothing in Release modes.
  • If there is a type in a module conforming to Reflectable, the compiler will emit an error (even if in Debug mode the metadata is actually emitted).
  1. Opt-in enabled (-enable-opt-in-reflection-metadata)
  • In Release mode, emit only for types that conform to Reflectable.
  • In Debug mode emit reflection in full.
  1. Fully enabled (current default level)
  • Emit reflection metadata for all types in Release and Debug modes.

Introducing a new flag to control the feature will allow us to safely roll it out and avoid breakages of the existing code. For those modules that get compiled with fully enabled metadata, nothing will change (all symbols will stay). For modules that have the metadata disabled, but are consumers of reflectable API, the compiler will emit the error enforcing the guarantee.

Source compatibility

The change won’t break source compatibility in versions prior to Swift 6, because of the gating by the new flag. If as proposed, it’s enabled by default in Swift 6, the code with types that has not been audited to conform to the Reflectable protocol will fail to compile if used with APIs that consume the reflection metadata.

Effect on ABI stability

Reflectable is a marker protocol, which doesn't have a runtime representation, has no requirements and doesn't affect ABI.

Effect on API resilience

This proposal has no effect on API resilience.

Alternatives considered

Dead Code Elimination and linker optimisations were also considered as a way to reduce the amount of present Reflection metadata in release builds. The optimiser could use a conformance to a Reflectable protocol as a hint about what reflection metadata should be preserved. However, turned out it was quite challenging to statically determine all usages of Reflection metadata even with hints.

It was also considered to use an attribute @reflectable on nominal type declaration to express the requirement to have reflection metadata, however, a lot of logic had to be re-implemented outside of type-checker to ensure all guarantees are fulfilled.

14 Likes

Totally makes sense, +1 for this.

Besides compiler flag to switch off all metadata of module, is there a way to turn off specific type which implicitly/indirectly conform to Reflectable protocol?

Furthermore, how about @_spi stuff? Does it also auto generated by default either in 5.x or 6 Swift

2 Likes

At my employer, we turned reflection metadata off to save on code size some time ago. The biggest challenge this created was that many enums description implementations relied on reflection metadata to produce a sensible value.

Is the reflection metadata emitted by -reflection-metadata-for-debugger-only and enable-opt-in-reflection-metadata still available in the program itself in debug builds? That is, if you call someEnum.description, or Mirror(reflecting: someOptedOutType) will you get a value that was created from the metadata?

I think the answer to the above should be "no." I think it will be difficult to educate engineers as to why code works in debug and not in production. At my employer, we opted to turn reflection metadata off in both debug and release, because we felt that engineers couldn't be trusted to know when their program relied on reflection metadata and when it did not.

I think that if this feature is turned on by default in Swift 6, there should be a migrator available in Xcode that attempts to identify cases where reflection metadata was indirectly used, and adds the conformance.

6 Likes

Would reflection metadata also be emitted if Reflectable conformance was added in an extension? If yes, would it even work if the type was defined in a different module than the extension?

2 Likes

+1, I'm very happy to see this!

I had big privacy concerns about the current reflection mechanism that prevented me from considering distributing closed-source frameworks due to a risk of easy reverse-engineering.

Also, as an author of some reflection-based functionality, this solves the reliability issue that would cause the functionality to silently fail to do its job because of missing reflection metadata.

I'm also really looking forward to better reflection, because accessing the Swift runtime using @_silgen_name is dirty and needs constant maintenance to make sure they still work.

1 Like

I’m curious how this would work for descendant elements, eg does SwiftUI require reflection on its view’s properties to work out whether they are Plain Old Data types vs complex types, whether they conform to equatable?

I’m curious whether this would infer reflection on the child elements within the reflected type?

3 Likes

Besides compiler flag to switch off all metadata of module, is there a way to turn off specific type which implicitly/indirectly conform to Reflectable protocol?

We didn't consider such a mechanism, because couldn't come up with the cases when it would be safe to remove Reflectability property from a type when it implicitly inherits it without breaking the guarantees that this proposal introduces. (Maybe you have such cases in mind?)
We also suggested to deprecate the -disable-reflection-metadata flag in Swift 6 to eliminate possiblity of any runtime issues in the future.

Furthermore, how about @_spi stuff? Does it also auto generated by default either in 5.x or 6 Swift

Could you provide more details on this? I am not very familiar with @_spi and thought it's related to how modules are imported.

many enums description implementations relied on reflection metadata to produce a sensible value

Do you use enums description in Release or Debug only builds?

Is the reflection metadata emitted by -reflection-metadata-for-debugger-only and -enable-opt-in-reflection-metadata still available in the program itself in debug builds? That is, if you call someEnum.description, or Mirror(reflecting: someOptedOutType) will you get a value that was created from the metadata?

We propose to emit reflection metadata in full in debug builds, because it might be consumed for debugging purposes, by the debugger or for cases you mentioned like enum description or print/dump functions. So yes, in Debug builds, it would work for all types in a program. In release builds, it wouldn't have any reflection metadata because it didn't conform to Reflectable protocol.

I think the answer to the above should be "no." I think it will be difficult to educate engineers as to why code works in debug and not in production. At my employer, we opted to turn reflection metadata off in both debug and release, because we felt that engineers couldn't be trusted to know when their program relied on reflection metadata and when it did not.

In general, I agree that we can't expect engineers to know when and how Reflection metadata is emitted, but I am not sure we can avoid that divergence between Debug and Release builds since it is used by the debugger and always should be emitted in a non-optimised mode. But what we can do is to make the usage of reflection consumed APIs safer without making them fail silently if reflection is missing. Developers still might not know how reflection is emitted, but with the proposal, they at least will be prevented from using it erroneously.

I also think, that the debug features like print/dump/enum.description, etc probably shouldn't be used in Release mode, since it might be a potential vector for attack, so it might be a good thing that they won't work.

I think that if this feature is turned on by default in Swift 6, there should be a migrator available in Xcode that attempts to identify cases where reflection metadata was indirectly used, and adds the conformance.

Could you give an example? I feel it might be quite challenging to find all cases where the Reflection metadata is used indirectly at compile time. My assumption is based on the fact that we already investigated the optimisation, which could be able to statically determine if the Reflection metadata is used, but it turned out it's an extremely difficult computational problem to solve.

@benpious

This is a good question!
Currently, it's allowed to add conformance to Reflectable through an extension, but it will work only until the type declaration and the extension are in the same module.

If a type declaration is in another module, it will be a no-op since it seems the compiler doesn't emit metadata for imported types.
There might be also some privacy concerns on top of it, but theoretically, this should be implementable if this is found useful since the compiler knows about the layout of imported types.

For now, perhaps the compiler needs to emit a warning preventing developers from doing that.

2 Likes

I'm also really looking forward to better reflection, because accessing the Swift runtime 5 using @_silgen_name is dirty and needs constant maintenance to make sure they still work.

Yeah, we also struggle with the current Mirror API implementation, especially because it only allows read access, but this is out of scope for this particular proposal.

does SwiftUI require reflection on its view’s properties to work out whether they are Plain Old Data types vs complex types

I believe the POD bit is stored in the struct's value witness table if the layout is known at runtime, so the absence of Reflection metadata shouldn't affect this part.

I’m curious how this would work for descendant elements

I tend to think that descendant elements of a reflectable type shouldn't have reflection metadata emitted for them recursively if they do not conform to Reflectable, otherwise, the fine-grained approach won't be fine-grained anymore.

In that case, a developer will be able to get the value of field b in type B, but won't be able to drill into it further, because type A won't have reflection metadata emitted.

struct A {}

struct B: Reflectable {
	let b = A()
}

This clarification should be added to the proposal text, I think. Thanks!

1 Like

Do you use enums description in Release or Debug only builds?

Both. Or at least, we used to.

Developers still might not know how reflection is emitted, but with the proposal, they at least will be prevented from using it erroneously.

Maybe I'm missing something, but this proposal doesn't seem to add any guardrails to prevent this. I don't see an explicit proposal to modify the API of Mirror to only accept types that conform to this protocol, for example.

Here's some potentially surprising examples of cases where reflection metadata is used:

enum E {
    case a
    case b
}

print(String(describing: E.a)) // prints "a" in DEBUG, "E(rawValue: 0)" without reflection metadata
let str = "\(E.a)"
print(str) // prints "a" in DEBUG, "E(rawValue: 0)" without reflection metadata

(I misremembered in my first post, enums don't implement description by default. This is what we were using. Sorry for any confusion this created)

Search for "get a string from an enum swift" and you'll find answers suggesting to hard code strings, but also quite a few folks suggesting String(describing:), even one on this forum. And the string interpolation example makes it even more unclear what the source of the string is.

Again, I might be missing something but I think it's very likely that if this change is accepted and landed a nontrivial number of apps on the App Store are going to break. Maybe that's an acceptable tradeoff for the future of the language, but in my opinion the potential for breakage needs to be explicitly planned for and addressed as part of this proposal.

Thanks for your feedback!

Maybe I'm missing something, but this proposal doesn't seem to add any guardrails to prevent this. I don't see an explicit proposal to modify the API of Mirror to only accept types that conform to this protocol, for example.

The proposal draws a line between optional and required reflection and focuses on a use case where reflection metadata is required e.g SwiftUI.

print/dump/String(describing:) are examples of consumers for optional metadata. We didn't initially have plans to require reflection metadata for using such APIs because not everybody wants to leak implementation details with reflection metadata.

However, we can try to address this issue if there is a request from the community.
I can see several ways how that can be resolved:

  1. Partially eliminate optionally consuming APIs like String(describing:). (Mirror API should stay optional I believe)
  2. A new variance of API can be introduced into stdlib that will require the presence of reflection metadata. (print(withReflection:_))
  3. Developers are asked to add conformance to Reflectable explicitly if they want rich output of optional APIs.
  4. Stdlib function consuming reflection optionally, might be tweaked not to use reflection if a type doesn't conform to Reflectable. In that case, the reflection metadata of such types will be used only by the debugger and will help to avoid the difference between Debug and Release builds.

Which one seems like a good way forward?

Again, I might be missing something but I think it's very likely that if this change is accepted and landed a nontrivial number of apps on the App Store are going to break. Maybe that's an acceptable tradeoff for the future of the language, but in my opinion the potential for breakage needs to be explicitly planned for and addressed as part of this proposal.

This is a valid concern.
If a concrete type is used on such a function we could provide a migrator for that, but it won't help for cases with dynamic types.
Theoretically, we also could just codemod all call-sites of print/dump/String(describing:) to arg as! Reflectable which would be an indicator to developers to add the conformance to the types used on it, but it seems quite radical to me :) At least, it won't fail silently and developers will be able to add the conformance explicitly.

1 Like

What about implicitly conforming all types to Reflectable unless -disable-reflection-metadata is passed? Then the stdlib could use have String(describing: Any) and String<R: Reflectable>(describing: R), thus re-using the existing language infrastructure for overloading a function based on arguments’ different capabilities.

2 Likes

I like the idea of overloading the APIs, but in Swift 6 we propose to depreciate -disable-reflection-metadata and -reflection-metadata-for-debugger-only options to eliminate the possibility of emitting incorrect code by the compiler which will leave us with opt-in and full-emission modes only.

I believe the question here is which one to make a default behaviour in Swift 6, based on the tradeoffs we are ready to accept.

  1. Initial idea was to enable Opt-in mode for Swift 6 by default, but yes, some apps will break but a binary size win will be there for everybody. (Swift 6 is going to break compatibility anyway, so we were ready to accept it as collateral damage)

  2. On the other hand, if we make Full-emission mode on by default, the compatibility won't be broken and everything will work as expected until a developer explicitly enables Opt-in mode.

For (1) we could also soften consequences, but I don't think we will be able to fully eliminate them as I mentioned in my post above.

As is, both options should be equally easy to implement, the mitigations for option (1) will bring lots of complications though, especially if we want it to be backward compatible.

Any thoughts? @ksluder @benpious

We didn't initially have plans to require reflection metadata for using such APIs because not everybody wants to leak implementation details with reflection metadata.

I'm not necessarily suggesting this. But I think if you want to prevent regressions, you need to require that types that are passed to the non-debug functions (String(describing:), string interpolation) implement one of the following:

  • Reflectable
  • CustomStringConvertible
  • TextOutputStreamable

I don't think you need to require this for debugging functions like print, and dump's documentation does imply the use of reflection metadata.

So my suggestion would be to make a new protocol, perhaps named StringConvertible, and have Reflectable, CustomStringConvertible, and TextOutputStreamable be subtypes of it. Then you can modify String(describing:) and String interpolation to only accept types conforming to StringConvertible, and the compiler will be able to tell folks when they are passing something to these functions that will have a different output in Swift 6.

Theoretically, we also could just codemod all call-sites of print/dump/String(describing:) to arg as! Reflectable

Let's not lose sight of the goal: to avoid breakages, not to force everyone to implement Reflectable. This will produce broken code in cases where people are relying on CustomStringConvertable instead. In fact, this might break any prints that are using Strings.

Another, unrelated question: what happens if I write

extension String: Reflectable {}

I don't own String, and the Swift stdlib has already been compiled. So presumably this implementation will do nothing, right?

Wouldn't this break ABI compatibility?

Ah, yeah, it would.

Imo continuing to emit full metadata (in all configurations) unless turned off explicitly for the module is the way to go. Having differences in behaviour between debug and release configurations by default sounds like a recipe for confusion to me, and the potential savings in binary size don't look big enough to me to make up for that. Disabling metadata on the default Vapor template reduced the binary (in release mode, Swift 5.6.1) from 32MB to 30MB (plus the version built without metadata failed to run properly).

I also wanna mention that I think the way print and string interpolation result in a useful description of a value, without the need to implement anything or even to conform to any marker protocol, is a very valuable feature of Swift and shouldn't be discarded lightly. Maybe even to the point where it might be worth somehow making that work with reflection metadata disabled.

1 Like

I agree that it would be best not to make this the default unless it can be made safe, but here’s some devil’s advocate perspective:

Disabling metadata on the default Vapor template reduced the binary (in release mode, Swift 5.6.1) from 32MB to 30MB

The Instagram app is 200mb installed today, according to the App Store. If it was written entirely in Swift, the binary size contribution of type metadata would probably be between 15 and 20mb (based on my anecdotal experience with the Uber apps and your number from Vapor, the contribution of type metadata is probably between 5-10% of the binary). I think it was 10mb for the Uber Rider app last we checked, and we are over 100mb in binary size.

So it’s not a trivial amount, especially if you’re a big company organized around the feature team model. I imagine if you’re trying to use Swift for some more systems oriented purpose it might also be useful to turn this off.

4 Likes