Basic quality-of-life improvements for @retroactive

Evolution Discussion
1

to me at least, @retroactive is one of the highest cost, lowest value features introduced with Swift 6. i can grasp the theoretical appeal of annotating every place where a conformance might not have the intended effect, but when you have a large code base with many dozens of packages, all authored by the same company, it just gets absurd.

consider the following protocol, which refines three other protocols, and has a bunch of retroactive conformances for types from an upstream package:

public protocol Storable: Serializable, Deserializable, CustomStringConvertible {}

you can’t just do this, because the compiler will complain that Storable is already in the same module as the extensions:

extension Account: @retroactive Storable {}
extension Incident: @retroactive Storable {}
extension Incident.Update: @retroactive Storable {}
extension Instrument: @retroactive Storable {}
extension Instrument.Update: @retroactive Storable {}
extension MetadataEntry: @retroactive Storable {}
extension Order: @retroactive Storable {}
extension Order.Update: @retroactive Storable {}
extension ConnectivityState: @retroactive Storable {}
extension ProductGroup: @retroactive Storable {}
extension PublicOrder: @retroactive Storable {}
extension PublicTrade: @retroactive Storable {}
extension PublicTrade.Update: @retroactive Storable {}
extension StoredProperties: @retroactive Storable {}
extension StoredProperties.Update: @retroactive Storable {}
extension Tag: @retroactive Storable {}
extension Tag.Update: @retroactive Storable {}
extension TickRule: @retroactive Storable {}
extension TickRule.Update: @retroactive Storable {}
extension Transaction: @retroactive Storable {}
extension Transaction.Update: @retroactive Storable {}
extension OperationRejection: @retroactive Storable {}
extension User: @retroactive Storable {}
extension Plugin: @retroactive Storable {}

you also can’t do this, because @retroactive doesn’t affect protocol compositions:

extension Account: Storable, @retroactive (Serializable & Deserializable & CustomStringConvertible) {}
extension Incident: Storable, @retroactive (Serializable & Deserializable & CustomStringConvertible) {}
extension Incident.Update: Storable, @retroactive (Serializable & Deserializable & CustomStringConvertible) {}
extension Instrument: Storable, @retroactive (Serializable & Deserializable & CustomStringConvertible) {}
extension Instrument.Update: Storable, @retroactive (Serializable & Deserializable & CustomStringConvertible) {}
extension MetadataEntry: Storable, @retroactive (Serializable & Deserializable & CustomStringConvertible) {}
extension Order: Storable, @retroactive (Serializable & Deserializable & CustomStringConvertible) {}
extension Order.Update: Storable, @retroactive (Serializable & Deserializable & CustomStringConvertible) {}
extension ConnectivityState: Storable, @retroactive (Serializable & Deserializable & CustomStringConvertible) {}
extension ProductGroup: Storable, @retroactive (Serializable & Deserializable & CustomStringConvertible) {}
extension PublicOrder: Storable, @retroactive (Serializable & Deserializable & CustomStringConvertible) {}
extension PublicTrade: Storable, @retroactive (Serializable & Deserializable & CustomStringConvertible) {}
extension PublicTrade.Update: Storable, @retroactive (Serializable & Deserializable & CustomStringConvertible) {}
extension StoredProperties: Storable, @retroactive (Serializable & Deserializable & CustomStringConvertible) {}
extension StoredProperties.Update: Storable, @retroactive (Serializable & Deserializable & CustomStringConvertible) {}
extension Tag: Storable, @retroactive (Serializable & Deserializable & CustomStringConvertible) {}
extension Tag.Update: Storable, @retroactive (Serializable & Deserializable & CustomStringConvertible) {}
extension TickRule: Storable, @retroactive (Serializable & Deserializable & CustomStringConvertible) {}
extension TickRule.Update: Storable, @retroactive (Serializable & Deserializable & CustomStringConvertible) {}
extension Transaction: Storable, @retroactive (Serializable & Deserializable & CustomStringConvertible) {}
extension Transaction.Update: Storable, @retroactive (Serializable & Deserializable & CustomStringConvertible) {}
extension OperationRejection: Storable, @retroactive (Serializable & Deserializable & CustomStringConvertible) {}
extension User: Storable, @retroactive (Serializable & Deserializable & CustomStringConvertible) {}
extension Plugin: Storable, @retroactive (Serializable & Deserializable & CustomStringConvertible) {}

instead, you actually have to insert @retroactive before every single upstream protocol that was refined by the local protocol:

extension Account: Storable, @retroactive Serializable, @retroactive Deserializable, @retroactive CustomStringConvertible {}
extension Incident: Storable, @retroactive Serializable, @retroactive Deserializable, @retroactive CustomStringConvertible {}
extension Incident.Update: Storable, @retroactive Serializable, @retroactive Deserializable, @retroactive CustomStringConvertible {}
extension Instrument: Storable, @retroactive Serializable, @retroactive Deserializable, @retroactive CustomStringConvertible {}
extension Instrument.Update: Storable, @retroactive Serializable, @retroactive Deserializable, @retroactive CustomStringConvertible {}
extension MetadataEntry: Storable, @retroactive Serializable, @retroactive Deserializable, @retroactive CustomStringConvertible {}
extension Order: Storable, @retroactive Serializable, @retroactive Deserializable, @retroactive CustomStringConvertible {}
extension Order.Update: Storable, @retroactive Serializable, @retroactive Deserializable, @retroactive CustomStringConvertible {}
extension ConnectivityState: Storable, @retroactive Serializable, @retroactive Deserializable, @retroactive CustomStringConvertible {}
extension ProductGroup: Storable, @retroactive Serializable, @retroactive Deserializable, @retroactive CustomStringConvertible {}
extension PublicOrder: Storable, @retroactive Serializable, @retroactive Deserializable, @retroactive CustomStringConvertible {}
extension PublicTrade: Storable, @retroactive Serializable, @retroactive Deserializable, @retroactive CustomStringConvertible {}
extension PublicTrade.Update: Storable, @retroactive Serializable, @retroactive Deserializable, @retroactive CustomStringConvertible {}
extension StoredProperties: Storable, @retroactive Serializable, @retroactive Deserializable, @retroactive CustomStringConvertible {}
extension StoredProperties.Update: Storable, @retroactive Serializable, @retroactive Deserializable, @retroactive CustomStringConvertible {}
extension Tag: Storable, @retroactive Serializable, @retroactive Deserializable, @retroactive CustomStringConvertible {}
extension Tag.Update: Storable, @retroactive Serializable, @retroactive Deserializable, @retroactive CustomStringConvertible {}
extension TickRule: Storable, @retroactive Serializable, @retroactive Deserializable, @retroactive CustomStringConvertible {}
extension TickRule.Update: Storable, @retroactive Serializable, @retroactive Deserializable, @retroactive CustomStringConvertible {}
extension Transaction: Storable, @retroactive Serializable, @retroactive Deserializable, @retroactive CustomStringConvertible {}
extension Transaction.Update: Storable, @retroactive Serializable, @retroactive Deserializable, @retroactive CustomStringConvertible {}
extension OperationRejection: Storable, @retroactive Serializable, @retroactive Deserializable, @retroactive CustomStringConvertible {}
extension User: Storable, @retroactive Serializable, @retroactive Deserializable, @retroactive CustomStringConvertible {}
extension Plugin: Storable, @retroactive Serializable, @retroactive Deserializable, @retroactive CustomStringConvertible {}

what value is disallowing extension Foo: @retroactive Storable {} actually bringing us?

9 Likes
2

It would be reasonable to loosen the rules so that@retroactive attached to a protocol defined in the same module is accepted so long as conformance to the protocol does ultimately imply associated conformances that are retroactive. I think that not accepting this is mostly an oversight and/or a concession to implementation complexity rather than an explicit decision that was made for soundness.

5 Likes
3

great! i assume this would this not need to go through Evolution again?

4

What happens when you fully qualify Storable with the module name instead of using @retroactive?

Amendments, even enhancements, would need to be reviewed unless the behavior is explicitly specified somewhere in the proposal already.

4 Likes
5

FWIW, I fixed this one a few days ago: [6.2] Sema: Don't diagnose implied conformance of imported type to Sendable by slavapestov · Pull Request #81694 · swiftlang/swift · GitHub. (The PR title is about Sendable, but there was a second related fix in there to @retroactive diagnostics.)

You should be advised that retroactive conformances, irrespective of @retroactive or not, are fundamentally unsound if used incorrectly (that is, in any scenario where more than one conformance for the same type/protocol is visible). Without knowing anything about your application, I would still suggest a design that does not involve retroactive conformances, if at all possible.

8 Likes
6

Is the broad @retroactive requirement overkill with MemberImportVisibility? It seems like it to me. Perhaps we only warn on such conformances if multiple conflicting ones are present when MemberImportVisibility is enabled. I think we might need a rethink of this feature in light of the new import mechanics.

7

No, MemberImportVisibility doesn't limit the visibility of conformances. Any retroactive conformance that is transitively visible can be used without a direct import of the module defining the conformance. I think we ought to fix that, but it will take a different proposal. I've also been told that the implementation would be fairly involved.

5 Likes
8

Oh, at the very least it seems like there are far fewer transitive imports coming in. If we fixed the remaining transitive conformances, perhaps this also subsumes the broad need for @retroactive.

9

Keep in mind that the dangers of retroactive conformances aren't just about what's statically visible in the transitive closure of the imports. Protocol conformances are also globally (w.r.t. the loaded image) registered in the runtime, so dynamic casts are risky even if conflicting conformances wouldn't necessarily be seen at compile time because they're in two distinct branches of the code somewhere.

2 Likes
10

I'm not sure what you mean by this, but MemberImportVisibility doesn't somehow limit what modules are visible to the compiler in general. What it limits is which member declarations are legal to reference from a given source file, given what has been directly imported in that file.

EDIT: You might be thinking of InternalImportsByDefault and SE-0409 in general. When upstream dependencies manage the visibility of their imports appropriately, that can reduce the number of transitive dependencies visible to dependents. I wouldn't consider that a solution to the problem, though, because many imports still have to be exposed and it would still potentially be useful to avoid implicit use of their retroactive conformances.

11

You can do this with the same mechanism as availability checking, I believe. When you walk the AST for a source file, you check lexical visibility of each conformance you visit, just like you check if its available from that source location.

What is much harder is making the overlapping conformances case work in a sound way. Presumably there you'd want to perform this filtering at conformance lookup time, instead of a separate pass, so that if conformance lookup produces ambiguous results you'd then diagnose the issue right there.

The problem is that we perform conformance lookups all over the place without any kind of source location information, and we expect to get the same results every time. Fixing this so that lookups are performed once and stashed in the AST from that point on would be a big undertaking.

4 Likes
12

i wrote this up as a pitch here:

1 Like