SE-0402: Generalize `conformance` macros as `extension` macros

[John_McCall]

Hello, Swift community.

The review of SE-0402: Generalize conformance macros as extension macros begins now and runs through July 17th, 2023.

Reviews are an important part of the Swift evolution process. All review feedback should be either on this forum thread or, if you would like to keep your feedback private, directly to me as the review manager by email or DM on the forums. When contacting me directly about this proposal, please put "[SE-0402]" at the start of the subject line.

Trying it out

If you'd like to try this proposal out, you can download a toolchain supporting it here (for macOS) or here (for Linux). You will need to add -enable-experimental-feature ExtensionMacros to your build flags.

What goes into a review?

The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of Swift. When writing your review, here are some questions you might want to answer in your review:

• What is your evaluation of the proposal?
• Is the problem being addressed significant enough to warrant a change to Swift?
• Does this proposal fit well with the feel and direction of Swift?
• If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
• How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

More information about the Swift evolution process is available at

https://github.com/apple/swift-evolution/blob/main/process.md

Thank you for contributing to Swift.

John McCall
Review Manager

[jrose (Jordan Rose)]

Overall in favor, but I do want to note that this exacerbates the local type problem, because conformance macros work(ed) on local types and extension macros (as proposed) do not.

[hborla (Holly Borla)]

@jrose

because conformance macros work(ed) on local types

This was an implementation mistake that has since been fixed on main and release/5.9. There are many cases where conformance macros applied to local types do not work due to the inability to resolve types written in a where clause. For example, I wrote a quick AddConformance macro that expands to extension S: P where T == Int {}:

public protocol P {
  associatedtype T
}

func test() {
  @AddConformance
  struct S<A> {
    typealias T = A
  }
}

The above code complains about self-referential generic requirements, and it succeeds if I write struct S at the top-level or nested inside another type.

I agree with the general point that extension macros exacerbate the inability to express extensions on local types, though.

[jlukas (Jacob Lukas)]

Will extension macros work on private types nested in other types?

struct A {
    @SomeExtensionMacro
    private struct B {}
}

As far as I know, there’s no way for a normal extension to extend A.B here, so I assume that would apply to extension macros as well, but I just wanted to confirm.

[s-k (Stephen Kockentiedt)]

First of all: I think the general idea of the proposal is great.

If I am not mistaken, the need to list all possible conformances in the macro declaration under conformances: is a restriction that was not present in the original conformance macros. I can imagine different scenarios where listing the conformances in advance is not possible. For example, the conformance may need to be different for every type the macro has been attached to.

I would suggest that the requirement is dropped or an arbitrary option is added such as for names:.

[s-k (Stephen Kockentiedt)]

Here is a simple macro possible with conformance macros but not with extension macros:

@AddProtocol
class Guide {
  func answer() -> Int { 42 }
}

// Generates:

extension Guide: GuideProtocol {}

protocol GuideProtocol {
  func answer() -> Int
}

In addition, this sentence would mean that the timing of the evolution process decides whether this macro will be possible or not:

If this proposal is accepted after 5.9, the conformance macro role will remain in the language as sugar for an extension macro that adds only a conformance.

[Zollerboy1 (Josef Zoller)]

@s-k

Here is a simple macro possible with conformance macros but not with extension macros:

@AddProtocol
class Guide {
  func answer() -> Int { 42 }
}

// Generates:

extension Guide: GuideProtocol {}

protocol GuideProtocol {
  func answer() -> Int
}

Why wouldn't this be possible with extension macros?

[s-k (Stephen Kockentiedt)]

@Zollerboy1

Why wouldn't this be possible with extension macros?

Because, as the proposal is currently written, the protocol name GuideProtocol would need to be listed under conformances: in the macro declaration. This is not possible because the name depends on the type the macro is attached to.

[Zollerboy1 (Josef Zoller)]

Ah yeah, this was a follow-up on your previous post. Yes, we should definitely have the option to declare an extension macro as providing arbitrary conformances.

[Joannis_Orlandos]

I was hoping this would be added. It's currently lacking, and as a library authors I've wished for this a couple of times already. I don't know if regular app developers would miss this, but I can definitely see this empowering protocol oriented development using Macros.

With this, I can get rid of the remainder of my code generation I would have to do using Sourcery. I've experimented with Macros a fair bit, and think this is the main missing puzzle piece. This is especially powerful for library/framework authors. I think the execution is almost perfect, and fits well.

I think that adding arbitrary protocol conformances can definitely be helpful. The "MockProtocol" listed above is a pretty good one, and aligns closely to the kind of use cases that I see in my backend projects.

[Douglas_Gregor]

@s-k

First of all: I think the general idea of the proposal is great.

If I am not mistaken, the need to list all possible conformances in the macro declaration under conformances: is a restriction that was not present in the original conformance macros. I can imagine different scenarios where listing the conformances in advance is not possible. For example, the conformance may need to be different for every type the macro has been attached to.

I would suggest that the requirement is dropped or an arbitrary option is added such as for names:.

Having a macro introduce arbitrary conformances are something we were trying to get away from. "Does type X conform to protocol P?" is a question often asked in the compiler, and a macro that can possibly answer that question for every P will be expanded nearly all the time. That's both a compile-time performance issue, and also can introduce cyclic dependencies in type checking when that macro has interesting arguments.

We could allow the same prefixed(X) and suffixed(X) forms in the conformances list that we do for names, so one can generate a protocol and a conformance to it together, with a name derived from the entity to which the macro is attached.

Doug

[s-k (Stephen Kockentiedt)]

@Douglas_Gregor

We could allow the same prefixed(X) and suffixed(X) forms in the conformances list that we do for names, so one can generate a protocol and a conformance to it together, with a name derived from the entity to which the macro is attached.

I think this would be a good solution

[Dan_Stenmark]

Under the current proposal and implementation, would the macro engine also permit expanding code into extensions on the protocol itself where Self is the implementor?

protocol Vehicle {}

@attached(extension, conformances: Vehicle, names: prefixed(build))
macro Vehicle = #externalMacro(...)

@Vehicle
struct Automobile {}

@Vehicle
struct Aircraft {}

// expanding to

extension Automobile: Vehicle {}
extension Vehicle where Self == Automobile {
  static func buildAutomobile() -> Self { ... }
}

extension Aircraft: Vehicle {}
extension Vehicle where Self == Aircraft {
  static func buildAircraft() -> Self { ... }
}

This would enable usage of these factory methods as follows:

let myCar: some Vehicle = .buildAutomobile()
let myPlane: some Vehicle = .buildAircraft()

[hborla (Holly Borla)]

@Dan_Stenmark

would the macro engine also permit expanding code into extensions on the protocol itself where Self is the implementor?

No, this proposal only allows extending the type the macro is attached to. Extending other types is a future direction, but all extended types would need to be listed in the @attached(extension) attribute so they're known prior to macro expansion. This came up in the pitch thread too:

@hborla

@stephencelis

This looks like a good improvement! Would it also pave the way for lifting some of the restrictions on extensions produced by macros?

Possibly. I could imagine a future addition to @attached(extension) that allows you to name the extended type in the macro attribute, e.g. @attached(extension, of: MyType), which defaults to the type the macro is attached to.

[liamnichols]

Great proposal!

Sorry that I haven’t been able to try the preview yet as I’m away from my laptop until next week, but I just wanted to double back and confirm if this change would allow us to attach the macro to extensions of types rather than the type itself?

In my use case, I want to be able to generate some test code in my test target so I would like attach the macro to an extension in the test target, for example:

// MyLib
struct User {
    let id: UUID
}

// MyLibTests
@ProvideFixture(using: User.init(id:))
extension User { } 

The expanded code here needs to add protocol conformance and also implement a method. I believe that this wasn’t possible before but I’m hoping that this will change.

Thanks!

[hborla (Holly Borla)]

@liamnichols

Sorry that I haven’t been able to try the preview yet as I’m away from my laptop until next week, but I just wanted to double back and confirm if this change would allow us to attach the macro to extensions of types rather than the type itself?

From the proposal:

Extension macros can only be attached to the primary declaration of a nominal type; they cannot be attached to typealias or extension declarations.

The reasons I subsetted out extension macros attached to extension declarations are

1. A major goal of this proposal is to fix the design of a feature in Swift 5.9, and packing in too much additional functionality could put the proposal at risk of not being ready in time for 5.9, leaving us with the two bad options of Swift 5.9 including a feature with suboptimal language design or ripping out the expressivity conformance macros provide entirely.
2. Supporting extension declarations is additive, so there's no reason it must be included in this proposal. Enabling extension macros on extension declarations can be proposed separately.
3. It's a little weird that an extension macro attached to an extension declaration generates entirely new extensions that can have different generic constraints than written on the original extension. I don't think any other alternative is better, but I'd feel more confident after more dedicated discussion about extension macros on extension declarations.

[liamnichols]

Thanks Holly, I totally missed that note in the proposal.

Your reasoning is sound and makes total sense, thanks for providing that context. I’ll have more of a think about what I’m trying to achieve and perhaps start up some separate discussions.

Liam

[woolsweater (Josh Caswell)]

The proposal shows the MyProtocol macro declaration in two places without a parameter clause:

@attached(extension, conformances: MyProtocol, names: named(requirement))
macro MyProtocol = #externalMacro(...)
//              ^ No parens

which would be a change to the macro declaration grammar. Could you clarify whether this is an oversight or an intentional part of the proposal?

[hborla (Holly Borla)]

@woolsweater

Could you clarify whether this is an oversight or an intentional part of the proposal?

Just a mistake in the proposal thanks for catching it!

[soumyamahunt (Soumya Ranjan Mahunt)]

I have one doubt regarding use-cases with class declarations, sorry if this covered elsewhere, but I couldn't find any info on it. The documentation for protocols variable in the macro conformance API mentions:

The list of protocols to add conformances to. These will always be protocols that type does not already state a conformance to.

Lets suppose a class SuperClass conforms to Codable and a new class SubClass that inherits from SuperClass, has an extension macro that generates Codable conformances. In this case will the macro generate new conformance or protocols array received in expansion API will be empty here?

If in previous case macro generates Codable conformances, will it have any context regarding SuperClass implementations i.e macro will be able to override SuperClass implementation and invoke super implementations in the overrides.