[Pitch] Relax the use of compiler-generated $-prefixed identifiers as closure argument names

klau · 2024-07-24T23:33:50.598Z

I am proposing to relax the use of compiler-generated $-prefixed identifiers in closures, specifically since the current restriction prevents the use of MacroExpansionContext.makeUniqueName(_:) as the identifier of a closure argument when expanding a macro.

Introduction

The prefix $ is currently reserved for compiler-generated identifiers, the Swift grammar explicitly states the use of $-prefixed identifiers as follows,

implicit-parameter-name → $ decimal-digits
property-wrapper-projection → $ identifier-characters

In practice, the compiler does not prohibit the use of $-prefixed identifiers in other context except for the use of $ decimal-digits outside of the closure scope and the use of $ identifier-characters as explicit closure argument names inside the closure scope.

Motivation

When expanding a macro, every invocation of MacroExpansionContext.makeUniqueName(_:) will return a name unique in the current macro context, preventing naming conflicts when the macro introduces new identifiers. Because these unique names are also prefixed with $ as they are generated by the compiler, as a result, the compiler hinders the use of these unique names as closure argument names in macro expansion, (example courtesy of @grynspan)

let arg0 = context.makeUniqueName("")
return """
{ \(arg0) in
  ...
}
"""

The compiler will report the following errors from the expanded source,

Inferred projection type 'Int' is not a property wrapper
Inferred projection type '() -> Int' is not a property wrapper

because the expanded source will look something like this,

{ $s<SOME_MINGLED_NAME> in
  ...
}

that the compiler interprets $s<SOME_MINGLED_NAME> as a property-wrapper-projection instead of an explicit closure argument.

Proposed Solution

The restriction on the use of $ identifier-characters as explicit closure argument names should be lifted. Such lift will not introduce any naming conflict since $ decimal-digits is still exclusively reserved for implicit closure argument names, while enabling macro authors working with closures to improve macro hygiene through the use of MacroExpansionContext.makeUniqueName(_:).

ktoso · 2024-07-25T00:18:12.944Z

It's not clear to me if this text is proposing to this lift the $ restriction in general or in macro expansions specifically.

It should probably clarify this aspect of the proposal.

In general though seems like a valuable change; I wonder if possible to restrict it to macro expansions though?

klau · 2024-07-25T00:41:39.457Z

I'll think about it further and rephrase my pitch soon, thanks!

jlukas · 2024-07-25T00:57:49.105Z

How would a macro invoke the property wrapper behaviour, if it was desired?

xwu · 2024-07-25T01:20:44.312Z

ktoso:

In general though seems like a valuable change; I wonder if possible to restrict it to macro expansions though?

How would that square with the policy we've articulated that macro expansion code is supposed to be plain utterable Swift?

ktoso · 2024-07-25T05:57:47.220Z

xwu:

How would that square with the policy we've articulated that macro expansion code is supposed to be plain utterable Swift?

That's already entirely not true though AFAICS?

E.g. variable identifiers like:

let $name = ""

example.swift:2:5: error: cannot declare entity named '$name'; the '$' prefix is reserved for implicitly-synthesized declarations
let $name = ""
    ^

while macros are freely doing just that, e.g. the TaskLocal macro in the stdlib:

    return [
      """
      \(access)\(staticKeyword)let $\(name)\(explicitTypeAnnotation) = TaskLocal(wrappedValue: \(initialValue))
      """
    ]

Unless I'm missing something here, this is somewhat similar of a situation. The macro introduced a $ name, which normally would not be okey. And the same is being asked for here, but for closure argument names.

Edit: I realized the nature of the emitted error is different:

example.swift:4:11: error: inferred projection type 'String' is not a property wrapper
closure { $name in }

So this isn't quite the same; it's not that the $names are banned but mis-interpreted... The proposal states this, but I missed it in the text, including an example with the error would help notice the specific issue being discussed.

Jumhyn · 2024-07-25T14:52:22.644Z

ktoso:

Unless I'm missing something here, this is somewhat similar of a situation. The macro introduced a $ name, which normally would not be okey. And the same is being asked for here, but for closure argument names.

Hm, this seems unfortunate given what I understood to be the general philosophy of Swift macros. IIRC we've declined to even suppress warnings coming from macro expansion on the basis that the code should be able to be copy-pasted out of a macro and into source without introducing any new diagnostics. cc @Douglas_Gregor: I don't recall if the tension here came up specifically (or if that philosophy had been totally solidified) when we originally reviewed attached macros, do you?

ktoso · 2024-07-25T15:02:33.784Z

As far as why the $name was allowed at all I believe the context may have been that we need to replace property wrappers with macros, and since property wrappers introduced $names... the macros needed to do this as well.

I'm not privy to any language/core team discussions why this was allowed or deemed troublesome or not; just from my observation why it was allowed in the specific macro case that I pasted code from.

Jumhyn · 2024-07-25T15:06:18.747Z

Right, the allowance of $ prefix was definitely very deliberately considered, I just don't recall if this specific intersection with "expansions should be fully valid Swift source" has been considered.

xwu · 2024-07-25T17:23:55.346Z

Jumhyn:

Right, the allowance of $ prefix was definitely very deliberately considered, I just don't recall if this specific intersection with "expansions should be fully valid Swift source" has been considered.

I think it's rationalizable that the "fully valid Swift source" policy can be held to apply modulo any sort of "name reservation" mechanism such as this when used by makeUniqueName(_:).

In theory, any macro expansion tooling could simply generate non-reserved unique identifiers when run in a user-facing tool context, and the fact that makeUniqueName(_:) happens to choose reserved names in code invisible to the user can be regarded as an implementation detail of how it guarantees uniqueness.

I was more raising my eyebrow at the notion that we should have macro expansion-specific code paths such that $ identifiers refer to a property wrapper projection in handwritten code but not in macro-synthesized code. That would be difficult to square with any interpretation of our policy but I think we're all on the same page that this isn't the end result we're going for.

Now, as to the example of the TaskLocal macro that deterministically synthesizes $\(name)\(explicitTypeAnnotation) = ..., which users can't write in their own source, I think that's a grey area... Indeed, that we permit this is sort of self-defeating: it's basically permitting users to create such identifiers anywhere they can invoke a macro, just with extra steps--one imagines someone could just write a #dollarPrefix macro.

Jumhyn · 2024-07-25T17:34:59.714Z

xwu:

In theory, any macro expansion tooling could simply generate non-reserved unique identifiers when run in a user-facing tool context, and the fact that makeUniqueName(_:) happens to choose reserved names in code invisible to the user can be regarded as an implementation detail of how it guarantees uniqueness.

Sure, if there's never any way for a user to observe the 'reserved' name, and all user-facing mechanisms spit out an appropriate user-definable identifier, then the distinction is moot. Though that raises the question of why we wouldn't just use the user-facing name internally at that point, since the internally-facing name is by assumption unobservable!

Val · 2024-07-27T11:26:26.938Z

Jumhyn:

IIRC we've declined to even suppress warnings coming from macro expansion on the basis that the code should be able to be copy-pasted out of a macro and into source without introducing any new diagnostics.

This is why I recently asked about the feasibility of downgrading the $-prefix error to a warning:

Downgrading the $-prefix error to a warning Discussion

I frequently use the "Inline macro" feature to debug or experiment with a macro. This is a fantastic feature that sets Swift macros apart. However, there is an annoyance: $-prefixed properties. Because these generate errors, it is not possible to quickly experiment with the macro implementation without changing a bunch of external references. Is it at all possible to downgrade that error to a warning? Or are there major downsides to allowing this even with a warning?