Piecemeal adoption of Swift 6 improvements in Swift 5.x

[allevato (Tony Allevato)]

@Douglas_Gregor

I don't care about the $ being concise, and it wasn't chosen for aesthetics. The $ syntax was chosen because existing Swift compilers already support it: $ can be at the start of a normal identifier (since at least Swift 3, probably earlier), but as with other $ identifiers in the language (for property wrappers, implicit closure arguments) you cannot define an entity with a $ name yourself. So the $ handy for compatibility with older tools... which is the point of the feature. (Note: @beccadax came up with the idea of using $ back when we introduced async / await into the compiler, I'm just codifying it now).

That makes total sense! Thanks for elaborating. Based on the examples you gave where you'd have to tack on additional conditions just to dodge older compilers (and the number of times this limitation has come up in other threads on these forums), using $ seems like a fine compromise.

[Zollerboy1 (Josef Zoller)]

Just a few questions out of curiosity:

Why is #if compiler(>=5.5) && $AsyncAwait used instead of just #if $AsyncAwait?

Does the compiler look after the && even when the expression before is false? E.g. could we write #if compiler(>=5.7) && hasFeature(AsyncAwait)?
In case we cannot do that today, could we maybe change the behaviour of the compiler now (i.e. stop parsing after a && in an #if when the expression before evaluates to false), so that in the future we could actually introduce a conditional check like hasFeature() and get away with it, because even if old compilers (starting from version 5.7 then) don't understand it, they can still just skip it?

[Douglas_Gregor]

@Zollerboy1

Why is #if compiler(>=5.5) && $AsyncAwait used instead of just #if $AsyncAwait ?

With the #if IDENTIFIER syntax, the compiler still parses the contents inside the #if body, but doesn't type check it. So an old compiler would still try to parse the declaration below and complain about the async:

#if $AsyncAwait
func f() async { } // error on older compilers, 'async' is unknown

The compiler(>=5.5) bit tells the compiler not to parse what's in the #if. It was introduced in SE-0212.

@Zollerboy1

Does the compiler look after the && even when the expression before is false? E.g. could we write #if compiler(>=5.7) && hasFeature(AsyncAwait) ?
In case we cannot do that today, could we maybe change the behaviour of the compiler now

Yes, the compiler looks after the &&. It parses the expression grammar, then interprets it. We could opt not to do that, but as you note, it would only provide benefits from point at which we make the change, so it doesn't help much with introducing hasFeature but could pave the way for some future check that doesn't fit any of the existing checks.

Doug

[xwu (Xiaodi Wu)]

@Douglas_Gregor

The compiler(>=5.5) bit tells the compiler not to parse what's in the #if . It was introduced in SE-0212.

Sounds like for many (most?) scenarios the real comparison, then, is between:

// (a)
#if compiler(>=5.7) && $SomeFeature

// …or (b)
#if compiler(>=5.7)
#if hasFeature(SomeFeature)

Admittedly, the latter is clunkier, but we could both (1) make && short-circuit in the fullness of time, and (2) make hasFeature imply from the get-go not parsing the source if the compiler doesn't have the feature, apt since features are being contemplated here for gating behind a flag specifically for source compatibility reasons.

I think that approach would land us in a better place after a transitional period than could be accomplished with the $ syntax. Because of the backward compatibility of the $ syntax, even if future compiler versions would know to implicitly restrict source parsing depending on feature support, we’d have to require an accompanying #if compiler conditional for older versions of the compiler.

[Douglas_Gregor]

@xwu

(1) make && short-circuit in the fullness of time,

I think this is a good idea, and while I think it could be an independent proposal, I'm fine with rolling it into this one.

@xwu

(2) make hasFeature imply from the get-go not parsing the source if the compiler doesn't have the feature

In all of our discussions about #if, it's been brought up that there is value in parsing the untaken if's when possible (e.g., for tools). Some new features on that list (existential any, regex literals, @preconcurrency) introduce new syntax that older compilers won't parse. The others (concise magic file names, forward-scan matching, implicitly opened existentials) are changes that affect semantics but don't change what code can be parsed. To me, that implies that the feature-check syntax (however it is spelled) shouldn't imply that an untaken if will not be parsed.

As for the $ vs. hasFeature, I'm bothered by the interim state of

#if compiler(>=5.7)
#if hasFeature(SomeFeature)
// new-fangled code
#else
// old code
#endif
#else
// exact copy of old code
#endif

I know it's an intermediate state, and 5 years down the road it won't matter, but we're designing this feature for folks who maintain compatibility with older tools, and it feels like we've missed the mark by adding a feature that they can't use cleanly. The amount of duplication required to use #if was brought up in other reviews (e.g., the primary associated types review), so the fact that "old code" in the example above has to be repeated twice bothers me.

Doug

[wes1 (Wes)]

@Douglas_Gregor

I don't think the SE numbers have a lot of meaning to folks outside this forum. Identifiers like AsyncAwait and ConciseMagicFile are far more immediately recognizable when you read them in, e.g., a package manifest or in source code.

Yes, the names are friendlier, after they are understood. Id's are not for meaning.

One partial fast-path to explanatory docs would be to have the compiler report the id of features it supports, when the option is given alone or with invalid input:

$ swift -enable-feature
swift-driver ...
...
Available features to enable: 

  SE-0296  AsyncAwait
  ... 

But since the SE is a vast discussion, it would be better to have a dedicated web portal with definitive docs:

$ swift -enable-feature
swift-driver ...
...
Available features to enable: 

 AsyncAwait  https://www.swift.org/docs/emerging/asyncawait
  ... 

Such a dedicated emerging-features site would alleviate most of my concerns.

Lacking that, what about doing both?

Can we permit the identifier as an optional suffix to the name?

-enable-feature AsyncAwait_SE-0296

That would enable package-writers to point users directly to docs, reducing the effort to adopt new features.

---

(What follows is just motivation; please skip unless you're on the fence)

I'm don't believe the name alone helps the larger numbers of developers who find a package they want to use and wonder if it's ok to enable that feature.

Both early-adopters and laggards don't understand yet, and both would benefit from the clearest/fastest path to docs on point.

None of those terms (AsyncAwait? "magic file"?), nor the bulk of new terminology introduced by Swift (existential? opaque?) is familiar to the average programmer, and even the vast majority of Swift programmers have only a vague sense of them, particularly when they are new. Nor is there a definitive description in the docs of emerging features. So I wouldn't expect users to be able to recognize, understand, and evaluate whether to use them, or whether to accept or reject a package using them, based on the name. (Exactly what aspect of concurrency is indicated by AsyncAwait?)

So I believe it's fair to say that someone not entirely familiar with the feature (i.e., whether early-adopter or laggard -- the people targeted) will need a fast path to definitive documentation. I also think these semi-oblivious users are the ones we want to ramp up as painlessly as possible.

Let's say the name can be followed by an optional id. Then the few package writers who want to use new features and bring along their many users can reduce the cost of adoption by including the suffix in their declaration.

Because it's optional, it doesn't tax those writers and readers who already know.

If we allow it as an option, later you can programmatically survey the population of Package.swift to get some measure of the relative utility of adding the id.

[xwu (Xiaodi Wu)]

@Douglas_Gregor

As for the $ vs. hasFeature , I'm bothered by the interim state of

#if compiler(>=5.7)
#if hasFeature(SomeFeature)
// new-fangled code
#else
// old code
#endif
#else
// exact copy of old code
#endif

That is definitely a bother and (I’d argue) unworkable. However, we can do:

#if compiler(<5.7)
// old code
#else
#if hasFeature(SomeFeature)
// newfangled code
#endif
#endif

Of course, I’d love to get rid of the nested conditional (I think this could be rolled into the short-circuiting proposal), but I think this is workable.

---

@Douglas_Gregor

Some new features on that list (existential any, regex literals, @preconcurrency ) introduce new syntax that older compilers won't parse. The others (concise magic file names, forward-scan matching, implicitly opened existentials) are changes that affect semantics but don't change what code can be parsed. To me, that implies that the feature-check syntax (however it is spelled) shouldn't imply that an untaken if will not be parsed.

With the hasFeature spelling, this could plausibly be determined on a feature-by-feature basis.

As you point out, some of these features involve spellings that just won’t parse in older compilers, and I’d imagine that almost every use of feature gating for those features will properly require an accompanying minimum compiler version—which, if not handled automatically, would require the user to specify correctly and likely without the benefit of a panoply of older compiler versions to test on.

Indeed, what a feature-by-feature determination of parsing here would allow is for parsing to be staged in separately from semantics, all without user intervention.

[fclout]

The syntax tradeoff seems to be that we have to pick between one of the following:

• a lovable, forward-looking syntax for feature detection
• a syntax that Swift 5 understands

and I think the technical reasons for this have been explained well enough. What most people seem to be annoyed with is that they expect Swift to be around for as long as Objective-C has been around, and it's a bummer that because we want to be compatible with Swift 5, Swift 25 (which may not be compile code older than Swift 24) will use $FEATURE for feature detection. How would we feel about an incremental plan such as this one?

• All future Swift compilers that support the Swift 5 language mode support both hasFeature(FEATURE) and $FEATURE for feature detection.
• Introduce a new compiler flag that tells up to which older version of Swift you expect your code to build for. It's a diagnostic with a fixit to use hasFeature(FEATURE) if it's not known that the minimum Swift version to parse this block is at least Swift 6, as determined either by the command-line-specified minimum version or whether we're currently in a #if compiler(>=6) block. SwiftPM can pass the right value to the compiler based on the minimum tools version.
• Within a few years all features detectable with $FEATURE are just assumed to be there, people stop testing for them, and the tests are organically removed. $FEATURE is still supported forever for all features introduced until the compiler stopped supporting the Swift 5 language mode, and everything else uses hasFeature.
• If for some reason something goes wrong with the plan, we can continue to do $FEATURE together with hasFeature for any necessary amount of time.

There's a small concern that people could keep using Swift 5 even after the compiler no longer supports $FEATURE for feature detection, but to me that concern is indistinguishable from the concern that the language becomes forked, so I think it might be OK to plan ahead assuming that it won't happen.

One reason I could see that we wouldn't want this would be that it makes it harder to explain which one of $FEATURE or hasFeature to use, but I still wanted to pitch it in case people think it's a reasonable transition plan. Another obvious reason to not be super excited with it is that it's more work than just using $FEATURE, and the people who will be stuck maintaining the end result obviously should have more say into this than me!

[Zollerboy1 (Josef Zoller)]

@xwu

However, we can do:

#if compiler(<5.7)
// old code
#else
#if hasFeature(SomeFeature)
// newfangled code
#endif
#endif

This still doesn't work for features that are added in Swift 5.8 and later. For these we would still have to write this:

#if compiler(<5.7)
// old code
#else
#if hasFeature(VariadicGenerics)
// code using variadic generics 
#else
// old code
#endif
#endif

[xwu (Xiaodi Wu)]

@Zollerboy1

This still doesn't work for features that are added in Swift 5.8 and later. For these we would still have to write this:

#if compiler(<5.7)
// old code
#else
#if hasFeature(VariadicGenerics)
// code using variadic generics 
#else
// old code
#endif
#endif

I don’t understand why you’d have to write that. Instead you’d write:

#if compiler(<5.8)
// old code
#else
#if hasFeature(VariadicGenerics)
// code using variadic generics 
#endif
#endif

[Zollerboy1 (Josef Zoller)]

@xwu

I don’t understand why you’d have to write that. Instead you’d write:

#if compiler(<5.8)
// old code
#else
#if hasFeature(VariadicGenerics)
// code using variadic generics 
#endif
#endif

What happens when the Swift version is greater than 5.8 but hasFeature(VariadicGenerics) is false? In this case you don't have any code at all.

[xwu (Xiaodi Wu)]

@Zollerboy1

What happens when the Swift version is greater than 5.8 but hasFeature(VariadicGenerics) is false? In this case you don't have any code at all.

I think you perhaps missed something. The reason an #if compiler conditional is required (in circumstances when it is) in conjunction with hasFeature (or $Feature, however spelt) is because the contents of #if blocks—other than #if compiler—are still parsed even if not taken. Therefore, code using a feature like variadic generics would cause an older compiler to stop parsing even if correctly gated with a feature conditional—that is, unless it is used within an additional #if compiler conditional requiring the minimum compiler version to be at least as high as the version in which the feature is introduced. So the answer to your question is, in the case when the Swift version is greater than 5.8 but hasFeature(VariadicGenerics) is false, you're holding it wrong and the file won't compile at all; this would be the case regardless of how you spell it and with any permutation of #ifs, #elseifs, and #elses.

Now, I'm advocating for some #if hasFeature blocks not taken also not to be parsed in future versions of Swift, perhaps on a feature-by-feature basis, but on reflection that would be unworkable as by construction any older compiler version would not have feature-by-feature knowledge of future features. Thus, unless we default to never parsing #if hasFeature blocks not taken (which may not be desirable, for the reasons that @DougGregor discusses above), the #if compiler conditional will be necessary.

[Douglas_Gregor]

@xwu

So the answer to your question is, in the case when the Swift version is greater than 5.8 but hasFeature(VariadicGenerics) is false, you're holding it wrong and the file won't compile at all;

Ah, I understand the disconnect now! I was thinking of the "compiler(<5.8)" check as only being there to make sure hasFeature was available, but it's communicating more than that in your example: 5.8 is also the place where the feature in our example was introduced.

However, this doesn't account for features that will be enabled in Swift 6 but are opt-in in prior language modes. There, we have to distinguish between "the compiler is 5.8 and the feature is disabled" and "the compiler is 5.8 and the feature is enabled". For such cases, the need to nest hasFeature forces you to have 3 branches (two of which will often be the same) vs. the $ formulation only needing 2 (is the feature on or off?).

@xwu

Now, I'm advocating for some #if hasFeature blocks not taken also not to be parsed in future versions of Swift, perhaps on a feature-by-feature basis, but on reflection that would be unworkable as by construction any older compiler version would not have feature-by-feature knowledge of future features.

Ah, that's a good point.

@fclout

All future Swift compilers that support the Swift 5 language mode support both hasFeature(FEATURE) and $FEATURE for feature detection.

As much as I'm loathe to introduce two features for the same thing at the same time... this does give the nicer syntax over time. We could say that the $ form is only allowed in Swift 5.x mode, and hasFeature is in all language modes.

Thank you, everyone, I think we have some fairly solid direction here, so I'm going to turn this pitch + the suggestions here into a full proposal.

Doug

[JetForMe (Rick M)]

I think this is a great idea.

• I prefer one of the hasFeature() spellings over $
• Please provide a way to list all available feature flags, e.g. swiftc -list-features, that ideally lists the feature flag name as well as a short description that points to the SE-XXXX.

[Douglas_Gregor]

Thank you everyone for the great feedback. I turned this into a proper proposal over at Piecemeal adoption of future language improvements by DougGregor · Pull Request #1660 · apple/swift-evolution · GitHub.

Doug

[hassila (Joakim Hassila)]

Looks really good. Quick read through — big +1. Nice integration with SPM extra bonus points! Thanks.

[benrimmington]

The compiler can define names with a leading $, but developers cannot, so it's effectively a reserved space for "magic" names.

I'm able to define $ names in Swift 5.6 (via Package.swift, I didn't try the command-line).

swiftSettings: [
  .define("$BareSlashRegexLiterals"),
]

#if $BareSlashRegexLiterals
#warning("$BareSlashRegexLiterals is defined") // ⚠️
#else
#error("$BareSlashRegexLiterals is undefined")
#endif

---

Could feature flags be added for the RequirementMachine?
This is currently inaccessible, except for unsafe flags.

swiftSettings: [
  .unsafeFlags([
    "-Xfrontend", "-requirement-machine-abstract-signatures=on",
    "-Xfrontend", "-requirement-machine-inferred-signatures=on",
    "-Xfrontend", "-requirement-machine-protocol-signatures=on",
  ]),
]

[stevapple (YR Chen)]

I'm a little hesitant about being so aggressive on removing feature checks. A warning would be better than an error when you try to enable a feature that is enabled by default, which will give users of lower versions time to migrate.

In the evolution of Swift we may encounter some feature that is not implementable or cannot be enabled on some platform as being blocked by platform-specific issues. This is especially a case for restricted or low-resource platforms (e.g. embedded / WASM), which may unavoidably miss some functionalities. It’s reasonable to provide a way to check for these partially-enabled features even they’re enabled by default.

[tkremenek (Ted Kremenek)]

@Douglas_Gregor

What we are trying to do here is limit permanent forks in the language. By limiting -enable-feature X to features that are going to be enabled in Swift 6, it means that folks can move along a path from Swift 5.x to Swift 6 incrementally.

I have a couple clarifying questions and the proposed behavior.

The compiler currently supports multiple language compatibility modes, such as "Swift 5", "Swift 4.2", "Swift 4". Would the feature flags apply only to "Swift 5" mode, or to all language compatibility modes? Would the compiler reject these flags in Swift 6 mode, or warn about them (since they become vestigial at that point)?

[Douglas_Gregor]

@benrimmington

The compiler can define names with a leading $ , but developers cannot, so it's effectively a reserved space for "magic" names.

I'm able to define $ names in Swift 5.6 (via Package.swift, I didn't try the command-line).

swiftSettings: [
  .define("$BareSlashRegexLiterals"),
]

Oh, oops. I think we meant to prohibit that. I'll fix the wording!

Doug

@benrimmington

Could feature flags be added for the RequirementMachine?
This is currently inaccessible, except for unsafe flags.

No, the requirement machine isn't meant to be a language dialect and shouldn't have flags. The flags are there so we could stage in the requirement machine as a new implementation of the generics system, leaving the old implementation as a workaround for a release or two while we iron out all of the wrinkles. In 5.7, the requirement machine is on by default. On main, the requirement machine is the only implementation and the flags are no-ops.

@stevapple

I'm a little hesitant about being so aggressive on removing feature checks. A warning would be better than an error when you try to enable a feature that is enabled by default, which will give users of lower versions time to migrate.

I don't see why one would need that time. Moving to a new language version (e.g., adopting -swift-version 6) is an explicit action the developer takes. At that point, the compiler errors will also have them remove now-default future feature flags.

@stevapple

In the evolution of Swift we may encounter some feature that is not implementable or cannot be enabled on some platform as being blocked by platform-specific issues. This is especially a case for restricted or low-resource platforms (e.g. embedded / WASM), which may unavoidably miss some functionalities. It’s reasonable to provide a way to check for these partially-enabled features even they’re enabled by default.

If at some point we decide that there is a limited set of features available to restricted environments, it would be possible to add hasFeature(X) checks for features that might not be available. Whether such a subset exists, or what features should be in or out of that subset, should be a separate discussion.

@tkremenek

The compiler currently supports multiple language compatibility modes, such as "Swift 5", "Swift 4.2", "Swift 4". Would the feature flags apply only to "Swift 5" mode, or to all language compatibility modes?

The proposal currently specifies that the feature flags are respected for all language modes up until the language mode where the feature is enabled by default. And in the language modes where the feature is enabled by default, it is an error to pass -enable-future-feature for that feature. We could be more picky, i.e., only allow -enable-future-feature X to be used in (say) Swift 5 when the feature is on-by-default in Swift 6, but I'm not sure there is much benefit to it.

Doug