What I was trying to show was that async could be generic. The syntax that I showed with the Bool generic parameter is the same syntax that is already used for integer parameters on InlineArray. The only changes that would need to be made to let this work are enabling Bool for value generics and allowing the async(isAsync) syntax to conditionally enable async for different specializations of the function.
Right I understand, and I think the direction is very interesting!
But I do not think it is equivalent to what InlineArray is doing, because that mechanism requires that the generic parameter is statically known. I do not believe that the type system supports doing this. For example, the following does not compile, and I think getting it to work would qualify as adding the "value-dependent type" concept.
let count = 5
let array: InlineArray<count, Int>
But again, I could be wrong!
Edit: Wait, I think I am wrong. My apologies!
Very sorry, again, for getting confused and distracting the conversation here. Generic arguments != function arguments apparently. I'll figure this Swift thing out one of these years.
If I may, I do still have a question about this. I assume the vision here is that the boolean generic argument will be inferred by the compiler at callsites?
And at that point, can you help me understand how this is meaningfully-different from a reasync effect specifier? Under what conditions would a generic-based conditional async look different? How would it allow for more expressiveness?
I fully believe that I'm just being slow and not seeing it. But I find the suggestion so interesting that I really want to better understand the concept.
2 Likes
rethrows <=> reasync, implicitly for the whole function, based on its actual closures.
Bool generics + async(Bool) or similar <=> typed throws, explicitly based on type inference from particular parameters.
But, the reason that rethrows + typed throws work is that all the variants can be compiled to one single implementation. And my understanding of the reason that reasync has been pushed back on is that an async function and a sync function fundamentally can't be compiled to one single implementation ā the async version is split up into independent functions at suspension points, and the sync version is not. reasync or async(Bool) would effectively have to emit two copies of the function, one for sync and one for async.
That's why the macro version is attractive: if you have to emit two copies anyway, why not keep it out of the language and just do it syntactically, with existing tools?
I'd still rather see this as a package (possibly under swiftlang) than embedded in the stdlib, though. I think the expectation of a stdlib macro would be that it works rather better than a syntax-based solution could ever hope to.
4 Likes
What I'm really looking for is a specific example with this generic-based syntax that can do something that reasync could not. I think I (now, finally) understand the concept. But I do not understand how it would be used practically in a way that provides more functionality.
contrived example, but
func do(
thisFirst: () async -> Void,
thenStartThis: () async -> Void,
) async -> Task<Void, Never> {
await thisFirst()
return Task(operation: thenStartThis)
}
If we just have reasync, then this is synchronous only if both closure parameters are synchronous, when it should be synchronous whenever the first closure parameter is synchronous; if we have a generics based solution then we can describe the desired behavior. Stealing the hypothetical syntax from upthread:
func do<let isAsync: Bool>(
thisFirst: () async(isAsync) -> Void,
thenStartThis: () async -> Void,
) async(isAsync) -> Task<Void, Never> {
await(isAsync) thisFirst()
return Task(operation: thenStartThis)
}
3 Likes
AFAIK typed throws are true generic function that can be specialized at compile time. When the functionās body is available to the compiler (it is in the same module or is inlinable), the compiler literally makes a copy of the function with the generics replaced with concrete types. Thatās specialization. When the compiler canāt specialize, it just treats all parameters as existential boxes (basically any). So, I see no reason why we couldnāt do the same for async. It could just strip out the suspension points as they arenāt needed for no-async code. And, the unspecialized variant could just leave the suspension points, but directly call the next chunk afterwards.
Iām not actually sure how rethrows does this internally, but it must be similar to typed throws because it can be migrated to typed throws without breaking ABI.
A suspension point isn't an instruction in a function body, the function is literally broken up into multiple functions at suspension points. So the compiled version of an async function looks nothing like the compiled version of a syntactically-similar synchronous function. Godbolt: Compiler Explorer
1 Like
Yes, but I think that splitting happens after generic specialization during compilation. The function is represented normally throughout generic specialization, all you have to do is not split the function up unless the condition in async(isAsync) is true.
For the unspecialized version, the function could just run like any async function, and at each suspension point it can directly call into the next chunk. It doesnāt need to actually suspend.
1 Like
The symbol graph and nested type issues affecting all peer macros will be fixed pending approval of two new PRs.
Symbol graph
In the meantime, the workaround when generating documentation with DocC is to create a separate DocC article for the synchronous overload. DocC recognizes the peer symbol and will attach the article to it when building the site.
Nested types
In the meantime, the workaround is to declare nested types outside the macro-attributed function.
Just to add to this - we've done similar things in Vapor and having a sync and async version of the same function has caused a ton of problems, API breaks and confusion in the compiler, especially when it comes to passing closures around. It would be good to solve these issues in the compiler first before adopting something like this
Could you share some specific examples of problems you ran into? These issues would already exist in any handwritten sync/async pair, so they're worth understanding regardless of how the declarations are produced.
4 Likes
IIRC initialisers don't work correctly and it always picks the sync version. It also introduced some breaking changes as we thought the compiler would pick the correct function when it had the option but this wasn't always the case so downstream users would get errors.
Closures have issues because it will fail to pick the async one, so you have to be explicit about it. E.g.
app.test(.GET, "route") { req in
let result = try await someAsyncCall() // ERROR
}
app.test(.GET, "route") { req async throws in
let result = try await someAsyncCall() // ERROR
}
Though I believe closures are out of scope for this?
Have you considered adding the Reasync attribute to sync functions, and for the macro to generate the async version instead? This was the approach I took as I was coincidentally creating something similar at the same time.
I think the name issue with overload resolution could be fixed by optionally providing a custom name (like @Reasync(named: "fooAsync").
My version:
2 Likes
Yes, but it can't be done correctly with a syntactic macro. There's no single correct place to insert await, since a synchronous function body doesn't generally have enough information to determine which calls should become asynchronous. The macro would need semantic knowledge which it doesn't have. More on this in the Alternatives Considered section of the full proposal.
I don't think there's an overload resolution issue to solve. The macro generates a true overload, and Swift's existing overload resolution selects the appropriate overload based on the calling context. A call from a synchronous context resolves to the generated synchronous overload, and a call from an asynchronous context resolves to the async source. More on this in the Overload Resolution subsection of Detailed Design.
3 Likes
After thinking about it a bit, there isn't really a way to make sure of this 100% of the time, even if we somehow perfectly analyse the function body. The only way to fix this would be for the developer to manually specify exceptions, which adds more friction. Doing the opposite would be easier.
Sorry, this was poor wording on my part. What I intended to say was to give the choice to rename the async function.
As for more feedback, I would propose either getting rid of @ReasyncMembers altogether or to add an additional macro, something like @ReasyncMembersIgnored.
The current rules for reasync to take effect on a function inside of a @ReasyncMembers declaration is too aggressive in my opinion, as from my understanding it also takes effect on functions that don't have closure parameters. In my view, @ReasyncMembers should only operate on functions that have async closure parameters, otherwise it would be pointless and would be extremely likely to cause a compile failure.
import Foundation
@ReasyncMembers
struct Foo {
func foo(operation: () async -> Void) async {
await operation()
}
func asyncBar() async throws {
let session = URLSession.shared
_ = try await session.data(from: URL(string: "https://swift.org")!)
}
}
I would recommend making @ReasyncMembers stricter, with it only taking effect on functions that have async closure parameters.
@ReasyncMembersIgnored could function like @ObservationIgnored, and just causes @ReasyncMembers to ignore that specific function declaration. This could help to solve the above case.
Overall a +1 from me with these changes considered.
Since this is pitching to add to the standard library, have we considered just adding a reasync attribute in a similar fashion to rethrows?
2 Likes
A language-level reasync has been discussed before, but no proposals have advanced. The pitch post quotes Doug Gregor's observation, where he notes that the motivation is easy, but the compiler implementation is a bunch of work, and that a peer macro can plausibly cover the common case without language-level changes.
This pitch also proposes addition of the macros as a swiftlang-org package, rather than as part of the standard library.
The full proposal discusses this further in the Alternatives Considered section.
1 Like
The language steering group evaluated this proposal in preparation for review, and we believe that this proposal addresses a significant enough problem to consider for the standard library. We would like to see the proposal incorporate some feedback from this pitch thread and address some of the following points prior to review:
Implementation as a macro
The pitch discussion has explored the limitations of a macro in addressing the problem of deriving a synchronous variant of an async function. The language steering group believes that a macro is a feasible solution worth considering for the standard library. Even if we pursued a more integrated reasync type system solution in the style of rethrows, it would need to have a macro-like underlying implementation, generating two separate machine-level functions, since synchronous and async functions cannot share a calling convention, so we get less benefit from the polymorphic behavior than we do from rethrows. Futhermore, async code offers many more possibilities for semantic distinctions between synchronous and async variations of a function, such as different interactions with isolation and sendability, or different parallel execution strategies that aren't readily available in synchronous code, so it isn't as clear-cut that there is a one-size-fits-most solution like rethrows for async. A macro potentially offers more flexibility to evolve in response to unanticipated needs. Being a macro, developers could even evolve their code by removing the macro and switching to a separately-written synchronous variant if necessary, without disturbing API or ABI, which would not necessarily be possible starting from a single reasync declaration.
The proposal text does acknowledge language-integrated reasync as an alternative solution, as well as a future direction, but the steering group does not see the implementation of reasync as a sure thing, and if we do end up accepting a proposal such as this one for a standard library macro that sees wide adoption, we anticipate it being even less likely that we would also pursue an integrated feature. We would like to see the proposal address reasync as a true alternative and discuss these tradeoffs and why the author sees the macro as the favored approach.
Alternative macro implementations, and interactions with other concurrency features
A number of community members compared the proposed implementation to their own implementations of macros providing similar functionality, such as David Catmull's @DeAsync and Matthew Yuen's stdlib-utils. Some of these implementations take different approaches or provide additional control over how the synchronous variation is generated. We would like to see the proposal address these alternative implementations, either adopting their functionality or providing rationale for eliding their features in the proposal's discussion of alternatives considered.
We also share concerns with some commenters that the proposal should further explore the interaction of the macro with @Sendable, isolation, and strict concurrency requirements. We note that many of the examples in the proposal do not compile in Swift 6 language mode, for instance:
% cat ~/test.swift
func sum(
_ a : Int,
_ b : Int,
using compute : (Int) async -> Int
) async -> Int
{
async let x : Int = compute(a)
async let y : Int = compute(b)
return await x + y
}
% ./bin/swiftc -swift-version 6 ~/test.swift
/Users/jgroff/test.swift:7:27: error: sending 'compute' risks causing data races [#RegionIsolation::SendingRisksDataRace]
5 | ) async -> Int
6 | {
7 | async let x : Int = compute(a)
| |- error: sending 'compute' risks causing data races [#RegionIsolation::SendingRisksDataRace]
| `- note: sending task-isolated 'compute' into async let risks causing data races between nonisolated and task-isolated uses
8 | async let y : Int = compute(b)
9 |
/Users/jgroff/test.swift:8:27: error: sending 'compute' risks causing data races [#RegionIsolation::SendingRisksDataRace]
6 | {
7 | async let x : Int = compute(a)
8 | async let y : Int = compute(b)
| |- error: sending 'compute' risks causing data races [#RegionIsolation::SendingRisksDataRace]
| `- note: sending task-isolated 'compute' into async let risks causing data races between nonisolated and task-isolated uses
9 |
10 | return await x + y
In order for this code to be valid under strict concurrency rules, the closure parameter would need to be @Sendable, and at least in this case, that @Sendable annotation could be filtered from the generated synchronous variant since the parallel execution is eliminated (though that may not necessarily be a universally applicable part of the transform).
To serve as the standard library solution to synchronous variant generation, the implementation should be prepared for strict concurrency, and capable of performing the transformation in the most common ways that will be necessary in practice to produce a valid synchronous variant.
Naming
The language steering group shares concerns with the community about the naming Reasync, since this macro is in many ways more general in its application than rethrows, and mechanically it removes async annotations rather than reintroducing asynchronous from closure parameters. The name both potentially undersells its capability to people familiar with rethrows and potentially misleads developers who are not. However, we also acknowledge that "reasync" is the name that many Swift developers ask for when looking for a feature like this, and we do not have a strong preference for an alternative. We anticipate that there will be a lot of feedback about naming during review, and it would be good for the proposal to gather the alternatives that have been raised during the pitch discussion in its alternatives considered section, but we are OK with starting the review with the name @Reasync. We do however think that it would be good for the title of the proposal to be more descriptive of what the macro does, beyond just including the name.
@ReasyncMembers
We think that the @ReasyncMembers macro could be separated from this proposal, since it is not part of the core functionality.
13 Likes