No. Test traits only run at runtime, which is far too late to affect the compiler.
The macro expansion can't affect the body of the test function as originally written, and it still needs to compile for the current target platform before the macro can do any useful work.
I was trying to look these functions up (because I have a similar situation as well). But got all sorts of other unrelated references.
What type/package are these part of?
POSIX 
If we feel a joiner word is necessary, there are better options than With. exitsReturning, exitsThrowing, exitsProducing, exitsFrom, exitsDueTo, all of them are better options. But I think #expect(exit: .failure), #expect(exit: .signal()) and #expect(exit: .code()) read really well by themselves. I also agree that exitCode is redundant if we're already labeling the parameter with exit in the name. We could also nest these failures inside failure itself. #expect(exit: .failure(.code(22)))
On a side note, I don't see any example output for these tests, it seems that like should be part of Testing proposals so we can ensure the output is what we'd expect.
I do discuss a few other options in the alternatives considered section, and I specifically called out "due to" as a possible alternative. None of the above options are factually correct except "due to", but as I wrote:
[Y]ou probably wouldn't say "exits due to success" in English.
#expect(exit: ...) does not follow the naming guidelines for Swift API. See here.
I hope that's helpful!
I'm not sure what you'd like to see? There is no command-line or IDE output specific to exit tests as they are represented in the event stream as expectations (either passing or failing.)
This seems unnecessarily facetious and bit insulting. I'm quite familiar with the guidelines, and it's well established that "fluent usage" does not mean "plain English" in Swift APIs. And I'd point out that "with" generally falls afoul of Swift's "no joiner words" guideline as well, as it provides no useful information, and they're filled in by readers anyway. I mean, you wouldn't say "exits with success" in typical English either. Spoken English isn't usually what we strive for in fluent API design anyway. If it were, we wouldn't have trailing closures, especially in this API (#expect({ someWork() }, to: .successfullyExit)). If there's more useful information to convey in the parameter name it should do so, so what are you thinking of for your "factually correct" naming?
This seems like a totally reasonable conversation to have; let's try to keep it friendly and productive.
Hi Jon,
Thank you for your feedback.
Please note that I keep a record of all suggested improvements. And we will go over all of them with the workgroup once the review period ends. Your suggestions, as well as the ones made by xwu, danny, macguru and mtsrodrigues, are all on the list.
KR Maarten
Apologies for the delay in replying—family matters and whatnot!
Sorry about that. I wasn't trying to be facetious and genuinely meant that I wanted to help! I'll try to reply to your points here with specifics that will hopefully make my thought process clearer.
They terms "fluent language" and "plain/spoken English" are certainly not meant to be isomorphic, but the guidelines state:
Strive for Fluent Usage
- Prefer method and function names that make use sites form grammatical English phrases. [emphasis theirs]
And in general, we don't avoid prepositions in symbol names in Swift. We do try to avoid redundant nouns unless they are required to avoid ambiguity (which is why .exitCode(_:) is not simply named .code(_:).)
That isn't really a rule of the Swift API Guidelines. Apple does tend to avoid the word "with", but Swift doesn't. For example, and in no particular order:
SIMD2.replace(with:where:) (and similar members on other SIMD types);String.init(unsafeUninitializedCapacity:initializingUTF8With:) and Array.init(unsafeUninitializedCapacity:initializingWith:);Dictionary.init(uniqueKeysWithValues:), init(_:uniquingKeysWith:), merge(_:uniquingKeysWith:), and merging(_:uniquingKeysWith:);Set.isDisjoint(with:);CheckedContinuation.resume(with:) and AsyncStream.Continuation.yield(with:); andCollection: starts(with:), replaceSubrange(_:with:), and update(with:).That's true. .success is the odd one out here because you can say:
SIGTERM"This is a constraint of English more than of Swift. The terms "normal" and "abnormal" are used to describe process termination in some of the relevant literature—perhaps this would work better if the .success and .failure cases were named .normally and .abnormally instead, but then we'd still need with on the exit code and signal cases in order for them to remain grammatical:
await #expect(exits: .abnormally) { ... }
await #expect(exits: .withSignal(SIGTERM)) { ... }
// or possibly:
await #expect(exits: .with(.signal(SIGTERM))) { ... }
Mulling that further, if we named the latter case .reporting(), it would read:
await #expect(exits: .reporting(.signal(SIGTERM))) { ... }
Which reads decently, although I'm not sure how discoverable it would be with Xcode's autocomplete engine.
"Omit needless words" is—or at least was—certainly part of the Swift API guidelines, and if it was eliminated editorially at some later point that would have been in error and not on the basis of community discussion.
At the time of guideline adoption, we certainly did use that rule to eliminate joiners, with particular attention to with when it introduced arguments. The examples you cite where with is currently used in the Swift standard library broadly fall into two categories:
labels for closures: which, by a convention first proposed by @dabrahams to replace a much more unwieldy earlier convention, often uses a preposition like by or with—precisely because it is vacuous and therefore can be dropped in trailing closure position without loss of information
in contexts where the absence of "with" totally changes (often, reverses) the meaning: for example, "replace x" and "replace with x" mean opposite things; "starts x" and "starts with x"; etc.
Any examples that don't fall under one of those categories (that is, vacuous uses of "with" other than those introducing a closure) ought to have been struck out at the time of Evolution review.
But it does not read fluently to say: "expect exits with exit code 5," and if you have to read in other words, like "expect [that it] exits with exit code 5," then go all the way: "expect exit code 5."
Hi xwu,
Does this mean that you propose this API:
#expect(exit: .failure)
as stated in your earlier reply: ST-0008: Exit Tests - #2 by xwu ?
That's the bucket this API falls into. Without "exits with" (or some other better phrase to replace "exits with"), the effect of the expression is ambiguous. For example, if we just wrote #expect(.failure) { ... } or #expect(.signal(SIGSEGV)) { ... } (whether or not these are good spellings isn't the point), it would be completely lost on the reader that the closures will run in external contexts.
Respectfully, eliding "that it" here, or rather "[that] the closure", is following another related rule:
Omit needless words. Every word in a name should convey salient information at the use site. [...] In particular, omit words thatmerely repeat type information.
The reader already knows there's a closure involved because it's present syntactically.
I think we need to workshop this more intensively then, if that's the goal.
It is completely lost on me, at least, that exitsWith is supposed to convey that the closure runs in an external context. I'm not sure that any label could convey that meaning, but my feedback would be that neither the "s" after "exit" nor the "with" does the job.
Sorry, to clarify, I'm not suggesting that you should add more words—yes, you should absolutely not write in "that" or other joiners: I'm suggesting that you slice more off until it reads more idiomatically.
I do like that form much better, although as I said it is a minor point.
However, I feel it's worth clarifying what those of us who were around during the API naming guideline debate meant at the time by them, as it sounds like they are not well understood anymore.
Then it's worth considering alternatives to the whole phrase "exits with", perhaps. Just using "exit" in this position would be grammatically incorrect, so we would want to look at other options.
I suspect we're talking past each other more than anything else. I understand the guidelines just fine, but I fully admit I'm not always the best at explaining my thought process. 
Edit: Any thoughts on something like:
await #expect(exits: .abnormally) { ... }
await #expect(exits: .reporting(.signal(SIGTERM))) { ... }
?
Yeah, I have to admit I'm not seeing the grammatical incorrectness. "Expect exit: failure" reads at a minimum (to me) at least as fluently as "Mission: Impossible II"—which is by now idiomatic English.
I think it's worth considering alternatives only if your overriding goal is to somehow work in an indication that the closure runs in an external context, but honestly I have trouble envisioning how to accomplish that :)
It is certainly fluent-reading!
Since you asked—speaking frankly but only for myself personally—it is the diametrically opposite direction from what I would have aimed for.
To put it into historical context, the API naming guidelines were written at a time when many APIs bridged from Objective-C had much longer base names and argument labels that were idiomatic in that language.
The guidelines for brevity and fluency were well settled before we added "-ed/-ing" rules to distinguish mutating/non-mutating pairs of APIs, and also before wide adoption of prepositions for argument labels, which evolved naturally as we discovered that some APIs could be more fluently and briefly spelled at use sites with a preposition instead.
It was never actively the point (at least, for me) that we should mash all of these naming guidelines together to add words or add suffixes with the sole or primary purpose of making an API read like a sentence. Overall, the effect is a sort of backsliding into an idiom that the naming guidelines were meant to take us away from in the first place.
My personal feeling is that the trend in parts of the Swift ecosystem to use every part of speech in this way—that is, library "Foo" vending APIs that read like FooingFoo.fooed(withFooable: Fooer.makeFoo().asFooingFoo)—is one of those unintended (and regrettable) interpretations of the guidelines that ought to have been nipped much earlier.
(And I say this as someone who over the past decade has thought approvingly of APIs such as allSatisfy and squareRoot—and I know there are others from that time who felt then and still now that even those are a bridge too far.)
Could there be a #subprocess macro, which doesn't need any particular termination status? Instead, the existing #require or #expect macros would be used afterwards.
let result = await #subprocess {
fatalError("message")
}
#expect(result.terminationStatus.isFailure)
Possibly the result's APIs would be similar to CollectedResult or ExecutionResult from SF-0007.
Mechanically yes, however Swift Testing is not trying to build a general-purpose replacement for fork() nor for Foundation API. (I mention integration with Foundation in the Future Directions section.)
Hello everyone,
We're coming up on the last couple of days for this review.
So far reactions about having this feature seem unanimously positive. However there is some debate about the API and several alternatives have been proposed.
My question is: do you have a preference for a specific API? It can be the API as stated in the proposal, but also one of the alternatives suggested in the thread. And even if you don't have a real preference, that's valuable feedback for us to learn about as well.
+1. If the exit is a code, it's an exit code, by definition. No?
The only reason I see for going with exitCode is that "exit code" is already a term of art. But I'm not convinced that justifies the verbose #expect(exitWith: .exitCode(...)).
And if the associated value for .exitCode is a local variable, the use site would look even more redundant:
let exitCode = expectedExitCode(for: condition)
#expect(exitWith: .exitCode(exitCode)) {
// ...
}
Vs maybe something like:
let expectedCode = expectedExitCode(for: condition)
#expect(exit: .code(exitCode)) {
// ...
}
I don't think #expect(.failure) totally changes the meaning of #expect(exitsWith: .failure). Sure, the "exit" bit adds important information, but removing it doesn't fundamentally challenge what I'd expect the API to do based on the name. Even more true for #expect(.signal(SIGSEGV)) where I don't see any other reasonable thing for the API to do.
As for conveying that it runs in an external context: when I read #expect(.signal(SIGSEGV)) I would expect the test to succeed when the code in the closure signals a SIGSEGV, without crashing the entire test suite. Which I think is all I'd need to know most of the time.
+1, it's also lost on me. Also agree that it's difficult for a short label to convey that it runs in an external process.
FWIW I read #expect(exit: .failure) as "#expect [an] exit [of type] failure. It's only grammatically incorrect if we consider the ommited subject is "this code", as in "#expect [this code] exits [with] failure".
Nested enum cases are a bit annoying to deal with. Plus although it does read well, I'm not really getting any extra information by the addition of .reporting(.signal(...)) vs just .signal(...). Of course it's reported, how would the API work otherwise?
I would prefer not having "with" spelled out in the label in any case. No strong opinion about #expect(exit: .failure) or #expect(exits: .failure) vs just #expect(.failure), though I think the "exit" bit is important an thus prefer the former (also, I agree that some people could confuse #expect(.failure) with something like XCTFail).
I also prefer #expect(exit: .failure). To me it reads just like any other #expect(value == 42) (except that reading out loud I would pronounce it “expect exit is failure” versus “expect value equals 42”).