[Pitch] Polling Expectations

Development Swift Testing
1

Something I've found to be of immense utility in tests is the ability to continuously re-evaluate an expression until it passes. Think monitoring changes made by a background activity when it doesn't make sense to add a callback mechanism or similar async mechanism (for example, that a ViewModel's property is updated as a result of a network request or db query). This proposal adds a way to check that an expression passes at some point during the course of a given timeout period, as well as a way to check that an expression always passes during the given timeout period.

Read the full proposal here.

4 Likes
2

First question: does this need to be a macro? Could it instead be something similar to withKnownIssue where it looks out for/intercepts issues? Something like:

await confirmationButNotNamingIsHard(.passes(10)) {
  let potato = try #require(await downloadPotato())
  #expect(potato.isTasty)
}

?

3

That's a very good point to bring up. This proposes macros without too much thought into alternatives - or how difficult it is to write & maintain macros :grin:. I was much more concerned with the expected behavior of this.

It really comes down to how often we expect or want people to use this. The test author experience is a bit more seamless if we use macros. One of my complaints about the existing confirmation api is that extra bit of setup required, which pushes me to use other mechanisms over callbacks. On the other hand having that little extra friction might be desirable to prevent overuse. Working with Nimble, I've definitely seen developers default to using polling expectations over standard expectations in order to avoid having to think about which is actually appropriate. Which isn't all that bad, but definitely is a smell. I don't know how to weigh that against the desire to make it easy to use.

4

It's… well, complicated to say the least. And it affects codegen and documentation in sometimes-unexpected ways (e.g. source location information is less reliable.) So if we can find a way to build API here without using them, that's preferable. That said, the closure-takin' #expect macro implementation is probably already sufficiently built out that adding something like this wouldn't involve much new macro code, if any.

Dunno if that scattered collection of thoughts helps… :melting_face:

1 Like