Reliably testing code that adopts Swift Concurrency?

Using Swift
,
90

Wild crazy ideas mode on, none of this is a thorough design;

That's a good example, if we start small with a thing like that we can start considering bigger examples.

Fundamentally I think there's two approaches here:

  • specific order where the test programmer is able to instruct execution to "I want the A task and then the B task"
    • I wonder if this could be done with a custom executor that would just inspect either a task-local within those jobs for an identifier, or using some other method to name tasks and "hold off scheduling one before the other"
  • fuzzing / tracing execution, we'd again need a way to identify some tasks and record the order in which we've executed them. (the task on ...:12 was first, next the one created on ...:24 etc), we could have a tracing executor which
    • I wonder if a simple form of that we could write as a specialized executor if we had the power to identify the job -> source location would be at least a brute "run many times" but once we hit a failure we would be able to dump the execution order of all tasks this executor has executed... This would then feed into the idea 1) where we force that execution order in order to debug issues :thinking:
    • the ultimate form of that is what e.g. foundationdb is able to pull off with their "system execution is 100% reproducible" but that would mean the entire system has to run on this "seeded, order fuzzing executor" and side effects cause problems to such system... I wonder what we could learn from such approaches though.

I think it'd definitely be worth exploring how far we could get with a custom executor if we'd be able to identify jobs to where they've been started -- I wonder if that would be useful already...

End of wild crazy ideas mode

3 Likes