Parallel computation DAG / Shared Futures?

@dmt (and @ktoso) That's not bad, thank you! I didn't realize that Task was essentially a future, but I see that it is. I was surprised to see that this didn't seem to be creating a huge number of thread and tended to execute tasks on the same thread where possible. I wonder what the actual synchronization cost is but this looks like a good start.

When I previously looked at this problem I seized on Operation because it seems to have some ability to handle dependencies, and presumably that could allow scheduling with less synchronization? Any insight there? I could of course try to measure the result. I think because operations are discovered I would still need an actor to coordinate and prevent operations from being duplicated.

Update:
I tried benchmarking @dmt's Task method vs using Operation and it looks a lot like Tasks win. They're certainly a lot simpler to use! And it's worth noting that it looks ike Operation doesn't really allow for dependency discovery in the same way; the dependencies really have to be discovered in a phase prior to starting a computation.

1 Like