Swift NIO `doSequentially` - confused about apparent stack overflow

I tried to write a doSequentially function for use with NIO futures. (It didn't seem like it was in the library.) I'm getting a EXC_BAD_ACCESS and the huge stack makes me think it's a stack overflow. The stack trace contains many frames of the whenSuccess closure below. I don't see what's wrong with my function. Seems like it should not pile up on the stack. If anyone familiar with NIO sees the problem, please let me know.

extension EventLoopFuture {
    static func doSequentially<A>(_ values: [A], on loop: EventLoop, once: @escaping (A) -> EventLoopFuture<Value>) -> EventLoopFuture<[Value]> {
        var i = 0
        var result = [Value]()
        let promise = loop.makePromise(of: [Value].self)
        func step() {
            if i >= values.count {
                promise.succeed(result)
                return
            }
            let input = values[i]
            let fut = once(input)
            fut.whenFailure { e in
                promise.fail(e)
            }
            fut.whenSuccess { b in
                result.append(b)
                i += 1
                step()
            }
        }
        step()
        return promise.futureResult
    }
        
}

I wrapped the body of my whenSuccess block in a loop.execute { } and that fixed it. Async programming drives me nuts...

Let’s talk a bit about what happens when you attach a callback to a NIO Future (e.g. by calling whenSuccess).

Roughly the following steps happen:

  1. NIO checks the Future’s EventLoop. This is the EventLoop on which the callback must execute. If the code currently running is not on that EventLoop, NIO will enqueue a block to the target EventLoop that starts this process again. At this point, the whenSuccess function (or similar) will return, as it can do no more.
  2. If we are currently executing on the Future’s EventLoop, we check whether the Future has a result already. If it does not, we attach the callback to the callback list and return from the whenSuccess function, as it can do no more.
  3. If the Future does have a result already then we immediately, synchronously invoke it.

This is what’s happening to you here. If the once() function returns a Future that a) belongs to the loop you’re currently on and b) is already complete, the whenSuccess block will immediately fire. This will recurse into this method inline. If once’s future is always completed (e.g. because you used makeSucceededFuture) then you will recurse until you have no more work to do.

Adding an eventloop.execute breaks this recursive chain and unwinds the stack, letting us start again.

Does execute guarantee that it's not going to try the same optimization and cause the same problem? It's function documentation (just one sentence) doesn't mention that.

We guarantee it, as we use it for this purpose ourselves.

1 Like
Terms of Service

Privacy Policy

Cookie Policy