NIOPipeBootstrap can't be bootstrapped in an eventLoop

I've been working on launching child processes asynchronously (and reading their output) for some transcoding jobs I want to do in the background. (The code can be found here if you're curious: process-nio)

It works well in my little testclient, but as soon as I try to integrate with a simple backend based on Vapor I hit the following precondition:
NIOPosix/Bootstrap.swift:1026: Precondition failed: limitation in SwiftNIO: cannot bootstrap PipeChannel on EventLoop.

What are good ways to work around this, or put differently: how would I design the way I'm trying to use NIO for async IO better around my use case? Could I 'abuse' some of the pre-existing infrastructure bootstrapped for the webserver to achieve my goals?

I strictly tried to not use NonBlockingFileIO since it uses a thread pool and I specifically wanted async reading since I'm using a Pipe. I could perhaps create the bootstrap before I'm in any event loops, however I'm not really sure if I could re-use that easily every time I want to launch a child process.

1 Like

I added this precondition and I think it was a mistake, I think we should just delete this line without replacement. CC @lukasa

The idea was that we need to check if the file descriptor is of the right kind (socket/pipe/...) and that's done with fstat.
Now in theory fstat could block if you passed in say a file descriptor for a file on a really slow NFS volume or something. And we thought it'd be good to protect users against that.

But: That was a terrible idea, for many reasons:

  • fstat will only block if you pass in a file descriptor for a file on a network volume. And if you did that, the bootstrap would fail anyway. There is absolutely no reason to assume that users would casually pass random file descriptors there...
  • Even if users did that, the kernel almost certainly has that information available anyway, it's an already-open file descriptor after all
  • there's no good way to fix this. AFAIK, there's no way in Linux to check the "type" of a file descriptor without blocking
  • There are so many other, much more scary ways of blocking an EventLoop, this one really isn't important enough to warrant this big limitation.

If I were you, I'd make a pull request to NIO which just deletes this precondition, it's silly and I should've never put it there. I think I wanted to see if somebody has an idea in this StackOverflow question.

2 Likes

I'll happily accept a PR that deletes that precondition and a test to confirm it works fine.

2 Likes

Thank you for the replies. This is exactly what I was looking for.

  • A solution, which I'll admit I was going to try to just delete the line and see what other parts of NIO would blow up and/or scream at me :innocent:
  • Understanding why the line is there and what it is trying to prevent is very valueable to me.
    I tried looking at the exact commit where it was introduced, hoping to find some more information on why it was put there.
    I've created a tiny pull request removing the precondition at github.com/apple/swift-nio/pull/1977
1 Like
Terms of Service

Privacy Policy

Cookie Policy