You can absolutely do distro-less Swift containers built from a SCRATCH base and make really small deployment containers. The trick is that the Swift runtime itself needs to be in a separate container that is only deployed once. I do precisely this in: GitHub - CSCIX65G/SwiftCrossCompilers: SPM toolchain to cross compile Raspberry Pi (arm64) and Amd64 Swift binaries on macOS , the Swift NIO echo server I create in the subdir there came in at <4MB when stripped the last time I looked. I expect a Vaporized version of something similar would only add 1MB or so. Probably even less if you moved Vapor and its dependencies into their own separate container as well.
I'd imagine that your dev-ops guy's objection is not nearly as much to size as it is to the use of any distro which would allow you to get to a shell prompt in his production environment. I mean if you are deploying to a lot of machines the difference between 5MB and 100MB is pretty important for deploy times alone, but the real benefit to distro-less containers is in minimizing the security cross-section, IMO.