Chiselled containers are based off the Distroless concept and provide very stripped down environments for your applications (no shell, no package manager etc) which provides smaller container images and a smaller attack surface.
It turns out, they're very easy to build. I've put a demo repo on GitHub but the container sizes built are:
Chiselled: 13.2MB
Non-chiselled: 83.7MB
As you can see, these are nearly an order of magnitude better in terms of size. The non-chiselled container uses an adapted version of the Vapor template Dockerfile which runs everything on a base Ubuntu image (so no Swift toolchain etc)
The builder image is the regular Swift image (see here) but the image used to generate the container is a chiselled image - there is no Swift runtime in the final image, otherwise it would be many GBs
It doesn't live anywhere. A Swift Docker container is used to compile the app, then it launches a Chiselled container and throws away everything with the runtime in - does that make sense?
Guess I'm not following. When I've build distroless containers for NIO-based apps before, I've always needed the following run-time shlibs to be available. (this is my current 5.7 run-time)
0 Apr 18 22:00 usr/lib/swift/
371424 Jan 15 13:55 usr/lib/libswiftDemangle.so
0 Apr 18 22:00 usr/lib/swift/pm/
0 Apr 18 22:00 usr/lib/swift/linux/
4488056 Jan 15 13:55 usr/lib/swift/linux/lib_InternalSwiftStaticMirror.so
12643328 Jan 15 14:23 usr/lib/swift/linux/libFoundation.so
100424 Jan 15 14:08 usr/lib/swift/linux/libswiftGlibc.so
0 Jan 15 14:21 usr/lib/swift/linux/libicui18nswift.so -> libicui18nswift.so.65.1
438136 Jan 15 14:21 usr/lib/swift/linux/libdispatch.so
199592 Jan 15 14:10 usr/lib/swift/linux/libswiftRegexBuilder.so
664992 Jan 15 14:08 usr/lib/swift/linux/libswift_Concurrency.so
2350936 Jan 15 14:21 usr/lib/swift/linux/libicuucswift.so.65.1
13800 Jan 15 14:21 usr/lib/swift/linux/libBlocksRuntime.so
647136 Jan 15 14:26 usr/lib/swift/linux/libXCTest.so
8679792 Jan 15 14:08 usr/lib/swift/linux/libswiftCore.so
204792 Jan 15 14:08 usr/lib/swift/linux/libswiftDistributed.so
121596632 Jan 15 13:59 usr/lib/swift/linux/lib_InternalSwiftScan.so
639984 Jan 15 14:08 usr/lib/swift/linux/libswift_Differentiation.so
1927712 Jan 15 14:26 usr/lib/swift/linux/libFoundationNetworking.so
0 Jan 15 14:21 usr/lib/swift/linux/libicui18nswift.so.65 -> libicui18nswift.so.65.1
387792 Jan 15 14:08 usr/lib/swift/linux/libswiftSwiftOnoneSupport.so
0 Jan 15 14:21 usr/lib/swift/linux/libicudataswift.so -> libicudataswift.so.65.1
1346544 Jan 15 14:10 usr/lib/swift/linux/libswift_StringProcessing.so
27981088 Jan 15 14:21 usr/lib/swift/linux/libicudataswift.so.65.1
21302432 Jan 15 13:59 usr/lib/swift/linux/lib_InternalSwiftSyntaxParser.so
0 Jan 15 14:21 usr/lib/swift/linux/libicuucswift.so -> libicuucswift.so.65.1
3980312 Jan 15 14:21 usr/lib/swift/linux/libicui18nswift.so.65.1
355992 Jan 15 14:21 usr/lib/swift/linux/libswiftDispatch.so
0 Jan 15 14:21 usr/lib/swift/linux/libicudataswift.so.65 -> libicudataswift.so.65.1
0 Jan 15 14:21 usr/lib/swift/linux/libicuucswift.so.65 -> libicuucswift.so.65.1
1594728 Jan 15 14:09 usr/lib/swift/linux/libswift_RegexParser.so
503280 Jan 15 14:23 usr/lib/swift/linux/libFoundationXML.so
876840 Jan 15 13:55 usr/lib/swift/linux/libswiftRemoteMirror.so
Generally my distroless containers have always come in about 9MB, but I need that set of shlibs at those version numbers plus whatever dependencies they pull in to be somehow available on the image to let my 9MB nio executable run.
My approach has been to put all the depends in a separate docker volume so that I can run my focal-compiled app on a focal or jazzy or bionic or Yocto instance and only redeploy the 9MB distroless container as I do upgrades.
It's all of those shlibs that might have to be available at run time that have complicated life here, so I was wondering how Chiselled was managing that. In particular I was wondering how you were chiselling out everything you didn't need from the underlying instance, i.e. is Chiselled taking your compiled app and doing the dependency analysis to figure out what parts of that list above you don't need? If you recompile and add a new dependency lib does that mean that you have to redeploy an entirely new Chiselled OS instance? (that last is the part that won't work for me managing a fleet of Raspberry Pi's where I can't upgrade the base OS easily)
If chiselled is actually doing the full dependency analysis for you and putting the depends in the underlying instance image then that is almost what I would actually want. I'd just need to take all those depends and put them in a docker volume that was separate from the distroless container.
Ok looking in more detail at your example, I probably can't use bare chisel, but I can take the chiselled instance that emerges from the process find out what shlibs it has pulled in for the app and make a new docker volume from that set of shlibs and hopefully not have to redeploy that volume very often.
Ahhh I understand - if you enable static linking (via swiftc or swift build) then all the runtime dependencies are part of the compiled binary. You get a fatter binary but it can be copied to where you need it.
(NB static executables don't really work, static linking of the standard library works well but you do need glibc and a few other dependencies depending on what you're linking. i.e. FoundationNetworking requires libcurl-dev to be installed in the runtime container etc)
ahhh.. I get it now. my objective is to make the actual docker container as small as it can be so that as I redeploy new versions to a large fleet I'm not running up my 4G bill. With a helloworld example that isn't a concern. when you drag in NIO and MQTT and SwiftyGPIO and a few other things you suddenly need more of the run time. I've been wrestling with how to have dynamic libs that I don't redeploy all the time for 4 years now.. :(. Chisel actually looks like it starts to address my problem, thanks for pointing this out!