Idea: runtime/slim images

Assuming you're going to be looking at a core dump offline, why is it necessary to include lldb as part of the docker image at all?

5 Likes

Ah, thanks for the save & great question. I misunderstood what was being provided. I’d run most containers as slim, and give best-effort to keeping a core dump for offline review later (using whatever container orchestration system/tools to do so).

You’re correct in I’d like to have an lldb-friendly container occasionally, but since I’d have core dumps from a slim container I would not care to bring along debugging tools for normal, prod containers.

it's online. In server environments you typically want to have your crash reports at least in Splunk (I say 'at least' because you might have a better story around crash reports but baseline would be Splunk/ELK/...) etc, and that's only useful if symbolicated. We run ./binary | symbolicate-swift-fatal ./binary so the crash output is already symbolicated when it hits stdout/err which gets send to splunk anyway.

Of course, we could build infrastructure to have access to the un-symbolicated stack trace, all binaries and also a way to identify which crash came from what deployment. Then we could have some software that then takes all this information and symbolicates, but that's just a lot of work for something that's much easier done straight away.

Arguably, this all shouldn't be necessary and all the stack traces should come symbolicated straight away.

3 Likes

this sounds great, but doesn‘t this already exist with the images of ibm: https://github.com/IBM-Swift/swift-ubuntu-docker
(they have a builder and runtime image)
Or what would be the difference?

Thanks everyone for your feedback. It's really great to hear people's interest in this.

Doing this requires Python to be installed as well, which will make the image even larger. Even the full swift:5.0 image doesn't include Python - if we're making Python a requirement maybe swift:5.0 should start FROM python... (joke)

This. Does anyone know what prevents that at the moment? I'm afraid I have no idea.

I think we should go further, and include it in the Linux release .tar.gz, as its so useful. Then it will be in the main swift:5.0 Docker image. It's not there today. What do people think?

I don't know whether Docker would accept that - the other languages I've seen only have two flavours: normal and slim. But we could ask, if having a third image is the solution we prefer. Personally I'd like to try hard to avoid it as it complicates things for users and image maintainers.

You're right, it isn't for offline analysis as you can docker commit the container containing the core file and then load the core into another image that contains LLDB for analysis.

IBM has maintained our own build and run images for several years. The images have some historical quirks though and we're keen to improve the upstream images so the whole community benefits.

1 Like

This is pretty standard fare in the Docker container world. You typically can post as many images and tags as are needed. Though the tag list should be kept as small as possible (to reduce complexity). The tags and their proper usage should be clearly and concisely communicated in the README.

I for one will benefit from a slim image. The value of including the debugger in the slim image isn’t worth the increased image size for something that is an irregular activity (for me) - especially as there are other options as noted above.

who is the right person that can answer if this is possible? it will drastically improve debuggability imo

2 Likes

I've opened https://bugs.swift.org/browse/SR-10439 to track shipping symbolicate-linux-fatal as part of the releases.

1 Like

This is true for normal accounts on Docker Hub, but the Swift images are part of the Docker Official Images program, which is curated by Docker to ensure best practices etc. So we'd need to work with Docker to make this happen, if we want it.

Yes, and it doesn't appear to be an issue for official images either. Just look at the Python image(s). It's a sizable matrix of:

  • python:<version>
  • python:<version>-slim
  • python:<version>-alpine
  • python:<version>-windowsservercore

:sunglasses:

2 Likes

Yes, I wasn't saying it was impossible, just that it's not entirely within our gift.

I would really like to see

  • swift:5.0-slim (base on Ubuntu)
  • swift:5.0-alpine
  • swift:5.0-alpine-slim

:grin:

3 Likes

Wow I'm coming late to this thread. @IanPartridge this is fantastic! I've been wanting this and I know others have been asking me this for a while now! :heart:

1 Like

I've got a PR up with prototype slim Dockerfiles for Swift 5.0.1. Please take a look and try them out!

8 Likes

The slim images are working for us (Kitura) in our testing, so I've merged them to master. They are not available on Docker Hub yet as I suggest we make them generally available alongside the next Swift release.

Please do try them out and post your experience. I'm particularly interested to check they work for Vapor.

2 Likes

Super awesome @IanPartridge! I'll give them a spin for sure. Out of curiosity, do you already have size comparison between the normal & slim images? :smile:

Full bionic (18.04): 1.35GB
Slim bionic (18.04): 226MB :tada:

Full xenial (16.04): 1.37GB
Slim xenial (16.04): 249MB :tada:

18 Likes

Woah! I expected improvements, but that's just mindblowing! Very nice work on this!

1 Like

Very nice job, but I think it can be smaller. I tried to compile Swift on Alpine, but got many errors and stop doing this, but I think Alpine would be the smallest base image for "cloud-native" Swift. :smirk:

We implemented something similar to this: https://medium.com/@jjacobson/a-minimal-swift-docker-image-b93d2bc1ce3c, in which we basically start from a slimed down version of Ubuntu (abt 50MB) and add just the libraries the compiled Swift program links to. We are able to get images in the range of 150MB including our own code (NIO based server stuff).

Earlier, we had issues with Alpine Linux not being able to resolve hostname to ip from inside the container (see https://github.com/gliderlabs/docker-alpine/issues/8 which was not really resolved as of late last year). We haven't tried Alpine since.

cc: @ilyaK

1 Like