# Strategies for Debugging Server-Side Swift in Production **URL:** \ **Category:** Using Swift\ **Tags:** server, debugging\ **Created:** [January 25, 2020, 1:22am UTC](https://forums.swift.org/t/strategies-for-debugging-server-side-swift-in-production/33031 "2020-01-25T01:22:29Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![rex-remind](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/rex-remind/32/449_2.png) [@rex-remind](https://forums.swift.org/u/rex-remind)\ **Post date:** [January 25, 2020, 1:22am UTC](https://forums.swift.org/t/strategies-for-debugging-server-side-swift-in-production/33031/1 "2020-01-25T01:22:29Z")
I'm curious what sorts of strategies people may be using to debug live server-side swift code in production or what sort of things are possible with a bit of work. E.g. Is there a way to: - performance trace live server-side code? - print out current stack traces of all running threads from the server? - capture crash logs for applications that crash? - log a stack trace on swift `Error`s? Any other strategies that come to mind would also be helpful. Thanks ---
**Author:** ![lukasa](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/lukasa/32/1232_2.png) [@lukasa](https://forums.swift.org/u/lukasa)\ **Post date:** [January 25, 2020, 8:00am UTC](https://forums.swift.org/t/strategies-for-debugging-server-side-swift-in-production/33031/2 "2020-01-25T08:00:12Z")
In production, server side Swift appears to the OS just like any other native binary. Therefore, the same debugging strategies apply. For performance tracing, `perf` is the main tool for the job, which you can use to generate flame graphs and other useful diagnostics. It can therefore also be used as a replacement for the macOS `sample` command for getting call stacks. For capturing crash logs, this is often a consideration for your runtime environment. Some process is responsible for starting and monitoring your service, whether that’s the Docker daemon or your init process (like `systemd`). That process should also be responsible for storing and delivering logs to some logging aggregation system. Exactly what your solution should look like here is hard to say because there are so many different choices, but this is a fairly standard log aggregation process. Your last point, logging a stack trace on Swift `Error`, is the most important one. This is not something you can do in general in Swift. `Error` does not capture a stack trace: it’s an extremely cheap error propagation mechanism, not much more complex at runtime than Go’s `(result, error)` return pattern. This is great in terms of performance but limiting in terms of metadata. For now there is no _general_ solution to the problem of capturing stack traces on `Error`. If this is an `Error` your code emits, you can always emit an appropriate error log that contains a stack, but if it is not you are limited in your ability to do much more at this time. ---
**Author:** ![rex-remind](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/rex-remind/32/449_2.png) [@rex-remind](https://forums.swift.org/u/rex-remind)\ **Post date:** [January 25, 2020, 7:41pm UTC](https://forums.swift.org/t/strategies-for-debugging-server-side-swift-in-production/33031/3 "2020-01-25T19:41:35Z")
Thank you very much for the thorough and quick explanation. This gives me a lot of information to start exploring. One follow up question though, you mentioned "you can always emit an appropriate error log that contains a stack". Is there a function in Swift that provides current stack information on a Linux / Docker env? Is it [Apple Developer Documentation](https://developer.apple.com/documentation/foundation/thread/1414836-callstacksymbols)? ---
**Author:** ![johannesweiss](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/johannesweiss/32/38183_2.png) [@johannesweiss](https://forums.swift.org/u/johannesweiss)\ **Post date:** [January 25, 2020, 10:03pm UTC](https://forums.swift.org/t/strategies-for-debugging-server-side-swift-in-production/33031/4 "2020-01-25T22:03:50Z")
What Cory says, and below a random collection of other stuff: - run production code in release mode: `swift build -c release` - until 5.2 lands, always pass a `-Xswiftc -g` to get debugging symbols, otherwise you won't have symbolicated stacktraces - if you have `--privileged`/`--security-opt seccomp=unconfined` containers or are running in VMs or even bare metal, you can run your binary with - if you don't have `--privileged` (or `--security-opt seccomp=unconfined`) containers (meaning you won't be able to use `lldb`) or you don't want to use `lldb`, I'd advise to use a library like [`swift-backtrace`](https://github.com/ianpartridge/swift-backtrace) to get stack traces on crash - for best performance in Swift 5.2 (once released), pass `-Xswiftc -cross-module-optimization` (this won't work in Swift versions below 5.2) - `perf` output, strack traces etc often appear "mangled" (lots of stuff looking like `3NIO14CircularBufferV5IndexV`), you can use `swift demangle` to turn that into readable things. So just do `cat file-containing-mangled-stuff | swift demangle` and it will look much more readable - make use of the sanitizers, so before running code in production, do the following: - make use of [logging](https://github.com/apple/swift-log) and [metrics](https://github.com/apple/swift-metrics) The good news is that creating a [deployment guide](https://swift.org/blog/sswg-update/#deployment-guides) is part of the [SSWG's focus areas for 2020](https://swift.org/blog/sswg-update/#future-focus-areas-for-2020). ---
**Author:** ![rex-remind](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/rex-remind/32/449_2.png) [@rex-remind](https://forums.swift.org/u/rex-remind)\ **Post date:** [January 26, 2020, 7:25am UTC](https://forums.swift.org/t/strategies-for-debugging-server-side-swift-in-production/33031/5 "2020-01-26T07:25:31Z")
Thank you so much for this cheat sheet!