The SwiftNIO team would like to ask the wider community to discuss a proposal to fundamentally change the TLS implementation used by swift-nio-ssl. This change would be scheduled to land in the next major release, which will also drop Swift 4 support (see Johannes’ proposal for more details).
The outline of the proposed change, as well as the reasoning and outcomes, is below. Please take a read and let us know your thoughts in the comments.
We propose to change swift-nio-ssl to stop linking against the system copy of libssl, and instead to provide a vendored copy of BoringSSL. This change would come with a number of subtle runtime behavioural changes, as well as a number of substantially more disruptive changes around application distribution and OS behaviour.
The proposed change would drop support for OpenSSL and LibreSSL. The reasoning for this choice is discussed later in the proposal, and is not necessarily mandatory.
Note that for Apple platforms the recommended TLS solution will still be to use swift-nio-transport-services.
One of the major difficulties encountered by server side software developers is that of distributing your built application. Most programs end up developing an implicit “build environment” that requires collaboration with the target deployment operating system to successfully configure.
This integration with the operating system presents developers with a burden, both during development, and during deployment. In development, developers need to set up what amounts to a clone of their deployed environment. This is made particularly difficult in Swift on Server because the most popular development environment is macOS, but the most popular deployment environment is Linux. Most developers are not able to perfectly homogenise their development environment with their build and deployment environment, which opens up the door to problems that only manifest on build or deployment.
One of the major difficulties here has come from libssl. Swift’s supported platforms cover multiple versions of the various Apple platforms, and 3 versions of Ubuntu. Each Ubuntu version ships with a different libssl API by default (OpenSSL 1.0.1 for Ubuntu 14.04, OpenSSL 1.0.2 for Ubuntu 16.04, and OpenSSL 1.1.0 for Ubuntu 18.04).
Apple platforms provide no libssl at all, requiring developers who wish to replicate their Linux build environment to go and obtain a copy of libssl. Frustratingly it is very hard for developers to obtain the exact libssl version that their deployment target will use. For example, Homebrew currently makes available (correct at the time of writing) OpenSSL 1.0.2p, OpenSSL 1.1.1, and LibreSSL 2.8.2, none of which corresponds to the stable libssl available on the target Linux platforms.
Unfortunately, the absence of libssl on Apple platforms requires developers that want to ship applications that build on both Linux and Apple platforms to either require that their users read an installation guide to obtain their dependencies, or to conditionally depend on both swift-nio-ssl and swift-nio-transport-services. This conditional dependency imposes particular requirements on the way projects use Swift Package Manager, and it locks out users from being able to use libssl-based TLS on Apple platforms even when they may want to.
The story for deployment is also tricky when it comes to libssl. This is because the produced application binary depends on having a copy of libssl available to it that is ABI-compatible with the one that it was built against. Unlike glibc, which aggressively remains ABI compatible, there are multiple incompatible libssl ABIs. The three major ones that affect Linux developers are LibreSSL’s, OpenSSL 1.0's, and OpenSSL 1.1's.
The result of this ABI compatibility concern is that a built binary can only run on a subset of Linux platforms: specifically, those that have a libssl that shares the same ABI as the one on the build machine.
Other popular languages in the server-side programming space, such as Go, have a compelling deployment story that simply involves copying a binary over to a new machine and running it. While this story is difficult for Swift to achieve exactly (due to our reliance on glibc), it is possible for Swift to get very close. However, any Swift application that links libssl is immediately ineligible for this kind of deployment model, as it will be indefinitely tied to this ABI constraint. Applications built on Ubuntu 14.04 against OpenSSL will run on Ubuntu 16.04, but not Ubuntu 18.04. This is an unfortunate, and unnecessary, limitation.
While we are actively using libraries like libssl widely in the Swift Server space, we will never have anywhere near as compelling a deployment story as languages like Go.
When it’s necessary to link against platform libraries, cross-compilation gets increasingly difficult. This is because your cross-compilation toolchain needs to have copies of all of the system library header files and binaries for the libraries against which you intend to link. This incurs all of the difficulty mentioned above about obtaining the specific matching copy of your target libssl, with the added difficulty of requiring that you ensure it is built for the correct platform.
An additional limitation of the widespread use of the system libssl is that several popular systems have relatively old copies of OpenSSL that lack important features. For example, none of the OpenSSL versions in the supported Linux distributions support TLS 1.3, and due to the maintenance process of the supported distributions, they never will. Additionally, the OpenSSL version in Ubuntu 14.04 does not support ALPN, meaning that applications on Ubuntu 14.04 will not be able to use HTTP/2. Looking forward to new protocols, as QUIC (the transport protocol for the forthcoming HTTP/3) uses the TLS 1.3 handshake protocol, the absence of TLS 1.3 support in any of the supported Linux distributions mean that applications using the system libssl on those platforms will never support QUIC.
While users can avoid this by bringing their own, non-system copy of libssl, this presents its own challenge. Because the entire application is required to link one and only one copy of libssl, the entire ecosystem must be capable of supporting that version. This limitation has already affected Swift Server applications on Ubuntu 18.04, as many different libraries have had to make source code changes for OpenSSL 1.1, and not all server libraries and frameworks have completed this change. Until all of those libraries have adopted the new API, using new OpenSSLs is not possible. Naturally, there is minimal incentive for much of the ecosystem to adopt new APIs that are not supported by any of the system libssl versions, which makes conditionally using newer copies very difficult.
As a final note, this new copy of libssl would become part of the deployment artifact for the application, further complicating the deployment story.
Some other languages, such as Go, attempt to avoid this difficulty by using self-contained TLS implementations that do not depend on the operating system.
The engineering cost of writing a pure-Swift crypto implementation is high, however. Rather than boil the ocean by saying that we should do this, it would be better to begin by hiding the underlying C implementation from users. This would open the door to replacing the core implementation with one written in Swift, rather than in C/C++, without further disruption. As Swift Package Manager is capable of building C and C++ code, this should be reasonably straightforward.
Note that, for performance reasons, almost all cryptography libraries require the use of accelerated helpers written in assembly language. This is true for BoringSSL as well. This proposal is therefore gated behind adding support for building assembly code to Swift Package Manager. Code for this has already landed in SwiftPM master, and assuming nothing surprising occurs should ship with Swift 5.
Currently swift-nio-ssl depends on swift-system-openssl, which causes Swift Package Manager to emit linker flags to whatever version of libssl can be found via pkg-config. This implicitly causes us to depend on the system libssl. We propose to replace this dependency with a new one, CNIOBoringSSL, which will contain a vendored copy of the source code of BoringSSL.
It has only recently become possible to make this change. As we need to perform symbol mangling to achieve this safely (see below), we have needed support from the BoringSSL project for this symbol mangling. This support was only added to BoringSSL on the 26th of August this year (in commit 8c7c635).
This new dependency will not be its own package: instead, it will be a target within swift-nio-ssl. The use of BoringSSL is intended as an implementation detail for swift-nio-ssl, and will not affect the rest of the package ecosystem.
This is because BoringSSL does not provide any API or ABI stability guarantees. They are free to, and indeed do, break APIs regularly. If BoringSSL were exposed as as Swift Package Manager package, it would bump its major version number every release. This would make it impossible for the wider community to depend on: different packages would be pinning to different major versions, and dependency resolution would immediately fail.
swift-nio-ssl will present an API that does not contain any part of BoringSSL. If users need to use libssl or libcrypto themselves, they will need to bring their own copy of libssl.
A concern of this approach is symbol clash. We will mangle the BoringSSL symbols to ensure they do not clash with any other package in the Swift ecosystem.
This concern is not theoretical: grpc-swift has already had to move away from BoringSSL once before due to symbol clash with the system libssl.
The problem here is that a Linux binary will only ever have one implementation of a given symbol name. If the binary is both dynamically linked against libssl, and uses a vendored copy of BoringSSL which has symbols that clash with the names in libssl, the portion of the binary that wanted the dynamically linked copy of libssl will instead get the BoringSSL implementation. This usually leads to memory corruption and eventual crashes, and is to be avoided.
The way we will avoid this is by mangling the symbol names of the BoringSSL symbols. The script that updates the vendored copy of BoringSSL will run a series of build steps that have the effect of prefixing all BoringSSL symbols vendored into swift-nio-ssl with a specific string unique to swift-nio-ssl. This will ensure that swift-nio-ssl does not bring symbols that clash with other copies of libssl.
As an example of this mangling, consider the libssl symbol
SSL_new. In swift-nio-ssl, this symbol will instead be
CNIOBoringSSL_SSL_new. As the string “CNIOBoringSSL” is entirely within the implicit NIO package namespace, there is no risk of clash with other libssl providers.
Other libraries in the ecosystem that may choose to vendor BoringSSL should do the same: this will ensure that multiple copies may inhabit the same program without conflict.
BoringSSL updates regularly, frequently dropping older, less-secure cipher suites and protocol versions. We will conceal these changes when they are non-functional, and follow semantic versioning when they are, thus hiding this rate of change from our users. We will provide all updates via Swift Package Manager.
One immediate effect would be that swift-nio-ssl will drop support for SSLv2 and SSLv3. The loss of these two protocol versions is highly unlikely to cause anyone any loss of sleep. Newer versions of OpenSSL have completely removed SSLv2, and SSLv3 is disabled across the vast majority of the web.
In the future, swift-nio-ssl will bring both new features and feature removals to all platforms simultaneously, rather than gating them on the specific operating system and OS version being used. This is a common deployment model: Chrome uses BoringSSL for this exact use-case, and many other tools and languages (such as Firefox and Go) ship their own TLS implementations they completely control.
Security updates will no longer be distributed by the OS package manager: instead, swift-nio-ssl will be carrying all TLS security updates, and users will need to update swift-nio-ssl to get them.
This is not a big difference from today. SwiftNIO and its ecosystem are a part of your application and can have security vulnerabilities of their own, and are not distributed by the operating system. As a result, it is already necessary to remain aware of the security content of Swift Package Manager package updates.
The SwiftNIO team has substantial experience managing security updates, both for SwiftNIO itself and for other projects. We have a system in place for deploying security updates, and will continue to use it for swift-nio-ssl when we’re distributing BoringSSL.
The SwiftNIO team will enhance swift-nio-ssl to ensure that it continues to discover the system-provided certificate store, where this is available.
One of the downsides of using BoringSSL is that it is no longer aware of the default certificate store on most Linux distributions. Fortunately, this procedure can be dealt with manually by searching for the CA bundle at startup. As an example, curl contains a list of possible locations that covers most Linux distributions (https://github.com/curl/curl/blob/f859b05c6686b2c5a41fd7805f164229b3c6d7c8/CMakeLists.txt#L684-L689).
Swift-nio-ssl will include this logic by default, so this should not result in a noticeable regression for any users.
One concern about using BoringSSL is that it has no notion of a stable version number. This absence of a stable version number can make it difficult to be confident about exactly what version of BoringSSL is in use when attempting to investigate issues with your application, or when investigating if you need an update to obtain a BoringSSL bug fix or security update.
To combat this, swift-nio-ssl will include a method that can be called to obtain the SHA of the git commit used to obtain the BoringSSL sources. This commit will be from the upstream repository, at https://boringssl.googlesource.com. This SHA will also be present in a code comment in the swift-nio-ssl Package.swift file.
The immediate result of this change will be that swift-nio-ssl gets easier to build for all users on all platforms. A large number of subtle compiler errors will no longer be encountered by users, particularly those new to the ecosystem who are attempting to start developing on macOS.
Additionally, it will become that little bit easier to build static binaries for Swift Server applications, and so to distribute Swift Server applications more easily. In particular, they will no longer be tied to the presence or absence of a libssl on the platform: they’ll bring their own along with them when they need it.
The gap between development and production environments will get smaller, as both environments will be running the same code using the same libraries.
Finally, we will improve the security and functionality of the entire Swift Server ecosystem, regardless of the operating system they are targeting. This will help keep more of our users more secure, as well as more up-to-date with the modern web.