RFC: Moving SwiftNIO SSL to BoringSSL

For the sake of posterity I should note that swift-nio-ssl doesn’t use BoringSSL on macOS at this time. It would be out of step with the desire to have a similar environment in dev and prod.

1 Like

Thanks. Just so I'm completely understanding the proposal :slight_smile:, although you will ship the entirety of BoringSSL you will only be using the bits of it needed to implement the public API of swift-nio-ssl, and BoringSSL itself will be inaccessible to users of swift-nio-ssl. In other words, people who depend on swift-nio-ssl in their Package.swift won't be able to import CNIOBoringSSL and call the mangled BoringSSL symbols. Correct?

Do you plan to remove CNIOSHA1 and use the implementation of SHA-1 from BoringSSL instead?

Correct.

No. CNIOSHA1 ships with the core NIO repository as it is required for websockets. We do not plan to make swift-nio-ssl a hard requirement of core NIO, nor do we intend to require that all NIO users download and compile BoringSSL when they are not using TLS.

2 Likes

As @lukasa has now opened a WIP PR for this I measured the build times I see:

swift-nio-ssl master:

real   0m20.852s
user   0m28.280s
sys    0m2.590s

BoringSSL branch:

real   0m39.140s
user   1m41.230s
sys    0m9.430s
1 Like

Ah right, I misremembered, it uses libressl, though, right? My point was more that it cannot use a library provided by the OS.

LibreSSL or OpenSSL, whatever it can find.

On Apple platforms that's absolutely correct.

They plan to back port OpenSSL 1.1.1 on 18.04 (there is already an SRU open), but the process is very slow. If we are lucky, maybe TLS 1.3 will be available on 18.04 before 2020.

And I think LibreSSL didn't even start to work on TLS 1.3.

I would really appreciate to don't have to bother with this kind of details when I have to deploy some apps, so I'm very exited about having BoringSSL embedded in Swift NIO.

1 Like

Speaking of a pure Swift TLS implementation. There is my SwiftTLS library, which is admittedly more a proof of concept, but nonetheless implements most of TLS 1.2 and 1.3. It's nowhere near ready for prime time of course, but could be a good starting point if anyone ever wants to pursue this.

2 Likes

Imho SSL isn't the biggest blocker not to use Swift for server stuff, so I like the idea of bundling a pure Swift solution that might not be suited for productive use, but more fun to play with (assuming it would still be possible to switch to an established lib easily).
Even if it's impossible to write a bulletproof implementation with Swift 5, I think it's good to start aiming for that goal now: Workarounds have a tendency to persist, and if Swift wants to be a general purpose language, having a constant reminder that there are topics where we can't compete with C yet might have a better effect than settling with an external lib.

This isn’t either-or. We can ship a production ready library today and be able to adopt an alternative tomorrow. swift-nio-ssl exposes approximately zero APIs that are tied to its underlying TLS implementation, and will continue to do so.

Again, I stress that we should not let the perfect be the enemy of the good. We can build something good now, and be useful now, while we buy time for alternative implementations to improve and mature.

2 Likes

don't ping me here ;-) The primarly obstacle here (in my opinion) is poor Swift runtime performance, not much how close the language is the hardware. History knows languages that are way further from the hardware than Swift is, and implements primitives (eg. on a daily basis in JavaScript world). Since Swift can call POSIX API and manually manage memory too, it is in theory possible to be as close to hardware as intended (excluding direct asm)... but... what swift advantage that would have?. Anyway, the performance obstacle is serious concern that apply to any pure Swift implementation, including eg. mentioned TLS. That said, it is possible to gain performance comparable to software implementations of OpenSSL for some primitives while working on raw memory.

While the discussion here has quieted down recently, I've had some personal conversation with participants in this thread that indicate that there are still people thinking over their positions on this issue. Given that there's no particular urgency here, I propose to give people the holiday period to mull over their positions on this issue and continue the discussion.

We'll try to come to a final decision in the first or second week of January 2019.

3 Likes

Overall I'm in favour of this.

On the one hand, this is a problem that every program that links against system libraries has to tackle, and we do it all the time using if #available for libraries like Foundation. In that sense it's not entirely unreasonable to tell those system administrators that they have to update the OS to patch security holes or take advantage of newer features like HTTP2/3.

On the other hand, it sounds like libssl has been a bit sloppy about API and ABI compatibility. In the absence of a stable alternative library, it makes sense to bundle our own implementation.

I would love to see us expose a public, Swift interface to the SSL/crypto functions one day, but that's a whole other thing.

I'm a bit concerned about how narrowly BoringSSL might be focused on their own needs and how closed it is to external needs/contributions. We can only try it and see - we'd still have the option to change the implementation we use if problems arise.

Also - Apple has open-sourced recent versions of many crypto frameworks (CommonCrypto, Security.framework, coretls). I'm not exactly sure how they all fit together, but is there not a complete/nearly-complete implementation there which we could use? I guess that interface is stable enough that we could create a nice Swift wrapper for it.

3 Likes

And system administrators would be more than happy to update there system, but they don't choose what packages are released on what system version. Today, you may ask Ubuntu TLS administrators to update to OpenSSL 1.1.1 to use TLS 1.3, but that version is not released yet on Bionic, and system administrator can't do much about it.

Hey folks, as the new year has begun I'd like to resurrect this thread and call for people's final opinions as to this proposal. I'll give this a week or so for further discussion and then will sum up at the end and work out the next steps.

4 Likes

+💯

1 Like

It sounds like moving to BoringSSL will help alleviate common build issues and ensure NIO can be an early adopter of new protocols and standards such as HTTP/3. I'm in full support of this. Thanks, Cory! :+1:

3 Likes

Thanks @lukasa for guiding the discussion.

Overall, we (Kitura) are OK with this proposal, but we don't love it. A few specific points...

Having a consistent implementation of TLS is a big win, and we definitely want the NIO team to be able to move forward with innovations like QUIC and HTTP/3.

However, shipping the entirety of BoringSSL and forcing everyone to compile the whole thing, when big chunks of it aren't even going to be used by SwiftNIO, is quite suboptimal. By "everyone" we're not talking just about frameworks like Kitura that consume SwiftNIO, but all the users of those frameworks too. It cascades down to everyone. I'd really like it if some way could be found to reduce the cost of that (e.g. through pre-processing the BoringSSL source code to remove unnecessary code). NIO already has some scripting around the NodeJS HTTP parser and SHA1 C implementations it uses - something similar could be done for BoringSSL? The diff here is currently "+403,592 −774" - anything we can do to mitigate that will benefit SwiftNIO users.

2 Likes

Thanks for raising this Ian.

It's a thing I have thought about, but I have not got high hopes for this kind of approach. The most granular level of division within BoringSSL is the difference between libcrypto and libssl. While the APIs we use are primarily in libssl, libssl has a hard dependency on libcrypto, so we cannot trivially drop that library.

If we try to make things finer-grained, we are ultimately playing with fire. It will be very hard to script things away, as the BoringSSL code moves around and is rewritten without our input, and we are unlikely to be able to simply excise entire files: only code within those files. Even there we don't have a lot of options for things to remove, as TLS exercises a substantial amount of the crypto code. Such a change would essentially have to be hand-rolled for every new update to BoringSSL, which greatly jeopardises our ability to stay close to the tip of BoringSSL development.

I should note that the diff makes the situation seem worse than it is. Many thousands of those lines of code are simply eliminated by the C preprocessor and not compiled at all, as they are platform-specific. A substantial amount of the LoC left are not compiled but assembled, which also reduces the overhead.

1 Like