Also, I would like to report a surprising difference in latency between Intel Mac and M1 (Max) Mac during the localhost development. This is especially noticeable when triggering routes that access a database (SQLite in my case).
Release build on Intel Mac responds in 7ms (according to the browser developer tools and Postman) but the same route on the M1 Mac responds in about 20ms.
Any ideas why? Or how to debug the cause of performance difference? I have not yet tried changing the async executor.
Lastly, simpler pages that do not use the database, report similar latencies, but also just a tiny bit shorter on Intel Mac — normally 1-2ms better.
If you are touching a database, one possible reason is the difference in page size if you are reading small data records randomly spread out. The intel machine has a 4k page size (as is way most common historically), while the m1 has 16k page size. For certain tests this can have a significant impact, just one difference to consider and analyze for.
I am sorry for the off-topic, but if I am to copy the entrypoint.swift, I am getting the runtime error. Am I missing something?
Precondition failed: BUG DETECTED: wait() must not be called when on an EventLoop.
Calling wait() on any EventLoop can lead to
- deadlocks
- stalling processing of other connections (Channels) that are handled on the EventLoop that wait was called on
Further information:
- current eventLoop: Optional(SelectableEventLoop { selector = Selector { descriptor = 7 }, thread = NIOThread(name = NIO-SGLTN-0-#4), scheduledTasks = PriorityQueue(count: 0): [] })
- event loop associated to future: SelectableEventLoop { selector = Selector { descriptor = 9 }, thread = NIOThread(name = NIO-SGLTN-0-#6) }
And here is my entrypoint.swift:
import Logging
import Vapor
import NIOCore
import NIOPosix
@main
enum Entrypoint {
static func main() async throws {
var env = try Environment.detect()
try LoggingSystem.bootstrap(from: &env)
let app = try await Application.make(env)
// This attempts to install NIO as the Swift Concurrency global executor.
// You should not call any async functions before this point.
let executorTakeoverSuccess = NIOSingletons.unsafeTryInstallSingletonPosixEventLoopGroupAsConcurrencyGlobalExecutor()
app.logger.debug("Running with \(executorTakeoverSuccess ? "SwiftNIO" : "standard") Swift Concurrency default executor")
do {
try await configure(app)
} catch {
app.logger.report(error: error)
try? await app.asyncShutdown()
throw error
}
try await app.execute()
try await app.asyncShutdown()
}
}
Have you migrated the shutdown to async as well? That's probably the cause. Annoyingly wrapping it in defer hides the warning about being unavailable from async contexts - yet another reason why transitive noasync warnings would be helpful
I am sorry for spamming it here, but 4.100.0 release killed my server. It now crashes on boot with: Redis/RedisStorage.swift:30: Fatal error: Modifying connection pools after application has booted is not supported.
I tried the async-lifecycle branch, but it did not help. However, I think I narrowed it slightly down. It seems to crash in this part of the asyncBoot():
for handler in self.lifecycle.handlers {
try await handler.didBootAsync(self)
}
I run asyncBoot function from the configure.swift once to boot redis (and the database too). It does some operations before starting listening for new requests. These operations succeed.
Then it calls try await app.execute() and crashes when calling the asyncBoot() again.
Keep in mind it is by its very nature a constructive critique, and written with the benefit of hindsight. No ill-will is meant towards anyone (not that I really expect anyone to read that in it, but I did point out quite a few mistakes we all made, none of which are meant to be taken personally). And it's inherently somewhat subjective; my subjective opinion.
I also apologise in advance for any mistakes or important things overlooked. I've been noodling away at it for what feels like forever now, and at this point I'm so done with editing and proof-reading it.
One small comment: you mistakenly wrote my name as Alex at one point, and it is in fact: Axel. You only did this once. Remappings to Alex happen regularly, that's the way the human brain is wired, but could you please fix it? Thanks!
Another thing: could you reference the first graphs that I created them, like you did with the latter?
Otherwise: amazing recap of all the 156 posts up here!
Lots of lessons learned during these discussions, some were way over my head.
In the end you write:
“Also, Axel has not yet done a follow-up with the final fixes & workarounds, to confirm that they do fully fix the benchmark’s results. ”
Do you mean do compare the results again with the other languages, to have ‘The final proof that swift reigns supreme’?
Actually I did it many times, but I thought I'd caught all of them in proof-reading - obviously not. Sorry about that! Axel is a cool name, but yeah as you guessed I've not met many Axel's in comparison to Alex's.
Ah yes, sorry about that - somewhere in the editing and rewriting those links got lost. Probably my fault (although possibly WordPress's - there's a few known, long-standing bugs where it loses or corrupts content as a result of unrelated edits ).
I mean with the fully fixed wrk and relevant socket kernel limits raised sufficiently. That should eliminate the weird failure rate behaviour you saw with Swift, making it behave like the other three (100% successes until max throughput is reached, then a fairly linear drop down to 0% as the overload is magnified).
In my local testing everything seems fixed, but your test setup is the more authoritative (and what you used in your posts, so it'd be good to have fully-corrected results).
As to whether "Swift reigns supreme", well, maybe. I don't think comparative benchmarks like this are all that conclusive as competitive measures. At least until the other three frameworks receive similar scrutiny. Maybe the BigInt implementation you used in PHP also isn't very efficient, for example.
Tangentially, if anyone does examine the Swift implementation further (e.g. analyse Vapor/SwiftNIO hot paths in detail, or the compiler's performance re. inlining etc) I for one would be very interested in reading about how that goes (irrespective of whether it results in major performance improvements).
To my knowledge there's not a single guide or postmortem or similar out there [yet] which shows how to really scrutinise and optimise the performance of a Vapor web server. This isn't unique to Vapor - there's precious little written about how to really optimise Mac and/or Swift software.
).