Vapor performance

Hello, I really like Swift and from time to time I go back to the server community to see how much have evolved and if I can think of using it for work or some side project. I am already using Swift for some CLI application so would be nice to use it for some web apps.

I am also in the process of learning Go, so out of curiosity I wanted to see how they compare in terms of speed. My day to day language is PHP so I added it to the test to have a baseline.

I generated a simple SQLite database with one table and ~10k rows a copied it on every project without modifications.

PHP
Brand new Laravel application with one single route, I am using Laravel Octane, with the Swoole extension. The code:

Route::get('/', function () {
    return Product::all()->toJson();
});

Swift
Brand new Vapor application, Swift 5.9, Fluent and Fluent SQLite Driver, one single route where I fetch everything e return it as json (I added Content to my Model).

import Vapor

func routes(_ app: Application) throws {
  app.get { req async throws in
    return try await Product.query(on: app.db).all()
  }
}

Go
One main.go file, server built with Fiber and for the database I used GORM. This is the whole code:

package main

import (
	"github.com/gofiber/fiber/v2"
	"gorm.io/driver/sqlite"
	"gorm.io/gorm"
)

type Product struct {
	ID    uint   `gorm:"primarykey" json:"id"`
	Code  string `json:"code"`
	Price int    `json:"price"`
}

func main() {
	app := fiber.New()

	db, err := gorm.Open(sqlite.Open("database.sqlite"), &gorm.Config{})

	if err != nil {
		panic("failed to connect database")
	}

	app.Get("/", func(c *fiber.Ctx) error {
		products := &[]Product{}

		db.Find(&products)

		return c.JSON(products)
	})

	app.Listen(":3000")
}

These are the results, in this order from left to right: Swift, Go and PHP

Swift seems quite slow, does to look right to you? Maybe I am missing something.

P.S. This was done on my Macbook Pro M2 Pro

5 Likes

Did you build in release mode? (-c release)

Yes, I built with swift build -c release and then I run .build/release/app

1 Like

These numbers definitely make me a bit suspicious. Are you running on macOS or Linux? Can you use either Instruments or Perf to produce a CPU profile and share it? I'd love to see where the time is going.

Going to loop @graskind in for extra explanation if needed but you've hit a couple of awkward points @gio

The first is that Vapor 4 was designed and created in a world where Swift Concurrency wasn't a thing and everything was built on top of event loops. That allowed us to make a number of assumptions around threads and thread safety that no longer hold true. To combat this (before we release the next major version) we've had to add a significant number of locks and boxes to ensure safety that will have a noticeable effect on performance due to the need to check locks and extra allocations etc. This won't be a significant hit though. (There's also another discussion around hopping between async contexts and jumping between concurrency and event loops).

The main issue you'll be hitting is Fluent relies on reflection and mirrors to do what it needs to do as it was the best tool at the time. Swift's reflection performance is...not great. I'd wager you'll see a massive bump if you drop Fluent models and use SQLKit instead. Fluent 5 will not have this issue.

11 Likes

Additionally, if you're returning all 10,000 rows in the response, you'll be paying for Swift's terrible JSON encoding performance too, which is orders of magnitude slower than Go.

2 Likes

Are there any resources where one can learn more about Vapor 5 and Fluent 5?

1 Like

Thanks for the explanations, very interesting. Indeed switching to SQLKit the results changed from ~8 req/sec to ~30 req/sec, which is not bad. Still far from Go but way better, it might be the json encoding as @Jon_Shier said.

2 Likes

What OS are you running on? The JSONEncoder on Linux is significantly faster than on macOS

There's a Vapor 5 and Fluent 5 channel on Vapor's Discord

I am on mac os, I tried using docker but the results didn't change

1 Like

I'm curious, when running your benchmark on Linux, is there any difference if you use JSONEncoder from the new swift-foundation?

2 Likes

Also, as a rule of thumb, it is generally not recommended to run a load generator on the same physical machine as the server.

You could try swift-foundation or ikigajson

5 Likes