Connection pooling is a critical component for many server-side applications. I would attempt to summarize connection pooling here, but honestly Wikipedia's page does a better job:
In software engineering, a connection pool is a cache of database connections maintained so that the connections can be reused when future requests to the database are required. Connection pools are used to enhance the performance of executing commands on a database.
There are several Swift connection pool implementations currently floating around:
EventLoop{Group}ConnectionPool
from Vapor's AsyncKit (contributed by myself, @MrLotU, @graskind, and @calebkleveter).
Vapor currently uses these pools for PostgreSQL, MySQL, SQLite, and APNS
ConnectionPool
from @Mordil's RediStack (contributed by @lukasa).ConnectionPool
from SSWG's AsyncHTTPClient (contributed by @adtrevor, @weissi, @artemredkin, @adam-fowler, and @dimitribouniol)
Given the variety of implementations currently available, it seems like a good time to compare and contrast approaches and see whether generic connection pool could help us combine efforts.
It's important to note that such a generic connection pool probably can't be a silver bullet. It's likely that AsyncHTTPClient for example will need to continue to ship its own implementation due to the complex requirements of the HTTP spec. However, as @lukasa said at the last SSWG meeting, we should be able to make progress on "homogenous connection" pooling. In other words, pools where all the connections can be treated equally.
Since I'm most familiar with Vapor's connection pools, I'm going to share an explanation of how they work, and what I believe are the pros / cons of the methods we chose. I invite anyone else who has experience with connection pooling to share their thoughts here. If I left something out from the list above, please let me know and I'll add it!
Vapor's AsyncKit package defines two connection pools:
EventLoopConnectionPool
: A pool of connections tied to a singleEventLoop
.EventLoopGroupConnectionPool
: A collection ofEventLoopConnectionPool
s tied to anEventLoopGroup
with one pool perEventLoop
.
These pools rely on two protocols:
ConnectionPoolItem
: What the connection pools hold.ConnectionPoolSource
: Responsible for creating new connection pool items.
The ConnectionPoolItem
protocol is very simple, but enforces a core assumption of Vapor's connection pools: Each connection belongs to an EventLoop
.
/// Item managed by a connection pool.
public protocol ConnectionPoolItem: class {
/// EventLoop this connection belongs to.
var eventLoop: EventLoop { get }
/// If `true`, this connection has closed.
var isClosed: Bool { get }
/// Closes this connection.
func close() -> EventLoopFuture<Void>
}
The isClosed
property is used by the connection pool to prune closed connections. The close()
method is used when the pool is shutting down.
The ConnectionPoolSource
is also quite simple and again assumes connections are created for a single event loop.
/// Source of new connections for `ConnectionPool`.
public protocol ConnectionPoolSource {
/// Associated `ConnectionPoolItem` that will be returned by `makeConnection()`.
associatedtype Connection: ConnectionPoolItem
/// Creates a new connection.
func makeConnection(logger: Logger, on eventLoop: EventLoop) -> EventLoopFuture<Connection>
}
Note that the Logger
the connection should use is passed here. Going forward, this method may need to accept additional context like BaggageContext
for tracing. This should be considered in advance to preempt protocol breakage.
A simplified version of EventLoopConnectionPool
's interface is supplied below. Some important things to note:
- If a connection is not available when requested, a new connection will be created until
maxConnections
is reached. - Closed connections are pruned, but the pool never closes connections itself unless shutting down.
- There is no "min connections" requirement.
- The
requestTimeout
prevents deadlocking if more connections are required at once than the pool can possibly yield. In other words, if you are waiting on three connections from the pool at once, and it can hold at most two, you would be waiting forever without this timeout. withConnection
simply callsrequestConnection
andreleaseConnection
around the supplied closure. There is no special logic here.- You can call this pool from any thread. It will return to the associated
EventLoop
before doing work. - This pool never locks and is meant to be used on its associated
EventLoop
. - Additional context passing besides just
Logger
should be considered.
final class EventLoopConnectionPool<Source>
where Source: ConnectionPoolSource
{
let source: Source
let maxConnections: Int
let requestTimeout: TimeAmount
let logger: Logger
let eventLoop: EventLoop
init(
source: Source,
maxConnections: Int,
requestTimeout: TimeAmount = .seconds(10),
logger: Logger = .init(label: "codes.vapor.pool"),
on eventLoop: EventLoop
)
func withConnection<Result>(
logger: Logger? = nil,
_ closure: @escaping (Source.Connection) -> EventLoopFuture<Result>
) -> EventLoopFuture<Result>
func requestConnection(
logger: Logger? = nil
) -> EventLoopFuture<Source.Connection>
func releaseConnection(
_ connection: Source.Connection,
logger: Logger? = nil
)
func close() -> EventLoopFuture<Void>
}
A simplified version of EventLoopGroupConnectionPool
is supplied below. Some important things to note:
- This pool is just a collection of
EventLoopConnectionPool
and does not implement any actual pooling logic. maxConnectionsPerEventLoop
setsmaxConnections
on eachEventLoopConnectionPool
. This means that you must have at least one connection event loop.pool(for:)
is the most important method this pool offers. The{with,request,release}Connection
methods all call into this method.- This pool locks during shutdown and to check that it is not already shutdown when other methods are called.
- This pool is meant to be used on the main thread. If you are on an
EventLoop
, you should ask for your respectiveEventLoopConnectionPool
. - Additional context passing besides just
Logger
should be considered.
final class EventLoopGroupConnectionPool<Source>
where Source: ConnectionPoolSource
{
let source: Source
let maxConnectionsPerEventLoop: Int
let eventLoopGroup: EventLoopGroup
let logger: Logger
public init(
source: Source,
maxConnectionsPerEventLoop: Int = 1,
requestTimeout: TimeAmount = .seconds(10),
logger: Logger = .init(label: "codes.vapor.pool"),
on eventLoopGroup: EventLoopGroup
)
public func withConnection<Result>(
logger: Logger? = nil,
on eventLoop: EventLoop? = nil,
_ closure: @escaping (Source.Connection) -> EventLoopFuture<Result>
) -> EventLoopFuture<Result>
public func requestConnection(
logger: Logger? = nil,
on eventLoop: EventLoop? = nil
) -> EventLoopFuture<Source.Connection>
public func releaseConnection(
_ connection: Source.Connection,
logger: Logger? = nil
)
func pool(for eventLoop: EventLoop) -> EventLoopConnectionPool<Source>
func syncShutdownGracefully() throws
func shutdownGracefully(_ callback: @escaping (Error?) -> Void)
}
Vapor integrates these pools into the framework and you normally don't interact with them directly. An example of using these pools without Vapor can be seen in PostgresKit's README.
import PostgresKit
let eventLoopGroup: EventLoopGroup = ...
defer { try! eventLoopGroup.syncShutdown() }
let configuration = PostgresConfiguration(
hostname: "localhost",
username: "vapor_username",
password: "vapor_password",
database: "vapor_database"
)
let pools = EventLoopGroupConnectionPool(
source: PostgresConnectionSource(configuration: configuration),
on: eventLoopGroup
)
defer { pools.shutdown() }
pools.withConnection { conn
print(conn) // PostgresConnection on randomly chosen event loop
}
let eventLoop: EventLoop = ...
let pool = pools.pool(for: eventLoop)
pool.withConnection { conn
print(conn) // PostgresConnection on eventLoop
}
Pros:
- Very concise implementation (~600 loc with lots of comments).
- Connections must be used on the same
EventLoop
meaning there is no hopping. CircularBuffer
is used to achieveO(1)
request
andrelease
.- LIFO ordering is used to help reduce connection timeouts.
- Lots of
trace
anddebug
logging to help diagnose issues. deinit
assertions to help quickly track down pools not being closed properly
Cons:
- You cannot share connections between
EventLoop
s and you must have at least one connection per loop. There is no way to set an application-wide max connection count. This has been a fairly consistent point of confusion for Vapor developers. - Very limited configurability, i.e., no min connection count option, "leaky connection" option, etc.
- No guarantees that connections are not being used after returned to the pool.