NIO-based Redis Driver
- Proposal: SSWG-NNNN
- Authors: Nathan Harris
- Sponsors: Vapor
- Review Manager: TBD
- Status: Implemented
- Pitch: Server/Pitches/SwiftNIO Redis Client
- Implementation: mordil/nio-redis
Package Description
Non-blocking Swift driver for Redis built on SwiftNIO.
| Package name | nio-redis |
| Proposed Maturity Level | Incubating |
| License | Apache 2 |
| Dependencies | SwiftNIO 2.x, Server Logging API 0.1.x |
Introduction
NIORedis is a module providing general implementations for connecting to Redis and executing
commands against an instance using its proprietary Redis Seralization Protocol (RESP).
These types are designed to work in a request / response loop, representing individual connections to Redis.
The goal of this library is to provide a semi-low level API to work with Redis, while still feeling like a normal Swift package that can be used as-is.
Motivation
Implementations of Swift Redis clients have been around for as long as Swift has, but most have been abandoned, rely on Objective-C runtimes, or use C libraries.
All of the currently maintained libraries either have framework specific dependencies, are not built with NIO, or do not provide enough extensibility while providing "out of the box" capabilities.
Existing Solutions
Proposed Solution
NIORedis provides the essential types for interacting with RESP and building NIO Channel pipelines for communicating with Redis, with default implementations designed to cover most use cases.
RESPValue
RESPValue represents the different types outlined in Redis' protocol as an enum.
public enum RESPValue {
case null
case simpleString(String)
case bulkString([UInt8])
case error(RedisError)
case integer(Int)
case array([RESPValue])
}
RESPValueConvertible
RESPValue needs to be translatable between Swift values and RESP more easily and generically - which is provided by the RESPValueConvertible protocol.
public protocol RESPValueConvertible {
init?(_ value: RESPValue)
func convertedToRESPValue() -> RESPValue
}
Default conformance is provided for:
- Optional where Wrapped: RESPValueConvertible
- Array where Element: RESPValueConvertible
- RedisError
- RESPValue
- String
- FixedWidthInteger (Int, Int8, Int16, ...)
- Double
- Float
RedisClient
As a protocol, RedisClient just defines a type that is capable of sending commands and receiving responses.
public protocol RedisClient {
var eventLoop: EventLoop { get }
func send(command: String, with arguments: [RESPValueConvertible]) -> EventLoopFuture<RESPValue>
}
In general, this is the type to rely on for executing commands - all convenience command extensions are applied to this type so that conformers may gain the benefits of implementations for.
An example of this is RedisPipeline, which conforms to RedisClient in order to implement it's internal pipelining logic.
RedisConnection
As the primary connection type (and RedisClient implementation), RedisConnection works with NIO's ClientBootstrap to build a pipeline for executing commands in a request / response cycles.
It is designed to be long-lived; being re-used between commands as needed.
public final class RedisConnection: RedisClient {
public var eventLoop: EventLoop { get }
public var isConnected: Bool { get }
public init(channel: Channel, logger: Logger = default)
@discardableResult
public func close() -> EventLoopFuture<Void>
public func makePipeline() -> RedisPipeline
public func send(command: String, with arguments: [RESPValueConvertible]) -> EventLoopFuture<RESPValue>
}
While RedisConnection can be created on the fly with just a Channel instance, it's more likely you won't have one and can instantiate a connection with the following static method:
extension RedisConnection {
public static func connect(
to socket: SocketAddress,
with password: String? = nil,
on eventLoopGroup: EventLoopGroup,
logger: Logger = default) -> EventLoopFuture<RedisConnection>
}
Example usage of RedisConnection:
import NIORedis
// create a new event loop group
let elg = MultiThreadedEventLoopGroup(numberOfThreads: 1)
defer { try! elg.syncShutdownGracefully() }
// create a new connection
let address = try SocketAddress(ipAddress: "127.0.0.1", port: 6379)
let connection = RedisConnection.connect(to: address, with: "password", on: elg.next()).wait()
defer { try! connection.close().wait() }
let value = connection.get("my_key")
RedisPipeline
Redis provides a decent overview of the benefits of providing pipelining, and NIORedis does this by
providing the RedisPipeline object, created from RedisConnection.makePipeline().
Roughly stated - a regular command through RedisConnection will be a "write and flush" operation, while RedisPipeline will write all of the commands, but not flush until execute() is called.
public final class RedisPipeline {
/// Number of queued commands
public var count: Int
public init(channel: channel, Logger: Logger = default)
/// Returns `self` to allow chaining
@discardableResult
public func enqueue<T>(operation: (RedisClient) -> EventLoopFuture<T>) -> RedisPipeline
/// Drains the queue
public func execute() -> EventLoopFuture<[RESPValue]>
}
An example of usage:
let connection = RedisConnection.connect(...)
let results = connection.makePipeline()
.enqueue { $0.set("my_key", to: 1) }
.enqueue { $0.get("my_key") }
.enqueue { $0.increment("my_key") }
.enqueue { $0.increment("my_key", by: 30) }
.execute()
.wait()
// results = [RESPValue]
// results[0].string == "OK"
// results[1].int == 1
// results[2].int == 2
// results[3].int == 32
RESPEncoder
As part of the pipeline, RESPValue needs to be translated to RESP byte streams, which is handled by the publicly available RESPEncoder.
This class conforms to NIO's MessageToByteEncoder protocol.
public final class RESPEncoder {
public init(logger: Logger = default)
public func encode(_ value: RESPValue, into buffer: inout ByteBuffer)
}
RESPDecoder
Inversely, RESPDecoder will translate RESP byte streams into RESPValues.
This also conforms to NIO's ByteToMessageDecoder protocol.
public final class RESPDecoder {
public enum ParsingState {
case notYetParsed
case parsed(RESPValue)
}
public func parse(at position: inout Int, from buffer: inout ByteBuffer) throws -> ParsingState
}
RedisCommandHandler
Redis' request / response cycle is a 1:1 mapping, and at the end of the Redis channel pipeline is RedisCommandHandler to marshal the queue of incoming and outgoing messages.
As RedisCommandContext instances are written to the channel, RedisCommandHandler pulls out the information to store callbacks in a queue.
When responses from Redis return, callbacks are popped off the queue and given the response as RESPValues.
public struct RedisCommandContext {
public let command: RESPValue
public let responsePromise: EventLoopPromise<RESPValue>
}
open class RedisCommandHandler {
public init(logger: Logger = default)
}
Maturity Justification
Until now, packages through the SSWG process have been accepted as Sandbox maturity - so it's appropriate to justify why NIORedis should be considered mature enough for Incubating.
This package supports:
- ~90 of Redis' commands as convenient extension methods
- 130+ unit tests, including RESP encoding / decoding and all command extensions
In addition, it meets the following criteria according to the SSWG Incubation Process:
As well as Vapor, and I, are in the processes of providing a (framework agnostic) higher-level library: RedisKit that Vapor has said they will be using as their implementation in version 4.
Seeking Feedback
- API-wise, what are things you do or don't like?
- Is there anything we could remove and still be happy?
- Is there anything you think is missing before it can be accepted?
- I'm working on Pub/Sub right now
- Known Issues / Feature Requests
. The core message here is: First, we should design the best API, then we should look how to implement this in the most efficient way. I believe that NIO is flexible enough that any reasonable API can be implemented in the most flexible way, of course using things like Redis pipelining without surfacing the word/concept of 'pipelining' to the user.
