[Discussion] gRPC Swift

gRPC Swift

Package Description

A gRPC client and server library with code generation.

Package Name GRPC
Module Name GRPC, protoc-gen-grpc-swift
Proposed Maturity Level Sandbox
License Apache 2
Dependencies SwiftNIO 2.8, SwiftNIO HTTP2 1.6, SwiftNIO SSL 2.4, SwiftNIO Transport Services 1.1, SwiftProtobuf 1.5, SwiftLog 1.0

Introduction

gRPC Swift is a community backed implementation of the gRPC over HTTP/2 protocol built on top of SwiftNIO. It includes a runtime (GRPC), a code generator (protoc-gen-grpc-swift), an extensive suite of tests and examples.

Motivation

gRPC is an industry standard protocol and a fundamental building block for microservices enabling interoperability with services written in other languages. Many implementations wrap a core C-library which can lead to memory safety issues and is difficult to debug. There are further rough edges on iOS where clients have to deal with network connectivity changes (e.g. LTE to WiFi). Having a gRPC server and client implementation in Swift built on top of Swift NIO will help to eliminate or reduce each of these issues.

Proposed solution

We will use SwiftNIO to provide the network layer, SwiftNIO SSL for TLS, SwiftNIO HTTP2 for HTTP/2, and SwiftProtobuf will be used for message serialization. We will also use SwiftNIO Transport Services to provide first-class support for Apple Platforms.

The following tutorials are intended as an introduction and high level overview of gRPC Swift:

  • Hello World: a quick-start guide to create and call your first gRPC service using Swift, and
  • Route Guide: a more in-depth tutorial covering service definition, generation of client and server code, and using the gRPC Swift API to create a client and server.

Design

This section assumes you have read the Route Guide tutorial which covers much of the high level API.

Server

The Server's NIO pipeline follows (channel handlers provided by gRPC Swift are denoted "[GS]"):

  • NIOSSLHandler (if TLS is being used)
  • If HTTP/1 (i.e. gRPC-Web):
    • HTTPServerPipelineHandler
    • WebCORSHandler [GS]
  • Otherwise (i.e. HTTP/2, "standard" gRPC):
    • NIOHTTP2Handler
    • HTTP2StreamMultiplexer
    • HTTP2ToHTTP1ServerCodec
  • HTTP1ToRawGRPCServerCodec [GS]: translates HTTP/1 types to gRPC metadata and length-prefixed messages, it also handles request/response state and message buffering since messages may span multiple frames. It is "Raw" since the messages are bytes (i.e. not deserialized Protobuf messages).
  • GRPCChannelHandler [GS]: configures the pipeline on receiving the request head by looking at the request URI and finding an appropriate service provider. This handler is removed from the pipeline when the handler has configured the rest of the pipeline.
  • GRPCCallHandler [GS]: handles the delivery of requests and responses to and from a user implemented call handler.

GRPCCallHandler has one class per RPC type, each inheriting from BaseCallHandler:

User logic is provided via the implementation of a generated protocol which conforms to CallHandlerProvider:

/// Provides `GRPCCallHandler` objects for the methods on a particular service name.
///
/// Implemented by the generated code.
public protocol CallHandlerProvider: class {
  /// The name of the service this object is providing methods for, including the package path.
  ///
  /// - Example: "io.grpc.Echo.EchoService"
  var serviceName: String { get }

  /// Determines, calls and returns the appropriate request handler (`GRPCCallHandler`), depending on the request's
  /// method. Returns nil for methods not handled by this service.
  func handleMethod(_ methodName: String, callHandlerContext: CallHandlerContext) -> GRPCCallHandler?
}

Functions on the generated protocol (the service provider protocol) correspond to RPCs in the service definition. A default implementation of handleMethod(_:callHandlerContext:) on the generated protocol maps the method name to an appropriate GRPCCallHandler (based on the call type) and method implementation (provided by the user) and calls it with the necessary context.

The service provider implementation is provided to the server as part of its configuration, which in turn is provided to GRPCChannelHandler:

let configuration = Server.Configuration(
  // Host and port to bind to.
  target: .hostAndPort("localhost", 0),
  // EventLoopGroup to run on.
  eventLoopGroup: eventLoopGroup,
  // Array of CallHandlerProviders, i.e. the services to offer.
  serviceProviders: [ServiceProviderImpl()]
  // An error delegate.
  errorDelegate: nil,
  // TLS configuration, a subset of NIO's TLSConfiguration.
  tls: nil
)

let server: EventLoopFuture<Server> = Server.start(configuration: configuration)

The configuration (Server.Configuration) includes what the server should bind to (host and port, Unix domain socket), the event loop group it use, TLS configuration (optional), error delegate (optional) and a list of CallHandlerProviders which it may use to serve requests.

The start method on Server uses the configuration to create a bootstrap and bind to the desired target.

Client

The client's channel pipeline follows, gRPC Swift provided handlers are denoted "[GS]":

  • NIOSSLHandler (if TLS is being used)
  • NIOHTTP2Handler
  • HTTP2StreamMultiplexer

Each call is made on an HTTP/2 stream channel whose pipeline is:

  • GRPCClientChannelHandler [GS]: translates between HTTP/2 frames to gRPC client request and response parts. The response parts include: the initial metadata (headers), response message(s), trailing metadata (headers), and the gRPC status (code and message). Manages the state of the request and response streams via GRPCClientStateMachine.
  • ClientResponseChannelHandler [GS]: surfaces the response parts listed above to the user via promises and callbacks (for streaming responses).

These handlers are currently work-in-progress while we replace an older implementation.

Note that other handlers exist in the pipeline for error handling and verification (i.e. TLS handshake was successful and a valid protocol was negotiated) but were omitted for brevity.

The differences between the four call types are just in their construction and request and response handlers.

Making Calls

The user makes RPC calls using a generated client and receives a ClientCall:

public protocol ClientCall {
  associatedtype RequestMessage: Message
  associatedtype ResponseMessage: Message

  /// Initial response metadata.
  var initialMetadata: EventLoopFuture<HTTPHeaders> { get }

  /// Status of this call which may be populated by the server or client.
  var status: EventLoopFuture<GRPCStatus> { get }

  /// Trailing response metadata.
  var trailingMetadata: EventLoopFuture<HTTPHeaders> { get }

  /// Cancel the current call.
  func cancel()
}

The calls which have a single response from the server (unary and client streaming) implement UnaryResponseClientCall which extends ClientCall to include a future response:

public protocol UnaryResponseClientCall: ClientCall {
  /// The response message returned from the service if the call is successful.
  /// This may be failed if the call encounters an error.
  var response: EventLoopFuture<ResponseMessage> { get }
}

For calls which have any number of responses from the server (server streaming and bidirectional streaming), constructing the call requires a response handler: (ResponseMessage) -> Void.

Calls sending a single request to the server (unary and server streaming) accept a single request on initialization. Calls which send any number of requests to the server (client streaming and bidirectional streaming) return a call which conforms to StreamingRequestClientCall which extends ClientCall to provide methods for sending messages to the server:

public protocol StreamingRequestClientCall: ClientCall {
  /// Sends a message to the service.
  func sendMessage(_ message: RequestMessage) -> EventLoopFuture<Void>
  func sendMessage(_ message: RequestMessage, promise: EventLoopPromise<Void>?)

  /// Sends a sequence of messages to the service.
  func sendMessages<S: Sequence>(_ messages: S) -> EventLoopFuture<Void> where S.Element == RequestMessage
  func sendMessages<S: Sequence>(_ messages: S, promise: EventLoopPromise<Void>?) where S.Element == RequestMessage

  /// Terminates a stream of messages sent to the service.
  func sendEnd() -> EventLoopFuture<Void>
  func sendEnd(promise: EventLoopPromise<Void>?)
}

Each of the four call types can be made from factory methods on the GRPCClient protocol. Their call signatures are:

public func makeUnaryCall<Request: Message, Response: Message>(
  path: String,
  request: Request,
  callOptions: CallOptions? = nil,
  responseType: Response.Type = Response.self
) -> UnaryCall<Request, Response>

public func makeServerStreamingCall<Request: Message, Response: Message>(
  path: String,
  request: Request,
  callOptions: CallOptions? = nil,
  responseType: Response.Type = Response.self,
  handler: @escaping (Response) -> Void
) -> ServerStreamingCall<Request, Response>

public func makeClientStreamingCall<Request: Message, Response: Message>(
  path: String,
  callOptions: CallOptions? = nil,
  requestType: Request.Type = Request.self,
  responseType: Response.Type = Response.self
) -> ClientStreamingCall<Request, Response>

public func makeBidirectionalStreamingCall<Request: Message, Response: Message>(
  path: String,
  callOptions: CallOptions? = nil,
  requestType: Request.Type = Request.self,
  responseType: Response.Type = Response.self,
  handler: @escaping (Response) -> Void
) -> BidirectionalStreamingCall<Request, Response>

This allows the code generation to be simple: the generated client stubs call these functions with some static information, such as the path (e.g. "/routeguide.RouteGuide/GetFeature") and the appropriate request and response types.

In cases where no client has been generated, an AnyServiceClient may be used. It provides the factory methods listed previous without any stubs a particular service:

let anyServiceClient: AnyServiceClient = ...

// Equivalent to: routeGuide.getFeature(...)
let getFeature = anyServiceClient.makeUnaryCall(
  path: "/routeguide.RouteGuide/GetFeature",
  request: Routeguide_Point.with {
    // ...
  },
  responseType: Routeguide_Feature.self
)

Making Clients

A client requires a connection to a gRPC server, this is done via ClientConnection which, like the server, is initialized with some configuration:

let configuration = ClientConnection.Configuration(
  target: .hostAndPort("localhost", "8080"),
  eventLoopGroup: group,
  // Delegates for observing errors and connectivity state changes:
  errorDelegate: nil,
  connectivityStateDelegate: nil,
  // TLS configuration, a subset of NIO's TLSConfiguration:
  tls: nil,
  // Connection backoff configuration:
  connectionBackoff: ConnectionBackoff()
)

let connection = ClientConnection(configuration: configuration)

Clients take a ClientConnection and an optional CallOptions struct on initialization:

// Call options used for each call unless specified at call time.
// Has support for custom metadata (headers) and call timeouts amongst a few
// other things.
let defaultCallOptions = CallOptions(timeout: .seconds(rounding: 90))

// Create a client, this would usually be generated from a proto.
let routeGuide = Routeguide_RouteGuideServiceClient(
  connection: connection,
  defaultCallOptions: defaultCallOptions  // optional
)
Connection lifecycle

The connection is created using the exponential backoff algorithm described by gRPC. The state of the connection is monitored (using the states defined by gRPC: idle, connecting, ready, transient failure, and shutdown) and will automatically reconnect (with backoff) if the channel is closed but the close was not initiated by the user. Users may optionally provide a connectivity state delegate to observe these changes:

public protocol ConnectivityStateDelegate {
  func connectivityStateDidChange(from oldState: ConnectivityState, to newState: ConnectivityState)
}

This text will be hidden

Apple Platform Support

The library also provides a means to run using NIO Transport Services instead where it’s supported on Apple platforms. The user only has to provide a correctly typed EventLoopGroup in their Configuration and gRPC Swift will pick the appropriate bootstrap. To aid this we provide some utility functions:

public enum NetworkPreference {
  // NIOTS when available, NIO otherwise
  case best
  // Pick manually
  case userDefined(NetworkImplementation).
}

public enum NetworkImplementation {
  // i.e. NIOTS (this has the appropriate @available/#if canImport(Network))
  case networkFramework
  // i.e. NIO
  case posix
}

public enum PlatformSupport {
  // Returns an EventLoopGroup of the appropriate type based on user preference.
  public static func makeEventLoopGroup(
    loopCount: Int,
    networkPreference: NetworkPreference = .best
  ) -> EventLoopGroup {
    // ...
  }
}

One thing to note with the NIO Transport Services support is that TLS is always provided by SwiftNIO SSL, even when Network.framework is being used. Ideally we would provide TLS via Network.framework if it’s being used, however abstracting over the different configuration for the two interfaces is not trivial.

Maturity Justification

gRPC Swift has approximately 300 tests including the gRPC Interoperability Test Suite and has CI on macOS and Ubuntu 18.04, each running Swift 5.0 and Swift 5.1. In CI the interoperability test suite is run against the gRPC C++ interoperability test server.

However, there is not enough production use to justify incubation maturity.

Alternatives considered

Wrapping the core gRPC library and providing bindings in Swift (the current gRPC Swift approach) has the issues set out in the Introduction. This approach is also not aligned with the SSWG minimal requirements since it wraps C instead of providing a native approach.

19 Likes

This looks great. I have no particular feedback to provide other than my sheer enthusiasm.

4 Likes

Very strong +1 from me. The SSWG absolutely needs a gRPC client/server and this implementation looks great!

I've used the SwiftNIO gRPC and it's much better than the previous non-NIO version (I've had connectivity issues and other implementation issues with it) Currently I'm using it in my Azure Functions Swift language worker Azure Functions Swift support. The very first challenge in building it was finding a good reliable Swift gRPC library, as the Azure Functions host communicates using gRPC. I was so happy to find the SwiftNIO gRPC! Also I used the code generator to generate the Swift protobufs

4 Likes

Thanks very much everybody. The proposal has been accepted (on the 19th February 2020) to "sandbox" and is now listed in the SSWG Package Index.

The SSWG identified one caveat: Currently, the default branch on Github (master) contains the old libgrpc based version and the new version (1.x series) is on the nio branch. The SSWG would prefer if the new version were on the master branch. That would also help with discoverability of the new version. CC @georgebarnett/@timburks/@MrMage/Michael Rebello.

2 Likes

Thanks @johannesweiss.

My proposal regarding branches would be:

  • Rename master to cgrpc
  • Set nio to be the default branch on GitHub
2 Likes

I agree to George's proposal. Should we do this right now or with 1.0? How far out would 1.0 be? Do we need Tim's technical assistance to make the switch?

We already recommend that people use it now so I'd be happy to change it now. I just checked and I don't have permissions to change it, so it looks like we would need Tim's help here.

1 Like

I support this change - Daniel now has admin-level access to the repo and can make the change as soon as he's ready.

Great, thanks @timburks

To clarify: I was not suggesting that we rename nio to master, however @MrMage prefers master over nio. I'm fine with either but wanted to check if anyone else has a preference on this?

I personally also care more about the default branch but other voices found it odd to have the default branch not be called master.

CC @server-workgroup

imo people generally assume master is where mainline development is happening so i would go for that

3 Likes

This has now been done thanks to @MrMage :pray:

  • master has the new NIO-based version of gRPC Swift, development will happen here
  • cgrpc has the old version of gRPC Swift and a notice redirect people to master
  • nio still exists to serve the purpose of redirecting people to master
5 Likes

nice. shell we close this topic now?

Thanks so much everybody involved, most importantly @georgebarnett & @MrMage . +1 on closing this.

3 Likes