Hello, I'd like to pitch adding Span family of overloads to read/write operations on FileDescriptor. You can see the gist here
Span-based File I/O for Swift System
- Proposal: SYS-0007
- Authors: Michael Ilseman, Guillaume Lessard
- Review Manager: TBD
- Status: Draft
- Implementation: apple/swift-system#290
Introduction
We propose adding Span-based APIs to Swift System's FileDescriptor type for reading and writing data. These new APIs complement the existing UnsafeRawBufferPointer-based methods, providing memory-safe alternatives.
Note: Because this is the first time we are revisiting old API to adopt Span types, we are writing this up as a proposal and posting on the forums. This proposal establishes some of the conventions and thinking regarding Span types and their adoption. Going forwards, Span adoption following similar patterns can generally be reviewed in PR.
Motivation
Swift System's existing file I/O methods predate Span and thus use the UnsafeRawBufferPointer family of types for borrowed views into contiguous memory. These types are unsafe and do not track partial initialization (e.g. for partial reads/writes).
Swift 6.2 introduces the Span family of types, which provide safe alternatives. For example, reading a chunk and processing it changes from:
let chunk = try [UInt8](unsafeUninitializedCapacity: 4096) { buf, count in
let rawBuf = UnsafeMutableRawBufferPointer(buf)
count = try fd.read(into: rawBuf)
}
process(chunk)
to:
let chunk = try [UInt8](unsafeUninitializedCapacity: 4096) { buf, count in
var output = OutputRawSpan(
buffer: UnsafeMutableRawBufferPointer(buf), initializedCount: 0)
try fd.read(into: &output)
count = output.byteCount
}
process(chunk)
Note that this particular example still requires constructing the OutputRawSpan manually due gaps in stdlib API. When the stdlib gains better conversions between typed and raw memory spans and better raw memory inits, this example will read more like:
// Note: Requires future stdlib Array init addition
let chunk = try [UInt8](rawCapacity: 4096) { buffer in
try fd.read(into: &buffer)
}
process(chunk)
Proposed Solution
We propose adding Span-based overloads to FileDescriptor that mirror the existing unsafe pointer APIs:
write(_: RawSpan)write(toAbsoluteOffset:_: RawSpan)writeAll(_: RawSpan)writeAll(toAbsoluteOffset:_: RawSpan)read(into: inout OutputRawSpan)read(fromAbsoluteOffset: Int64, into: inout OutputRawSpan)
Write operations take RawSpan (read-only bytes). Read operations take inout OutputRawSpan, which tracks how many bytes have been initialized, making it a natural fit for partial reads.
We also propose new convenience methods which provide analogues to writeAll:
read(filling: inout OutputRawSpan)read(fromAbsoluteOffset: Int64, filling: inout OutputRawSpan)
These loop until the buffer is full (or EOF), handling partial reads automatically.
Note: These additions require a Swift 6.2 compiler.
Detailed Design
Write Operations
extension FileDescriptor {
/// Writes the contents of a buffer at the current file offset.
///
/// - Parameters:
/// - data: The region of memory that contains the data being written.
/// - retryOnInterrupt: Whether to retry the write operation
/// if it throws ``Errno/interrupted``.
/// The default is `true`.
/// Pass `false` to try only once and throw an error upon interruption.
/// - Returns: The number of bytes that were written.
///
/// After writing,
/// this method increments the file's offset by the number of bytes written.
/// To change the file's offset,
/// call the ``seek(offset:from:)`` method.
///
/// The corresponding C function is `write`.
@available(SwiftStdlib 6.2, *)
@_alwaysEmitIntoClient
public func write(
_ data: RawSpan,
retryOnInterrupt: Bool = true
) throws(Errno) -> Int
/// Writes the contents of a buffer at the specified offset.
///
/// - Parameters:
/// - offset: The file offset where writing begins.
/// - data: The region of memory that contains the data being written.
/// - retryOnInterrupt: Whether to retry the write operation
/// if it throws ``Errno/interrupted``.
/// The default is `true`.
/// Pass `false` to try only once and throw an error upon interruption.
/// - Returns: The number of bytes that were written.
///
/// Unlike ``write(_:retryOnInterrupt:)``,
/// this method leaves the file's existing offset unchanged.
///
/// The corresponding C function is `pwrite`.
@available(SwiftStdlib 6.2, *)
@_alwaysEmitIntoClient
public func write(
toAbsoluteOffset offset: Int64,
_ data: RawSpan,
retryOnInterrupt: Bool = true
) throws(Errno) -> Int
/// Writes the entire contents of a buffer, retrying on partial writes.
///
/// - Parameters:
/// - data: The region of memory that contains the data being written.
/// - retryOnInterrupt: Whether to retry the write operation
/// if it throws ``Errno/interrupted``.
/// The default is `true`.
/// Pass `false` to try only once and throw an error upon interruption.
/// - Returns: The number of bytes that were written.
///
/// After writing,
/// this method increments the file's offset by the number of bytes written.
/// To change the file's offset,
/// call the ``seek(offset:from:)`` method.
///
/// The corresponding C function is `write`.
@available(SwiftStdlib 6.2, *)
@_alwaysEmitIntoClient
public func writeAll(
_ data: RawSpan,
retryOnInterrupt: Bool = true
) throws(Errno) -> Int
/// Writes the entire contents of a buffer at the specified offset,
/// retrying on partial writes.
///
/// - Parameters:
/// - offset: The file offset where writing begins.
/// - data: The region of memory that contains the data being written.
/// - retryOnInterrupt: Whether to retry the write operation
/// if it throws ``Errno/interrupted``.
/// The default is `true`.
/// Pass `false` to try only once and throw an error upon interruption.
/// - Returns: The number of bytes that were written.
///
/// Unlike ``writeAll(_:retryOnInterrupt:)``,
/// this method leaves the file's existing offset unchanged.
///
/// The corresponding C function is `pwrite`.
@available(SwiftStdlib 6.2, *)
@_alwaysEmitIntoClient
public func writeAll(
toAbsoluteOffset offset: Int64,
_ data: RawSpan,
retryOnInterrupt: Bool = true
) throws(Errno) -> Int
}
Read Operations
Read operations use OutputRawSpan, which tracks both the capacity of the buffer and how many bytes have been initialized.
extension FileDescriptor {
/// Reads bytes at the current file offset into a buffer.
///
/// - Parameters:
/// - buffer: The region of memory to read into.
/// - retryOnInterrupt: Whether to retry the read operation
/// if it throws ``Errno/interrupted``.
/// The default is `true`.
/// Pass `false` to try only once and throw an error upon interruption.
/// - Returns: The number of bytes that were read.
///
/// After reading,
/// this method increments the file's offset by the number of bytes read.
/// To change the file's offset,
/// call the ``seek(offset:from:)`` method.
///
/// The corresponding C function is `read`.
@available(SwiftStdlib 6.2, *)
@_alwaysEmitIntoClient
public func read(
into buffer: inout OutputRawSpan,
retryOnInterrupt: Bool = true
) throws(Errno) -> Int
/// Reads bytes at the specified offset into a buffer.
///
/// - Parameters:
/// - offset: The file offset where reading begins.
/// - buffer: The region of memory to read into.
/// - retryOnInterrupt: Whether to retry the read operation
/// if it throws ``Errno/interrupted``.
/// The default is `true`.
/// Pass `false` to try only once and throw an error upon interruption.
/// - Returns: The number of bytes that were read.
///
/// Unlike ``read(into:retryOnInterrupt:)``,
/// this method leaves the file's existing offset unchanged.
///
/// The corresponding C function is `pread`.
@available(SwiftStdlib 6.2, *)
@_alwaysEmitIntoClient
public func read(
fromAbsoluteOffset offset: Int64,
into buffer: inout OutputRawSpan,
retryOnInterrupt: Bool = true
) throws(Errno) -> Int
}
Looping Read Operations
The read(filling:) methods loop until the buffer is completely filled (or EOF), handling partial reads automatically. This is the read-side analog to writeAll.
extension FileDescriptor {
/// Reads bytes into a buffer, retrying until it is full.
///
/// - Parameters:
/// - buffer: The region of memory to read into.
/// - retryOnInterrupt: Whether to retry the read operation
/// if it throws ``Errno/interrupted``.
/// The default is `true`.
/// Pass `false` to try only once and throw an error upon interruption.
/// - Returns: The number of bytes that were read.
///
/// After reading,
/// this method increments the file's offset by the number of bytes read.
/// To change the file's offset,
/// call the ``seek(offset:from:)`` method.
///
/// The corresponding C function is `read`.
@available(SwiftStdlib 6.2, *)
@_alwaysEmitIntoClient
@discardableResult
public func read(
filling buffer: inout OutputRawSpan,
retryOnInterrupt: Bool = true
) throws(Errno) -> Int
/// Reads bytes at the specified offset into a buffer,
/// retrying until it is full.
///
/// - Parameters:
/// - offset: The file offset where reading begins.
/// - buffer: The region of memory to read into.
/// - retryOnInterrupt: Whether to retry the read operation
/// if it throws ``Errno/interrupted``.
/// The default is `true`.
/// Pass `false` to try only once and throw an error upon interruption.
/// - Returns: The number of bytes that were read.
///
/// Unlike ``read(filling:retryOnInterrupt:)``,
/// this method leaves the file's existing offset unchanged.
///
/// The corresponding C function is `pread`.
@available(SwiftStdlib 6.2, *)
@_alwaysEmitIntoClient
@discardableResult
public func read(
fromAbsoluteOffset offset: Int64,
filling buffer: inout OutputRawSpan,
retryOnInterrupt: Bool = true
) throws(Errno) -> Int
}
Source Compatibility
This proposal is purely additive. The existing UnsafeRawBufferPointer-based methods remain available and unchanged. Existing code continues to compile and function identically.
ABI Compatibility
This proposal is purely additive. All new methods use @_alwaysEmitIntoClient, meaning they are inlined at the call site and do not affect the library's ABI. This enables deployment to older OS versions that support Swift System but predate Swift 6.2.
Implications on Adoption
Adopting the new Span-based APIs requires:
- Swift 6.2 or later (for Span types)
- No minimum OS deployment target changes (due to
@_alwaysEmitIntoClient)
Existing code using the unsafe APIs can migrate incrementally; both API styles can coexist.
Alternatives Considered
Deprecate the Unsafe Pointer APIs
We considered deprecating the existing UnsafeRawBufferPointer-based methods in favor of the new Span APIs. Since Swift System targets low-level systems programming where C interoperability and direct pointer manipulation are common, we retain the unsafe APIs for now.
This proposal is the first instance of updating existing System API for Span, so it sets precedent. We are keeping the pre-existing UnsafeRawBufferPointer overloads rather than deprecating them. This does not set a precedent of adding UnsafeRawBufferPointer overloads for new API going forward; new API should prefer Span.
We may revisit deprecation in the future, pending investigation into how easily code can migrate between the two interfaces.
Add OutputSpan<UInt8> Overloads
We considered providing OutputSpan<UInt8> overloads in addition to OutputRawSpan. This would allow callers using standard library APIs like Array.init(capacity:initializingWith:) (which provides an OutputSpan<Element>) to pass the buffer directly without conversion.
However, there is currently no direct conversion path from OutputSpan<UInt8> to OutputRawSpan. Rather than work around this limitation by doubling the API surface, we encourage the standard library and language to provide better interconversion between typed and raw span types. File I/O at the System level operates below concerns like memory layout and endianness; we are simply moving bytes. The raw span types are the natural fit for this use case.
For write operations, Span<UInt8> already has the .bytes property to obtain a RawSpan, so no additional overloads are needed there.
Add MutableRawSpan overloads for read operations
We use OutputRawSpan rather than MutableRawSpan for read operations because it better models the semantics of reading. Reads should always be treated as partial (due to EOF) and OutputRawSpan models this.
A MutableRawSpan represents fully initialized memory. In the event of a partial fill, the span would still claim its full size is valid, requiring careful bookkeeping and checking. This is an error-prone pattern common in C buffer APIs that OutputRawSpan explicitly solves.
Naming: read(filling:) vs readAll
We chose read(filling:) over readAll because it describes the operation more accurately: reading until the provided buffer is completely filled. The name readAll suggests reading all of the file, not filling all of the buffer.
Returning Count vs Throwing on EOF
read(filling:) returns the byte count rather than throwing on EOF because EOF is not an error. The return value and the buffer's initialized count provide redundant but consistent information: callers can use whichever is more convenient. To check for a complete fill, compare the return value against the original free capacity, or check if the buffer is full.
The return value is marked @discardableResult since callers can inspect the buffer instead.
Typed Span Variants
We do not provide typed variants like OutputSpan<T> or Span<T> for arbitrary element types. File I/O operates on raw bytes, and reading or writing multi-byte types requires explicit handling of endianness, even for trivial bitwise-copyable types. General typed variants could be added in a follow-on proposal once ecosystem patterns for byte-level serialization are established.
Future Directions
InputSpan for Write Operations
This proposal reveals an asymmetry in the Span family. For reads, OutputRawSpan tracks how many bytes have been initialized, so partial reads update the span automatically. For writes, RawSpan has no equivalent tracking, so callers must manually slice the buffer after a partial write:
var remaining = data
while remaining.byteCount > 0 {
let written = try fd.write(remaining)
remaining = remaining.extracting(droppingFirst: written)
}
An InputSpan type that tracks consumption would complete the symmetry:
// Future direction (not proposed)
var input = InputRawSpan(data)
while input.byteCount > 0 {
try fd.write(consuming: &input) // span updates automatically
}
This motivates future work on the Span family.
Vectored I/O (readv/writev)
Vectored I/O operations that accept multiple spans could improve performance for scatter-gather patterns:
// Future direction (not proposed)
extension FileDescriptor {
func write(gatheringFrom spans: some Sequence<RawSpan>) throws(Errno) -> Int
func read(scatteringInto spans: inout some MutableCollection<OutputRawSpan>) throws(Errno) -> Int
}
This kind of API is not currently expressible in Swift. Span types are non-escapable and have lifetime dependencies that prevent them from being stored in standard collections or passed as sequences.
Acknowledgments
This proposal builds on the Span family of types designed by Guillaume Lessard and the Swift Standard Library team. The design follows patterns established in the ongoing Swift System evolution.