Pitch 2: Add FilePath to stdlib

Evolution Pitches
1

Hello all, I've written a second draft for inclusion of FilePath in the stdlib. I've trimmed many of the syntactic operations, focusing on what is essential to get in Swift: the types, fundamental conformances, and behavior fixes (like trailing slash). Future proposals can include many of the convenience methods.

Full proposal viewable at FilePath in the stdlib - pitch 2 · GitHub

Add FilePath to the Standard Library

Introduction

We propose adding FilePath and its essential operations to the Swift standard library. FilePath parses platform-specific path syntax on the developer's behalf, provides a normalized view of path components, and enables resolution against the filesystem. This proposal focuses on establishing the core type, its conformances, and the essential operations needed to adopt it as a currency type. Additional syntactic convenience methods and platform-specific path decomposition API are planned as future additions. See Future Directions.

Motivation

Swift has no standard representation for file system paths. The swift-system package introduced FilePath in System 0.0.1, and it has since gained a comprehensive set of syntactic operations for platform-correct path manipulation. But because FilePath lives in an external package, it cannot be depended on by the standard library or the Swift runtime, nor can it appear in API in toolchain libraries such as Foundation.

Every new API that needs to name a file path faces the same dilemma around using String that SE-0513 faces. String is a poor fit for paths as it does not capture the structure of a file path. Path string representations are also platform-specific and path components are not necessarily valid Unicode.

FilePath stores the path in its native platform encoding, enabling strongly-typed programming with paths. It encapsulates the platform-specific details of path syntax (separators, root/prefix forms, special components) so that developers can work with paths correctly without needing to understand each platform's conventions.

Proposed solution

We propose adding FilePath, FilePath.Component, and FilePath.ComponentView to the Swift module, alongside essential functionality for construction, decomposition, resolution, and C interoperability.

var path: FilePath = "/var/www/static/index.html"

path.isAbsolute                      // true
path.hasTrailingSeparator            // false

for component in path.components {
    print(component, component.kind) // "var": .regular, "www": .regular, ...
}

// Resolve against the filesystem
let resolved = try path.resolve()

// Construct a path through mutation
var config: FilePath = "/etc/nginx"
config.components.append("nginx.conf")
// config is "/etc/nginx/nginx.conf"

Detailed design

See full proposal

Source compatibility

All changes are additive.

SystemPackage.FilePath and/or System.FilePath can migrate to the standard library's FilePath via a conditional typealias:

#if compiler(>=6.4) // or whichever version this lands in
public typealias FilePath = Swift.FilePath
#else
public struct FilePath { ... }
#endif

Existing functionality from SystemPackage.FilePath (such as syntactic convenience methods) can be added as extensions on Swift.FilePath on toolchain versions that include this change. This enables a smooth source-compatible migration path.

ABI compatibility

This proposal is purely an extension of the ABI of the standard library and does not change any existing features.

On Darwin, System.FilePath currently has ABI commitments. Migration from the System module on Darwin can be handled through ABI-level redirection so that existing binaries linked against System.FilePath continue to work.

Implications on adoption

Adopters will need a toolchain that includes this change. The type cannot be back-deployed to older runtimes without additional work.

For existing users of swift-system, SystemPackage (the SwiftPM package) can use #if conditionals as described above. System (the Darwin framework) can perform ABI migration, redirecting the existing System.FilePath symbol to the standard library implementation, preserving binary compatibility for existing Darwin binaries.

Future directions

Syntactic operations and path decomposition

Future API for FilePath could include additional syntactic operations and more path decomposition, such as originally pitched. This includes convenience accessors like lastComponent, stem, extension, removingLastComponent(), and mutation methods like append, push, removePrefix. These can be expressed in terms of the ComponentView and are back-deployable.

Lexical resolution and resolve-beneath

Lexical operations that collapse .. components without consulting the filesystem, and operations that ensure a subpath does not escape a base directory, are important for sandboxing and security-sensitive code. These were part of the original pitch and are planned as future additions. Real resolution (via resolve()) is provided in this proposal to ensure developers have the correct tool available from day one.

Path prefix type and analysis

Windows path prefixes and Darwin roots carry more semantic content than Linux. A future FilePath.Prefix type could expose this structure: UNC server/share decomposition, XNU resolve flags, Windows volume identifiers, and more. The isAbsolute, isRooted, and driveLetter queries in this proposal cover the most common needs while the design of a full prefix type matures.

Platform string APIs and CInterop

swift-system defines a CInterop namespace with typealiases for platform-specific character types (PlatformChar, PlatformUnicodeEncoding) and provides init(platformString:) APIs on FilePath and its subtypes. This proposal provides withCString/withWCString for passing paths to C APIs, and Span-based access on components. Future work could include a PlatformChar typealias and more functionality along these lines.

Path parsing APIs and verbatim storage modes

Future work includes exposing FilePath's internal parser for uses over borrowed storage (such as Span), allowing developers to parse paths in-place without copying into a FilePath.

Some use cases (logging, diagnostics) benefit from preserving the exact bytes of a path as originally provided, without any normalization. These would be better handled by referring to the actual original, possibly facilitated with path parsing API over borrowed storage. But, if it proves to be an essential use case, FilePath as pitched is sufficiently resilient that we could add a verbatim storage mode in the future.

SystemString

swift-system internally uses a SystemString type that handles the underlying storage for FilePath. A similar type may be independently useful as a public type for working with null-terminated platform-encoded strings with opaque contents.

More path types and filesystem-specific functionality

Future work could include a paths library or package that adds:

  • Type-enforced invariants: AbsolutePath, ResolvedPath, etc.
  • Platform-specific foreign path types: XNUPath, Win32Path, WinNTPath, etc.
  • File-system specific (and configuration specific) functionality over components: case conversion, Unicode normalization, etc.

Testing Windows path behavior on Linux/macOS (and vice versa) is a known gap that platform-specific path types could address; these could conform to a common protocol while each implementing the semantics of its target platform.

Legacy device name handling

On Windows, legacy device names (CON, NUL, COM1, etc.) receive special treatment from the Win32 layer, and their exact handling varies across Windows versions. Future work could add detection or mitigation for these names, potentially as part of a resolve-beneath or sandboxing API.

Alternatives considered

Do more in Swift 6.4

The original pitch included a FilePath.Root type, convenience methods (lastComponent, stem, extension, starts(with:), append, push), and lexical resolution operations. We chose to defer these in favor of shipping the essential core: the type, its conformances, the component view, and real resolution. The deferred API can be added as extensions on FilePath and is back-deployable where expressed in terms of the component view. Shipping the type and conformances first is critical because those cannot be added retroactively without migration costs (e.g. someone conforming FilePath to Hashable themselves).

Include Codable conformance

FilePath intentionally does not conform to Codable. Serialized paths are inherently platform-specific: a path serialized on Unix is not meaningful on Windows. Path prefix semantics can also change across OS versions (Windows 10 and 11 differ in device name recognition; Darwin may add new resolve flag prefixes). The existing Codable conformance on System.FilePath uses an awkward binary encoding that has been a source of friction. Rather than commit the standard library to a serialization format, we leave serialization to application-level code that can choose a format appropriate to its needs. String(decoding:) and init(_ string:) provide the necessary conversion.

Do more: bring all of swift-system into the toolchain

As discussed in system-in-the-toolchain, it may also make sense to have a System module in the toolchain for low-level OS interfaces and low-level currency types (like FileDescriptor, Errno, etc).

FilePath is different from these other types in that it transcends the entire tech stack, from kernel-level programming to high level scripts and automation. Note that we are not pulling in syscalls such as FilePath.stat, those will remain in System/SystemPackage.

Add FilePath to a separate standard library module

FilePath could live in a new module (e.g. FilePaths, Path, Files, ...) that ships with the toolchain but requires an explicit import. Currency types lose much of their value when they require an import. String, Array, Int, and Result are all in the Swift module; it is our (weakly held) opinion FilePath should be too.

Use Foundation's URL

Foundation's URL is designed for URI semantics, including scheme parsing and percent-encoding. File system paths and URIs have different structure and different invariants. For example, URL(fileURLWithPath:) and URL.appendingPathComponent make blocking file system calls, which is surprising for what appears to be a pure data type. On Unix, paths containing bytes that are not valid UTF-8 cannot survive conversion to a file:// URL, which requires percent-encoding. Foundation also sits high in the dependency stack; the Swift runtime and toolchain components cannot depend on it.

Do nothing

Every new API that needs to name a file will continue using String, perpetuating the loss of structure, platform correctness, and type safety that FilePath was designed to address.

Acknowledgments

Thanks to Saleem Abdulrasool for co-authoring the original FilePath syntactic operations design and implementation in swift-system. Thanks to the participants in the System-in-toolchain discussion, the SE-0513 review, and the first pitch thread for helping shape this proposal.

17 Likes
2

Summary of changes from the original pitch:

Focused on the essential core. This revision ships the type, its conformances, ComponentView (as BidirectionalCollection & RangeReplaceableCollection), and real filesystem resolve(). Convenience methods (lastComponent, stem, extension, append, push, starts(with:), etc.) are deferred as future work, back-deployable via ComponentView.

Root type removed. Windows paths like C:foo don't decompose cleanly into a root. Replaced by targeted queries: isAbsolute, isRelative, isRooted (new), and driveLetter (new). A formal Prefix type is future work.

Lexical operations deferred. lexicallyNormalize, isLexicallyNormal, and lexicallyResolving are removed. A real func resolve() throws -> FilePath replaces them.

Trailing separators preserved. The original pitch stripped them on construction. This revision preserves them and makes them significant for equality ("/tmp/foo" != "/tmp/foo/"), with API to query and manipulate them.

ComponentView normalization specified. The view coalesces repeated separators, converts / to \ on Windows, and drops interior . components (except on Windows verbatim paths). .. is always presented.

Components are opaque byte bags. No stem or extension on Component. You get a Span of platform bytes.

Platform prefix handling expanded. XNU resolve flags (/.resolve/1/), volume identifiers (/.vol/1234/5678/), and UNC trailing separators are now correctly recognized.

Conformance changes. Comparable added to FilePath, Component, and ComponentView. Codable dropped. Span-based byte access and C interop (withCString/withWCString) added directly.

2 Likes
3

Just wanted to note that on a brief reading, I definitely prefer the simpler approach outlined here compared to the original.

4

Are we missing a typedef that abstracts over “platform character” here, so that we could get rid of the platform conditions from most of the interface and have API in terms of Span<FilePath.Character> or Span<PlatformCharacter> instead?

2 Likes
5

It might make sense to avoid committing to the file path code unit being the one true "platform character" and having the type alias be FilePath.CodeUnit or something like that scoped to the FilePath type itself. (Though I'm not sure how much that gives you over the bag-of-bytes approach the proposal follows, given the minimal nature of the rest of the API here.)

3 Likes
6

As a type alias, this does sound like the best path to me (ha).

1 Like
7

I uploaded a new version of the pitch with this change and some related tweaks: https://gist.github.com/milseman/438cd1566a2ac7ba20b36b383da05aaf/revisions

Summary of changes in version 2.1:

  • Added typealias FilePath.CodeUnit = CChar/UInt16 to remove the conditionals around the spans.
  • Span properties and argument labels are now named codeUnits
  • Added OutputSpan<FilePath.CodeUnit> init
  • withCString now takes a pointer of FilePath.CodeUnits
  • withWCString is now named withCString on Windows
6 Likes
9

I think this is a very good starting place. Given the immediate intense commentary on the last version of this pitch, and the fairly slow, gentle commentary on this one, I think we’ve landed in a good place. Your work on this topic is much appreciated

1 Like
10

Hi everyone, this thread is pretty quiet and non-controversial. I urge that we consider advancing this to a formal review for inclusion in Swift 6.4.

I posted a small update to the gist:

**Summary of changes in version 2.2**:

- Added `var nullTerminatedCodeUnits: Span<FilePath.CodeUnit>` to replace `withCString`, no unsafe pointers or closures anymore

This removes any use of unsafe pointers from new API by instead delegating to Span. The downside is we need to pick a name for this property, and I went with nullTerminatedCodeUnits. I recommend we hash out names during the formal review.

FilePath as proposed parses paths the way your platform's kernel parses paths, which is surprisingly non-trivial as Darwin and Windows paths have obscure features and platform-specific prefixes/suffixes. FilePath parses these correctly so that the developer doesn't have to, but they are semantically relevant and affect things like == comparisons.

I want to mention three high-value, but technically severable, additions that would help developers deal with these things when they arise (e.g. by explicitly ignoring or dropping them). Workarounds exist for all three, so it's a question for the Language Steering Group whether any are worth including in the formal review or whether they should be future work. I do not want them to hold up the start of the formal review.

These are presented here for awareness, not as a request for another round of pitch discussion. They can be evaluated during the formal review as severable additions, or deferred entirely.

1. FilePath.Anchor (cross-platform). An opaque type for the prefix region of a path, enabling var anchor: Anchor? { get set } and a reconstruction init FilePath(anchor:components:hasTrailingSeparator:). This completes the decomposition model and provides a deprecation path for swift-system's Root. Without it, transplanting or replacing a path's prefix is done through the existing structured API:

var path: FilePath = #"C:\Users\dev\project"#
...

// Now we want to make it a verbatim-component path using `\\?\`

// With Anchor:
path.anchor = #"\\?\C:\"# 

// Without: construct a new path and transplant the components and anything else that's needed
var result: FilePath = #"\\?\C:\"#
result.components = path.components
if path.hasTrailingSeparator {
    result = result.withTrailingSeparator()
}
path = result

The workaround is functional, but it's easy to lose information in some corner cases. On Darwin, the reconstruction init using Anchor would similarly assist in removing or ignoring resolve flags (an internal and obscure, but semantically-meaningful and important, detail).

Python's pathlib has used .anchor for this concept since 3.4 and we consider this the better name than Root as Root is technically inaccurate for Windows paths such as C:foo. Rust uses the name "prefix" for this concept but it's too general to be obvious what its role is and Sequence.prefix exists with a somewhat different meaning.

Anchor would then position us for future additions either in the stdlib or a library such as swift-system or a dedicated paths module: Future anchor additions · GitHub

2. Resource fork bools (Darwin-only). FilePath already correctly parses /..namedfork/rsrc suffixes: they affect equality/hashing and are hidden from ComponentView. They are not relative path components, but instead they are directly analogous to (and mutually exclusive with) trailing separators. Without any API, developers can't observe what FilePath parsed other than looking at the raw bytes or string representation. If we add isResourceFork, withResourceFork(), withoutResourceFork(), removeResourceFork(), mirroring the trailing separator pattern, then it's much easier for developers to work with or around them.

// With API:
if path.isResourceFork { path.removeResourceFork() }

// Without:
let s = String(decoding: path)
if s.hasSuffix("/..namedfork/rsrc") {
    path = FilePath(String(s.dropLast("/..namedfork/rsrc".count)))
}

3. Component.init(verbatim:) (Windows-only). Inside \\?\ paths, / is a legal character in component names, but the standard Component.init rejects it. A verbatim init would allow constructing such components directly. Without it:

// Construct a throwaway path to extract the component
var temp = FilePath(#"\\?\x\weird/name"#)
let c = temp.components.last  // extracts "weird/name" as one component
...

Feedback on any of the above is welcome. We'd like to get the core proposal into formal review soon.

8 Likes
11

I updated the S-E PR with the formal proposal. I believe this is ready to run.

Summary of changes in proposal version 1.0:

  • Added FilePath.Anchor as a fundamental decomposition primitive (and a replacement for swift-system's Root type), exposing isRooted, driveLetter, and isVerbatimComponent.
  • Added FilePath.Component.init(verbatim:) overloads for string and span (Windows-only).
  • Added Darwin-specific API for querying resource forks (akin to trailing separators).
  • Added FilePath.init(anchor:_:hasTrailingSeparator:) and expounded on path decomposition and reconstitution.
1 Like
12

Swift Testing cannot use Span directly or indirectly, so unsafe pointers are still necessary for us to adopt.

The above only applies on Darwin, where we don't potentially need a dependency on FilePath at this time. Ignore me!

1 Like
13

What does the right way to unsafèdly convert the Span into a C string look like?

SE-0529: Add FilePath to the Standard Library
14

For now, that would be

nullTerminatedCodeUnits.withUnsafeBufferPointer {
    let cStringPointer = $0.baseAddress
    return myCFunction(cStringPointer)
}

(strlen will return ($0.count-2)).

Note that this is not the final answer. We are interested in investigating linear buffers coded by their terminating element (0 for a C string,) as well as a family of interop-only non-escapable pointer wrappers that would move safe C-interop out of closures.

2 Likes
15

Given that a very common use case for FilePath is interacting with platform-specific C I/O API, we probably do want withCString or similar.

1 Like
16

We would like to stop adding C-interop API to specific types; do you think that is a bad goal?

SE-0529: Add FilePath to the Standard Library
17

I think it's a great goal for Swift code to always be able to pass around local file paths as FilePaths. However, most path manipulation is ultimately in service of creating a path that can be passed off to some OS API, and if there's no standard way for arbitrary code to actually do that without writing its own copy-into-a-buffer code, that seems to undermine that goal rather than support it. Presumably we will eventually have a batteries-included local I/O library that just works with FilePath, but we don't have that now, and even when we do, it won't wrap every OS API that takes a path, much less every third-party C API. I think not having a convenient way to get a const char * / LPCWSTR / whatever out of this API would be a mistake.

9 Likes
18

Hmm, I just noticed that this discussion is happening in the pitch thread, not the currently active review thread. I'm going to copy my post over there and close this thread; please continue any discussion over there.

4 Likes