Hi all,
Thank you all so much for the feedback on my original post on Foundation URL Improvements. I gathered so many excellent suggestions. After carefully reading through them and evaluating the pros and cons, I've revised the proposal on the URL improvements.
Here's a brief overview of the changes since the original draft:
- Changed
URL.currentDirectoryto a closure to ensure its value does not change within the closure body. - Added
@_alwaysEmitIntoClientto theStaticStringinitializer so the older OSs can also take advantage of it. - Introduced
appending(component:)andappending(queryItem:) - Introduced
URL'sExpressibleByStringLiteralconformance - Introduced
DirectoryHint(read more about it in the proposal) - Added
percentEncodedPathand friends as properties onURL - Changed
init(fileURLWithPath: String)andfunc appendingPathComponent(String) -> URLand friends to soft deprecation
I also want to let everyone know that I've heard the suggestions on some "big picture" improvements loud and clear, including:
- Introduce and utilize
FilePathin Foundation - Decouple
URLResourceValuesfromURL - Make
URLmore of a "model" type without dependencies on the file system - Update the parser for
URLandURLComponents - Split
URLintoFileURLandWebURL
All these ideas are very intriguing. I've carefully read each one of them and I will take them into consideration for future improvements. Unfortunately, this proposal has a smaller scope so we will have to hold off on these more ambitious items for now.
Thanks, and please let me know your thoughts on the revision.
URL Enhancements
- Proposal: FOU-XXXX
- Author(s): Charles Hu
- Status: Active review
Revision History
- v1 Initial version
-
v2 Addressed community feedbacks
- Changed
URL.currentDirectoryto a closure - Added
@_alwaysEmitIntoClientwhen available - Introduced
appending(component:)andappending(queryItem:) - (Re)introduced
URL'sExpressibleByStringLiteralconformance - Introduced
DirectoryHint - Added
percentEncodedPathand friends - Added clarification that we are still exploring the possibility of using
FilePath - Changed
init(fileURLWithPath: String)andfunc appendingPathComponent(String) -> URLand friends to soft deprecation
- Changed
Introduction
URL is one of the most common types used by Foundation for File IO as well as network-related tasks. The goal of this proposal is to improve the ergonomics of URL by introducing some convenience methods and renaming some verbose/confusing ones. Specifically, we proposed to:
- Introduce a new
StaticStringinitializer - Introduce a new
DirectoryHinttype - Refine the existing "file path" initializers
- Refine
appendingPathComponentand friends - Introduce common directories as static
URLproperties - Introduce
percentEncodedPathand friends
We will discuss each improvement in the following sections.
The New StaticString Initializer and ExpressibleByStringLiteral Conformance
URL represents two concepts: 1) internet address such as https://www.apple.com initialize by init?(string: String), and 2) file path such as /System/Library/ initialized by init(fileURLWithPath: String). We propose to introduce a new StaticString initializer that could initialize URL for both concepts. This initializer will require the StaticString to contain the URL scheme and initialize Strings with the file:/// scheme as file paths and everything else as web addresses. Because this initializer takes a StaticString known at compile-time, it considers "schemeless" Strings as a programmer error and asserts the existence of a scheme. It will also precondition any invalid web addresses, which mainly includes empty strings and strings with invalid characters. This means malformed web addresses will cause a runtime error for both debug builds AND release builds. Here are some examples:
| String Literal | URL Type | URL absoluteString
|
|---|---|---|
"https://www.apple.com" |
Web Address | https://www.apple.com |
"file:///var/mobile" |
File Path | file:///var/mobile |
"www.apple.com" |
- | Assertion failure |
/System/Library/Frameworks" |
- | Assertion failure |
"file://www.ap le.com" |
File Path | file:///current/dir/www.ap%20le.com |
"https://www.ap le.com" |
Web Address | Assertion failure |
@available(macOS 9999, iOS 9999, tvOS 9999, watchOS 9999, *)
extension URL {
/// Initialize an URL with a String literal. This initializer **requires** the
/// string literal contains a URL scheme such as `file://` or `https://` to correctly
/// interpret the URL. For web addresses (URLs that doesn't start with the `file` scheme),
/// it will `precondition` that the addresses are valid.
@_alwaysEmitIntoClient
init(_ string: StaticString)
}
This new "universal" StaticString initializer allows us to further add ExpressibleByStringLiteral conformance to URL:
@available(macOS 9999, iOS 9999, tvOS 9999, watchOS 9999, *)
extension URL : ExpressibleByStringLiteral {
typealias StringLiteralType = StaticString
/// Initialize an URL with a String literal. This initializer **requires** the
/// string literal contains a URL scheme such as `file://` or `https://` to correctly
/// interpret the URL. For web addresses (URLs that doesn't start with the `file` scheme),
/// it will `precondition` that the addresses are valid.
init(stringLiteral value: Self.StringLiteralType)
}
ExpressibleByStringLiteral conformance will further simplify various URL construction sites. For example:
// Move an item
try FileManager.default.moveItem(at: "file:///Dir/File", to: "file:///Dir2")
// URLSession tasks
let (data, _) = try await URLSession.shared.data(from: "https://example.com/file", delegate: nil)
NOTE: We are actively monitoring the pitch on Compile-Time Constant Values. When the const keyword support is ready, we will update the StaticString initializer to const String initializer:
@available(macOS 9999, iOS 9999, tvOS 9999, watchOS 9999, *)
extension URL {
/// Initialize an URL with a String literal. This initializer **requires** the
/// string literal contains a URL scheme such as `file://` or `https://` to correctly
/// interpret the URL. For web addresses (URLs that doesn't start with the `file` scheme),
/// it will `precondition` that the addresses are valid.
init(_ string: const String)
}
The New DirectoryHint Type
One of the lesser-known behavior of URL is that it will try to consult the filesystem (aka lstat) to determine whether the URL refers to a directory when you call the variants of methods and file path constructors that do not have the explicit isDirectory parameter. The list of methods include:
init(fileURLWithPath: String)init(fileURLWithPath: String, relativeTo: URL?)func appendPathComponent(String)func appendingPathComponent(String) -> URL
File URLs need to keep track of whether it refers to a directory for numerous reasons. For instance, if you are trying to construct a file URL relative to a base URL, the URL RFC dictates that the last component of the base URL should be stripped off if it's not a directory:
let base = URL(filePath: "/test/file", isDirectory: false)
let url = URL(filePath: "/directory/file2", relativeTo: base)
print(url.path) // prints "/test/directory/file2
However, since this behavior is not widely known and does have performance implications, we propose to change this behavior with the new methods and constructors that we are introducing in this proposal. We also propose to introduce a new type, URL.DirectoryHint, to allow the call site to decide whether a filesystem check should be performed.
extension URL {
@available(macOS 9999, iOS 9999, tvOS 9999, watchOS 9999, *)
public enum DirectoryHint {
case isDirectory // Specifies that the URL does reference a directory
case isNotDirectory // Specifies that the URL does NOT reference a directory
case checkFileSystem // Specifies that URL should check with the file system to determine whether it references a directory
case inferFromPath // Specifies that URL should infer whether is references a directory based on whether it has a trialing slash
}
}
The newly introduced methods will use DirectoryHint to specify how to determine whether a URL references a directory, replacing the existing boolean parameter isDirectory. We propose to set inferFromPath as the default value for this parameter. inferFromPath leaves the developers the responsibility to insert the trailing slash in the path to specify that it is a directory. Here are some examples:
| URL | .hasDirectoryPath |
|---|---|
URL(filePath: "/test/file") |
false |
URL(filePath: "/test/directory/") |
true |
URL(filePath: "/test/apple", directoryHint: .isDirectory) |
true |
URL(filePath: "/test/apple", directoryHint: .isNotDirectory) |
false |
URL(filePath: "/System/Library/Frameworks/Foundation.framework", directoryHint: .checkFileSystem) |
true |
Revisit the FilePath Initializers
The new StaticString initializer conflicts with the existing FilePath based initializer because neither has an argument label and FilePath conforms to ExpressibleByStringLiteral. The FilePath initializer is problematic because it could lead to unexpected results. For example, contrary to the assumption that most developer will have, URL("https://www.apple.com") actually initializes a file path (isFileURL returns true) because "https://www.apple.com" is interpreted as a FilePath due to ExpressibleByStringLiteral conformance. We therefore propose to rename the FilePath family initializers to init(filePath: FilePath) and deprecate the original versions.
Similarly, we propose to rename the existing family of file path initializers, init(fileURLWithPath path: String) to init(filePath: String), to slightly improve the ergonomics of these methods due to the more concise naming. This will, of course, conflict with the new FilePath initializer, so we will mark them as @_disfavoredOverload because they require additional transformations to and from FilePath.
extension URL {
@available(*, deprecated, renamed: "init?(filePath:)")
public init?(_ path: FilePath)
@available(*, deprecated, renamed: "init?(filePath:isDirectory)")
public init?(_ path: FilePath, isDirectory: Bool)
@available(macOS 9999, iOS 9999, tvOS 9999, watchOS 9999, *)
public init?(filePath: FilePath, directoryHint: DirectoryHint = .inferFromPath)
}
extension URL {
@available(macOS 9999, iOS 9999, tvOS 9999, watchOS 9999, *)
public init(filePath: String, directoryHint: DirectoryHint = .inferFromPath, relativeTo: URL? = nil)
@available(macOS, introduced: 10.9, deprecated: 100000.0, message: "Use init(filePath:directoryHint:relativeTo:) instead")
@available(iOS, introduced: 7.0, deprecated: 100000.0, message: "Use init(filePath:directoryHint:relativeTo:) instead")
@available(tvOS, introduced: 9.0, deprecated: 100000.0, message: "Use init(filePath:directoryHint:relativeTo:) instead")
@available(watchOS, introduced: 2.0, deprecated: 100000.0, message: "Use init(filePath:directoryHint:relativeTo:) instead")
public init(fileURLWithPath: String)
@available(macOS, introduced: 10.9, deprecated: 100000.0, message: "Use init(filePath:directoryHint:relativeTo:) instead")
@available(iOS, introduced: 7.0, deprecated: 100000.0, message: "Use init(filePath:directoryHint:relativeTo:) instead")
@available(tvOS, introduced: 9.0, deprecated: 100000.0, message: "Use init(filePath:directoryHint:relativeTo:) instead")
@available(watchOS, introduced: 2.0, deprecated: 100000.0, message: "Use init(filePath:directoryHint:relativeTo:) instead")
public init(fileURLWithPath: String, isDirectory: Bool)
@available(macOS, introduced: 10.9, deprecated: 100000.0, message: "Use init(filePath:directoryHint:relativeTo:) instead")
@available(iOS, introduced: 7.0, deprecated: 100000.0, message: "Use init(filePath:directoryHint:relativeTo:) instead")
@available(tvOS, introduced: 9.0, deprecated: 100000.0, message: "Use init(filePath:directoryHint:relativeTo:) instead")
@available(watchOS, introduced: 2.0, deprecated: 100000.0, message: "Use init(filePath:directoryHint:relativeTo:) instead")
public init(fileURLWithPath: String, relativeTo: URL?)
@available(macOS, introduced: 10.9, deprecated: 100000.0, message: "Use init(filePath:directoryHint:relativeTo:) instead")
@available(iOS, introduced: 7.0, deprecated: 100000.0, message: "Use init(filePath:directoryHint:relativeTo:) instead")
@available(tvOS, introduced: 9.0, deprecated: 100000.0, message: "Use init(filePath:directoryHint:relativeTo:) instead")
@available(watchOS, introduced: 2.0, deprecated: 100000.0, message: "Use init(filePath:directoryHint:relativeTo:) instead")
public init(fileURLWithPath: String, isDirectory: Bool, relativeTo: URL?)
}
We propose to "soft deprecate" the old constructors init(fileURLWithPath) and friends because they are widely used and the new constructor init(filePath:directoryHint:relativeTo:) will have slightly different behavior due to DirectoryHint. This means we will mark the deprecation version to 100000.0 instead of a concrete version.
Revisit appendingPathComponent and Friends
We also propose to rename the (painfully) long, but popular appendingPathComponent() and friends to .appending(path:), as well as adding a few additions:
-
.appending(paths:): appends a collection of path components. -
.appending(component:): appends a single component. It percent-encodes the String before appending. -
.appending(queryItem:): appends aURLQueryItemto the URL.
Similarly to the old constructors, we propose to soft deprecate the old methods since they are also widely used.
extension URL {
@available(macOS 9999, iOS 9999, tvOS 9999, watchOS 9999, *)
public mutating func append(path: String, directoryHint: DirectoryHint = .inferFromPath)
@available(macOS 9999, iOS 9999, tvOS 9999, watchOS 9999, *)
public func appending(path: String, directoryHint: DirectoryHint = .inferFromPath) -> URL
@available(macOS 9999, iOS 9999, tvOS 9999, watchOS 9999, *)
public mutating func append<C: Sequence>(paths: C, directoryHint: DirectoryHint = .inferFromPath) where C.Element : StringProtocol
@available(macOS 9999, iOS 9999, tvOS 9999, watchOS 9999, *)
public func appending<C: Sequence>(paths: C, directoryHint: DirectoryHint = .inferFromPath) where C.Element : StringProtocol -> URL
@available(macOS 9999, iOS 9999, tvOS 9999, watchOS 9999, *)
public mutating func append(component: String, directoryHint: DirectoryHint = .inferFromPath)
@available(macOS 9999, iOS 9999, tvOS 9999, watchOS 9999, *)
public func appending(component: String, directoryHint: DirectoryHint = .inferFromPath)
@available(macOS 9999, iOS 9999, tvOS 9999, watchOS 9999, *)
public mutating func append(queryItem: URLQueryItem)
@available(macOS 9999, iOS 9999, tvOS 9999, watchOS 9999, *)
public func appending(queryItem: URLQueryItem)
@available(macOS, introduced: 10.9, deprecated: 100000.0, message: "Use append(path:directoryHint:) instead")
@available(iOS, introduced: 7.0, deprecated: 100000.0, message: "Use append(path:directoryHint:) instead")
@available(tvOS, introduced: 9.0, deprecated: 100000.0, message: "Use append(path:directoryHint:) instead")
@available(watchOS, introduced: 2.0, deprecated: 100000.0, message: "Use append(path:directoryHint:) instead")
public mutating func appendPathComponent(String)
@available(macOS, introduced: 10.9, deprecated: 100000.0, message: "Use append(path:directoryHint:) instead")
@available(iOS, introduced: 7.0, deprecated: 100000.0, message: "Use append(path:directoryHint:) instead")
@available(tvOS, introduced: 9.0, deprecated: 100000.0, message: "Use append(path:directoryHint:) instead")
@available(watchOS, introduced: 2.0, deprecated: 100000.0, message: "Use append(path:directoryHint:) instead")
public mutating func appendPathComponent(String, isDirectory: Bool)
@available(macOS, introduced: 10.9, deprecated: 100000.0, message: "Use appending(path:directoryHint:) instead")
@available(iOS, introduced: 7.0, deprecated: 100000.0, message: "Use appending(path:directoryHint:) instead")
@available(tvOS, introduced: 9.0, deprecated: 100000.0, message: "Use appending(path:directoryHint:) instead")
@available(watchOS, introduced: 2.0, deprecated: 100000.0, message: "Use appending(path:directoryHint:) instead")
public func appendingPathComponent(String) -> URL
@available(macOS, introduced: 10.9, deprecated: 100000.0, message: "Use appending(path:directoryHint:) instead")
@available(iOS, introduced: 7.0, deprecated: 100000.0, message: "Use appending(path:directoryHint:) instead")
@available(tvOS, introduced: 9.0, deprecated: 100000.0, message: "Use appending(path:directoryHint:) instead")
@available(watchOS, introduced: 2.0, deprecated: 100000.0, message: "Use appending(path:directoryHint:) instead")
public func appendingPathComponent(String, isDirectory: Bool) -> URL
}
The addition of .appending(paths:) will make appending multiple path components to URL simpler. For example:
let baseURL: URL = ...
let id = UUID().uuidString
// Before
let photoURL = baseURL
.appendingPathComponent("Photos")
.appendingPathComponent("\(id.first!)")
.appendingPathComponent(id)
// After
let photoURL = baseURL.appending(paths: ["Photos", "\(id.first!)", id])
Common Directories as Static URL Properties
We propose to add all "get URL" style methods from FileManager to URL as static methods. This allows the call site to use Swift's powerful static member lookup to get the URLs to predefined directories instead of always needing to spell out FileManager.default. We also propose to add a few more static directory URLs that correspond to FileManager.SearchPathDirectory. Note that these are not stored properties and will not run in O(1). We will add documentations to specify that.
@available(macOS 9999, iOS 9999, tvOS 9999, watchOS 9999, *)
extension URL {
/// The working directory of the current process.
/// This properties has been implemented as a closure
/// to ensure the value of `currentDirectory` does not change
/// within the `body`
/// Calling this property will issue a `getcwd` syscall.
public static func withCurrentDirectory<R>(_ body: (URL) -> throws R) rethrows -> R
/// The home directory for the current user (~/).
/// Complexity: O(1)
public static var homeDirectory: URL
/// The temporary directory for the current user.
/// Complexity: O(1)
public static var temporaryDirectory: URL
/// Discardable cache files directory for the
/// current user. (~/Library/Caches).
/// Complexity: O(n) where n is the number of significant directories
/// specified by `FileManager.SearchPathDirectory`
public static var cachesDirectory: URL
/// Supported applications (/Applications).
/// Complexity: O(n) where n is the number of significant directories
/// specified by `FileManager.SearchPathDirectory`
public static var applicationDirectory: URL
/// Various user-visible documentation, support, and configuration
/// files for the current user (~/Library).
/// Complexity: O(n) where n is the number of significant directories
/// specified by `FileManager.SearchPathDirectory`
public static var libraryDirectory: URL
/// User home directories (/Users).
/// Complexity: O(n) where n is the number of significant directories
/// specified by `FileManager.SearchPathDirectory`
public static var userDirectory: URL
/// Documents directory for the current user (~/Documents)
/// Complexity: O(n) where n is the number of significant directories
/// specified by `FileManager.SearchPathDirectory`
public static var documentsDirectory: URL
/// Desktop directory for the current user (~/Desktop)
/// Complexity: O(n) where n is the number of significant directories
/// specified by `FileManager.SearchPathDirectory`
public static var desktopDirectory: URL
/// Application support files for the current
/// user (~/Library/Application Support)
/// Complexity: O(n) where n is the number of significant directories
/// specified by `FileManager.SearchPathDirectory`
public static var applicationSupportDirectory: URL
/// Downloads directory for the current user (~/Downloads)
/// Complexity: O(n) where n is the number of significant directories
/// specified by `FileManager.SearchPathDirectory`
public static var downloadsDirectory: URL
/// Movies directory for the current user (~/Movies)
/// Complexity: O(n) where n is the number of significant directories
/// specified by `FileManager.SearchPathDirectory`
public static var moviesDirectory: URL
/// Music directory for the current user (~/Music)
/// Complexity: O(n) where n is the number of significant directories
/// specified by `FileManager.SearchPathDirectory`
public static var musicDirectory: URL
/// Pictures directory for the current user (~/Pictures)
/// Complexity: O(n) where n is the number of significant directories
/// specified by `FileManager.SearchPathDirectory`
public static var picturesDirectory: URL
/// The user’s Public sharing directory (~/Public)
/// Complexity: O(n) where n is the number of significant directories
/// specified by `FileManager.SearchPathDirectory`
public static var sharedPublicDirectory: URL
/// Trash directory for the current user (~/.Trash)
/// Complexity: O(n) where n is the number of significant directories
/// specified by `FileManager.SearchPathDirectory`
public static var trashDirectory: URL
/// Returns the home directory for the specified user.
public static func homeDirectory(forUser user: String) -> URL?
/// Locates and optionally creates the specified common directory in a domain.
///
/// - parameter directory: The search path directory. The supported values are
/// described in FileManager.SearchPathDirectory.
/// - parameter domain: The file system domain to search. The value for this
/// parameter is one of the constants described in
/// `FileManager.SearchPathDomainMask`. You should specify only one domain for
/// your search and you may not specify the allDomainsMask constant for
/// this parameter.
/// - parameter url: The file URL used to determine the location of the returned
/// URL. Only the volume of this parameter is used. This parameter is ignored
/// unless the directory parameter contains the value
/// `FileManager.SearchPathDirectory.itemReplacementDirectory` and the domain
/// parameter contains the value `userDomainMask`.
/// - parameter shouldCreate: Whether to create the directory if it does not
/// already exist.
public init(
for directory: FileManager.SearchPathDirectory,
in domain: FileManager.SearchPathDomainMask,
appropriateFor url: URL?,
create shouldCreate: Bool) throws
}
Now code for common file tasks such as writing files to the Downloads directory will be much cleaner:
let secretData = ...
// Before
let downloadDirectoryURL = FileManager.default.urls(for: .downloadsDirectory, in: .userDomainMask)[0]
let targetFolderURL = downloadDirectoryURL.appendingPathComponent("TopSecrets")
try FileManager.default.createDirectory(at: targetFolderURL, withIntermediateDirectories: true, attributes: nil)
let secretDataURL = targetFolderURL.appendingPathComponent("Secret.file")
try secretData.write(to: secretDataURL)
// After
let targetFolderURL: URL = .downloadsDirectory.appending(path: "TopSecrets")
try FileManager.default.createDirectory(at: targetFolderURL, withIntermediateDirectories: true, attributes: nil)
try secretData.write(to: targetFolderURL.appending(path: "Secret.file"))
percentEncodedPath and Friends
URL currently does not offer percentEncodedXX properties such as percentEncodedPath and percentEncodedHost, forcing the developers to fallback to URLComponents. We propose to add these missing properties:
extension URL {
@available(macOS 9999, iOS 9999, tvOS 9999, watchOS 9999, *)
public var percentEncodedFragment: String? { get }
@available(macOS 9999, iOS 9999, tvOS 9999, watchOS 9999, *)
public var percentEncodedHost: String? { get }
@available(macOS 9999, iOS 9999, tvOS 9999, watchOS 9999, *)
public var percentEncodedPassword: String? { get }
@available(macOS 9999, iOS 9999, tvOS 9999, watchOS 9999, *)
public var percentEncodedPath: String { get }
@available(macOS 9999, iOS 9999, tvOS 9999, watchOS 9999, *)
public var percentEncodedQuery: String? { get }
@available(macOS 9999, iOS 9999, tvOS 9999, watchOS 9999, *)
public var percentEncodedUser: String? { get }
}
Impact on Existing Code
Very minimal. Most changes introduced are additive with a few additional method renames.
Alternatives Considered
Use FilePath for File IO Tasks
Swift System introduced FilePath to represent a location on the file system, which functionally overlaps with URL for File IO tasks. Although FilePath is more lightweight and has better cross-platform support, replacing URL with FilePath for File IO tasks in Foundation could lead to two major issues:
- Bigger impact on existing code. Callers of
FileManagerwill need to be updated to useFilePath. - URL supports resource value caching, a feature that many systems depends on.
FilePathwill need to support fetching and cachingURLResourceValuesbefore it could replaceURLfor file tasks.
Please note that although we are not considering FilePath for this proposal, we are not "shutting down" the possibility of using FilePath in Foundation. We will explore Foundation's usage for FilePath in the future.
Uniformed append Method with AppendingType
Since we have several append methods, we could also introduce a new type, tentatively called URL.AppendingType, that allows us to have a more "uniformed" append experience with one append method and maybe even a custom operator!
extension URL {
@available(macOS 9999, iOS 9999, tvOS 9999, watchOS 9999, *)
public enum AppendingType {
case path(String) // appends the String as a path, treating `/` as directory separators
case component(String) // appends the **percent encoded** String as a single component
case components([String]) // appends an array of **percent encoded** Strings
case pathExtension(String) // appends the String as a path extension
case queryItem(URLQueryItem) // appends an URLQueryItem
}
@available(macOS 9999, iOS 9999, tvOS 9999, watchOS 9999, *)
public func appending(_ type: AppendingType, hint: DirectoryHint = .inferFromPath)
@available(macOS 9999, iOS 9999, tvOS 9999, watchOS 9999, *)
public static func +(lhs: URL, rhs: AppendingType) -> URL
}
Here are some examples:
let base = URL("file:///System/Library/Frameworks")
let infoURL = base + .component("Foundation") + .pathExtension(".framework") + .path("Resources/Info.plist")
// Alteratively
let infoURL = base.appending(.components(["Foundation.framework", "Resources", "Info.plist"]))
We decided to not move forward with this approach for several reasons:
- The downside of having to maintain a brand new type
AppendingTypeout weights the value it provides. - We want to nudge developers into explicitly providing
DirectoryHint, but the+operator does not give developers that ability.
, really - there's no substitute for tests). That is something that only build-time checking can provide. This thing gives you a false sense of security; it's just as misguided as those who proclaim that all force-unwraps are evil or "code smells".