Add JSONDecoder.URLDecodingStrategy in JSONDecoder

Introduction

Add an enum JSONDecoder.URLDecodingStrategy in JSONDecoder to customize decoding URL.

Motivation

Currently JSONDecoder in Foundation framework implements URL decoding like below

 } else if type == URL.self {
            guard let urlString = try self.unbox(value, as: String.self) else {
                return nil
            }

            guard let url = URL(string: urlString) else {
                throw DecodingError.dataCorrupted(DecodingError.Context(codingPath: self.codingPath,
                                                                        debugDescription: "Invalid URL string."))
            }
            return url
}

JSONDecoder decodes URL from string.
Sometimes, We need preprocessing to init URL.
There is some example

/// case1 : When we need trimming white spaces before initialize URL
let trimmedString = stringValue.trimmingCharacters(in: .whitespacesAndNewlines)
let url1 = URL.init(string: trimmedString)

/// case2 : When we need combine domainURL and path
let domainURLString = "https://forums.swift.org"
let pitchPath = "/c/evolution/pitches"
let url2 = URL.init(string: domainURLString + pitchPath)

Unfortunatly, Current URL Decoding implementation can't decode that cases.

Proposed solution

So, I suggest to JSONDecoder to add enum JSONDecoder.URLDecodingStrategy to JSONDecoder

Fortunatly, func decode<T>(T.Type, from data: Data) -> T method converts json data to json object.

// MARK: - Decoding Values
    /// Decodes a top-level value of the given type from the given JSON representation.
    ///
    /// - parameter type: The type of the value to decode.
    /// - parameter data: The data to decode from.
    /// - returns: A value of the requested type.
    /// - throws: `DecodingError.dataCorrupted` if values requested from the payload are corrupted, or if the given data is not valid JSON.
    /// - throws: An error if any value throws an error during decoding.
    open func decode<T : Decodable>(_ type: T.Type, from data: Data) throws -> T {
        let topLevel: Any
        do {
            topLevel = try JSONSerialization.jsonObject(with: data, options: .allowFragments)
        } catch {
            throw DecodingError.dataCorrupted(DecodingError.Context(codingPath: [], debugDescription: "The given data was not valid JSON.", underlyingError: error))
        }

        let decoder = _JSONDecoder(referencing: topLevel, options: self.options)
        guard let value = try decoder.unbox(topLevel, as: type) else {
            throw DecodingError.valueNotFound(type, DecodingError.Context(codingPath: [], debugDescription: "The given data did not contain a top-level value."))
        }

        return value
    }

Detailed design

class JSONDecoder {
    enum URLDecodingStrategy {
         case fromString // decode URL from string
         case custom((Decoder) throws -> URL)
    }
    var urlDecodingStrategy: URLDecodingStrategy = .fromString // default value
}

let jsonDecoder = JSONDecoder()
jsonDecoder.urlDecodingStrategy = .custom { decoder -> URL in
    guard let codingPath = decoder.codingPath.last else {
        throw DecodingError.dataCorrupted(DecodingError.Context.init(codingPath: decoder.codingPath, debugDescription: "coding key is not exist"))
    }

// trimming whitespaces and new lines before initialize URL
    let urlString = try decoder.singleValueContainer().decode(String.self).trimmingCharacters(in: .whitespacesAndNewlines)

    if let url = URL.init(string: urlString) {
        return url
    } else {
        throw SomeError
    }
}

If JSONDecoder provides URLDecodingStrategy, we can decode above two cases and more cases.

let jsonDecoder = JSONDecoder()
jsonDecoder.urlDecodingStrategy = .custom {
}

Source compatibility

It is not affect to previous swift version.

1 Like

I’m not sure that I understand the use case. We have e.g. a DateDecodingStrategy because JSON has no native date type and there are multiple equally valid ways to represent a date. However, there’s only one way to represent a URL — if it’s not a string that conforms to RFC 1738, then it isn’t really a URL.

Is there a real-world scenario you encountered where this pitch would be helpful? Even if there was, I don’t think handling this case falls within the scope of JSONDecoder since it’s not specific to the JSON format. You’re not really decoding an URL from JSON, you’re decoding some other object and constructing a URL out of it. Instead of extending JSONDecoder, what about using a custom property wrapper?

I fully understand the motivation - many JSON responses contain relative URLs, not absolute. Like when you use URL(string: "index.html", relativeTo: rootURL). But in such case I would rather suggest adding a baseURL property on the JSONDecoder rather than adding a decoding strategy...

I'll just note that there's a JIRA ticket tracking JSONEncoder's URL encoding format — specifically, adding an option to make it round-trip losslessly (i.e. falling back to the default URL implementation). [There is an internal Radar tracking the ticket too.] If that ever gets added, a .custom strategy might be reasonable too.

1 Like

I understand the motivation, but as a maintainer of a custom coder (specifically for coding and decoding XML) I can't endorse this (or even any other current) coding strategy API.

I had some experience using coders extensively and had a closer look at how coders are implemented while working on feature requests to XMLCoder. While coding strategies were an acceptable solution at the time when the Codable API was introduced, sometimes they aren't a good fit when your values are coded differently, but processed by the same coder. Good luck with coding dates, that on some levels of nesting are expected to be ISO 8601 dates, while on other levels for other entities are expected to be epoch dates, for example.

Same for URLs, you could easily expect different types or even properties of the same type to have different base URLs. And even the API itself, with closures wrapped in enum values assigned on a property, is not very discoverable and is not widely adopted in Swift in general as a pattern. In cases where different types need different coding strategies for dates (and other value types) within the same coder, I still think this requires some amount of helper code to make it work properly and avoid a giant switch statement in strategy closures.

Not considering all that, the problem is that JSONDecoder and JSONEncoder are Foundation APIs, which are owned by Apple and don't go through the evolution process, as far as I'm aware. And this exposes the bigger problem with strategies: they have to be implemented separately for every coder. So what you're proposing would be quite useful for XMLCoder for example, but as a I maintainer I would have to implement it separately. Same for the CBOR coder, ObjectEncoder, CodableFirebase and many others.

It doesn't have to be that way, especially after Swift 5.1 which introduced property wrappers. Have a look at BetterCodable or ResilientCoding libraries which give some great examples of how the Codable API could be improved. It's super nice that all these wrappers are coder-agnostic. XMLCoder, and virtually any other coder where these transformations are applicable, gain them basically for free. If a user needs a new strategy, they need neither XMLCoder nor Foundation to do anything special.

The summary is, I think this should be either proposed as a property wrapper for inclusion in the standard library, probably as a part of a bigger proposal to deprecate strategy enums and replace with property wrappers. Or submitted as a PR to BetterCodable or one of the other open source libraries out there that maintain Codable property wrappers.

4 Likes

Oh, I didn't think about property wrapper:(
Thank you for advice.
I think property wrapper can cover that cases too!.
I will try use property wrapper.
Thank you.

@Max_Desiatov
Can you tell me how to contribute to Foundation framework If I want to contribute to Foundation framework?

The source code is available, but I'm not sure that PRs with significant changes to the API would be merged without a corresponding pitch thread like this one and gathering some amount of feedback. Maybe a proposal that passes a formal review would work after all. Having another look at Swift Evolution, the original Codable addition was one of those rare proposals that did change Foundation.

1 Like

Codable was added to the standard library. Foundation is an Apple-owned library, and its API is not subject to Swift Evolution except where it touches on the standard library or the Swift language itself. The open-source swift-corelibs-foundation is a reimplementation of Foundation for non-Apple platforms but is constrained to expose the same API, and the Swift overlay for Darwin Foundation (still visible for now in the main Swift repository) has been returned to the control of the Foundation team.

2 Likes