URL Literals


(Erica Sadun) #1
let x = #imageLiteral(resourceName:"nothere.jpg")
print(x)

This compiles. It crashes at runtime. I don't see why URLs should be any different.

-- E

···

On Dec 19, 2016, at 11:53 AM, David Sweeris via swift-evolution <swift-evolution@swift.org> wrote:

On Dec 19, 2016, at 1:26 AM, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> wrote:

URLs are unlikely to be something that can be validated by regex. See, for instance, this discussion: <https://webkit.org/blog/7086/url-parsing-in-webkit/>. The full spec is here: <https://url.spec.whatwg.org <https://url.spec.whatwg.org/>>. If Swift were to implement parsing of URLs at the level of the compiler or core library, I'd expect it to be the full spec, as we do with Unicode.

On Mon, Dec 19, 2016 at 2:26 AM, Benjamin Spratling via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
Howdy,
  Yes, I was also intrigued by the “Regex” validation mentioned in another post. It could offer a convenient way to get some literals support in without the headaches associated with the constexpr C++ approach.
  I’m curious, though, how many types can we image in can be validated by this method? If it really is just URL’s, then I’d actually lean towards making this a compiler magic feature.

  Someone else mentioned fetching the URL’s for a preview. Given that we might be coding “deletes” in URL’s (yes, I recently met a backend developer who coded a delete as a GET), I really highly suggest we not ping people’s API’s artificially. At least we shouldn’t for non-file-scheme URLs. IMHO, verifying that a service is active isn’t really the Swift compiler’s job. It might happen as part of coordinated run-time tests, which sometimes have to be balanced to keep test data correct, something the IDE wouldn’t know how to enforce correctly.
-Ben

On Dec 19, 2016, at 1:41 AM, David Sweeris via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Dec 17, 2016, at 1:12 PM, Micah Hainline via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

I'd love a fleshed out elegant example for URL that shows what a complete implementation of that special init method would look like.

Sorry this took so long… the weekend kinda got away from me.

Anyway, I was thinking something like this (which has been very simplified on account of my regexing being sub-sketchy, and me not knowing exactly what’s valid in an URL anyway):
#literalpatterns += (name: “URLLiteralType”, components: (name: url, type: StringLiteralType, pattern: “(http|https)://(www.)?[a-z|A-Z|0-9]+.(com|org|net)(/[a-z|A-Z|0-9]+)*(/[a-z|A-Z|0-9]+.[a-z|A-Z|0-9]+)?”), protocol: ExpressibleByURLLiteral)
This would let the compiler know pretty much everything it needs to know… that the “new” type is called “URLLiteralType", that it starts out life as young StringLiteralType with a bright future in the computer industry, that in order to succeed it has to match a given pattern, and what protocol a type has to conform to in order to use an URLLiteral. In practice, the compiler would synthesize a struct containing the specified members and validate the literal with the specified pattern before making an “instance” of it (since we’re talking about literals and compile-time code here, I’m pretty sure that “instance" the wrong terminology… pardon my ignorance)
struct URLLiteralType: {
    let url: StringLiteralType
}
A tuple would be better, IMHO, but according to the playground, single-element tuples can’t have element labels. As for the implementation of the init function:
init(urlLiteral value: URLLiteralType) {
    let urlString = value.url
    //Do whatever URL is doing now, except there’s no need to check for errors since the compiler pre-validated it for us
}

If it’d be more useful, the pattern could be split into multiple pieces:
#literalpatterns += (name: “URLLiteralType”,
                     components: ((name: “`protocol`", type: StringLiteralType, pattern: “(http|https)”),
                                  (name: _, type: StringLiteralType, pattern: “://”),
                                  (name: “domain", type: StringLiteralType, pattern: “(www.)?[a-z|A-Z|0-9]+.(com|org|net)”),
                                  (name: “path”, type: StringLiteralType, pattern: "(/[a-z|A-Z|0-9]+)*(/[a-z|A-Z|0-9]+.[a-z|A-Z|0-9]+)?”))
                     protocol: ExpressibleByURLLiteral)
This would result in URLLiteralType looking like this:
struct URLLiteralType: {
    let `protocol`: StringLiteralType
    let domain: StringLiteralType
    let path: StringLiteralType
}
And in the init would start out like this:
init(urlLiteral value: URLLiteralType) {
    let protocolType = value.protocol
    let domain = value.domain
    let path = value.path
    //Do whatever with the components
}

The “base” types of literals like Int or String that don’t refine pre-existing literal types would still need a bit of compiler magic (or at least a different mechanism for becoming actual types), but as long as a type doesn’t take advantage of reference semantics in its stored properties or something, I *think* pretty much any data type could become “literalizeable” with something like this. Oh, and there’s nothing particularly magical about regular expressions as far as this idea is concerned; they’re just usually the first thing that comes to mind when I think of pattern matching in a string.

I know this looks like a lot of code, but the scary-looking parts with the regex stuff only has to be written once for each “type” of literal… types that want to be expressible by such a literal just have to write an init function.

It doesn’t have to be regex per se… instead of
#literalpatterns += (name: “URLLiteralType”, components: (name: url, type: StringLiteralType, pattern: “(http|https)://(www.)?[a-z|A-Z|0-9]+.(com|org|net)(/[a-z|A-Z|0-9]+)*(/[a-z|A-Z|0-9]+.[a-z|A-Z|0-9]+)?”), protocol: ExpressibleByURLLiteral)
I probably should’ve written something more like:
#literalpatterns += (name: “URLLiteralType”, components: (name: url, type: StringLiteralType, matching: Regex(“(http|https)://(www.)?[a-z|A-Z|0-9]+.(com|org|net)(/[a-z|A-Z|0-9]+)*(/[a-z|A-Z|0-9]+.[a-z|A-Z|0-9]+)?”)), protocol: ExpressibleByURLLiteral)
where the `matching` argument can be anything that can (“@purely”-ly) use some specified mechanism (I’d vote for the ~= operator) with a literal to test whether it matches. Also, there is no existing `Regex` struct/class/mechanism in Swift, unless you count `NSRegularExpression`. I didn’t want to use that for a couple reasons… 1) I don’t think it’s part of the stdlib, and 2) it doesn’t have a non-failable init that just takes a string, so using it unmodified would kinda put us in a “it’s turtles all the way down” kind of situation. What I’d started doing was to look for the existing mechanism for specifying literals in the compiler so I could use the existing name for it (somehow I doubt there’s a actually an array of patterns called “literalpatterns" in the compiler) and copy the existing methods for specifying a valid literal. After being unsuccessful for some amount of time, I decided I was getting too tired and made up what I sent last night.

The more I think about it, the more I’m starting to be of the opinion that we really ought to have two mechanisms here… One for specifying what constitutes a “base” literal (like `43`, `[“foo”, “bar”]`, or `true`), and one for types that merely need to perform some sort of validation on existing “base” literals. The first mechanism probably should be fairly arcane and involved, because you’d essentially be able to create new syntaxes, which should be kinda scary and hard to understand because it’s most certainly not an area beginners should be in. The second mechanism — something like that `ExpressibleByValidatedStringLiteral` idea — isn’t nearly as complicated. In the case of URLs, I’d vote the second approach. We only really need two extra features to implement it (“@constexpr” and the compiler being able to use the REPL to evaluate @costexpr statements), and both of them have more uses other than just getting a few more compile-time checks or allowing for more inits to be non-failable. With both of those in place, getting an url “literal” becomes just this:
protocol ExpressibleByValidatedStringLiteral {
    init?(stringLiteral value: StringLiteralType)
}
struct URL : ExpressibleByValidatedStringLiteral {
    //Stuff
    //...
    init?(stringLiteral value: StringLiteralType) {
        //Perform validation here; return nil if it fails
    }
    //...
    //Things
}
var lru: URL = "foo" //Compiler throws this to the init? function, it returns nil, the compiler raises a syntax error
var url: URL = "http://www.some.valid.url.com <http://www.some.valid.url.com/>" //Compiler throws this to the init? function, it returns an optional URL, the compiler unwraps it and does the assignment

I still very much want a way to define custom literals (precisely because it’d let me make new syntaxes), but I’m starting to think that something like the second, disappointingly easy idea, is probably the way to go in this case.

- Dave Sweeris


(David Sweeris) #2

They shouldn’t be. The print function can already handle things that aren’t `CustomStringConvertible`:
struct Foo {}
let x = Foo()
Foo.self is CustomStringConvertible //says `false`
print(x) //says `"Foo()\n"`
so, IMHO, that’s a bug.

- Dave Sweeris

···

On Dec 19, 2016, at 11:21 AM, Erica Sadun via swift-evolution <swift-evolution@swift.org> wrote:

let x = #imageLiteral(resourceName:"nothere.jpg")
print(x)

This compiles. It crashes at runtime. I don't see why URLs should be any different.

— E


(David Sweeris) #3

Oh, hey, wait a second… I didn’t catch the meaning of the resource name, and thought you meant it was crashing on the print statement. I still think it’s a bug, because the compiler should be able to check if the resource exists.

- Dave Sweeris

···

On Dec 19, 2016, at 11:36 AM, David Sweeris via swift-evolution <swift-evolution@swift.org> wrote:

On Dec 19, 2016, at 11:21 AM, Erica Sadun via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

let x = #imageLiteral(resourceName:"nothere.jpg")
print(x)

This compiles. It crashes at runtime. I don't see why URLs should be any different.

— E

They shouldn’t be. The print function can already handle things that aren’t `CustomStringConvertible`:
struct Foo {}
let x = Foo()
Foo.self is CustomStringConvertible //says `false`
print(x) //says `"Foo()\n"`
so, IMHO, that’s a bug.


(Xiaodi Wu) #4
let x = #imageLiteral(resourceName:"nothere.jpg")
print(x)

This compiles. It crashes at runtime. I don't see why URLs should be any
different.

— E

They shouldn’t be. The print function can already handle things that
aren’t `CustomStringConvertible`:

struct Foo {}
let x = Foo()
Foo.self is CustomStringConvertible //says `false`
print(x) //says `"Foo()\n"`

so, IMHO, that’s a bug.

Oh, hey, wait a second… I didn’t catch the meaning of the resource name,
and thought you meant it was crashing on the print statement. I still think
it’s a bug, because the compiler should be able to check if the resource
exists.

I think the point here is that what exists at compile time may not exist at
runtime and vice versa, so while a warning might be elegant, it's not
helpful for the compiler to refuse to proceed on the basis that the image
does not yet exist. At the end of the day, an image literal hardcodes the
path to an image, not the image itself. Whether it ought to is another
kettle of fish.

···

On Mon, Dec 19, 2016 at 1:44 PM, David Sweeris via swift-evolution < swift-evolution@swift.org> wrote:

On Dec 19, 2016, at 11:36 AM, David Sweeris via swift-evolution < > swift-evolution@swift.org> wrote:
On Dec 19, 2016, at 11:21 AM, Erica Sadun via swift-evolution < > swift-evolution@swift.org> wrote:

- Dave Sweeris

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(David Sweeris) #5

Ah, ok, I didn’t realize that image literals weren’t actually created until runtime... In that case, I retract my statement that they shouldn't be any different, because they while they both may be called “literals", they use that word to mean to two quite different things.

- Dave Sweeris

···

On Dec 19, 2016, at 11:48 AM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Mon, Dec 19, 2016 at 1:44 PM, David Sweeris via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Dec 19, 2016, at 11:36 AM, David Sweeris via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Dec 19, 2016, at 11:21 AM, Erica Sadun via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

let x = #imageLiteral(resourceName:"nothere.jpg")
print(x)

This compiles. It crashes at runtime. I don't see why URLs should be any different.

— E

They shouldn’t be. The print function can already handle things that aren’t `CustomStringConvertible`:
struct Foo {}
let x = Foo()
Foo.self is CustomStringConvertible //says `false`
print(x) //says `"Foo()\n"`
so, IMHO, that’s a bug.

Oh, hey, wait a second… I didn’t catch the meaning of the resource name, and thought you meant it was crashing on the print statement. I still think it’s a bug, because the compiler should be able to check if the resource exists.

I think the point here is that what exists at compile time may not exist at runtime and vice versa, so while a warning might be elegant, it's not helpful for the compiler to refuse to proceed on the basis that the image does not yet exist. At the end of the day, an image literal hardcodes the path to an image, not the image itself. Whether it ought to is another kettle of fish.


(David Sweeris) #6

I want to elaborate on this a bit… While both of my ideas might (depending on what all can be marked @constexpr) allow someone to check whether an URL actually points to something at compile-time, in all my examples, I was only intending to check that the URL in question was syntactically valid, not whether accessing it would result in some sort of “not found” error. You’d still have to check at runtime that the server is up & responding, that the file is still where you think it is, etc. However, if someone wants to expressly make it a syntax error to compile their code while “www.my.hardcoded.site.com” is down for maintenance, their internet connection is spotty, or whatever... Well… I can’t fathom why that’d be a good idea, but IMHO the pros of being able to get compile-time validation of literals far out-ways the downside of giving people one more opportunity to write some “questionable” code.

Actually, I just thought of one kinda-sorta-not really potentially valid use… In a group environment, it could be used to check that a SVC server is up, as a guard against trying to work with out-dated code. Personally, I think that’s the wrong solution to that particular “problem" (to the extent that it is one), but it is a solution (sorta), and I don’t think I can confidently say that it’s impossible for something like that to be the right decision for someone else. (Yes, it’s a real stretch… I know… the point is that in the amount of time it took me to write one paragraph, I went from “this would never be a good idea”, to “eh… this would probably never be a good idea”. Given more time to think on the possibilities, perhaps someone will think of something really cool and useful to do with it.)

- Dave Sweeris

···

On Dec 19, 2016, at 11:55 AM, David Sweeris via swift-evolution <swift-evolution@swift.org> wrote:

On Dec 19, 2016, at 11:48 AM, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> wrote:

On Mon, Dec 19, 2016 at 1:44 PM, David Sweeris via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Dec 19, 2016, at 11:36 AM, David Sweeris via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Dec 19, 2016, at 11:21 AM, Erica Sadun via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

let x = #imageLiteral(resourceName:"nothere.jpg")
print(x)

This compiles. It crashes at runtime. I don't see why URLs should be any different.

— E

They shouldn’t be. The print function can already handle things that aren’t `CustomStringConvertible`:
struct Foo {}
let x = Foo()
Foo.self is CustomStringConvertible //says `false`
print(x) //says `"Foo()\n"`
so, IMHO, that’s a bug.

Oh, hey, wait a second… I didn’t catch the meaning of the resource name, and thought you meant it was crashing on the print statement. I still think it’s a bug, because the compiler should be able to check if the resource exists.

I think the point here is that what exists at compile time may not exist at runtime and vice versa, so while a warning might be elegant, it's not helpful for the compiler to refuse to proceed on the basis that the image does not yet exist. At the end of the day, an image literal hardcodes the path to an image, not the image itself. Whether it ought to is another kettle of fish.

Ah, ok, I didn’t realize that image literals weren’t actually created until runtime... In that case, I retract my statement that they shouldn't be any different, because they while they both may be called “literals", they use that word to mean to two quite different things.


(Xiaodi Wu) #7

The behavior is not altogether clear to me either. Since XCode does show a
preview of images, it's fair to think that there's some sort of
compile-time validation going on.

I would not be opposed to image and file literals changing their behavior
so that when a user drags a file into the IDE the file contents are what
are embedded via a data URL, rather than a file path. I do know that in a
Playground, when you choose a file for a file literal, an alias is made to
that file stored in a different folder for Playground resources, and it's
that alias which is referenced in the resulting URL. This seems brittle to
me.

···

On Mon, Dec 19, 2016 at 13:55 David Sweeris <davesweeris@mac.com> wrote:

On Dec 19, 2016, at 11:48 AM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Mon, Dec 19, 2016 at 1:44 PM, David Sweeris via swift-evolution < > swift-evolution@swift.org> wrote:

On Dec 19, 2016, at 11:36 AM, David Sweeris via swift-evolution < > swift-evolution@swift.org> wrote:

On Dec 19, 2016, at 11:21 AM, Erica Sadun via swift-evolution < > swift-evolution@swift.org> wrote:

let x = #imageLiteral(resourceName:"nothere.jpg")
print(x)

This compiles. It crashes at runtime. I don't see why URLs should be any
different.

— E

They shouldn’t be. The print function can already handle things that
aren’t `CustomStringConvertible`:

struct Foo {}
let x = Foo()
Foo.self is CustomStringConvertible //says `false`
print(x) //says `"Foo()\n"`

so, IMHO, that’s a bug.

Oh, hey, wait a second… I didn’t catch the meaning of the resource name,
and thought you meant it was crashing on the print statement. I still think
it’s a bug, because the compiler should be able to check if the resource
exists.

I think the point here is that what exists at compile time may not exist
at runtime and vice versa, so while a warning might be elegant, it's not
helpful for the compiler to refuse to proceed on the basis that the image
does not yet exist. At the end of the day, an image literal hardcodes the
path to an image, not the image itself. Whether it ought to is another
kettle of fish.

Ah, ok, I didn’t realize that image literals weren’t actually created
until runtime... In that case, I retract my statement that they shouldn't
be any different, because they while they both may be called “literals",
they use that word to mean to two quite different things.

- Dave Sweeris