New literal for string from contents of file

Evolution Pitches
,
21

I don't understand how this would work. How would the compiler create something of type NSData without Foundation?

22

It will be able because a Foundation type should conform to the protocol in case it uses NSData as LiteralType. Without importing Foundation, you won’t be able to use that specific implementation.

23

What would the protocol to which NSData conforms look like? Literals aren't magic; at some point, the compiler needs to instantiate a built-in or standard library type to pass as an argument to the literal initializer. What is that type?

24

Wouldn't a stream-like type be the good choice here? This could help controlling memory consumption, and support huge resources. Some conforming types may fail early consuming the input data (imagine a json-based type). Some conforming types may reduce the input data (in a hashed value, for example).

25

Like the one I had in my previous reply. And for other types as well. For the Standard Library, we can have the String variant of the literal, for instance. If you need other encoding or [NS]Data, import Foundation.

26

Good idea!

27

I think you're misunderstanding my point. The literal initializer for NSData must receive the contents of the file in an argument of some type. What is that type? It cannot be NSData, obviously.

28

I'm not familiar enough with streams to pop a good type out of my memory. And I thought Foundation streams had a bad reputation. What existing type do you know, that could be consumed on demand?

29

I think we'd probably have to create one--haven't thought this one through, honestly. Otherwise, it could just be a buffer pointer.

30

Thanks for clarifying. String from NSData.init(contentsOfFile: String)

31

UnsafeRawBufferPointer?

32

OK. And how would the compiler instantiate this String? Keep in mind it has no access to decoding functions in Foundation.

33

That works, and we circle back to my initial reply:

1 Like
34

I think the answer to this question will help:
What does the compiler use for the color literal?

35

It uses init(colorLiteralRed: Double, green: Double, blue: Double, alpha: Double). It just passes along the values from the end user's #colorLiteral(red:green:blue:alpha:) exactly as written.

Edit: ah, you mean that NSData should receive the location of the file as a String. The compiler can definitely do that, but that's exactly today's #fileLiteral. It does not address @mklbtz's goal of embedding the data at compile time.

1 Like
36

Correct me if needed. This is roughly my view of the implementation:


protocol ExpressibleByColorLiteral {
    
    init(colorLiteralRed: Double, green: Double, blue: Double, alpha: Double)
}

// In UIKit

extension UIColor: ExpressibleByColorLiteral {
    
    required init(colorLiteralRed: Double, green: Double, blue: Double, alpha: Double) {
        // initialize with given values
    }
}


protocol ExpressibleByFileContentLiteral {
    
    associatedtype FileContentLiteralType
    
    init(resource: FileContentLiteralType)
}

// In Foundation

extension NSData: ExpressibleByFileContentLiteral {
    
    typealias FileContentLiteralType = String
    
    required init(resource: FileContentLiteralType) {
        // initialize like NSData(contentsOfFile: String)
    }
}

let data: NSData = #fileContentLiteral(resource: "path")

let color = #colorLiteral(red: r, green: g, blue: b, alpha: a)

The color instantiates at compile time and depicts as a rounded square, so it turns out the data too can (instantiate)?

37

There appears to be no difference between your proposed #fileContentLiteral and the existing #fileLiteral. The compiler will not read data at compile time because the contents can change at runtime.

38

If this feature is going to be a general "initialize a value of some type embedding the contents of some other file into the executable", and if we don't want to put constraints onto the types, then the compiler cannot directly do preprocessing or prevalidation, such as codepoint translation. This model means that the feature is a sugar feature that turns:

let x = #yoursyntax(file: "foo.dat") as YourType

into something like:

 let buffer = [0x1, 0x2, ... ]  // contents of foo.dat, in bytes.
 let x = YourType(someLabel: buffer)

It would make sense to use UnsafeBufferPointer to avoid the array object, but you get the point. In this model, the protocol is just there to link up the type with the init that has the right keyword arguments. The currency of this syntax would always be an array/buffer of bytes. We could even take the protocol out of the situation and allow extra arguments, which could enable:

let x = #yoursyntax(file: "foo.txt", encoding: .utf8) as String

to turn into:

 let x = YourType(someLabel: buffer, encoding: .utf8)

The only disadvantage I see to this approach is that we can't diagnose invalid stuff with just this or transcode data, but I think that is ok. We have other tools in the toolbox that allow this, similar to how we diagnose integer overflows in a mandatory SIL pass using well-known builtins. This gives us the advantage of generality for things like Data and [UInt8] while allowing custom behavior for important special cases like String.

-Chris

5 Likes
39

Agree, this seems like a good fit for a #dataLiteral. It'd work like magic for many types; the default DataLiteralType could even be Foundation.Data, following on the model of #colorLiteral, with the user prompted to import Foundation if necessary. That way, when converting to String, the full set of encoding functionality is available.

2 Likes
40

Great point, I agree, the natural default type should be Data.

2 Likes