# Prepitch: Character integer literals **URL:** \ **Category:** Pitches\ **Created:** [February 28, 2018, 11:08pm UTC](https://forums.swift.org/t/prepitch-character-integer-literals/10442 "2018-02-28T23:08:40Z")\ **Posts on this page:** 20\ **Page:** 15
**Author:** ![johnno1962](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/johnno1962/32/6805_2.png) [@johnno1962](https://forums.swift.org/u/johnno1962)\ **Post date:** [January 11, 2019, 10:44pm UTC](https://forums.swift.org/t/prepitch-character-integer-literals/10442/283 "2019-01-11T22:44:56Z")
> [@Douglas\_Gregor](#): > > I'm encouraging you to consider adopting the existing protocols _as they are_ . It feels like we don't lose much---the protocol names aren't perfect, but they're rarely seen anyway, for example, and we could probably use the constant-evaluation work to help get overflow diagnostics. The benefit is that it reduces the ABI effect of the proposal down to only added conformances... the rest is in the source-level language, reducing risk/churn/schedule pressure a bit. > [@xwu](#): > > The alternative would be to go after a design without ABI impact along the lines that Doug has sketched out. These are the constraints that guided the new implementation. Aside from being more complicated if we introduced ExpressibleByASCIILiteral at this stage it would also break the deal we made with ourselves to leave the door open for Option 2. ---
**Author:** ![taylorswift](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/taylorswift/32/43413_2.png) [@taylorswift](https://forums.swift.org/u/taylorswift)\ **Post date:** [January 11, 2019, 11:50pm UTC](https://forums.swift.org/t/prepitch-character-integer-literals/10442/284 "2019-01-11T23:50:03Z")
can we _please_ just get this to review? this thread has been going in circles for _a year_ now ---
**Author:** ![SDGGiesbrecht](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/sdggiesbrecht/32/4768_2.png) [@SDGGiesbrecht](https://forums.swift.org/u/SDGGiesbrecht)\ **Post date:** [January 11, 2019, 11:57pm UTC](https://forums.swift.org/t/prepitch-character-integer-literals/10442/285 "2019-01-11T23:57:19Z")
> [@johnno1962](#): > > it would also break the deal we made with ourselves to leave the door open for Option 2. Not really. That would be no more difficult than adding conformances for `Int` types to `ExpressibleByUnicodeScalarLiteral`. In fact, that set‐up would allow you to trivially add such a conformance in your own module if you understood what you were doing and wanted to live dangerously. The way we have it now, the conformance is already there and in the way, so you cannot do it yourself. (...Well, though uglier, you _could_ add a conformance to the character version to work around it, so maybe never mind.) But more people are thinking this than are outgoing enough to say it: > [@taylorswift](#): > > can we _please_ just get this to review? this thread has been going in circles for _a year_ now ---
**Author:** ![taylorswift](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/taylorswift/32/43413_2.png) [@taylorswift](https://forums.swift.org/u/taylorswift)\ **Post date:** [January 12, 2019, 12:14am UTC](https://forums.swift.org/t/prepitch-character-integer-literals/10442/286 "2019-01-12T00:14:52Z")
> [@SDGGiesbrecht](#): > > Not really. That would be no more difficult than adding conformances for `Int` types to `ExpressibleByUnicodeScalarLiteral` it’s not that, it’s the fact that we’re changing `ExpressibleByUnicodeScalarLiteral` from a base protocol to an inherited one. this means the requirements for `ExpressibleByUnicodeScalarLiteral`, and every literal protocol that inherits it, are different now. to users, there’s no change, since the new requirements all get default implementations, but again, idk how this affects ABI ---
**Author:** ![SDGGiesbrecht](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/sdggiesbrecht/32/4768_2.png) [@SDGGiesbrecht](https://forums.swift.org/u/SDGGiesbrecht)\ **Post date:** [January 12, 2019, 1:06am UTC](https://forums.swift.org/t/prepitch-character-integer-literals/10442/287 "2019-01-12T01:06:02Z")
I meant the present change would not make John’s desired future addition any more difficult at that future time. I understand the difference it makes at the present. ---
**Author:** ![xwu](https://avatars.discourse-cdn.com/v4/letter/x/b487fb/32.png) [@xwu](https://forums.swift.org/u/xwu)\ **Post date:** [January 13, 2019, 10:02am UTC](https://forums.swift.org/t/prepitch-character-integer-literals/10442/288 "2019-01-13T10:02:22Z")
> [@taylorswift](#): > > can we _please_ just get this to review? this thread has been going in circles for _a year_ now On the contrary, I think this thread is just starting to go in some interesting directions. Given that it won’t make it into Swift 5.0, I don’t see any reason to cut off such fruitful discussion. ---
**Author:** ![Tino](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/tino/32/1073_2.png) [@Tino](https://forums.swift.org/u/Tino)\ **Post date:** [January 13, 2019, 11:07am UTC](https://forums.swift.org/t/prepitch-character-integer-literals/10442/289 "2019-01-13T11:07:10Z")
I have no idea what merit continuing this thread may have or if anyone reads all those messages, but as it is tagged as a pitch and even labeled as "prepitch", I think it's sensible to move forward by creating a new topic. Also, imho it would be a good idea to start this with a introduction to the fundamental problems of encodings (I guess there are some good resources that could be linked), and their impact on Swift: It's quite hard to find information about _source file_ encoding for swiftc, and `swiftc --help` doesn't even mention what encoding it expects. ---
**Author:** ![taylorswift](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/taylorswift/32/43413_2.png) [@taylorswift](https://forums.swift.org/u/taylorswift)\ **Post date:** [January 13, 2019, 2:01pm UTC](https://forums.swift.org/t/prepitch-character-integer-literals/10442/290 "2019-01-13T14:01:17Z")
> [@Tino](#): > > Also, imho it would be a good idea to start this with a introduction to the fundamental problems of encodings (I guess there are some good resources that could be linked), and their impact on Swift about midway through the thread we added exactly such a section, but we removed it because people said it “wasn’t relevant”. the “background and terminology” section is a vestige of that originally much longer background section. ---
**Author:** ![Jean-Daniel](https://avatars.discourse-cdn.com/v4/letter/j/977dab/32.png) [@Jean-Daniel](https://forums.swift.org/u/Jean-Daniel)\ **Post date:** [January 13, 2019, 7:21pm UTC](https://forums.swift.org/t/prepitch-character-integer-literals/10442/291 "2019-01-13T19:21:14Z")
> [@Nevin](#): > > It is the wrong question. We are not trying to encode “é” at all. We are trying to treat it as an integer. And there is only one integer which canonically maps to it, namely 233. And what the use case for that ? Your int will not be usable with any UTF-8 byte buffer, which is the only representation using UInt8 actually. ---
**Author:** ![Tino](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/tino/32/1073_2.png) [@Tino](https://forums.swift.org/u/Tino)\ **Post date:** [January 14, 2019, 12:42pm UTC](https://forums.swift.org/t/prepitch-character-integer-literals/10442/292 "2019-01-14T12:42:00Z")
> [@taylorswift](#): > > about midway through the thread we added exactly such a section, but we removed it because people said it “wasn’t relevant” SE is hard... ;-) but afaics, there's not even a link to the current proposal text in the first post (imho that alone should be reason enough to start a fresh thread), and it's tedious to find this deleted section. I have no strong opinion how detailed the "basics" should be addressed in the proposal text, but I think it would be beneficial to have at least a link to a page that explains all those points that may seem trivial for experts, but still have potential to confuse less experienced readers. Of course, there are some resources that explain why strings in Swift are so much more "complicated" than in other languages, but I'm not aware of any official document (mhh, looking at recent entries from [https://swift.org/blog/:](https://swift.org/blog/:) Maybe someone with write access could post an updated version of [mikeash.com: Friday Q&A 2015-11-06: Why is Swift's String API So Hard?](https://www.mikeash.com/pyblog/friday-qa-2015-11-06-why-is-swifts-string-api-so-hard.html) there? ;-) ---
**Author:** ![johnno1962](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/johnno1962/32/6805_2.png) [@johnno1962](https://forums.swift.org/u/johnno1962)\ **Post date:** [January 14, 2019, 1:03pm UTC](https://forums.swift.org/t/prepitch-character-integer-literals/10442/293 "2019-01-14T13:03:28Z")
The original proposal document you’re looking for is [here](https://github.com/apple/swift-evolution/blob/4026a6336b1f875ad88a99dc793aebefd8b33e66/proposals/0233-codepoint-and-character-literals.md#proposed-solution). It’s Unicode that is hard and thankfully we are able to avoid a detailed discussion about it since we limited the integer conversion feature to ASCII (a.k.a. Option 4). Swift does a great job of hiding the vagaries of Unicode under an clean abstraction. We’re not proposing anything that would change that other than separating Character/Unicode.Scalar literals from String literals which allows us to add making integers expressible by the new character literals for ASCII characters. ---
**Author:** ![dwaite](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/dwaite/32/1200_2.png) [@dwaite](https://forums.swift.org/u/dwaite)\ **Post date:** [January 15, 2019, 6:54pm UTC](https://forums.swift.org/t/prepitch-character-integer-literals/10442/294 "2019-01-15T18:54:35Z")
To emphasize your point, even the source document (where this character literal is present) is in an encoding, and also risks being normalized or denormalized by other tools. With ASCII, the risk there is that a tool might make code that didn't compile before work, or code which did compile before fail. Extending beyond ASCII, the primary risk is that people will get the codes wrong (e.g. not understand that the character literal will be interpreted as Latin 1/UCS-2/UTF-32). I do not know if source files in swift can be in non-UTF8 encoding, where this becomes even more of a confusing issue for the developer. With UCS-2/UTF-32, the risk is that a valid scalar will be interpreted at compile time as a different valid scalar. Initializing a Unicode.Scalar from a literal is a risk but one people could be expected to understand considering they are working with literal initialization of unicode scalars. IMHO, it's far easier just to limit it to ASCII only, and (probably more arguably) [U]Int8 only. > [@lorentey](#): > > I have bad news: that is, by definition, an encoding. To treat a character as a number, you need to select an encoding. What you are arguing for is simply this particular set of encodings: > > UInt8 : Latin-1 > UInt16 : UCS-2 > UInt32 : UTF-32 > > It’s not a great choice. ---
**Author:** ![fswarbrick](https://avatars.discourse-cdn.com/v4/letter/f/54ee81/32.png) [@fswarbrick](https://forums.swift.org/u/fswarbrick)\ **Post date:** [January 16, 2019, 12:25am UTC](https://forums.swift.org/t/prepitch-character-integer-literals/10442/295 "2019-01-16T00:25:43Z")
> [@dwaite](#): > > Extending beyond ASCII, the primary risk is that people will get the codes wrong (e.g. not understand that the character literal will be interpreted as Latin 1/UCS-2/UTF-32). I do not know if source files in swift can be in non-UTF8 encoding, where this becomes even more of a confusing issue for the developer. FWIW, the [IBM Toolkit for Swift on z/OS](https://developer.ibm.com/mainframe/products/ibm-toolkit-swift-z-os/) allows for source code in any of 3 different EBCDIC code pages. As an EBCDIC user this whole thread makes me a bit antsy, but I don't specifically know if it will be unusable or hard to use on an EBCDIC based OS. ---
**Author:** ![SDGGiesbrecht](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/sdggiesbrecht/32/4768_2.png) [@SDGGiesbrecht](https://forums.swift.org/u/SDGGiesbrecht)\ **Post date:** [January 16, 2019, 1:04am UTC](https://forums.swift.org/t/prepitch-character-integer-literals/10442/296 "2019-01-16T01:04:05Z")
Doesn’t the following assertion hold when compiled on z/OS right now? ```swift assert("a".unicodeScalars.first?.value == 0x61) ``` If so, nothing will really change. Since `'a'` will be treated the same way, nothing will be unusable and source code will be completely compatible between the two systems. This assertion will also hold: ```swift assert('a' == 0x61) ``` If that is not actually how you _want_ it to behave, then please elaborate. ---
**Author:** ![fswarbrick](https://avatars.discourse-cdn.com/v4/letter/f/54ee81/32.png) [@fswarbrick](https://forums.swift.org/u/fswarbrick)\ **Post date:** [January 16, 2019, 1:28am UTC](https://forums.swift.org/t/prepitch-character-integer-literals/10442/297 "2019-01-16T01:28:55Z")
> [@SDGGiesbrecht](#): > > Doesn’t the following assertion hold when compiled on z/OS right now? > > ```swift > assert("a".unicodeScalars.first?.value == 0x61) > > ``` Yes, the above assertion holds on z/OS. My issue is that there might be a case where I want to call a z/OS subsystem service or API that uses an EBCDIC codepage as its native encoding. Let's say it expects to be called with an 8-byte EBCDIC character string. If only ASCII/UTF-8 is supported then I can't just do the following: `let r = zOSAPI1(['T', 'E', 'S', 'T', ' ', ' ', ' '. ' '])` Or perhaps even: `let r = zOSAPI1('TEST ')` Because that would pass an array of 8 ASCII characters, not an array of 8 EBCDIC characters. That being said, I don't know how realistic this example is. Certainly there are many z/OS systems that have this type of API. But to make them useful and more Swifty one would want to write Swift wrappers around them anyway, where we'd just use Swift strings and then convert the strings to EBCDIC character arrays within the wrapper, using the Foundation supplied String.Encoding method(s). So what is the likelihood of a user application calling the API directly like this with a literal? Perhaps not all that likely. One issue I have with this entire thread is the example uses cases are very small and perhaps not all that realistic either. I'd like to see some more fleshed out uses cases for me to be able to think about how they might work in an EBCDIC world. Perhaps I'm making a mountain out of a mole hill. I don't know... ---
**Author:** ![SDGGiesbrecht](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/sdggiesbrecht/32/4768_2.png) [@SDGGiesbrecht](https://forums.swift.org/u/SDGGiesbrecht)\ **Post date:** [January 16, 2019, 2:12am UTC](https://forums.swift.org/t/prepitch-character-integer-literals/10442/298 "2019-01-16T02:12:25Z")
> [@fswarbrick](#): > > write Swift wrappers around them anyway, where we'd just use Swift strings and then convert the strings to EBCDIC character arrays _...sigh..._ The dangers of that sort of method are part of why this pitch exists: ```swift let bytes: [UInt8] = ebcdic37("â") // 0x42 ``` The above code would crash at runtime if the “â” got decomposed while the source file was being handled. The compiler would instead have created the Unicode sequence [U+0041, U+0302] which fails conversion to EBCDIC 37: [0x81, _invalid_] But I guess you would be insulated as long as the source file itself stayed in the EBCDIC encoding, so you are not as vulnerable as the rest of us. * * * > [@fswarbrick](#): > > `let r = zOSAPI1(['T', 'E', 'S', 'T', ' ', ' ', ' '. ' '])` I don’t know how to make that sort of thing possible in both of two different encodings at the compiler level. But you could still do it at runtime. In pseudocode: ```swift func zOSAPI1(_ parameter: [Character]) { let safetyCheckedScalars = parameter.map({ $0.precomposedStringWithCanonicalMapping.unicodeScalars.first! }) let bytes = safetyCheckedScalars.map({ $0.ebcdic37Value }) zOS.zOSAPI1(bytes) // ← Call the actual primitive. } let r = zOSAPI1(['T', 'E', 'S', 'T', ' ', ' ', ' '. ' ']) ``` ---
**Author:** ![ben-cohen](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/ben-cohen/32/296_2.png) [@ben-cohen](https://forums.swift.org/u/ben-cohen)\ **Post date:** [February 26, 2019, 9:14pm UTC](https://forums.swift.org/t/prepitch-character-integer-literals/10442/299 "2019-02-26T21:14:18Z")
Hi everyone, In prepping this proposal for review, we've encountered a snag in the implementation that will need to be addressed. Once Swift is in the OS, all new types shipped in the OS will be subject to availability via `@available` annotations. However, Swift does not yet have a way of annotating a _conformance_ with availability. This is a capability that could be added to the language in future, but does not exist right now. With new protocols, this is not a problem: the protocol itself will have availability, and therefore any conformances will too. Likewise new structs will have availability so can conform to existing protocols. But this won't apply to new conformances of existing types to existing protocols. The pitch proposes adding exactly this: of conforming existing integer types to existing `ExpressibleBy...` protocols. This means we will need to adapt the proposal to accommodate this. The core team's preferred approach would be that we subset the proposal to just introducing the `'x'` syntax for the scalar and grapheme literals. This addresses the pain point of having to give `Character` literals type context. It will also allow users to conform the integer types themselves via a short extension. Not as convenient as having it built in to the standard library, to be sure, but not too taxing. Once we have the ability to apply conformances retroactively, we could then put them in the standard library too. Let me know your thoughts, Ben p.s. the ability to declare availability on a conformance is certainly something the core team expects to see added to the language in future versions of Swift – any thoughts on the design/implementation of that feature would be welcome over on the [compiler development](https://forums.swift.org/c/development/compiler) forum. ---
**Author:** ![taylorswift](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/taylorswift/32/43413_2.png) [@taylorswift](https://forums.swift.org/u/taylorswift)\ **Post date:** [February 26, 2019, 9:32pm UTC](https://forums.swift.org/t/prepitch-character-integer-literals/10442/300 "2019-02-26T21:32:53Z")
> [@ben-cohen](#): > > that we subset the proposal to just introducing the `'x'` syntax for the scalar and grapheme literals. This addresses the pain point of having to give `Character` literals type context. It will also allow users to conform the integer types themselves via a short extension. Not as convenient as having it built in to the standard library, to be sure, but not too taxing. The core of the proposal is the _statically checked_ ascii literals,, the character and unicode scalar literal syntax is just a side benefit. the first thing isn’t possible in user-defined extensions, the second is just a syntactic difficulty from not having enough type context. taking the second without the first feels like cutting out the cake for the icing. > [@ben-cohen](#): > > p.s. the ability to declare availability on a conformance is certainly something the core team expects to see added to the language in future versions of Swift how long should we expect before this gets unblocked? ---
**Author:** ![Nevin](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/nevin/32/617_2.png) [@Nevin](https://forums.swift.org/u/Nevin)\ **Post date:** [February 26, 2019, 10:03pm UTC](https://forums.swift.org/t/prepitch-character-integer-literals/10442/301 "2019-02-26T22:03:27Z")
> [@taylorswift](#): > > The core of the proposal is the _statically checked_ ascii literals,, the character and unicode scalar literal syntax is just a side benefit. the first thing isn’t possible in user-defined extensions, the second is just a syntactic difficulty from not having enough type context. taking the second without the first feels like cutting out the cake for the icing. As long as the static checking for ASCII literals is confined to single-quoted characters, what’s the problem? ```swift protocol ExpressibleBySingleQuotedCharacterLiteralOrWhatever { … } extension Int: ExpressibleBy… { … } let x: Int = 'a' // 97 ``` ---
**Author:** ![taylorswift](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/taylorswift/32/43413_2.png) [@taylorswift](https://forums.swift.org/u/taylorswift)\ **Post date:** [February 26, 2019, 10:21pm UTC](https://forums.swift.org/t/prepitch-character-integer-literals/10442/302 "2019-02-26T22:21:51Z")
there can be no static checks in user extensions? [Previous page](https://forums.swift.org/t/prepitch-character-integer-literals/10442.md?page=14) [Next page](https://forums.swift.org/t/prepitch-character-integer-literals/10442.md?page=16)