SE-0243: Codepoint and Character Literals

Evolution Proposal Reviews
269

I think so far there’s been five serious alternatives if this does get returned for revision, so I figured it’s worth summarizing the pros, cons, and implications of each so we can settle on a design moving forward.

1. 'a'.ascii, callable member

let codepoint:Unicode.Scalar = 'a'
return codepoint.ascii
Single quoted literals default to Unicode.Scalar
Implementation difficulty Easy
Compile-time validation? No

Summary:
The Unicode.Scalar type will get an .ascii computed property, which provides its value with the trapping precondition that value < 0x80.

Pros:

  • Readable, concise, and clearly indicates encoding used.
  • Has high discoverability as a callable member.
  • No new compiler or language features needed.
  • No new syntax or semantics.

Cons:

  • Character literals will continue to require type context.
  • Character literals either cannot be expressed with single quotes, or would result in ambiguous expressions like 'é' as Character.
  • Impossible to provide compile-time validation guarantees. (The best we can do is a warning heuristic.)
  • Member .ascii would be available on all Unicode.Scalar values, including run-time values (foo.ascii), which doesn’t seem appropriate from an API standpoint.
  • Exposes users to run-time trapping.
  • Privileges ASCII subset of Unicode.Scalar.
  • Overloads on return type.
  • Strongly ABI-coupled.

2. 'a'.ascii, “literal-bound” member

return 'a'.ascii
Single quoted literals default to Character
Implementation difficulty Hard
Compile-time validation? Yes

Summary:
Swift will support a new method attribute @literalself, essentially a more restrictive version of @constexpression on self. The Character type will get an .ascii computed property which is @literalself, and provides its ASCII value subject to the compile-time condition that it consists of a single codepoint within the ASCII range. Note that this would still be vulnerable to '\r\n' folding.

Pros:

  • Readable, concise, and clearly indicates encoding used.
  • Provides compile-time validation guarantee.
  • Decoupled from ABI.

Cons:

  • Extremely magical, could be considered an abuse of dot . notation.
  • Effectively introduces entire new kind of instance method to the language, depends on @constexpression to generalize into a language feature.
  • Very low discoverability.
  • Privileges ASCII subset of Unicode.Scalar.
  • Overloads on return type.

3. 'a' as UInt8

return 'a' as UInt8
Single quoted literals default to Character
Implementation difficulty Hard
Compile-time validation? Yes

Summary:
Swift will introduce the concept of non-expressible literal coercions, which would allow “opt-in” literal coercions through the use of the as operator. (Note that this is not an overload on the as operator, it merely makes this operator mandatory if requested.) Contrast with Swift’s existing expressible literal coercions, which are “opt-out”, and make the as operator optional. All FixedWidthInteger types would receive a non-expressible literal conformance to unicode scalar literals. This is essentially identical to the proposal as written, except it requires an explicit as (U)Int8 everywhere a codepoint literal→ASCII coercion takes place.

Pros:

  • Readable (though not as concise).
  • Makes it obvious that a literal coercion is taking place.
  • Provides compile-time validation guarantee.
  • Decoupled from ABI.

Cons:

  • Does not indicate ASCII as the specific encoding used.
  • Effectively adds a new feature to the literals system (see this post), depends on @constexpression to generalize into a language feature.

4. a'a'

return a'a'
Single quoted literals default to Character (u'a' defaults to Unicode.Scalar)
Implementation difficulty Medium
Compile-time validation? Yes

Summary:
Single-quoted literals will be subdivided into multiple prefixed literal sorts. Unprefixed tokens will be parsed as character literals, u-prefixed tokens will be parsed as unicode scalar literals, and a-prefixed tokens will be parsed as integer literals, constrained to the ASCII-range.

Pros:

  • Readable, highly concise, indicates encoding used.
  • Very few compiler modifications needed, no new language features needed.
  • Provides compile-time validation guarantee.
  • Decoupled from ABI.
  • Easily extensible to provide unambiguous syntaxes for Unicode.Scalar (u'a') and Character ('a') literals, as well as alternative character encodings.
  • No new semantics.

Cons:

  • Introduces new syntax to the language. (As opposed to 2 and 3, which only introduce new semantics.)
  • Users need to remember single-character abbreviations for each prefix (“a for ascii”, “u for unicode scalar”, etc).
  • Low discoverability.

5. Full ASCII struct

return ('a' as ASCII).value
Single quoted literals default to Character
Implementation difficulty Medium
Compile-time validation? Yes

Summary:
The standard library will gain a full 7-bit ASCII type which is expressible by unicode scalar literals. Compile-time validation will be performed in the compiler in a semi-magical fashion, just like the current proposal as written.

Pros:

  • The most conservative and strongly-typed design.
  • Few compiler modifications needed, no new language features needed.
  • Provides compile-time validation guarantee.
  • No new syntax or semantics.
  • Mid–high discoverability.

Cons:

  • Limited utility. (Useful for generating outputs, but useless for processing input bytestrings.)
  • May encourage users to bind raw buffers to this type, which is incorrect. (An arbitrary (U)Int8 cannot be safely reinterpreted as 7-bit ASCII value.)
  • Member .value effectively overloads on return type.
  • Strongly ABI-coupled.
5 Likes