ExpressibleByStringInterpolation vs. String re-evaluation vs. Regex


(Jacob Bandes-Storch) #1

In the past, there has been some interest in refining the behavior of
ExpressibleByStringInterpolation (née StringInterpolationConvertible), for
example:

- Ability to *restrict the types that can be used* as interpolation segments
- Ability to *distinguish the string-literal segments* from interpolation
segments whose type is String

Some prior discussions:
- "StringInterpolationConvertible and StringLiteralConvertible inheritance"
https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20160516/017654.html
- Sub-discussion in "Allow multiple conformances to the same protocol"
https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20160606/020746.html
- "StringInterpolationConvertible: can't distinguish between literal
components and String arguments" https://bugs.swift.org/browse/SR-1260
/ rdar://problem/19800456&18681780
- "Proposal: Deprecate optionals in string interpolation"
https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20160516/018000.html

About Swift 4, Chris wrote:

- *String re-evaluation:* String is one of the most important
fundamental types in the language. The standard library leads have
numerous ideas of how to improve the programming model for it, without
jeopardizing the goals of providing a unicode-correct-by-default model.
Our goal is to be better at string processing than Perl!

I'd be interested in any more detail the team can provide on this. I'd like
to talk about string interpolation improvements, but it wouldn't be wise to
do so without keeping an eye towards possible regex/pattern-binding syntax,
and the String refinements that the stdlib team has in mind, if there's a
chance they would affect interpolation.

Discuss!

-Jacob


(Brent Royal-Gordon) #2

I'm not one of the core team, so all I can really provide is a use case.

Given a LocalizedString type like:

    /// Conforming types can be included in a LocalizedString.
    protocol LocalizedStringConvertible {
        /// The format to use for this instance. This format string will be included in the key when
  /// this type is interpolated into a LocalizedString.
        var localizedStringFormat: String { get }
    
        /// The arguments to use when formatting to represent this instance.
        var localizedStringArguments: [CVarArg] { get }
    }

    extension NSString: LocalizedStringConvertible {…}
    extension String: LocalizedStringConvertible {…}
    extension LocalizedString: LocalizedStringConvertible {…}
    
    extension Int: LocalizedStringConvertible {…}
    // etc.

    struct LocalizedString {
        /// Initializes a LocalizedString by applying the `arguments` to the format string with the
        /// indicated `key` using `String.init(format:arguments:)`.
        ///
        /// If the `key` does not exist in the localized string file, the `key` itself will be used as
        /// the format string.
        init(key: String, formattedWith arguments: [CVarArg]) {…}
    }
    
    extension String {
        init(_ localizedString: LocalizedString) {
            self.init(describing: localizedString)
        }
    }

    extension LocalizedString {
        /// Initializes a LocalizedString with no arguments which uses the indicated `key`. `%`
        /// characters in the `key` will be converted to `%%`.
        ///
        /// If the `key` does not exist in the localized string file, the `key` itself will be used as
        /// the string.
        init(key: String) {…}
    
        /// Initializes a LocalizedString to represent the indicated `value`.
        init(_ value: LocalizedStringConvertible) {…}
    
        /// Initializes a LocalizedString to represent the empty string.
        init() {…}
    }

    extension LocalizedString: CustomStringConvertible {…}
    
    extension LocalizedString: ExpressibleByStringLiteral {
        init(stringLiteral value: String) {
            self.init(key: value)
        }
        …
    }

The current ExpressibleByStringInterpolation protocol has a number of defects.

  1. We want to only permit LocalizedStringConvertible types, or at least *use* the LocalizedStringConvertible conformance; neither of these appears to be possible. (`is` and `as?` casts always fail, overloads don't seem to be called, etc.)

  2. The literal parts of the string are interpreted using `String`'s `ExpressibleByStringLiteral` conformance; we really want them to use `LocalizedString`'s instead.

  3. We don't want the literal parts of the string to pass through `init(stringInterpolationSegment:)`, because we want to treat interpolation and literal segments differnetly.

In other words, we want to be able to write something like this:

  extension LocalizedString: ExpressibleByStringInterpolation {
    typealias StringInterpolatableType = LocalizedStringConvertible
    
    init(stringInterpolation segments: LocalizedString) {
      self.init()
      for segment in segments {
        formatKey += segment.formatKey
        arguments += segment.arguments
      }
    }
    
    init(stringInterpolationSegment expr: LocalizedStringConvertible) {
      self.init(expr)
    }
  }

And change the code generated by the compiler from (given the statement `"foo \(bar) baz" as LocalizedString`) this:

  LocalizedString(stringInterpolation:
    LocalizedString(stringInterpolationSegment: String(stringLiteral: "foo ")),
    LocalizedString(stringInterpolationSegment: bar),
    LocalizedString(stringInterpolationSegment: String(stringLiteral: " baz"))
  )

To this:

  LocalizedString(stringInterpolation:
    LocalizedString(stringLiteral: "foo "),
    LocalizedString(stringInterpolationSegment: bar),
    LocalizedString(stringLiteral: " baz")
  )

This would obviously require a few changes to the ExpressibleAsStringInterpolation protocol:

  // You cannot accept interpolations unless you can also be a plain literal.
  // Necessary for literal segments.
  protocol ExpressibleByStringInterpolation: ExpressibleByStringLiteral {
    // An associated type for the type of a permitted interpolation
    associatedtype StringInterpolatableType = Any
    
    // No changes here
    init(stringInterpolation segments: Self...)
    
    // No longer generic; instead uses StringInterpolatableType existentials.
    // Also a semantic change: this is only called for the actual interpolations.
    // init(stringLiteral:) is called for literal segments.
    init(stringInterpolationSegment expr: StringInterpolatableType)
    
    // Given the change in roles, we might want to consider renaming the initializers:
    //
    // init(stringInterpolation:) => init(combinedStringLiteral:) or init(stringInterpolationSegments:)
    // init(stringInterpolationSegment:) => init(stringInterpolation:)
  }

Or perhaps we would hoist the combining initializer up into ExpressibleAsStringLiteral, and generate an `init(combinedStringLiteral:)` call every time string literals are used.

  protocol ExpressibleByStringLiteral {
    associatedtype StringLiteralType: _ExpressibleByBuiltinStringLiteral = String
    
    init(stringLiteralSegments segments: Self...)
    init(stringLiteral value: StringLiteralType)
  }

  protocol ExpressibleByStringInterpolation: ExpressibleByStringLiteral {
    associatedtype StringInterpolatableType = Any
    
    init(stringInterpolation expr: StringInterpolatableType)
  }

  // "foo" as LocalizedString
  LocalizedString(stringLiteralSegments:
    LocalizedString(stringLiteral: "foo")
  )
  
  // "foo \(bar) baz" as LocalizedString
  LocalizedString(stringInterpolation:
    LocalizedString(stringLiteral: "foo "),
    LocalizedString(stringInterpolation: bar),
    LocalizedString(stringLiteral: " baz")
  )

Now, it's quite possible--perhaps even likely--that there are really good reasons for the current design. But I've been thinking about this for two years and I don't know what they are yet; nor can I find much relevant design documentation. I, too, would love to find out why the current design was selected.

···

On Jul 30, 2016, at 10:35 PM, Jacob Bandes-Storch via swift-evolution <swift-evolution@swift.org> wrote:

In the past, there has been some interest in refining the behavior of ExpressibleByStringInterpolation (née StringInterpolationConvertible), for example:

- Ability to restrict the types that can be used as interpolation segments
- Ability to distinguish the string-literal segments from interpolation segments whose type is String

Some prior discussions:
- "StringInterpolationConvertible and StringLiteralConvertible inheritance" https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20160516/017654.html
- Sub-discussion in "Allow multiple conformances to the same protocol" https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20160606/020746.html
- "StringInterpolationConvertible: can't distinguish between literal components and String arguments" https://bugs.swift.org/browse/SR-1260 / rdar://problem/19800456&18681780
- "Proposal: Deprecate optionals in string interpolation" https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20160516/018000.html

About Swift 4, Chris wrote:
- String re-evaluation: String is one of the most important fundamental types in the language. The standard library leads have numerous ideas of how to improve the programming model for it, without jeopardizing the goals of providing a unicode-correct-by-default model. Our goal is to be better at string processing than Perl!

I'd be interested in any more detail the team can provide on this. I'd like to talk about string interpolation improvements, but it wouldn't be wise to do so without keeping an eye towards possible regex/pattern-binding syntax, and the String refinements that the stdlib team has in mind, if there's a chance they would affect interpolation.

Discuss!

--
Brent Royal-Gordon
Architechies


(Dave Abrahams) #3

In the past, there has been some interest in refining the behavior of
ExpressibleByStringInterpolation (née StringInterpolationConvertible), for
example:

- Ability to *restrict the types that can be used* as interpolation segments
- Ability to *distinguish the string-literal segments* from interpolation
segments whose type is String

Hi Jacob,

I see you've already filed a Jira for the second bullet. Can you file
one for the first one? We're going to redesign
ExpressibleByStringInterpolation for Swift 4 and solve these problems.

Thanks,

···

on Sat Jul 30 2016, Jacob Bandes-Storch <swift-evolution@swift.org> wrote:

Some prior discussions:
- "StringInterpolationConvertible and StringLiteralConvertible inheritance"
https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20160516/017654.html
- Sub-discussion in "Allow multiple conformances to the same protocol"
https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20160606/020746.html
- "StringInterpolationConvertible: can't distinguish between literal
components and String arguments" https://bugs.swift.org/browse/SR-1260
/ rdar://problem/19800456&18681780
- "Proposal: Deprecate optionals in string interpolation"
https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20160516/018000.html

About Swift 4, Chris wrote:

- *String re-evaluation:* String is one of the most important
fundamental types in the language. The standard library leads have
numerous ideas of how to improve the programming model for it, without
jeopardizing the goals of providing a unicode-correct-by-default model.
Our goal is to be better at string processing than Perl!

I'd be interested in any more detail the team can provide on this. I'd like
to talk about string interpolation improvements, but it wouldn't be wise to
do so without keeping an eye towards possible regex/pattern-binding syntax,
and the String refinements that the stdlib team has in mind, if there's a
chance they would affect interpolation.

Discuss!

-Jacob
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
-Dave


(Jacob Bandes-Storch) #4

Here's another example use case: Auto Layout visual format strings.

https://gist.github.com/jtbandes/9c1c25ee4996d2554375#file-constraintcollection-swift-L85-L87

Since only views and numeric types are supported, the generic init<T> has
to be a run-time error. Ideally it could be a compile-time error.

>
> In the past, there has been some interest in refining the behavior of
ExpressibleByStringInterpolation (née StringInterpolationConvertible), for
example:
>
> - Ability to restrict the types that can be used as interpolation
segments
> - Ability to distinguish the string-literal segments from interpolation
segments whose type is String
>
> Some prior discussions:
> - "StringInterpolationConvertible and StringLiteralConvertible
inheritance"
https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20160516/017654.html
> - Sub-discussion in "Allow multiple conformances to the same protocol"
https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20160606/020746.html
> - "StringInterpolationConvertible: can't distinguish between literal
components and String arguments" https://bugs.swift.org/browse/SR-1260 /
rdar://problem/19800456&18681780
> - "Proposal: Deprecate optionals in string interpolation"
https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20160516/018000.html
>
>
> About Swift 4, Chris wrote:
> - String re-evaluation: String is one of the most important fundamental
types in the language. The standard library leads have numerous ideas of
how to improve the programming model for it, without jeopardizing the goals
of providing a unicode-correct-by-default model. Our goal is to be better
at string processing than Perl!
>
> I'd be interested in any more detail the team can provide on this. I'd
like to talk about string interpolation improvements, but it wouldn't be
wise to do so without keeping an eye towards possible regex/pattern-binding
syntax, and the String refinements that the stdlib team has in mind, if
there's a chance they would affect interpolation.
>
> Discuss!

I'm not one of the core team, so all I can really provide is a use case.

Given a LocalizedString type like:

    /// Conforming types can be included in a LocalizedString.
    protocol LocalizedStringConvertible {
        /// The format to use for this instance. This format string will
be included in the key when
        /// this type is interpolated into a LocalizedString.
        var localizedStringFormat: String { get }

        /// The arguments to use when formatting to represent this
instance.
        var localizedStringArguments: [CVarArg] { get }
    }

    extension NSString: LocalizedStringConvertible {…}
    extension String: LocalizedStringConvertible {…}
    extension LocalizedString: LocalizedStringConvertible {…}

    extension Int: LocalizedStringConvertible {…}
    // etc.

    struct LocalizedString {
        /// Initializes a LocalizedString by applying the `arguments` to
the format string with the
        /// indicated `key` using `String.init(format:arguments:)`.
        ///
        /// If the `key` does not exist in the localized string file, the
`key` itself will be used as
        /// the format string.
        init(key: String, formattedWith arguments: [CVarArg]) {…}
    }

    extension String {
        init(_ localizedString: LocalizedString) {
            self.init(describing: localizedString)
        }
    }

    extension LocalizedString {
        /// Initializes a LocalizedString with no arguments which uses the
indicated `key`. `%`
        /// characters in the `key` will be converted to `%%`.
        ///
        /// If the `key` does not exist in the localized string file, the
`key` itself will be used as
        /// the string.
        init(key: String) {…}

        /// Initializes a LocalizedString to represent the indicated
`value`.
        init(_ value: LocalizedStringConvertible) {…}

        /// Initializes a LocalizedString to represent the empty string.
        init() {…}
    }

    extension LocalizedString: CustomStringConvertible {…}

    extension LocalizedString: ExpressibleByStringLiteral {
        init(stringLiteral value: String) {
            self.init(key: value)
        }
        …
    }

The current ExpressibleByStringInterpolation protocol has a number of
defects.

        1. We want to only permit LocalizedStringConvertible types, or at
least *use* the LocalizedStringConvertible conformance; neither of these
appears to be possible. (`is` and `as?` casts always fail, overloads don't
seem to be called, etc.)

        2. The literal parts of the string are interpreted using
`String`'s `ExpressibleByStringLiteral` conformance; we really want them to
use `LocalizedString`'s instead.

        3. We don't want the literal parts of the string to pass through
`init(stringInterpolationSegment:)`, because we want to treat interpolation
and literal segments differnetly.

Yep, this is what I filed https://bugs.swift.org/browse/SR-1260 for.

···

On Tue, Aug 2, 2016 at 6:10 PM, Brent Royal-Gordon <brent@architechies.com> wrote:

> On Jul 30, 2016, at 10:35 PM, Jacob Bandes-Storch via swift-evolution < > swift-evolution@swift.org> wrote:

In other words, we want to be able to write something like this:

        extension LocalizedString: ExpressibleByStringInterpolation {
                typealias StringInterpolatableType =
LocalizedStringConvertible

                init(stringInterpolation segments: LocalizedString) {
                        self.init()
                        for segment in segments {
                                formatKey += segment.formatKey
                                arguments += segment.arguments
                        }
                }

                init(stringInterpolationSegment expr:
LocalizedStringConvertible) {
                        self.init(expr)
                }
        }

And change the code generated by the compiler from (given the statement
`"foo \(bar) baz" as LocalizedString`) this:

        LocalizedString(stringInterpolation:
                LocalizedString(stringInterpolationSegment:
String(stringLiteral: "foo ")),
                LocalizedString(stringInterpolationSegment: bar),
                LocalizedString(stringInterpolationSegment:
String(stringLiteral: " baz"))
        )

To this:

        LocalizedString(stringInterpolation:
                LocalizedString(stringLiteral: "foo "),
                LocalizedString(stringInterpolationSegment: bar),
                LocalizedString(stringLiteral: " baz")
        )

This would obviously require a few changes to the
ExpressibleAsStringInterpolation protocol:

        // You cannot accept interpolations unless you can also be a plain
literal.
        // Necessary for literal segments.
        protocol ExpressibleByStringInterpolation:
ExpressibleByStringLiteral {
                // An associated type for the type of a permitted
interpolation
                associatedtype StringInterpolatableType = Any

                // No changes here
                init(stringInterpolation segments: Self...)

                // No longer generic; instead uses
StringInterpolatableType existentials.
                // Also a semantic change: this is only called for the
actual interpolations.
                // init(stringLiteral:) is called for literal segments.
                init(stringInterpolationSegment expr:
StringInterpolatableType)

                // Given the change in roles, we might want to consider
renaming the initializers:
                //
                // init(stringInterpolation:) =>
init(combinedStringLiteral:) or init(stringInterpolationSegments:)
                // init(stringInterpolationSegment:) =>
init(stringInterpolation:)
        }

Or perhaps we would hoist the combining initializer up into
ExpressibleAsStringLiteral, and generate an `init(combinedStringLiteral:)`
call every time string literals are used.

        protocol ExpressibleByStringLiteral {
                associatedtype StringLiteralType:
_ExpressibleByBuiltinStringLiteral = String

                init(stringLiteralSegments segments: Self...)
                init(stringLiteral value: StringLiteralType)
        }

        protocol ExpressibleByStringInterpolation:
ExpressibleByStringLiteral {
                associatedtype StringInterpolatableType = Any

                init(stringInterpolation expr: StringInterpolatableType)
        }

        // "foo" as LocalizedString
        LocalizedString(stringLiteralSegments:
                LocalizedString(stringLiteral: "foo")
        )

        // "foo \(bar) baz" as LocalizedString
        LocalizedString(stringInterpolation:
                LocalizedString(stringLiteral: "foo "),
                LocalizedString(stringInterpolation: bar),
                LocalizedString(stringLiteral: " baz")
        )

Now, it's quite possible--perhaps even likely--that there are really good
reasons for the current design. But I've been thinking about this for two
years and I don't know what they are yet; nor can I find much relevant
design documentation. I, too, would love to find out why the current design
was selected.

--
Brent Royal-Gordon
Architechies


(Jacob Bandes-Storch) #5

Hi Dave,
I just filed https://bugs.swift.org/browse/SR-2303.

Brainstorming: is it important that the init(stringInterpolation:) and
init(stringInterpolationSegment:) requirements are on the same type?
Perhaps it would work to separate these two requirements, allowing the
segments to construct intermediate types, and only requiring the type
adopting ExpressibleByStringInterpolation to implement
init(stringInterpolation:).

This would be nice because it would be much easier for types which aren't
enums to conform to ExpressibleByStringInterpolation. In my auto layout
example (https://gist.github.com/jtbandes/9c1c25ee4996d2554375), the
ConstraintCollection type is only an enum because it has to provide all the
initializers, but it's strange that the cases are accessible publicly;
ideally it would just be a struct with no public initializers besides
init(stringInterpolation:). For example:

    enum InterpolationSegment<T: InterpolationSegmentProtocol> {
        case stringLiteral(String)
        case interpolatedValue(T)
    }

    protocol InterpolationSegmentProtocol {
        // Might want to implement init(stringInterpolationSegment:) for
multiple types,
        // so we can't require a single associated value (same with
ExpressibleByStringLiteral today)
    // associatedtype Value
    // init(stringInterpolationSegment value: Value)
    }

    protocol MyExpressibleByStringInterpolation {
        associatedtype Segment: InterpolationSegmentProtocol
        init(stringInterpolation: InterpolationSegment<Segment>...)
    }

    // Foo is constructed from a string interpolation containing only
    // String pieces and Foo.Segment pieces.
    struct Foo: MyExpressibleByStringInterpolation {
        struct Segment: InterpolationSegmentProtocol {
            init(stringInterpolationSegment value: Int) {}
            init(stringInterpolationSegment value: Double) {}
        }
        init(stringInterpolation: InterpolationSegment<Segment>...) {
            // ...
        }
    }

    let x: Foo = "abc\(3)def"
    // translated to
    Foo(stringInterpolation:
        .stringLiteral("abc"),
        .interpolatedValue(.init(stringInterpolationSegment: 3)),
        .stringLiteral("def"))

Jacob

···

On Mon, Aug 8, 2016 at 5:57 PM, Dave Abrahams via swift-evolution < swift-evolution@swift.org> wrote:

on Sat Jul 30 2016, Jacob Bandes-Storch <swift-evolution@swift.org> wrote:

> In the past, there has been some interest in refining the behavior of
> ExpressibleByStringInterpolation (née StringInterpolationConvertible),
for
> example:
>
> - Ability to *restrict the types that can be used* as interpolation
segments
> - Ability to *distinguish the string-literal segments* from interpolation
> segments whose type is String

Hi Jacob,

I see you've already filed a Jira for the second bullet. Can you file
one for the first one? We're going to redesign
ExpressibleByStringInterpolation for Swift 4 and solve these problems.

Thanks,

> Some prior discussions:
> - "StringInterpolationConvertible and StringLiteralConvertible
inheritance"
> https://lists.swift.org/pipermail/swift-evolution/
Week-of-Mon-20160516/017654.html
> - Sub-discussion in "Allow multiple conformances to the same protocol"
> https://lists.swift.org/pipermail/swift-evolution/
Week-of-Mon-20160606/020746.html
> - "StringInterpolationConvertible: can't distinguish between literal
> components and String arguments" https://bugs.swift.org/browse/SR-1260
> / rdar://problem/19800456&18681780
> - "Proposal: Deprecate optionals in string interpolation"
> https://lists.swift.org/pipermail/swift-evolution/
Week-of-Mon-20160516/018000.html
>
> About Swift 4, Chris wrote:
>
>> - *String re-evaluation:* String is one of the most important
>> fundamental types in the language. The standard library leads have
>> numerous ideas of how to improve the programming model for it, without
>> jeopardizing the goals of providing a unicode-correct-by-default model.
>> Our goal is to be better at string processing than Perl!
>
> I'd be interested in any more detail the team can provide on this. I'd
like
> to talk about string interpolation improvements, but it wouldn't be wise
to
> do so without keeping an eye towards possible regex/pattern-binding
syntax,
> and the String refinements that the stdlib team has in mind, if there's a
> chance they would affect interpolation.
>
> Discuss!
>
> -Jacob
> _______________________________________________
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>

--
-Dave

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Dave Abrahams) #6

Hi Dave,
I just filed https://bugs.swift.org/browse/SR-2303.

Brainstorming: is it important that the init(stringInterpolation:) and
init(stringInterpolationSegment:) requirements are on the same type?

As far as I'm concerned the design space is wide open.

Perhaps it would work to separate these two requirements, allowing the
segments to construct intermediate types, and only requiring the type
adopting ExpressibleByStringInterpolation to implement
init(stringInterpolation:).

This would be nice because it would be much easier for types which aren't
enums to conform to ExpressibleByStringInterpolation. In my auto layout
example (https://gist.github.com/jtbandes/9c1c25ee4996d2554375), the
ConstraintCollection type is only an enum because it has to provide all the
initializers, but it's strange that the cases are accessible publicly;

I'm not sure whether what you're describing is just a limitation in our
access control (a public type conforming to a protocol must expose all
of that protocol's requirements as public members). If so, maybe we
should fix *that*, rather than morphing designs to work around it.

That said, I don't really have time to think about the design of
ExpressibleByStringInterpolation in detail at the moment, as we're right
up against the Swift 3 ship date (sorry!) If you'd bring it up in a few
months I'm sure I'll have a lot more bandwidth for it.

Thanks,

···

on Mon Aug 08 2016, Jacob Bandes-Storch <swift-evolution@swift.org> wrote:

--
-Dave