# String hygiene

**URL:** <https://forums.swift.org/t/string-hygiene/12826>\
**Category:** Discussion\
**Created:** [May 18, 2018, 11:06pm UTC](https://forums.swift.org/t/string-hygiene/12826 "2018-05-18T23:06:08Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Erica\_Sadun](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/erica_sadun/32/243_2.png) [@Erica\_Sadun](https://forums.swift.org/u/Erica_Sadun)\
**Post date:** [May 18, 2018, 11:06pm UTC](https://forums.swift.org/t/string-hygiene/12826/1 "2018-05-18T23:06:08Z")

</div>

A thread to discuss trim/pad/center/etc. cc @Michael_Ilseman @armcknight @Dante-Broggi @tkrajacic

Updated gist to remove errant `mutating`: [trim.md · GitHub](https://gist.github.com/erica/e635a86405bda185d80f696949a4272d)

---

<div class="post-metadata">

**Author:** ![Michael\_Ilseman](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/michael_ilseman/32/44640_2.png) [@Michael\_Ilseman](https://forums.swift.org/u/Michael_Ilseman)\
**Post date:** [May 18, 2018, 11:12pm UTC](https://forums.swift.org/t/string-hygiene/12826/2 "2018-05-18T23:12:27Z")

</div>

What should be the result of the following?

```swift
" \u{301}abc\n\u{301}".trimmed()

```

---

<div class="post-metadata">

**Author:** ![armcknight](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/armcknight/32/3486_2.png) [@armcknight](https://forums.swift.org/u/armcknight)\
**Post date:** [May 18, 2018, 11:34pm UTC](https://forums.swift.org/t/string-hygiene/12826/3 "2018-05-18T23:34:04Z")

</div>

I think this would be a view over the characters of a string, so that the unicode grapheme clusters have already been resolved. I'm guessing that since swift gave me back ` ́abc\ń` for your example, the grapheme clustering pass happens before escape sequences like `\n`.

So in this case, you get back exactly ` ́abc\ń`, and any trimmable characters you added on either end of your input should be removed.

---

<div class="post-metadata">

**Author:** ![Michael\_Ilseman](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/michael_ilseman/32/44640_2.png) [@Michael\_Ilseman](https://forums.swift.org/u/Michael_Ilseman)\
**Post date:** [May 18, 2018, 11:50pm UTC](https://forums.swift.org/t/string-hygiene/12826/4 "2018-05-18T23:50:13Z")

</div>

If you implemented this using `trimmingCharacters`, then it's actually what [NSString returned](https://github.com/apple/swift/blob/7ff12fb88242377af9edad035486df37f4138b97/stdlib/public/SDK/Foundation/NSStringAPI.swift#L1122). ~~Since we'll want this for the standard library itself, we'll need an implementation that doesn't depend on Foundation~~ (edit: this is an implementation detail, not important to this discussion). We'll need to define the semantics in these situations.

> [@armcknight](#):
>
> So in this case, you get back exactly `́abc\ń` , and any trimmable characters you added on either end of your input should be removed.

More precisely:

```swift
Array(" \u{301}abc\n\u{301}".trimmed().unicodeScalars)
// => Array<Unicode.Scalar> = ["\u{0301}", "a", "b", "c", "\n", "\u{0301}"]

```

So it did drop the leading space. This means that:

```swift
let str = " \u{301}abc\n\u{301}"
str.count // => 6
str.trimmed().count // => 6
(str + str).count // => 12
(str + str.trimmed()).count // => 11

```

Which might not be intuitive, as trimming results in an isolated combining scalar. This is further complicated by the fact that Unicode pre-4.1 recommended the use of a space followed by a combining scalar as a technique for displaying isolated combining scalars. If someone was doing this, they certainly don't want it trimmed! But, this was a bad idea (because computers use spaces as separators ), and they reverted the suggestion in 4.1 and later.

I'm not arguing either way, I'm just curious what the proper semantics should be. There might not be a good answer, so we'd have to figure out what the least bad answer is.

---

<div class="post-metadata">

**Author:** ![armcknight](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/armcknight/32/3486_2.png) [@armcknight](https://forums.swift.org/u/armcknight)\
**Post date:** [May 19, 2018, 12:01am UTC](https://forums.swift.org/t/string-hygiene/12826/5 "2018-05-19T00:01:01Z")

</div>

Sorry, I'm not sure my example translated well. The first character is actually a space with an acute accent superset, so it combined your leading space and acute accent codepoint. Check out the unicode scalars for your example:

```
Array(" \u{301}abc\n\u{301}".unicodeScalars)
// => $R1: [Unicode.Scalar] = 7 values {
// [0] = U' '
// [1] = U'́'
// [2] = U'a'
// [3] = U'b'
// [4] = U'c'
// [5] = U'\n'
// [6] = U'́'
// }

```

Those accents are, for my lack of vocabulary, before the empty character, haha. It's the difference between these:

```
  7> " \u{0301}"
$R5: String = " ́"
  8> "\u{0301}"
$R6: String = "́"

```

Also though, ideally we could maintain some mapping from the original literal string definition to the unicode resolved version, and use those relationships to extract the correct ranges of characters from the original literal to return as a new set of backing unicode points. I hope that made any amount of sense.

---

<div class="post-metadata">

**Author:** ![armcknight](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/armcknight/32/3486_2.png) [@armcknight](https://forums.swift.org/u/armcknight)\
**Post date:** [May 19, 2018, 12:14am UTC](https://forums.swift.org/t/string-hygiene/12826/6 "2018-05-19T00:14:41Z")

</div>

Actually when I plug those resulting strings from my second block back into swift and extract unicode scalars again, that information seems to be preserved:

```
  9> let b = " ́"
b: String = " ́"
 11> Array(b.unicodeScalars)
$R8: [Unicode.Scalar] = 2 values {
  [0] = U' '
  [1] = U'́'
}
 12> let c = "́"
c: String = "́"
 13> Array(c.unicodeScalars)
$R9: [Unicode.Scalar] = 1 value {
  [0] = U'́'
}

```

---

<div class="post-metadata">

**Author:** ![DevAndArtist](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/devandartist/32/287_2.png) [@DevAndArtist](https://forums.swift.org/u/DevAndArtist)\
**Post date:** [May 19, 2018, 6:06am UTC](https://forums.swift.org/t/string-hygiene/12826/7 "2018-05-19T06:06:28Z")

</div>

@Erica_Sadun do we really want it to be a in `Foundation` or do we want to generalize it without any dependencies? If it's meant for the stdlib then the implementation should not rely on `Foundation`.

---

<div class="post-metadata">

**Author:** ![jawbroken](https://avatars.discourse-cdn.com/v4/letter/j/cc9497/32.png) [@jawbroken](https://forums.swift.org/u/jawbroken)\
**Post date:** [May 20, 2018, 6:16am UTC](https://forums.swift.org/t/string-hygiene/12826/8 "2018-05-20T06:16:14Z")

</div>

It's clear to me from @Michael_Ilseman's example that the `NSString` implementation is incorrect for Swift anyway, regardless of the dependency issues. `" \u{301}"` is not whitespace when processed as a single Swift `Character`, and `String` functionality should generally operate on the `Character` level. Are there any exceptions in the current `String` API, setting aside the `unicodeScalars`, `utf16` and `utf8` views? I guess you could say technically some of the edge cases with `append`, etc. because adjacent `Characters` may merge.

---

<div class="post-metadata">

**Author:** ![Erica\_Sadun](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/erica_sadun/32/243_2.png) [@Erica\_Sadun](https://forums.swift.org/u/Erica_Sadun)\
**Post date:** [May 20, 2018, 8:30pm UTC](https://forums.swift.org/t/string-hygiene/12826/9 "2018-05-20T20:30:23Z")

</div>

Can we use CharacterSet or does that fit into "no, that needs a redesign too"?

---

<div class="post-metadata">

**Author:** ![Michael\_Ilseman](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/michael_ilseman/32/44640_2.png) [@Michael\_Ilseman](https://forums.swift.org/u/Michael_Ilseman)\
**Post date:** [May 20, 2018, 8:33pm UTC](https://forums.swift.org/t/string-hygiene/12826/10 "2018-05-20T20:33:03Z")

</div>

There is nothing wrong with using `CharacterSet` if it gives the semantics we want. What I'm looking for is: what semantics do we want, and why?

---

<div class="post-metadata">

**Author:** ![pvieito](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/pvieito/32/33613_2.png) [@pvieito](https://forums.swift.org/u/pvieito)\
**Post date:** [May 20, 2018, 8:40pm UTC](https://forums.swift.org/t/string-hygiene/12826/11 "2018-05-20T20:40:30Z")

</div>

IMHO this API (like [`trimmingCharacters(in:)`](https://developer.apple.com/documentation/swift/stringprotocol/2923360-trimmingcharacters)) fits in `Foundation`, not in the Standard Library.

---

<div class="post-metadata">

**Author:** ![Michael\_Ilseman](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/michael_ilseman/32/44640_2.png) [@Michael\_Ilseman](https://forums.swift.org/u/Michael_Ilseman)\
**Post date:** [May 20, 2018, 9:02pm UTC](https://forums.swift.org/t/string-hygiene/12826/12 "2018-05-20T21:02:44Z")

</div>

> [@jawbroken](#):
>
> It's clear to me from @Michael_Ilseman's example that the `NSString` implementation is incorrect for Swift anyway, regardless of the dependency issues. `" \u{301}"` is not whitespace when processed as a single Swift `Character` , and `String` functionality should generally operate on the `Character` level. Are there any exceptions in the current `String` API, setting aside the `unicodeScalars` , `utf16` and `utf8` views? I guess you could say technically some of the edge cases with `append` , etc. because adjacent `Characters` may merge.

I wouldn't say that it is incorrect for Swift. I would say that it seems inconsistent with a view of String as a collection of graphemes. These semantics may be perfect for, e.g., `String.UnicodeScalarView.trimmed()`.

Degenerate graphemes are a reality that we have to deal with, and we (unfortunately) cannot have a nice algebraic String because of them. But, it does seem odd for String to have a common-use API that produces them.

> [@armcknight](#):
>
> I hope that made any amount of sense.

I don't think I communicated my point well. My example was meant to demonstrate some corner cases that we need to have a strategy for, even if a random package's implementation doesn't care about the details. I see (at least) 3 potential results, each of which has some upsides and downsides. The question is what the result _should_ be.

I’ll use \u{020} to represent a space explicitly.

`“\u{020}\u{301}abc\n\u{301}”.trimmed()` could yield:

1. “abc”, under the view that String APIs operate on graphemes and a grapheme leading with whitespace is considered whitespace.
2. “\u{020}\u{301}abc\n\u{301}”, under the view that String APIs operate on graphemes and a grapheme with a combining character is **not** whitespace.
3. “\u{301}abc\n\u{301}”, which results in a degenerate leading grapheme.

I feel like #3 makes sense for a `trimmed()` on the UnicodeScalarView or the the code unit views. For `String`, we need to figure out which of these 3 (or perhaps an unknown 4th option) is the least harmful behavior.

---

<div class="post-metadata">

**Author:** ![Erica\_Sadun](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/erica_sadun/32/243_2.png) [@Erica\_Sadun](https://forums.swift.org/u/Erica_Sadun)\
**Post date:** [May 20, 2018, 10:24pm UTC](https://forums.swift.org/t/string-hygiene/12826/13 "2018-05-20T22:24:44Z")

</div>

I've updated [the gist](https://gist.github.com/erica/e635a86405bda185d80f696949a4272d). Is this more like you want? I avoided `CharacterSet` but that can be easily re-introduced.

(I went with I believe #2, but can again adjust accordingly)

---

<div class="post-metadata">

**Author:** ![kiel](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/kiel/32/1392_2.png) [@kiel](https://forums.swift.org/u/kiel)\
**Post date:** [May 21, 2018, 12:48am UTC](https://forums.swift.org/t/string-hygiene/12826/14 "2018-05-21T00:48:36Z")

</div>

I'm wondering if the trimming functions should return a `String` or `Substring`. For example, if an algorithm was trimming a lot of strings, might memory usage increase undesirably?

I've no data for a warranted concern, just throwing the question out there. I would gladly accept the response that returning a `Substring` would be a case of premature optimisation. Perhaps we could find some data from usage of the Core Foundation APIs or the standard libraries of other general purpose languages?

If those sources show undesirable memory usage, maybe the proposal should include a range API for locating the desired text and the trimming functions use those?

---

<div class="post-metadata">

**Author:** ![kiel](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/kiel/32/1392_2.png) [@kiel](https://forums.swift.org/u/kiel)\
**Post date:** [May 21, 2018, 12:52am UTC](https://forums.swift.org/t/string-hygiene/12826/15 "2018-05-21T00:52:35Z")

</div>

Another thought: parameterise the `Set<Character>` so users can trim specific characters, such as spaces but not new lines? Obviously, it could still use `Trim.whitespaceAndNewlineCharacters` by default.

---

<div class="post-metadata">

**Author:** ![Erica\_Sadun](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/erica_sadun/32/243_2.png) [@Erica\_Sadun](https://forums.swift.org/u/Erica_Sadun)\
**Post date:** [May 21, 2018, 12:53am UTC](https://forums.swift.org/t/string-hygiene/12826/16 "2018-05-21T00:53:28Z")

</div>

I wrote it both ways for the non-mutating version and could support either consensus.

---

<div class="post-metadata">

**Author:** ![kiel](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/kiel/32/1392_2.png) [@kiel](https://forums.swift.org/u/kiel)\
**Post date:** [May 21, 2018, 1:26am UTC](https://forums.swift.org/t/string-hygiene/12826/17 "2018-05-21T01:26:19Z")

</div>

A few more thoughts.

The `Trim` enum has option semantics. If that's correct, maybe it should inherit from `OptionSet` instead? In that case, the `.full` case/option is redundant because `full` is just the set with `start` and `end` options specified. Specifying `[.start, .end]` better corresponds to the concept of trimming, too (`full` feels awkward, like an attempt to say `bothSides` (or `bothEnds`) but with less characters).

Trivially, parameterising the ends of the string to trim as a `OptionSet` saves you the cost of instantiating both `[.start, .full]` and `[.end, .full]` during the course of the implementation. Again, of course, the default value for this `endsToTrim` (or whatever) parameter would obviously be `[.start, .end]`.

(It just occurred to me: my sincerest apologies if this is not the right place to give feedback about the trimming gist, but not sure where else).

---

<div class="post-metadata">

**Author:** ![Erica\_Sadun](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/erica_sadun/32/243_2.png) [@Erica\_Sadun](https://forums.swift.org/u/Erica_Sadun)\
**Post date:** [May 21, 2018, 1:34am UTC](https://forums.swift.org/t/string-hygiene/12826/18 "2018-05-21T01:34:16Z")

</div>

I added the directionality for @sbromberger. My first implementation of the side-trim used option sets. Then I started using the type and I hated how it looks, so I re-did everything to not use option sets, justifying the readability for the two specialized use cases: start and end. I defaulted it out with "full" so that can be elided for normal use.

Compare:

```swift
string.trimmed()
string.trimmed(from: .start)
string.trimmed(from: .end)

```

with

```swift
string.trimmed(from: [.start])
string.trimmed(from: [.end])

```

and possibly this horror[1]:

```swift
string.trimmed(from: [.start, .end])

```

* * *

[1] I apply the "Lattner Rule of Gross™" here. cc @Chris_Lattner3

---

<div class="post-metadata">

**Author:** ![kiel](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/kiel/32/1392_2.png) [@kiel](https://forums.swift.org/u/kiel)\
**Post date:** [May 21, 2018, 1:44am UTC](https://forums.swift.org/t/string-hygiene/12826/19 "2018-05-21T01:44:06Z")

</div>

`OptionSet` doesn't seem to require the use of the array literal notation for a single option. For example, I'd expect `string.trimmed(from: .start)` and `string.trimmed(from: [.start])` to compile and achieve the same thing.

If the default is `[.start, .end]`, there'd be less chance of seeing such horrors at the call site.

---

<div class="post-metadata">

**Author:** ![TellowKrinkle](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/tellowkrinkle/32/3370_2.png) [@TellowKrinkle](https://forums.swift.org/u/TellowKrinkle)\
**Post date:** [May 21, 2018, 2:49am UTC](https://forums.swift.org/t/string-hygiene/12826/20 "2018-05-21T02:49:19Z")

</div>

Quick question, is there any reason `trimmed` couldn't return a `Substring`? It only removes from the front and end of the string, so is there any time it would have to modify the original string?

[Next page](https://forums.swift.org/t/string-hygiene/12826.md?page=2)