# OrderedDictionary decoding

**URL:** <https://forums.swift.org/t/ordereddictionary-decoding/66829>\
**Category:** Collections\
**Tags:** dictionary, decoding, decodable\
**Created:** [August 18, 2023, 6:15pm UTC](https://forums.swift.org/t/ordereddictionary-decoding/66829 "2023-08-18T18:15:56Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Nostromo](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/nostromo/32/44325_2.png) [@Nostromo](https://forums.swift.org/u/Nostromo)\
**Post date:** [August 18, 2023, 6:15pm UTC](https://forums.swift.org/t/ordereddictionary-decoding/66829/1 "2023-08-18T18:15:56Z")

</div>

Hello, I created the bug here: [OrderedDictionary decoding. · Issue #310 · apple/swift-collections · GitHub](https://github.com/apple/swift-collections/issues/310)  
But now I'm curious, what is the correct approach to decoding here.  
Should it be fully compatible with an original Dictionary or be implemented like now (expects `unkeyedContainer`)?  
This is probably related to the nature of the KeyedDecodingContainer's keys, which are also not ordered.

---

<div class="post-metadata">

**Author:** ![Morten\_Bek\_Ditlevsen](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/morten_bek_ditlevsen/32/28133_2.png) [@Morten\_Bek\_Ditlevsen](https://forums.swift.org/u/Morten_Bek_Ditlevsen)\
**Post date:** [August 18, 2023, 6:35pm UTC](https://forums.swift.org/t/ordereddictionary-decoding/66829/2 "2023-08-18T18:35:53Z")

</div>

To me it sounds reasonable that encoding and decoding preserves the order as it does today, so my guess is that it is intentional and not a bug.

---

<div class="post-metadata">

**Author:** ![Jon\_Shier](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/jon_shier/32/921_2.png) [@Jon\_Shier](https://forums.swift.org/u/Jon_Shier)\
**Post date:** [August 18, 2023, 9:27pm UTC](https://forums.swift.org/t/ordereddictionary-decoding/66829/3 "2023-08-18T21:27:16Z")

</div>

This is indeed expected, as it's technically correct that many encodings don't support ordered dictionaries (like JSON, technically). However, that makes the default coding behavior rather useless, as the vast majority of people would simply expect to be able to preserve ordering into a dictionary (like JSON, practically). Suggested workaround is to wrap your `OrderedDictionary` into a type that preserves order as expected.

---

<div class="post-metadata">

**Author:** ![wadetregaskis](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/wadetregaskis/32/8494_2.png) [@wadetregaskis](https://forums.swift.org/u/wadetregaskis)\
**Post date:** [August 18, 2023, 9:34pm UTC](https://forums.swift.org/t/ordereddictionary-decoding/66829/4 "2023-08-18T21:34:32Z")

</div>

It seems reasonable to me that it should use the same encoded form as `Dictionary`.

[The JSON specification](https://www.ecma-international.org/wp-content/uploads/ECMA-404_2nd_edition_december_2017.pdf) states:

> The JSON syntax … does not assign any significance to the ordering of name/value pairs. These are all semantic considerations that may be defined by JSON processors or in specifications defining specific uses of JSON for data interchange.

Generic decoding can't make any assumptions about the use, but users of the JSON may need to re-encode it at some point and may require that order be preserved (per their "specific use" semantics, as the standard says), therefore `OrderedDictionary` must be an option (otherwise, what is?).

---

<div class="post-metadata">

**Author:** ![tera](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/tera/32/38201_2.png) [@tera](https://forums.swift.org/u/tera)\
**Post date:** [August 19, 2023, 12:34am UTC](https://forums.swift.org/t/ordereddictionary-decoding/66829/5 "2023-08-19T00:34:32Z")

</div>

Ditto here, I'd vote for a change where encoded OrderedDictionary representation is normal JSON object so this test works:

```swift
let x: [String: String] = ["foo" : "1", "bar" : "2", "baz" : "2"]
let data = try! JSONEncoder().encode(x)
let y = try! JSONDecoder().decode(OrderedDictionary<String, String>.self, from: data)
let data2 = try! JSONEncoder().encode(y)
let z = try! JSONDecoder().decode([String: String].self, from: data2)

```

specifically IMHO this should be true:

```swift
OrderedDictionary -> JSON -> OrderedDictionary // preserves order
OrderedDictionary -> JSON -> Dictionary // order is n/a obviously
Dictionary -> JSON -> OrderedDictionary // ordered "as per JSON"

```

If it's too late to change the current default behaviour maybe we can opt in by specifying an option on encoder/decoder object similar to how we do it with key decoding strategy and output formatting.

---

<div class="post-metadata">

**Author:** ![itaiferber](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/itaiferber/32/33840_2.png) [@itaiferber](https://forums.swift.org/u/itaiferber)\
**Post date:** [August 19, 2023, 4:00am UTC](https://forums.swift.org/t/ordereddictionary-decoding/66829/6 "2023-08-19T04:00:54Z")

</div>

> [@wadetregaskis](#):
>
> [The JSON specification](https://www.ecma-international.org/wp-content/uploads/ECMA-404_2nd_edition_december_2017.pdf) states:

> [@tera](#):
>
> OrderedDictionary representation is normal JSON object so this test works:

Keep in mind that `OrderedDictionary`'s `Codable` conformance extends to _all_ `Codable` formats, and not just JSON. Using an `UnkeyedEncodingContainer` is entirely correct for maintaining the order of the dictionary, even if in certain formats it produces something that doesn't _look_ like a dictionary; the key is that the _runtime_ representation of the `OrderedDictionary` is the same.

> [@wadetregaskis](#):
>
> > The JSON syntax … does not assign any significance to the ordering of name/value pairs.

For JSON specifically, the spec indeed places absolutely no restrictions on the meaning of key-value pairs in dictionaries — which means that _in practice_, most JSON parsers and encoders don't preserve order at all, because it usually requires more work to do so. This is at least partially due to the fact that in many languages, dictionaries are themselves unordered, and decoding is very often eager, rehydrating an entire payload into language-native collections. (This is part of the reason why, at the moment, `JSONDecoder` can't preserve ordering: because it uses `JSONSerialization` under the hood, it gets back fully-hydrated `Dictionary` values back, so ordering has already been lost by the time that `JSONDecoder` gets the data. This is something that may change with the upcoming `swift-foundation` work and rewrite.)

This means that if you're using JSON to communicate with any sort of service, you have to be very careful because can't really rely on "ordered dictionary " → service → "ordered dictionary" producing the same results. (Unless you encode the dictionary as an array, in which case it can't ever reasonably be reordered, unless intentionally.)

> [@tera](#):
>
> ```swift
> OrderedDictionary -> JSON -> OrderedDictionary // preserves order
> OrderedDictionary -> JSON -> Dictionary // order is n/a obviously
> Dictionary -> JSON -> OrderedDictionary // ordered "as per JSON"
> 
> ```

This is actually possible today, without changing the encoded representation of `OrderedDictionary` at all:

1. `OrderedDictionary` → JSON → `OrderedDictionary` is obviously possible
2. `OrderedDictionary` → JSON → `Dictionary` is already possible: `Dictionary` [already knows](https://github.com/apple/swift/blob/594469abf1d1d11386a2f9f4cf215af1dc4ae0e7/stdlib/public/core/Codable.swift#L5712)\* how to decode from an array of key-value pairs, in the same way that `OrderedDictionary` encodes
3. `Dictionary` → JSON → `OrderedDictionary` is also possible, by having `OrderedDictionary.init(from:)` check for a keyed container, and if one is present, decode from _that_ (i.e., the reverse of what `Dictionary` does). The caveat here is that `OrderedDictionary` has no choice but to trust that the order of the keyed given by the keyed contain is the same as was in the original data, which currently is _not_ the case (and in general, decoders for a variety of formats can't make that promise) — but for `JSONDecoder` specifically, could be

\*The tricky thing is that at the moment, if you try to decode a dictionary keyed by either `String` or `Int`, `Dictionary` assumes that you must have a keyed container representation, so you'll get an error if you try to decode `OrderedDictionary<String, ...>` as `Dictionary<String, ...>`, but this can be loosened. You can also work around this by using a wrapper key type that encodes/decodes as a `String` (even a `RawRepresentable` type should work), so this does work:

```swift
import Foundation
import OrderedCollections

let orderedDict: OrderedDictionary = ["one": 1, "two": 2, "three": 3]
print(orderedDict) // => [one: 1, two: 2, three: 3]

let encoded = try! JSONEncoder().encode(orderedDict)
print(String(data: encoded, encoding: .utf8)!) // => ["one",1,"two",2,"three",3]

struct Key: RawRepresentable, Hashable, Decodable {
    let rawValue: String
}

let decoded = try! JSONDecoder().decode([Key: Int].self, from: encoded)
print(decoded) // => [Key(rawValue: "three"): 3, Key(rawValue: "one"): 1, Key(rawValue: "two"): 2]

```

> [@tera](#):
>
> I'd vote for a change where encoded OrderedDictionary representation

Also keep in mind that changing the encoded representation of a type isn't something you can do loosely (or in some cases, at all). Once the encoded representation of a type is public, that representation can be written out to disk and persisted indefinitely; changing the representation means that older software versions (the app/package/whatever), unless written in a very forward-thinking manner, will choke on the new data format.

The bar for breaking backwards compatibility in data serialization (in the general case, at least) is exceedingly high, because you risk throwing away support for otherwise perfectly valid software which can't necessarily be updated.

---

<div class="post-metadata">

**Author:** ![tera](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/tera/32/38201_2.png) [@tera](https://forums.swift.org/u/tera)\
**Post date:** [August 19, 2023, 9:36pm UTC](https://forums.swift.org/t/ordereddictionary-decoding/66829/7 "2023-08-19T21:36:00Z")

</div>

> [@itaiferber](#):
>
> Using an `UnkeyedEncodingContainer` is entirely correct for maintaining the order of the dictionary, even if in certain formats it produces something that doesn't _look_ like a dictionary; the key is that the _runtime_ representation of the `OrderedDictionary` is the same.

Maybe we are missing an abstraction here, like "SortedKeyedContainer". I see you point of how we'd got to what we have, just the end result doesn't look ideal.

> [@itaiferber](#):
>
> `Dictionary` → JSON → `OrderedDictionary` is also possible, by having `OrderedDictionary.init(from:)` check for a keyed container, and if one is present, decode from _that_ (i.e., the reverse of what `Dictionary` does).

Please clarify: it is possible to implement in the standard library just not implemented now, and could be considered in the future, or you mean something else?

> [@itaiferber](#):
>
> \*The tricky thing is that at the moment, if you try to decode a dictionary keyed by either `String` or `Int`, `Dictionary` assumes that you must have a keyed container representation, so you'll get an error if you try to decode `OrderedDictionary<String, ...>` as `Dictionary<String, ...>`, but this can be loosened.

Heh, "String" key was the thing I checked, so I didn't notice it could work.

OTOH, it does feel somewhat wrong that I can encode an array: ["foo", "1", "bar", "2"] into JSON and read it back as a `Dictionary`. I'd never assume that's possible and would consider that a bug if you didn't tell this is by design.

> [@itaiferber](#):
>
> Also keep in mind that changing the encoded representation of a type isn't something you can do loosely (or in some cases, at all). ... The bar for breaking backwards compatibility in data serialization (in the general case, at least) is exceedingly high...

That's why I said:

> [@tera](#):
>
> If it's too late to change the current default behaviour maybe we can opt in by specifying an option on encoder/decoder object similar to how we do it with key decoding strategy and output formatting.

IMHO, if we have the facilities like changing the key names completely or key capitalisation between snake/camel case) we could as well have this other option of what container to choose.

> [@itaiferber](#):
>
> This means that if you're using JSON to communicate with any sort of service, you have to be very careful because can't really rely on "ordered dictionary " → service → "ordered dictionary" producing the same results.

Well, passing dictionary formatted key value pairs through a text JSON file and back would preserve the order, but it of course would be naïve to assume that passing it through an arbitrary "service" that could change the order by re-encoding JSON would keep the order intact. That's expected. That's where the option of array serialisation would be helpful (be it the default or the opt-in behaviour).

---

<div class="post-metadata">

**Author:** ![itaiferber](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/itaiferber/32/33840_2.png) [@itaiferber](https://forums.swift.org/u/itaiferber)\
**Post date:** [August 21, 2023, 2:07pm UTC](https://forums.swift.org/t/ordereddictionary-decoding/66829/8 "2023-08-21T14:07:37Z")

</div>

> [@tera](#):
>
> > [@itaiferber](#):
> >
> > `Dictionary` → JSON → `OrderedDictionary` is also possible, by having `OrderedDictionary.init(from:)` check for a keyed container, and if one is present, decode from _that_ (i.e., the reverse of what `Dictionary` does).
> 
> Please clarify: it is possible to implement in the standard library just not implemented now, and could be considered in the future, or you mean something else?

What I mean is that it's possible to update `OrderedDictionary.init(from:)` to try to fetch an unkeyed container, as it does today — if one is present, then it can decode with the same approach that it already does. However, if attempting to fetch an unkeyed container fails due to a type mismatch, it can attempt to fetch a keyed container instead, and attempt to decode from the keyed container (like `Dictionary` currently does).

Because `OrderedDictionary` lives in [swift-collections](https://github.com/apple/swift-collections/), no update to the standard library is necessary; it would just be a package update.

> [@tera](#):
>
> That's why I said:
> 
> > If it's too late to change the current default behaviour maybe we can opt in by specifying an option on encoder/decoder object similar to how we do it with key decoding strategy and output formatting.

Indeed, I just wanted to reinforce that.

> [@tera](#):
>
> IMHO, if we have the facilities like changing the key names completely or key capitalisation between snake/camel case) we could as well have this other option of what container to choose.

Sure, encoders and decoders can offer this option! Nothing stopping that — though it's outside the purview of the standard library. Foundation can choose to offer toggles for what to do with ordered collections on `JSONEncoder`/`JSONDecoder` (though it does get a bit tricky, because Foundation can't import `swift-collections` to know about `OrderedDictionary`, so some protocol conformances might be necessary to make this happen).

> [@tera](#):
>
> but it of course would be naïve to assume that passing it through an arbitrary "service" that could change the order by re-encoding JSON would keep the order intact. That's expected.

Assertions about naïveté aside, the definition of "service" here can be pretty wide. Passing the data to older versions of your app using an stdlib/Foundation that doesn't support ordered containers, passing your ordered dictionary to any encoder/decoder that doesn't care to preserve order, storing ordered data on disk but later attempting to manipulate it with a tool that doesn't support preserving ordering — all are implicit ways of unexpectedly losing ordering without necessarily being aware of it.

It's not a bad thing, necessarily, and it doesn't at all mean that maintaining ordering during decoding shouldn't be supported, but it does mean you need to be very careful: you can only really assume your `OrderedDictionary` means anything if you know that _you_ wrote the data out with ordered keys and are reading it back as-is, without anything else having touched it. Pretty much any other scenario is up for grabs.

---

<div class="post-metadata">

**Author:** ![NeedleInAJayStack](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/needleinajaystack/32/19881_2.png) [@NeedleInAJayStack](https://forums.swift.org/u/NeedleInAJayStack)\
**Post date:** [December 20, 2024, 7:02am UTC](https://forums.swift.org/t/ordereddictionary-decoding/66829/9 "2024-12-20T07:02:30Z")

</div>

I'd love to reopen this discussion if possible. As a maintainer of the server-side GraphQL implementation (which does have [a requirement for specific ordering of encoded JSON fields](https://spec.graphql.org/October2021/#sec-Serialized-Map-Ordering)), I'd love to have a more standard solution. Currently, our solution has involved [copying a large portion of the JSONEncoder code](https://github.com/GraphQLSwift/GraphQL/blob/main/Sources/GraphQL/Map/GraphQLJSONEncoder.swift) to avoid any usage of unordered dictionaries, which has obvious maintainability problems.

> If it's too late to change the current default behaviour maybe we can opt in by specifying an option on encoder/decoder object similar to how we do it with key decoding strategy and output formatting.

Is this seen as the most viable path forward for control over encoding order? I'd be happy to contribute if so. Or do you see alternatives for my use case?