Feedback on SE-0166 and SE-0167


#1

Hello,

I want to provide real-life feedback for the Swift Archival & Serialization (SE-0166) and Swift Encoders (SE-0167) proposals that currently ship in Swift 4 snapshots.

The context: I'm the author of GRDB.swift [1], a SQLite library that, among other goals, aims at easing the conversion between database rows and custom models (structs and class hierarchies):

  // Sample code
  let arthur = Player(name: "Arthur", score: 100)
  try arthur.insert(db)
  print(arthur.id)
  
  let topPlayers = try Player
    .order(Column("score").desc)
    .limit(10)
    .fetchAll(db) // [Player]

Due to the lack of any introspection in Swift, GRDB currently wants you to perform explicit conversion:

  struct Player {
    var id: Int64?
    let name: String
    let score: Int
  }
  
  extension Player : RowConvertible {
    init(row: Row) {
      id = row.value(named: "id")
      name = row.value(named: "name")
      score = row.value(named: "score")
    }
  }
  
  extension Player : TableMapping, MutablePersistable {
    static let databaseTableName = "player"
    var persistentDictionary: [String: DatabaseValueConvertible?] {
      return ["id": id, "name": name, "score: score]
    }
  }

That's enough, but that's still too much.

SE-0166 and SE-0167 sound like the promise that some boilerplate code could be automatically generated.

Along with JSONDecoder and PListDecoder, let's introduce DatabaseRowDecoder! The current state of the work is at https://github.com/groue/GRDB.swift/tree/Swift4

At first, it's very satisfying. Decodable keeps some of it promises:

  struct Player : RowConvertible, Decodable {
    static let databaseTableName = "player"
    var id: Int64?
    let name: String
    let score: Int
  }
  
  // Yeah, no more extra code necessary for this to work!
  let topPlayers = try Player
    .order(Column("score").desc)
    .limit(10)
    .fetchAll(db)

But there are some issues.

### Issue 1: SE-0166/0167 merge the concepts of keyed objects and values

This is a problem. Let's take this example:

  enum Color: Int, Codable {
    case blue, green, red
  }
  
  struct Flower : RowConvertible, Decodable {
    let name: String
    let color: Color
  }
  
The way to decode a color comes from KeyedDecodingContainerProtocol:

  protocol KeyedDecodingContainerProtocol {
    func decode<T>(_ type: T.Type, forKey key: Key) throws -> T where T : Decodable
    func decodeIfPresent<T>(_ type: T.Type, forKey key: Key) throws -> T? where T : Decodable
  }

But the ability to decode a Color from a database row comes from the DatabaseValueConvertible, which I can't invoke since I can't test if type T conforms to this protocol:

  struct RowKeyedDecodingContainer<Key: CodingKey>: KeyedDecodingContainerProtocol {
    let row: Row
    
    // Not OK: no support for values
    func decode<T>(_ type: T.Type, forKey key: Key) throws -> T where T : Decodable {
      if <T conforms to DatabaseValueConvertible> {
        let databaseValue: DatabaseValue = row.value(named: key.stringValue)
        return T.fromDatabaseValue(databaseValue)
      } else { ... }
    }
  }

So the current state of the Codable library disallow GRDB from supporting value properties which are not the trivial Int, Int32, etc. Of course, GRDB itself makes it possible, with explicit user code. But we're talking about removing boilerplate and relying on the code generation that Codable is blessed with, here. We're talking about sharing the immense privilege that Codable is blessed with.

However, if I can't decode **values**, I can still decode **complex keyed objects** (in this case the row behaves like a hierarchical container - a concept already present in GRDB and allows it to consume complex rows like results of joins):

  struct Book : RowConvertible, Decodable { ... }
  struct Author : RowConvertible, Decodable { ... }
  struct Pair : RowConvertible, Decodable {
    let book: Book
    let author: Author
  }
  
  struct RowKeyedDecodingContainer<Key: CodingKey>: KeyedDecodingContainerProtocol {
    let row: Row
  
    // OK, support for other decodable objects
    func decode<T>(_ type: T.Type, forKey key: Key) throws -> T where T : Decodable {
      if let scopedRow = row.scoped(on: key.stringValue) {
        return try T(from: RowDecoder(row: scopedRow, codingPath: codingPath + [key]))
      } else {
        throw DecodingError.keyNotFound(key, DecodingError.Context(codingPath: codingPath, debugDescription: "missing scope"))
      }
    }
  }

Yet this use case is much less frequent.

Is it possible to workaround this problem? Did I miss something?

### Issue 2: Encodable can not be used to derive other persistence strategies.

The use case here is to derive other types of persistence from Encodable (and take profit from the compiler-generated code).

For example, I want to write:

  extension MutablePersistable where Self: Encodable {
    // Required by MutablePersistable
    var persistentDictionary: [String: DatabaseValueConvertible?] {
      return ...
    }
  }

If it were possible, we could get the full picture, with all boilerplate removed:
  
  // Wouldn't it be great?
  struct Player : RowConvertible, MutablePersistable, Codable {
    static let databaseTableName = "player"
    var id: Int64?
    let name: String
    let score: Int
  }

  let arthur = Player(name: "Arthur", score: 100)
  try arthur.insert(db)
  print(arthur.id)

  let topPlayers = try Player
    .order(Column("score").desc)
    .limit(10)
    .fetchAll(db) // [Player]

Unfortunately, it's impossible: the Encodable protocol doesn't allow iteration on the coding keys. I can't generate anything useful.

Again, is it possible to workaround this problem? Did I miss something?

Thanks for your attention,
Gwendal Roué

[1] https://github.com/groue/GRDB.swift


(Itai Ferber) #2

Hi Gwendal,

First, thanks for taking the time to adopt some of our new work. Feedback is always good, and we appreciate it.
Some comments inline.

Hello,

I want to provide real-life feedback for the Swift Archival & Serialization (SE-0166) and Swift Encoders (SE-0167) proposals that currently ship in Swift 4 snapshots.

The context: I'm the author of GRDB.swift [1], a SQLite library that, among other goals, aims at easing the conversion between database rows and custom models (structs and class hierarchies):

  // Sample code
  let arthur = Player(name: "Arthur", score: 100)
  try arthur.insert(db)
  print(arthur.id)
  
  let topPlayers = try Player
    .order(Column("score").desc)
    .limit(10)
    .fetchAll(db) // [Player]

Due to the lack of any introspection in Swift, GRDB currently wants you to perform explicit conversion:

  struct Player {
    var id: Int64?
    let name: String
    let score: Int
  }
  
  extension Player : RowConvertible {
    init(row: Row) {
      id = row.value(named: "id")
      name = row.value(named: "name")
      score = row.value(named: "score")
    }
  }
  
  extension Player : TableMapping, MutablePersistable {
    static let databaseTableName = "player"
    var persistentDictionary: [String: DatabaseValueConvertible?] {
      return ["id": id, "name": name, "score: score]
    }
  }

That's enough, but that's still too much.

SE-0166 and SE-0167 sound like the promise that some boilerplate code could be automatically generated.

Along with JSONDecoder and PListDecoder, let's introduce DatabaseRowDecoder! The current state of the work is at https://github.com/groue/GRDB.swift/tree/Swift4

At first, it's very satisfying. Decodable keeps some of it promises:

  struct Player : RowConvertible, Decodable {
    static let databaseTableName = "player"
    var id: Int64?
    let name: String
    let score: Int
  }
  
  // Yeah, no more extra code necessary for this to work!
  let topPlayers = try Player
    .order(Column("score").desc)
    .limit(10)
    .fetchAll(db)

But there are some issues.

### Issue 1: SE-0166/0167 merge the concepts of keyed objects and values

This is a problem. Let's take this example:

  enum Color: Int, Codable {
    case blue, green, red
  }
  
  struct Flower : RowConvertible, Decodable {
    let name: String
    let color: Color
  }
  
The way to decode a color comes from KeyedDecodingContainerProtocol:

  protocol KeyedDecodingContainerProtocol {
    func decode<T>(_ type: T.Type, forKey key: Key) throws -> T where T : Decodable
    func decodeIfPresent<T>(_ type: T.Type, forKey key: Key) throws -> T? where T : Decodable
  }

But the ability to decode a Color from a database row comes from the DatabaseValueConvertible, which I can't invoke since I can't test if type T conforms to this protocol:

  struct RowKeyedDecodingContainer<Key: CodingKey>: KeyedDecodingContainerProtocol {
    let row: Row
    
    // Not OK: no support for values
    func decode<T>(_ type: T.Type, forKey key: Key) throws -> T where T : Decodable {
      if <T conforms to DatabaseValueConvertible> {
        let databaseValue: DatabaseValue = row.value(named: key.stringValue)
        return T.fromDatabaseValue(databaseValue)
      } else { ... }
    }
  }

What is preventing you from doing this check? I looked at GRDB code, but AFAICT, DatabaseValueConvertible has not associated types, nor does SQLExpression, which it adopts.

You should be able to express this code as:

    if T.self is DatabaseValueConvertible.Type {
        let databaseValue: DatabaseValue = row.value(named: key.stringValue)
        return (T.self as! DataBaseValueConvertible.Type).fromDatabaseValue(databaseValue) as! T
    } else { … }

So the current state of the Codable library disallow GRDB from supporting value properties which are not the trivial Int, Int32, etc. Of course, GRDB itself makes it possible, with explicit user code. But we're talking about removing boilerplate and relying on the code generation that Codable is blessed with, here. We're talking about sharing the immense privilege that Codable is blessed with.

However, if I can't decode **values**, I can still decode **complex keyed objects** (in this case the row behaves like a hierarchical container - a concept already present in GRDB and allows it to consume complex rows like results of joins):

  struct Book : RowConvertible, Decodable { ... }
  struct Author : RowConvertible, Decodable { ... }
  struct Pair : RowConvertible, Decodable {
    let book: Book
    let author: Author
  }
  
  struct RowKeyedDecodingContainer<Key: CodingKey>: KeyedDecodingContainerProtocol {
    let row: Row
  
    // OK, support for other decodable objects
    func decode<T>(_ type: T.Type, forKey key: Key) throws -> T where T : Decodable {
      if let scopedRow = row.scoped(on: key.stringValue) {
        return try T(from: RowDecoder(row: scopedRow, codingPath: codingPath + [key]))
      } else {
        throw DecodingError.keyNotFound(key, DecodingError.Context(codingPath: codingPath, debugDescription: "missing scope"))
      }
    }
  }

Yet this use case is much less frequent.

Is it possible to workaround this problem? Did I miss something?

### Issue 2: Encodable can not be used to derive other persistence strategies.

The use case here is to derive other types of persistence from Encodable (and take profit from the compiler-generated code).

For example, I want to write:

  extension MutablePersistable where Self: Encodable {
    // Required by MutablePersistable
    var persistentDictionary: [String: DatabaseValueConvertible?] {
      return ...
    }
  }

If it were possible, we could get the full picture, with all boilerplate removed:
  
  // Wouldn't it be great?
  struct Player : RowConvertible, MutablePersistable, Codable {
    static let databaseTableName = "player"
    var id: Int64?
    let name: String
    let score: Int
  }

  let arthur = Player(name: "Arthur", score: 100)
  try arthur.insert(db)
  print(arthur.id)

  let topPlayers = try Player
    .order(Column("score").desc)
    .limit(10)
    .fetchAll(db) // [Player]

Unfortunately, it's impossible: the Encodable protocol doesn't allow iteration on the coding keys. I can't generate anything useful.

To be clear, is the problem here that Encodable doesn’t expose the CodingKeys on a type? Because this is a deliberate decision — it is important for types to be able to encapsulate their CodingKeys and keep them private.

···

On May 26, 2017, at 7:26 AM, Gwendal Roué via swift-evolution <swift-evolution@swift.org> wrote:

Again, is it possible to workaround this problem? Did I miss something?

Thanks for your attention,
Gwendal Roué

[1] https://github.com/groue/GRDB.swift

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(David Hart) #3

Can you explain what’s the problem with Issue 2?

Am I correct in suggesting that Issue 1 is more of a missing generics feature than a problem with SE-0166/0167?

David.

···

On 26 May 2017, at 16:26, Gwendal Roué via swift-evolution <swift-evolution@swift.org> wrote:

Hello,

I want to provide real-life feedback for the Swift Archival & Serialization (SE-0166) and Swift Encoders (SE-0167) proposals that currently ship in Swift 4 snapshots.

The context: I'm the author of GRDB.swift [1], a SQLite library that, among other goals, aims at easing the conversion between database rows and custom models (structs and class hierarchies):

  // Sample code
  let arthur = Player(name: "Arthur", score: 100)
  try arthur.insert(db)
  print(arthur.id)
  
  let topPlayers = try Player
    .order(Column("score").desc)
    .limit(10)
    .fetchAll(db) // [Player]

Due to the lack of any introspection in Swift, GRDB currently wants you to perform explicit conversion:

  struct Player {
    var id: Int64?
    let name: String
    let score: Int
  }
  
  extension Player : RowConvertible {
    init(row: Row) {
      id = row.value(named: "id")
      name = row.value(named: "name")
      score = row.value(named: "score")
    }
  }
  
  extension Player : TableMapping, MutablePersistable {
    static let databaseTableName = "player"
    var persistentDictionary: [String: DatabaseValueConvertible?] {
      return ["id": id, "name": name, "score: score]
    }
  }

That's enough, but that's still too much.

SE-0166 and SE-0167 sound like the promise that some boilerplate code could be automatically generated.

Along with JSONDecoder and PListDecoder, let's introduce DatabaseRowDecoder! The current state of the work is at https://github.com/groue/GRDB.swift/tree/Swift4

At first, it's very satisfying. Decodable keeps some of it promises:

  struct Player : RowConvertible, Decodable {
    static let databaseTableName = "player"
    var id: Int64?
    let name: String
    let score: Int
  }
  
  // Yeah, no more extra code necessary for this to work!
  let topPlayers = try Player
    .order(Column("score").desc)
    .limit(10)
    .fetchAll(db)

But there are some issues.

### Issue 1: SE-0166/0167 merge the concepts of keyed objects and values

This is a problem. Let's take this example:

  enum Color: Int, Codable {
    case blue, green, red
  }
  
  struct Flower : RowConvertible, Decodable {
    let name: String
    let color: Color
  }
  
The way to decode a color comes from KeyedDecodingContainerProtocol:

  protocol KeyedDecodingContainerProtocol {
    func decode<T>(_ type: T.Type, forKey key: Key) throws -> T where T : Decodable
    func decodeIfPresent<T>(_ type: T.Type, forKey key: Key) throws -> T? where T : Decodable
  }

But the ability to decode a Color from a database row comes from the DatabaseValueConvertible, which I can't invoke since I can't test if type T conforms to this protocol:

  struct RowKeyedDecodingContainer<Key: CodingKey>: KeyedDecodingContainerProtocol {
    let row: Row
    
    // Not OK: no support for values
    func decode<T>(_ type: T.Type, forKey key: Key) throws -> T where T : Decodable {
      if <T conforms to DatabaseValueConvertible> {
        let databaseValue: DatabaseValue = row.value(named: key.stringValue)
        return T.fromDatabaseValue(databaseValue)
      } else { ... }
    }
  }

So the current state of the Codable library disallow GRDB from supporting value properties which are not the trivial Int, Int32, etc. Of course, GRDB itself makes it possible, with explicit user code. But we're talking about removing boilerplate and relying on the code generation that Codable is blessed with, here. We're talking about sharing the immense privilege that Codable is blessed with.

However, if I can't decode **values**, I can still decode **complex keyed objects** (in this case the row behaves like a hierarchical container - a concept already present in GRDB and allows it to consume complex rows like results of joins):

  struct Book : RowConvertible, Decodable { ... }
  struct Author : RowConvertible, Decodable { ... }
  struct Pair : RowConvertible, Decodable {
    let book: Book
    let author: Author
  }
  
  struct RowKeyedDecodingContainer<Key: CodingKey>: KeyedDecodingContainerProtocol {
    let row: Row
  
    // OK, support for other decodable objects
    func decode<T>(_ type: T.Type, forKey key: Key) throws -> T where T : Decodable {
      if let scopedRow = row.scoped(on: key.stringValue) {
        return try T(from: RowDecoder(row: scopedRow, codingPath: codingPath + [key]))
      } else {
        throw DecodingError.keyNotFound(key, DecodingError.Context(codingPath: codingPath, debugDescription: "missing scope"))
      }
    }
  }

Yet this use case is much less frequent.

Is it possible to workaround this problem? Did I miss something?

### Issue 2: Encodable can not be used to derive other persistence strategies.

The use case here is to derive other types of persistence from Encodable (and take profit from the compiler-generated code).

For example, I want to write:

  extension MutablePersistable where Self: Encodable {
    // Required by MutablePersistable
    var persistentDictionary: [String: DatabaseValueConvertible?] {
      return ...
    }
  }

If it were possible, we could get the full picture, with all boilerplate removed:
  
  // Wouldn't it be great?
  struct Player : RowConvertible, MutablePersistable, Codable {
    static let databaseTableName = "player"
    var id: Int64?
    let name: String
    let score: Int
  }

  let arthur = Player(name: "Arthur", score: 100)
  try arthur.insert(db)
  print(arthur.id)

  let topPlayers = try Player
    .order(Column("score").desc)
    .limit(10)
    .fetchAll(db) // [Player]

Unfortunately, it's impossible: the Encodable protocol doesn't allow iteration on the coding keys. I can't generate anything useful.

Again, is it possible to workaround this problem? Did I miss something?

Thanks for your attention,
Gwendal Roué

[1] https://github.com/groue/GRDB.swift

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


#4

Can you explain what’s the problem with Issue 2?

The problem was me, I guess :slight_smile: Of course nobody knows the list of keys, but the type itself. It's a matter of injecting an encoder. I'll do that.

Thanks also Itai for your answer.

Am I correct in suggesting that Issue 1 is more of a missing generics feature than a problem with SE-0166/0167?

There are two ways to see such issue: either a language is not ready, either a library isn't designed for its language. :slight_smile: But this is not the case here. Again, Itai has the correct answer:

   if T.self is DatabaseValueConvertible.Type {
       let databaseValue: DatabaseValue = row.value(named: key.stringValue)
       return (T.self as! DataBaseValueConvertible.Type).fromDatabaseValue(databaseValue) as! T
   } else { ... }

This is the way to test a type against a protocol - I didn't know this was even possible!

Thanks a lot, Itai and David: SE-0166 and SE-0167 are delivering their promises, and GRDB will make good use from them :slight_smile:
Gwendal

···

Le 26 mai 2017 à 22:30, David Hart <david@hartbit.com> a écrit :

David.

On 26 May 2017, at 16:26, Gwendal Roué via swift-evolution <swift-evolution@swift.org> wrote:

Hello,

I want to provide real-life feedback for the Swift Archival & Serialization (SE-0166) and Swift Encoders (SE-0167) proposals that currently ship in Swift 4 snapshots.

The context: I'm the author of GRDB.swift [1], a SQLite library that, among other goals, aims at easing the conversion between database rows and custom models (structs and class hierarchies):

  // Sample code
  let arthur = Player(name: "Arthur", score: 100)
  try arthur.insert(db)
  print(arthur.id)
  
  let topPlayers = try Player
    .order(Column("score").desc)
    .limit(10)
    .fetchAll(db) // [Player]

Due to the lack of any introspection in Swift, GRDB currently wants you to perform explicit conversion:

  struct Player {
    var id: Int64?
    let name: String
    let score: Int
  }
  
  extension Player : RowConvertible {
    init(row: Row) {
      id = row.value(named: "id")
      name = row.value(named: "name")
      score = row.value(named: "score")
    }
  }
  
  extension Player : TableMapping, MutablePersistable {
    static let databaseTableName = "player"
    var persistentDictionary: [String: DatabaseValueConvertible?] {
      return ["id": id, "name": name, "score: score]
    }
  }

That's enough, but that's still too much.

SE-0166 and SE-0167 sound like the promise that some boilerplate code could be automatically generated.

Along with JSONDecoder and PListDecoder, let's introduce DatabaseRowDecoder! The current state of the work is at https://github.com/groue/GRDB.swift/tree/Swift4

At first, it's very satisfying. Decodable keeps some of it promises:

  struct Player : RowConvertible, Decodable {
    static let databaseTableName = "player"
    var id: Int64?
    let name: String
    let score: Int
  }
  
  // Yeah, no more extra code necessary for this to work!
  let topPlayers = try Player
    .order(Column("score").desc)
    .limit(10)
    .fetchAll(db)

But there are some issues.

### Issue 1: SE-0166/0167 merge the concepts of keyed objects and values

This is a problem. Let's take this example:

  enum Color: Int, Codable {
    case blue, green, red
  }
  
  struct Flower : RowConvertible, Decodable {
    let name: String
    let color: Color
  }
  
The way to decode a color comes from KeyedDecodingContainerProtocol:

  protocol KeyedDecodingContainerProtocol {
    func decode<T>(_ type: T.Type, forKey key: Key) throws -> T where T : Decodable
    func decodeIfPresent<T>(_ type: T.Type, forKey key: Key) throws -> T? where T : Decodable
  }

But the ability to decode a Color from a database row comes from the DatabaseValueConvertible, which I can't invoke since I can't test if type T conforms to this protocol:

  struct RowKeyedDecodingContainer<Key: CodingKey>: KeyedDecodingContainerProtocol {
    let row: Row
    
    // Not OK: no support for values
    func decode<T>(_ type: T.Type, forKey key: Key) throws -> T where T : Decodable {
      if <T conforms to DatabaseValueConvertible> {
        let databaseValue: DatabaseValue = row.value(named: key.stringValue)
        return T.fromDatabaseValue(databaseValue)
      } else { ... }
    }
  }

So the current state of the Codable library disallow GRDB from supporting value properties which are not the trivial Int, Int32, etc. Of course, GRDB itself makes it possible, with explicit user code. But we're talking about removing boilerplate and relying on the code generation that Codable is blessed with, here. We're talking about sharing the immense privilege that Codable is blessed with.

However, if I can't decode **values**, I can still decode **complex keyed objects** (in this case the row behaves like a hierarchical container - a concept already present in GRDB and allows it to consume complex rows like results of joins):

  struct Book : RowConvertible, Decodable { ... }
  struct Author : RowConvertible, Decodable { ... }
  struct Pair : RowConvertible, Decodable {
    let book: Book
    let author: Author
  }
  
  struct RowKeyedDecodingContainer<Key: CodingKey>: KeyedDecodingContainerProtocol {
    let row: Row
  
    // OK, support for other decodable objects
    func decode<T>(_ type: T.Type, forKey key: Key) throws -> T where T : Decodable {
      if let scopedRow = row.scoped(on: key.stringValue) {
        return try T(from: RowDecoder(row: scopedRow, codingPath: codingPath + [key]))
      } else {
        throw DecodingError.keyNotFound(key, DecodingError.Context(codingPath: codingPath, debugDescription: "missing scope"))
      }
    }
  }

Yet this use case is much less frequent.

Is it possible to workaround this problem? Did I miss something?

### Issue 2: Encodable can not be used to derive other persistence strategies.

The use case here is to derive other types of persistence from Encodable (and take profit from the compiler-generated code).

For example, I want to write:

  extension MutablePersistable where Self: Encodable {
    // Required by MutablePersistable
    var persistentDictionary: [String: DatabaseValueConvertible?] {
      return ...
    }
  }

If it were possible, we could get the full picture, with all boilerplate removed:
  
  // Wouldn't it be great?
  struct Player : RowConvertible, MutablePersistable, Codable {
    static let databaseTableName = "player"
    var id: Int64?
    let name: String
    let score: Int
  }

  let arthur = Player(name: "Arthur", score: 100)
  try arthur.insert(db)
  print(arthur.id)

  let topPlayers = try Player
    .order(Column("score").desc)
    .limit(10)
    .fetchAll(db) // [Player]

Unfortunately, it's impossible: the Encodable protocol doesn't allow iteration on the coding keys. I can't generate anything useful.

Again, is it possible to workaround this problem? Did I miss something?

Thanks for your attention,
Gwendal Roué

[1] https://github.com/groue/GRDB.swift

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(David Hart) #5

Can you explain what’s the problem with Issue 2?

The problem was me, I guess :slight_smile: Of course nobody knows the list of keys, but the type itself. It's a matter of injecting an encoder. I'll do that.

Thanks also Itai for your answer.

Am I correct in suggesting that Issue 1 is more of a missing generics feature than a problem with SE-0166/0167?

There are two ways to see such issue: either a language is not ready, either a library isn't designed for its language. :slight_smile: But this is not the case here. Again, Itai has the correct answer:

  if T.self is DatabaseValueConvertible.Type {
      let databaseValue: DatabaseValue = row.value(named: key.stringValue)
      return (T.self as! DataBaseValueConvertible.Type).fromDatabaseValue(databaseValue) as! T
  } else { … }

I didn’t know that was possible either! Really cool. Even better:

  if let databaseValueType = T.self as? DatabaseValueConvertible.Type {
      let databaseValue: DatabaseValue = row.value(named: key.stringValue)
      return databaseValueType.fromDatabaseValue(databaseValue) as! T
  } else { … }

···

On 27 May 2017, at 08:59, Gwendal Roué <gwendal.roue@gmail.com> wrote:

Le 26 mai 2017 à 22:30, David Hart <david@hartbit.com> a écrit :

This is the way to test a type against a protocol - I didn't know this was even possible!

Thanks a lot, Itai and David: SE-0166 and SE-0167 are delivering their promises, and GRDB will make good use from them :slight_smile:
Gwendal

David.

On 26 May 2017, at 16:26, Gwendal Roué via swift-evolution <swift-evolution@swift.org> wrote:

Hello,

I want to provide real-life feedback for the Swift Archival & Serialization (SE-0166) and Swift Encoders (SE-0167) proposals that currently ship in Swift 4 snapshots.

The context: I'm the author of GRDB.swift [1], a SQLite library that, among other goals, aims at easing the conversion between database rows and custom models (structs and class hierarchies):

  // Sample code
  let arthur = Player(name: "Arthur", score: 100)
  try arthur.insert(db)
  print(arthur.id)
  
  let topPlayers = try Player
    .order(Column("score").desc)
    .limit(10)
    .fetchAll(db) // [Player]

Due to the lack of any introspection in Swift, GRDB currently wants you to perform explicit conversion:

  struct Player {
    var id: Int64?
    let name: String
    let score: Int
  }
  
  extension Player : RowConvertible {
    init(row: Row) {
      id = row.value(named: "id")
      name = row.value(named: "name")
      score = row.value(named: "score")
    }
  }
  
  extension Player : TableMapping, MutablePersistable {
    static let databaseTableName = "player"
    var persistentDictionary: [String: DatabaseValueConvertible?] {
      return ["id": id, "name": name, "score: score]
    }
  }

That's enough, but that's still too much.

SE-0166 and SE-0167 sound like the promise that some boilerplate code could be automatically generated.

Along with JSONDecoder and PListDecoder, let's introduce DatabaseRowDecoder! The current state of the work is at https://github.com/groue/GRDB.swift/tree/Swift4

At first, it's very satisfying. Decodable keeps some of it promises:

  struct Player : RowConvertible, Decodable {
    static let databaseTableName = "player"
    var id: Int64?
    let name: String
    let score: Int
  }
  
  // Yeah, no more extra code necessary for this to work!
  let topPlayers = try Player
    .order(Column("score").desc)
    .limit(10)
    .fetchAll(db)

But there are some issues.

### Issue 1: SE-0166/0167 merge the concepts of keyed objects and values

This is a problem. Let's take this example:

  enum Color: Int, Codable {
    case blue, green, red
  }
  
  struct Flower : RowConvertible, Decodable {
    let name: String
    let color: Color
  }
  
The way to decode a color comes from KeyedDecodingContainerProtocol:

  protocol KeyedDecodingContainerProtocol {
    func decode<T>(_ type: T.Type, forKey key: Key) throws -> T where T : Decodable
    func decodeIfPresent<T>(_ type: T.Type, forKey key: Key) throws -> T? where T : Decodable
  }

But the ability to decode a Color from a database row comes from the DatabaseValueConvertible, which I can't invoke since I can't test if type T conforms to this protocol:

  struct RowKeyedDecodingContainer<Key: CodingKey>: KeyedDecodingContainerProtocol {
    let row: Row
    
    // Not OK: no support for values
    func decode<T>(_ type: T.Type, forKey key: Key) throws -> T where T : Decodable {
      if <T conforms to DatabaseValueConvertible> {
        let databaseValue: DatabaseValue = row.value(named: key.stringValue)
        return T.fromDatabaseValue(databaseValue)
      } else { ... }
    }
  }

So the current state of the Codable library disallow GRDB from supporting value properties which are not the trivial Int, Int32, etc. Of course, GRDB itself makes it possible, with explicit user code. But we're talking about removing boilerplate and relying on the code generation that Codable is blessed with, here. We're talking about sharing the immense privilege that Codable is blessed with.

However, if I can't decode **values**, I can still decode **complex keyed objects** (in this case the row behaves like a hierarchical container - a concept already present in GRDB and allows it to consume complex rows like results of joins):

  struct Book : RowConvertible, Decodable { ... }
  struct Author : RowConvertible, Decodable { ... }
  struct Pair : RowConvertible, Decodable {
    let book: Book
    let author: Author
  }
  
  struct RowKeyedDecodingContainer<Key: CodingKey>: KeyedDecodingContainerProtocol {
    let row: Row
  
    // OK, support for other decodable objects
    func decode<T>(_ type: T.Type, forKey key: Key) throws -> T where T : Decodable {
      if let scopedRow = row.scoped(on: key.stringValue) {
        return try T(from: RowDecoder(row: scopedRow, codingPath: codingPath + [key]))
      } else {
        throw DecodingError.keyNotFound(key, DecodingError.Context(codingPath: codingPath, debugDescription: "missing scope"))
      }
    }
  }

Yet this use case is much less frequent.

Is it possible to workaround this problem? Did I miss something?

### Issue 2: Encodable can not be used to derive other persistence strategies.

The use case here is to derive other types of persistence from Encodable (and take profit from the compiler-generated code).

For example, I want to write:

  extension MutablePersistable where Self: Encodable {
    // Required by MutablePersistable
    var persistentDictionary: [String: DatabaseValueConvertible?] {
      return ...
    }
  }

If it were possible, we could get the full picture, with all boilerplate removed:
  
  // Wouldn't it be great?
  struct Player : RowConvertible, MutablePersistable, Codable {
    static let databaseTableName = "player"
    var id: Int64?
    let name: String
    let score: Int
  }

  let arthur = Player(name: "Arthur", score: 100)
  try arthur.insert(db)
  print(arthur.id)

  let topPlayers = try Player
    .order(Column("score").desc)
    .limit(10)
    .fetchAll(db) // [Player]

Unfortunately, it's impossible: the Encodable protocol doesn't allow iteration on the coding keys. I can't generate anything useful.

Again, is it possible to workaround this problem? Did I miss something?

Thanks for your attention,
Gwendal Roué

[1] https://github.com/groue/GRDB.swift

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


#6

Yes, that's even more clear like that :slight_smile:

And for Encodable support, it works just great (not a single line of boilerplate code):

    private enum Color: String, DatabaseValueConvertible, Encodable {
        case red, green, blue
    }
    
    private struct EncodableStruct : Persistable, Encodable {
        static let databaseTableName = "myTable"
        let name: String
        let color: Color?
    }
    
    try dbQueue.inDatabase { db in
        try EncodableStruct(name: "Arthur", color: .red).insert(db)
    }

Gwendal

···

Le 27 mai 2017 à 11:59, David Hart <david@hartbit.com> a écrit :

I didn’t know that was possible either! Really cool. Even better:

if let databaseValueType = T.self as? DatabaseValueConvertible.Type {
     let databaseValue: DatabaseValue = row.value(named: key.stringValue)
     return databaseValueType.fromDatabaseValue(databaseValue) as! T
} else { … }


(Itai Ferber) #7

Excellent! Glad that ended up working out — looking forward to seeing this improve quality of life for GRDB users and others. :slight_smile:

···

On 27 May 2017, at 3:40, Gwendal Roué wrote:

> Le 27 mai 2017 à 11:59, David Hart <david@hartbit.com> a écrit :

I didn’t know that was possible either! Really cool. Even better:

if let databaseValueType = T.self as? DatabaseValueConvertible.Type {
     let databaseValue: DatabaseValue = row.value(named: key.stringValue)
     return databaseValueType.fromDatabaseValue(databaseValue) as! T
} else { … }

Yes, that's even more clear like that :slight_smile:

And for Encodable support, it works just great (not a single line of boilerplate code):

    private enum Color: String, DatabaseValueConvertible, Encodable {
        case red, green, blue
    }

    private struct EncodableStruct : Persistable, Encodable {
        static let databaseTableName = "myTable"
        let name: String
        let color: Color?
    }

    try dbQueue.inDatabase { db in
        try EncodableStruct(name: "Arthur", color: .red).insert(db)
    }

Gwendal


(David Hart) #8

As a user of GRDB: :heart_eyes:

···

On 27 May 2017, at 12:40, Gwendal Roué <gwendal.roue@gmail.com> wrote:

Le 27 mai 2017 à 11:59, David Hart <david@hartbit.com <mailto:david@hartbit.com>> a écrit :

I didn’t know that was possible either! Really cool. Even better:

if let databaseValueType = T.self as? DatabaseValueConvertible.Type {
     let databaseValue: DatabaseValue = row.value(named: key.stringValue)
     return databaseValueType.fromDatabaseValue(databaseValue) as! T
} else { … }

Yes, that's even more clear like that :slight_smile:

And for Encodable support, it works just great (not a single line of boilerplate code):

    private enum Color: String, DatabaseValueConvertible, Encodable {
        case red, green, blue
    }
    
    private struct EncodableStruct : Persistable, Encodable {
        static let databaseTableName = "myTable"
        let name: String
        let color: Color?
    }
    
    try dbQueue.inDatabase { db in
        try EncodableStruct(name: "Arthur", color: .red).insert(db)
    }

Gwendal