Emulating a key value store, are there any downsides to my approach?

Related Projects GRDB
1

Hello, I am evaluating GRDB for use in my app and wanted to get some input on whether something I'm trying is appropriate, or if it's abusing GRDB, or otherwise terrible for some reason. Thanks in advance for any advice!

For some of the data in my app, I want to use GRDB's records traditionally: set up specific tables for them, properly migrate them between app versions, etc. But for much of my data, I would like to use GRDB as a key-value store of codable structs. It would be used for some long-term ish caching purposes, but can be cleared at any time and if a record format changes between app releases, it's fine because it can just re-fetched. For quick illustration purposes, the hierarchy I'm setting up can be represented with this swift dictionary:

var store = [String: [String: (value: Any, metadata: Any?)]]

Top level are "collections", and the inner level are "keys". A key can have a value and optional metadata associated with it. Continuing with the dictionary version, looking up an Article with id "1" would look like:

let article = store["Article"]?["1"]?.value as? Article

My app would be storing in the low 1000s of entries.

GRDB

Here's a slightly modified version of how I'm accomplishing this with GRDB.

First I have two protocols:

public protocol Persistable: Codable {
    static var collection: String { get }
    var id: String { get }
}

public protocol PersistableWithMetadata: Persistable {
    associatedtype Metadata: Codable
}

These are what my app's records would deal with. Everything below would be "private" to a wrapper framework. Various abstractions for reading Persistables and PersistableWithMetadatas would be built, so I could write code like Article.value(for: "1", in: database)

I then create two record wrapper types:

struct DatabaseValueWrapper<Wrapped: Persistable>: Codable, FetchableRecord, PersistableRecord {
    static var databaseTableName: String {
        "values"
    }

    enum Columns {
        static var id: Column { Column(CodingKeys.id) }
        static var value: Column { Column(CodingKeys.value) }
    }

    var id: String

    var value: Wrapped

    init(_ wrapped: Wrapped) {
        self.id = "\(Wrapped.collection).\(wrapped.id)"
        self.value = wrapped
    }
}

struct DatabaseMetadataWrapper<Wrapped: PersistableWithMetadata >: Codable, FetchableRecord, PersistableRecord {
    static var databaseTableName: String {
        "metadatas"
    }

    enum Columns {
        static var id: Column { Column(CodingKeys.id) }
        static var metadata: Column { Column(CodingKeys.metadata) }
    }

    var id: String

    var metadata: Wrapped.Metadata

    init(id: String, _ wrapped: Wrapped.Metadata) {
        self.id = "\(Wrapped.collection).\(id)"
        self.metadata = wrapped
    }
}

I then set up my tables like this:

registerMigration("createWrapperTables") { d in
    try d.create(table: "values") { table in
        table.column("id", .text).indexed().primaryKey(onConflict: .replace)
        table.column("value", .blob)
    }

    try d.create(table: "metadatas") { table in
        table.column("id", .text).indexed().primaryKey(onConflict: .replace)
        table.column("metadata", .blob)
    }
}
2

Hello @noremac,

You can implement a key-value store in SQLite, with GRDB on top of it. Some apps do need to escape the Relational Model and give up with Database Normalization or Data Integrity.

But if your app needs to deal with a fixed and know set of entities (articles, etc), then it may be useful to setup a regular database schema, with tables and typed columns. You'll be able to express constraints such as "all articles MUST have a title and a content", or "all articles MUST belong to a category", without having to reimplement them, sometimes with great difficulty, in application code. Requests such as "all articles published in 2021" will be trivial to implement. YMMV.

My rule of thumb is: the more SQLite works for me, the less bugs I ship.

3

Thanks @gwendal.roue, I'll take that into consideration and talk to the team! If I do end up sticking with a key-value store approach for certain types, does the one I've outlined seem reasonable? I'm also currently investigating collapsing my two tables, values/metadatas into one.

4

You and your team look quite able to implement a robust key-value store, so I'm not sure I understand the nature and the purpose of your question. If your code runs and performs the expected effects, then I guess you're on the right track. It looks "engineered", and I suppose this is because you have constraints to fulfill.

I'm also currently investigating collapsing my two tables, values/metadatas into one.

Its common that one eventually grasps the advantages of the relational model ;-)

1 Like