SDF Swift Data Format


(Paul Ossenbruggen) #1

My coworker Dan Steward and I, have been getting into the strongly typed nature of Swift and realizing that JSON is lacking in this regard. I think Swift needs its own strongly typed Data Definition format, that fixes shortcomings of JSON. We came up with this, any ideas or feedback?

Swift Data Format(SDF): A strictly typed data description format.

  • Adopts much of Swift's syntax so should feel natural to Swift developers but different enough to not mistake for Swift code.
  • Prevents errors that are common in JSON of malformed or invalid structures. Utilizes strict type checking.
  • Does not specify class or struct that is up to the loading program.
  • Allows both swift style comments
  • You can define a constant that can be read from a Swift file
  • Easily decoded and encoded using Codable in your Swift code.
  • Generates to binary
  • Reminiscent of the the old Rez file format but more aligned with Swift. Equivalent tools to Rez and DeRez would exist and constants could be exported for easy sharing in Swift code so there is only one place to change.
  • Alternative to JSON which would validate the schema before letting it be read.
  • Editors could validate as code is typed.
  • Definitions can be separate or in the same file.
  • Allows definitions of own named entities that can be built up like you can with structs.
  • Optionals are supported.
  • Default values can be defined in the definition.
  • Allows definitions, with let that later get evaluated. (I see no need for var).
  • Should support mathematical constructs so if you need to calculate a value 10 + 4 * 3 would be allowed.
  • Supports the base Swift types (Int, String) etc
  • Haven't thought about how protocols could be imported but might be an interesting direction.
  • Associated values for Swift enums, I can't currently see how that would work, but interesting area to explore.
  • Enums would likely have to specify a type but open to ideas.
  • Could eventually replace Storyboard XML, Xcode Project format, and Plist formats as prefered format and even the SPM Package format.
  • Would need to be made cross platform to support use as data transfer format.
  • Reference: OpenDDL (didn't feel Swifty enough).

Detailed example:

// Comments are supported
/*
    this style comment allowed
*/

// NOTE THIS IS NOT SWIFT, it is a Swift like data definition language.

// Definitions
import OtherDefinitons // can put other definitions in separate files.

def Pet(
    name: String
    age: Int
    legs: Int = 10 // default
)

enum Kind: String (
    case employee
    case manager = "Manager"
)

def Person(
    kind: Kind,
    height: Int,
    age: Double,
    name: String,
    friends: [Person]
    nice: Bool?
    pets: [Pet]
)

def Vertex(
    x: Float,
    y: Float,
    z: Float
)

def Color(
    red: Double,
    green: Double,
    blue: Double,
    alpha: Double = 1.0,
    enum Predefined {
        case yellow: Color(red: 0.0, green: 1.0, blue: 1.0)
        case blue: Color(red: 0.0, green: 0.0, blue: 1.0)
        case red: Color(red: 1.0, green: 0.0, blue: 0.0, alpha: 0.4)
    }
)

def Vehicle(
    color: Color,
    name: String
)

def People: [Person]

// instances

let george = Person(
    kind: .employee,
    height: 10,
    age: 49,
    name: "george",
    nice: true,
    friends = [john, paul, ringo]
    pets: []
)

let john = Person(
    kind: .manager,
    height: 10,
    age: 49,
    name: "john",
    friends: [george, paul, ringo],
    nice: true,
    pets: [ Pet(
                name: "fluffy",
                age: 10,
                legs: 4
            ),
            Pet(
                name: "RaptorMax",
                age: 12 // legs defaulted.
                )
    ]
)

let paul = Person(
    kind: .manager,
    height: 10,
    age: 49,
    name: "Paul",
    nice: true,
    friends: [john, george, ringo]
    pets: []
)

let ringo = Person(
    kind: .employee,
    height: 10,
    age: 49,
    name: "Ringo", // optional "nice" not declared here for example, not saying anything either way about example person.
    friends: [george, paul, john]
    pets: []
)

// can define using let or in line, 'let' does not output anything just defines it.
let vertex3 = Vertex(
    x: 9.9, y: 2.9, z: 4.5
)
let verticies = [
    Vertex(
        x: 0.0,
        y: 0.0,
        z: 0.0
    ),
    Vertex(
        x: 10.0,
        y: 10.0,
        z: 10.0
    ),
    vertex3
]

// generates objects into result file, nothing generated otherwise, kind of like how Playgrounds don't do
// anything unless you output result.

ringo
verticies
Vehicle(
    color: .Predefined.yellow,
    name: "Yellow Submarine"
)

(TJ Usiyan) #2

Is speed a concern here? wouldn't the definition aspect complicate parsing such that it would only have an expressivity advantage over JSON. Similarly, I think that both comment styles might be more than needed. (I am likely incorrect about this but it is an immediate concern since I think that this would hinder adoption.)


(Paul Ossenbruggen) #3

Yes, performance is definitely important. Once parsed into binary format, it should not need to be rechecked. Parsing performance should not be much different from parsing XML and hopefully closer to JSON, as a goal. Saving the schema into the format would also be important, but could be transferred or handled separately from the data file.

We put in two styles of comments to have parity with Swift, and to support multi-line commenting out of sections. This would not be generated into the binary output This would mainly be for documentation of source files and definitions. Since JSON does not support commenting at all, this seems like an improvement to me but only preserved for the textual source file representation similar to compiled code.


(Jacob Williams) #4

#Bikeshedding I like the idea of a Swift Data Format, but why not call it Swift Object Notation just so we have the acronym SON. This is much better than SDF in my opinion ;) (and much easier to remember)


(Paul Ossenbruggen) #5

Oh yes, I am definitely open to better names, this was the first thing we thought of. I am sure there are better names.


(Jacob Williams) #6

Does this mean there will be a utility/format, like jsonschema, that would allow developers to specify the expected schema for validation?


(George) #7

JSON is lacking, but there are a few formats which address its shortcomings and have existing ecosystems. Cap'n Proto comes to mind. It supports generics and enumerations and it is even extended with an optional RPC layer. I wonder if that could fit your needs.


#8

I'm not sure what the use case for it would be. Is it supposed to be written by hand? If so it would be type-checked and maybe error, just like JSON. Or maybe it is supposed to be written by a type-safe library, in which case it would be type-checked when creating the values to encode, just like with JSON.


(Paul Ossenbruggen) #9

Thank you I am definitely interested in seeing what is out there as well. I am not sure these would be a natural as a Swift like format but will investigate.


(Paul Ossenbruggen) #10

http://json-schema.org also thank you for alternatives, will investigate this too and yes, this would define a schema to validate against.


(Paul Ossenbruggen) #11

It should be able to be written by hand, or generated from something like IB. I was thinking that for people who don't like Storyboards because they prefer code, you could write all the resource descriptions using this, hopefully swapping back and forth between IB, and this hand coding this format. This would also allow you separate the layout data from the object definitions so that that could be tweaked more easily, and not cause as many source control conflicts as frequently happens with Storyboards.


(Paul Ossenbruggen) #12

Capt'n Proto does look like a good binary format to base things on and if it is an existing standard automatically get interoperability with that format. So maybe this project becomes Implementing a Swift like version with this binary format, something that would reinvent the wheel otherwise and hopefully capitalize on the lessons learned there. I am thinking that this should be a 100% Swift implementation and hopefully look more like Swift as laid out in the example above.