QM isn't talking about memory leaks. He's talking about whether you can construct the objects in the first place.
And your example isn't a memory leak in and of itself. It's only a leak if no code breaks the cycle.
1 Like
If no code breaks the loop, Coding does not require two-step initialization.
I repeat myself: Coding requires no more weak references than Swift requires to avoid memory leaks.
You keep mentioning memory leaks, but memory leaks are not the issue.
1 Like
Can we please focus on the topic at hand and move this discussion into a side channel? It is very distracting.
5 Likes
It would also be great to have a mapping between KeyPaths and CodingKeys of a type.
I recognize this is a giant can of worms (there may be custom CodingKeys that aren't 1-1, some KeyPaths won't have coding keys, etc) however something as simple as an optional keyPath on a CodingKey (nil if there's no related KeyPath) could unlock some pretty sweet stuff.
For example, with alchemy the ORM models are Codable to leverage the free compiler synthesized CodingKeys. Unfortunately there isn't a way to map KeyPaths to CodingKeys so the QueryBuilder is forced to be stringly typed.
There are ways, of course, to build a strongly typed query builder (see vapor's fluent) but it requires property wrappers for each field, with manual passing in of the column key.
Though now that I think about it, perhaps this is better solved with more advanced reflection features. If there were a surefire way to get a string for each key path (like when using NSObject) this would be doable.
1 Like
exactly
:)
1 Like
XML is not a data serialization format
I was referring to any concrete XML format that supports arbitrary ordered nodes, like HTML. My apologies for not making that more clear.
I imagine it would be extremely difficult to write an HTML decoder that works as I described.
Since we are talking about extending Codable to generic serialization formats, would be nice to include communities at large such as committer for Flatbuffer's Swift implementation into the discussion: mustiikhalil · GitHub
HTML isn't an XML format, it's an SGML format with some changes made so that common XMLisms (e.g. self closing BR) aren't considered errors.
Okay. You're arguing pointless semantics. My point still stands.
To repeat Dario’s request:
Please keep the discussion on topic and stop the side-topics or split them out into separate threads. If necessary we’ll ask moderators to clean up the thread.
Thank you.
4 Likes
Folks, the goal of the thread is discussing technical aspects of serialization in Swift.
Posts that do not directly contribute to that goal are better discussed in separate threads, and offensive posts would need to be moderated.
This thread has been immensely useful thus far, and we should resume the healthy discourse.
11 Likes
When talking about code-gen for serialization, would this be only for generating Swift code? It could be very interesting if we could use plug-in tools in the code-gen to generate schemas for serialization methods like Protocol Buffers, or Cap'n Protocol (who's schema language looks very close to Swift code).
1 Like
I primarily use Codable to encode/decode coarse-grained molecular simulations. I don't have many complaints here other than it is extremely slow.
Codable fills a few gaps in Swift's poor reflection capabilities. I also use Codable for two more things: 1) to generate property views of structs at runtime and 2) to implement deep copying within my simulations. My trick to making these things work is to register types with these systems at module load time.
I look in envy at languages with powerful metaprogramming. Everyone knows about Rust, but I would also like to point out Zig and Circle. Zig can manipulate types as values using compile-time execution and lazy evaluation. Circle adds metaprogramming on top of C++. Like others in this thread, I think Swift should implement serialization on top of a metaprogramming system.
These languages have had great success using metaprogramming to implement serialization and other related features. Metaprogramming could be the foundation for so many other valuable things. Beyond my examples above, I'm excited about its GPU shader interop and reflection applications and perhaps even differentiable programming. What other uses are going to come along in the future? Baking serialization into the compiler feels like a hack and a half measure. We should implement serialization codecs in Swift by interacting with a powerful type system. Metaprogramming is a massive language feature, but I think it would pay off in many unexpected ways.
15 Likes
As much as people keep talking about metaprogramming, I think it's reasonable to think that there's a fair chance we'll eventually get it. Assuming that MP would, in fact, provide for a complete serialization solution, do we want to spend the time & effort to build another interim solution? What about backwards compatibility? Can we keep calling the protocols Codable and have everything work, or will we need to be maintaining multiple "official Swift" serialization schemes?
[representing my own opinion, not the core team position] one interesting venue to explore here is how Codable synthesis can be extracted from the compiler to be based on an MP solution. IOW, can we design an MP solution that is flexible enough to extract Codable synthesis from the compiler and into a library, as well as support the development of additional Serialization solutions side by side to Codable
23 Likes
I think probably "yes", as long as the code that's currently synthesized has the same visibility as regular source code. That is to say, the only part of the compiler that "knows" it was synthesized is the part that does the synthesis. Although I suppose it also comes up if you ask your IDE where in your source code the implementations are, but I don't think an error there would prevent compilation or linking, would it?
Speaking of generating schemas, PerfectCRUD is using Codable to generate SQL schemas, although it comes with some bizarre limitations (max two Bools per table) probably due to how it interacts with Codable.
2 Likes
I wrote an experimental Swift encode / decode package (similar to Codable at interface level) that does not treat reference types as second-class citizens.
I called it GraphCodable. You can check it here.
GraphCodable:
- encodes type information;
- supports reference types inheritance;
- never duplicates the same object (as defined by ObjectIdentifier);
- in other words, it preserves the structure and the types of your model unaltered during encoding and decoding;
- is fully type checked at compile time;
- supports keyed and unkeyed coding, also usable simultaneously;
- supports conditional encoding;
- implements an userInfo dictionary;
- implements a type version system;
- implements a type substitution system during decode;
The package comes with a fairly complete documentation with many examples, a table summarizing the coding rules, a description of the data format.
I think you might find interesting the two examples concerning the encoding and decoding of DAG (Directed Acyclic Graph) and DGC (Directed Acyclic Graph).
I can confirm that deferDecode(...) is only required to decode a weak variable used to break ARC memory strong cycles. In any other situation, decode(...) must be used. If you don't believe me, check it out.
The main difficulty, in my opinion, is that you have to keep the types repository (simplifying, a singleton "typeName: GCodable type" dictionary) updated during the development of an application. The absence of a single 'GCodable' type from the types repository makes it impossible to decode a data file containing it.
I see two ways of solving this problem:
- a compiler magic that auto-register all GCodable types encountered during compilation. Is this possible with generic types?;
- better yet, a way to serialize and deserialize
Any.Type, if possible.
If there is a third one, I would be happy to learn it.
9 Likes
The team at Jet brains just released v1.20 of their Kotlin Serialization framework it looks impressive
3 Likes