Async Codable decoder?

I am working on a QUIC application that receives data from peers. I am decoding the data using my custom decoder that using Codable interface.

One issue is that I need an online decoder because the messages are variable size and there is no separator between them. I cannot know where is the data boundary without decode the data first. Obviously I don't want to decode some data to see if it is complete, if not, wait for more incoming data, and try again, until a success decode. And I need the ability to reject bad format data quickly (e.g. not after received 5MB of garbage). But there is no way for me to say pause a decoding action or await when waiting for more data with Codable.

I can ditch Codable and write my own version but I lost the compiler synthesis for it. It is an option but I want to keep it as the last resort.

I am currently trying to run the decode task on a separate thread and use old school semaphore to signal need more data. The code is ugly and I think I have to spawn a new thread for it, which have large overhead.

What other options do I have? Is there any existing open source code doing similar thing that I can reference? Are there any existing library that can help?

1 Like

This is an interesting question, but let me press on this for a moment. Is this a hard requirement? Do you have any ability to change that design?

yeah it is probably better to require a length prefix for messages. will see if it is possible to make such change to the protocol

1 Like