MultipartKit

[0xTim (Tim)]

Introduction

The Vapor Core Team would like to pitch MultipartKit to the SSWG. MultipartKit is a Swift library for encoding and decoding multipart form data based on SwiftNIO. It contains a C library used for the actual parsing of the request data and a Swift wrapper to integrate the C library with SwiftNIO.

Motivation

Multipart is a popular mechanism for sending data to servers, especially when doing file uploads. It's supported by both browsers and HTTP libraries and we feel that having a Swift implementation in the SSWG is an important step for the ecosystem.

MultipartKit was originally built into Vapor for the release of Vapor 4 with the intention of splitting it out at a later date and pitching it to the SSWG so here we are! Since the implementation has been used in Vapor 4 it's fairly well battle tested and deployed.

[mackoj (Jeffrey Macko)]

Thanks for pushing this forward.

I would love to see something like multipart form data with an official fully working implementation. I think it's a missing piece of the ecosystem.

I did tried MultipartKit a couple weeks ago and I did't see a way to send something that look like the example below(a few form data and the content of a file). Maybe I did misunderstood how to use it.
I didn't find the documentation for this project.

example

Content-Type: multipart/form-data; boundary=Boundary-0A7D819D-074D-4DEE-957F-51AD5F602D06

    --Boundary-0A7D819D-074D-4DEE-957F-51AD5F602D06
    Content-Disposition: form-data; name="file_format"

    text_file
    --Boundary-0A7D819D-074D-4DEE-957F-51AD5F602D06
    Content-Disposition: form-data; name="foo"

    bar
    --Boundary-0A7D819D-074D-4DEE-957F-51AD5F602D06
    Content-Disposition: form-data; name="file"; filename="hello.txt"
    Content-Type: text/plain

    Hello

[0xTim (Tim)]

Yep documentation is definitely missing. Plan is to add a proper usage/API description to the full proposal and those will be added to the README before the proposal is finalised. This should help people understand how to use it and is definitely something we should do if we want the library to be adopted outside of Vapor.

[lukasa (Cory Benfield)]

While I think MultipartKit is a very important addition to the Swift on Server ecosystem, I would like to see MultipartKit remove its C dependency as soon as possible.

The SSWG baseline guidelines call out a desire to avoid C where possible:

• Prefer native Swift over C wrapping, where appropriate

One of the goals of this guidance is to drastically reduce the exposure of server side code to the kinds of memory safety issues that are unavoidably tied up in C code. Multipart parsing is explicitly done on untrusted input, and so is a perfect attack vector for compromising server side processes. It's exactly the place where the most effort should be put into moving away from a C-based implementation.

In this case the need to move is also tractable. multipartparser.c is a 340-line C file. Performing a translation of the algorithms into Swift should be reasonably straightforward. It shouldn't even be too hard to replace the entire thing with a much more Swift-native design, one that produces structures and uses slicing instead of delivering the individual parts via callbacks.

While this is definitely an important project worth adopting, I really do think we should press on removing the C from it as soon as practicably possible.

[0xTim (Tim)]

I agree - the C implementation has been there because it's come from the very early Vapor days as a quick way to build a multipart implementation. Moving to Swift would also reduce the barrier to contributing.

I'll see if we can schedule some work to migrating to pure Swift soon

[siemensikkema]

@lukasa FYI I started work on rewriting the parser in Swift here. It still depends on callbacks in order to support parsing incoming data in chunks (streaming) but it uses buffer slicing for sending multipart bodies.