REST Up: Consuming a Multipart/Form-Data REST Method With Low-Code

A couple of weeks ago, I was asked to use a low-code platform - OutSystems - to create an interface with a REST API from an external supplier. Except for a lack of functional documentation from the supplier, the process was pretty straightforward until I hit a stumbling block: one of the methods for uploading documents used something called multipart/form-data, which is not natively supported by OutSystems.

Since I am active in the OutSystems Community Forum, I have regularly seen people running into this problem when trying to consume a REST service but had not experienced it myself. So, I did two things: first, I created a Community Idea for OutSystems to support it natively, and second, I came up with an idea of how to satisfy the REST API with minimal impact on my implementation of its interface, which I describe in this article.

REST and OutSystems

The first thing I want to explain is how OutSystems implements REST. REST is a protocol on top of HTTP (the protocol used for retrieving web pages) and typically (though not exclusively) uses JSON to communicate data.

When sending data via HTTP, the sender must specify the type of content. Since OutSystems supports JSON only,[1] it sends “application/json” in the HTTP “Content-Type” header. In the body of the message, the actual JSON is sent compacted (no line breaks), like this:

{"name":"Test for POST","type":"PACKAGE","language":"en","description":"This is a test for posting a package"}

This works for probably 95% of REST methods, so it’s in line with the OutSystems philosophy of supporting the most commonly used software patterns, but not necessarily all of them.

First, instead of just the content type, a client (like a browser or REST consuming app) sending multipart/form-data sets the HTTP content type to “multipart/form-data,” but it also needs to specify something called a “boundary.” This is a unique text string that must be guaranteed not to appear anywhere in the message, so it can signal the several sections or parts (it’s multipart, after all). It is customary to start the boundary with a string of dashes followed by some kind of GUID, although a client is free to set the boundary to whatever it likes (within limits), as long as the rule I just stated is met. The boundary is added after the content type with a semicolon, followed by the “boundary=” and followed by the boundary itself between double quotes:

Content-Type: multipart/form-data;boundary="MyBoundary"

The body of the message consists of one or more sections, each part starting with two dashes (--) followed by the boundary text. On the next line, there’s the “Content-Disposition” text: form-data; name="name,” where the name in quotes is the name of the item. If the section contains file data, the line also contains the filename, like this:

--MyBoundary
Content-Disposition: form-data; name="myFile" filename="myfile.docx"

After this line, there’s a blank line, followed by the value associated with “name.” For simple input fields, this is their value; for files, this is their binary data. Directly after the data, a new section starts (without a blank line in between).

After the last section, to signal the end of the multipart/form-data message, the boundary is added again. This time, however, it not only starts with two dashes, but it also ends with them:

--MyBoundary--

Of course, I’m not the first one to attempt to solve the problem of sending multipart/form-data. Forge Components for various REST APIs have solved it in a number of different ways, and there is also a Forge component for sending multipart/form-data independently from REST.

But none of that was entirely to my liking. So I decided to create an implementation that suited my needs (and my taste for esthetics).

Implementation Details

Whatever implementation you choose, in the end, you need to do two things: send the right Content-Type header and send the correctly formatted body (as explained above). If you include files that contain binary data (i.e., anything other than a text file), you need to declare that the REST API Method has a parameter with a data type of “Binary Data, and its “Send In” property is set to "Body."

This allows the binary data to be sent unmodified, which is what we want. If also prevents the Platform from adding a Content-Type tag in the Header, which you then need to add (via a Parameter with “Send In” set to “Header”).

The (small) downside is that, to compose the message, you need the Binary Concat Forge component so you can concatenate the various parts. You also need Actions from the BinaryData Extension (which is a system component that is already installed) to convert the text-based parts of the message to binary before concatenating it with the binary data of the files.

Another approach, which only works when the message does not contain any binary data (e.g., when sending text files only), is to declare the REST API method as a “normal’ JSON one, and modify the message before it is sent to the REST service.

In OutSystems, this is done by defining an OnBeforeRequest Action in the advanced properties of a REST API. Choose “New OnBeforeRequest” from the drop-down menu (not the “Advanced” one), and the following REST API action will appear:

Select “OnBeforeRequest” for great justice!

There it is, for your modifying pleasure.

As can be seen in the image, the OnBeforeRequest has one input parameter and one output parameter. Both are of type HTTPRequest, and contain everything you need to modify the REST message:

Important notes about the HTTPRequest Structure: The OnBeforeRequest is a generic way of modifying the REST message and not specific to multipart/form-data. Also note that there is a single OnBeforeRequest for the entire REST API, not one per method. This means that inside the OnBeforeRequest, you need to check which method is called by inspecting the URLPath and HTTPMethod Attributes of the Request Input Parameter. [2]

My Solution

I needed to send binary data, so I had to choose the first approach. [3] This meant I added two input parameters: one for the content type and one for the binary body. Using my Forge component, I added the multiple parts (including a JSON part) to the message, after which I assigned the concatenated parts via the Binary Data parameter.

The Forge Component

Regardless of the specific solution that makes sense for your project, I created a Forge Component, aptly named Multipart/Form-Data, that consists of a single eSpace with a number of public actions that help create the right content:

• MultipartFormDataCreate: The main action, this creates a text body that consists of specified parts, in a multipart/form-data compatible format.
• ContentTypeGet: Returns the value to be used for the Content-Type header of the REST message based on the boundary used.
• PartAdd: A helper function that can be used to easily create a List of parts.

Wrapping Up

I hope you find this useful when you are implementing REST with multipart/form-data. You can ask any questions in the OutSystems Community Forum, the component subforum, or, as a last resort, send me a PM.

[1] This is not entirely true for consuming REST services, but for the sake of this article, we’ll assume it is, as the other supported types are of no use for the “multipart/form-data” we want to support.

[2] Another solution would be to isolate a single method that needs multipart/form-data in its own REST API, but I would advise against this, as there’s no logical grouping of Methods anymore, and you need to configure the same base URL twice in Service Center, which is prone to errors.

[3] In a previous version of this article, I mentioned choosing the second approach. This was based on the erroneous assumption I could make it work by converting binary data to text, which turned out to be not the case.

Thanks to Kiarash Irandoust.