500 error during API call

500 error during API call

  

Hi.

I'm trying to upload a file using a REST call. The call is working fine from Postman.

Outsystems however, is causing a few issues. The endpoint in question is Yammer. Find the documentation here. There is a part in the documentation which specifies addition of attachments, which requires a multipart file upload.

Its working great on Postman, regardless of file type.

Outsystems is able to upload .txt files properly (not very useful though).

Excel and PPT files are being uploaded, but are getting corrupted for some reason. I get error which say "We're sorry, we couldn't open your workbook. It's possibly corrupt or using a file format that's not supported."

Image uploads are causing a 500 error.

Now, I have references the Box extension, available on Forge, in order to perform a multipart API call, so I'm sure the call format is right. I have, however, skipped the OnBeforeRequest and OnAfterResponse functions out.

Is there something I can do to make it work properly?

Thank you!

Hi Saurav,

Looking at your OML I see you have your authentication in there. You might want to remove that! Also, I see you have three different REST APIs defind all called "Yammer". I'd advise you to add those methods to a single Yammer REST API.

That said, have you set the Logging Level of the API to Full, so you can see exactly what is being sent to Yammer? If not, in order to do so, use Service Center, go to the eSpace (via Factory / eSpaces), select the Integrations tab, select the REST API, and set Logging Level to Full. Next run your test eSpace, and check the Integration Logging (Monitoring / Integrations).

Kilian Hekhuis wrote:

Hi Saurav,

Looking at your OML I see you have your authentication in there. You might want to remove that! Also, I see you have three different REST APIs defind all called "Yammer". I'd advise you to add those methods to a single Yammer REST API.

That said, have you set the Logging Level of the API to Full, so you can see exactly what is being sent to Yammer? If not, in order to do so, use Service Center, go to the eSpace (via Factory / eSpaces), select the Integrations tab, select the REST API, and set Logging Level to Full. Next run your test eSpace, and check the Integration Logging (Monitoring / Integrations).

Hi Kilian.

I had a look by changing the Logging Level. This is what the request looks like:

    POST https://www.yammer.com/api/v1/messages.json HTTP/1.1
    User-Agent: OutSystemsPlatform
    Content-Type: multipart/form-data; boundary=r9sYHBv3hfqoEy8geJ1CpJrw8qlKmFRq
    Authorization: Bearer XXXXX-XXXXXXXXXXXXXXXXX
    Host: www.yammer.com
    Content-Length: 68992

    <BINARY DATA>


Since I'm converting all the text to binary data before sending it out, I'm not able to see what is being sent. Unless this format is not right?

Edit: Just checked for .txt file (which is the only format which is working). Request is successful, and the request format looks the same as above.


Hi Saurav,

You ought to be able to see the multiple parts you create (with the Boundary you set), and within each part Base64-encoded binary data. There are online tools to decode Base64 data, so you can check whether the right data is sent.

Kilian Hekhuis wrote:

Hi Saurav,

You ought to be able to see the multiple parts you create (with the Boundary you set), and within each part Base64-encoded binary data. There are online tools to decode Base64 data, so you can check whether the right data is sent.

But that means that my request format is incorrect? Is it supposed to show up by default on Service Center?

Thanks.


Hi Saurav,

The header information seems correct, but since you didn't include the full trace I can't know for sure. Perhaps you can click the "Download HTTP Trace" button and attach an output here?

Kilian Hekhuis wrote:

Hi Saurav,

The header information seems correct, but since you didn't include the full trace I can't know for sure. Perhaps you can click the "Download HTTP Trace" button and attach an output here?

That is the entire request. I've attached the HTTP trace anyway.

Thanks, Kilian.


Mmm, ok, that's not what I would've expected. The binary data should very likely be Base64-encoded, and not be actual binary data.

Kilian Hekhuis wrote:

Mmm, ok, that's not what I would've expected. The binary data should very likely be Base64-encoded, and not be actual binary data.

Is that how it generally is on Outsystems?


Yammer wants form encoded content, and as far as I know, that means binary data should be base64 encoded. So it's not what OutSystems wants, but what Yammer wants (note that if you create or consume a REST service and have a Binary parameter, that will be base64 encoded as well).

Kilian Hekhuis wrote:

Yammer wants form encoded content, and as far as I know, that means binary data should be base64 encoded. So it's not what OutSystems wants, but what Yammer wants (note that if you create or consume a REST service and have a Binary parameter, that will be base64 encoded as well).

Hi Kilian,

You're right. Yammer does expect form encoded content. So is there any other way I can achieve this on Outsystems?

Thank you for your help.


Hi Saurav,

Instead of using the BinaryConcats, I'd use a string builder (from the Text system extension)*, lose the TextToBinaryData everywhere, and put a BinaryToBase64() around the Upload1.Content. This way, you should get all text and the binary nicely converted to Base64.

*You could just use the good ol' + operator to concatenate Text, but that's very inefficient and slow when using large pieces of text (like a Base64-encoded binary).

Kilian Hekhuis wrote:

Hi Saurav,

Instead of using the BinaryConcats, I'd use a string builder (from the Text system extension)*, lose the TextToBinaryData everywhere, and put a BinaryToBase64() around the Upload1.Content. This way, you should get all text and the binary nicely converted to Base64.

*You could just use the good ol' + operator to concatenate Text, but that's very inefficient and slow when using large pieces of text (like a Base64-encoded binary).

Hi Kilian.

I tried your method using Base64 conversion. Images are getting uploaded now, but its still corrupt. I've attached the HTTP trace of your method. It seems to be formatted currently, and the message is going through. Here's the error messages for .xlsx files and .jpeg files.

Thanks.


I used a Base64 decode to decode the XLSX file, and Excel indeed reports a problem. The funny thing is though that it is a valid ZIP file (which XLSX files are), containing what seems valid XML documents that comprise an Excel file. So the only thing I can think of is that whatever produces the Excel is doing something wrong. Can you attach the original XLSX that was sent in the message you attached above? Then I can try to compare them.

Kilian Hekhuis wrote:

I used a Base64 decode to decode the XLSX file, and Excel indeed reports a problem. The funny thing is though that it is a valid ZIP file (which XLSX files are), containing what seems valid XML documents that comprise an Excel file. So the only thing I can think of is that whatever produces the Excel is doing something wrong. Can you attach the original XLSX that was sent in the message you attached above? Then I can try to compare them.

I've attached it.

Thanks Kilian!


Hi Saurav,

The file sent via the API has a bunch of trailing bytes that shouldn't be there. Looking into the trace, I see there may be a malformatted boundery directly at the end of the Base64-encoded part:

+h4AAGRvY1Byb3BzL2NvcmUueG1sUEsBAi0AFAAGAAgAAAAhAGFJCRCJAQAAEQMAABAAAAAAAAAAAAAAAAAAeyEAAGRvY1Byb3BzL2FwcC54bWxQSwUGAAAAAAoACgCAAgAAOiQAAAAA------WebKitFormBoundarym0vCJKBpUYdCIWQG--

You can see that the boundary is appended directly to the Base64-part, without a newline. Could you try adding a newline after the Base-64 part?

Kilian Hekhuis wrote:

Hi Saurav,

The file sent via the API has a bunch of trailing bytes that shouldn't be there. Looking into the trace, I see there may be a malformatted boundery directly at the end of the Base64-encoded part:

+h4AAGRvY1Byb3BzL2NvcmUueG1sUEsBAi0AFAAGAAgAAAAhAGFJCRCJAQAAEQMAABAAAAAAAAAAAAAAAAAAeyEAAGRvY1Byb3BzL2FwcC54bWxQSwUGAAAAAAoACgCAAgAAOiQAAAAA------WebKitFormBoundarym0vCJKBpUYdCIWQG--

You can see that the boundary is appended directly to the Base64-part, without a newline. Could you try adding a newline after the Base-64 part?

Hi Kilian. I tried it. Same result. I've attached the HTTP trace.


Could you also attach the original PNG of this one so I can compare again?

Solution

Kilian Hekhuis wrote:

Could you also attach the original PNG of this one so I can compare again?

Hi Kilian.

I got in touch with Outsystems support. They helped me find a solution. It seems like a used an extra "\n" in my code, which caused the boundary to get combined with the Base64 data for some reason.

Thanks for your help though. :)


Solution

Hi Saurav,

I'm glad you found the solution. It seemed my hunch the boundary was involved was right :).

@Kilian Sir,

One thing I learnt through this that boundary plays very important role in multipart REST web services and people should read the API documentation very carefully. Also people need to study the HTTP request generated in postman or other tool very carefully and once it is done, consuming multipart REST web services is just like using aggregate  :)

Cheers!!

Hello,


For those who may be still facing this issue: I've published a Forge component which helps submitting multipart/form-data content.

https://www.outsystems.com/forge/component/3731/multipart-upload/


Thanks.