REST encoding problem

Victor Salvalagio Pereira

Solved

Question

Hi,

I'm consuming a REST web service from a client and the data returned has accents and other special characters. When the data has accents or special characters I get question marks where the characters should be. I believe this is a encoding problem.
Can I set the encoding of a REST?

I'm attaching a screenshot.

Thanks!

REST_Error.png

12 Nov 2015

João Rosado

Staff

Solution

Hi Victor,

The REST respects the character set specified in the response, so if for example the encoding is in UTF-8 the client needs to reply with "content-type:application/json;charset=UTF-8".
The platform by default assumes the default specified in the HTTP spec (ISO-8859-1) and not UTF-8 as some services assume the default is, so it can cause problems like that.

You can try to use a OnAfterResponse to:

and read it ResponseBinary
use the BinaryDataToText action from the BinaryData extension
set the result into the ResponseText

~~I'm not sure 100% if the ResponseBinary is filled when the output of the method is not a binary...~~
Edit: just tested and it's filled corretly, so you should have no problem doing this.

Tell me if you get it working.

Regards,
João Rosado

12 Nov 2015

2 replies

Last reply 22 Dec 2023

Show thread

Hide thread

Caio Santana Magalhães

MVP

João Rosado wrote:

and read it ResponseBinary
use the BinaryDataToText action from the BinaryData extension
set the result into the ResponseText

João,

Adding to your answer: in my case, I had to specify the encoding as "utf-8" when using the BinaryDataToText function in order for it to work.

Thanks.

27 May 2018

Steven Decock

Hi João,

I know this is an old thread, but I wanted to set some things straight here.

The source that is mentioned here as saying that "ISO-8859-1 is the default for HTTP" dates from 2006 and probably talks only about the default for text media types.

Indeed, before 2014, ISO-8859-1 was the default encoding for any text media types such as text/html, text/plain, ... (source).

Quote from the RFC 2616 spec:
"When no explicit charset parameter is provided by the sender, media subtypes of the "text" type are defined to have a default charset value of "ISO-8859-1" when received via HTTP"

Then in 2014, the RFC 7231 spec changed this to:
"The default charset of ISO-8859-1 for text media types has been removed; the default is now whatever the media type definition says." (see Appendix B)

So that means that the default charset for application/json (as used in REST) should be defined by the JSON spec (RFC 8259), which, in section 8.1 says:

"JSON text exchanged between systems that are not part of a closed ecosystem MUST be encoded using UTF-8"

Conclusion:

ISO-8859-1 is NOT, and probably never has been, the default encoding for Content-Type application/json. So this seems to be an incorrect implementation of the HTTP spec by OutSystems.

Kind regards,

Steven

22 Dec 2023

Stefano Valente

We just encountered this problem. thanks for the solution/answer.

Small addition:

The TextToBinary must contain the charset/encoding of the supplying system.

For us this was ISO-8591-1.

This is ofcourse a very bad way to decode: We only use the binary conversion util to translate encoded text.

06 Sep 2019

David Salgueiro

Using the BinaryDataToText, I did set encoding input to utf-8 and it worked.

23 Jun 2023

Bogdan Boglea

Hello,

I'm facing very similar issue. Although at first sight I thought it solved my issue, I found some cases where encoding is still incorrect (see below example)

If I didn't use the OnAfterResponse and convert the encoding, second row also had issues, but after the change you can see that first example is still wrong...

Any ideas? :)

01 Aug 2023

Community GuidelinesBe kind and respectful, give credit to the original source of content, and search for duplicates before posting.

See the full guidelines