12
Views
14
Comments
Solved
API response in East Asian text cannot be processed in OutSystems
Question

Greetings all,

I realized an API response in East Asian text cannot be processed in OutSystems. It seems as though East Asian text gets transformed into strange symbols immediately upon entering the OutSystems environment.

How can we address this?


Thanks!

mvp_badge
MVP
Solution

Hi YS,

application/json is fine, since there's JSON returned. However, there's a problem with the charset: it should be "utf-8" (with a dash), not "utf8". So that's a bug in the other party's REST service! If the other party can't change this, then indeed you should the On After Response to fix the charset so that it reads "utf-8".

Can you try to test this API with postman?

I tried with Postman. It works with Postman (see image). But not with OutSystems ):

mvp_badge
MVP

Hi YS,

Have you increased the REST logging and checked the output? Does it show wrong there as well? Did you try to display the result on the screen? Does it go wrong there?

mvp_badge
MVP

Hi YS,

OutSystems is fully Unicode compliant, so if the East Asian (Chinese? Japanese? Korean?) text is sent as Unicode (utf-8) it should work.

Thanks Kilian! It is Chinese. Is there any possible reason why it works with Postman (see image above) but not with OutSystems?

mvp_badge
MVP

The only thing I know of that can be not Unicode-compliant is the database (like on this Forum, you can't write Chinese, it will end up gibberish), but with REST-services I haven't seen this. The only thing I can think of is that the REST service doesn't  send the right encoding string so the Platform doesn't know it's in UTF-8, hence my recommendation in my other post to increase the logging level and check the exact reply sent.

mvp_badge
MVP

I just did a quick test, but sending and receiving Chinese text from OutSystems itself is no problem, as expected:

Thanks Kilian,

I followed your suggestion of setting logging level to "full" and displaying the output on a screen. It is still gibberish ):

Is it possible that it works for traditional Chinese but not simplified Chinese? (As the first line in your quick test is traditional Chinese.)


Thanks!

mvp_badge
MVP

Both traditional and simplified Chinese use the same Unicode encoding, so I don't think that can be a problem. It would be interesting however to see what the value of the Content-Type header is, it should be this:

Content-Type: text/plain; charset=utf-8

If the "utf-8" is missing, or has a different value, it might be the Platform is getting confused about the right character set.

 

Greetings Kilian,

The Content-Type is application/json instead of text/plain.

Should I just change the response format to plain text?:

Or should I do a text manipulation in the On After Response?

Thanks!

mvp_badge
MVP
Solution

Hi YS,

application/json is fine, since there's JSON returned. However, there's a problem with the charset: it should be "utf-8" (with a dash), not "utf8". So that's a bug in the other party's REST service! If the other party can't change this, then indeed you should the On After Response to fix the charset so that it reads "utf-8".

That's very sharp, Kilian! (:

I implemented an On After Response:

However, the screen still shows gibberish. Perhaps my modification comes too late in the flow and the platform is already confused?

mvp_badge
MVP

Yes, that could be, I don't know exactly when the Platform interprets the data in the body. I would expect that it would do that after the modification of the headers.

As for your change, I'd use a ListIndexOf to check for the header containing the "charset=", then using the the result to index the Headers to do the Replace. This ensures you really are changing the right header.

Community GuidelinesBe kind and respectful, give credit to the original source of content, and search for duplicates before posting.