Hello Internet
I've been trying to connect the DeepL REST API for two days now and I'm failing with the "encoding". As you can see on this DemoPage (OML attached):
https://personal-wlfdncte.outsystemscloud.com/DeepLDemo/DeepL
texts e.g. to Greek, Chinese or Japanese are not translated correctly but only converted into a mess of numbers and characters. I have read various forum entries and tried to take them into account. Unfortunately it did not work or I did something wrong.
Would be great if you could show me my mistake.
Kind regards
Daniel
PS: DeepLAPI Key is included. Will change it after a solution is found.
LINKS:
Hi Daniel,
Please find attached the solution require to handle the encoding correctly, all required logic in OnAfterResponse:
To deal with the Japanese characters correctly we need to deal with the encoding, consumed service is not defining "Content-Type" header with "charset=utf-8" (example response in postman):
I created a new TEST page and a new REST reference ApifreeDeepl2 to deal with this.
This page also helps http://string-functions.com/encodedecode.aspx to deal with encodings:
What's happing is that OutSystems when consumes a rest api and the charset is not specified we use default encoding specified in the rfc of the http definition that is iso-8859-1, in this case we need to force to use the utf-8 in the way described attached.
Hope this helps you!
Thanks for this great answer, I hope the OP can solve their problem now. However, I noticed the first image in your post is broken, perhaps you can try to fix that? Thanks!
Image fixed! Thank you for your feedback.
Thx!
@Daniel Martins ,Since 2014 this is not a fact.
When not specified, the default encoding to consider should be UTF-8, and not ISO-8859-1.
According to RFC7231 - Hypertext Transfer Protocol (HTTP/1.1): Semantics and Content https://datatracker.ietf.org/doc/html/rfc7231#appendix-B
"The default charset of ISO-8859-1 for text media types has been removed; the default is now whatever the media type definition says."
And according to RFC7159 - The JavaScript Object Notation (JSON) Data Interchange Formathttps://datatracker.ietf.org/doc/html/rfc7159#section-8.1
"JSON text SHALL be encoded in UTF-8, UTF-16, or UTF-32. The default encoding is UTF-8, and JSON texts that are encoded in UTF-8 are interoperable in the sense that they will be read successfully by the maximum number of implementations"
Nontheless, there workaround works.
Tiago,
You are obviously wrong, as the OP faced this problem, and Daniel's solution solved it. There is a difference between what some RFC says, and the way the world (and in this case, OutSystems) interprets it. So unless you can prove that this problem doesn't exist anymore in the OutSystems Platform, your answer (posted 3.5 years after the last reply!) isn't of much value.
Hi Kilian, nice to meet you to.
Of course that if I'm replying do this, is because I have crossed an equivalent issue and found this post to find a way to workaround it in the platform.
And my response is based on an RFC (as Daniel also did), because it sustains our technical world on how we should behave. OutSystems is a platform that should follow standards (as it does most of the times), but is not the golden buzzer for the truth.
What a stroll down to the memory lane, that your reply took me!
Glad that you found the workaround, that's the point of this community and solution given to this particular case.
I believe you got the point, I can't recall exactly why I replied 4 years ago that you should expect the OutSystems platform to make use of iso-8859-1 enconding by default.
Nevertheless, nowadays given chatgpt's and modern search engines like Brave AI search...it seems pretty standard assumption that UTF-8 should be use as default:
"The default encoding for REST APIs is typically UTF-8, as it ensures that the response content can always be correctly rendered and covers the full range of characters supported by the application. This is the default and generally preferred encoding for REST APIs."
If current version of the platform still assumes iso-8859-1, you should give the inputs gathered in this post and submit support case.
Best wishes!
I'll see if I can find someone at OutSystems to confirm whether ISO-8859-1 is the default, or whether it's UTF-8 like it should.
OutSystems has confirmed that currently, ISO-8859-1 is indeed the default for REST responses in absence of an explicit charset. I've asked them to consider changing that, but meanwhile, the answer given in this topic still stands.
Ok, last update: since changing it would be a breaking change, it will not be changed. The documentation will be updated to reflect the actual working though.
God damn, finally I got this to work! Thanks to you.
I haven't had any issues like with APIs before. When I tried a GET method with 'Test', it brought back all letters like it should with UTF-8: the Scandinavian letter "ä" was "ä" and "ö" was "ö".
But, for some reason, when the response was assigned to a structure, that was by the way automatically created when the body was copied to the response after testing, debugging and UI brought the letter "ä" back as "ä" and "ö" as "ö".
I was hitting my head to a wall for a few hours, and actually tried very similar methods but didn't get it to work. At some point I think I might have gotten it to work, we never know, because I might have forgotten to refresh dependencies at some point.Anyway, after doing what you did in OnAfterResponse, the letters showed as they should. I'm grateful.
Thanks, works perfect!
Thanks for the demo, it also solved the problem here.
Solved my problem calling GPT API with this. Thanks you! upvoted.