ASCII Text in a Binary file doesn't show special characters correctly

ASCII Text in a Binary file doesn't show special characters correctly

Hi Guys,

I have some ASCII file as a resource in service studio. There is an action that convert this binary file to text and updates the database line for line. When encountering special characters like " the database doesn’t save the correct character but something like this: ?
We tried to use the encodesql() function but it gives the same result.

When first converting the same file to UTF8 format and use this file in service studio the same action runs like expected and the special characters are saved correct in the database.

Can someone explain to me why there is different behaviour and if there is a way to achieve the correct behaviour with the ASCII files?

Hi Martijn,

You supply little information about the actual actions used to convert the ASCII file to text. I can make some observations however:
1) You talk about "this binary file", but an ASCII file is typically not considered binary. It is text. And in fact, plain ASCII (characters 32-127) map 1:1 to UTF-8, so converting to UTF-8 shouldn't have any effect!
2) You talk about "an action that converts" - is that action under your control? If so, what action does it use to "convert" the file? If you use BinaryDataToText (from the BinaryData extension), there's a parameter "encoding" that lets you specify the encoding of the source file. Is it set to "ascii"?
3) You say you have the file "as a resource". But if you do, what's the problem using the "UTF-format" file and including that as a resource? I can understand there's a problem if the user can upload a file, or you read something from a file system or via FTP, but if it's a fixed resource you can prepare it beforehand?
Hi Kilian,

Just a quick reply.
Yes we use the BinaryDataToText action and No we don't use the encoding parameter. I'm going to try this now. Can it be that the default of this action uses UTF-8? Because then it would make sense right?

Further at the moment we use the files indeed as recourse and we can fixe the files beforehand. The plan is that the user can upload the files on runtime.
Hi Martijn,

It wouldn't surprise me if the default is UTF-8, and I it might be that the ASCII files aren't true ASCII but something like Windows-1252 (because, as I stated above, true ASCII is a subset of UTF-8).
Hi Kilian,

I think you are right, that the original file isn't ASCII. I'm getting the feeling this has something to do with the settings of the database and the runtime setting of the session. There isn't a quick fix what I hoped for.