Specific File Type and Encoding

Hi Everyone,

This is a very specific problem so i'm just looking for some pointers here.

Basically i have to run a process which, after running, generates a file which has to be with UTF-8 BOM Encoding. This is a closed prerequisite, the Byte Order Mark has to be present.

Well i created an extension which runs the process, and i've tried the following:

1 - Outputting a string, calling TextToBinary and using Download Widget (Outputs utf-8 no BOM)

2 - Outputting binary array and using Download widget (Outputs utf-8 no BOM)

3 - and finally what i'm trying now is creating the file directly in the extension (.net using binaryWriter) outputting to a file and doing nothing on the Outsystems end afterwards. This i'm having no luck because either i don't have permissions to access the folder where i want to save the file OR maybe the extension just can't get there. 

It's probably worth it to say that i'm doing this in my personal cloud of Outsystems 10, however it is to be integrated on a production environment.


If anyone can help with some insights, maybe someone has had similar problems, or maybe limitations that i may not be aware of, i would greatly appreciate some help on this subject.


Best Regards,

Bruno Gonçalves

Solution

Hi Bruno,

You could use the Binary Concat Forge component, and concatenate the BOM to the rest of the file.

Solution

Thank you Kilian! 

Managed to inject the bom in the way described and it is recognized as UTF-8 BOM.


For further information, the string used for the BOM is as follows: ""


if you insert that string in the TextToBinary, it becomes the bom, which you can concatenate to the rest of your binary content.

Thank you again and Best Regards,

Bruno Gonçalves

Great you got it solved! Happy coding :)

Hello guys,

I'm having the same issue. I need to export a .txt file with BOM.

I tried to used the Kilian solution to concatenate the BOM this "", with the text part that is being generated by a list to csv.  Still not getting the desired result. 

Can you please check if is this the correct way to concatenate?


Best Regards


Hi Sergio,

The problem is you are converting text to binary using UTF-8, while "" is already UTF-8, so you're double converting it. You might try to use "ascii" instead of "utf-8".

Hi Kilian, 


thanks for your help, but unfortunatly I've tried using "ascii" with no luck. What it did was adding "" to the text of my list.

Actually what I am trying to achieve is saving a .txt file in UCS-2 Little Endian. This is what notepad does when I save it in Unicode.


Thanks

Sergio

Hi Sergio,

If you want to save using UCS-2 (or rather, UTF-16), you need to use "utf-16" instead of "utf-8" with TextToBinary, and use Chr(255) + Chr(254) with "ascii" for the BOM.

Hi Kilian,

thanks again for helping, but still no luck. Doing that saves the file in ANSI.

Have to manually open with notepad and save it Unicode.


Thanks



Hi Sergio,

I did some experimenting myself, and it seems my memory had failed me. Instead of using "ascii" as encoding for the BOM, do not use anything. So to get a binary using UTF-16 with a leading BOM, use a BinaryConcat, with as first parameter:

TextToBinaryData(Chr(254) + Chr(255))

and as second:

ConvertEncoding(TextToBinaryData(l_Text, "utf-8"), "utf-8", "utf-16")

The ConvertEncoding is needed because all text in OutSystems is utf-8, so first you need to convert the text to binary using utf-8, then convert this utf-8 encoding to utf-16.