Reading in Delimited Txt Files Extension?

Reading in Delimited Txt Files Extension?

  
Hey guys, I am still quite new to outsystems and struggling lately, as many things are limited

Outsystems comes with excel to recordlist widget and this is very limited as my clients data are usually over a couple of millions thus excel is not an option, however, they are able to get tab delimited text files thus I need a widget somehow read that into the entity. It seems like there are no widget nor any extension in the forge so I was wondering how can I write my own extension? in C#? in Javascript? any video tutorial on making an extension? or a reading in tab delimited extension already there?

Thanks
David -

It's pretty easy to write this yourself, without any C# needed (though you could do it in C# if you really wanted to).

Do the following:

* Get the text of the file (Upload Widget + BinaryToText).
* String_Split using NewLine() as the delimiter.
* Iterate over that with For Widget.
* Inside the iteration, run String_Split on the .Current of the record list you are iterating over, passing in Chr(9) are the delimiter (ASCII code 9 is for Tab).
* Iterate over the second record list using For to get your field values.

You will need to get more sophisticated, of course, if you are handling a file that uses quote-delimited fields to handle field values containing tabs.

J.Ja
Hi JJ thanks for your solution tip, but I got it working yesterday and my solution is very similar. I'll post a screenshot below, not sure if anyone can make it simplier lol. All done in Text,Binary Extensions, this solution can also accept comma delimited as well


This solution probably has the worse performance. Our client do have tab delimited files with half a million to a million records with no more than 10 columns.
David -

For that number of rows, this is a bad approach. The issue is that you are going to need a TON of RAM, and you're making a giant collection with a million records it in.

For something of that size, you are better off using the Index function to get the position of Chr(13) to find where the line ends, and using Substr to read just one line at a time into a temporary variable, and use String_Split on that variable on the tabs or commas to get the fields.

This approach will require slightly more coding, but will have a huge improvement in speed.

J.Ja
David,
Concerning the amout of data, I think, it's even better to have a SQL/Oracle server import batchjob and then process it in OutSystems ...
David Chen wrote:

Please provide the code down load for this implementaion
Hi JJ thanks for your solution tip, but I got it working yesterday and my solution is very similar. I'll post a screenshot below, not sure if anyone can make it simplier lol. All done in Text,Binary Extensions, this solution can also accept comma delimited as well