Bulk csv to external database importer

i was working on an ssis integration and wrote a script task in c# for automating my imports from csv file to my DB's. It works great but i needed  help with removing commas from strings in pipes(|aaa,aaa| or |a|,|a|) in the csv. for example "Address, city wide". i want a function that can remove that comma(,) . i would paste a snippet of my code and what i have done so far

I wrote a script that automates import of csv from a script path into my database source. now i want to do some formatting before import in an automated fashion to allow successful imports. i was using ssis for integration


I have decided to build a tool for that purpose in outsystems platform. I need to automate bulk import of csv's into databases , were i can preset configs like delimeters and then select a sourcepath folder directory containing all csv and then import and imports all csv to the database similar to what the import data function does in sql management studio which imports a csv and creates a table in database along with all the csv data but in my case i want to be able to do bulk import of multiple csv fles at once. We do lots of huge imports in csv format to external databases and i think building a tool would help out a lot. I would like to know if there are any forge components that can fit my needs or at least give me  a good head start in this project.Thanks

Hi Tmlewin,

There are some great Forge plugins for CSV files.

Take a look at this page

https://success.outsystems.com/Evaluation/Integration/03_Import_and_export_Excel_CSV_and_XML_files

and this forge plugin

https://www.outsystems.com/forge/component-overview/636/csvutil

Additionally, OutSystems can import Excel files with the Excel to Record List widget.  If it is an option for you, import direct from Excel is very easy to do in OutSystems.

Hopefully, using those will mean you don't need to do the cleanup step removing the comma.  However, if that is still necessary, check out the Text API module of the platform, there are a number of actions which could be useful for you such as String_Split, and Regex_Replace.

Also, these built in functions should come in useful

  • Index(text, search, startIndex, searchFromEnd, ignoreCase)
  • Replace(text, search, replace) and
  • Substr(text, start, length)

Hope this helps!

Kind regards,

Stuart.

Thanks alot stuart for your help.  What I need is to import multiple csv files in bulk to an external database.i would have to setup a database connection in service  center. But my use case is to create a an app were I can setup a source folder path which is the directory  were all my csv files are and on import it imports all the csv files as tables in my external database.  I would explore the csv util if it has modules that can help me with my use case.