672
Views
31
Comments
Solved
How to upload CSV files to a database

I have successfully transferred the Excel file to the database, but I am unable to convert it into an Excel file using CSV and transfer it to the database. Can you help me?

Upload.oml
2023-02-20 05-20-57
Nam Nguyen
Solution

Hi Naku Sitsu,


The goal here is to upload csv file to the database.

Your input is csv file.

  • Use LoadCSV2RecordList of cvsutil to extract the file to RecordList. 
  • Perform insert to the database with the BulkInsert action. The best practice is to perform database operations once instead of iterating the list to insert a single record.

Your flow should be like this. 

Cheers, 
Nam

UserImage.jpg
Naku Sitsu

I have mastered this method(LoadCSV2RecordList). I want to try using the method of converting CSV to Excel and uploading it to the database.

UserImage.jpg
subhajit ghosh

Hi Nam Sitsu,

where do I get GetphysicalTableName and BulkInsert actions?

Many Thanks in advance.

UserImage.jpg
subhajit ghosh

can you share your oml Naku Sitsu..Thanks, I am a new learner want study your flow.

UserImage.jpg
subhajit ghosh

Hi Nam Nguyen ,

can you please share above screenshot oml file

2023-02-20 05-20-57
Nam Nguyen

Hi @subhajit ghosh,


Sorry, I can't share the oml because business involves.


Bulkinsert is in the forge.
GetPhysicalTable is an action that I create.  Here is the action detail.


First, you need {Entity} from Systems,

Then, filter it to get the Physical_Table_Name. Here are the filter conditions.



Important: This action and your table must be in the same module.

If you have any questions regarding this subject, please DM me.


Cheers,

Nam



UserImage.jpg
Naku Sitsu

CSV conversion to Excel file upload missing data | OutSystems 


Please review this article.

UserImage.jpg
subhajit ghosh

Hi @Nam Nguyen,

I am successfully uploaded data from excel file but for csv file still I am struggling.

Not able to understand what to fill up in (recordlist,startindex,maxnum areas(loadcsv2list)).

also for bulkupload(tablename,recordlist)

sharing my oml can you please help me out .

Thanks




UserImage.jpg
subhajit ghosh

Hi @Nam Nguyen,

I am successfully uploaded data from excel file but for csv file still I am struggling.

Not able to understand what to fill up in (recordlist,startindex,maxnum areas(loadcsv2list)).

also for bulkupload(tablename,recordlist)

sharing my oml can you please help me out .

Thanks




UserImage.jpg
subhajit ghosh

Hi @Nam Nguyen,

I am successfully uploaded data from excel file but for csv file still I am struggling.

Not able to understand what to fill up in (recordlist,startindex,maxnum areas(loadcsv2list)).

also for bulkupload(tablename,recordlist)

sharing my oml can you please help me out .

Thanks




UserImage.jpg
subhajit ghosh

Hi @Nam Nguyen,

I am successfully uploaded data from excel file but for csv file still I am struggling.

Not able to understand what to fill up in (recordlist,startindex,maxnum areas(loadcsv2list)).

also for bulkupload(tablename,recordlist)

sharing my oml can you please help me out .

Thanks




UserImage.jpg
subhajit ghosh

Hi @Nam Nguyen,

I am successfully uploaded data from excel file but for csv file still I am struggling.

Not able to understand what to fill up in (recordlist,startindex,maxnum areas(loadcsv2list)).

also for bulkupload(tablename,recordlist)

sharing my oml can you please help me out .

Thanks




UserImage.jpg
subhajit ghosh


Hi @Nam Nguyen,

I am successfully uploaded data from excel file but for csv file still I am struggling.

Not able to understand what to fill up in (recordlist,startindex,maxnum areas(loadcsv2list)).

also for bulkupload(tablename,recordlist)

sharing my oml can you please help me out .

Thanks

test.oml
testexcel - Copy.xlsx
2023-02-20 05-20-57
Nam Nguyen
UserImage.jpg
subhajit ghosh

Hi @nam nguyen,

I have checked your oml but the problem is you are using older version of that server action which show me a error.

but when I am using latest server action of that loadcsv2 record list it keep asking me start index max,maxnum 


do you have any idea about this??

Thanks 

2023-02-20 05-20-57
Nam Nguyen


I am using version 1.11.8 (latest version).

From the image you share, you are using modified extension.
Please re-check it.

Nam

UserImage.jpg
subhajit ghosh

Hi Nam Nguyen,

Thank you for your support. I have updated latest version and now not showing any error in my code but when I run the application, process showing done but not inserting any new data .

can you please recheck my code or oml.

sample- csv I have attached.


exceltextv1.oml
Income_OUC_Translation (1).csv
2023-02-20 05-20-57
Nam Nguyen

Hi @subhajit ghosh,


Your code is not running because LoadCSV2RecordList action is not success.
Check LoadCSV2RecordList.DebugInfo to view detail. 
The reason is the structure of record list is not the same with your csv.
First option:
Just insert column Id in your csv. Since the Identifier entity is auto number, so that the id will auto increase.

Note : I used csv file from Resources and that file is not contain Id column. Update or replace it with your csv.



Second option is to create a structure without id to receive your csv file. After that assign it to the original record list.


Cheers, 

Nam



UserImage.jpg
subhajit ghosh

Hi Nam Nguyen,

I have implemented first step now it is success fully uploading csv files But I want to learn the second step also I have created one structure but  how I will assign it to the original record list can you elaborate?

and one more question we have to use after loadcsv server action right?

cheers,

S G


UserImage.jpg
subhajit ghosh

@nam nguyen can you please suggest me any videos or link regarding that structure ?

2023-02-20 05-20-57
Nam Nguyen

Hi @subhajit ghosh,

The second option:

Instead of a RecordList with the same structure with your entity (Id,solution_space_name,.... ), you create a RecordList that has structure without Id (solution_space_name ,space_key,...) for the LoadCSV2RecordList action then assign it to the original RecordList ( the one with Id). 

With this approach, you don't have to create a column Id in your excel file because the new structure is match with your csv


Cheers, 
Nam

2024-01-04 15-15-51
Abed Al Banna

Hi Naku,


You are not able to convert the Excel file because the server action "ExcelToRecordList" you are using for that purpose is missing the "Sheet Name" property. Try passing it the name of the Excel sheet you would like to store in the database, you can also refer to the "BootstrapSheet1s" server action for the implementation.

Let me know if this solves your issue!


Best regards,

Aabed

UserImage.jpg
Naku Sitsu

Do you mean that I need to add a sheet name to convert CSV files into Excel files and transfer them to the database?

2024-01-04 15-15-51
Abed Al Banna

You should pass it the Sheet Name as it is inside Excel, and not the Excel file name. The Sheet Name in this case would be "Sheet 1".


2024-01-04 15-15-51
Abed Al Banna

If you would like to do that dynamically, you can leverage the Advanced Excel extension.

1. Click on Manage Dependencies and go to the Advanced Excel extension

2. Add "Worksheet_SelectByIndex" (to select the first of X sheet from the uploaded Excel file) and "Worksheet_GetName" (to get the name of the sheet you have selected) server actions

3. Add them to the flow right before the "ExcelToRecordList" server action, and provide the output of the "Worksheet_GetName" to the "ExcelToRecordList" as input. Below are some screenshots of the configuration:


 


 

UserImage.jpg
Naku Sitsu

I followed your instructions and the webpage has been unable to appear. What should I do?

Upload (1)_renrenren.oml
2024-01-04 15-15-51
Abed Al Banna

My bad, I missed out on 1 step.


1. Add "Workbook_Open_BinaryData" as a dependency from Advanced_Excel

2. Pass it the BinaryData as input

3. Pass the generated output to "Worksheet_SelectByIndex" as input for the "Workbook" property

4. Keep the rest of the code the same as before


This should work :)

UserImage.jpg
Naku Sitsu


Uploading Excel files is possible, but CSV files are still not possible, and the webpage cannot be displayed.

Please use my csv file. Thank you very much!

Upload (1)_renrenren.oml
User_Details.csv
User_Details.xlsx
UserImage.jpg
Jorz

Hi Naku,

Convert your CSV file to text before you store the data on database. for the conversion of file you can use the below forge component.

https://www.outsystems.com/forge/component-overview/636/csvutil


Thanks,

2023-02-20 05-20-57
Nam Nguyen
Solution

Hi Naku Sitsu,


The goal here is to upload csv file to the database.

Your input is csv file.

  • Use LoadCSV2RecordList of cvsutil to extract the file to RecordList. 
  • Perform insert to the database with the BulkInsert action. The best practice is to perform database operations once instead of iterating the list to insert a single record.

Your flow should be like this. 

Cheers, 
Nam

UserImage.jpg
Naku Sitsu

I have mastered this method(LoadCSV2RecordList). I want to try using the method of converting CSV to Excel and uploading it to the database.

UserImage.jpg
subhajit ghosh

Hi Nam Sitsu,

where do I get GetphysicalTableName and BulkInsert actions?

Many Thanks in advance.

UserImage.jpg
subhajit ghosh

can you share your oml Naku Sitsu..Thanks, I am a new learner want study your flow.

UserImage.jpg
subhajit ghosh

Hi Nam Nguyen ,

can you please share above screenshot oml file

2023-02-20 05-20-57
Nam Nguyen

Hi @subhajit ghosh,


Sorry, I can't share the oml because business involves.


Bulkinsert is in the forge.
GetPhysicalTable is an action that I create.  Here is the action detail.


First, you need {Entity} from Systems,

Then, filter it to get the Physical_Table_Name. Here are the filter conditions.



Important: This action and your table must be in the same module.

If you have any questions regarding this subject, please DM me.


Cheers,

Nam



UserImage.jpg
Naku Sitsu

CSV conversion to Excel file upload missing data | OutSystems 


Please review this article.

UserImage.jpg
subhajit ghosh

Hi @Nam Nguyen,

I am successfully uploaded data from excel file but for csv file still I am struggling.

Not able to understand what to fill up in (recordlist,startindex,maxnum areas(loadcsv2list)).

also for bulkupload(tablename,recordlist)

sharing my oml can you please help me out .

Thanks




UserImage.jpg
subhajit ghosh

Hi @Nam Nguyen,

I am successfully uploaded data from excel file but for csv file still I am struggling.

Not able to understand what to fill up in (recordlist,startindex,maxnum areas(loadcsv2list)).

also for bulkupload(tablename,recordlist)

sharing my oml can you please help me out .

Thanks




UserImage.jpg
subhajit ghosh

Hi @Nam Nguyen,

I am successfully uploaded data from excel file but for csv file still I am struggling.

Not able to understand what to fill up in (recordlist,startindex,maxnum areas(loadcsv2list)).

also for bulkupload(tablename,recordlist)

sharing my oml can you please help me out .

Thanks




UserImage.jpg
subhajit ghosh

Hi @Nam Nguyen,

I am successfully uploaded data from excel file but for csv file still I am struggling.

Not able to understand what to fill up in (recordlist,startindex,maxnum areas(loadcsv2list)).

also for bulkupload(tablename,recordlist)

sharing my oml can you please help me out .

Thanks




UserImage.jpg
subhajit ghosh

Hi @Nam Nguyen,

I am successfully uploaded data from excel file but for csv file still I am struggling.

Not able to understand what to fill up in (recordlist,startindex,maxnum areas(loadcsv2list)).

also for bulkupload(tablename,recordlist)

sharing my oml can you please help me out .

Thanks




UserImage.jpg
subhajit ghosh


Hi @Nam Nguyen,

I am successfully uploaded data from excel file but for csv file still I am struggling.

Not able to understand what to fill up in (recordlist,startindex,maxnum areas(loadcsv2list)).

also for bulkupload(tablename,recordlist)

sharing my oml can you please help me out .

Thanks

test.oml
testexcel - Copy.xlsx
2023-02-20 05-20-57
Nam Nguyen
UserImage.jpg
subhajit ghosh

Hi @nam nguyen,

I have checked your oml but the problem is you are using older version of that server action which show me a error.

but when I am using latest server action of that loadcsv2 record list it keep asking me start index max,maxnum 


do you have any idea about this??

Thanks 

2023-02-20 05-20-57
Nam Nguyen


I am using version 1.11.8 (latest version).

From the image you share, you are using modified extension.
Please re-check it.

Nam

UserImage.jpg
subhajit ghosh

Hi Nam Nguyen,

Thank you for your support. I have updated latest version and now not showing any error in my code but when I run the application, process showing done but not inserting any new data .

can you please recheck my code or oml.

sample- csv I have attached.


exceltextv1.oml
Income_OUC_Translation (1).csv
2023-02-20 05-20-57
Nam Nguyen

Hi @subhajit ghosh,


Your code is not running because LoadCSV2RecordList action is not success.
Check LoadCSV2RecordList.DebugInfo to view detail. 
The reason is the structure of record list is not the same with your csv.
First option:
Just insert column Id in your csv. Since the Identifier entity is auto number, so that the id will auto increase.

Note : I used csv file from Resources and that file is not contain Id column. Update or replace it with your csv.



Second option is to create a structure without id to receive your csv file. After that assign it to the original record list.


Cheers, 

Nam



UserImage.jpg
subhajit ghosh

Hi Nam Nguyen,

I have implemented first step now it is success fully uploading csv files But I want to learn the second step also I have created one structure but  how I will assign it to the original record list can you elaborate?

and one more question we have to use after loadcsv server action right?

cheers,

S G


UserImage.jpg
subhajit ghosh

@nam nguyen can you please suggest me any videos or link regarding that structure ?

2023-02-20 05-20-57
Nam Nguyen

Hi @subhajit ghosh,

The second option:

Instead of a RecordList with the same structure with your entity (Id,solution_space_name,.... ), you create a RecordList that has structure without Id (solution_space_name ,space_key,...) for the LoadCSV2RecordList action then assign it to the original RecordList ( the one with Id). 

With this approach, you don't have to create a column Id in your excel file because the new structure is match with your csv


Cheers, 
Nam

UserImage.jpg
Nani

Hi

Naku Sitsu

Is your issue resloved then mark a solution.

if not just let me know what is the issue your facing and what requirements need.

Community GuidelinesBe kind and respectful, give credit to the original source of content, and search for duplicates before posting.