HI Community,
There is the use case scenario where I wanted to download/extract a large amount of records (like 15Lakhs or 25Lakhs) in to only one CSV file. Since excel file has limitation to numbers of rows but that is not the case with csv file.
Let me know any pointers what should be the best approach so that there should not be connection time out issues.
Hey! I’ve faced a similar situation before when exporting large datasets, and the best balance I found was to handle the generation asynchronously in the background.
Instead of building the entire CSV during the user’s request (which will almost always hit a timeout), you can:
Trigger a background job (using a Timer or a BPT Process) when the user clicks “Export CSV”.
In that process, fetch data in batches (something like 100k rows per iteration) and build the CSV progressively in memory.
You can append each batch to a BinaryData variable or use a TextBuilder to handle the text efficiently.
Once all batches are processed, store the CSV temporarily (for example, in the app’s temporary storage or as a transient binary in the database — not in a network drive).
When it’s done, just expose a download link or send a notification to the user saying the export is ready.Hope this helps! GG.
Hi @Gustavo Gonçalves
I did the same as you mentioned above.
Generated binary file by csv util component and store that binary data file in a Outsystems table.
Binary data file size is almost 700MB (which is vary large in size)
I gave the download link to download this 700 MB file data which is ultimately gives time out error in the UI.
Definitely, We need another approach for this use case.
Hi @Rohan J ,
Trying to serve a 700MB binary file directly through the OutSystems UI via a download link is likely to cause timeouts, memory issues, or performance degradation, especially in browser-based environments.
Benefits:
If you must keep the file in OutSystems (not recommended for 700MB):
But this is complex and not ideal for UI download scenarios.
Hi @Rohan J
The most common enterprise solution in this case could be to compress the file.
CSV size is driven by row width, not row count alone. Compression ratio for CSV usually yields 60–85% reduction. So, around 1.2 GB CSV → 300–400 MB ZIP. So, the steps would be:
So, there would be still one file, faster download and less DB stress.