I am writing this to seek some expertise regarding some issues we are currently facing while developing an application on the OutSystems platform.
Issue Faced - Timeout with file upload
Our application is a log analyzer, designed to process .bin log files (time series data) uploaded by users.
We are encountering a problem where the upload of .bin files larger than 30MB results in a connection timeout error. To address this, we added a 'SetRequestTimeout' action from the HTTPRequestHandler at the beginning of the 'UploadOnClick' client action, setting the request timeout to -1. However, the connection timeout error persists when uploading larger .bin files. Experimented by increasing the server request timeout of server action "CreateorUpdate" action to 60, 120 etc, and that of the Post API's timeout to 60,120,180 etc, sometimes it works, sometimes not. Sometimes the RefreshData action(GetFiles) action also gets timed out even when there is only one file in the database. We are unsure if this is an inherent limitation of OutSystems for large file uploads, or if there is a workaround available for this.Would greatly appreciate any assistance with this question.
Note: The .bin files needed to be uploaded and processed has a size range of 40 MB - 200 MB generally.
Hi Nithin,
Are you getting a HTTP 413 (Content Too Large) or similar error?
Usually the process to upload big files is as follows:
If you do not have the factory configuration check this link:
https://www.outsystems.com/forge/component-overview/25/factory-configuration
Then used this documentation for the setup:
https://success.outsystems.com/support/troubleshooting/application_runtime/altering_file_upload_limits_in_outsystems_apps/
And finally note that in reactive the max file size is 250mb with appears to be good for what you need ;)
https://success.outsystems.com/documentation/11/reference/outsystems_language/interfaces/designing_screens/upload/
Hope this helps,
Rúben
Hi Rúben,
Thank you for your detailed response. I appreciate the insights you provided regarding the HTTP 413 error during file uploads.
Upon implementing your suggestions, specifically increasing the server request timeout and exploring the Factory Configuration App, I encountered an issue. When attempting to log in using my Service Studio credentials, I received the following error message: "The request failed with HTTP status 404: Not Found."
I would like to inquire whether access to the Factory Configuration App is restricted for users with free tier Service Studio accounts. Clarifying this point would greatly assist me in determining the feasibility of utilizing the Factory Configuration App to address the HTTP 413 error for file uploads exceeding 100 MB.
Best regards,
Nithin
Using the factory configuration app then increase the maximum request length allowed by the application server.
Factory configuration link:
Install this and create a new shared configuration and all should work fine.
Hi Joshua,
Thank you for your suggestion. I attempted to install the Factory Configuration App as per the provided link (Link). However, I encountered an issue with logging in using my Service Studio credentials, receiving the message "The request failed with HTTP status 404: Not Found."
Your assistance is much appreciated.
Did you set the frontend module server request timeout value?
Hi Stefan :)
I'm curious, any reason to do this at the module level?
I think that is not a good practice, you should only increase the server request timeout where you need. Also last time I checked this will lead to all server actions in the module being flagged by AI Mentor (not sure if this is still the case)...
If i understood the problem above correctly then the user initiates an action flow from the browser calling a client action that itself calls a server action. The frontend now waits the specified module timeout value. if it exceeds the value this leads to a timeout, which in turn leads to a failed server action call and aborted transaction.
You are of course right that increasing this value is not best-practice, but he wants to upload a large file to a server action. At least from my understanding. Personally i dont recommend storing any files in the outsystems database and instead use direct uploads to a AWS S3 bucket.
Best
Stefan
Hi Stefan,
Thanks for your suggestion! Adjusting the frontend module's server timeout value to 60 seconds resolved the connection timeout issue that I was facing before.
However, while addressing the file upload problem for files larger than 100 MB, I encountered a login issue with the Factory Configuration App. Trying to log in the app using the service studio credentials led to a message: "The request failed with HTTP status 404: Not Found."
Any insights on resolving this login issue would be greatly appreciated.
Best Regards,
Could it be that you are using a Personal environment? If so please note that the factory configuration app only works in the commercial editions of OutSystems.
In regards to large file upload i would strongly recommend to not upload it to the database but instead do i direct upload to an S3 store. You find more information here https://medium.com/itnext/file-uploads-from-outsystems-reactive-web-client-to-aws-s3-using-pre-signed-urls-18e9a42dee17
Thank you for the clarification regarding the Factory Configuration App. Yes, I am currently using a personal environment, and I appreciate your insight that the app is only functional in commercial editions of OutSystems.
I'll explore the alternative approach you suggested, opting for a direct upload to an S3 store.
If you have any further tips or considerations on this approach, I would be grateful for your guidance.
Creating a new shared factory configuration and associating it with your application should do the job.
Well depending on the file size, the network, etc.. that will most likely not be enough. That is why I recommended those two steps. One thing is the connection timeout and another is the maximum request length allowed by the application server.
Previously i have already faced this issue and i have looked for solution but i failed to find.
But i wrote my logic to resolve my issues.
Solution = I have used two apis, we can also achive With one api for that i should know the complete flow if you share your oml it will easy to me.
Hi Nani,
Thank you for sharing your experience with the file upload issue. I appreciate your willingness to help.
Certainly, I can provide you with the OML file. I'm attaching it to this message for your reference. Please feel free to review the logic and let me know your insights.
Looking forward to your feedback.
i dont have much time so couldn't spend more time on it, but i have implemented same logic flow for you it will work.
Check the the omls.
But it have one limitations first the file should be able to create in enitity then only logic flow work.
When creating a file it depends on file size and Network speed, if the network speed is slow increase the module server request time because it will take time to carry the binary file from client side to server side, if the file is created in entity you will have no problem your are good go.
After that go to the "TestApi_Core" module and open the "CreateOtherSideFile" expose api and add your logic in between the "BinaryDataSize" node and "assgin" node you can see in the image.
I have test the 80mb File, its working.
Thank you.