[Amazon Simple Storage Service (S3)] Downloading large (~100MB) files, any common pattern?
Reactive icon
Forge component by Platform Maintenance

Good day,

We're building a Reactive application that includes storing documents from Users. We already store the binaries itself on AWS S3 utilizing the Amazon Simple Storage Service component.
When Users download a single file we retrieve the binary from AWS and combine it with the metadata (filename) stored in our database to offer it as download.

Users were facing time outs for larger (>50MB) files. The default time-out of 10 seconds has already been increased to 120 seconds. However, Users sometimes still encounter time-outs. 

We understand everything has its limits and up/downloading gigabytes of data is better done on external services, but with a pretty decent 1MB/sec download speed a 100MB file can still take almost two minutes.

Searching around the forums and Forge, I see numerous (mobile) solutions for uploading large files (such as ReactiveMultipleFileUpload) but not downloading them. Ideally we would like to show a progress bar, similar to how modern browsers offer a large download.

We also tried the GetPresignedUrl from AmazonS3_IS, but unfortunately this only offers the binary blob without the filename.

Hopefully this common use case also has a common pattern we could follow in OutSystems/lowcode.

Thank you in advance :)


The best option is to create a presigned url as you already figured out by yourself. Unfortunately the S3 connector by OutSystems has limitations. What you are looking for is to create a presigned url including a custom response Content-Disposition header. To do this you have to extend the S3 connector.

1. Clone the AmazonS3_ext component using Integration Studio

2. Add a new non mandatory input property "ResponseContentDisposition" to the S3_GetPresignedUrl action.

3. Open the code in visual studio. Then open the AmazonS3_extConnector.cs file in the Connector folder. Modify the method like this.

4. Open the AmazonS3_ext.cs file in the root and modify the MssS3_GetPresignedUrl method accordingly.

5. Remove the NewtonsoftJson.dll reference and add a new NewtonsoftJson.dll reference using the dll from C:\Program Files\OutSystems\Service Studio 11\Service Studio\Newtonsoft.Json.dll (or wherever you installed Service Studio)

6. Build the solution and then publish the cloned extension in Integration Studio.

Next open AmazonS3_IS module. This is a protected module so you will create a clone automatically. 

Open Dependency Manager and remove all dependencies to the "old" extension module and add new dependencies to your new extension module. Then refresh the dependencies if necessary.

7. Update the Object_GetPreSignedUrl and add a non mandatory input parameter ResponseContentDisposition.

8. Pass the ResponseContentDisposition input parameter to the S3_GetPresignedUrl action in the flow.

Publish you custom Integration component and you are good to go,

Setting the ResponseContentDisposition is like this. Lets say you have a filename "MyPicture.png" stored in a property called PictureFileName. then you would set

ResponseContentDisposition -> "attachment; filename=""" + PictureFileName + """"

Note the escaped quotation marks.



Sorry for the late response (holidays :) ), but thanks for your elaborate reply!

We're not very fond of cloning and customizing components, as this will increase complexity and maintenance load for our applications. Especially components that are managed/published by OutSystems, as we'll miss out on future improvements. So it is unfortunate that it is actually possible with the S3 connector, but not offered to developers (in a low code way).

Hopefully anyone else (or OutSystems) is able to help out with this general use case (downloading files that take a while).

True. I already asked if they would incorporate my changes to the offical connector. Though i havent got a response yet.

Community GuidelinesBe kind and respectful, give credit to the original source of content, and search for duplicates before posting.