Architecture

Detailed Functionalities

1. MultipartUpload_CW: Exposes core functionalities of the component.

• Block
  • MultipartUpload: Upload widget enables uploading multiple large files to OutSystems' database at once.
  • MultipartUpload_CSV: Upload widget specialized for uploading large CSV files efficiently by splitting them into 10MB chunks (ending at line boundaries), appending headers to each chunk, and uploading these as individual parts.
    • Ideal for data import scenarios, allowing batch processing of CSV data for reliability and performance.
  • MultipartUpload_S3: Same functionality as MultipartUpload block but upload files to AWS S3.
    • Supports adding prefixes to S3 objects.
  • MultipartUpload_S3_CSV: Same functionality as MultipartUpload_CSV block but upload files to AWS S3.
    • Supports adding prefixes to S3 objects.
• Client Actions
  • FileDownload: Download a file locally using REST API instead of OutSystems' Download Action given the File UID. Improved upload speed with progress tracking by the browser.
• Server Actions
  • FileDelete: Delete a file from the database given the File UID.
  • FileGet: Get the file information & binary from the database given the File UID.

2. MultipartUpload_API: REST API for file uploading & downloading.

• Integrations/REST/v1: Expose methods for single & multi-part file uploading, enables file download through REST API.
  • Custom authentication: Checks if the File UID exists in the request URL and if that File UID exists in the File table.
• Server Actions
  • GetPreSignedUrl_Download
  • GetPreSignedUrl_Multiple
  • GetPreSignedUrl_Single

3. MultipartUpload_CS: Database tables and Server Actions to support file uploading, downloading, and concatenating binary chunks.

• Database tables
  • AWS_S3_Config: Configurations to support AWS S3 upload.
  • File: File information (name, size, content-type).
  • FileBinary: File actual binary after concatenating chunks/parts.
  • FilePart: File"s binary chunks/parts.
• Server Actions
  • AbortMultipartUpload
  • CompleteMultipartUpload
  • CreateMultipartUpload
  • CheckHasProcessFinished
• Processes
  • ConcatFileParts: Invoke after all file parts have been uploaded and ready for concatenation.
  • DeleteFileParts: Invoke after concatenating file parts in the ConcatFileParts Process.

4. MultipartUpload_AmazonS3_Config: Admin screens (backoffice) to manage credentials for AWS S3 uploading.

• Screens
  • AWSS3Configs: List of existing AWS S3 configurations.
  • AWSS3ConfigDetail: Detail of each AWS S3 configuration, allows updating existing configurations or creating new ones.

5. MultipartUpload_AmazonS3_CS: Server Actions to validate, encrypt, decrypt, and store AWS S3 configurations.

• Server Actions
  • CryptoAPI folder: Get/set environment key, encrypt/decrypt data.
  • S3 folder: Wrappers around AWS S3 Actions, handle authentication to AWS.
  • S3_Config folder: Validate and get/set AWS S3 configurations.
• Role
  • MultipartUploadAdmin: Allows end-users to access screens in MultipartUpload_AmazonS3_Config module and update configurations.
• Site Properties
  • PresignedUrlExpiresIn: Number of seconds which after then the presigned url is expired, default = 1800 seconds (or 30 minutes).
  • SavedKey: Environment key for CryptoAPI.

6. MultipartUpload_AmazonS3_ext: Cloned from "Amazon Simple Storage Service (S3)" Forge asset to add custom methods

• Server Action
  • S3_GetPreSignedUrl_MultipartUpload
    • Accept two additional input parameters: UploadId and PartNumber.
    • Enables direct multipart uploading to AWS S3 using Presigned URLs, allowing clients to upload parts directly to S3 without routing data through OutSystems servers.

AWS S3 Setup

1. In AWS Console:

• Create a S3 bucket.
• In the "Permissions" tab, add "Cross-origin resource sharing (CORS)" settings:

[

    {

        "AllowedHeaders": [

            "*"

        ],

        "AllowedMethods": [

            "HEAD",

            "GET",

            "PUT",

            "POST",

            "DELETE"

        ],

        "AllowedOrigins": [

            "*"

        ],

        "ExposeHeaders": [

            "ETag"

        ]

    }

]

• Create an IAM user with permission to access the S3 bucket.
  • Remember to save the IAM user credentials.

2. In MultipartUpload_AmazonS3_Config module:

• Go to "https://<your enviroment>/MultipartUpload_AmazonS3_Config/".
• Click "Add Config +".
• In the AWSS3ConfigDetail screen, provide these info (mandatory):
  • Name: Name of the configuration, displayed in the AWS S3 Config List.
  • Access Key Id: IAM user credential.
  • Secret Access Key: IAM user credential.
  • Bucket Name: Name of the bucket you created in the previous step.
  • Region: For example, "APNortheast1" or "USWest1".
  • Entry Espace: Reactive or Mobile module referencing the MultipartUpload_CW module.
• Click "Save".

Notes:

• If you encounter a "Server Request Timeout" exception (especially when uploading files larger than 200MB), consider increasing the "Server Request Timeout" property of the "MultipartUpload_CW/Client Actions/MultipartUpload/CompleteMultipartUpload/CompleteMultipartUpload" Action node.

Dependencies

• OutSystems UI (built-in)
• Amazon Simple Storage Service (S3)
• Binary Concat
• CryptoAPI

Architecture

• MultipartUpload_CW (Core Widget) → MultipartUpload_API (Exposed API) → MultipartUpload_CS (Core Service) → MultipartUpload_AmazonS3_ext (Extension)
• MultipartUpload_AmazonS3_Config → MultipartUpload_CS (Core Service)

Detailed Functionalities

1. MultipartUpload_CW: Exposes core functionalities of the component.

• Block
  • MultipartUpload: Upload widget enables uploading multiple large files to OutSystems" database at once.
  • MultipartUpload_CSV: Upload widget specialized for uploading large CSV files efficiently by splitting them into 10MB chunks (ending at line boundaries), appending headers to each chunk, and uploading these as individual parts.
    • Ideal for data import scenarios, allowing batch processing of CSV data for reliability and performance.
  • MultipartUpload_S3: Same functionality as MultipartUpload block but upload files to AWS S3.
    • Supports adding prefixes to S3 objects.
  • MultipartUpload_S3_CSV: Same functionality as MultipartUpload_CSV block but upload files to AWS S3.
    • Supports adding prefixes to S3 objects.
• Client Actions
  • FileDownload: Download a file locally using REST API instead of OutSystems" Download Action given the File UID. Improved upload speed with progress tracking by the browser.
• Server Actions
  • FileDelete: Delete a file from the database given the File UID.
  • FileGet: Get the file information & binary from the database given the File UID.

2. MultipartUpload_API: REST API for file uploading & downloading.

• Integrations/REST/v1: Expose methods for single & multi-part file uploading, enables file download through REST API.
  • Custom authentication: Checks if the File UID exists in the request URL and if that File UID exists in the File table.
• Service Actions
  • GetPreSignedUrl_Download
  • GetPreSignedUrl_Multiple
  • GetPreSignedUrl_Single

3. MultipartUpload_CS: Database tables and Server Actions to support file uploading, downloading, and concatenating binary chunks; Server Actions to validate, encrypt, decrypt, and store AWS S3 configurations.

• Database tables
  • AWS_S3_Config: Configurations to support AWS S3 upload.
  • File: File information (name, size, content-type).
  • FileBinary: File actual binary after concatenating chunks/parts.
  • FilePart: File"s binary chunks/parts.
• Server Actions
  • AbortMultipartUpload
  • CompleteMultipartUpload
  • CreateMultipartUpload
  • New actions in S3 folder to support multi-part upload with AWS S3
  • New actions in S3_Config folder to validate, encrypt, decrypt, and store AWS S3 configurations.
• Processes
  • DeleteFileParts: Invoke after concatenating file in CompleteMultipartUpload action.

4. MultipartUpload_AmazonS3_ext: Cloned from "Amazon Simple Storage Service (S3)" Forge asset to add custom methods

• Server Actions
  • S3_GetPreSignedUrl_MultipartUpload
    • Enhanced to accept two additional input parameters: UploadId and PartNumber.
    • Enables direct multipart uploading to AWS S3 using Presigned URLs, allowing clients to upload parts directly to S3 without routing data through OutSystems servers.
    • This approach avoids the extra memory consumption associated with the standard Object_UploadPart action from AmazonS3_IS, which loads file parts into OutSystems server RAM before forwarding to S3.

5. MultipartUpload_AmazonS3_Config: Admin screens (backoffice) to manage credentials for AWS S3 uploading.

• Screens
  • AWSS3Configs: List of existing AWS S3 configurations.
  • AWSS3ConfigDetail: Detail of each AWS S3 configuration, allows updating existing configurations or creating new ones.

AWS S3 Setup

1. In AWS Console:

• Create a S3 bucket.
• In the "Permissions" tab, add "Cross-origin resource sharing (CORS)" settings:

[

    {

        "AllowedHeaders": [

            "*"

        ],

        "AllowedMethods": [

            "HEAD",

            "GET",

            "PUT",

            "POST",

            "DELETE"

        ],

        "AllowedOrigins": [

            "*"

        ],

        "ExposeHeaders": [

            "ETag"

        ]

    }

]

• Create an IAM user with permission to access the S3 bucket.
  • Remember to save the IAM user credentials.

2. In MultipartUpload_AmazonS3_Config module:

• Go to "https://<your enviroment>/MultipartUpload_AmazonS3_Config/".
• Click "Add Config +".
• In the AWSS3ConfigDetail screen, provide these info (mandatory):
  • Name: Name of the configuration, displayed in the AWS S3 Config List.
  • Access Key Id: IAM user credential.
  • Secret Access Key: IAM user credential.
  • Bucket Name: Name of the bucket you created in the previous step.
  • Region: For example, "APNortheast1" or "USWest1".
  • Entry Espace: Reactive or Mobile modules referencing the MultipartUpload_CW module.
• Click "Save".

Notes:

• If you encounter a "Server Request Timeout" exception (especially when uploading files larger than 200MB), consider increasing the "Server Request Timeout" property of the "MultipartUpload_CW/Client Actions/MultipartUpload/CompleteMultipartUpload/CompleteMultipartUpload" Action node.

Dependencies

• OutSystems UI (built-in)
• Amazon Simple Storage Service (S3)
• Binary Concat
• CryptoAPI

Architecture

• MultipartUpload_CW (Core Widget) → MultipartUpload_API (Exposed API) → MultipartUpload_CS (Core Service) → MultipartUpload_AmazonS3_ext (Extension)
• MultipartUpload_AmazonS3_Config → MultipartUpload_CS (Core Service)

Detailed Functionalities

1. MultipartUpload_CW: Exposes core functionalities of the component.

• Block
  • MultipartUpload: Upload widget enables uploading multiple large files to OutSystems" database at once.
  • MultipartUpload_CSV: Upload widget specialized for uploading large CSV files efficiently by splitting them into 10MB chunks (ending at line boundaries), appending headers to each chunk, and uploading these as individual parts.
    • Ideal for data import scenarios, allowing batch processing of CSV data for reliability and performance.
  • MultipartUpload_S3: Same functionality as MultipartUpload block but upload files to AWS S3.
    • Supports adding prefixes to S3 objects.
  • MultipartUpload_S3_CSV: Same functionality as MultipartUpload_CSV block but upload files to AWS S3.
    • Supports adding prefixes to S3 objects.
• Client Actions
  • FileDownload: Download a file locally using REST API instead of OutSystems" Download Action given the File UID. Improved upload speed with progress tracking by the browser.
• Server Actions
  • FileDelete: Delete a file from the database given the File UID.
  • FileGet: Get the file information & binary from the database given the File UID.

2. MultipartUpload_API: REST API for file uploading & downloading.

• Integrations/REST/v1: Expose methods for single & multi-part file uploading, enables file download through REST API.
  • Custom authentication: Checks if the File UID exists in the request URL and if that File UID exists in the File table.
• Service Actions
  • GetPreSignedUrl_Download
  • GetPreSignedUrl_Multiple
  • GetPreSignedUrl_Single

3. MultipartUpload_CS: Database tables and Server Actions to support file uploading, downloading, and concatenating binary chunks; Server Actions to validate, encrypt, decrypt, and store AWS S3 configurations.

• Database tables
  • AWS_S3_Config: Configurations to support AWS S3 upload.
  • File: File information (name, size, content-type).
  • FileBinary: File actual binary after concatenating chunks/parts.
  • FilePart: File"s binary chunks/parts.
• Server Actions
  • AbortMultipartUpload
  • CompleteMultipartUpload
  • CreateMultipartUpload
  • New actions in S3 folder to support multi-part upload with AWS S3
  • New actions in S3_Config folder to validate, encrypt, decrypt, and store AWS S3 configurations.
• Processes
  • DeleteFileParts: Invoke after concatenating file in CompleteMultipartUpload action.

4. MultipartUpload_AmazonS3_ext: Cloned from "Amazon Simple Storage Service (S3)" Forge asset to add custom methods

• Server Actions
  • S3_GetPreSignedUrl_MultipartUpload
    • Enhanced to accept two additional input parameters: UploadId and PartNumber.
    • Enables direct multipart uploading to AWS S3 using Presigned URLs, allowing clients to upload parts directly to S3 without routing data through OutSystems servers.
    • This approach avoids the extra memory consumption associated with the standard Object_UploadPart action from AmazonS3_IS, which loads file parts into OutSystems server RAM before forwarding to S3.

5. MultipartUpload_AmazonS3_Config: Admin screens (backoffice) to manage credentials for AWS S3 uploading.

• Screens
  • AWSS3Configs: List of existing AWS S3 configurations.
  • AWSS3ConfigDetail: Detail of each AWS S3 configuration, allows updating existing configurations or creating new ones.

AWS S3 Setup

1. In AWS Console:

• Create a S3 bucket.
• In the "Permissions" tab, add "Cross-origin resource sharing (CORS)" settings:

[

    {

        "AllowedHeaders": [

            "*"

        ],

        "AllowedMethods": [

            "HEAD",

            "GET",

            "PUT",

            "POST",

            "DELETE"

        ],

        "AllowedOrigins": [

            "*"

        ],

        "ExposeHeaders": [

            "ETag"

        ]

    }

]

• Create an IAM user with permission to access the S3 bucket.
  • Remember to save the IAM user credentials.

2. In MultipartUpload_AmazonS3_Config module:

• Go to "https://<your enviroment>/MultipartUpload_AmazonS3_Config/".
• Click "Add Config +".
• In the AWSS3ConfigDetail screen, provide these info (mandatory):
  • Name: Name of the configuration, displayed in the AWS S3 Config List.
  • Access Key Id: IAM user credential.
  • Secret Access Key: IAM user credential.
  • Bucket Name: Name of the bucket you created in the previous step.
  • Region: For example, "APNortheast1" or "USWest1".
  • Entry Espace: Reactive or Mobile modules referencing the MultipartUpload_CW module.
• Click "Save".

1.1.0 Update: denote in red

Notes:

• If you encounter a 'Server Request Timeout' exception (especially when uploading files larger than 200MB), consider increasing the 'Server Request Timeout' property of the 'MultipartUpload_CW/Client Actions/MultipartUpload/CompleteMultipartUpload/CompleteMultipartUpload' Action node.

Dependencies

• OutSystems UI (built-in)
• Amazon Simple Storage Service (S3)
• Binary Concat
• CryptoAPI

Architecture

• MultipartUpload_CW (Core Widget) → MultipartUpload_API (Exposed API) → MultipartUpload_CS (Core Service) → MultipartUpload_AmazonS3_ext (Extension)
• MultipartUpload_AmazonS3_Config → MultipartUpload_CS (Core Service)

Detailed Functionalities

1. MultipartUpload_CW: Exposes core functionalities of the component.

• Block
  • MultipartUpload: Upload widget enables uploading multiple large files to OutSystems' database at once.
  • MultipartUpload_CSV: Upload widget specialized for uploading large CSV files efficiently by splitting them into 10MB chunks (ending at line boundaries), appending headers to each chunk, and uploading these as individual parts.
    • Ideal for data import scenarios, allowing batch processing of CSV data for reliability and performance.
  • MultipartUpload_S3: Same functionality as MultipartUpload block but upload files to AWS S3.
    • Supports adding prefixes to S3 objects.
  • MultipartUpload_S3_CSV: Same functionality as MultipartUpload_CSV block but upload files to AWS S3.
    • Supports adding prefixes to S3 objects.
• Client Actions
  • FileDownload: Download a file locally using REST API instead of OutSystems' Download Action given the File UID. Improved upload speed with progress tracking by the browser.
• Server Actions
  • FileDelete: Delete a file from the database given the File UID.
  • FileGet: Get the file information & binary from the database given the File UID.

2. MultipartUpload_API: REST API for file uploading & downloading.

• Integrations/REST/v1: Expose methods for single & multi-part file uploading, enables file download through REST API.
  • Custom authentication: Checks if the File UID exists in the request URL and if that File UID exists in the File table.
• Service Actions
  • GetPreSignedUrl_Download
  • GetPreSignedUrl_Multiple
  • GetPreSignedUrl_Single

3. MultipartUpload_CS: Database tables and Server Actions to support file uploading, downloading, and concatenating binary chunks; Server Actions to validate, encrypt, decrypt, and store AWS S3 configurations.

• Database tables
  • AWS_S3_Config: Configurations to support AWS S3 upload.
  • File: File information (name, size, content-type).
  • FileBinary: File actual binary after concatenating chunks/parts.
  • FilePart: File's binary chunks/parts.
• Server Actions
  • AbortMultipartUpload
  • CompleteMultipartUpload
  • CreateMultipartUpload
  • New actions in S3 folder to support multi-part upload with AWS S3
  • New actions in S3_Config folder to validate, encrypt, decrypt, and store AWS S3 configurations.
• Processes
  • DeleteFileParts: Invoke after concatenating file in CompleteMultipartUpload action.

4. MultipartUpload_AmazonS3_ext: Cloned from 'Amazon Simple Storage Service (S3)' Forge asset to add custom methods

• Server Actions
  • S3_GetPreSignedUrl_MultipartUpload
    • Enhanced to accept two additional input parameters: UploadId and PartNumber.
    • Enables direct multipart uploading to AWS S3 using Presigned URLs, allowing clients to upload parts directly to S3 without routing data through OutSystems servers.
    • This approach avoids the extra memory consumption associated with the standard Object_UploadPart action from AmazonS3_IS, which loads file parts into OutSystems server RAM before forwarding to S3.

5. MultipartUpload_AmazonS3_Config: Admin screens (backoffice) to manage credentials for AWS S3 uploading.

• Screens
  • AWSS3Configs: List of existing AWS S3 configurations.
  • AWSS3ConfigDetail: Detail of each AWS S3 configuration, allows updating existing configurations or creating new ones.

AWS S3 Setup

1. In AWS Console:

• Create a S3 bucket.
• In the 'Permissions' tab, add 'Cross-origin resource sharing (CORS)' settings:

[

    {

        "AllowedHeaders": [

            "*"

        ],

        "AllowedMethods": [

            "HEAD",

            "GET",

            "PUT",

            "POST",

            "DELETE"

        ],

        "AllowedOrigins": [

            "*"

        ],

        "ExposeHeaders": [

            "ETag"

        ]

    }

]

• Create an IAM user with permission to access the S3 bucket.
  • Remember to save the IAM user credentials.

2. In MultipartUpload_AmazonS3_Config module:

• Go to “https://<your enviroment>/MultipartUpload_AmazonS3_Config/”.
• Click “Add Config +”.
• In the AWSS3ConfigDetail screen, provide these info (mandatory):
  • Name: Name of the configuration, displayed in the AWS S3 Config List.
  • Access Key Id: IAM user credential.
  • Secret Access Key: IAM user credential.
  • Bucket Name: Name of the bucket you created in the previous step.
  • Region: For example, 'APNortheast1' or 'USWest1'.
  • Entry Espace: Reactive or Mobile modules referencing the MultipartUpload_CW module.
• Click “Save”.

1.0.1 Update: denote in red

Notes:

If you encounter a 'Server Request Timeout' exception (especially when uploading files larger than 200MB), consider increasing the 'Server Request Timeout' property of the 'MultipartUpload_CW/Client Actions/MultipartUpload/CompleteMultipartUpload/CompleteMultipartUpload' Action node.

Dependencies

• OutSystems UI (built-in)
• Binary Concat
• Amazon Simple Storage Service (S3)

Architecture

MultipartUpload_CW (Core Widget) → MultipartUpload_API (Exposed API) → MultipartUpload_CS (Core Service) → MultipartUpload_AmazonS3_ext (Extension)

Detailed Functionalities

1. MultipartUpload_CW: Exposes core functionalities of the component.

• Block
  • MultipartUpload: Upload widget enables uploading multiple large files to OutSystems' database at once.
  • MultipartUpload_CSV: Upload widget specialized for uploading large CSV files efficiently by splitting them into 10MB chunks (ending at line boundaries), appending headers to each chunk, and uploading these as individual parts.
    • Ideal for data import scenarios, allowing batch processing of CSV data for reliability and performance.
  • MultipartUpload_S3: Same functionality as MultipartUpload block but upload files to AWS S3.
    • Supports adding prefixes to S3 objects.
  • MultipartUpload_S3_CSV: Same functionality as MultipartUpload_CSV block but upload files to AWS S3.
    • Supports adding prefixes to S3 objects.
• Client Actions
  • FileDownload: Download a file locally using REST API instead of OutSystems' Download Action given the File UID. Improved upload speed with progress tracking by the browser.
• Server Actions
  • FileDelete: Delete a file from the database given the File UID.
  • FileGet: Get the file information & binary from the database given the File UID.

2. MultipartUpload_API: REST API for file uploading & downloading.

• Integrations/REST/v1: Expose methods for single & multi-part file uploading, enables file download through REST API.
  • Custom authentication: Checks if the File UID exists in the request URL and if that File UID exists in the File table.
• Service Actions
  • GetPreSignedUrl_Download
  • GetPreSignedUrl_Multiple
  • GetPreSignedUrl_Single

3. MultipartUpload_CS: Database tables and Server Actions to support file uploading, downloading, and concatenating binary chunks.

• Database tables
  • File: File information (name, size, content-type).
  • FileBinary: File actual binary after concatenating chunks/parts.
  • FilePart: File's binary chunks/parts.
• Server Actions
  • AbortMultipartUpload
  • CompleteMultipartUpload
  • CreateMultipartUpload
  • New actions in S3 folder to support multi-part upload with AWS S3
• Processes
  • DeleteFileParts: Invoke after concatenating file in CompleteMultipartUpload action.

4. MultipartUpload_AmazonS3_ext: Cloned from 'Amazon Simple Storage Service (S3)' Forge asset to add custom methods

• Server Actions
  • S3_GetPreSignedUrl_MultipartUpload
    • Enhanced to accept two additional input parameters: UploadId and PartNumber.
    • Enables direct multipart uploading to AWS S3 using Presigned URLs, allowing clients to upload parts directly to S3 without routing data through OutSystems servers.
    • This approach avoids the extra memory consumption associated with the standard Object_UploadPart action from AmazonS3_IS, which loads file parts into OutSystems server RAM before forwarding to S3.

AWS S3 Setup

1. In AWS Console:

• Create a S3 bucket.
• In the 'Permissions' tab, add 'Cross-origin resource sharing (CORS)' settings:

[

    {

        "AllowedHeaders": [

            "*"

        ],

        "AllowedMethods": [

            "HEAD",

            "GET",

            "PUT",

            "POST",

            "DELETE"

        ],

        "AllowedOrigins": [

            "*"

        ],

        "ExposeHeaders": [

            "ETag"

        ]

    }

]

• Create an IAM user with permission to access the S3 bucket.
  • Remember to save the IAM user credentials.

2. In OutSystems Service Center:

• Go to module 'MultipartUpload_CS'.
• In the 'Site Properties' tab, provide these info:
  • Mandatory
    • AccessKeyId: IAM user credential.
    • BucketName: Name of the bucket you created in the previous step.
    • Region: For example, 'APNortheast1' or 'USEast1'.
    • SecretAccessKey: IAM user credential.
  • Optional
    • PresignedUrlExpiresIn: Time in seconds that the Presigned Urls are going to expire, defaults to 1800 seconds or 30 minutes.

Dependencies

• OutSystems UI (built-in)
• Binary Concat
• Amazon Simple Storage Service (S3)

Architecture

MultipartUpload_CW (Core Widget) → MultipartUpload_API (Exposed API) → MultipartUpload_CS (Core Service) → MultipartUpload_AmazonS3_ext (Extension)

Detailed Functionalities

1. MultipartUpload_CW: Exposes core functionalities of the component.

• Block
  • MultipartUpload: Upload widget enables uploading multiple large files to OutSystems' database at once.
  • MultipartUpload_CSV: Upload widget specialized for uploading large CSV files efficiently by splitting them into 10MB chunks (ending at line boundaries), appending headers to each chunk, and uploading these as individual parts.
    • Ideal for data import scenarios, allowing batch processing of CSV data for reliability and performance.
  • MultipartUpload_S3: Same functionality as MultipartUpload block but upload files to AWS S3.
    • Supports adding prefixes to S3 objects.
  • MultipartUpload_S3_CSV: Same functionality as MultipartUpload_CSV block but upload files to AWS S3.
    • Supports adding prefixes to S3 objects.
• Client Actions
  • FileDownload: Download a file locally using REST API instead of OutSystems' Download Action given the File UID. Improved upload speed with progress tracking by the browser.
• Server Actions
  • FileDelete: Delete a file from the database given the File UID.
  • FileGet: Get the file information & binary from the database given the File UID.

2. MultipartUpload_API: REST API for file uploading & downloading.

• Integrations/REST/v1: Expose methods for single & multi-part file uploading, enables file download through REST API.
• Service Actions
  • GetPreSignedUrl_Download
  • GetPreSignedUrl_Multiple
  • GetPreSignedUrl_Single

3. MultipartUpload_CS: Database tables and Server Actions to support file uploading, downloading, and concatenating binary chunks.

• Database tables
  • File: File information (name, size, content-type).
  • FileBinary: File actual binary after concatenating chunks/parts.
  • FilePart: File's binary chunks/parts.
• Server Actions
  • AbortMultipartUpload
  • CompleteMultipartUpload
  • CreateMultipartUpload
  • New actions in S3 folder to support multi-part upload with AWS S3
• Processes
  • DeleteFileParts: Invoke after concatenating file in CompleteMultipartUpload action.

4. MultipartUpload_AmazonS3_ext: Cloned from 'Amazon Simple Storage Service (S3)' Forge asset to add custom methods

• Server Actions
  • S3_GetPreSignedUrl_MultipartUpload
    • Enhanced to accept two additional input parameters: UploadId and PartNumber.
    • Enables direct multipart uploading to AWS S3 using Presigned URLs, allowing clients to upload parts directly to S3 without routing data through OutSystems servers.
    • This approach avoids the extra memory consumption associated with the standard Object_UploadPart action from AmazonS3_IS, which loads file parts into OutSystems server RAM before forwarding to S3.

AWS S3 Setup

1. In AWS Console:

• Create a S3 bucket.
• In the 'Permissions' tab, add 'Cross-origin resource sharing (CORS)' settings:

[

    {

        "AllowedHeaders": [

            "*"

        ],

        "AllowedMethods": [

            "HEAD",

            "GET",

            "PUT",

            "POST",

            "DELETE"

        ],

        "AllowedOrigins": [

            "*"

        ],

        "ExposeHeaders": [

            "ETag"

        ]

    }

]

• Create an IAM user with permission to access the S3 bucket.
  • Remember to save the IAM user credentials.

2. In OutSystems Service Center:

• Go to module 'MultipartUpload_CS'.
• In the 'Site Properties' tab, provide these info:
  • Mandatory
    • AccessKeyId: IAM user credential.
    • BucketName: Name of the bucket you created in the previous step.
    • Region: For example, 'APNortheast1' or 'USEast1'.
    • SecretAccessKey: IAM user credential.
  • Optional
    • PresignedUrlExpiresIn: Time in seconds that the Presigned Urls are going to expire, defaults to 1800 seconds or 30 minutes.

0.1.0 Update: denote in red

Dependencies

• OutSystems UI (built-in)
• Binary Concat

Architecture

MultipartUpload_CW (Core Widget) → MultipartUpload_API (Exposed API) → MultipartUpload_CS (Core Service)

Detailed Functionalities

1. MultipartUpload_CW: Exposes core functionalities of the component.

• Block
  • MultipartUpload: Upload widget enables uploading multiple large files at once.
  • MultipartUpload_CSV: New block specialized for uploading large CSV files efficiently by splitting them into 10MB chunks (ending at line boundaries), appending headers to each chunk, and uploading these as individual parts.
    • Ideal for data import scenarios, allowing batch processing of CSV data for reliability and performance.
• Client Actions
  • FileDownload: Download a file locally using REST API instead of OutSystems' Download Action given the File UID. Improved upload speed with progress tracking by the browser.
• Server Actions
  • FileDelete: Delete a file from the database given the File UID.
  • FileGet: Get the file information & binary from the database given the File UID.

2. MultipartUpload_API: REST API for file uploading & downloading.

• Integrations/REST/v1: Expose methods for single & multi-part file uploading, enables file download through REST API.
• Service Actions
  • GetPreSignedUrl_Download
  • GetPreSignedUrl_Multiple
  • GetPreSignedUrl_Single

3. MultipartUpload_CS: Database tables and Server Actions to support file uploading, downloading, and concatenating binary chunks.

• Database tables
  • File: File information (name, size, content-type).
  • FileBinary: File actual binary after concatenating chunks/parts.
  • FilePart: File's binary chunks/parts.
• Server Actions
  • AbortMultipartUpload
  • CompleteMultipartUpload
  • CreateMultipartUpload
• Processes
  • DeleteFileParts: Invoke after concatenating file in CompleteMultipartUpload action.

Dependencies

• OutSystems UI (built-in)
• Binary Concat

Architecture

MultipartUpload_CW (Core Widget) → MultipartUpload_API (Exposed API) → MultipartUpload_CS (Core Service)

Detailed Functionalities

1. MultipartUpload_CW: Exposes core functionalities of the component.

• Block
  • MultipartUpload: Upload widget enables uploading multiple large files at once.
• Client Actions
  • FileDownload: Download a file locally using REST API instead of OutSystems' Download Action given the File UID. Improved upload speed with progress tracking by the browser.
• Server Actions
  • FileDelete: Delete a file from the database given the File UID.
  • FileGet: Get the file information & binary from the database given the File UID.

2. MultipartUpload_API: REST API for file uploading & downloading.

• Integrations/REST/v1: Expose methods for single & multi-part file uploading, enables file download through REST API.
• Service Actions
  • GetPreSignedUrl_Download
  • GetPreSignedUrl_Multiple
  • GetPreSignedUrl_Single

3. MultipartUpload_CS: Database tables and Server Actions to support file uploading, downloading, and concatenating binary chunks.

• Database tables
  • File: File information (name, size, content-type).
  • FileBinary: File actual binary after concatenating chunks/parts.
  • FilePart: File's binary chunks/parts.
• Server Actions
  • AbortMultipartUpload
  • CompleteMultipartUpload
  • CreateMultipartUpload
• Processes
  • DeleteFileParts: Invoke after concatenating file in CompleteMultipartUpload action.