Performance Optimisation for OutSystems ListAppendAll Function
456
Views
3
Comments
New
Builtin & User functions

The proposal aims to enhance the performance of the OutSystems built-in ListAppendAll function. Currently, ListAppendAll iterates over the source list using a foreach loop, invoking the ListAppend action on each iteration. This results in multiple dynamic array resizings, as each call to ListAppend extends the underlying array incrementally, leading to performance overhead due to repeated memory allocations and data copies.

In contrast, in native C#, the List.AddRange method optimises this process by pre-allocating the required space in the underlying array in a single operation. It then efficiently inserts elements using Array.Copy, minimising reallocations and improving execution time. Implementing a similar bulk append mechanism in ListAppendAll would reduce memory overhead and improve performance.

The attached files include the BenchmarkDotNet results and the code used for performance testing.

Tested on .NET 4.8.1 and Intel Core i7 12800HX processor.

While the code does not exactly replicate the OutSystems implementation, it follows a similar approach to demonstrate potential performance improvements.

2025_02_25_13_48_39__ConsoleApp_ListBenchMark[1].cs
ConsoleApp_ListBenchMark.ListInsertBenchmark.html

Your proposal to optimize the ListAppendAll function by pre-allocating space and using efficient copying (similar to List.AddRange in C#) is a great idea. Further improvements could include minimizing reallocations by dynamically growing the list size and leveraging parallelization for large datasets. Testing across real-world scenarios and ensuring compatibility with the OutSystems framework would make the proposal even stronger. This could significantly enhance performance in data-intensive applications. 

You should create a forge component for this to allow others to take advantage of the tool.

For generic lists (text, number, dates etc) is should be easy to put it in to a forge component, however... I don't know how much of a performance loss there would be having to pass both lists to the component and then get the new list back again.

For custom lists a generic template could be added to the forge for people to customise themselves, but again, it's unclear how it would perform.