How to Deal with Complex Data Structures
A data structure is a specialized format for organizing, processing, retrieving and storing data. They’re designed to arrange data to suit a specific purpose so that it can be accessed and worked with in appropriate ways.
There are two types of data structures:
- Linear data structures, where data is stored sequentially, such as in a stack, linked list, array or queue.
- Non-linear data structures, such as graphs and trees, that store data non-sequentially.
When it comes to choosing between a linear and non-linear data structure, there is no right or wrong answer – it’s a matter of choosing the right one for the job at hand. This isn’t a problem when working with programming languages like Java that have the ability to work with both types. However, with OutSystems, you need to approach data structures in a slightly different way.
That’s what I’ll show you in this article.
Understanding Data Structures in OutSystems
In the OutSystems platform, a structure is a compound data type used to encapsulate groups of related attributes – it’s a custom data type that you can use in your model. Unlike other programming languages, OutSystems structures can be used on the server side as well as the client side.
OutSystems structures are a collection of attributes that support different data types, such as basic types like text or integers, along with more complex data types like records. While structures can be as complex as you can imagine, there are a few things you can do differently in OutSystems to overcome limitations or increase performance. Let’s look at two scenarios.
Scenario 1: Recursive Nested Structure with Undefined Depth
A recursive structure is one that holds itself as an attribute. OutSystems doesn’t allow recursive structures and requires that structure depth be defined at design instead of allowing depth to be dynamic.
One possible solution is to transform the recursive structure into a flattened list structure. To do so, you modify the data from a REST API by setting an OnAfterResponse. In the OnAfterResponse callback action, manipulate the response data by using something like the “ardoJSON” component from the Forge to flatten the JSON and make it match one of your structures. OutSystems will then use the OnAfterResponse before returning back from the REST call.
Another solution is to use .NET code to pass the API response as text input to a C# extension that would take JSON as an input, which would then allow us to deserialize the text into hierarchical objects that you can flatten into a hierarchical database structure list that can be used in OutSystems.
Scenario #2: Hierarchical Defined Depth Structures with Key Value List
Because key value pairs can exist deep within a hierarchical defined depth structure, retrieving data from this type of structure can impact application performance. Traditionally, you would apply a filter at each level to get your desired node and then retrieve the property value for a given key. However, this can result in a heavy cost in terms of performance, especially if there are several property values to be retrieved.
One way to solve this problem is to transform the data into key value pairs and then deserialize them to be used in the OutSystems structure. Again, “ardoJSON” is the perfect tool for this task: use JSONSelect to select properties from JSON, and then use JSONListify to transform it into the desired format. Finally, you would deserialize the transformed JSON to the list of Property (Key Value) structures.
By transforming a nested structure with a defined depth into hashmap objects, you can achieve a significant increase in data retrieval performance. Hashmap objects can be future-enhanced to optimize performance based on application requirements.
Complex Data Structures Made Easy
OutSystems makes working with data very easy, but that simplicity can make it hard to understand how to deal with more complex use cases. In their session Dealing with Complex Structures in OutSystems, my colleagues Baghel, OutSystems Architect, and Junaid Syed, Technical Lead, have done a wonderful job in demystifying how to deal with these cases, and I definitely recommend checking out what they have to say.