Process iterations

Process iterations

  

Hi there,


During development we stumbled upon a following problem:


We try to download data from external database.

1. We want to download data but not all at once! Let's say we Are downloading sales data grouped by product name

2. We want to perform downloading process for each product separately, therefore we do the following:

 a) we create 'for each' iteration loop that goes "by product"

 b) in the loop we run the operation process that downloads data for the given product.


PROBLEM:

Because the processes are asynchronous, loop does not wait for the end of the process, it initiates another one! Because of that connection to the external DB is swarmed with requests of processes and soon dies out. Is there any way to make the loop wait for the process to finish?


We can (and probably will) add some kind of semaphore, but is there any other way?


Warm regards,

Lukasz Cybula

Hi Lukasz,

As far as I can understand, you want to fetch a product, then use a process to do something with that product related records. And you want to do this for all products you have and waiting for the processing of one product to end to start all over again with the next product (fetch product related rows and execute the process).

A solution I came up (I don't if it is the better one), in the action after you place the "Lauch Process" node you should have a "mutex". That mutex should wait for the processs to end to release the flow:

  1. The System entity "Processes" contains all the processes you execute in your app. To get the specific process you can use the "ProcessId" output of the "Launch Process" node;
  2. Inside the "Processes" entity you have a attribute named "Status_Id" which is an identifier for the static entity "Process_Status" (present in the Processes API that you should reference);
  3. The mutex in your action should be an aggregate after the "Launch Process" node getting the status of the process and an "If" node which condition check if the status of the process is "Closed" (This is the status when the process is..closed); 
  4. While it is not "Closed" the "If" node will be continuously evaluating the condition till the processed is closed;
  5. To avoid an active listening of the "If" node (that would be something bad for the performance of the app) you can use a sleep action. Since there is no Sleep node available in OutSystems you can check this component or implement your own extension (you only need an action with a line of code).

    Something like this:

I don't know if this solves your problem entirely but I hope it can help you.

Regards,
Thomas

Hi Thomas,

This is pretty much the same solution we thought about ourselves, we were just hoping that there is some kind of built-in mechanism.

Good to know "we are not the only one" ;)


Regards,

Lukasz Cybula



ProcessExecutionExample.oml

Hi Lukasz,

Thomas solution is indeed possible and serves the purpose of running processes sequentially.

I would propose a different solution to better take advantage of the BPT engine and avoid having "sleeps" in your action/timer, which will make it take as long to run as the time all processes take. Check the module attached for an implementation example.

In my solution you have a "parent" process that takes care of executing the "child" processes (which fetch data for each product) and use a Wait activity to do the same thing as the "sleep" but all in a controlled fashion. You also have the advantage that the parent process only finishes when all child processes finish, so if you want to know whether all products are done or not (or monitor the time the full operation takes), you can just check the parent process.

All the processing and waiting is done in the BPT engine (no action/timer execution pending) and can be monitored easily using "Process Monitoring" screen in Service Center.

Hope it helps!