We have a system-to-system interface with several vendors to retrieve data. The interface is based upon REST whereby we start the data loads with timers. We retrieve 165K records with one vendor in particular (on a daily basis). This web services takes too long, and it generate a timeout error. Our Timer Execution Attempts is set to 3 and the timeout in minutes for the timer is 20 minutes. The challenge we have is that we noticed that the record count started to increase while loading the data records from our vendor to OutSystems. But after the time out the record count was exactly the same as prior to the data load.

The records are simply stored in the memory while the web service is active? Any suggestions what we can do differently to store the data records prior to the time out of the web service? See attached screenshot for our business logic. 



Hello Jeffrey

It seems that the problem is that the ACTION called by the timer successfully calls the web service and retrieve the data, but than you have a timeout before the process is completed.

When this happens, by default, the transaction will be rollout and no data changes will persist in the storage.
It is the database who takes care of keep it in a "temporary" state before the end of the transaction.

I noticed in the image that the web service can be called many times, depending on the response (multiple pages, maybe?). In this case, you should implement logic to check if the timeout is about to expire before calling the web service again. You can do it computing, at the begining of the action, from the current datetime + the timeout - safe time (like 10% of the timeout) to know when to restart the processe.

In this case, if the limit is reach, you wake the timer (use the Wake action associated with the timer), and finish the action.

When there are no more records to process, you finish the action.



Adding on top of Eduardo's reply, I would advise you against having a 20 min timer running in your servers. A limited amount of threads (3 per FE by default) is available to run timers, and having one that runs for so long may impact on other processes.

I would also advise to commit each N records crested/updated to avoid repeating work if an exception is raised. This will also prevent strange behaviors if you're using SQL Server as the platform runs there with READ UNCOMMITED isolation level.

Another tip is to define a boolean Site Property to allow you to stop the timer by configuration. Not related to your problem, but very useful in batch processes with cycles.

Thanks all for replying. Eduardo pointed us in the direction we needed. Thanks!