Hi,

We have been noticing increase in time on each activity by 10 secs - 2 mins which should be completing in 1 second as number of process increases. Noticed that actions inside activity isn't taking much time but takes time move to next activity. I am not seeing anything unusual in error / general log, no slow sql were reported. Checked all application server, database server and see no issues on memory / cpu utilization. What could slow down the process ? Should we expect slowness when concurrent process are running ?

Application is hosted on on-premise environment with 2 front ends.

Outsystems version 10 Java platform.

Solution

Hi Prabhu,


I'd say it is or can be the result of two things:


1. You may be running too much processes in parallel and each front end has a defined limit, usually 10 if I'm not mistaken.

2. The amount of data - whenever you create a process, there is a lot of data generated in Process and Activity entities, just to name a few. These tables support the process and activities actions and therefore become slower as the number of records in those tables increase. If you don't need this history (for auditing purposes for instance) deleting those might help you. My advise is to check components like DBCleaner or DBCleaner on Steroids, which should have cleaning mechanisms for that.


On another note, you mentioned you're running JAVA stack, you might want to consider migrating to .NET as JAVA stack is discontinued and not updated anymore (see here).


Also, you may want to check LBPT which is a way faster BPT with one automatic activity if it suits your use case.


Hope it helps.


Cheers,

João

Solution

João Marques wrote:

Hi Prabhu,


I'd say it is or can be the result of two things:


1. You may be running too much processes in parallel and each front end has a defined limit, usually 10 if I'm not mistaken.

2. The amount of data - whenever you create a process, there is a lot of data generated in Process and Activity entities, just to name a few. These tables support the process and activities actions and therefore become slower as the number of records in those tables increase. If you don't need this history (for auditing purposes for instance) deleting those might help you. My advise is to check components like DBCleaner or DBCleaner on Steroids, which should have cleaning mechanisms for that.


On another note, you mentioned you're running JAVA stack, you might want to consider migrating to .NET as JAVA stack is discontinued and not updated anymore (see here).


Also, you may want to check LBPT which is a way faster BPT with one automatic activity if it suits your use case.


Hope it helps.


Cheers,

João

 

 

Hi João,

Thanks for your quick reply. 

We did a major release and that's when we started to notice the problems. I understand its combination of #1 and #2 but was not something noticed prior to our release. I see every activity in process is slowing down with increase in numbers (they match with prior to release) no change was done on most of activities on release.

There isnt any slowsql reported on general logs. Logging shows actions within the activity runs pretty fast but it takes time to close and move to next activity. Is there anything we could compare prior to our new release that could tell why these activities are slowing down

Prabhu Gnanasekar wrote:

João Marques wrote:

Hi Prabhu,


I'd say it is or can be the result of two things:


1. You may be running too much processes in parallel and each front end has a defined limit, usually 10 if I'm not mistaken.

2. The amount of data - whenever you create a process, there is a lot of data generated in Process and Activity entities, just to name a few. These tables support the process and activities actions and therefore become slower as the number of records in those tables increase. If you don't need this history (for auditing purposes for instance) deleting those might help you. My advise is to check components like DBCleaner or DBCleaner on Steroids, which should have cleaning mechanisms for that.


On another note, you mentioned you're running JAVA stack, you might want to consider migrating to .NET as JAVA stack is discontinued and not updated anymore (see here).


Also, you may want to check LBPT which is a way faster BPT with one automatic activity if it suits your use case.


Hope it helps.


Cheers,

João

 

 

Hi João,

Thanks for your quick reply. 

We did a major release and that's when we started to notice the problems. I understand its combination of #1 and #2 but was not something noticed prior to our release. I see every activity in process is slowing down with increase in numbers (they match with prior to release) no change was done on most of activities on release.

There isnt any slowsql reported on general logs. Logging shows actions within the activity runs pretty fast but it takes time to close and move to next activity. Is there anything we could compare prior to our new release that could tell why these activities are slowing down

 

 We have multiple process running in parallel from main process and one of the process which is initiated multiple times from main process had long running query and resources were not released. This piled up the queue count which was noticeable in Environment heath. Disabled the new feature added in application which caused long running query and things got back to normal. I could see slow sql reported on this process but overlooked at it as we were figuring out why main process was slowing down.