Hi All,

We are attempting to evolve our architecture into a globally distributed instance and are facing a lot of issues with performance and the general setup.  Wonder if any of you have had any experience with this that you can share:

  • We are using OutSystems deployed in on-premise setup on AWS.
  • We have deployment controller setup in London region.
  • We are trying to deploy a front-end server in Sydney using deployment zone.
  • Application database is in Sydney, OutSystems database is in London.
  • In doing so, we have encountered performance issues due to latency and found that its primarily due to front-end server/app calls to OutSystems Database (Users/Roles/Main generally?).

Thanks all for your help

Dan

Hi Daniel,

I've come across the exact same scenario in the past. 

It was decided that the Australian FrontEnds would be sitting next to the UK servers. The user latency was not that much and some tests were done without issues.

There is more network traffic between the servers themselves (across the globe) rather than users to frontend.


There is also another way of fixing this, but it really depends on your requirements. The other solution is to have an AUS environment, like a PROD2. This will only work if you're not using a SaaS approach. 

Feel free to hit me up in case you want to discuss a little more!

Kind regards!




Forgot to add, feel free to use this great tool to test your applications from all around the globe:

https://www.dotcom-tools.com/website-speed-test.aspx


Hi Rui, thank you for the suggestion. However, we have a data restriction which controls that we can not move application data outside of Sydney thus the Architecture/requirement is non-negotiable in reference to Outsystems core database and application database.

Have there been any other configuration option(s) that may have been tried/tested?

Thank you in advance.

Regards,

Kumar

Hi Kumar, 

Sorry for the delay. 

You may have 2 scenarios left:


1) Create a new environment on AUS with all data on AUS and all the data is independent.


2) Create a new environment on AUS but consume master data from UK server while using Services.

Sorry for my lack of powerpoint skills!


Thanks again, Rui, and apologies for being unable to get back to you earlier (got no notification, Grrr!!).

Your powerpoint skills are definitely better than mine ;)

Scenario 1 we are trying to avoid as it means additional license cost and investment in duplicate infrastructure.

Scenario 2 is what I tested recently and is causing the performance issues which I am unable to understand the possible cause of.

in the second scenario for some reasons, OutSystems app published via frontend in Sydney still try and connect with master database in the UK (and the DC) every time the application data is refreshed via browser (rather than using the cache) thus adding to the latency of data being accessible via the frontend.

I requested OS support to help me debug it and am still awaiting confirmation if could get some help, let's see.


If you have users scattered all over the world, maybe look into something like Cloudflare or CacheFly.

Accelerating these kinds of web applications is quite literally their thing

Indeed, thanks Carlos, but for some reason, the app consistently try and fetch data from OS database, even though the app database is next to the application - thus adding to latency.

I was hoping to be able to debug the issue with OS support because it seems that OS is built this way and it would be very difficult to find an alternative without there helping, however, without avail.

To be fair, to take full advantage of Cloudflare or CacheFly services, you'd probably need to do a lot of refactoring to your application, which might solving the issue by itself.

But you should also look into caching, which the platform supports.

In web screens:


And in Aggregates and SQL queries:







Also note that if you know an Aggregate or SQL will only return 1 record, you should specify it, as above, since that helps the platform generate more efficient code.

But do be careful with query caching, because this can cause subtle errors that may go unnoticed.

And also:

https://success.outsystems.com/Documentation/Best_Practices/Performance_Best_Practices/Improving_performance_with_distributed_caching


Just to complement Carlos, you may also use (carefully) caching on server actions. 

These will avoid the request going from the frontend to the database, which is also a good scenario since you don't want your requests to travel a lot around the globe.

thanks, Rui, is it distributed cache you are referring to (https://success.outsystems.com/Documentation/Best_Practices/Performance_Best_Practices/Improving_performance_with_distributed_caching) if so, yes it was tried and made no difference. The calls to the deployment controller were still recorded.

Thanks again everyone for your inputs/discussion. :)

Daniel Asserati wrote:

Hi All,

We are attempting to evolve our architecture into a globally distributed instance and are facing a lot of issues with performance and the general setup.  Wonder if any of you have had any experience with this that you can share:

  • We are using OutSystems deployed in on-premise setup on AWS.
  • We have deployment controller setup in London region.
  • We are trying to deploy a front-end server in Sydney using deployment zone.
  • Application database is in Sydney, OutSystems database is in London.
  • In doing so, we have encountered performance issues due to latency and found that its primarily due to front-end server/app calls to OutSystems Database (Users/Roles/Main generally?).

Thanks all for your help

Dan

Hi Daniel, 

I am also investigating such a set up. Your post made me curious and hence these questions !!

  1. Are you using MS SQL server as your database stack?  
  2. If you are using Database catalogs to separate application data from platform data, then Outsystems expects these 2 databases to be on the same SQL server instance. Was wondering how you managed to overcome this challenge?