283
Views
12
Comments
Solved
Multi tenancy with already existing data from multiple tenants
Application Type
Reactive

Hi all,


I am currently working on an app which requires multiple tenancy.
However, I also have a core module which is not multi-tenant which loads (bootstraps) all the existing data from multiple Excel files. However this data is coming from a main database (external) but with data from all tenants.

I watched all the videos (masterclasses) and all the documentation (how to build..../scalable multi tenant....) and I know how to filter the aggregates when the TenantId is exposed in an entity, but I would like to keep the entities automatically filtering. (so without the checkbox enabled)


I have an idea of how to do this, but I wanted to see if there are other options?


My idea was to create "clone" entities in the Front-End app which are multi-tenant and to keep the entities in the Core single-tenant, then load the App entities which are multi tenant with the filtered data from 1 tenant from the core on login. Kind of like this: https://www.outsystems.com/forums/discussion/76161/bootstrap-data-into-multi-tenant-app/ , so every time a tenant logs in their data is filtered from the core then loaded in to a multi-tenant entity in the Front-End app. ..... I hope you get the idea.


Any feedback on how to tackle something like this is appreciated.


Kind regards,


Remi

2018-05-15 08-41-11
Davide Periquito
Solution

Well, you have a really nice challenge ahead. 


Let me try to make this simple and cover a lot of topics. Multi-tenancy is all about data isolation between tenants. So this is the first question that you need to answer when you create a table. "Is this table content specific for each tenant?"

Going into implementation and more specific:

  1. Data is not specific to a tenant (e.g. some configurations)
    No one should be able to change this data per tenant. you can go with what I said, having single tenant entities.
  2. Data is specific to a tenant and it might change per tenant.
    Your tables need to be multi-tenant, and Data need to be duplicated for the number of tenants that you have (I will describe how can you achieve this below)
  3. Data is specific to a tenant and it is not possible to change the bootstrapped data (hardest scenario, different approaches can be suggested)
    Option 1, Set the main table to single tenant where you store all the information that you need, (e.g. labels, description, product cost, whatever)
    Create an extension table multi-tenant that has an FK to the main table and for every query that you do you need to join those 2 tables. (this approach is very similar to the multi-lingual approach if you want to check deeper)
    Option 2, The table is multi-tenant and has the data duplicated for the number of tenants that you have,  add a field called bootstrapped data, and for that data is not possible to edit. (business logic)

The approach used depends on many things, size of bootstrapped data, performance, and so on. There is no swiss army knife here.


Finally for point 2, bootstrap data, probably you are using some timers, these also have a multi-tenant property. This means that the timer will run as many times as the tenants that you have so it will populate the tables filter by tenant Id. If you have 10 tenants the timer will run 10 times. 


Do you need to create tenants dynamically? (If so this get even longer so I will not describe right way)


Creating a local table brings some complexity but some advantages, I wouldn't suggest going that way due to all the complexity of building, managing and destroying the cache.


Let me know if you need more information. Sorry for the long post

UserImage.jpg
Remi Roo

Hi Davide,

Thank you so much for the info!

I think I'm now really really close to getting it done following your excellent advice.

From what I understand now, your point 2 says that I can bootstrap the data on a table as many times as there are tenants, using the multi-tenant feature of the timer. I then also need to filter the multi-tenant tables on TenantId.?

So this means that I populate the table already filtered on TenantId each time I bootstrap per Tenant, right? Like, say I have a table called "Projects" with FKs to "Customers" table. The bootstrap timers then run e.g. 10 times to bootstrap "Projects" because I have 10 tenants (which are also my customers). Then there is somthing with filters which I don't quite understand.


In any case, I think I can maybe work something out with this.

So on first publish, the data from the bootstrap will be loaded in multiple times and the data will be connected to a TenantId (if you do the filtering right on bootstrap). 

From then on I can just use the multi-tenant table like normal I guess, as OutSystems will be filtering it automatically on TenantId if a tenant logs in. This part is quite easy as this is what makes it so useful.


Your point 3 is not so relevant for me I think as data should always able to be changed during runtime.

Creating dynamic tenants is currently not so important for me, as our firm already created a custom solution for this and it is out of scope for my app.


Again Davide, your help is very much appreciated. I am now just 4 weeks in OS and I am happy there is such a great community to help.


Kind regards,


Remi



2018-05-15 08-41-11
Davide Periquito

In this case, you just need to have the timer as multi-tenant...

UserImage.jpg
Remi Roo

Yep, that's what I figured indeed.

Thanks again!

2018-05-15 08-41-11
Davide Periquito

Hey Remi,


You have multiple configurations when it comes to multi-tenant, I am not sure if I got the question.


But if you want to have a module filtering everything based on the active tenant you need to keep the check box active as image below


Although you have an additional multi-tenant configuration per entity, I think this is your case. The module needs to be multi-tenant, although a specific table would not be multi-tenant or have the identifier exposed (check the screenshot below)


Set the Is Multi-tenant to false

UserImage.jpg
Remi Roo

Hey Davide,


Thank you so much for your feedback!

I posted some more context now to better understand my use case.


But in general, I think you might have got it indeed. You are basically advising me to set the app to multi-tenant, but to leave the entities themselves single-tenant right?

Does this indeed create "user-specific entities" when for example 2 people are logged in at the same time? 

So if 2 user (different users from different tenants) are both fetching from the same WebApp entities on the screen (I mean the names in the expressions are the same), their results will be different from each other at runtime?


This would be great, however I would still struggle to know how to "load" these WebApp entities with Tenant-specific data if the TenantId is not yet defined in the database model.

Again the issue of: I have customers which need to be separated, but how to I link these with the tenants?


Kind regards,


Remi

2020-09-15 13-07-23
Kilian Hekhuis
 
MVP

Hi Remi,

You can use the System action "TenantSwitch" to switch between tenants. Thus you don't need to expose the Tenant Id.

2018-05-15 08-41-11
Davide Periquito

I don't know the exact use case.


but careful with the tenant switch, the session gets lost, description below:

"'TenantSwitch' Change the context to the given tenant identifier. When switching tenant, the session is cleared (equivalent to an implicit logout, before changing the tenant), the OnSessionStart actions are executed, and the current transaction is committed. "

2020-09-15 13-07-23
Kilian Hekhuis
 
MVP

That seems like the description for the client-side action. Server-side, when running a timer, there's no need for session variables and there's no OnSessionStart.

2018-05-15 08-41-11
Davide Periquito

Not exactly that was the server side. I believe there is no system tenant switch for client.


2020-09-15 13-07-23
Kilian Hekhuis
 
MVP

You are completely right, I have never used the OnSessionStart, and forgot it existed.

UserImage.jpg
Remi Roo

Thank you all for the replies, a bit more background on this use case might help to understand exactly what I'm looking for. But I've already read some great insights.

So to start with, I have 2 apps: a Core and a WebApp.

The Core app (or module) is only used to simply import a whole lot of data from Excel with various bootstrap actions.

 










This results in a whole lot of entities, all mixed data.

Now, I want a customer to log in and for them only to get their data fetched in each screen.

As I understand it, I can make users and map them to tenants and intern map those tenants to customers as well. So the Customers see it as "their app" while actually the app is used for multiple customers.


Now I understand relatively well how the multi-tenants work and what the "expose tenant Id" does, but what always lacked for me in those explanations, was how to make it so that this Customer-Tenant link is there.

Like, how does OS know what data to filter if say I have the "Projects" entity which I want to view? Yes it may filter automatically on the TenantId in the background, but if the data is already there, how to I say to OS that the CustomerId has to be mapped to a tenant? (especially using core-app relations)


What I wanted to try now:

  • Create the same entities as in the core also in the front-end WebApp
  • On login of the user, check their TenantId and filter the Core data down to the right CustomerId (using an extra entity which maps TenantIds to CustomerIds) in the core
  • Copy the filtered core data records to the "local" entities which are set to multi-tenant
  • Display the data on the screens from the "local" entities

@Kilian Hekhuis @Davide Periquito 

2018-05-15 08-41-11
Davide Periquito
Solution

Well, you have a really nice challenge ahead. 


Let me try to make this simple and cover a lot of topics. Multi-tenancy is all about data isolation between tenants. So this is the first question that you need to answer when you create a table. "Is this table content specific for each tenant?"

Going into implementation and more specific:

  1. Data is not specific to a tenant (e.g. some configurations)
    No one should be able to change this data per tenant. you can go with what I said, having single tenant entities.
  2. Data is specific to a tenant and it might change per tenant.
    Your tables need to be multi-tenant, and Data need to be duplicated for the number of tenants that you have (I will describe how can you achieve this below)
  3. Data is specific to a tenant and it is not possible to change the bootstrapped data (hardest scenario, different approaches can be suggested)
    Option 1, Set the main table to single tenant where you store all the information that you need, (e.g. labels, description, product cost, whatever)
    Create an extension table multi-tenant that has an FK to the main table and for every query that you do you need to join those 2 tables. (this approach is very similar to the multi-lingual approach if you want to check deeper)
    Option 2, The table is multi-tenant and has the data duplicated for the number of tenants that you have,  add a field called bootstrapped data, and for that data is not possible to edit. (business logic)

The approach used depends on many things, size of bootstrapped data, performance, and so on. There is no swiss army knife here.


Finally for point 2, bootstrap data, probably you are using some timers, these also have a multi-tenant property. This means that the timer will run as many times as the tenants that you have so it will populate the tables filter by tenant Id. If you have 10 tenants the timer will run 10 times. 


Do you need to create tenants dynamically? (If so this get even longer so I will not describe right way)


Creating a local table brings some complexity but some advantages, I wouldn't suggest going that way due to all the complexity of building, managing and destroying the cache.


Let me know if you need more information. Sorry for the long post

UserImage.jpg
Remi Roo

Hi Davide,

Thank you so much for the info!

I think I'm now really really close to getting it done following your excellent advice.

From what I understand now, your point 2 says that I can bootstrap the data on a table as many times as there are tenants, using the multi-tenant feature of the timer. I then also need to filter the multi-tenant tables on TenantId.?

So this means that I populate the table already filtered on TenantId each time I bootstrap per Tenant, right? Like, say I have a table called "Projects" with FKs to "Customers" table. The bootstrap timers then run e.g. 10 times to bootstrap "Projects" because I have 10 tenants (which are also my customers). Then there is somthing with filters which I don't quite understand.


In any case, I think I can maybe work something out with this.

So on first publish, the data from the bootstrap will be loaded in multiple times and the data will be connected to a TenantId (if you do the filtering right on bootstrap). 

From then on I can just use the multi-tenant table like normal I guess, as OutSystems will be filtering it automatically on TenantId if a tenant logs in. This part is quite easy as this is what makes it so useful.


Your point 3 is not so relevant for me I think as data should always able to be changed during runtime.

Creating dynamic tenants is currently not so important for me, as our firm already created a custom solution for this and it is out of scope for my app.


Again Davide, your help is very much appreciated. I am now just 4 weeks in OS and I am happy there is such a great community to help.


Kind regards,


Remi



2018-05-15 08-41-11
Davide Periquito

In this case, you just need to have the timer as multi-tenant...

UserImage.jpg
Remi Roo

Yep, that's what I figured indeed.

Thanks again!

Community GuidelinesBe kind and respectful, give credit to the original source of content, and search for duplicates before posting.