Native support for database views
6556
Views
75
Comments
On our RadarOn our Radar
Backend
Quite often, I am repeating the same set of joins in queries (both normal and advanced), that I would ordinarily put in a view. Think e.g. of a CRM system that regularly needs "all active clients" or an ordering system that needs "all open orders". Of course if you need just that, it's easy to put it in an action, but I often find myself needing the result and join that with some other tables.

Unfortunately, for some reason OS does not allow to define database views, which imho is a great gap. Therefore, I'd really like to see support for database views or something similar (e.g. queries you can define on eSpace level and join the result of with another query, but that amounts to the same).
While you can't define views in ServiceStudio, you can define them in the database directly and import them with Integration Studio.
Thanks, I was aware of that, but it has the obvious downsides of having to maintain things outside the platform, which I'd rather not do.
May this idea be merged with this one: Create and using Views?
It'd be good to merge the two, although I think mine has a clearer title and description (all imho of course).
This is simliar to what I suggested in https://www.outsystems.com/ideas/465/.

Maybe votes can be combined.
Yeah, can we request the merge of ideas somewhere? 41 votes is better dan 21.
I see they're still not merged, can we request that somewhere?
One issue that we have had with using views imported through integration studio is that it will not allow us to use an advanced query that contains both a view and an outsystems table (standard join etc). 

However, if we query that view directly through a database connection, we are able to query both the view and the outsystems table at the same time. 

Having this limitation has pushed us away from using integration studio in many cases. I would love to see this fixed.  
Sometimes you start to have performance issues in database operations (locks, queries time out, too many records, etc.) in you applications and you have to re-think the database model.

One approach could be having flags indicating that a record is deleted (or active/inactive) and hiding it from your users, and remove this records afterwards with a timer then the application has less use.

Having to change all your queries in order to use the flag that you've created can be a headache. You can miss one or two, or having to change complex advanced queries, etc.

It would be nice if one could create a view that would filter records and return only those that did not have the flag marked as deleted and then simply use the Find & Replace Usages functionality to change all queries in your application that once referred directly to the table and now use the view that you've created.

This would help a lot when you have to use this approach and greatly reduce the time developing and testing your application afterwards.

Merged from 'Create and using Views' (idea created on 2010-05-11 14:35:43 by Fernando Sousa), on 2014-03-17 23:00:48 by Gonçalo Borrêga
This one should be merged with this: https://www.outsystems.com/ideas/Idea_View.aspx?IdeaId=1019 and probably this: https://www.outsystems.com/ideas/465/vritual-entities/

Merged from 'Create and using Views' (idea created on 2010-05-11 14:35:43 by Fernando Sousa), on 2014-03-17 23:00:50 by Gonçalo Borrêga
Be able to create an entity that gets its data via a simple query or advanced query.  Then have the ability to use that entity in other queries.  This way you can simplify your queries and also improve application performance.  (For the SQL Server side it would create a view in the database)

Merged from 'Vritual Entities' (idea created on 2010-08-27 15:19:57 by Rebecca Hall), on 2014-03-17 23:01:08 by Gonçalo Borrêga
I believe that is a drill down on the idea of centralized queries I already posted at:

https://www.outsystems.com/ideas/446/


Merged from 'Vritual Entities' (idea created on 2010-08-27 15:19:57 by Rebecca Hall), on 2014-03-17 23:01:08 by Gonçalo Borrêga
I was going to post a similar idea, but then i found this one, that is a fine approach to what i was thinking. 

I would also suggest that we could use directly the query inside another query (as an entity of course). This could be a way of doing more complex queries just like the usual "nested" queries we can make in SQL.

Another option would be the ability to put the result of a query in a structure and use the structure in another query, but just like a normal entity (allowing joins with other entities and so on). As i can tell the normal use of structures now is like a more advanced variable to be filled at runtime...


Merged from 'Vritual Entities' (idea created on 2010-08-27 15:19:57 by Rebecca Hall), on 2014-03-17 23:01:08 by Gonçalo Borrêga
What's described above is basically Views, so I think it should be merged with this: https://www.outsystems.com/ideas/Idea_View.aspx?IdeaId=1019 and this: https://www.outsystems.com/ideas/91/create-and-using-views/

Merged from 'Vritual Entities' (idea created on 2010-08-27 15:19:57 by Rebecca Hall), on 2014-03-17 23:01:08 by Gonçalo Borrêga
In the entities section of Service Studio we could create\manage views.
They could be defined like an Advanced query, and with the 1 click publish the platform would generate the view in the platform database. 

PROS:
1) Don't need to create views directly in the DB Server to import them later with Integration Studio;
2) Changes to views would be easier and faster;
3) The views would be validated with the TrueChange technology, and any entity change would be identified;
4) Less errors;


Merged from 'Create\Manage views from Service Studio' (idea created on 2014-12-05 11:43:08 by Carlos Henriques), on 2017-11-24 18:33:53 by Gonçalo Borrêga
2014-01-21 08-35-57
Andrew Burgess
This would also help to remove "repeated" code in your applications. For example, if you have the concept of "customers with active subscriptions", you might need to use that to populate data for a list screen, but also as the basis for further queries in the business logic, such "customers with active subscriptions that are in arrears." Currently, the filters required for "customers with active subscriptions" would need defining twice. That's wasted effort, but is also a pain if the definition changes - it has to be fixed in many places.

Merged from 'Create\Manage views from Service Studio' (idea created on 2014-12-05 11:43:08 by Carlos Henriques), on 2017-11-24 18:33:53 by Gonçalo Borrêga

It would be great if we could not only create Entities but Views also.

At the Data Tab -> under database make a segregation of Entities (Tables) and Views.

Database

 1. Entities

  2.Views

Provide the same option to make the views public  (YES/NO).

Make it possible to use the views in both query options ( Aggregate / Advanced Query)


Benefit:

This can alleviate a lot of creation of complex queries with a lot of joins in the Aggregate and  Advanced queries. Queries)




Merged from 'Create Database Views through OutSystems' (idea created on 2016-12-05 14:19:57 by Alexei), on 2017-11-24 18:31:32 by Gonçalo Borrêga

Merged from 'Create\Manage views from Service Studio' (idea created on 2014-12-05 11:43:08 by Carlos Henriques), on 2017-11-24 18:33:53 by Gonçalo Borrêga

Agreed. The View4Entities component is good, but fills a bit of a different role than this, and you'd still need to add the views back in through an Extension... AND you cant do joins, filtering, etc...

J.Ja



Merged from 'Create Database Views through OutSystems' (idea created on 2016-12-05 14:19:57 by Alexei), on 2017-11-24 18:31:32 by Gonçalo Borrêga

Merged from 'Create\Manage views from Service Studio' (idea created on 2014-12-05 11:43:08 by Carlos Henriques), on 2017-11-24 18:33:53 by Gonçalo Borrêga

Ability to create (filtered) views and reuse them as a kind of entity. 

On the view you could define CRUD using actions (or restrict when absent)

Usage could improve security and maintainability. For instance; the using party only sees the views and not the entities, which could be different.

This would also improve the usage in connecting to legacy systems. You do not have to replicate or expose the original entity, but you could encapsulate it within a view. 

Single source of Truth, but adapted to your use



Merged from 'Create/Manage (filtered) Views on entities (or saved Aggregates)' (idea created on 2017-01-20 10:37:38 by Patrick Baanvinger), on 2017-11-24 18:34:30 by Gonçalo Borrêga

This has been suggested several times, most recently here -
https://www.outsystems.com/ideas/2771/create-database-views-through-outsystems



Merged from 'Create/Manage (filtered) Views on entities (or saved Aggregates)' (idea created on 2017-01-20 10:37:38 by Patrick Baanvinger), on 2017-11-24 18:34:30 by Gonçalo Borrêga
2016-11-21 23-23-05
Gonçalo Borrêga
Merged this idea with 'Create and using Views' (created on 2010-05-11 14:35:43 by Fernando Sousa)

Why is this still not there?

Could we at least know whether this is on the roadmap or not? 


Find it indeed a bit disappointing that it still doesn't exist..

Hi Frank,

It's in the radar as you can see I'm the status of the first post.

Hi, are there any updates on this request,?

HI Christoff,

Thanks for your question. We aware of how this feature will help development speed but specially allow to have a centralized place where the query is defined and then reused in several other modules.

Although you would loose some of the optimizations that aggregates bring OOTB, but you can create an public server action that returns a query result, with this approach you would be able to have a reusable query.

Nevertheless we are looking into ways and possible impact (security and governance) of bringing this feature into the product.


Thanks,

We have been building our own backend views for sometime to be able to implement a few custom views.  They are used to provide an easy-to-use module for security as well as to build out reports that we can't build using aggregates or advanced queries easily.  Using an extension to expose the view though makes it so you can't really identify usage of the view or easily apply updates to the applications.  For optimization, I would consider that these views are in the same category as an Advanced Query where it is up to the company to provide them.   Security could be provided by module entities or by consumed references (something intended to be shared)

2021-04-27 10-08-39
Daniel Marques
Merged this idea with 'Allow views similar to database views' (created on 17 Sep 2018 17:52:53 by Munjal Subodh)
2018-09-17 17-54-14
Munjal Subodh

Databases have views that make it easy to run frequently run queries. It would be good if there was an equivalent in OutSystems.



This comment was:
- originally posted on idea 'Allow views similar to database views' (created on 17 Sep 2018 by Munjal Subodh)
- merged to idea 'Native support for database views' on 23 Jan 2020 16:38:39 by Daniel Marques
Changed the category to
Aggregates & Queries


This comment was:
- originally posted on idea 'Allow views similar to database views' (created on 17 Sep 2018 by Munjal Subodh)
- merged to idea 'Native support for database views' on 23 Jan 2020 16:38:39 by Daniel Marques
2021-04-27 10-08-39
Daniel Marques
Merged this idea with 'Create DB View to easily pass data to consumer' (created on 20 Sep 2019 11:17:45 by Vincent Koning)

We have several modules that contains entities that are used by several consumers. One of these is, for example, our Project database. In this database all our projects are stored with all relevant information. This database is used by a lot of other applications (consumers) in one way or another.

The problem I frequently run into is that for each of the applications we want to consume the Project data but with a different filter. Some applications only want to see projects that are in the Lead phase. Some applications only want to see projects that are currently in progress and some only want to see projects that are closed.

Right now I need to create the needed filter on each and every Aggregate we use in one of these applications. This is error prone and when you need to make a change because we need to add or remove a certain requirement we need to find all used aggregates and again, this is error prone. 

I would like therefor to suggest that I can create "Views" that define the filters and sorting relevant for the consuming application. This consuming application then uses only this view and not the Entities directly anymore. A change in the requirements of this View is then only done in one location saving a lot of work and possible errors. 

A view does not change the returning data type. It's still of type "Project". I can still update these entities through the normal process (for us that's via Server Actions since Entities are published read-only). So this will not change any underlying database or coding mechanics. 



This comment was:
- originally posted on idea 'Create DB View to easily pass data to consumer' (created on 20 Sep 2019 by Vincent Koning)
- merged to idea 'Native support for database views' on 23 Jan 2020 16:43:29 by Daniel Marques
Merged this idea with 'Add Entity of type view to Data tab' (created on 23 Jun 2020 06:39:01 by Auke Quist)

It would be helpful to define view(s) on an entity so you can define subsets of your data during design. As an example let's presume we have an entity Project. Projects are open (tendered), lost or won. When writing applications that make use of only Won projects you don't want to filter in every GetProject aggregate on that particular status. It would be easier to derive/define an Entity View WonPrjects based on the Entity Project in the data tab. This pre-filtered entity can then be used througout modules/applications. Advantage: definition of view/filter on 1 place instead of multiple aggregates.



This comment was:
- originally posted on idea 'Add Entity of type view to Data tab' (created on 23 Jun 2020 by Auke Quist)
- merged to idea 'Native support for database views' on 27 Jun 2020 02:02:55 by Justin James

Hi!

Views is something that has been requested since 2012, 8 years ago.

A quick search in Ideas on "views" and you would find this old idea:

https://www.outsystems.com/ideas/1019/native-support-for-database-views

--Tiago Bernardo



This comment was:
- originally posted on idea 'Add Entity of type view to Data tab' (created on 23 Jun 2020 by Auke Quist)
- merged to idea 'Native support for database views' on 27 Jun 2020 02:02:55 by Justin James

Hi, okay perfect. I voted for that one, not sure how i can close my idea then.

Auke



This comment was:
- originally posted on idea 'Add Entity of type view to Data tab' (created on 23 Jun 2020 by Auke Quist)
- merged to idea 'Native support for database views' on 27 Jun 2020 02:02:55 by Justin James

You cannot close your Idea. It has to be done by some Community Moderator (OutSystems Staff or MVPs).


To a Community Moderator or MVP:

We think this Idea (Id 9223, "Add Entity of type view to Data tab") should be merged with this one:

https://www.outsystems.com/ideas/1019/native-support-for-database-views


--Tiago Bernardo



This comment was:
- originally posted on idea 'Add Entity of type view to Data tab' (created on 23 Jun 2020 by Auke Quist)
- merged to idea 'Native support for database views' on 27 Jun 2020 02:02:55 by Justin James
Merged this idea with 'Allow creation of views' (created on 22 May 2019 13:14:19 by Joey Moree)

In our applications we have a few screens which require alot of joins to gather multiple fields of data from several entities.

It would be very nice to be able to create views in Outsystems (since we can't access the database ourselves the platform is denying us this functionality, which could save us alot of performance).

Now I can do this manually, by creating triggers for update/create events for the tables I wish to view on and create a seperate entity for this, but it will cause alot of extra effort just to get this extra performance gain.

I'd love to see Outsystems implement this (or give control to the database to be able to setup views myself).




This comment was:
- originally posted on idea 'Allow creation of views' (created on 22 May 2019 by Joey Moree)
- merged to idea 'Native support for database views' on 30 Jun 2020 15:09:07 by Justin James
2018-10-29 13-00-29
Magda Pereira
Changed the category to
Aggregates & Queries


This comment was:
- originally posted on idea 'Allow creation of views' (created on 22 May 2019 by Joey Moree)
- merged to idea 'Native support for database views' on 30 Jun 2020 15:09:07 by Justin James
Merged this idea with 'Defining aggregate views' (created on 27 Jun 2020 18:26:15 by Fred Stoopendaal)

This comment was:
- originally posted on idea 'Allow creation of views' (created on 22 May 2019 by Joey Moree)
- merged to idea 'Native support for database views' on 30 Jun 2020 15:09:07 by Justin James

In applications i like to define complex database queries once. So i like to create complex aggregates or advanced sql’s and use them as source in other aggregates. This way you can define database views in outsystems.



This comment was:
- originally posted on idea 'Defining aggregate views' (created on 27 Jun 2020 by Fred Stoopendaal)
- merged to idea 'Allow creation of views' on 30 Jun 2020 15:07:39 by Justin James


This comment was:
- originally posted on idea 'Allow creation of views' (created on 22 May 2019 by Joey Moree)
- merged to idea 'Native support for database views' on 30 Jun 2020 15:09:07 by Justin James
Merged this idea with 'Idea - Aggregate reuse for common queries by providing centralised template aggregates..' (created on 16 Jul 2020 12:09:10 by Richard Tilbury)

We have a number of recurring queries rendered as aggregates e.g. getting customer details, getting client information, etc.  We are having to code these as actions but this loses the benefits of the aggregate optimisation.  We'd like to centralise these to re-use logic as much as possible. 

Suggestion: 

Create a new section in Data for common aggregates - these are defined in the host module as you would an aggregate elsewhere but the aggregate can be consumed anywhere.  These aggregates can be made public and thus exposed for reuse in other modules.

The definition is made once only.  The object is consumed anywhere and executes the query defined in the aggregate, but has all the filters and joins defined in one centralised location - this allows us to capture business logic in one place and ensure the data is retrieved consistently.

Aggregates within actions are not optimized, but this way they could be as they are derived from essentially a central template object.

Timeouts and Caching would be defined on the base object and inherited automatically.




This comment was:
- originally posted on idea 'Idea - Aggregate reuse for common queries by providing centralised template aggregates..' (created on 16 Jul 2020 by Richard Tilbury)
- merged to idea 'Native support for database views' on 19 Jul 2020 16:54:05 by Justin James
Merged this idea with 'Aggregate state snapshot' (created on 05 Dec 2020 22:04:56 by Tiago Ribeiro)

As a developer,

I would like to have an area in the IDE to save "aggregate snapshots"

So that I can easily reuse aggregates


Aggregates are an amazing way to access data in an optimized way, as the platform will check what fields need to be fetched.

Imagine a scenario where I need the same aggregate (with complex filters, sorting, joins) in different screens and actions. I typically have two options:

- Replicate de aggregate everywhere (and lose on maintenance)

- Encapsulate the aggregate in an action (and lose on platform optimization - with a more generic output the platform won't be able to fetch only the needed fields)


I propose a new area / folder in the Data tab where the aggregates snapshot (with filters, sort, joins) could be saved.

When changing the snapshot in this area, all actions and screen that use this snapshot would be updated with the new code.




This comment was:
- originally posted on idea 'Aggregate state snapshot' (created on 05 Dec 2020 by Tiago Ribeiro)
- merged to idea 'Native support for database views' on 07 Dec 2020 18:14:30 by Justin James
Merged this idea with 'Implement better interfaces by means of database views' (created on 02 Mar 2021 16:15:01 by Matthieu de Graaf)

Hi,

As OutSystems applications become bigger it would be nice if an additional way of interfacing between applications/subsystems would be available. 

Currently we have 2 options

1) Hard references by means of referencing the entities In this case changes in the datamodel very often result in big deployments (all modules, using this entity have to be deployed)

2) Service actions. Technically being restful webservices, this option comes with an overhead. 

I would be nice if you could have an entity like objects base on an aggregate or advance sql. Technically you would implement this as a database view. In this way you can expose a limited, simpler datamodel to other applications/subsystems without losing the performance benefits of the database.

Because you have a clear/simpler interface to other application/subsystems deployments will be much simpler.

It would be nice if you could use these views in an aggregate/advanced sql. This especially helpful if you want to combine data of different applications/subsystems. The is a clear benefit with respect to service functions. In this case you have to combine your data by means of looping over the data in de mid-tier.

Best regards,

Matthieu de Graaf




This comment was:
- originally posted on idea 'Implement better interfaces by means of database views' (created on 02 Mar 2021 by Matthieu de Graaf)
- merged to idea 'Native support for database views' on 03 Mar 2021 08:12:55 by Daniël Kuhlmann

Good idea. Would also add that having a database view that you can use like an entity in service studio would help in bigger applications when you want to reuse the same SQL statement across different parts of your application.

For example: you have a view that gives you all the students that are currently enrolled in a given year. If in order to get this data you need joins and filters on a number of tables, by having a view you could share it across different parts of the application. Then, whenever this logic needs update, you just need to update the view.



This comment was:
- originally posted on idea 'Implement better interfaces by means of database views' (created on 02 Mar 2021 by Matthieu de Graaf)
- merged to idea 'Native support for database views' on 03 Mar 2021 08:12:55 by Daniël Kuhlmann

Hi Thiago,


I totally agree with you on the additional advantage. It would certainly be nice to reuse complex aggregates/advanced sql.


Matthieu



This comment was:
- originally posted on idea 'Implement better interfaces by means of database views' (created on 02 Mar 2021 by Matthieu de Graaf)
- merged to idea 'Native support for database views' on 03 Mar 2021 08:12:55 by Daniël Kuhlmann

Good idea, but it already exits since 2012, so I will merge it.



This comment was:
- originally posted on idea 'Implement better interfaces by means of database views' (created on 02 Mar 2021 by Matthieu de Graaf)
- merged to idea 'Native support for database views' on 03 Mar 2021 08:12:55 by Daniël Kuhlmann
Merged this idea with 'Create Views in Service Studio' (created on 18 Mar 2021 17:33:29 by Jorge Fonte)

The idea is to create Views in design time, much like we create entities and allow expose those views to be consumed and create queries with Aggregates or Advance SQL queries.



This comment was:
- originally posted on idea 'Create Views in Service Studio' (created on 18 Mar 2021 by Jorge Fonte)
- merged to idea 'Native support for database views' on 18 Mar 2021 20:59:11 by Daniël Kuhlmann

Nice idea, but its around already 9 years, so I will merge yours into it.



This comment was:
- originally posted on idea 'Create Views in Service Studio' (created on 18 Mar 2021 by Jorge Fonte)
- merged to idea 'Native support for database views' on 18 Mar 2021 20:59:11 by Daniël Kuhlmann
Merged this idea with 'View to be used in aggreates and queries' (created on 25 Mar 2022 07:27:46 by Marlies Quaadgras)

It would be helpfull if it is possible to create a view in outsystems, like the Oracle view.

When you have some entities, that are often used together to retrieve data, it should be great if they are combined in a view. 

Of course you can make a serveraction, to retrieve the result in a structure, but that can not be used in an aggregate or query.

When you have one view to combine in your aggregate, you are sure that you always retrieve the correct information.

This is the solution for creating reusable data objects.

It's a good Idea. But for that be useful it must be guaranteed that if the data changes the view is updated, like in Oracle.

This is a great idea! Creating public aggregates that act like entities would make our lives to much easier!

Great idea. In fact a view is a kind of a reusable aggregrate and it would be very helpfull avoiding duplicate aggregates.

The Query model in the Outsystems Domain Driven Design can be implemented by using views.
See the link Domain Driven Design with OutSystems - OutSystems on what the query model is.
That is what I meant with my comment: 'This is the solution for creating reusable data objects.' 


great idea
Merged this idea with 'SQL views from aggregates ' (created on 28 Mar 2023 13:14:30 by Damian Fonville)

One helpful feature would be the capability to generate aggregates as SQL views, which could be reused in other aggregates and have predefined filters and joins applied, making it more flexible and developer-friendly. Moreover, using a view would minimize the number of database reads needed to filter data, as it would require only a single read instead of multiple actions.

Sounds like an interesting idea. I don't know too much about SQL views but I know they allow much more performant data fetches. Interested to see where this idea goes. I also am curious to know if OutSystems already does use views but just not visible to us as end users.

Please merge this idea with https://www.outsystems.com/ideas/1019/native-support-for-database-views/

I had the idea to better control data access rights via Public Data Views. For example, you could leave the entity private to prevent direct access by other modules. On the other hand, the public views could then allow direct but controlled access via aggregates and advanced queries from these modules:

  • Restriction of visible fields by projection (e.g., hiding of security-relevant information such as hashes that are only evaluated within the core service)
  • Accessing only relevant records by selection (e.g. hiding logically deleted entries)
  • Enforcing selection restrictions through joins (e.g. enforcing row-based user rights using ACLs)

Of course, all these scenarios can also be implemented using private entities and public server actions, but this means that you lose the possibility of joining other data outside the module with the entities protected in this way. This leads to additional implementation effort and performance load on the application servers.

Merged this idea with 'enable to create Entity as a view' (created on 16 Oct 2023 11:02:27 by Marwen Reguigui)

My Idea its to create some entity that would contain selected columns from the other entities. 

it can be quicker to access. Once you open the application, you can quickly access the information you seek 

Hi Kilian 

Just to add to your idea, Outsystems should also support materialized views. This kind of view is a database object (is pre-computed) so is very fast to consult.

The use case for this kind of view is to create a denormalized model that support searches (help to increase search performance).

Regards.

Merged this idea with 'Database views Maintenance' (created on 12 Nov 2023 06:24:49 by Jerome Rajadurai J)

A suggestion to create and edit database views from Service Studio This helps in selecting a subset of reusable data. This can also be helpful in abstraction.

Any idea if this is close to being implemented? 

I wouldn't hold my breath 🥲