Lots of database elements left behind

  
Some days ago I was looking in the Forge and i found a small app: DBCleaner. After running one of the scripts generated by this tool I found that there are a lot of database elements, mainly tables, that are left behind after I delete a ESpace or after i delete an entity.

My main question as developer is: why are those elements left behind? If a delete an espace i would expect every element related whit it would be eliminated.
Can I prevent this situation by using some sort of configuration? Do I need to periodically run that DBCleaner component to keep clean my database? 
I believe their intent is to err on the side of caution and preserve the data in case you need it.
Hello Johannes,

This is a very pertinent question.

There are two main reasons for the platform not deleting "physically" tables and/or columns from the database whenever their logical counterparts are removed from the application meta model.

The first point is that if, for some reason, a developer deletes by mistake an entity or an attribute and publishes their solution, they can avoid an "oh noes" moment because their data remains in the database and they can simply undo/revert the change they made inadvertedly, with no harm done.

The second point is that the platform has a feature that allows you to "go back" to any version of an application that you have published in the past (e.g. service center -> factory -> eSpaces -> eSpaceDetails -> versions tab). Since data in the database is preserved whenever an entity/attribute is deleted, when you wish to go to an old version of an application, you get a database state where most if not all of the data that was available at the time when that particular application version existed.

I would like to stress that data is not deleted from the database only in scenarios where structurally destructive modifications are carried out (e.g. deleting whole entities and/or entity attributes). If you delete a particular entity record, it is physically removed from the database.

As for your other questions, currently, there is no way for you to prevent this behaviour by the platform, it is there by default, and cannot be disabled.

If you really feel the need for clearing up space from your database, you can use the DBCleaner component.

If you have any further questions, please do not hesitate to place them.

With best regards,

Stoyan Garbatov
I've successfully used another alternative for accomplishing this.  I found it in another thread (which I cannot locate right now) but it works very well if you are absolutely sure that you do not need any previous versions of the database structure or eSpace code.

<< This is a destructive procedure with no recovery.  Be sure this is what you want before proceeding!!  >>

1) Open the eSpace with the entities you are cleaning up in Service Studio.
2) Click on Module | Export | Save As (this is 9.0, may be different on other versions) and export as an OML file (this is just a precaution)
3) Click on Module | Module Management (this is 9.0, may be different on oither versions) which will bring up Life Time for the module
4) Click Delete eSpace
5) In Service Studio make some insignificant change so that the 1-Click Publish button appears
6) Click the 1-Click Publish button
7) Once published, the eSpace will be located in the Independent Modules area.  Move it back to the original application using the Move To function.

Hope this helps,
Curt

Stoyan, Thanks a lot for your very detailed answer. It's really helpfull to understand this behaviour about database elements and its relationship with how Outsystems platform manages versions.


Stoyan Garbatov wrote:
Hello Johannes,

This is a very pertinent question.

There are two main reasons for the platform not deleting "physically" tables and/or columns from the database whenever their logical counterparts are removed from the application meta model.

The first point is that if, for some reason, a developer deletes by mistake an entity or an attribute and publishes their solution, they can avoid an "oh noes" moment because their data remains in the database and they can simply undo/revert the change they made inadvertedly, with no harm done.

The second point is that the platform has a feature that allows you to "go back" to any version of an application that you have published in the past (e.g. service center -> factory -> eSpaces -> eSpaceDetails -> versions tab). Since data in the database is preserved whenever an entity/attribute is deleted, when you wish to go to an old version of an application, you get a database state where most if not all of the data that was available at the time when that particular application version existed.

I would like to stress that data is not deleted from the database only in scenarios where structurally destructive modifications are carried out (e.g. deleting whole entities and/or entity attributes). If you delete a particular entity record, it is physically removed from the database.

As for your other questions, currently, there is no way for you to prevent this behaviour by the platform, it is there by default, and cannot be disabled.

If you really feel the need for clearing up space from your database, you can use the DBCleaner component.

If you have any further questions, please do not hesitate to place them.

With best regards,

Stoyan Garbatov
 
 
Hello everyone,

Just a little bit of extra info. As of version 9.0.0.23 (released on the 23rd of January 2015) there is a new system API called DBCleaner. The DbCleaner API provides you with functionality for freeing up database space. This is accomplished through actions that enable you to:
 
- Drop tables/columns corresponding to entities/attributes that are no longer in use by any of your applications;
- Delete module versions that are too old and no longer needed.

For more information, please consult the API documentation located here.


With best regards,

Stoyan Garbatov
Hi Stoyan. This means that DBCleaner functionality, that was already available as an independent application, is now part of the "official" plataform API's? Or, are they similar but different implementations?
Hello Johannes,

The internal implementations are different and the functionality offered is similar, but not a perfect match. Nonetheless, to answer your question, yes, the ability to effectively delete (from the database) tables/columns/versions of "elements" that are no longer "active" is now officially supported by the platform.

I imagine that at some point in the (near) future, there will be a version of the DBCleaner component that uses the new API. People can continue using it, if its functionality was enough for their purposes, or they can use the newly made available API to implement their own custom logic maintenance/cleaning actions.

If you have any further questions, please do not hesitate to put them forward.


With best regards,

Stoyan Garbatov
This is fantastic news, thanks!

J.Ja
Great News, thanks!
In fact Very good news !!!
This mean that in our personal enviroment can now delete the columns that no longer need and produce those blue warnings when we publish the applications?
Gonçalo. I didnt try yet! But for what I read in help. Yes!!!
Hello everyone,

The OutSystems Success Team should have more information about the upgrade of personals to 9.0.0.23. If you wish to get more info, try sending an email to success@outsystems.com.

With best regards,

Stoyan Garbatov
Is this API implemented in the cloud personal environment?
Hello David,

Yes, this API will be available in the cloud personal environments, when they get upgraded to 9.0.0.23.
That is great news. I have some columns that I thought I would need and now they just keep reporting that my database model is inconsistent when I publish.