9
 Followers
32
 Likes

One click clone production database to development environment

Backend
On our radar
I would like to see a "Clone Database" button in LifeTime, which copies the Production database backwards into the QA and Dev environments. 


Each time a problem occurs on our production environment that can't be recreated on the dev environment for lack of similar data I get more and more frustrated with this platform.

When developing on a dynamic web platform you often need real world, live data to test a behavior, even more so when troubleshooting an unusual behavior that has been reported by a user. This necessitates being able to quickly and easily clone the production database into the development environment.

Unfortunately, OutSystems has an incredibly complex key and token system which turns a simple database clone into such a Herculean task it requires a 20 page technote! https://www.outsystems.com/home/document-download/1484/8/0/0
This is completely unacceptable.



I would like to see a "Clone Database" button in LifeTime, which copies the Production database backwards into the QA and Dev environments. This would make diagnosing complex and nuanced issues that happen in the Production environment much easier and greatly improve the RAD timeframe and lifestyle.


Sincerely,
Braxton Bragg
Created on 29 Oct 2015
Comments (19)
Yeah, that would be nice. However, there's a few issues to take into account:
1) Law. In many countries, you are not allowed to have unmodified production data on your testing environments, if it contains customer data.
2) Unless you actually copy physical database files (which the platform couldn't or shouldn't do), and depending on the size of the database, transferring the data could take ages and effectively shut down the production and test servers and/or the network. We have a database that's 100GB. We are a small company. Imagine a large company, which could easily have 100TB. You really, really, really don't want that transferred over your netword record by record...
3) It is technically insanely complex if you want to restore the data record by record because of foreign key dependencies. There may be tricks to circumvent this, but it ain't easy...
4) *Everything* is stored in the database, including users and eSpaces. I assume your development environment has different users than production. I'm pretty sure your development environment has eSpaces with different content than production. This has all to be taken into account as well, adding to the complexity.

That said, I totally recognize the use case. I just don't think that it is feasible to ever implement something like this.
You could in theory restrict the restore job to individual applications/espaces like you do deploys via Lifetime.
This could check to make sure any applications that reference the database also are restored as well etc.

This changes your restore from 100TB to something reasonable since you're no longer restoring all old espaces, outsystems logs, etc.
Braxton,

You could, but it'd be a monumental task. CoolProfs has developed such a tool, which took them ages and it's still not perfect.
Merged this idea with 'One button push for production server to development server database migration' (created on 15 Mar 2016 16:11:57 by Jason Herrington)
Often issues in the production server are a result of the data that has been entered into the production server database.  This makes it somewhat hard to troubleshoot and fix these issues as we want to minimize production server downtime.  It would be very useful to have one button (think a lifetime in reverse) where only the production server database could be quickly copied over to the development server database for troubleshooting.

This comment was:
- originally posted on idea 'One button push for production server to development server database migration' (created on 15 Mar 2016 by Jason Herrington)
- merged to idea 'One click clone production database to development environment' on 16 Aug 2020 01:21:23 by Justin James
This is one of the big items keeping some companies off the OutSystems cloud. :(

J.Ja

This comment was:
- originally posted on idea 'One button push for production server to development server database migration' (created on 15 Mar 2016 by Jason Herrington)
- merged to idea 'One click clone production database to development environment' on 16 Aug 2020 01:21:23 by Justin James
Merged this idea with 'Data sync form prod to lower envs with masking' (created on 03 Jun 2020 07:54:57 by Tushar Panpaliya)

This comment was:
- originally posted on idea 'One button push for production server to development server database migration' (created on 15 Mar 2016 by Jason Herrington)
- merged to idea 'One click clone production database to development environment' on 16 Aug 2020 01:21:23 by Justin James

Out if the box feature to sync data form Prod to lower envs(dev/test) for testing/ debugging purpose. Maybe an additional option to mask sensitive data before syncing in.



This comment was:
- originally posted on idea 'Data sync form prod to lower envs with masking' (created on 03 Jun 2020 by Tushar Panpaliya)
- merged to idea 'One button push for production server to development server database migration' on 05 Jun 2020 03:32:42 by Justin James


This comment was:
- originally posted on idea 'One button push for production server to development server database migration' (created on 15 Mar 2016 by Jason Herrington)
- merged to idea 'One click clone production database to development environment' on 16 Aug 2020 01:21:23 by Justin James
Merged this idea with 'Production data porting' (created on 06 Aug 2020 09:28:36 by ronny)

This comment was:
- originally posted on idea 'One button push for production server to development server database migration' (created on 15 Mar 2016 by Jason Herrington)
- merged to idea 'One click clone production database to development environment' on 16 Aug 2020 01:21:23 by Justin James

It would be good the production data can be ported to the testing environment if there is any production data issues that need to be replicated in the testing environment.



This comment was:
- originally posted on idea 'Production data porting' (created on 06 Aug 2020 by ronny)
- merged to idea 'One button push for production server to development server database migration' on 10 Aug 2020 16:44:52 by Justin James


This comment was:
- originally posted on idea 'One button push for production server to development server database migration' (created on 15 Mar 2016 by Jason Herrington)
- merged to idea 'One click clone production database to development environment' on 16 Aug 2020 01:21:23 by Justin James

Beware with GDPR directives! It could be a bad idea.



This comment was:
- originally posted on idea 'Production data porting' (created on 06 Aug 2020 by ronny)
- merged to idea 'One button push for production server to development server database migration' on 10 Aug 2020 16:44:52 by Justin James


This comment was:
- originally posted on idea 'One button push for production server to development server database migration' (created on 15 Mar 2016 by Jason Herrington)
- merged to idea 'One click clone production database to development environment' on 16 Aug 2020 01:21:23 by Justin James
Merged this idea with 'Production Database -Data Replicate in Development environment' (created on 06 May 2019 14:03:51 by Nilesh Rokade)

In data driven application ,when production issue is not reproducible in development environment . It becomes important to run code on same dataset as that of production.


For non outsystem application , simple database restore and connection string change does the trick but this option is not there for outsystems based application.


Idea is to provide UI Interface to select table from production and copy their data to development environment . It should generate select script and populate it in development environment .






This comment was:
- originally posted on idea 'Production Database -Data Replicate in Development environment' (created on 06 May 2019 by Nilesh Rokade)
- merged to idea 'One click clone production database to development environment' on 16 Aug 2020 01:21:45 by Justin James

Changed the category to Database




This comment was:
- originally posted on idea 'Production Database -Data Replicate in Development environment' (created on 06 May 2019 by Nilesh Rokade)
- merged to idea 'One click clone production database to development environment' on 16 Aug 2020 01:21:45 by Justin James
Merged this idea with 'Data Copy from Production to other server (Development/QA Environment)' (created on 24 Mar 2020 07:40:38 by Nikko Gonzales)

It would be great if the platform does have a copying of data from production to other server/environment. Purpose is to replicate the exact issue/bug scenario from production environment.


This may be challenging since the primary key per user is not always match with the other environment. And also some organization have different server in production.



This comment was:
- originally posted on idea 'Data Copy from Production to other server (Development/QA Environment)' (created on 24 Mar 2020 by Nikko Gonzales)
- merged to idea 'One click clone production database to development environment' on 18 Sep 2020 13:23:18 by Justin James

Changed the category to Database




This comment was:
- originally posted on idea 'Data Copy from Production to other server (Development/QA Environment)' (created on 24 Mar 2020 by Nikko Gonzales)
- merged to idea 'One click clone production database to development environment' on 18 Sep 2020 13:23:18 by Justin James
Merged this idea with 'Copy data content from an entity through different environments, on demand' (created on 15 Sep 2020 10:11:51 by Larisa Tanasa)

This comment was:
- originally posted on idea 'Data Copy from Production to other server (Development/QA Environment)' (created on 24 Mar 2020 by Nikko Gonzales)
- merged to idea 'One click clone production database to development environment' on 18 Sep 2020 13:23:18 by Justin James

Hello everyone,


I would propose we have a functionality in lifetime where we can select which entities' data is useful for the environment we are pushing to.

Or maybe have a Boolean property per entity that would be marked in case we want it's data copied to the other environments.

I know Bootstraping is an easy way to populating entities, but I think this will be even easier, just 1 Click Publish away.


Kind regards,

Larisa



This comment was:
- originally posted on idea 'Copy data content from an entity through different environments, on demand' (created on 15 Sep 2020 by Larisa Tanasa)
- merged to idea 'Data Copy from Production to other server (Development/QA Environment)' on 18 Sep 2020 13:22:50 by Justin James


This comment was:
- originally posted on idea 'Data Copy from Production to other server (Development/QA Environment)' (created on 24 Mar 2020 by Nikko Gonzales)
- merged to idea 'One click clone production database to development environment' on 18 Sep 2020 13:23:18 by Justin James
views
1857
Followers
9