BPT bootstrap of existing business data

BPT bootstrap of existing business data

  
Hello folks,
 
I’m working close with a customer to implement a BPT module for their ERP system.
 
We have so far concluded 50% of the process design and I’m currently analyzing how to bootstrap BPT with the existing data (currently 12932 records).
 
I see 2 approaches, or a mix of both: 
  1. Add Conditional Starts to my process design
    • This will work as an automatic state transition mechanism to bootstrap the existing data;
  2. Create a timer to initialize the processes through an API
    • Calling LaunchProcess API action for each business record (12932 in total);
    • Calling LaunchActivity API action to jump start each business record to the proper point in time;
    • This can actually be a problem if the process gets launched and immediately gets hooked to an initial activity…
 
Some remarks about the designed process:
  • it is based on an existing application workflow, so BPT process is actually a “slave” of the existing state machine;
  • it is divided into several sub processes, each one a potential starting point of the bootstrapped records;
  • currently, it only uses Human Activity and Decision kinds (and this may simplify the design for bootstrap purposes);
 
Do you recommend any of the above approaches (or a mix)?
Do you know any other way to do this?
What lessons have we learned in similar situations before?
 
Kind regards,
Hi Pedro,

Can you tell us how did you solve this?

Many thanks,

Rúben Gonçalves
Hi Rúben,

Thanks, I'm really glad you asked.

I ended up with a mix of the 2 approaches above. By including the conditional starts as a kind of scaffolding pattern, the process initialization allows all launched processes to fast-forward to their respective starting activity. The only issue is that once the processes get started through this workflow, removing the bootstrap conditionals will impact it, meaning I'll only be able to remove these once all the processes with the bootstrap. So far it only affected BAM visualization, so no problem.





An important step along with the implementation was to create a basic administration screen to monitor the process initialization and export data for validation (before and after running the batch). Timer logic was pretty basic but good trace auditing messages were essential for troubleshooting, health monitoring and timings. Finally, the most important of all: a full-day simulation was performed to get a good feel of this process, as well as to exercise and fine-tune a detailed plan prepared for this go-live.

During development it was also important to have a handy SQL script to reset all the BPT instance data-model. Although this is highly dangerous, messing up with the platform's system entities, it would be a headache to ask a database restore for each test and since it was only used in development environment everything was good. We should be having a new API to purge a process instance from the datamodel, in the midterm - thanks to Support/Product Management!

That was it. Let me know if you have any other questions or details about it. I'll be happy to share!

Special thanks to Gonçalo Borrêga, Paulo Cunha and Paulo Garrudo for their suggestions and support.
Just a heads-up: contrary to my previous post where I raised an issue about removing the bootstrap conditionals which would impact the existing running processes (by suspending them), this is not a problem at all. It can be easily resolved by automating the reactivation of processes through the platform's BPT API.

For this, you will require an extension call to be made to internal method OutSystems.HubEdition.RuntimePlatform.DBRuntimePlatform.RemoveSuspendedProcessUpgradeData(). Further details can be found on the attached extension.

It is also a good approach that system entity Process_Upgrade guides the implementation of the reactivation of processes, so that you know which situations can automatically perform this. Be sure to inspect this entity in your administration/backoffice screen to reactivate suspended BPT processes.

Cheers!


Pedro Gonçalves wrote:
Hi Rúben,

Thanks, I'm really glad you asked.

I ended up with a mix of the 2 approaches above. By including the conditional starts as a kind of scaffolding pattern, the process initialization allows all launched processes to fast-forward to their respective starting activity. The only issue is that once the processes get started through this workflow, removing the bootstrap conditionals will impact it, meaning I'll only be able to remove these once all the processes with the bootstrap. So far it only affected BAM visualization, so no problem...
 
 
Hey Pedro,

Did removing the conditional starts actually suspend any process?
From what I remember the conditinal starts should not affect suspensions unless they have input parameters.


João Rosado wrote:
Hey Pedro,

Did removing the conditional starts actually suspend any process?
From what I remember the conditinal starts should not affect suspensions unless they have input parameters.

 
 
Hi João,

Thank you for your reply.

Yes, indeed. I actually recorded the publish output after deleting one of the conditional starts:

    - Suspending 4 'Processo Orçamento' processes: activity 'Autorizar Orcamento' was moved back in the flow path.
    - Suspending 5 'Processo Orçamento' processes: activity 'Indicar Projetista' was moved back in the flow path.
    - Suspending 236 'Processo Orçamento' processes: activity 'Resposta Autorizar Orçamento' was moved back...
    - Suspending 243 'Processo Orçamento' processes: activity 'Resposta Indicar Projetista' was moved back...

Processo Orçamento is the main process holding the deleted conditional starts.
The activities are subprocess calls and decisions which were affected.
The removal of conditional starts was further recorded in the Process_Upgrade system entity.
This discussion was on our internal forums, but everyone agreed it would be cool to share with the Community. Enjoy!


Miguel
Hi Miguel,
Thank you for sharing this information that is very interesting and useful, related to BPT bootstrap..Good work..
Hope you share more posts like this in the future..

Cheers,
Gonçalo Martins