Add batch processing, parallel processing/asynchronous processing capabilities

On our radar

We really need something like BPT, but designed for a few additional use cases:

* Batch processing (potentially millions of instances, potentially long-term execution, etc.)
* Parallel processing or a fork/join style mechanism or a Tasks system... to handle complex situations
* Real time asynchronous processing, where it does not use Scheduler on each step, but keeps going.

I think that each of these scenarios could be enabled within the existing BPT framework, if BPT was modified a bit.


Created on 19 Jan 2016
Comments (4)
Most of my work is converting mainframe systems where batch processing is prevelant.  I try and reduce this as much as possible in any new implementation but there always seem to be some processes that require this so any improvement in batch processing helps me.

Thanks for the idea J!

BPT is too much of an overhead. it burdens more than it's handy.

but, the idea behind BPT should be great for implementing your ideas.

J. -

The biggest problem with BPT mright now, is that it depends on the Scheduler Service and operates like a timer-based service bus. That's great for what BPT is meant for, but unless you have a very strong understanding of BPT and how it works, it looks too much like the "B" stands for "Batch", and next thing you know the developer has built something expected to pump 200 transactions per second through a system limited to 10 threads at once, or they try building long batches in something limited to 5 minutes processes.

BPT looks like the hammer for every nail, when really it is a specialized hammer designed to only hit one kind of nail well, can hit a few other nails OK, and destroys anything else it hits.

What BPT should be used for, it is great. But its technical implementation needs to meet the expectations... or a replacement needs to be made that can work the way people want/need BPT to work.


 Just to share that there is a forge component that shows how to achieve this with BPT.

Best regards,

Daniel Martins