README.adoc

camunda-platform-7-custom-batch User Guide

What Is Camunda Batch

Currently Camunda Batch is mainly used for offloading the workload of internal commands like e.g. process migration or process cancellation.

So if you for example select 10000 processes for process migration, the engine will create a batch which will handle those instances in small chunks. By default, the batch will create 100 jobs, where each job will work on one process migration. (In it’s own transaction!) After this, it will create another 100 jobs and so on, until the work on all instances is finished. Benefit of this is that the migration will not block anything and that there is not just one big transaction.

Additionally, the cockpit already provides a view to monitor batches. (See also)

For more information regarding Camunda Batch, visit also Camunda’s user guide: User Guide

Why This Extension?

Using Camunda Batch for your own purposes is a very time consuming and complex thing. With this extension it will be much more easier to create an own batch. This could be helpful to offload huge workload like doing reassignments of user tasks.

How to Use

Overview

A new batch is created with help of the the CustomBatchBuilder and a specific BatchJobHandler for a set of batch data. The BatchJobHandler has two functions: Help creating all the necessary jobs and processing the individual items from the batch data.

With the default configuration, there will exist one job for each data item. The jobs are created in small chunks which have to be processed until the next chunk is created.

Each job will have it’s piece of data processed by calling the same BatchJobHandler.

Implementation Steps

Provide Implementation of CustomBatchJobHandler

You have to provide two methods:

• execute(List<T> jobData, CommandContext commandContext)

This method will be called by the batch job to process a piece of the batch data. It depends on the configuration of the batch (invocationsPerBatchJob) how many items you will get. In other words, jobData will contain a maximum of invocationsPerBatchJob number of items.

• getType()

The name / id of the job handler.

Example Implementation:

@Component
public class PrintStringBatchJobHandler extends CustomBatchJobHandler<String> {
  @Override
  public void execute(List<String> jobData, CommandContext commandContext) {
      data.forEach(dataEntry -> logger.info("Work on data entry: " + dataEntry));
  }

  @Override
  public String getType() {
      return "print-string-batch-handler";
  }
}

The abstract CustomBatchJobHandler takes care about:

• Creating Jobs + saving Configuration to ByteArray Table

• Reading data from ByteArray Table for each Batch Jobs

• Cleanup of Jobs + Configuration

Register CustomBatchJobHandler Implementation

You have to register the implementation of the job handler during startup of engine. This can be done by passing the job handler to the CustomBatchHandlerPlugin and adding this plugin to the engine.

With spring boot it would be enough to provide this as a bean:

@Bean
public ProcessEnginePlugin customBatchHandlerPlugin(PrintStringBatchJobHandler printStringBatchJobHandler) {
  return new CustomBatchHandlerPlugin(Collections.singletonList(printStringBatchJobHandler));
}

If you just use camunda.cfg.xml / applicationContext.xml to configure your engine:

    <property name="processEnginePlugins">
      <list>

        <bean class="org.camunda.community.batch.plugin.CustomBatchHandlerPlugin">
          <constructor-arg>
            <bean class="org.camunda.community.batch.example.simple.PrintStringBatchJobHandler"/>
          </constructor-arg>
        </bean>

      </list>
    </property>

Examples:

• applicationContext.xml

• camunda.cfg.xml

Create the Batch

For creating the batch, you have to use the CustomBatchBuilder.

In the minimum setting (with default batch values and configuration from Context) this looks like:

 final Batch batch = CustomBatchBuilder.of(data) #List of Objects which should be processed
        .jobHandler(printStringBatchJobHandler)
        .create();

When calling create, the batch will be inserted and the job executor will start to work on the batch with it’s next run.

The builder takes care about:

• Creating the Batch (Entity)

• Creating the Seed and Monitor Job

• Saves the BatchConfiguration data, no matter of the data type

• Saves you from handling with ConfigurationBytes

Configuration of the Batch

The batch could be configured during building it with CustomBatchBuilder:

 final Batch batch = CustomBatchBuilder.of(data)
        .configuration(configuration)
        .jobHandler(printStringBatchJobHandler)
        .jobsPerSeed(10)
        .invocationsPerBatchJob(5)
        .jobPriority(0L)
        .exclusive(true)
        .create(configuration.getCommandExecutorTxRequired());

Property configuration

Per default, the builder tries to get the process engine configuration from context.

Context.getProcessEngineConfiguration()

Custom Configuration could be set with:

CustomBatchBuilder.of().configuration(configuration)

Property jobHandler

Here you have to provide the batch job handler which should be used by the batch.

CustomBatchBuilder.of().jobHandler(printStringBatchJobHandler)

Property jobsPerSeed

Number of batch execution jobs created per seed job invocation. The batch seed job is invoked until it has created all batch execution jobs required by the batch.

CustomBatchBuilder.of().jobsPerSeed(10)

Default is 100

Property invocationsPerBatchJob

How many data should be passed to the job handler.

E.g., for the process instance migration batch this specifies the number of process instances which are migrated per batch execution job.

CustomBatchBuilder.of().invocationsPerBatchJob(5)

Default is 1

Property jobPriority

Priority of the seed job + monitoring job + batch jobs.

Note: The batch jobPriority is only considered when using Job Executor with the corresponding Acquisition Strategy jobExecutorAcquireByPriority. (see camunda documentation)

Default is 0l

CustomBatchBuilder.of().jobPriority(5L)

Default is 1

Property exclusive

Should a batch job be handled exclusive, take care when setting this to false! (See Camunda Job Docs)

CustomBatchBuilder.of().exclusive(false)

Default is true