This will be the last episode of our Jakarta EE Batch adventure. In this post we will see our example application in action and see how to restart a batch job and the mechanics behind it.

Test Data

To have our application actually do anything we need some test data. I found a nice web site which generates up to 100k data records and you can define how the data should look like. And it is even for free. Great service! Thanks onlinedatagenerator.com.

We can download this test data set from the Bitbucket project site download section and upload it to the IFS on our IBM i server. The data is stored as SQL INSERT statements and also includes the SQL CREATE TABLE statement. We just need a library where we will store the data and we are good to go.

1
RUNSQLSTM SRCSTMF('/home/mihael/customers.sql') COMMIT(*NONE)  DFTRDBCOL(RPGNEXTGEN)

Note: The data set contains 100k data records. Depending on your environment you may get some messages in your job log and the job log may even overflow. So you might want to look out for an inquiry message which you need to answer.

Install Application

We can download a precompiled version of the application including all dependencies from the download section of the project site. We can either run it from our PC, IBM i server or any other server. Depending on the host you run the application you will need to adjust the configuration of the database connection.

Run Application

Our application is a Java batch application which can be executed in the same way as any other Java application (and that is one reason I Iike it so much because you don’t have to jump through any hoops to get where you want to go):

Put every jar in the classpath and call the main class.

And we will do exactly that!

1
java -cp batch.jar:libs/* rpgnextgen.batch.Main start customerTransfer --customer 100

We also appended some parameters to the call.

  • ˋstartˋ : Action parameter (resolved by the class ˋrpgnextgen.batch.BatchStarterˋ
  • ˋcustomerTransferˋ : Batch job id from the XML file under ˋMETA-INF/batch-jobsˋ
  • ˋ–customer 100ˋ : These two parameters identify the customer to process. These parameters will get injected into our ˋCustomerReaderˋ class. Optional.

Application Logic

The application either processes one customer (parameter customer) or processes all active customers starting from a given change date (parameter date). If neither customer nor date is provided the application will process all active customers from the current date.

Note: There are no active customers in the given data set. But feel free to modify the data as you want or need.

Note: The date parameter expects the date in ISO format like 2022-01-31 .

Database Configuration

But before we can call our nice little batch program we need to configure our database connection.

Luckily we used MicroProfile Config for our configuration. Now we have multiple ways to get our configuration into the application. We have a base config file in META-INF named microprofile-config.properties. If we don’t do anything else the application will take database configuration from this properties file.

We can override these value by passing Java system properties on the call as additional parameters.

But we can also use environment variables. So under linux we would just execute the following in a shell:

1
2
3
4
export ibmi_host=localhost
export ibmi_user=mihael
export ibmi_password=uwillneverknow
export ibmi_libraries=RPGNEXTGEN

Note: You will need to adjust those values to match your environment and setup.

Restart Application

The Jakarta EE Batch specification also states that a failed batch run needs to be able to be restarted. For this to work you need to provide some checkpoint information. Checkpoint information can be used in the ItemReader and/or ItemWriter.

And the usage is pretty direct and easy: You just implement the corresponding checkpointInfo methods where you return the checkpoint information for the current state of the reader or writer. It can be any object which can be serialized (implements Serializable).

On a restart of an application the open method is passed the last checkpoint information which we provided in the checkpointInfo method.

1
2
3
4
5
@Override
public Serializable checkpointInfo() throws Exception {
	logger.finer("Saving reader checkpoint data: " + lastReturnedCustomer);
	return lastReturnedCustomer == null ? null : lastReturnedCustomer.id;
}

In a real application this could be the last processed customer number (object of class Integer), see class CustomerReader line 64. We would get that Integer on a restart as a parameter on the open method and are checking it at line 43.

1
2
3
4
if (checkpoint != null) {
	lastCustomerId = (Integer) checkpoint;
	logger.info("Restarting after customer " + lastCustomerId);
}

If we have a lastCustomerId we are starting our processing after that customer by specifying in our SQL the lastCustomerId as our offset to start at.

1
2
3
@SqlQuery("SELECT id, company, street, city, country, discount, active FROM customer "
		+ "WHERE active = 1 AND id > :offset AND changeDate >= :changeDate")
ResultIterator<Customer> listActiveCustomers(@Bind("offset") int offset, @Bind("changeDate") LocalDate changeDate);

But how do we actually restart the application?

For restarting the application we need the last failed batch job id (not IBM i job). We can get this from the log output of the original run. It states:

1
rpgnextgen.batch.BatchStarter: Starting batch job with id 401

So 401 is the batch job id we need to pass to on our restart.

1
java -cp batch.jar:libs/* rpgnextgen.batch.Main restart 401 --date 2022-01-01

Note: It is important to pass the same batch parameters to the restart as on the original batch run. Those parameters are not persisted by the framework.

Wrap Up

I think the batch specification is quite well written and fulfills my needs for batch processing quite well. We can do all the necessary work in a structured way which lets us reuse components and split up workloads in multiple ways and even restart batch runs. Great work!

Happy batching!

Mihael