Spring Batch 3 Annotation Example using MySQL and Gradle

By Arvind Rai, November 06, 2014

In this page we will learn how to run Spring Batch 3 using annotation. To run the example we are using MySQL database. We are building our project using spring batch boot in gradle. Spring Batch is the game of ItemReader, ItemProcessor and ItemWriter. Read the large data using ItemReader from any resource, apply business login to it using ItemProcessor and then finally write the data to any resource using ItemWriter. In our example we will read data from csv file and apply a business logic and then write it to MySQL database. Find the detailed description with example.

Required Software to Run Example

To run Spring Batch 3 Annotation Example below software is needed.
1. JDK 7
2. MySQL 5
3. Gradle
4. Eclipse

Gradle Build File for Spring Batch Boot

Find the gradle build file. Spring provides spring batch boot to resolve JAR dependency.
build.gradle

apply plugin: 'java'
apply plugin: 'eclipse'
repositories {
    mavenCentral()
}
dependencies {
    compile 'org.springframework.boot:spring-boot-starter-batch:1.1.8.RELEASE'
    compile 'mysql:mysql-connector-java:5.1.31'
    compile 'commons-dbcp:commons-dbcp:1.4'
}

Create Batch Configuration File using @EnableBatchProcessing Annotation

To create spring batch configuration file we need to use @EnableBatchProcessing annotation. EnableBatchProcessing provides spring batch feature in configuration.
BatchConfiguration.java

package com.concretepage.springbatch;
import javax.sql.DataSource;
import org.apache.commons.dbcp.BasicDataSource;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.launch.support.RunIdIncrementer;
import org.springframework.batch.item.ItemProcessor;
import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.ItemWriter;
import org.springframework.batch.item.database.BeanPropertyItemSqlParameterSourceProvider;
import org.springframework.batch.item.database.JdbcBatchItemWriter;
import org.springframework.batch.item.file.FlatFileItemReader;
import org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper;
import org.springframework.batch.item.file.mapping.DefaultLineMapper;
import org.springframework.batch.item.file.transform.DelimitedLineTokenizer;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.io.ClassPathResource;
import org.springframework.jdbc.core.JdbcTemplate;
@Configuration
@EnableBatchProcessing
public class BatchConfiguration {
    @Bean
    public ItemReader<Student> reader() {
        FlatFileItemReader<Student> reader = new FlatFileItemReader<Student>();
        reader.setResource(new ClassPathResource("student-data.csv"));
        reader.setLineMapper(new DefaultLineMapper<Student>() {{
            setLineTokenizer(new DelimitedLineTokenizer() {{
                setNames(new String[] {"stdId", "subMarkOne", "subMarkTwo" });
            }});
            setFieldSetMapper(new BeanWrapperFieldSetMapper<Student>() {{
                setTargetType(Student.class);
            }});
        }});
        return reader;
    }
    @Bean
    public ItemWriter<Marksheet> writer(DataSource dataSource) {
        JdbcBatchItemWriter<Marksheet> writer = new JdbcBatchItemWriter<Marksheet>();
        writer.setItemSqlParameterSourceProvider(new BeanPropertyItemSqlParameterSourceProvider<Marksheet>());
        writer.setSql("INSERT INTO marksheet (studentId,totalMark) VALUES (:stdId,:totalSubMark)");
        writer.setDataSource(dataSource);
        return writer;
    }
    @Bean
    public ItemProcessor<Student, Marksheet> processor() {
        return new StudentItemProcessor();
    }
    @Bean
    public Job createMarkSheet(JobBuilderFactory jobs, Step step) {
        return jobs.get("createMarkSheet")
                .flow(step)
                .end()
                .build();
    }
    @Bean
    public Step step(StepBuilderFactory stepBuilderFactory, ItemReader<Student> reader,
            ItemWriter<Marksheet> writer, ItemProcessor<Student, Marksheet> processor) {
        return stepBuilderFactory.get("step")
                .<Student, Marksheet> chunk(5)
                .reader(reader)
                .processor(processor)
                .writer(writer)
                .build();
    }
    @Bean
    public JdbcTemplate jdbcTemplate(DataSource dataSource) {
        return new JdbcTemplate(dataSource);
    }
    @Bean
	public DataSource getDataSource() {
	    BasicDataSource dataSource = new BasicDataSource();
	    dataSource.setDriverClassName("com.mysql.jdbc.Driver");
	    dataSource.setUrl("jdbc:mysql://localhost:3306/concretepage");
	    dataSource.setUsername("root");
	    dataSource.setPassword("");
	    return dataSource;
	}
}

Configure Spring Batch ItemReader and ItemWriter

ItemReader reads data from different resources. We can read data from flat file, xml and database and in each case respective ItemReader instance is returned. Using annotation, create an ItemReader bean in configuration file. In our example we are reading data from a flat file and so using FlatFileItemReader.
ItemWriter writes data to flat file, xml or database using different ItemWriter class. ItemWriter uses the data read by ItemReader. In our example we are writing data to mysql database and so using JdbcBatchItemWriter.

Configure Spring Batch ItemProcessor

ItemProcessor is the link between ItemReader and ItemWriter. ItemProcessor is used to apply any business login in the data read by ItemReader and then the data is written using ItemWriter. To create our business logic we will create a class and implement ItemProcessor interface. ItemProcessor has a method process() which needs to be defined to implement our business logic. In our example we have done a sample business login in ItemProcessor class.
StudentItemProcessor.java

package com.concretepage.springbatch;
import org.springframework.batch.item.ItemProcessor;
public class StudentItemProcessor implements ItemProcessor {
    @Override
    public Marksheet process(final Student student) throws Exception {
    	int totalMark = student.getSubMarkOne()+student.getSubMarkTwo();
    	System.out.println("student id:"+student.getStdId() +" and Total mark:"+ totalMark);
    	Marksheet marksheet = new Marksheet(student.getStdId(), totalMark);
        return marksheet;
    }
}

Find the Student class which will be used by ItemReader to read data.
Student.java

package com.concretepage.springbatch;
public class Student {
	private String stdId;
	private int subMarkOne;
	private int subMarkTwo;
	public String getStdId() {
		return stdId;
	}
	public void setStdId(String stdId) {
		this.stdId = stdId;
	}
	public int getSubMarkOne() {
		return subMarkOne;
	}
	public void setSubMarkOne(int subMarkOne) {
		this.subMarkOne = subMarkOne;
	}
	public int getSubMarkTwo() {
		return subMarkTwo;
	}
	public void setSubMarkTwo(int subMarkTwo) {
		this.subMarkTwo = subMarkTwo;
	}
}

Find the Marksheet class which will be used by ItemWriter to write the data.
Marksheet.java

package com.concretepage.springbatch;
public class Marksheet {
	private String stdId;
	private int totalSubMark;
	public Marksheet(String stdId, int totalSubMark){
		this.stdId = stdId;
		this.totalSubMark = totalSubMark;
	}
	public String getStdId() {
		return stdId;
	}
	public void setStdId(String stdId) {
		this.stdId = stdId;
	}
	public int getTotalSubMark() {
		return totalSubMark;
	}
	public void setTotalSubMark(int totalSubMark) {
		this.totalSubMark = totalSubMark;
	}
}

We need to introduce our ItemProcessor in configuration class. So create a bean for ItemProcessor using annotation.

Configure Spring Batch Step using StepBuilderFactory

We need to configure Step to run batch processing. Step is steps in batch processing like read, process and then write. Use StepBuilderFactory to define spring batch Step. Step provides to process data in chunk. We can set a chunk size.

Configure Spring Batch Job using JobBuilderFactory

Spring Batch Job calls the Step defined to process spring batch. We can restart the Step using Job . Declare a bean for Job in configuration that is necessary part to run spring batch.

Run Spring Batch using SpringApplication.run

To run the batch use SpringApplication.run(). This will call the batch processing.
Main.java

package com.concretepage.springbatch;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.util.List;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.context.ApplicationContext;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.jdbc.core.RowMapper;
@ComponentScan
@EnableAutoConfiguration
public class Main {
    public static void main(String[] args) {
        ApplicationContext ctx = SpringApplication.run(Main.class, args);
        List result = ctx.getBean(JdbcTemplate.class).query("select studentId,totalMark FROM marksheet", 
        		new RowMapper() {
            @Override
            public Marksheet mapRow(ResultSet rs, int row) throws SQLException {
                return new Marksheet(rs.getString(1), Integer.parseInt(rs.getString(2)));
            }
        });
        System.out.println("Number of Record:"+result.size());
    }
}

Project Configuration in Eclipse

Spring Batch 3 Annotation Example using MySQL and Gradle

Flat File for Demo

Find the CSV file which is acting as input file for spring batch processing in our example.
student-data.csv

student-1,10,15
student-2,12,17
student-3,15,13

MySQL Table Schema

Find the schema of the table which we are using in our example.
Table: marksheet

CREATE TABLE `marksheet` (
	`studentId` VARCHAR(50) NULL DEFAULT NULL,
	`totalMark` INT(11) NULL DEFAULT NULL
)

Output

Console output will be as below.

student id:student-1 and Total mark:25
student id:student-2 and Total mark:29
student id:student-3 and Total mark:28

Number of Record:3

Find the data inserted into table.

Spring batch creates other table too in the database for its processing.

Download Source Code

spring-batch-3-annotation-example-using-mysql-gradle.zip

POSTED BY

ARVIND RAI

LEARN MORE

JAVA SPRING BOOT ANGULAR Angular Material NGRX JAVASCRIPT ANDROID THYMELEAF FREEMARKER Hibernate Struts 2 Testing MAVEN APACHE API JACKSON API JUNIT 5