Spring Batch Pipeline Guide to Product Data Processing

1. Project Setup

To start, we need a Maven project with the necessary dependencies. Here’s the pom.xml:

<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-batch</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-data-jdbc</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-data-jpa</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>
    <dependency>
        <groupId>com.mysql</groupId>
        <artifactId>mysql-connector-j</artifactId>
        <scope>runtime</scope>
    </dependency>
</dependencies><dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-batch</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-data-jdbc</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-data-jpa</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>
    <dependency>
        <groupId>com.mysql</groupId>
        <artifactId>mysql-connector-j</artifactId>
        <scope>runtime</scope>
    </dependency>
</dependencies>

2. Database Configuration

In the application.properties file, configure the database connection and Hibernate settings:

spring.datasource.url=jdbc:mysql://localhost:3306/bootWork
spring.datasource.username=root
spring.datasource.password=root@123
spring.jpa.hibernate.ddl-auto=update
spring.jpa.show-sql=true
spring.batch.jdbc.initialize-schema=always

3. Model Class: Product

The Product class represents the data model. It includes fields like productId, title, description, price, discount, and discountedPrice, along with their respective getters and setters.

public class Product {
    private int productId;
    private String title;
    private String description;
    private BigDecimal price;
    private BigDecimal discount;
    private BigDecimal discountedPrice;
    // Getters and Setters
}

4. Configuration Class: BatchConfig

The BatchConfig class is the central configuration for the Spring Batch job. It defines the job, step, reader, processor, and writer.

Job Definition: The jobBean method creates a batch job with a single step.

Step Definition: The step method defines a step that reads, processes, and writes Product data.
Reader: The FlatFileItemReader reads data from product.csv.

Processor: The CustomeItemProcessor processes each Product to calculate the discounted price.

Writer: The JdbcBatchItemWriter writes the processed data to the PRODUCTTS table in the MySQL database

@Configuration
public class BatchConfig {
    @Bean
    public Job jobBean(JobRepository jobRepository, JobCompletionNotficationImpl listener, Step step) {
        return new JobBuilder("job", jobRepository)
                .listener(listener)
                .start(step)
                .build();
    }
    @Bean
    public Step step(JobRepository jobRepository, DataSourceTransactionManager transactionManager,
                     ItemReader<Product> reader, ItemProcessor<Product, Product> itemProcessor,
                     ItemWriter<Product> writer) {
        return new StepBuilder("jobStep", jobRepository)
                .<Product, Product>chunk(5)
                .reader(reader)
                .processor(itemProcessor)
                .writer(writer)
                .transactionManager(transactionManager)
                .build();
    }
    @Bean
    public FlatFileItemReader<Product> reader() {
        return new FlatFileItemReaderBuilder<Product>()
                .name("itemReader")
                .resource(new ClassPathResource("product.csv"))
                .linesToSkip(1)
                .delimited()
                .names("productId", "title", "description", "price", "discount", "discountedPrice")
                .targetType(Product.class)
                .build();
    }
    @Bean
    public ItemProcessor<Product, Product> itemProcessor() {
        return new CustomeItemProcessor();
    }
    @Bean
    public ItemWriter<Product> writer(DataSource dataSource) {
        return new JdbcBatchItemWriterBuilder<Product>()
                .sql("INSERT INTO PRODUCTTS (product_id, title, description, price, discount, discounted_price) VALUES (:productId, :title, :description, :price, :discount, :discountedPrice)")
                .dataSource(dataSource)
                .beanMapped()
                .build();
    }
}

5. Processing Logic: CustomeItemProcessor5. Processing Logic: CustomeItemProcessor

The CustomeItemProcessor class implements the ItemProcessor interface and contains the logic to calculate the discounted price for each product.

public class CustomeItemProcessor implements ItemProcessor<Product , Product> {
    @Override
    public Product process(Product item) throws Exception {
        BigDecimal discountPercentage = item.getDiscount();
        BigDecimal originalPrice = item.getPrice();
        BigDecimal discountAmount = originalPrice.multiply(discountPercentage).divide(BigDecimal.valueOf(100), RoundingMode.HALF_UP);
        BigDecimal finalPrice = originalPrice.subtract(discountAmount);
        item.setDiscountedPrice(finalPrice);
        return item;
    }
    }

6. Job Listener: JobCompletionNotficationImpl6.

This class implements JobExecutionListener to log the status of the job before and after execution.

@Component
public class JobCompletionNotficationImpl implements JobExecutionListener {
    private Logger logger = LoggerFactory.getLogger(JobCompletionNotficationImpl.class);
    @Override
    public void beforeJob(JobExecution jobExecution) {
        logger.info("Job started");
    }
    @Override
    public void afterJob(JobExecution jobExecution) {
        if (jobExecution.getStatus() == BatchStatus.COMPLETED) {
            logger.info("Job completed successfully");
        }
    }
}

7. Running the Application

Finally, the BootbatchApplication class runs the Spring Boot application, initiating the batch job.

Copy code
@SpringBootApplication
public class BootbatchApplication {
    public static void main(String[] args) {
        System.exit(SpringApplication.exit(SpringApplication.run(BootbatchApplication.class, args)));
    }
}

8. Summary

In this blog, we walked through creating a Spring Batch application that reads product data from a CSV file, processes it to apply discounts, and writes the results into a MySQL database. This application demonstrates the core concepts of Spring Batch, including job configuration, step processing, and data handling with Spring Batch components.

Creating a Spring Batch Pipeline for Product Data Processing