SpringBatch高阶应用：大数据批处理框架实战指南

开发运维 2024-05-06 大白菜程序猿 手机阅读

本篇文章主要内容：通过Spring Batch从一个库中读取数据进过处理后写入到另外一个库中。

1. 环境准备

1.1 引入依赖


  org.springframework.boot
  spring-boot-starter-batch


  org.springframework.boot
  spring-boot-starter-data-jpa

2.2 配置Job

配置Job启动器

@Bean
JobLauncher userJobLauncher(JobRepository userJobRepository) {
  SimpleJobLauncher jobLauncher = new SimpleJobLauncher() ;
  jobLauncher.setJobRepository(userJobRepository) ;
  return jobLauncher ;
}

配置任务Repository存储元信息

@Bean
JobRepository userJobRepository(DataSource dataSource, PlatformTransactionManager transactionManager) {
  JobRepositoryFactoryBean factory = new JobRepositoryFactoryBean() ;
  factory.setDatabaseType("mysql") ;
  factory.setTransactionManager(transactionManager) ;
  factory.setDataSource(dataSource) ;
  try {
    factory.afterPropertiesSet() ; 
    return factory.getObject() ;
  } catch (Exception e) {
    throw new RuntimeException(e) ;
  }
}

配置ItemReader读取器

@Bean

ItemReader userReader(JobOperator jobOperator) throws Exception {

  JpaPagingItemReaderBuilder builder = new JpaPagingItemReaderBuilder() ;

  builder.entityManagerFactory(entityManagerFactory) ;

  // 每次分页查询多少条数据

  builder.pageSize(10) ;

  builder.queryString("select u from User u where u.uid