spring-batch groovy y gradle
Post on 13-Aug-2015
66 Views
Preview:
TRANSCRIPT
Spring Batch Groovy & Gradle integrationSpring Batch Groovy & Gradle integration
Because we like it…Because we like it… because we have fun with itbecause we have fun with it
About me:About me:
Born with C and Java
Github: http://github.com/amr390 (mostly drops, and tests)
Working with Java and Javascript, take any opportunity to use groovy and python
(I love for both of them)
Grown up with Javasince 2000
Twitter: 390amr (not much into it)
Spring Batch. Groovy and GradleSpring Batch. Groovy and Gradle
What is Spring BatchWhat is Spring Batch
Why integrating spring-batch with Groovy and GradleWhy integrating spring-batch with Groovy and Gradle
Requirements Requirements ➢ DependenciesDependencies
➢ Gradle tasksGradle tasks
What is Spring BatchWhat is Spring Batch
➢Configuring Jobs, Tasks, Steps and stufConfiguring Jobs, Tasks, Steps and stuf
➢Running and Handling JobsRunning and Handling Jobs
Improving Spring Batch performanceImproving Spring Batch performance
➢Multi-threadingMulti-threading
➢Master-Slave approachMaster-Slave approach
➢Parallelizing Step executionsParallelizing Step executions
What is Spring Batch I
Batch Framework
Technical services
Functionality to process large volumes of records
ItemReader, ItemWriter, ItemProcessor
Stateless or Stateful step executions
High-volumeHigh-performanceHigh-scalability
Trough optimization Partitioning techniques
What is Spring Batch IIWhat is Spring Batch II
What is Spring Batch IIIWhat is Spring Batch III
What is Spring Batch IVWhat is Spring Batch IV
STEP:➔ Custom tasklet➔ Restartable➔ Skippable➔ Retry➔ Rollback➔ Listeners Before/After
✗ Reader✗ Processor✗ Writer✗ Chunk✗ Step
➔ Step flow (on status)
What is Spring Batch: exampleWhat is Spring Batch: example
Philae Lander dataPhilae Lander data
Philae Lander is a spacecraft that sends analysis data from comet 67P/Churyumov–Gerasimenko (https://en.wikipedia.org/wiki/Philae_(spacecraft))
Sends tones data in files encrypted in Klingon or Murciano.
Files are zipped, and organized with a naming rule of file1, file2 etc ...
It is always fun using groovy
Can be nicely coupled to other platforms Grifon, Grails...
Power of Gradle
Homogeneous Architecture
Why integrating Spring-batch with Why integrating Spring-batch with groovy and Gradlegroovy and Gradle
Plug insPlug insRepositoriesRepositories
DependenciesDependencies TasksTasks
Requeriments IIRequeriments II
Repositories
Plugins
Dependencies IIIDependencies III
Dependencies
Requirements IVRequirements IV
Gradle tasks
MainClassName: Sets the project main class (because it is a global definition)
run: sets the required arguments for run task, in this case the job configuration class and the Job to be launched.
@Configuration@EnableBatchProcessing
Injects
a JobRepository a JobLauncher a JobRegistry a PlatformTransactionManagera JobBuilderFactorya StepBuilderFactory
Spring Batch Groovy & Gradle Spring Batch Groovy & Gradle integrationintegration
Job definition configurations, steps, tasklets and stuf
Spring Batch Groovy & Gradle Spring Batch Groovy & Gradle integrationintegration
Running and Handling Jobs
Spring batch admin
Spring Batch Groovy & Gradle Spring Batch Groovy & Gradle integrationintegration
Running and Handling Jobs
Job Launchers
org.springframework.batch.core.launch
Interface JobLauncher
All Known Implementing Classes:
SimpleJobLauncher
JobExecution run(Job job, JobParameters jobParameters) throws JobExecutionAlreadyRunningException, JobRestartException, JobInstanceAlreadyCompleteException, JobParametersInvalidException
Spring Batch Groovy & Gradle Spring Batch Groovy & Gradle integrationintegration
Running and Handling Jobs
Job Launchers
org.springframework.batch.core.launch.support
Class CommandLineJobRunner
Spring Batch Groovy & Gradle Spring Batch Groovy & Gradle integrationintegration
Performance improvement
There different approached in order to improve the job execution performance
Multi-threading: Using a ThreadPoolTaskExecutor
Remote chunking:Master executes read phase of step slaves are distributed in nodes and execute processing and writing from a chunkProvider.
Partitioning: Clone Job execution step and run steps in parallel
Remote Partitioning:
Instead of running the steps clone locally steps are
distributed
Parallel steps:step execution is splitted to
execute several steps in parallel flows.
Spring Batch Groovy & Gradle Spring Batch Groovy & Gradle integrationintegration
Multi
threading
Spring Batch Groovy & Gradle Spring Batch Groovy & Gradle integrationintegration
Multi
threading
Using task namespace <task:executor id="taskExecutor" pool-size="10" />
By java configuration method:ThreadPoolTaskExecutor taskExecutor ...
Steps must be thread safe
Spring Batch Groovy & Gradle Spring Batch Groovy & Gradle integrationintegration
Master-Slave
approach
PartitionHandler: Interface defining the responsabilities of controlling the execution of partitioned step
Partitioner: Interface to determine the way of creating partitions (key-range, file-chunks...)
Spring Batch Groovy & Gradle Spring Batch Groovy & Gradle integrationintegration
Master-Slave
approach
Spring Batch Groovy & Gradle Spring Batch Groovy & Gradle integrationintegration
Parallelizing Step executions
Spring Batch Groovy & Gradle Spring Batch Groovy & Gradle integrationintegration
Parallelizing Step executions
@Beanpublic SimpleFlow mainFlow() { SimpleFlow splitFlow = new FlowBuilder<SimpleFlow>("Split Flow") .split(new SimpleAsyncTaskExecutor()) .add(flow2(), flow3()) .build(); return new FlowBuilder<SimpleFlow>("Main Flow") .start(flow1()) .next(splitFlow) .end();}
<batch:step id="step1" next="">...</batch:step><batch:split id="splitSteps" next="step3">
<batch:flow> <batch:step … /></batch:flow><batch:flow> <batch:step … /></batch:flow>
</batch:split><batch:step id="step3">
<batch:tasklet ref="tasklet" /></batch:step>
Demo Github: https://github.com/amr390/groovy-springbatch-demo
Spring batch reference page: http://docs.spring.io/spring-batch/trunk/reference/Spring batch in action: http://www.manning.com/templier/
Spring batch horizontal-vertical scaling:http://www.ontheserverside.com//blog/2014/07/23/horizontal-and-vertical-scaling-strategies-for-batch-applications
Spring batch admin: http://docs.spring.io/spring-batch-admin/
Spring Integration reference page: http://docs.spring.io/spring-integration/reference/
Groovy: http://www.groovy-lang.org/documentation.html
Gradle: https://docs.gradle.org/current/release-notes
Spring Batch Groovy & Gradle Spring Batch Groovy & Gradle integrationintegration
References
top related