apache flink · 2016-07-18 · apache flink flink core features - general purpose data processing...
TRANSCRIPT
Apache Flink
Flink Core Features - General purpose data processing for clusters
- Compatibility: Kafka, Hadoop YARN, HDFS, …
- Fully pipelined native streaming runtime
- Built-in program optimizations
- Stateful operators and UDF support
- A single system for batch & stream processing
Platform Overview and Related Systems
AnOpenSourcePla/ormforDistributedStreamandBatchDataProcessing
The Flink Modular System Stack Flink in the Analytics Ecosystem
Flink History and Community
Flink Success Stories / Use Cases
Stay Tuned!
MailingList:[email protected] flink.apache.org/blog @ApacheFlink
WhydidZalandochoseFlink?• Flinkprocesseseventstreamsathigh
throughputswithconsistentlylowlatencies.• Flinkprovidesanefficient,easytouse,key/
valuebasedstate.• Flinkisatruestream-processingframework:
EventsareprocessedoneataEmeandeacheventhasitsownEmewindow.
• StreamimperfecEonslikeout-of-ordereventsareeasilyhandledusingtheframework’seventEmeprocessingsupport.
• FlinksupportwasperceivedtobebeHerthanSpark’s.ZalandoisabletocontactFlinkdevelopersdirectlyandtheyareeagertoimproveFlinkandaddressourissues.
Source:hHps://tech.zalando.de/blog/apache-showdown-flink-vs.-spark/
FormoreinformaEonvisit:hHp://flink.apache.orghHp://www.dima.tu-berlin.dehHp://data-arEsans.com
2008 2009 2010
201120122014
2015
2016
1st Flink Forward Conference
2nd Flink Forward ConferenceSeptember 2016flink-forward.org
Proposal for Stratosphere I
Spinning Fast Ite-rative Dataflowspaper published
Founded
Grant Award Start of Stratosphere I
Nephele / PACTs paper published
Proposal for Stratosphere II
Grant Award Start of Stratosphere II
Flink Incubator Project
Stratosphere System-paper published
Initial vision for a big data analytics platform
FlinkTop Level Project
21 Meetups Worldwide 186 Contributors6326 Members20 Cities12 Countries
Flink Community GroupsAcross Europe
2010
2012
2015 2015
20162016
Berlin 758Paris 500Madrid 384Stockholm 313Brussels 279London 190