apache flink first half of 2015 community update
TRANSCRIPT
Bay Area Apache Flink Meetup #1
Community Update June 2015
Robert MetzgerCommitter and PMC
@rmetzger_
Flink’s Recent History
April 2014 April 2015Dec 2014
Top Level Project Graduation
0.7
0.6
0.5
0.9
0.9-m1
This talk
3
December 2014 Graduation as Top Level Project
flink.apache.org
Official Press release: https://blogs.apache.org/foundation/entry/the_apache_software_foundation_announces69
• Officially acknowledged as a healthy, vibrant and growing open-source community, following the Apache Way.
• Attracted many new contributors and committers during incubation
• More than 75+ contributors
4
January 2015Release 0.8.0
• Lots of stability improvements• Scala Streaming API• Improved streaming windowing
semantics• Kryo as the new fallback serializer• Extended FileSystem support
(Hadoop compat)
flink.apache.org
5
February 2015
• Table API added• Gelly – Graph API
added• flink-hcatalog added• Apache Flink +
Apache SAMOA (incubating) integration Machine Learning on streams
flink.apache.org
6
March 2015
• Flink runner for Google DataFlow API available
• First code of Flink ML added
flink.apache.org
7
New Wiki pages with system internals
• Data Exchange between tasks
• Type Extraction and Serialization• Memory Management in (Batch API)• Akka and Actors
flink.apache.org
8
April 2015• Flink 0.9-milestone1 has been
released• Storm compatibility layer pull request• Interactive Scala Shell added• Experimental Python API added
flink.apache.org
9flink.apache.org
Apache Flink got 3 Google Summer of Code students accepted:• Distributed Pattern Matching for Apache
Flink• Exact and Approximate Statistics for Data
Streams and Windows in Flink• Apache Flink: Asynchronous Iterations and
Updates
10
May 2015• Gelly Scala API started (Community
contrib)• Stockholm and Bay Area Meetup
groups started
flink.apache.org
Meetup in BudapestMeetup in Stockholm
11
Website redesign
flink.apache.org
12
June 2015• Zeppelin + Flink Integration merged• Static Code Analysis added• VOTE for Flink 0.9.0 started
Biggest release so far
• Chicago Meetup group started
flink.apache.org
13
Flink + Zeppelin integration
flink.apache.org
14
Flink 0.9.0 Highlights
• APIs on top:– Flink Machine Learning Library– Gelly Graph Processing API– Table API (SQL-style operations)
• Internals– Enhanced YARN (single jobs, container restart)– Flink On Apache Tez– Akka for process communication– secure HDFS / YARN support
• Streaming Fault-Tolerance• Reworked intermediate results representation
flink.apache.org
15
Ways to get in touch with the Community
• Mailing lists– [email protected] (announcements only,
low traffic)– [email protected] (user support, medium
traffic)– [email protected] (internal discussions, high
traffic)
• Twitter (@ApacheFlink)• flink.apache.org/blog• Local Meetup groups:
http://apache-flink.meetup.com/flink.apache.org
16
Meetup Groups
flink.apache.org
17
Flink Forward registration & call for abstracts is open now
flink.apache.org
• 12/13 October 2015• Kulturbrauerei Berlin• With Flink Workshops / Trainings!