apache flink first half of 2015 community update

Bay Area Apache Flink Meetup #1

Community Update June 2015

Robert MetzgerCommitter and PMC

Memberrmetzger@apache.org

@rmetzger_

Flink’s Recent History

April 2014 April 2015Dec 2014

Top Level Project Graduation

0.7

0.6

0.5

0.9

0.9-m1

This talk

3

December 2014 Graduation as Top Level Project

flink.apache.org

Official Press release: https://blogs.apache.org/foundation/entry/the_apache_software_foundation_announces69

• Officially acknowledged as a healthy, vibrant and growing open-source community, following the Apache Way.

• Attracted many new contributors and committers during incubation

• More than 75+ contributors

4

January 2015Release 0.8.0

• Lots of stability improvements• Scala Streaming API• Improved streaming windowing

semantics• Kryo as the new fallback serializer• Extended FileSystem support

(Hadoop compat)

flink.apache.org

5

February 2015

• Table API added• Gelly – Graph API

added• flink-hcatalog added• Apache Flink +

Apache SAMOA (incubating) integration Machine Learning on streams

flink.apache.org

6

March 2015

• Flink runner for Google DataFlow API available

• First code of Flink ML added

flink.apache.org

7

New Wiki pages with system internals

• Data Exchange between tasks

• Type Extraction and Serialization• Memory Management in (Batch API)• Akka and Actors

flink.apache.org

8

April 2015• Flink 0.9-milestone1 has been

released• Storm compatibility layer pull request• Interactive Scala Shell added• Experimental Python API added

flink.apache.org

9flink.apache.org

Apache Flink got 3 Google Summer of Code students accepted:• Distributed Pattern Matching for Apache

Flink• Exact and Approximate Statistics for Data

Streams and Windows in Flink• Apache Flink: Asynchronous Iterations and

Updates

10

May 2015• Gelly Scala API started (Community

contrib)• Stockholm and Bay Area Meetup

groups started

flink.apache.org

Meetup in BudapestMeetup in Stockholm

11

Website redesign

flink.apache.org

12

June 2015• Zeppelin + Flink Integration merged• Static Code Analysis added• VOTE for Flink 0.9.0 started

Biggest release so far

• Chicago Meetup group started

flink.apache.org

13

Flink + Zeppelin integration

flink.apache.org

14

Flink 0.9.0 Highlights

• APIs on top:– Flink Machine Learning Library– Gelly Graph Processing API– Table API (SQL-style operations)

• Internals– Enhanced YARN (single jobs, container restart)– Flink On Apache Tez– Akka for process communication– secure HDFS / YARN support

• Streaming Fault-Tolerance• Reworked intermediate results representation

flink.apache.org

15

Ways to get in touch with the Community

• Mailing lists– news@flink.apache.org (announcements only,

low traffic)– user@flink.apache.org (user support, medium

traffic)– dev@flink.apache.org (internal discussions, high

traffic)

• Twitter (@ApacheFlink)• flink.apache.org/blog• Local Meetup groups:

http://apache-flink.meetup.com/flink.apache.org

16

Meetup Groups

flink.apache.org

17

Flink Forward registration & call for abstracts is open now

flink.apache.org

• 12/13 October 2015• Kulturbrauerei Berlin• With Flink Workshops / Trainings!