Apache

Flink

Language Integrated Queries

(Preview)

Aljoscha Krettek

aljoscha@apache.org

What is This?

• Declarative Expression API for Flink

• Can freely mix declarative operations and

“classic” operations

• Elegance through conciseness

• Robustness through type analysis

• Speed through code-generation

flink.apache.org 1

Introductory Example

flink.apache.org 2

case class Book(title: String, price: Int, …)

val books = env.fromElements(…)

val result = books.filter('price > 10)

Filter

flink.apache.org 3

val result = books.filter('price > 10 && 'author === "Foo")

Join

flink.apache.org 4

val result = books

.join(authors).where('bAuthor === 'aAuthor && price > 10)

.select('aAuthor, 'aAddress)

Aggregation

flink.apache.org 5

val result = books.group('author).select('author, 'price.avg)

Expressions

flink.apache.org 6

val result = books.select('author.substring(0, 'price.avg + 5))

val result = books.select(('price + 5).avg % 2, 'title.count)

Back to the Classic API

flink.apache.org 7

case class MyResult(avg: Int, cnt: Int)

val result: DataSet[MyResult] = books

.select(('price + 5).avg % 2 as 'avg, 'title.count as 'cnt)

.as[MyResult]

.map { … }

Java API

flink.apache.org 8

DataSet<Row> result = books.select("author", "price.avg + 5")

DataSet<Row> result = books

.join(authors).where("bAuthor = aAuthor && price > 10")

.select("aAuthor", "aAddress")

github.com/aljoscha/flink/tree/lin

q

flink.apache.org

github.com/apache/flink

meetup.com/Apache-Flink-

Meetup