flightcaster presentation hadoop

15

Upload: hadoop-user-group

Post on 17-Jan-2015

1.893 views

Category:

Documents


0 download

DESCRIPTION

 

TRANSCRIPT

Page 1: Flightcaster  Presentation  Hadoop
Page 2: Flightcaster  Presentation  Hadoop
Page 3: Flightcaster  Presentation  Hadoop

75% of major delays     

less than 30 min. before departure

Page 4: Flightcaster  Presentation  Hadoop
Page 5: Flightcaster  Presentation  Hadoop
Page 6: Flightcaster  Presentation  Hadoop

hadoopcascading     ->  clojure

infer cascading-clojurecrane

Open Source

Page 7: Flightcaster  Presentation  Hadoop
Page 8: Flightcaster  Presentation  Hadoop
Page 9: Flightcaster  Presentation  Hadoop

Discrete Event Simulation?

Fuzzy Join with funky predicate?

Page 10: Flightcaster  Presentation  Hadoop

group-by with secondary sort.

accumulator to simulate a discrete event simultor.

Page 11: Flightcaster  Presentation  Hadoop
Page 12: Flightcaster  Presentation  Hadoop

cascading-clojure: motivation

FP

dev

deploy

Page 13: Flightcaster  Presentation  Hadoop

cascading-clojure: lessons learned

planner

serialization

data structures for wide data

Page 14: Flightcaster  Presentation  Hadoop

cascading-clojure: future direction

functions + data structures

abstractions for planner and M/R

Page 15: Flightcaster  Presentation  Hadoop