fluxx streaming

20
Lee Sheng [email protected]

Upload: yelp-engineering

Post on 16-Mar-2018

195 views

Category:

Technology


0 download

TRANSCRIPT

Page 1: Fluxx Streaming

Lee Sheng

[email protected]

Page 2: Fluxx Streaming

Fluxx is a simple card game where the rules

and goals are ever changing. Cards include:

• Rules - how to play the game

• Goals - victory conditions

• Keepers - help you achieve goals

• Creepers - prevent you from winning

INTRODUCTION TO

Page 3: Fluxx Streaming

Per Fluxx rules, we start with basic

logging:

Log one line:logging.info(“Playing Streaming Fluxx!”)

Read one line:tail -1 streaming_fluxx.log

BASIC LOGGINGB

AS

IC L

OG

GIN

G To log data:

Log one line

Read one line

Page 4: Fluxx Streaming

CENTRALIZED LOGGINGC

EN

TR

AL

IZE

D L

OG

GIN

G

Page 5: Fluxx Streaming

FACEBOOK SCRIBEF

AC

EB

OO

K S

CR

IBE

Page 6: Fluxx Streaming

OVER 9000 HOSTSO

VE

R 9

000 H

OS

TS

!

Page 7: Fluxx Streaming

LOG AGGREGATORSL

OG

AG

GR

EG

AT

OR

S

Page 8: Fluxx Streaming

ALL THE USE CASES!A

LL

TH

E U

SE

CA

SE

S!

Page 9: Fluxx Streaming

APACHE KAFKAA

PA

CH

E K

AF

KA

Page 10: Fluxx Streaming

TRAFFIC IMBALANCE!T

RA

FF

IC I

MB

AL

AN

CE

!

Page 11: Fluxx Streaming

BUFFER NODESB

UF

FE

R N

OD

ES

Page 12: Fluxx Streaming

RELIABLE MESSAGESR

EL

IAB

LE

ME

SS

AG

ES

Page 13: Fluxx Streaming

KAFKA ACKSK

AF

KA

AC

KS

publish(acks=all)

ACK

Page 14: Fluxx Streaming

INFINITE RETENTIONIN

FIN

ITE

RE

TE

NT

ION

Page 15: Fluxx Streaming

PINTEREST SECORP

INT

ER

ES

T S

EC

OR

Page 16: Fluxx Streaming

Build incrementally!

KEY TAKEAWAY 1

Page 17: Fluxx Streaming

Don’t shy from replacing what used to work

KEY TAKEAWAY 2

Page 18: Fluxx Streaming

FINAL PRODUCTION SIDE ARCHITECTURE

Page 19: Fluxx Streaming

WINNING SYSTEM! (for now…)

Page 20: Fluxx Streaming

TA

LK

OV

ER

CO

NT

AC

T C

AR

D

Lee Sheng

[email protected]

@bogosort