Download - Genn.ai introduction for Buzzwords
Genn.ai (源内)Attached realtime marketing platform
1
http://genn.ai/
Who am I?Takeshi NAKANO
Senior Researcher / Architect.Co-authered Getting started with Solr in Japanese.Co-authered Hadoop Hacks in Japanese.
2
Writing “Getting started with Kafka” in Japanese.
http://genn.ai/3
Company Name Recruit Holdings Co., Ltd
Founded March 31, 1960 (incorporated: August 26, 1963)※governance changed from Recruit Co., Ltd since October 1st, 2012
Headquarters Tokyo, Japan
Number of Group Employees 22,368(as of the end of March,2012)
Number of Group Companies 102(Consolidated Subsidiaries/as of the end of June,2012)
Capital 3,002.64 million yen(since March 1st, 1995)
Consolidated Sales 806,661 million yen (from April 1st 2011 to March 31st 2012)
Consolidated Ordinary Income
117,617 million yen( from April 1st 2011 to March 31st 2012)
http://genn.ai/4
Recruit Holdings
HeadquartersFunction
Global Business
Business R&D
Recruit Career Co., Ltd
Recruit Jobs Co., Ltd
Recruit Sumai Company Ltd
Recruit Marketing Partners Co., Ltd
Recruit Lifestyle Co., Ltd
Recruit Administration Co., Ltd
Recruit Technologies Co., Ltd
Recruit Communications Co., Ltd
Operating Companies Domain
Recruit Staffing Co., Ltd
STAFF SERVICE HOLDINGS CO., LTD
http://genn.ai/
What is our challenge?Clients can get the information about their customers checking their items on our web sites.
Providing a printed report monthly. ▽
Send them more detailed report in real time.
In real time fashion, the value of the information will be dramatically increased!
5
http://genn.ai/
Agenda
What is genn.ai?OverviewHow to gather clicks 0) Injecting our javascripts
6
Genn.ai core details Structure 1) Setting up filters 2) Filtering customers 3) Visualizing them
FutureDealing with historical streamsGenn.ai wants friends.
http://genn.ai/
Overview of using Genn.ai
7
CustomerA
CustomerB
CustomerC
Realtime Analysis Platform
Genn.ai
じゃらんWeb Sites
Client(Hotels)
MarketerAnalyst
http://genn.ai/
Demo videos!•They are based on a hotel reservation site like Expedia.com.
•An animation and a screencapture what the marketting staff see using Genn.ai.
At this moment ..•What prefectures are all my users searching for? (0:26)•Comparing the number of searches in today’s and yesterday’s regarding places. (2:50)
8
http://genn.ai/
Overview of using Genn.ai
9
CustomerA
CustomerB
CustomerC
Realtime Analysis Platform
Genn.ai
じゃらんWeb Sites
Client(Hotels)
MarketerAnalyst
http://genn.ai/
Overview of using Genn.ai
10
CustomerA
CustomerB
CustomerC
じゃらんWeb Sites
Realtime Analysis Platform
Genn.ai
Client(Hotels)
MarketerAnalyst
http://genn.ai/
How to gain access to Genn.ai
11
CustomerA
CustomerB
CustomerC
App(ex.Tomcat) Apache
Genn.ai will be able to be attached!
http://genn.ai/
How to gain access to Genn.ai
A custom apache module for embeding our code on returning html sources from App.
12
CustomerB
Apache
Genn.ai will be able to be attached!
mod
Genn.ai
App(ex.Tomcat)
http://genn.ai/
How to gain access to Genn.ai
13
The embedded values and js on a html.
http://genn.ai/
Structure
14
http://genn.ai/
Steps for getting resultsThere are 3 steps! (after getting clicks)
15
1 2 3
Pouring clicks into Kafka
Recording themas behaviors
Visualizing them
0gather clicks
Backend for Genn.ai
http://genn.ai/
Structure (Pouring into Kafka)
16
StormTopology
(Filter)
Kafka
kakan.puTomcatiPad(APP)
MongoDB
321
Genn.ai Front(Based on Finagle)
Querygenn.ai/web)Click Streams
Frontend for Genn.ai
Visualizer for Genn.ai
Zoomdata
Backend for Genn.ai
http://genn.ai/
Structure (Recording as behaviors)
17
StormTopology
(Filter)
Kafka
kakan.puTomcatiPad(APP)
MongoDB
Genn.ai Front(Based on Finagle)
Querygenn.ai/web)Click Streams
321
Visualizer for Genn.ai
Frontend for Genn.ai
Zoomdata
Backend for Genn.ai
Visualizer for Genn.ai
http://genn.ai/
Structure (Visualizing customers)
18
StormTopology
(Filter)
Kafka
kakan.puTomcatiPad(APP)
MongoDB
Genn.ai Front(Based on Finagle)
Querygenn.ai/web)Click Streams
321
Frontend for Genn.ai
Zoomdata
http://genn.ai/
How is Genn.ai composed?Genn.ai Frontend! : Gathering clicks.Genn.ai Backend! : Recording behaviors.Genn.ai Visualizer! : Visualizing customers’ attrs.
↑Kafka : Used for message busMongoDB : Storing Additional tablesStorm : Processing click streams
19
http://genn.ai/
Details of recording behaviors.
20
Backend for Genn.ai
http://genn.ai/
Structure (Recording behaviors)
21
StormTopology
(Filter)
Kafka
kakan.puTomcatiPad(APP)
MongoDB
Genn.ai Front(Based on Finagle)
Querygenn.ai/web)Click Streams
321
Visualizer for Genn.ai
Zoomdata
Frontend for Genn.ai
http://genn.ai/
Calculated valuesRecording behaviorsUseful values for each sitesEx.)Hotel reservation site,
22
{ "_id": 154, "largeAreaCntAcl": { "Tokyo": 3, "Berlin": 1 }, "largeAreaCntPre": { "Tokyo": 4, "Berlin": 3 }, "rateAvgAcl": 4200, "rateAvgPre": 6200, "rateCntAcl": [ 4, 1, 0, 0, 0 ], "rateCntPre": [ 4, 1, 2, 0, 0 ], "lastUpdate": "2013-04-16T04:04:37.432Z"}
http://genn.ai/
Calculated valuesRecording behaviorsUseful values for each sitesEx.)Hotel reservation site,Where is the favorite place?
23
{ "_id": 154, "largeAreaCntAcl": { "Tokyo": 3, "Berlin": 1 }, "largeAreaCntPre": { "Tokyo": 4, "Berlin": 3 }, "rateAvgAcl": 4200, "rateAvgPre": 6200, "rateCntAcl": [ 4, 1, 0, 0, 0 ], "rateCntPre": [ 4, 1, 2, 0, 0 ], "lastUpdate": "2013-04-16T04:04:37.432Z"}
http://genn.ai/
Calculated valuesRecording behaviorsUseful values for each sitesEx.)Hotel reservation site,Where is the favorite place?Where is the this searching place?
24
{ "_id": 154, "largeAreaCntAcl": { "Tokyo": 3, "Berlin": 1 }, "largeAreaCntPre": { "Tokyo": 4, "Berlin": 3 }, "rateAvgAcl": 4200, "rateAvgPre": 6200, "rateCntAcl": [ 4, 1, 0, 0, 0 ], "rateCntPre": [ 4, 1, 2, 0, 0 ], "lastUpdate": "2013-04-16T04:04:37.432Z"}
http://genn.ai/
Calculated valuesRecording behaviorsUseful values for each sitesEx.)Hotel reservation site,Where is the favorite place?Where is the this searching place?What price range has he booked?
25
{ "_id": 154, "largeAreaCntAcl": { "Tokyo": 3, "Berlin": 1 }, "largeAreaCntPre": { "Tokyo": 4, "Berlin": 3 }, "rateAvgAcl": 4200, "rateAvgPre": 6200, "rateCntAcl": [ 4, 1, 0, 0, 0 ], "rateCntPre": [ 4, 1, 2, 0, 0 ], "lastUpdate": "2013-04-16T04:04:37.432Z"}
etc..
http://genn.ai/
Functions for recording behaviorsTime shift function
Time sensitive counter (old item will be expired and deleted)
Facet ( like Solr )Categorized counter
Range Facet ( like Solr )Dynamic categorized counter
26
321
http://genn.ai/
Functions for recording behaviorsTime shift function
Time sensitive counter (old item will be expired and deleted)
Facet ( like Solr )Categorized counter
Range Facet ( like Solr )Dynamic categorized counter
27
321
http://genn.ai/
Time Shift function?We make 3 min time window and now start. ( no clicks here )
28
t
Now
http://genn.ai/
Time Shift function?In our 3 min time window, 1.5 min are captured. 2 clicks have been recorded.
29
1.5 minsNow
Time window
▲Click
▲Click
2 clicks are captured.
t
http://genn.ai/
Time Shift function?There are 3 clicks in the past.The length of the capturing window becomes max.
30
3 minsNow
Time window
▲Click
▲Click
▲Click
t
http://genn.ai/
Time Shift function?There are 4 clicks in the past.Ignore the first 2 clicks
31
▲Click
▲Click
Now
Time window
▲Click
▲Click
1.5 min 3 mins
t
http://genn.ai/
Time Shift function?There are 4 clicks in the past.Ignore the first 3 clicks
32
▲Click
▲Click
Now
Time window
▲Click
▲Click
3 mins3 mins
t
http://genn.ai/
Functions for recording behaviorsTime shift function
Time sensitive counter (old item will be expired and deleted)
Facet ( like Solr )Categorized counter
Range Facet ( like Solr )Dynamic categorized counter
33
321
http://genn.ai/
Functions for recording behaviorsTime shift function
Time sensitive counter (old item will be expired and deleted)
Facet ( like Solr )Categorized counter
Range Facet ( like Solr )Dynamic categorized counter
34
321
http://genn.ai/
Counting the frequencyThis function is under the effect of the Time shift function.
35
{ "_id": 154, "largeAreaCntAcl": { "Tokyo": 3, "Berlin": 1 }, "largeAreaCntPre": { "Tokyo": 4, "Berlin": 3 }, "rateAvgAcl": 4200, "rateAvgPre": 6200, "rateCntAcl": [ 4, 1, 0, 0, 0 ], "rateCntPre": [ 4, 1, 2, 0, 0 ], "lastUpdate": "2013-04-16T04:04:37.432Z"}
http://genn.ai/
Counting the frequencyThis function is under the effect of the Time shift function.You have booked 1 stay in Berlin and 3 stays in Tokyo in this 3 weeks.
36
▲L.A.
▲S.F.
t
Now
Time window
▲Berlin
▲Tokyo
1.5 week 3 weeks
▲Tokyo "largeAreaCntAcl": {
"Tokyo": 3, "Berlin": 1}
▲Tokyo
http://genn.ai/
Details of visualization.
37
Backend for Genn.ai
http://genn.ai/
Visualizing the customers
38
StormTopology
(Filter)
Kafka
kakan.puTomcatiPad(APP)
MongoDB
Genn.ai Front(Based on Finagle)
Querygenn.ai/web)Click Streams
321
Visualizer for Genn.ai
Backend for Genn.ai
http://genn.ai/
Visualizing the customers
39
StormTopology
(Filter)
Kafka
kakan.puTomcatiPad(APP)
MongoDB
Genn.ai Front(Based on Finagle)
Querygenn.ai/web)Click Streams
321
Visualizer for Genn.ai
http://genn.ai/
Kafka consumerKakan.puThe program for relaying clicks from Kafka to the other systems.
The enriching function (joining the MongoDB data)The converting function (for following systems)
40
Backend for Genn.ai
http://genn.ai/
Visualizing the customers
41
StormTopology
(Filter)
Kafka
kakan.puTomcatiPad(APP)
MongoDB
Genn.ai Front(Based on Finagle)
Querygenn.ai/web)Click Streams
321
Visualizer for Genn.ai
Zoomdata
http://genn.ai/
Visualizing toolZoomdataA product containing a set of server app and ipad app for visualizing event data.
42
321
http://genn.ai/
ZoomdataZoomdata has the functions like..WebAPI for streaming data.Own event processor for vizualizing.
43
321
The bar and bubble graph on default setting.Customizable visualizing templates.
http://genn.ai/
Feature plan.
44
http://genn.ai/
Genn.ai EvolutionAdding functions on Genn.ai core.Genn.ai should have filtering / aggrigation functions for federating with other systems.
Historical rewinding function will be implemented with the Hadoop cluster.
45
http://genn.ai/
Future planGenn.ai wants friends!
We are looking for co-challenging partners for tuning and improving Genn.ai.
You can use Genn.ai on your web sites for totally free (infrastructure etc..) if we could share the dream.
Genn.ai codes will be opensourced in the future.
Please feel free to contact us!http://genn.ai/
46
http://genn.ai/
Thanks!
47
48