couchdoop: connecting hadoop with couchbase
If you can't read please download the document
DESCRIPTION
This presentation was made during the Couchbase London Meetup where the following topics were presented: 1. Couchbase and Hadoop 2. Sub-millisecond response times with Couchbase Speakers: Alex Bordei- Techie Product Manager at Bigstep, Calin Burloiu- Big Data Engineer at Avira and Radu Pastia - Big Data Team Leader at Avira. Because Hadoop and any app based on this software framework requires a reliable and powerful infrastructure, Avira choose our Full Metal Cloud for their Couchdoop tests. What they say about our infrastructure: "We used bare metal instances. The Hadoop cluster had 7 worker nodes. The Couchbase cluster initially had only 2 nodes. Bigstep’s infrastructure allowed us to provision an additional Couchbase node within minutes. Each node had two 2 CPUs with a total number of 16 physical cores. This means 32 virtual cores with hyper-threading. Each node has 192 GiB and used local hard disks. The nodes were connected with 10 gigabits Ethernet connections."TRANSCRIPT
- 1. Two-tier ArchitectureReal-time Tier (Couchbase) Detects user intentGives next best recommendation or dealData Bridge (Couchdoop)Batch Tier (Hadoop) Recommends productsUser eventsRecommendations
2. Importing Data{user: Rudy,action: view,product: Fender Guitar}{user: Rudy,action: click,product: Guitar Amplifier} {user: Emma,action: buy,product: Blue Skirt}CouchdoopMachine Learning RecommendationsHadoopIMPORTHDFS 3. {user: Rudy,recommendations: [[Ibanez Acoustic Guitar,450],[Guitar Tuner, 120],[Sound Mixer, 30]]}Exporting DataCouchdoopMachine Learning RecommendationsHadoop 4. {user: Rudy,recommendations: [[Ibanez Acoustic Guitar,450],[Guitar Tuner, 120],[Sound Mixer, 30]]}Updating DataCouchdoopMachine Learning RecommendationsHadoop