megastore - id2220 presentation
DESCRIPTION
Paper presentation for Advanced Topics in Distributed System. Original paper can be found here -> http://research.google.com/pubs/pub36971.htmlTRANSCRIPT
![Page 1: Megastore - ID2220 Presentation](https://reader033.vdocuments.us/reader033/viewer/2022042601/554e5b01b4c905ad178b500c/html5/thumbnails/1.jpg)
Megastore
Providing Scalable, Highly Available Storage for Interactive Services
Paper by Jason Barker et al. Presented by Arinto Murdopo
7/11/2012 1
![Page 2: Megastore - ID2220 Presentation](https://reader033.vdocuments.us/reader033/viewer/2022042601/554e5b01b4c905ad178b500c/html5/thumbnails/2.jpg)
Outline
7/11/2012 2
Motivation Megastore:
Features Scalability Availabilty Putting them all together
Observation Conclusions
![Page 3: Megastore - ID2220 Presentation](https://reader033.vdocuments.us/reader033/viewer/2022042601/554e5b01b4c905ad178b500c/html5/thumbnails/3.jpg)
Motivation
Conflicting requirements • RDBMS – easy to use, but not scale • NoSQL – scale, but not easy to use
Interactive online services • Highly available and fast response time
7/11/2012 3
![Page 4: Megastore - ID2220 Presentation](https://reader033.vdocuments.us/reader033/viewer/2022042601/554e5b01b4c905ad178b500c/html5/thumbnails/4.jpg)
Here comes Megastore
easy to use • ACID semantics
scalable • data partitioning
highly available • synchronous replication through modified
Paxos
7/11/2012 4
![Page 5: Megastore - ID2220 Presentation](https://reader033.vdocuments.us/reader033/viewer/2022042601/554e5b01b4c905ad178b500c/html5/thumbnails/5.jpg)
Easy to use - Features
cost-transparent APIs • No API for joins • Joins are implemented in application code
data model • schema, table (entity), property • entity clustering • indexes: local, global • Bigtable column name == Megastore table
name and property name, i.e User.name
7/11/2012 5
![Page 6: Megastore - ID2220 Presentation](https://reader033.vdocuments.us/reader033/viewer/2022042601/554e5b01b4c905ad178b500c/html5/thumbnails/6.jpg)
Easy to use - Features
transactions and concurrency control • Bigtable for concurrency control • transaction lifecycle: read, application logic,
commit, apply, clean up
others • backup system of transaction logs • encryption
7/11/2012 6
![Page 7: Megastore - ID2220 Presentation](https://reader033.vdocuments.us/reader033/viewer/2022042601/554e5b01b4c905ad178b500c/html5/thumbnails/7.jpg)
Scalable
Scale the replication scheme Data partitioning
• Entity group concept
Data locality
• Entity group locality • Bigtable instances locality
7/11/2012 7
![Page 8: Megastore - ID2220 Presentation](https://reader033.vdocuments.us/reader033/viewer/2022042601/554e5b01b4c905ad178b500c/html5/thumbnails/8.jpg)
Entity Groups
7/11/2012 8
Entity is like instance of table. Entity group is group of entities. i.e
Email Application • Email account
Blog Application • User Profile • Blog post + metadata • Blog unique name
![Page 9: Megastore - ID2220 Presentation](https://reader033.vdocuments.us/reader033/viewer/2022042601/554e5b01b4c905ad178b500c/html5/thumbnails/9.jpg)
Entity Groups
7/11/2012 9
![Page 10: Megastore - ID2220 Presentation](https://reader033.vdocuments.us/reader033/viewer/2022042601/554e5b01b4c905ad178b500c/html5/thumbnails/10.jpg)
Highly Available
7/11/2012 10
Replicate mutations of write-ahead log inside entity groups using modified Paxos, but let’s revisit original Paxos…
![Page 11: Megastore - ID2220 Presentation](https://reader033.vdocuments.us/reader033/viewer/2022042601/554e5b01b4c905ad178b500c/html5/thumbnails/11.jpg)
Modified Paxos – Fast Reads
7/11/2012 11
Read in original Paxos
![Page 12: Megastore - ID2220 Presentation](https://reader033.vdocuments.us/reader033/viewer/2022042601/554e5b01b4c905ad178b500c/html5/thumbnails/12.jpg)
Modified Paxos – Fast Reads
7/11/2012 12
Contact Coordinator and read locally if possible
![Page 13: Megastore - ID2220 Presentation](https://reader033.vdocuments.us/reader033/viewer/2022042601/554e5b01b4c905ad178b500c/html5/thumbnails/13.jpg)
Modified Paxos – Fast Writes
7/11/2012 13
Skip “prepare” stage in subsequent write of same leader, provided no write from other writers
![Page 14: Megastore - ID2220 Presentation](https://reader033.vdocuments.us/reader033/viewer/2022042601/554e5b01b4c905ad178b500c/html5/thumbnails/14.jpg)
Modified Paxos – New Replica Types
7/11/2012 14
Full Replicas all replicas that we have seen until now
Witness Replicas
are able to vote store but do not apply write-ahead logs do not store entity data
Read-only Replicas
are not able to vote snapshots of entity data
![Page 15: Megastore - ID2220 Presentation](https://reader033.vdocuments.us/reader033/viewer/2022042601/554e5b01b4c905ad178b500c/html5/thumbnails/15.jpg)
Putting them all together
7/11/2012 15
Megastore Architecture
![Page 16: Megastore - ID2220 Presentation](https://reader033.vdocuments.us/reader033/viewer/2022042601/554e5b01b4c905ad178b500c/html5/thumbnails/16.jpg)
Reads
7/11/2012 16
Query Local
Find Position
Catchup
Validate
Query Data
![Page 17: Megastore - ID2220 Presentation](https://reader033.vdocuments.us/reader033/viewer/2022042601/554e5b01b4c905ad178b500c/html5/thumbnails/17.jpg)
Writes
7/11/2012 17
Accept Leader
Prepare
Accept
Invalidate
Apply
![Page 18: Megastore - ID2220 Presentation](https://reader033.vdocuments.us/reader033/viewer/2022042601/554e5b01b4c905ad178b500c/html5/thumbnails/18.jpg)
Observation - Availability
7/11/2012 18
![Page 19: Megastore - ID2220 Presentation](https://reader033.vdocuments.us/reader033/viewer/2022042601/554e5b01b4c905ad178b500c/html5/thumbnails/19.jpg)
Observation – Latency
7/11/2012 19
![Page 20: Megastore - ID2220 Presentation](https://reader033.vdocuments.us/reader033/viewer/2022042601/554e5b01b4c905ad178b500c/html5/thumbnails/20.jpg)
Conclusion
7/11/2012 20
Megastore and its motivation
Features of megastore • It has ACID semantics • But need to define entity groups • Need to handle inter-group updates
Scalability and Availability
More experiments are needed