access2011 van garderen-suhonos-part2
TRANSCRIPT
![Page 1: Access2011 van garderen-suhonos-part2](https://reader033.vdocuments.us/reader033/viewer/2022052907/558e7e6c1a28ab9a0b8b456c/html5/thumbnails/1.jpg)
BIG DATABIG DATA
![Page 2: Access2011 van garderen-suhonos-part2](https://reader033.vdocuments.us/reader033/viewer/2022052907/558e7e6c1a28ab9a0b8b456c/html5/thumbnails/2.jpg)
![Page 3: Access2011 van garderen-suhonos-part2](https://reader033.vdocuments.us/reader033/viewer/2022052907/558e7e6c1a28ab9a0b8b456c/html5/thumbnails/3.jpg)
"datasets that grow so large that they become
difficult to work with using relational
databases and within a tolerable elapsed time"
![Page 4: Access2011 van garderen-suhonos-part2](https://reader033.vdocuments.us/reader033/viewer/2022052907/558e7e6c1a28ab9a0b8b456c/html5/thumbnails/4.jpg)
BIG DATA IS BIGBIG DATA IS BIG
![Page 5: Access2011 van garderen-suhonos-part2](https://reader033.vdocuments.us/reader033/viewer/2022052907/558e7e6c1a28ab9a0b8b456c/html5/thumbnails/5.jpg)
LIKE, REALLY BIGLIKE, REALLY BIG
![Page 6: Access2011 van garderen-suhonos-part2](https://reader033.vdocuments.us/reader033/viewer/2022052907/558e7e6c1a28ab9a0b8b456c/html5/thumbnails/6.jpg)
FACEBOOK: 140 BILLION PHOTOS
HUMAN GENOME: 3 BILLIONBASE PAIRS
GOOGLE: 50 BILLIONWEB PAGES
WORLDCAT: 1.5 BILLIONITEM RECORDS
![Page 7: Access2011 van garderen-suhonos-part2](https://reader033.vdocuments.us/reader033/viewer/2022052907/558e7e6c1a28ab9a0b8b456c/html5/thumbnails/7.jpg)
NOT REALLYNOT REALLY
![Page 8: Access2011 van garderen-suhonos-part2](https://reader033.vdocuments.us/reader033/viewer/2022052907/558e7e6c1a28ab9a0b8b456c/html5/thumbnails/8.jpg)
EUROPEANA: 20 MILLION(715K / COUNTRY)
LIBRARY OF CONGRESS:
1.9 MILLION
CANADIANA: 1 MILLION
LIBRARY AND ARCHIVES CANADA:
3.5 MILLION(ARCHIVAL DESCRIPTIONS)
![Page 9: Access2011 van garderen-suhonos-part2](https://reader033.vdocuments.us/reader033/viewer/2022052907/558e7e6c1a28ab9a0b8b456c/html5/thumbnails/9.jpg)
BIG DATABIG DATAIS COMPLICATEDIS COMPLICATED
![Page 10: Access2011 van garderen-suhonos-part2](https://reader033.vdocuments.us/reader033/viewer/2022052907/558e7e6c1a28ab9a0b8b456c/html5/thumbnails/10.jpg)
1966
![Page 11: Access2011 van garderen-suhonos-part2](https://reader033.vdocuments.us/reader033/viewer/2022052907/558e7e6c1a28ab9a0b8b456c/html5/thumbnails/11.jpg)
1976
![Page 12: Access2011 van garderen-suhonos-part2](https://reader033.vdocuments.us/reader033/viewer/2022052907/558e7e6c1a28ab9a0b8b456c/html5/thumbnails/12.jpg)
≠
![Page 13: Access2011 van garderen-suhonos-part2](https://reader033.vdocuments.us/reader033/viewer/2022052907/558e7e6c1a28ab9a0b8b456c/html5/thumbnails/13.jpg)
≠
![Page 14: Access2011 van garderen-suhonos-part2](https://reader033.vdocuments.us/reader033/viewer/2022052907/558e7e6c1a28ab9a0b8b456c/html5/thumbnails/14.jpg)
NOT REALLYNOT REALLY
![Page 15: Access2011 van garderen-suhonos-part2](https://reader033.vdocuments.us/reader033/viewer/2022052907/558e7e6c1a28ab9a0b8b456c/html5/thumbnails/15.jpg)
ಠ_ಠ
![Page 16: Access2011 van garderen-suhonos-part2](https://reader033.vdocuments.us/reader033/viewer/2022052907/558e7e6c1a28ab9a0b8b456c/html5/thumbnails/16.jpg)
![Page 17: Access2011 van garderen-suhonos-part2](https://reader033.vdocuments.us/reader033/viewer/2022052907/558e7e6c1a28ab9a0b8b456c/html5/thumbnails/17.jpg)
![Page 18: Access2011 van garderen-suhonos-part2](https://reader033.vdocuments.us/reader033/viewer/2022052907/558e7e6c1a28ab9a0b8b456c/html5/thumbnails/18.jpg)
SCALABILITYSCALABILITY
![Page 19: Access2011 van garderen-suhonos-part2](https://reader033.vdocuments.us/reader033/viewer/2022052907/558e7e6c1a28ab9a0b8b456c/html5/thumbnails/19.jpg)
● ICA-AtoM (LAMP)
● BENCHMARK 3.5M RECORDS
● 100% OPEN SOURCE SOFTWARE
● COMMODITY HARDWARE
![Page 20: Access2011 van garderen-suhonos-part2](https://reader033.vdocuments.us/reader033/viewer/2022052907/558e7e6c1a28ab9a0b8b456c/html5/thumbnails/20.jpg)
![Page 21: Access2011 van garderen-suhonos-part2](https://reader033.vdocuments.us/reader033/viewer/2022052907/558e7e6c1a28ab9a0b8b456c/html5/thumbnails/21.jpg)
CAN WE DO IT?CAN WE DO IT?
![Page 22: Access2011 van garderen-suhonos-part2](https://reader033.vdocuments.us/reader033/viewer/2022052907/558e7e6c1a28ab9a0b8b456c/html5/thumbnails/22.jpg)
WRITE SPEEDWRITE SPEED
![Page 23: Access2011 van garderen-suhonos-part2](https://reader033.vdocuments.us/reader033/viewer/2022052907/558e7e6c1a28ab9a0b8b456c/html5/thumbnails/23.jpg)
READ SPEEDREAD SPEED
![Page 24: Access2011 van garderen-suhonos-part2](https://reader033.vdocuments.us/reader033/viewer/2022052907/558e7e6c1a28ab9a0b8b456c/html5/thumbnails/24.jpg)
WRITE MEMORYWRITE MEMORY
![Page 25: Access2011 van garderen-suhonos-part2](https://reader033.vdocuments.us/reader033/viewer/2022052907/558e7e6c1a28ab9a0b8b456c/html5/thumbnails/25.jpg)
READ MEMORYREAD MEMORY
![Page 26: Access2011 van garderen-suhonos-part2](https://reader033.vdocuments.us/reader033/viewer/2022052907/558e7e6c1a28ab9a0b8b456c/html5/thumbnails/26.jpg)
NOSQL vs. SQLNOSQL vs. SQL(a.k.a. ODM vs. ORM)
● 4x - 10x FASTER
● 50% - 90% LESS MEMORY
![Page 27: Access2011 van garderen-suhonos-part2](https://reader033.vdocuments.us/reader033/viewer/2022052907/558e7e6c1a28ab9a0b8b456c/html5/thumbnails/27.jpg)
![Page 28: Access2011 van garderen-suhonos-part2](https://reader033.vdocuments.us/reader033/viewer/2022052907/558e7e6c1a28ab9a0b8b456c/html5/thumbnails/28.jpg)
![Page 29: Access2011 van garderen-suhonos-part2](https://reader033.vdocuments.us/reader033/viewer/2022052907/558e7e6c1a28ab9a0b8b456c/html5/thumbnails/29.jpg)
RELATIONAL DATABASESSCALE WELL
IF YOUR DATAIS NOT HIERARCHICAL
SOLRSCALES WELL
IF YOU HAVE INFINITE RAM
BEWARE THEDOGMA OF SQL
NOSQL IS AVIABLE OPTION
THINK SIDEWAYS SCALE OUT →
![Page 30: Access2011 van garderen-suhonos-part2](https://reader033.vdocuments.us/reader033/viewer/2022052907/558e7e6c1a28ab9a0b8b456c/html5/thumbnails/30.jpg)
![Page 31: Access2011 van garderen-suhonos-part2](https://reader033.vdocuments.us/reader033/viewer/2022052907/558e7e6c1a28ab9a0b8b456c/html5/thumbnails/31.jpg)
THE CLOUD IS A LIETHE CLOUD IS A LIE
![Page 32: Access2011 van garderen-suhonos-part2](https://reader033.vdocuments.us/reader033/viewer/2022052907/558e7e6c1a28ab9a0b8b456c/html5/thumbnails/32.jpg)
“big data is less about size, and more about
freedom”
open source tools+ distributed design= new opportunities