scalable web softwareweb.stanford.edu/class/cs193s/cs193s-1-07.pdf · the web is just right •...
TRANSCRIPT
![Page 1: Scalable Web Softwareweb.stanford.edu/class/cs193s/CS193S-1-07.pdf · The Web is Just Right • Many basic computer science ... • They are so called because in a log/log plot they](https://reader033.vdocuments.us/reader033/viewer/2022050206/5f594dca755330579c67df08/html5/thumbnails/1.jpg)
Scalable Web SoftwareCS193S - Jan Jannink - 1/07/10
![Page 2: Scalable Web Softwareweb.stanford.edu/class/cs193s/CS193S-1-07.pdf · The Web is Just Right • Many basic computer science ... • They are so called because in a log/log plot they](https://reader033.vdocuments.us/reader033/viewer/2022050206/5f594dca755330579c67df08/html5/thumbnails/2.jpg)
Administrative Stuff• Computer Forum Career Fair: Wed. 13,
11-4
• Lawn between Hewlett Teaching Center and Gilbert Building
• Looking forward to your emails! Already received a dozen or so
• Any problems dealing with eclipse/gwt setup?
![Page 3: Scalable Web Softwareweb.stanford.edu/class/cs193s/CS193S-1-07.pdf · The Web is Just Right • Many basic computer science ... • They are so called because in a log/log plot they](https://reader033.vdocuments.us/reader033/viewer/2022050206/5f594dca755330579c67df08/html5/thumbnails/3.jpg)
Weekly Syllabus1. Scalability: (Jan.)
2. Agile Practices
3. Ecology/Mashups*
4. Browser/Client
5. Data/Server: (Feb.)
6. Security/Privacy
7. Analytics*
8. Cloud/Map-Reduce
9. Publish APIs: (Mar.)*
10. Future
* assignment due
![Page 4: Scalable Web Softwareweb.stanford.edu/class/cs193s/CS193S-1-07.pdf · The Web is Just Right • Many basic computer science ... • They are so called because in a log/log plot they](https://reader033.vdocuments.us/reader033/viewer/2022050206/5f594dca755330579c67df08/html5/thumbnails/4.jpg)
Quick Review• Think Big
• Scalability means thriving and growing in a dynamic environment
• Java + open source: a great environment to learn scalable practices
• Engineers & investors understand innovation differently
![Page 5: Scalable Web Softwareweb.stanford.edu/class/cs193s/CS193S-1-07.pdf · The Web is Just Right • Many basic computer science ... • They are so called because in a log/log plot they](https://reader033.vdocuments.us/reader033/viewer/2022050206/5f594dca755330579c67df08/html5/thumbnails/5.jpg)
Scale Fail
![Page 6: Scalable Web Softwareweb.stanford.edu/class/cs193s/CS193S-1-07.pdf · The Web is Just Right • Many basic computer science ... • They are so called because in a log/log plot they](https://reader033.vdocuments.us/reader033/viewer/2022050206/5f594dca755330579c67df08/html5/thumbnails/6.jpg)
Scale Fail
how do you get change for this?
![Page 7: Scalable Web Softwareweb.stanford.edu/class/cs193s/CS193S-1-07.pdf · The Web is Just Right • Many basic computer science ... • They are so called because in a log/log plot they](https://reader033.vdocuments.us/reader033/viewer/2022050206/5f594dca755330579c67df08/html5/thumbnails/7.jpg)
The Web is Just Right
• Many basic computer science algorithms are well understood
• Complexity theory defines practically unsolvable problems
• Scalable web coding fits in the border region between basic and complex
![Page 8: Scalable Web Softwareweb.stanford.edu/class/cs193s/CS193S-1-07.pdf · The Web is Just Right • Many basic computer science ... • They are so called because in a log/log plot they](https://reader033.vdocuments.us/reader033/viewer/2022050206/5f594dca755330579c67df08/html5/thumbnails/8.jpg)
Complexity Classes
![Page 9: Scalable Web Softwareweb.stanford.edu/class/cs193s/CS193S-1-07.pdf · The Web is Just Right • Many basic computer science ... • They are so called because in a log/log plot they](https://reader033.vdocuments.us/reader033/viewer/2022050206/5f594dca755330579c67df08/html5/thumbnails/9.jpg)
YouTube Example• Piggybacked growth on MySpace.com
• Previously unheard of bandwidth
• Growth delayed feature development
• Could not afford to continue to grow on its own
• Acquisition significantly delayed revenue model development
![Page 10: Scalable Web Softwareweb.stanford.edu/class/cs193s/CS193S-1-07.pdf · The Web is Just Right • Many basic computer science ... • They are so called because in a log/log plot they](https://reader033.vdocuments.us/reader033/viewer/2022050206/5f594dca755330579c67df08/html5/thumbnails/10.jpg)
Properties of Web Data• Quickly, iteratively, accumulated by
contributors, seen by many viewers
• New additions are not independent of prior ones
• The dependency induces several power law distributions over the data
• Frequently called ‘Long Tail data’
![Page 11: Scalable Web Softwareweb.stanford.edu/class/cs193s/CS193S-1-07.pdf · The Web is Just Right • Many basic computer science ... • They are so called because in a log/log plot they](https://reader033.vdocuments.us/reader033/viewer/2022050206/5f594dca755330579c67df08/html5/thumbnails/11.jpg)
Power Law Distributions
• Every aggregated collection of human artifacts seems expressible in this way
• They are so called because in a log/log plot they generally follow a straight line
• the slope of the line s corresponds to a power coefficient: i.e., y = x-s
![Page 12: Scalable Web Softwareweb.stanford.edu/class/cs193s/CS193S-1-07.pdf · The Web is Just Right • Many basic computer science ... • They are so called because in a log/log plot they](https://reader033.vdocuments.us/reader033/viewer/2022050206/5f594dca755330579c67df08/html5/thumbnails/12.jpg)
Web Traffic Referral Plot
![Page 13: Scalable Web Softwareweb.stanford.edu/class/cs193s/CS193S-1-07.pdf · The Web is Just Right • Many basic computer science ... • They are so called because in a log/log plot they](https://reader033.vdocuments.us/reader033/viewer/2022050206/5f594dca755330579c67df08/html5/thumbnails/13.jpg)
Why This is Relevant
• Back end infrastructure
• Database partitioning
• Data replication
• Caching technique
• Web page design
![Page 14: Scalable Web Softwareweb.stanford.edu/class/cs193s/CS193S-1-07.pdf · The Web is Just Right • Many basic computer science ... • They are so called because in a log/log plot they](https://reader033.vdocuments.us/reader033/viewer/2022050206/5f594dca755330579c67df08/html5/thumbnails/14.jpg)
Other Examples
• Human Languages (word usage)
• Social Networks (connections)
• Financial Markets (transactions)
• Research Citations
• Called them Nexus in my PhD research
![Page 15: Scalable Web Softwareweb.stanford.edu/class/cs193s/CS193S-1-07.pdf · The Web is Just Right • Many basic computer science ... • They are so called because in a log/log plot they](https://reader033.vdocuments.us/reader033/viewer/2022050206/5f594dca755330579c67df08/html5/thumbnails/15.jpg)
Power Law Generators• item creation
• object in a universe
• link creation
• any relationship between items
• new link probability based on existing structure
• item & link removal
![Page 16: Scalable Web Softwareweb.stanford.edu/class/cs193s/CS193S-1-07.pdf · The Web is Just Right • Many basic computer science ... • They are so called because in a log/log plot they](https://reader033.vdocuments.us/reader033/viewer/2022050206/5f594dca755330579c67df08/html5/thumbnails/16.jpg)
Stuff to Ponder
![Page 17: Scalable Web Softwareweb.stanford.edu/class/cs193s/CS193S-1-07.pdf · The Web is Just Right • Many basic computer science ... • They are so called because in a log/log plot they](https://reader033.vdocuments.us/reader033/viewer/2022050206/5f594dca755330579c67df08/html5/thumbnails/17.jpg)
Dynamic Environments• Habitats/Ecosystems
• species surviving to occupy a niche
• Organizations/Societies
• unifying force and leadership
• Markets/Economies
• producer consumer relationship
![Page 18: Scalable Web Softwareweb.stanford.edu/class/cs193s/CS193S-1-07.pdf · The Web is Just Right • Many basic computer science ... • They are so called because in a log/log plot they](https://reader033.vdocuments.us/reader033/viewer/2022050206/5f594dca755330579c67df08/html5/thumbnails/18.jpg)
Back to Software• So many technologies, so little time
• Start with a single platform, language
• Java/gwt/junit
• Single developer code/test cycle
• code/checkin/test/repeat
• We’ll continue today with SQL
![Page 19: Scalable Web Softwareweb.stanford.edu/class/cs193s/CS193S-1-07.pdf · The Web is Just Right • Many basic computer science ... • They are so called because in a log/log plot they](https://reader033.vdocuments.us/reader033/viewer/2022050206/5f594dca755330579c67df08/html5/thumbnails/19.jpg)
MySQL Install• download from website & install, download & unzip popnames.zip
• minor Mac OS X command line edit
• sudo echo /usr/local/mysql/bin > /etc/paths.d/mysql
• mysql < popnames.txt
• select name, count(name), sum(count) from popnames group by name order by count(name) desc limit 20;
• select gender, name, sum(count) as total from popnames where year(year)>1958 group by name, gender order by total desc limit 20;
• select left(name,1) as i, sum(count) from popnames group by i;
![Page 20: Scalable Web Softwareweb.stanford.edu/class/cs193s/CS193S-1-07.pdf · The Web is Just Right • Many basic computer science ... • They are so called because in a log/log plot they](https://reader033.vdocuments.us/reader033/viewer/2022050206/5f594dca755330579c67df08/html5/thumbnails/20.jpg)
Untaught MySQL Tools• mysqlimport
• mysqldump
• workbench
• connectors
• in mysql client
• data analysis : group by, order by
![Page 21: Scalable Web Softwareweb.stanford.edu/class/cs193s/CS193S-1-07.pdf · The Web is Just Right • Many basic computer science ... • They are so called because in a log/log plot they](https://reader033.vdocuments.us/reader033/viewer/2022050206/5f594dca755330579c67df08/html5/thumbnails/21.jpg)
Popular Names Dataset
• Source data:• http://www.ssa.gov/OACT/babynames/
• Command line tools extract data from html tables, into tab separated text files• curl, sed, awk, python
• Import into MySQL• load data local infile “” into table ... fields
terminated by ‘’ (@A, @B, @C) set ...;
![Page 22: Scalable Web Softwareweb.stanford.edu/class/cs193s/CS193S-1-07.pdf · The Web is Just Right • Many basic computer science ... • They are so called because in a log/log plot they](https://reader033.vdocuments.us/reader033/viewer/2022050206/5f594dca755330579c67df08/html5/thumbnails/22.jpg)
User Account Table• name
• hashed password
• identifier (email)
• current email
• payment info
• (secure stuff)
• everything else (potentially public data) should be kept separate
![Page 23: Scalable Web Softwareweb.stanford.edu/class/cs193s/CS193S-1-07.pdf · The Web is Just Right • Many basic computer science ... • They are so called because in a log/log plot they](https://reader033.vdocuments.us/reader033/viewer/2022050206/5f594dca755330579c67df08/html5/thumbnails/23.jpg)
Q & A Topics
• Nexus research
• Database vs. flat file
• Hibernate vs. ODBC access layer
![Page 24: Scalable Web Softwareweb.stanford.edu/class/cs193s/CS193S-1-07.pdf · The Web is Just Right • Many basic computer science ... • They are so called because in a log/log plot they](https://reader033.vdocuments.us/reader033/viewer/2022050206/5f594dca755330579c67df08/html5/thumbnails/24.jpg)
Worth Checking Out
• MySQL downloads• http://dev.mysql.com/downloads/
• The Goldilocks Enigma
• Paul Davies
• SQLite• http://www.sqlite.org/