the evolution of data science
TRANSCRIPT
![Page 1: The Evolution of Data Science](https://reader033.vdocuments.us/reader033/viewer/2022052401/55d08edcbb61eb9b748b4645/html5/thumbnails/1.jpg)
The Evolution of Data ScienceKenny DanielCTO, Algorithmia
July 24, 2015
![Page 2: The Evolution of Data Science](https://reader033.vdocuments.us/reader033/viewer/2022052401/55d08edcbb61eb9b748b4645/html5/thumbnails/2.jpg)
Kenny Daniel - CTO, Algorithmia• Graduate research in Artificial Intelligence and Mechanism Design• Multiple published algorithms and papers in Machine Learning• Received $1 million from DOT “Engineering Tomorrow’s Transportation Market”• B.S. Carnegie Mellon University, M.S., Ph.D. (on leave) USC• Data Scientist and Computer Vision specialist for Delectable, Inc• Initial and current overall architect of Algorithmia Platform
![Page 3: The Evolution of Data Science](https://reader033.vdocuments.us/reader033/viewer/2022052401/55d08edcbb61eb9b748b4645/html5/thumbnails/3.jpg)
Make state-of-the-art algorithms
accessible and discoverable by
everyone.
![Page 4: The Evolution of Data Science](https://reader033.vdocuments.us/reader033/viewer/2022052401/55d08edcbb61eb9b748b4645/html5/thumbnails/4.jpg)
Evolution of Data Science
● History of data science
● Modern data science
● Future speculation
![Page 5: The Evolution of Data Science](https://reader033.vdocuments.us/reader033/viewer/2022052401/55d08edcbb61eb9b748b4645/html5/thumbnails/5.jpg)
Pre-cloud● Mainframes● Universities● Research Facilities● Finance● PhD researchers, highly specialized
More pre-planning, less exploratory
![Page 6: The Evolution of Data Science](https://reader033.vdocuments.us/reader033/viewer/2022052401/55d08edcbb61eb9b748b4645/html5/thumbnails/6.jpg)
Source and Inspiration: http://www.slideshare.net/AlbertWenger/the-no-stackstartup
1990s Connectivity$10,000 per month
Servers$20,000 per box
Storage$1,000/GB
2000s Connectivity$1,000 per month
Servers$1,000 per box
Storage$10/GB
2010s Connectivity10 cents/GB
Servers20 cents/hour
Storage12 cents/GB
NOW Backend using ParseSearch using AlgoliaSynchronization using FirebaseVideo calls and SMS using TwilioPayments using StripeVideo recording using ZiggeoSend and track emails using MailgunCustomer service using IntercomShip product using Shyp
![Page 7: The Evolution of Data Science](https://reader033.vdocuments.us/reader033/viewer/2022052401/55d08edcbb61eb9b748b4645/html5/thumbnails/7.jpg)
“no one got fired for using AWS”cost, security, convenience
![Page 8: The Evolution of Data Science](https://reader033.vdocuments.us/reader033/viewer/2022052401/55d08edcbb61eb9b748b4645/html5/thumbnails/8.jpg)
“We used to leak memory.
Now we leak instances.
Soon we will leak entire data centers.”
- Dan Kaminsky
![Page 9: The Evolution of Data Science](https://reader033.vdocuments.us/reader033/viewer/2022052401/55d08edcbb61eb9b748b4645/html5/thumbnails/9.jpg)
Previously, data analysis was done by domain experts
Now, shift toward data science as its own field
A new field is born
![Page 10: The Evolution of Data Science](https://reader033.vdocuments.us/reader033/viewer/2022052401/55d08edcbb61eb9b748b4645/html5/thumbnails/10.jpg)
![Page 11: The Evolution of Data Science](https://reader033.vdocuments.us/reader033/viewer/2022052401/55d08edcbb61eb9b748b4645/html5/thumbnails/11.jpg)
“Hi, I’m a Data Scientist”
![Page 12: The Evolution of Data Science](https://reader033.vdocuments.us/reader033/viewer/2022052401/55d08edcbb61eb9b748b4645/html5/thumbnails/12.jpg)
Lots of Data
Little Intelligence
![Page 13: The Evolution of Data Science](https://reader033.vdocuments.us/reader033/viewer/2022052401/55d08edcbb61eb9b748b4645/html5/thumbnails/13.jpg)
“Data is inherently dumb. It doesn’t actually do anything unless
you know how to use it...
The next digital gold rush will be focused on how you do
something with data.”
- Peter Sondergaard (Gartner Research)
![Page 14: The Evolution of Data Science](https://reader033.vdocuments.us/reader033/viewer/2022052401/55d08edcbb61eb9b748b4645/html5/thumbnails/14.jpg)
1990s TechnologyHPC, Mainframes
2000s
2010s
NOW Generalist Big Data such as Amazon EMRLarge Data Processing such as DatabricksReal Time Processing such as Amazon KinesisData Repositories such as SocrataData Collectors such as KimonoDSaaS for Customer Analytics such as CaptricityDSaaS for Marketing such as AcxiomDSaaS for Security such as FortscaleHosted Machine Learning such as BigML, DatoAlgorithms-as-a-Service such as Algorithmia
TechnologyIn-house clusters
TechnologiesCloud, Hadoop, Spark
UsersCorporations, tech startups
UsersIndividual data scientists
UsersResearchers, hw engineers, committees
![Page 15: The Evolution of Data Science](https://reader033.vdocuments.us/reader033/viewer/2022052401/55d08edcbb61eb9b748b4645/html5/thumbnails/15.jpg)
Behold...
Data Sciencein a Spreadsheet
![Page 16: The Evolution of Data Science](https://reader033.vdocuments.us/reader033/viewer/2022052401/55d08edcbb61eb9b748b4645/html5/thumbnails/16.jpg)
Future of Data Science● How will these trends continue?
● What will future tools look like?
● What is the role of data scientists going forward?
![Page 17: The Evolution of Data Science](https://reader033.vdocuments.us/reader033/viewer/2022052401/55d08edcbb61eb9b748b4645/html5/thumbnails/17.jpg)
Data is less structured, and less amenable to traditional
data analysis without pre-processing
● Unstructured text
● Images
● Video
Future… new data sources
![Page 18: The Evolution of Data Science](https://reader033.vdocuments.us/reader033/viewer/2022052401/55d08edcbb61eb9b748b4645/html5/thumbnails/18.jpg)
Future… building blocks
Topic Analysis
Twitter Youtube Satellite Imagery
Computer Vision
Artificial Neural Networks
![Page 19: The Evolution of Data Science](https://reader033.vdocuments.us/reader033/viewer/2022052401/55d08edcbb61eb9b748b4645/html5/thumbnails/19.jpg)
Future… more autonomous
AutoMLEnsemble learningHyperparameter optimization
![Page 20: The Evolution of Data Science](https://reader033.vdocuments.us/reader033/viewer/2022052401/55d08edcbb61eb9b748b4645/html5/thumbnails/20.jpg)
JOIN: algorithmia.com/signup?invite=SeattleDS(will post to meetup group)
REACH OUT: [email protected]