r the ultimate crawl creating a tour using web services and a sprinkling of r code
TRANSCRIPT
![Page 1: R The Ultimate Crawl Creating a tour using web services and a sprinkling of R code](https://reader036.vdocuments.us/reader036/viewer/2022062718/56649e735503460f94b726cd/html5/thumbnails/1.jpg)
R
The Ultimate CrawlCreating a tour using web services and a sprinkling of R code
![Page 2: R The Ultimate Crawl Creating a tour using web services and a sprinkling of R code](https://reader036.vdocuments.us/reader036/viewer/2022062718/56649e735503460f94b726cd/html5/thumbnails/2.jpg)
R Lee Hawthorn, Acma, CGMA, Ba(Hons)› Many roles over the years
– Software engineer– Management Accountant– Data Analyst
› Motivated to learn and share, to solve problems
› Current : Data Analyst @ Payzone UK
› uk.linkedin.com/in/leehawthorn/
› @lee_hawthorn
› Leehbi.com
![Page 3: R The Ultimate Crawl Creating a tour using web services and a sprinkling of R code](https://reader036.vdocuments.us/reader036/viewer/2022062718/56649e735503460f94b726cd/html5/thumbnails/3.jpg)
R Agenda
› 1. Setting the stage
› 2. Plan of action
› 3. Code walkthrough
› 4. Conclusion
![Page 4: R The Ultimate Crawl Creating a tour using web services and a sprinkling of R code](https://reader036.vdocuments.us/reader036/viewer/2022062718/56649e735503460f94b726cd/html5/thumbnails/4.jpg)
R The data revolution
› Traditional
› On premise data held within databases or data-warehouse– BI tools, Excel, MDX, SQL, Web reporting– Basic queries with a few calculations
› The future
› SaaS & PaaS is moving data to the cloud– Services & Data delivered over API’s– Restful, JSON, XML– With R & other modern tools we have the power to solve hard
problems by consuming services.
SaaS/PaaSUnstructure
d Data
OLTP / OLAP
![Page 5: R The Ultimate Crawl Creating a tour using web services and a sprinkling of R code](https://reader036.vdocuments.us/reader036/viewer/2022062718/56649e735503460f94b726cd/html5/thumbnails/5.jpg)
R Touring around Chester
![Page 6: R The Ultimate Crawl Creating a tour using web services and a sprinkling of R code](https://reader036.vdocuments.us/reader036/viewer/2022062718/56649e735503460f94b726cd/html5/thumbnails/6.jpg)
R Fail to plan, plan to fail
Venues
RouteDistance
TSP
Map
Yelp.com
Google Maps
TSP package
GGMAP
GGMAP
![Page 7: R The Ultimate Crawl Creating a tour using web services and a sprinkling of R code](https://reader036.vdocuments.us/reader036/viewer/2022062718/56649e735503460f94b726cd/html5/thumbnails/7.jpg)
R Code Walkthrough
› The code can be found on Rpub
› http://rpubs.com/leehbi/tour
VenueTour.html
![Page 8: R The Ultimate Crawl Creating a tour using web services and a sprinkling of R code](https://reader036.vdocuments.us/reader036/viewer/2022062718/56649e735503460f94b726cd/html5/thumbnails/8.jpg)
R Next Steps
› Refactor - eliminate overlap in data.
› Add tests (sorry Chris)
› Add Start position into TSP Route function
› Explore deployment to Shiny (Coursera – March)
› Investigate Concorde or some other brute-force algorithm
![Page 9: R The Ultimate Crawl Creating a tour using web services and a sprinkling of R code](https://reader036.vdocuments.us/reader036/viewer/2022062718/56649e735503460f94b726cd/html5/thumbnails/9.jpg)
R Key Learning
› JSON is much easier to work with than XML
› Oauth lets us authenticate with keys…after coding this up I discovered the ROauth package
› Still tempted to write For Loops but no need with R
› Think of GGMAP like a multi-layered photoshop image
![Page 10: R The Ultimate Crawl Creating a tour using web services and a sprinkling of R code](https://reader036.vdocuments.us/reader036/viewer/2022062718/56649e735503460f94b726cd/html5/thumbnails/10.jpg)
R