predicting popular areas of a tiled web map as a strategy for server-side caching sterling quinn
TRANSCRIPT
Predicting popular areas of a Predicting popular areas of a tiled Web map as a strategy tiled Web map as a strategy
for server-side cachingfor server-side caching
Sterling QuinnSterling Quinn
IntroductionIntroduction
This project presents a predictive model This project presents a predictive model for popular areas of a Web mapfor popular areas of a Web map
Model output indicates where server-side Model output indicates where server-side cache of map tiles should be createdcache of map tiles should be created
Selectively caching based on popular map Selectively caching based on popular map areas can save time and disk spaceareas can save time and disk space
Project objectivesProject objectives
Describe server-side caching of map tilesDescribe server-side caching of map tiles
Describe the need for selective cachingDescribe the need for selective caching
Present a predictive model for popular areas of Present a predictive model for popular areas of the mapthe map
Describe ways the model could be used and Describe ways the model could be used and evaluatedevaluated
Web map optimization and the Web map optimization and the advent of server-side cachingadvent of server-side caching
Organizing large maps in Organizing large maps in manageable “tiles” is not newmanageable “tiles” is not new
Large paper map Large paper map series are indexed in series are indexed in organized gridsorganized grids
CGIS, a pioneering CGIS, a pioneering GIS, used “frames” to GIS, used “frames” to organize data (right)organize data (right)
From Tomlinson, Calkins, & Marble, 1976, p. 56.
Other techniques for organizing Other techniques for organizing maps in tiles or grid systemsmaps in tiles or grid systems
Pyramid technique Pyramid technique successively successively generalizes rasters in generalizes rasters in groups of four cells groups of four cells (right)(right)
Quadtree structures Quadtree structures index datasets in a index datasets in a hierarchy of hierarchy of quadrantsquadrants
From De Cola & Montagne, 1993, p. 1394.
Server-side caching of map Server-side caching of map tiles is newtiles is new
Tiled maps allow users to retrieve just the Tiled maps allow users to retrieve just the needed pieces of the mapneeded pieces of the map
Cached map tiles give extremely fast Cached map tiles give extremely fast performanceperformance
Traditional map servers (ArcIMS, WMS) draw Traditional map servers (ArcIMS, WMS) draw the image on the flythe image on the fly
Early static map servers returned the entire map Early static map servers returned the entire map at onceat once
Advent of tiled maps and server-Advent of tiled maps and server-side cachingside caching
Microsoft Terra Server an early Microsoft Terra Server an early deployment of massive amounts of cached deployment of massive amounts of cached imagery tilesimagery tiles
Google Maps serves cached map tiles Google Maps serves cached map tiles with AJAX techniques to create a with AJAX techniques to create a “seamless” Web mapping experience“seamless” Web mapping experience
Many sites have followed Google’s patternMany sites have followed Google’s pattern
Caching optionsCaching options
Current caching optionsCurrent caching options
Current GIS software allows analysts to Current GIS software allows analysts to create tile caches for their own mapscreate tile caches for their own maps ESRI’s ArcGIS ServerESRI’s ArcGIS Server MapnikMapnik Microsoft MapCruncherMicrosoft MapCruncher
Caching can require enormous Caching can require enormous resources on the serverresources on the server
Caches covering big areas at large scales Caches covering big areas at large scales can include millions of tilescan include millions of tiles Many gigabytes, or even terabytes of storageMany gigabytes, or even terabytes of storage Days, weeks, or sometimes months to Days, weeks, or sometimes months to
generategenerate
Many GIS shops lack resources to Many GIS shops lack resources to maintain large cachesmaintain large caches
Selective caching as a strategy for Selective caching as a strategy for saving resourcessaving resources
Administrator can cache only the areas Administrator can cache only the areas anticipated to be most visitedanticipated to be most visited
Remaining areas can be:Remaining areas can be: Added to the cache “on-demand” when first Added to the cache “on-demand” when first
user navigates thereuser navigates there Covered with a “Data not available” tileCovered with a “Data not available” tile Left blankLeft blank
Implications of selective cachingImplications of selective caching
Wise because some tiles (ocean, desert) Wise because some tiles (ocean, desert) will rarely, if never, be accessedwill rarely, if never, be accessed
Requires an admission that some areas Requires an admission that some areas are more important than othersare more important than others
Poses challenge of predicting popular Poses challenge of predicting popular areas before the map is released.areas before the map is released.
The need for a predictive modelThe need for a predictive model
Project presents a predictive model Project presents a predictive model for where to pre-cache tilesfor where to pre-cache tiles
““Which places are most interesting?”Which places are most interesting?”
Inputs are datasets readily available to Inputs are datasets readily available to GIS analystGIS analyst
Output vector features are a template for Output vector features are a template for where to pre-cache tileswhere to pre-cache tiles
Purpose of the modelPurpose of the model
Help majority of users see a fast Web map Help majority of users see a fast Web map while minimizing cache creation time and while minimizing cache creation time and storage spacestorage space
Not a descriptive modelNot a descriptive model
Descriptive model would Descriptive model would show where existing show where existing users have already users have already viewedviewed
Microsoft Hotmap a good a good example of a descriptive example of a descriptive tool (right)tool (right)
Descriptive models are Descriptive models are useful in deriving useful in deriving predictive modelspredictive models
Source: Microsoft Hotmaphttp://hotmap.msresearch.us
Advantages of a predictive modelAdvantages of a predictive model
Doesn’t require the map to be deployed Doesn’t require the map to be deployed alreadyalready
Can include fixed and varying geographic Can include fixed and varying geographic phenomenaphenomena
Has applications far beyond map cachingHas applications far beyond map caching
Proposed methodsProposed methods
Study area and conditionsStudy area and conditions
Model will predict popular places for a Model will predict popular places for a general base mapgeneral base map
Study area of CaliforniaStudy area of California
May create models for thematic maps if May create models for thematic maps if time allowstime allows
Input datasetsInput datasets
Populated / developed areasPopulated / developed areas
Road networksRoad networks
CoastlinesCoastlines
Points of interestPoints of interest
Populated / developed areasPopulated / developed areas
Human Influence Human Influence Index grid created by Index grid created by the Socioeconomic the Socioeconomic Data and Applications Data and Applications Center (SEDAC) at Center (SEDAC) at Columbia UniversityColumbia University
Model selects all grid Model selects all grid cells over a certain cells over a certain valuevalue
Road networksRoad networks
Major roads buffered by a given distanceMajor roads buffered by a given distance
AllAll roads within national parks, roads within national parks, monuments, historical sites, and monuments, historical sites, and recreation areas, buffered by a given recreation areas, buffered by a given distancedistance
CoastlinesCoastlines
All coastlines buffered by a given distance All coastlines buffered by a given distance (wider buffer on inland side)(wider buffer on inland side)
Points of interestPoints of interest
Set of 50 interesting points chosen by model Set of 50 interesting points chosen by model authorauthor Mountain peaksMountain peaks Theme parksTheme parks Sports arenasSports arenas Etc.Etc.
Represents a flexible layer that could be tailored Represents a flexible layer that could be tailored to local needsto local needs
Deriving the outputDeriving the output
Merge all layers togetherMerge all layers together
Clip to California outline buffered by ½ Clip to California outline buffered by ½ milemile
Remove small holes and polygonsRemove small holes and polygons
Dissolve into one multipart featureDissolve into one multipart feature
Using the model outputUsing the model output
Output is a vector dataset that can be used as a Output is a vector dataset that can be used as a template for creating cached tilestemplate for creating cached tiles
Compare model output area with total area to Compare model output area with total area to understand percent coverageunderstand percent coverage
Compare model output with actual usage over Compare model output with actual usage over timetime
Refine if necessaryRefine if necessary
LimitationsLimitations
Models on a world or country level should Models on a world or country level should account for Internet connectivityaccount for Internet connectivity
Input datasets have varying collection datesInput datasets have varying collection dates
Input datasets vary in resolution and precisionInput datasets vary in resolution and precision
Maps with many scales might require multiple Maps with many scales might require multiple iterations and variations of the modeliterations and variations of the model
Questions?Questions?