towards the wikipedia of world wide sensors
TRANSCRIPT
Jie Liu
Principal Researcher Microsoft Research Redmond, WA 98052
Towards the Wikipedia of World Wide Sensors
With thanks to Yan Xu, Suman Nath, Aman Kansal, Heitor Ramos, and Qiang Wang
Computing in the Real World
Energy
Climate Change
Environment
Homeland Security
Disaster Response
Critical Infrastructure
Transportation
Asset Management
Healthcare
Assisted Living
...
Experimental, theoretical, and computational, and data-driven science.
4th Paradigm of Scientific Discovery
Microsoft Sensing Research
With applications in
• Environmental monitoring
• Data center operation and energy management
• Mobile computing
Collaboration Collection Extraction
S
N
Atlantic Rainforest Micrometeorology Sensor Network in Brazil (University of São Paulo, Microsoft Research, Johns Hopkins University)
(Images courtesy of Humberto Rocha, Rob Fatland, and Andreas Terzis)
Towers
50meters
Serra do Mar
SwissEx
Put all data together for better understanding Share data with other scientists
Snow Soil Streams Temperature Humidity
• Sensor networking
• Energy management
• Data yield improvements
• Deployment strategy
• Data management
• Data interoperability
• Data archival
• Sensor tasking
• Data visualization
• Temporal-spatial indexing
• Online aggregation and representation
Key Technical Challenges
Participatory Environmental Monitoring Toolkit
Objectives
• Facilitate socially inclusive environmental observation
o Time & GPS location
o Temperature & Humidity
o CO2
o H2S
• Leverage existing Microsoft technologies and user communities
• Deliver a HW+SW toolkit in open source form
Key technologies
• Microsoft Research low energy GPS location sensing and mobile data collection services
• OData
• World Wide Telescope(WWT)
• Windows Azure
* In development
Sense Web: The Wikipedia of sensors
Real-time indexing, aggregation, and tasking.
Cypress: Data Stream Compression
• Compress data to reduce storage and I/O cost
• Answer queries directly on compressed data
• 100X compression typical sensor data streams
Columns
• Take advantage of data types.
• Trim data based on sufficient precision.
Trickles
• Spectrum analysis – store data based on frequency bands
• Store anomalies separately
• Use sketches to compress “noise” – preserve data correlation.
GAMPS:
• Find correlations among data streams.
• Store data as differences or ratios to reference streams.
Open Data Sharing
*Cyberinfrastructure for the waters networks: a Survey of AEESP and CUSHAI Members, K.A. Lawrence et al, May 2006,
Popular Software Packages* Factors Influencing Technology Adoption*
SQL
The Lowest Common Denominator OData • Easy of use • Additional value • Professional technical support
WWT and Geo-Data Visualization
WorldWide Telescope (WWT)
• A visualization software environment
• Enables a computer to function as a virtual telescope
• (astronomers call it “the best VO (virtual observatory) implementation”)
• Visualizes geo-data in 4D (space + time)
• Integrated with Excel
• Allows data sharing with controlled access – WWT Community
• Empowers high-quality, intuitive, and interactive visual presentation via “WWT tour”
• Datasets under consideration
• Seismic event distribution against sbuductionslab slab models (USGS NEIC)
• Standardized-format datasets (OGC, WxS, NetCDF, Shapefile, CSV, HDF, …)
• Dataset and model output concept: plugging data generators directly into WWT
• Draped raster, e.g. MODIS ocean, land and atmospheric products
• Alternatice topgraphy, e.g. ice sheet thickness and bathymetry
• Climate change thematic datasets, e.g. monthly sea ice extent from NSIDC
• Free for research and education use
* In development
WWT and Dust Storm Simulation
• A mutually beneficial case study
• Mind-swap, e.g. at Open Data for Open Science Developers Training
• Improve science modeling
• Improve computer engineering
* In development
Wikipedia of Environmental Sensing
• Open platform, open data
• Free participation
• Discoverable, searchable, interoperable
• Visualized, annotated, built on top of each other
Microsoft Environmental Informatics Since 2010 • Vision: facilitate seamless access to environmental data and information
• Focus: data discoverability, accessibility, and consumability
• Objectives:
• advance the technology use in environmental research
• create design wins using Microsoft technologies to
• Foster innovations in computational environmental research
• Advance interoperability of data and information sharing
• Facilitate citizen science for environmental research
• Build a community among multiple disciplines and stakeholders