datafed challenge. value-adding processes integrated datadatasets std. interface data views std....
TRANSCRIPT
DataFed Challenge
Value-Adding Processes
Integrated DataDatasets
Std
. In
terf
ace
Data ViewsS
td.
Inte
rface
Data
Control
Reports
Reports
Obs. & Models Decision Support System
Analyzing
Filter/IntegrateAggregate/FuseAssimilate
Organizing
DocumentStructureFormat
Exploring
Display BrowseCompare
Valu
e-A
dd
ing
P
rocesses
Data Manager Technical Analysts
DB System Analysis System Reporting System
Loosely Coupled Data Access through Standard ProtocolsOGC Web Coverage Service (WCS)
Client request Capabilities
Server returns Capabilities and data ‘Profile’
Client requests data by ‘where, when, what’ query
Server returns data ‘cube’ in requested format
GetCapabilities
GetData
Capabilities, ‘Profile’
Data
Where? When? What? Which Format?
Server
Back End S
td.
Inte
rface
Client
Front EndS
td.
Inte
rface
Query GetData Standards
Where? BBOX OGC, ISO
When? Time OGC, ISO
What? Temperature CF
Format netCDF, HDF.. CF, EOS, OGC
T2T1
Integrated DataDatasets
Std
. In
terf
ace
Data ViewsS
td.
Inte
rface
Data
Control
Reports
Reports
Obs. & Models Decision Support System
Web Services and Workflow for Loose Coupling
Integrated DataDatasets
Std
. In
terf
ace
Data ViewsS
td.
Inte
rface
Data
Control
Reports
Reports
Obs. & Models Decision Support System
Service Broker
Service Provider
PublishFind
BindServiceUser
Web Service Interaction Service Chaining & Workflow
Collaborative Reporting and Dynamic Delivery
Integrated DataDatasets
Std
. In
terf
ace
Data ViewsS
td.
Inte
rface
Data
Control
Reports
Reports
Obs. & Models Decision Support System
Co Writing - Wiki
Screencast
Analysis Reports:
Information supplied by manyNeeds continuous program feedbackReport needs many authorsWiki technologies are for collaborative writing
Dynamic Delivery:
Much of the content is dynamicAnimated presentations are compellingMovies and screencasts are for dynamic delivery
The Network Effect:Less Cost, More Benefits through Data Reuse
ProgramPublic
Data Organization
Data
Data Program
Program
OrganizationData
Data
ProgramData
Orgs Develop Programs
Programs ask/get Data Public sets
up Orgs
Pay only once Richer content
Less Prog. Cost More Knowledge
Less Soc. Cost More Soc. Benefit
Data Re-Use Network Effect
Data are costly resource – should be reused (recycled) for multiple applications
Data reuse saves $$ to programs and allows richer knowledge creation
Data reuse, like recycling takes some effort: labeling, organizing, distributing
Scheffe Challenge
GEOSS
Eco-informatics
Accountability/indicatorsSIPs, nat.rules
designations
PHASE
PM research
Risk/exposureassessments
AQ forecastingPrograms
NAAQS setting
EPANOAA
NASA
NPS
USDA
DOE
PrivateSector
States/TribesRPO’s/Interstate
Academia
NARSTO
NAS, CAAACCASAC, OMB
Enviros
Organizations
CDC
Supersites
IMPROVE, NCorePM monit, PAMS
CASTNET
Lidarsystems
NADP Satellite data
Intensive studies
PM centers
Other networks:SEARCH, IADN..
Data sources
CMAQGEOS-CHEM
EmissionsMeteorology
Health/mort.records
The Scheffe Challenge: ‘Organizations - Programs – Data Mess’
Info System Challenges:
What’s the overall dependency
Information Flow
Forces and Controls on Data Flow
Cooperation, Competition, Co-Opetition
Information System of Systems
Air Quality Management System: Components and Functions
Public
Analyzing
Interpreting Evaluating Separating Synthesizing
Organizing
Quality controlFormattingDocumentingDisplaying
Deciding
Evaluate optionsMatching goals CompromisingChoosing
Data Manager, Organizer
Technical Analysts, Program Manager
Policy Analysts, Decision Maker
Valu
e A
dd
ing
P
rocesses
Human Agents
Decision Support System (DSS)
The primary purpose of data systems is to serve programs/projects
Programs perform analysis for Orgs., the DSS is within programs
The big decisions of societal importance are Organizations
(This needs more wisdom from the practioners)
Relationship Between
Organizations - Programs – DataVersion 0.1
Goals $$Info needs, $$
Data need, $$
Judge, Decide, ActAnalyze, Report
Actionable Knowledge
Decision, Action
Public
Measure, Organize
Organized Data
Flow of InformationData systems organize the measurements and models and provide them to programs.
Programs analyze the data and provide actionable knowledge to organizations.
Organizations evaluate multiple information sources, make decisions and act.
Flow of ControlPublic and special interest groups set up organizations and provides them with funding
Organizations develop programs, define their scope, governance and funding
Programs satisfy their information needs by monitoring or by using other’s data
Data sources acquire the data for their parent programs and also expose them for reuse
Information System Components for AQ Programs
Integrated DataDatasets
Std
. In
terf
ace
Data Views
Std
. In
terf
ace
Data
Control
Reports
Reports
Public
Human Information SystemMachine Information System
Obs. & Models Decision Support System
Flow of Data and Usage Control
Flow of Control
Reports are commissioned by programs
Analysts select, explore, and process data for a report
Flow of Data
Providers expose data to analysts who extract the needed subset
The data is pulled into data exploration or processing software
Integrated DataDatasets
Std
. In
terf
ace
Data Views
Std
. In
terf
ace
Data
Control
Reports
Reports
Obs. & Models Decision Support System
Flow of Data and Usage Control
Data
Control
Requesting Information
Providing Information
Sensors
Acquisition processing User
Agencies
User Progra
ms
NAAQS SIPs Forecast GEOSS …
Info System
hhh
Integrated DataDatasets
Std
. In
terf
ace
Data Views
Std
. In
terf
ace
Data
Control
Reports
Reports
Obs. & Models Decision Support System
Architechture Challenge
GEOSS Architecture Mapped to Air QualityThe L. Friedl, K. Fontaine Challenge
Integrated DataDatasets
Std
. In
terf
ace
Data ViewsS
td.
Inte
rface
Data
Control
Reports
Reports
Obs. & Models Decision Support System
Not part of DataFed
Policy, Management, Personal Decisions
Special Architectural Features:
Implied data assimilation into models
Decision Support black box
Focus on model predictions, not ‘simulation’
Pollutant ‘Characterization’ is implicit
IGOS Architecture Mapped to DataFedThe Len Barrie Challenge
Integrated DataDatasets
Std
. In
terf
ace
Data ViewsS
td.
Inte
rface
Data
Control
Reports
Reports
Obs. & Models Decision Support System
Not part of DataFed
Quality assurance, CalVal?
Integrated Data Archives? -
Special Architectural Features:
Real-time data assimilation into AQ models
Characterization of global pattern
Young Challenge
The Steve Young Challenges
Integrated DataDatasets
Std
. In
terf
ace
Data ViewsS
td.
Inte
rface
Data
Control
Reports
Reports
Obs. & Models Decision Support System
• Real-time monitoring & watch for surprises• Inform decision-makers & assess outcomes• Support for adaptive management
S. Young Challenge #1: Real-time monitoring & watching for surprises
Integrated DataDatasets
Std
. In
terf
ace
Data ViewsS
td.
Inte
rface
Data
Control
Reports
Reports
Obs. & Models Decision Support System
Choose Data & Tools Browse, Explore Trigger Response
S. Young Challenge #2:Inform decision-makers & assess outcomes
Integrated DataDatasets
Std
. In
terf
ace
Data ViewsS
td.
Inte
rface
Data
Control
Reports
Reports
Obs. & Models Decision Support System
• This needs more work• Ideas? How could the info-system help here?
AirQuality
AssessmentCompare to GoalsPlan ReductionsTrack Progress
Controls (Actions)
Monitoring(Sensing)
Close the sensory-motor loop!
S. Young Challenge #3. Support for Adoptive AQ Management
From ‘Stovepipe’ to Workflow Software
Integrated DataDatasets
Std
. In
terf
ace
Data ViewsS
td.
Inte
rface
Data
Control
Reports
Reports
Obs. & Models Decision Support System
Flow of DataFlow of Control
Air Quality Data
Meteorology Data
Emissions Data
Informing Public
AQ Compliance
Status and Trends
Network Assess.
Tracking Progress
Data to Knowledge Transformation
Loosely Coupled Workflow