uncertainty computation,visualization, and validation suresh k. lodha computer science university of...
DESCRIPTION
Overview Uncertainty representation & computation Data/information fusion Quality-of-service issues Uncertainty visualization Uncertainty validationTRANSCRIPT
![Page 1: Uncertainty Computation,Visualization, and Validation Suresh K. Lodha Computer Science University of California, Santa Cruz (831)-459-3773](https://reader036.vdocuments.us/reader036/viewer/2022081517/5a4d1af17f8b9ab05997e450/html5/thumbnails/1.jpg)
Uncertainty Computation,Visualization, and Validation
Suresh K. LodhaComputer Science
University of California, Santa [email protected]
(831)-459-3773
![Page 2: Uncertainty Computation,Visualization, and Validation Suresh K. Lodha Computer Science University of California, Santa Cruz (831)-459-3773](https://reader036.vdocuments.us/reader036/viewer/2022081517/5a4d1af17f8b9ab05997e450/html5/thumbnails/2.jpg)
Personnel
• Suresh K. Lodha– PhD, CS, Rice University, 1992– Scientific & information visualization,
uncertainty quantification & visualization,multi-modal visualization,computer-aided geometric design
– Uncertainty research supported byNational Science Foundation &Department of Energy ASCI Program
(LLNL, LANL, SNL)
![Page 3: Uncertainty Computation,Visualization, and Validation Suresh K. Lodha Computer Science University of California, Santa Cruz (831)-459-3773](https://reader036.vdocuments.us/reader036/viewer/2022081517/5a4d1af17f8b9ab05997e450/html5/thumbnails/3.jpg)
Overview
• Uncertainty representation & computation• Data/information fusion• Quality-of-service issues• Uncertainty visualization• Uncertainty validation
![Page 4: Uncertainty Computation,Visualization, and Validation Suresh K. Lodha Computer Science University of California, Santa Cruz (831)-459-3773](https://reader036.vdocuments.us/reader036/viewer/2022081517/5a4d1af17f8b9ab05997e450/html5/thumbnails/4.jpg)
Uncertainty Visualization Pipeline
Physicalphenomenon
Informationacquisition
-sensors -humans -databases
Uncertainty computation
Uncertaintyvisualization
-target deformation -glyphs -animation
Info. processing& fusion
-computational algorithms
Analysis
-interaction -decision
Transformation
-sampling -quantization -compression
![Page 5: Uncertainty Computation,Visualization, and Validation Suresh K. Lodha Computer Science University of California, Santa Cruz (831)-459-3773](https://reader036.vdocuments.us/reader036/viewer/2022081517/5a4d1af17f8b9ab05997e450/html5/thumbnails/5.jpg)
Sources of Uncertainty
• Sensor and human limitations• Noise, clutter, jamming etc.• Modeling assumptions• Algorithm limitations• Data compression• Communication errors• Visualization-induced errors
![Page 6: Uncertainty Computation,Visualization, and Validation Suresh K. Lodha Computer Science University of California, Santa Cruz (831)-459-3773](https://reader036.vdocuments.us/reader036/viewer/2022081517/5a4d1af17f8b9ab05997e450/html5/thumbnails/6.jpg)
Uncertainty Representation
• Uncertainty formalisms used by the fusion community– Probability– Dempster-Shafer evidence theory– Fuzzy sets and possibility theory
• Uncertainty representation in visualization research – Confidence intervals– Estimation error– Uncertainty range
![Page 7: Uncertainty Computation,Visualization, and Validation Suresh K. Lodha Computer Science University of California, Santa Cruz (831)-459-3773](https://reader036.vdocuments.us/reader036/viewer/2022081517/5a4d1af17f8b9ab05997e450/html5/thumbnails/7.jpg)
Uncertainty Computation(Previous Work)
• Data/Information Fusion– Knowledge-based systems– Random sets (Goodman, Nguyen, Mahler)
• Visualization– NIST/ NCGIA `91 (Beard et al.)– BattleSpace `98 (Durbin et al.)– Visualization Software `96 (Globus, Uselton)– Scientific Visualization `96 -- (Lodha, Pang,
Wittenbrink)
![Page 8: Uncertainty Computation,Visualization, and Validation Suresh K. Lodha Computer Science University of California, Santa Cruz (831)-459-3773](https://reader036.vdocuments.us/reader036/viewer/2022081517/5a4d1af17f8b9ab05997e450/html5/thumbnails/8.jpg)
“Any battlefield necessarily deals with uncertainty, and it is necessary to determine ways to represent and encode the confidence level that exists for each piece of battlefield data.” – Durbin et al ’98 (NRL)
![Page 9: Uncertainty Computation,Visualization, and Validation Suresh K. Lodha Computer Science University of California, Santa Cruz (831)-459-3773](https://reader036.vdocuments.us/reader036/viewer/2022081517/5a4d1af17f8b9ab05997e450/html5/thumbnails/9.jpg)
Designing Error Metrics
• True vs. measured/observed/anticipated• Observed vs. simulated• High resolution vs. low resolution• Continuous vs. discrete• Individual source vs. multiple sources• Static vs. dynamic• Time-independent vs. time-critical• Error-free vs. error-prone communication
![Page 10: Uncertainty Computation,Visualization, and Validation Suresh K. Lodha Computer Science University of California, Santa Cruz (831)-459-3773](https://reader036.vdocuments.us/reader036/viewer/2022081517/5a4d1af17f8b9ab05997e450/html5/thumbnails/10.jpg)
Examples of Error Metrics
• Local metrics -- distance metric-- curvature metric-- sampling-number or depth metric
(distribution of error)• Global metrics
-- Topology metric
![Page 11: Uncertainty Computation,Visualization, and Validation Suresh K. Lodha Computer Science University of California, Santa Cruz (831)-459-3773](https://reader036.vdocuments.us/reader036/viewer/2022081517/5a4d1af17f8b9ab05997e450/html5/thumbnails/11.jpg)
Uncertainty Metrics : Isosurfaces
![Page 12: Uncertainty Computation,Visualization, and Validation Suresh K. Lodha Computer Science University of California, Santa Cruz (831)-459-3773](https://reader036.vdocuments.us/reader036/viewer/2022081517/5a4d1af17f8b9ab05997e450/html5/thumbnails/12.jpg)
Uncertainty Metrics : Fluid Flow Topology
Original
332 cp65%
55%
![Page 13: Uncertainty Computation,Visualization, and Validation Suresh K. Lodha Computer Science University of California, Santa Cruz (831)-459-3773](https://reader036.vdocuments.us/reader036/viewer/2022081517/5a4d1af17f8b9ab05997e450/html5/thumbnails/13.jpg)
Research Issues
• Representations and data structures for uncertainty measures
• Design and integration of error metrics• Uncertainty-aware and uncertainty-reducing
data processing (algorithms and models)• Common consistent uncertainty
representation over a distributed mobile network ?
![Page 14: Uncertainty Computation,Visualization, and Validation Suresh K. Lodha Computer Science University of California, Santa Cruz (831)-459-3773](https://reader036.vdocuments.us/reader036/viewer/2022081517/5a4d1af17f8b9ab05997e450/html5/thumbnails/14.jpg)
Uncertainty Visualization
• How to convey uncertainty to human users?– Uncluttered display– Intuitive metaphors for mapping– Data characteristics– Multi-modality
• Do NOT hide processes that produce problems for the human users?– Visualize the abstraction (e.g uncertainty
pipeline, graphical models)
![Page 15: Uncertainty Computation,Visualization, and Validation Suresh K. Lodha Computer Science University of California, Santa Cruz (831)-459-3773](https://reader036.vdocuments.us/reader036/viewer/2022081517/5a4d1af17f8b9ab05997e450/html5/thumbnails/15.jpg)
Uncertainty Visualization
• Display devices /environment– screen space (monitor, PDA, workbench,..)– mobility
• Modality– vision– audio– haptics
![Page 16: Uncertainty Computation,Visualization, and Validation Suresh K. Lodha Computer Science University of California, Santa Cruz (831)-459-3773](https://reader036.vdocuments.us/reader036/viewer/2022081517/5a4d1af17f8b9ab05997e450/html5/thumbnails/16.jpg)
Uncertainty Visualization
• Data types/ characteristics– scalar/vector/tensor– discrete/continuous– static/dynamic
• Levels of fusion– data-level (raw/abstract)– image-level (physical phenomena)– feature-level (compressed view)– decision-level (super-compressed)
![Page 17: Uncertainty Computation,Visualization, and Validation Suresh K. Lodha Computer Science University of California, Santa Cruz (831)-459-3773](https://reader036.vdocuments.us/reader036/viewer/2022081517/5a4d1af17f8b9ab05997e450/html5/thumbnails/17.jpg)
Uncertainty Visualization (continued)
• Techniques– glyphs– deformation– transparency– texture– linking– superimposing/backgrounding– augmented reality– modality
![Page 18: Uncertainty Computation,Visualization, and Validation Suresh K. Lodha Computer Science University of California, Santa Cruz (831)-459-3773](https://reader036.vdocuments.us/reader036/viewer/2022081517/5a4d1af17f8b9ab05997e450/html5/thumbnails/18.jpg)
Unc Viz: Example 3Fluid Flow Visualization
![Page 19: Uncertainty Computation,Visualization, and Validation Suresh K. Lodha Computer Science University of California, Santa Cruz (831)-459-3773](https://reader036.vdocuments.us/reader036/viewer/2022081517/5a4d1af17f8b9ab05997e450/html5/thumbnails/19.jpg)
Unc Viz: Example 5 Geometric Uncertainty
![Page 20: Uncertainty Computation,Visualization, and Validation Suresh K. Lodha Computer Science University of California, Santa Cruz (831)-459-3773](https://reader036.vdocuments.us/reader036/viewer/2022081517/5a4d1af17f8b9ab05997e450/html5/thumbnails/20.jpg)
Uncertainty Visualization: Example 4
![Page 21: Uncertainty Computation,Visualization, and Validation Suresh K. Lodha Computer Science University of California, Santa Cruz (831)-459-3773](https://reader036.vdocuments.us/reader036/viewer/2022081517/5a4d1af17f8b9ab05997e450/html5/thumbnails/21.jpg)
Research Issues
• Uncertainty mapping and metaphors for different modalities, data types and fusion tasks
• Display support for a variety of uncertainty metrics/formalisms
• Interactive display for uncertainty-source -> task analysis
• Integration and analysis of uncertainty for decision-making?
![Page 22: Uncertainty Computation,Visualization, and Validation Suresh K. Lodha Computer Science University of California, Santa Cruz (831)-459-3773](https://reader036.vdocuments.us/reader036/viewer/2022081517/5a4d1af17f8b9ab05997e450/html5/thumbnails/22.jpg)
Uncertainty Validation
• Does addition of uncertainty information help human users in making decisions?
• Can humans integrate qualitative and quantitative (or heterogeneous information) when there is uncertainty?– Task definitions– Careful design of experiments– Usability studies– Statistical analysis
![Page 23: Uncertainty Computation,Visualization, and Validation Suresh K. Lodha Computer Science University of California, Santa Cruz (831)-459-3773](https://reader036.vdocuments.us/reader036/viewer/2022081517/5a4d1af17f8b9ab05997e450/html5/thumbnails/23.jpg)
Uncertainty Validation
• Task definitions– primary level tasks (raw estimation)– secondary level tasks (correlation or simple spatio-
temporal relationships)– higher level tasks
• Examples– feature existence (binary decision)– feature recognition (finite multiple choices)– target aiming (zone-centered decision within a
specified space-time region)
![Page 24: Uncertainty Computation,Visualization, and Validation Suresh K. Lodha Computer Science University of California, Santa Cruz (831)-459-3773](https://reader036.vdocuments.us/reader036/viewer/2022081517/5a4d1af17f8b9ab05997e450/html5/thumbnails/24.jpg)
Validation Strategies
• Formative vs. summative studies• With or without uncertainty mapping• Representative sampling of tasks, data and
uncertainty mappings• Constrained, interactive or free-form
environment• Within-subjects/between-subjects and
tabular designs
![Page 25: Uncertainty Computation,Visualization, and Validation Suresh K. Lodha Computer Science University of California, Santa Cruz (831)-459-3773](https://reader036.vdocuments.us/reader036/viewer/2022081517/5a4d1af17f8b9ab05997e450/html5/thumbnails/25.jpg)
Uncertainty Validation (Previous Work)
• Validation of user interfaces (CHI `90s) • Validation of multi-modal mappings
(Melara, Marks, Massaro (UCSC))• Validation of uncertainty mappings
![Page 26: Uncertainty Computation,Visualization, and Validation Suresh K. Lodha Computer Science University of California, Santa Cruz (831)-459-3773](https://reader036.vdocuments.us/reader036/viewer/2022081517/5a4d1af17f8b9ab05997e450/html5/thumbnails/26.jpg)
Uncertainty Validation: Example 1 (with M. Hansen)
• Protein structural alignment (intuitive metaphors)
![Page 27: Uncertainty Computation,Visualization, and Validation Suresh K. Lodha Computer Science University of California, Santa Cruz (831)-459-3773](https://reader036.vdocuments.us/reader036/viewer/2022081517/5a4d1af17f8b9ab05997e450/html5/thumbnails/27.jpg)
Uncertainty Validation: Example 1 (continued)
• Protein structural alignment-- accuracy of discrimination
![Page 28: Uncertainty Computation,Visualization, and Validation Suresh K. Lodha Computer Science University of California, Santa Cruz (831)-459-3773](https://reader036.vdocuments.us/reader036/viewer/2022081517/5a4d1af17f8b9ab05997e450/html5/thumbnails/28.jpg)
Uncertainty Validation: Example 2 (GIS)
Rainbow Saturated
![Page 29: Uncertainty Computation,Visualization, and Validation Suresh K. Lodha Computer Science University of California, Santa Cruz (831)-459-3773](https://reader036.vdocuments.us/reader036/viewer/2022081517/5a4d1af17f8b9ab05997e450/html5/thumbnails/29.jpg)
Uncertainty Validation:Tasks(averaging, comparisons)
![Page 30: Uncertainty Computation,Visualization, and Validation Suresh K. Lodha Computer Science University of California, Santa Cruz (831)-459-3773](https://reader036.vdocuments.us/reader036/viewer/2022081517/5a4d1af17f8b9ab05997e450/html5/thumbnails/30.jpg)
Uncertainty Validation: (with Wittenbrink & Pang)
• Vector uncertainty glyph evaluation
![Page 31: Uncertainty Computation,Visualization, and Validation Suresh K. Lodha Computer Science University of California, Santa Cruz (831)-459-3773](https://reader036.vdocuments.us/reader036/viewer/2022081517/5a4d1af17f8b9ab05997e450/html5/thumbnails/31.jpg)
Research Issues
• Construction of user evaluation environments
• Conduct user evaluation studies for efficiency and accuracy
• Data analysis and statistical testing• Feedback loop to improve performance• Integrated decision tool combining
uncertainty approaches in visualization and command and control?
![Page 32: Uncertainty Computation,Visualization, and Validation Suresh K. Lodha Computer Science University of California, Santa Cruz (831)-459-3773](https://reader036.vdocuments.us/reader036/viewer/2022081517/5a4d1af17f8b9ab05997e450/html5/thumbnails/32.jpg)
Concluding Remarks I
• Provide human users with uncertainty information– Representation and computation of uncertainty– Uncertainty-aware and uncertainty-reducing
algorithms and models– Uncertainty visualization– Visualization of uncertainty pipeline or hidden
processes or abstract models
» (continued)
![Page 33: Uncertainty Computation,Visualization, and Validation Suresh K. Lodha Computer Science University of California, Santa Cruz (831)-459-3773](https://reader036.vdocuments.us/reader036/viewer/2022081517/5a4d1af17f8b9ab05997e450/html5/thumbnails/33.jpg)
Concluding Remarks II
• Effective and clutter-free visualization of uncertainty along with the data/information– Sensitive to data characteristics/ fusion level/
tasks/ display environments (intuitive and cognitively accurate metaphors)
– Multi-modality – Usability studies
![Page 34: Uncertainty Computation,Visualization, and Validation Suresh K. Lodha Computer Science University of California, Santa Cruz (831)-459-3773](https://reader036.vdocuments.us/reader036/viewer/2022081517/5a4d1af17f8b9ab05997e450/html5/thumbnails/34.jpg)
Collaboration
• Uncertainty representation/ fusion(UCSC, Syracuse GTech, UCB,USC)
• Uncertainty visualization(UCSC, Gtech UCB, USC)
• Multi-modal interaction(UCSC USC, GTech)
• Other MURIS/ DoD?