next generation 4-d distributed modeling and visualization of battlefield next generation 4-d...
TRANSCRIPT
Next Generation 4-D Next Generation 4-D Distributed Modeling and Distributed Modeling and Visualization of BattlefieldVisualization of Battlefield
Avideh ZakhorUC Berkeley
MURI on Mobile Augmented Battlespace Visualization
Participants: Avideh Zakhor, (UC Berkeley) Bill Ribarsky, (Georgia Tech) Ulrich Neumann (USC) Pramod Varshney (Syracuse) Suresh Lodha (UC Santa Cruz)
Battlefield VisualizationDetailed, timely and accurate picture of the modern battlefield vital to militaryMany sources of info:
eye witness, aerial photographs, sonar, Synthetic Aperture Radar (SAR), Multi-Spectral Imaging (MSI), Hyper-Spectral Imaging (HSI). Foliage PENetration (FOPEN) radar, Electro-Optic (EO), Infra-Red (IR), Moving Target Imaging (MTI)
Major Challenges
Disparate/conflicting sources of info must be combined.
Impossible for ONE individual to collect and comprehend
Specially trained technicians for each info source Effectiveness of information combining and
fusion determined by its usability. Must avoid information overload in presenting
the data.
Historical Perspective
Sand box:
box filled with sand shaped to replicate battlespace terrain.
Commanders moved around small physical replicas of battlefield objects to direct situation.
Historical Perspective
Paper maps and acetate: As intelligence arrives, technicians use
grease pencils to mark new info on acetate.
Commanders draw on the acetate to plan battlefield situations.
Time consuming: several hours to print, distribute and update.
Many opportunities for introducing errors.
Historical Perspective
Joint Maritime Command Information System (JMCIS): computerized battle space visualization two fundamental limitations:
clutter when displaying too much info.Two dimensional display: 3D info is lost.
Historical Perspective
Responsive workbench with Dragon: Inherently 3D. Workstation renders 3D, back-projected
on a horizontal screen. Users in the viewing area interact with
the bench through 3D mouse, pinch gloves, speech recognition.
Stereographic display with LCS shutter glass.
Improvements over WRB/Dragon
Workbench not suitable for mobile soldier with PDA. Augmented reality can enhance Dragon/WRB:
sensors distributed among the soldiers, can be used both to navigate and to update the visualization database.
Need to deal with uncertainty: represent, compute, and visualize uncertainty information without cluttering.
Time should become the 4th dimension. 4D model construction. Play back/visualize the last 24 hours.
Agile, Mobile, Collaborative Testbed
A networked, collaborative whole Earth terrain database linking workstations, large projected displays, and mobile handheld systems.
Mobile users carry systems with handheld or augmented displays providing 3D terrain visualizations.
Mobile users receive, record, and transmit information about the world.
Users of stationary 3D displays collect and evaluate information provided by mobile users and route data and intelligence back to mobile users.
Collaboration is through annotation of the virtual geo-spatial database.
Central Terrain Database
NAVE Virtual Workbench
Mobile Displays
Collaboration Channels
Wireless Terrain Datalink
Terrain Datalink
Position, Terrain Markup, Route Finding, Weather,
Friendly/Foe advisories, etc.
w/GPS, bearing, tilt sensors and wireless data.
Local databases download and cache data according to bandwidth, movement, and
rendering speed of each platform.
Wireless
Wireless
Architecture: Agile, Mobile, Collaborative Testbed
Research Agenda
Model construction: initially constructed by registering sensor imagery to
reference imagery, maps, elevation data, etc. Four dimensional: time treated on the same footing as space.
Model update: Distributed mobile or stationary users contribute to updating the database via their sensors.
Mobile, real time visualization, interaction and navigation within the database; augmented reality sensors tracking and registration required;
Uncertainty processing for model construction and update, as well as uncertainty visualization.
Tracking & Registration
Database Visualization
Model Update & Construction
Uncertainty processing
Uncertainty visualization
Technical Challenges
3-D and 4D Model Construction
Develop a framework for fast, automatic and accurate 3D model construction for objects, scenes, rooms, buildings (interior and exterior), urban areas, and cities. Incorporate time element (4-D).
Models must be easy to compute, compact to represent, suitable for insertion in large hierarchical visualization databases, to facilitate high quality view synthesis and visualization from views that were not necessarily captured during data collection process.
Strategy: Fusion of multiple data sources: intensity, range, GPS, panoramic
cameras. Incorporate apriori models, e.g. CAD, DEM, DTED, elevation data,
maps
Geo-registration and Tracking
Develop techniques for unemcumbered, wide area and real time tracking for (a) augmentation and (b) visualization.
Strategy: Estimate real time 6-DOF tracking by fusion of multiple data streams with variable uncertainty: GPS, vision, inertial gyros, accelerometers, compass, laser range finders
Real time prototype will use the resulting tracking algorithms on an augmented reality PDA to geo-register and navigate the user within the geo-spatial database.
Mobile Visualization
VGIS: general framework for global geo-spatial data and visualization.
Hierarchical, scalable data structures with fast access that include (a) time; (b) uncertainty; c) all varied products.
Dynamic data structures with fast update: Real time data is put in dynamic cache until system finds time to integrate it with the online data base.
Automated detail management for (a) uncertainty; (b) all visual products;
Intelligent retrieval and visual data mining. Multi-modal interaction in multiple environments.
Technical Challenges: Uncertainty Computation and Visualization
Physicalphenomenon
Informationacquisition
-sensors -humans -databases
Uncertainty computation
Uncertaintyvisualization
-target deformation -glyphs -animation
Info. processing& fusion
-computational algorithms
Analysis
-interaction -decision
Transformation
-sampling -quantization -compression
Uncertainty Processing
Representation and computation issues: many formalisms: probability, possibility, evidence theory;
Transformations; Fusion: data, feature and decision levels
uncertainty aware fusion algorithms for dynamic distributed networks; various topologies for fusion networks: serial, parallel, tree network, non-tree feedback network.
decentralized statistical inference algorithms Time Critical computation and quality of service issues:
Data continually arriving, requiring re-computation. tradeoff between precision and speed of uncertainty
computation; fast, yet imprecise answers.
Uncertainty Visualization and Validation
Present uncertainty in an intuitive uncluttered way; Display devices: screen space, mobility Modality: vision, audio, haptics Data types: scalar/vector/tensor, discrete/continuous,
static/dynamic Uncertainty visualization techniques: glyphs,
deformation, transparency, texture, superimposing/backgrounding, augmented reality.
Validation with novice and expert users: Tasks for mobile battlefield; measure accuracy
and speed performance of user; conduct stat. analysis.
Proof of Concept : Multi-modal Interactions for PDA
Extend interaction methods for workstations to PDA;
Wired (electromagnetic) and unwired (vision) interaction modes, hand gestures
Visible and infrared sensors added to PDA Demonstrate a hybrid vision/inertial tracking PDA
demonstrate VGIS visualization on portable display context
Multiple mobile users collaborate on identifying and locating features or targets.
Facilities and Equipment
G. Tech: GVU Future Computing Lab (FCL); Mobile augmented test-bed.
USC: IMSC lab; CGIT lab.UCSC: VizLabSyracuse: Sensor Fusion Laboratory
(SFL)Berkeley: Video and Image Processing
(VIP) Lab.
Collaboration and Management issues
Dr. Hassan Foroosh currently with University of Maryland, will be the technical liason amongst the five universities.
Telephone conference amongst team members periodically to discuss status of the project.
Live seminars broadcast via video conferencing amongst five campuses, once every two months.
Annual workshops and retreats for PI and graduate students and all other researchers to exchange ideas.
Transitions
Air Force Research Lab (Graniero)Army Research Lab (Emmerman, Tocarcik)Navy Research Lab (Rosenblum)Lawernce Livermore Lab (Ucelton)Sun Microsystems (Sowizraj)Intel (Liang)Planet 9 (Colleen)Geometrix (Zwern)Hughes Research Lab (Azuma)
Cross Collaboration
UCB USC G.T. SYR UCSC
Model const. & update
X x x x
Tracking & reg. x X x xMobile visual. database
x x X x x
Uncertain.processing
x x X x
Uncertain.Visualization.
x x X
Outline of Talks
3D model construction for visualization (UC Berkeley)
Geo-registration and tracking for augmentation and visualization(USC)
Mobile visualization in dynamic, augmented battlespace (Georgia Tech)
Uncertainty processing and information fusion (Syracuse)
Uncertainty visualization and validation (UC Santa Cruz)