Telesupervised Adaptive Ocean Sensor Fleet Year 1 Interim Review

Feb. 23, 2007Carnegie Mellon University

NASA Goddard Space Flight FacilityNASA Wallops Flight Facility

Jet Propulsion Laboratory

2

Outline

• Project and system overview (slides 2-4)

• Technical status (slides 5-21)

• Schedule, milestones, and work planned (slides 22-25)

• Critical issues (slide 26)

• Financial status (slide 27)

• Educational outreach (slide 28)

• Acronyms/glossary (slide 29)

3

Telesupervised Adaptive Ocean Sensor Fleet (TAOSF)

Objective

Key Milestones

TRLin = 4

• Improved in-situ study of Harmful Algal Blooms (HAB), coastal pollutants, oil spills, and hurricane factors

• Expanded data-gathering effectiveness and science return of existing NOAA OASIS (Ocean Atmosphere Sensor Integration System) surface vehicles

• Establishment of sensor web capability combining ocean-deployed and space sensors

• Manageable demands on scientists for tasking, control, and monitoring

Approach• Telesupervision of a networked fleet of NOAA surface autonomous vehicles (OASIS)• Adaptive repositioning of sensor assets based on environmental sensor inputs (e.g., concentration gradients)• Integration of complementary established and emergent technologies (System Supervision Architecture (SSA), Inference Grids, Adaptive Sensor Fleet (ASF), Instrument Remote Control (IRC), and OASIS)• Thorough, realistic, step-by-step testing in relevant environments• Gregg Podnar / CMU• Jeffrey Hosler, John Moisan, Tiffany Moisan / GSFC• Alberto Elfes / JPL

PI: John Dolan, CMU

Co-I’s/Partners

Artist's conception of telesupervised sensor fleet investigating a Harmful Algal Bloom.

• Interface Definition Document Feb 2007• Test components on one platform in water May 2007• Autonomous multi-platform mapping of dye Jul 2007• Science requirements for Inference Grid Feb 2008• Multi-platform concentration searchsimulation May 2008• HAB search in estuary for high concentration Jul 2008• Moving water test plan & identify location Feb 2009• Simulate test using in-situ and MODIS data May 2009• Use MODIS data to target and reassign fleet Jul 2009

4

TAOSF Program Synergy

ESTO Office

Inputs

4 PhD, MS, and BS students

ESTO Office

Collaborative Partner

Telesupervised Adaptive Ocean

Sensor Fleet Project

AIST Value Added Outputs

Tools and Technology Users

GSFC

OASIS Platforms

Adaptive Sensor Fleet / Instrument Remote Control

Multi-Robot Telesupervision

Architecture

Planetary Exploration

HAB Detection

5

OASIS Mapping of Harmful Algal Blooms

• System Components• System Supervision Arch. (SSA)• Adaptive Sensor Fleet (ASF)• Instrument Remote Control (IRC)• Inference Grids (IG)• Marine platforms (OASIS)

High-level planning and monitoring

High-bandwidth, single-platformtelepresence

Low-bandwidth, multi-platform telemetry

6

Technical Status

• Near-complete TAOSF architecture design (slides 6-9)

• Software architecture integration progress (SSA-ASF-IRC-OASIS) (slides 10-11)

• Ongoing Harmful Algal Bloom (HAB) dataset acquisition and analysis (slide 12)

• Initial design and testing of ground-truthing system (slides 13-20)

• OASIS platform development and testing (slide 21)

7

TAOSF Architecture Design (1)

OASIS ASV

System (EST)

Platform Communicator

(GSFC)

Multi-Platform Simulation

Environment(GSFC)

Adaptive Sensor

Fleet(GSFC)

SystemSupervision Architecture

(CMU/JPL)

OASIS Driver API Instrument Remote Control MySQL HTTP

OASIS ASV

System (EST)

OASIS ASV

System (EST/WFF)

Connectivity of high-level components

CMU: Carnegie Mellon UniversityGSFC: Goddard Space Flight CenterWFF: Wallops Flight FacilityEST: Emergent Space TechnologiesJPL: Jet Propulsion Laboratory

8

GSFC: Multi-Platform Simulation Environment

EST: OASIS ASV System

OASISPlatform

PlatformGateway

Mission Operations Environment

Fleet Environment

OASISPlatform

NetworkServices

EngineeringInterface

Logging

PlatformDriver

Environmental Models

Platform BehaviorModels

SimulationManager

GSFC: Platform Communicator

MessageReceiver

MessageConverter

StateModel

OASIS Driver API Instrument Remote Control

Detailed view of platforms, simulator, and communicator

TAOSF Architecture Design (2)

9

GSFC: Adaptive Sensor Fleet System

GoalsDatabase

Comm.Client

FleetManager

ASFWeb GUI

Science/GoalAnalyzer

States/ModelsDatabase

TAOSF Architecture Design (3)

Fleet Manager

Sends position commands to boats based on plans developed in the Science/Goal Analyzer.

Communications Client

Provides bidirectional communications with the real or simulated platforms.

Science/Goal Analyzer Plans efficient multi-platform coverage of designated regions based on hexagonal tesselation of the environment.

ASF Web GUI

Allows web-based specification of user goals via ASF. Can be bypassed by the SSA (see next slide) to insert automatically generated goals or user-generated goals at the SSA level.

10

CMU/JPL: System Supervision Architecture

ASFClient

Science Data Analyzer

OCU Interface

DataStorage Handler

Data Storage

RobotController

OCU

External Science

Data

RemoteData

Display

RemoteData

Interface

OCU

Operator Control Unit provided by SPAWAR Systems Center San Diego with modifications by CMU

Robot Controller

Provides tasking and monitoring of individual robots and groups. Based on existing CMU Robot Supervision Architecture.

Science Data Analyzer

Combines data from the robots and other sources (satellite imagery, buoys, etc.) to predict HAB locations.

Remote Data Interface / Display

Allows remotely-located scientists to review data both in real-time and as recorded playback.

TAOSF Architecture Design (4)

11

Software Integration

• Nov 2006: API for Adaptive Sensor Fleet (ASF)-OASIS communications developed

• Dec 2006: Conducted dry test of ASF commands sent to and engineering telemetry received from OASIS

• Feb 2006: Initial integration of System Supervision Architecture (SSA) with ASF and existing U.S. Navy OCU (MOCU1) preparatory to SSA-ASF-OASIS end-to-end software test

1MOCU ( Multi-Robot Operator Control Unit) is developed by SPAWAR Systems Center San Diego (SSC-SD)

12

MOCU sending waypoints to and receiving engineering telemetry from ASF

Engineering telemetry

OASIS platformfollowing waypoint

trajectory

13

HAB Dataset Acquisition/Analysis

• Based on ROMS model of the Chesapeake Bay, investigating correlation between surface temperature and salinity

• Obtained chlorophyll A and sea surface temperature MODIS data for the Delmarva region

• Obtained descriptions of five potential HAB regions of study in the Chesapeake and Coastal Bays from the Maryland Dept. of Natural Resources

Chesapeake Bay

Temperature Salinity

MODIS sea surface temperature

14

Ground-Truthing System

• Purpose: confirm data from OASIS platforms Platform positions and “bloom” concentration measures

• Means: aerial sensor/communications package

Carried aloft by an aerostat tethered to a human-piloted research boat

Sensor package: GPS position, barometric altimeter, magnetic compass, video camera filtered to enhance rhodamine WT imaging

Use existing JPL software for mosaicing and object recognition

15

Ground-Truthing System

To confirm data from OASIS platforms: • Aerial camera with sensors: latitude, longitude, altitude & heading • Image the bloom and the boats

Will use existing JPL software to geolocate boats and bloom.

16

Ground-Truthing System

Initial Test — 2006-Nov-14 at JPL

Simple initial test conducted with recording GPS and Digital Camcorder lifted on a tethered weather balloon.

1) GPS data was used to recover an aerial image of the test site from Google Earth (GE).2) Camcorder images were overlaid on GE image.

17

Ground-Truthing System

Initial Test — 2006-Nov-14 at JPL

• Mosaic of camcorder images (sharp) overlaid on Google Earth image (blurry)

• Position reconstructed from recorded GPS track data

• Heading recovered manually

18

Ground-Truthing System

Second Test — 2007-Feb-16 at JPL

• Avionics package:• GPS• Barometric altimeter• Magnetic compass• Serial data link• Wide-angle monochrome camera• Video transmitter

• Fins on package to limit rotation

1) GPS data was used to recover an aerial image of the test site from Google Earth (GE).2) Camcorder images were overlaid on GE image.

19

Ground-Truthing System

Second Test — 2007-Feb-16 at JPL

Two frames from camera over parking lot test site annotated with GPS position, altitude above ground, and heading showing uncertainty.

These data recorded simultaneously from sensor package.

20

Ground-Truthing System

Second Test — 2007-Feb-16 at JPL

• Test Image 1 overlaid on Google Earth image of parking lot

• Test Image center within 3m of Google Earth GPS mark

• Heading uncertainty includes Google Earth’s North. This will be improved with a more stable aerostat.

21

Ground-Truthing System

Second Test — 2007-Feb-16 at JPL

•Test Image 2 overlaid on Google Earth image of parking lot

•Test Image center within 2m of Google Earth GPS mark

• Heading off by more than 30˚ from Google Earth’s North. This will be improved with a more stable aerostat.

22

• 15 Nov 06: First open-ocean deployment of OASIS-2

• OASIS-2 has barometer, fluorometer, and temperature, humidity,

and salinity sensors• OASIS-2 currently conducting long-term (2-3 day) operations

testing• OASIS-1 being upgraded to OASIS-2 level

OASIS at seaOASIS about to launch

OASIS Platform Development

23

Year 1 Schedule

Software integration

Overall architecture design

Interface Definition Document

Subsystem integration

Adaptive sampling

HAB data acquisition

Initial algorithm development

Sensors

Ground-truthing system development

Science sensor placement

System testing

In-water subsystem test

In-water 1-platform test

In-water multi-platform test

Validate autonomous dye detection

3Q06 4Q06 1Q07 2Q07 3Q07 4Q07

Yr. 1 start date: Sept. 5, 2006 Yr. 1 end date: Sept. 4, 2007

24

Year 1 Milestones

• Conduct initial ground-truthing tests (at JPL) Nov 2006

• Complete/test ASF-OASIS interface Dec 2006

• Conduct interim ground-truthing tests (at JPL) Feb 2007

• Complete Interface Definition Document Feb 2007

• Test fully integrated (SSA-ASF-OASIS) software Apr 2007

• Test components on one platform in water May 2007

• Autonomous single-platform mapping of dye Jun 2007

• Autonomous multi-platform mapping of dye Jul 2007

25

Key Project Milestones

• Interface Definition Document Feb 2007

• Autonomous multi-platform mapping of dye Jul 2007

• Multi-platform HAB search in estuary Jul 2008

• Use MODIS data to target and reassign fleet Jul 2009

26

Work Planned

• Finalize Year 1 TAOSF architecture design

• Test end-to-end software integration first in simulation, then with one platform in the water: issue commands to and receive engineering and science telemetry from OASIS

• Use ROMS data to investigate ability to infer detailed salinity and temperature characteristics from sparse samples

• Stabilize, refine, and conduct additional testing of ground-truthing system

• Follow up initial contacts at Feb 2007 San Diego meeting– Stephan Kolitz expressed interest in inserting the dynamic replanning

component of the Earth Phenomena Observing System (EPOS) as a module in the TAOSF system

– We may be able to use Internet tasking of the EO-1 satellite (POC Dan Mandl)

– Confer with Robert Morris about inserting his planning work in TAOSF

27

Critical Issues

• The availability of the third OASIS platform for the July 2007 multi-platform test is dependent on the platform development schedule and NOAA funding of this parallel project.

• The ability of the ground-truthing system to accurately detect rhodamine WT dye needs to be validated.

• We have had difficulty obtaining good HAB or HAB-related datasets that would allow algorithm development and off-line testing of adaptive sampling.

28

PROJECT FINANCIAL STATUSTelesupervised Adaptive Ocean Sensor Fleet

Cost Status

0

50

100

150

200

250

300

350

400

450

500

Co

st $

K

Cum Cost Plan 35 73 111 148 186 224 261 299 337 368 399 431 471

Cum Cost Actual 17 34 52 75 89

Variance -18 -39 -58 -73 -97

Sep-06 Oct-06 Nov-06 Dec-06 Jan-07 Feb-07 Mar-07 Apr-07 May-07 Jun-07 Jul-07 Aug-07 Sep-07

Notes: 1. Wallops has not charged the award yet, but will do so later in the year in lumpsum(s) reflecting the planned average $10K/month spending rate. 2. GSFC began charging to the award in December 2006. Their planned average spending is $8K/mo.

29

Educational Outreach

• Steve Stancliff– Ph.D. student,

Robotics– Carnegie

Mellon University

• Ellie Lin– Ph.D. student,

Robotics– Carnegie Mellon

University

• Jeff Baker– B.S. student,

Computer Science– Duquesne University

• Sandra Mau– Master’s student,

Robotics– Carnegie Mellon

University

30

Acronyms/Glossary

• API – Application Program Interface• ASF – Adaptive Sensor Fleet• CMU – Carnegie Mellon University• Delmarva – Delaware/Maryland/Virginia• EST – Emergent Space Technologies• GSFC – Goddard Space Flight Center• HAB – Harmful Algal Bloom• IG – Inference Grids• IRC – Instrument Remote Control• JPL – Jet Propulsion Laboratory• MOCU – Multi-Robot Operator Control Unit• MODIS – Moderate-Resolution Imaging Spectrometer • MySQL – My Structured Query Language, a popular database management system• NOAA – National Oceanic and Atmospheric Administration• OASIS – Ocean Atmosphere Sensor Integration System• Rhodamine WT – a non-toxic liquid red dye commonly used in water-tracing studies• ROMS – Regional Ocean Modeling System• SPAWAR – Space and Naval Warfare Systems• SSA – System Supervision Architecture• TAOSF – Telesupervised Adaptive Ocean Sensor Fleet• WFF – Wallops Flight Facility