aist-qrs-06-0026 esto interim review
DESCRIPTION
The Detection and Tracking of Satellite Image Features Associated with Extreme Physical Events for Sensor Web Targeting Observing. AIST-QRS-06-0026 ESTO Interim Review. Principle Investigator: John F Moses, GSFC Code 586 Data Systems Co-Investigator:Liping Di, George Mason University CSISS - PowerPoint PPT PresentationTRANSCRIPT
The Detection and Tracking of Satellite Image Features
Associated with Extreme Physical Events for Sensor Web Targeting
Observing
AIST-QRS-06-0026ESTO Interim Review
Principle Investigator: John F Moses, GSFC Code 586 Data SystemsCo-Investigator: Liping Di, George Mason University CSISSCo-Investigator : Wayne Feltz, U of Wisconsin SSEC/CIMSSCo-Investigator : Jason Brunner, U of Wisconsin SSEC/CIMSSCo-Investigator: Robert Rabin, National Severe Storms Laboratory
2
The Detection and Tracking of Satellite Image Features Associated with Extreme Physical Events for Sensor Web
Targeting Observing
Objective
Key Milestones
TRLin = 2
•Modify WCS server to work with GOES data prototype detection and tracking algorithms
03/2007•Install and configure virtual sensor platform
06/2007•Develop automated algorithms for overshooting tops and Enhanced-V, and methods to assess convective initiation detection skills
09/2007•Finish assessment of physical data models and methods on GEO and LEO case study datasets
12/2007
•Our technical approach involves developing a capability to evaluate prototype components for image event detection that can support the needs of complex multi-discipline physical models.
•The prototype detection and tracking algorithms and techniques will serve as a basis for comparative analysis of detailed implementation approaches for the virtual sensor platform and event data models.
Approach
This project will demonstrate a capability to detect, track and rank radiance structures in satellite image data that are associated with extreme physical events. Our objective is to define key elements of a generalized technology capable of populating cross-discipline target ranking models for Sensor Web application. Secondly, a single interface is desired for event detection and tracking algorithms to access data from multiple, diverse sensors and models. The third objective is to enable discovery of new physical event detection models by implementing the capability to measure and rank geometric characteristics and show application in detection of the onset of convection.
Liping Di / GMUWayne Feltz, Jason Brunner / U. of WIRobert Rabin / NOAA
PI: John Moses, GSFC
Co-I’s/Partners
Dec 2006
A prototype algorithm detected and tracked over a thousand objects in the September 18, 2003 GOES IR image with Hurricane Isabel. The detected objects appear as an overlay of orange dots; if tracked, objects appear as green lines.
3
Detector ProjectObjectives and Goals
• Principle area of research: Data model for detection of structured radiance events in remote sensor images
• Basis: case histories and best practices for discovering satellite image ‘features’ associated with extreme physical events
– Using principles for discovery of physical structures and their relations to extreme phenomena
• Data Model Extensions– Incorporate geostationary and low earth orbit data structures utilizing WCS and
WFS platforms– Examine implications of alternate data models for detection
• Cross-Correlation methods• Contour Integral methods
– Extend to radiance field structures by instrument• Proof of Concept Demonstrations
– Feature selection & ranking methods• Enhanced-V• Convective Initiation
– Tracking and Winds for forecasts– Storm event reports for determining missed events and false alarm rates
4
Detector Project Formulation
• Web Service Studies (GMU)
• Science Case Studies (UofWisc)
• Cross-Correlation methods (UofWisc)
• Contour Integral methods (GSFC)
• Tracking (NSSL)
5
Technical Status: Prototype
6
Web Service Architecture
7
Web Processing Service (WPS)
WPSWeb Processing ServiceWeb Processing Service
GetCapabilities
Communication over the web using HTTP
ExecuteDescribeProcess
Algorithms Repository…
TrackingDetection
Data Handler Repository…
…GML Data Handler
WPS-CLIENTWPS-CLIENT
WPS is a standard interface that can offer any sort of GIS functionality to clients across a network – getCapabilities, describeProcess, and execute.WPS support web-based geo-processes; Geo-processes become interoperable through Web Services;The data required by the WPS can be delivered across a network, or available at the server.
8
WPS USE Example
Here is a request to execute new-added “DetectContour” and its process result.
HTTP://localhost:8080/wps?service=“WPS”&version=“0.4.0”&request=“Execute”&store=“true”&Identifier=“DetectContour”&DataInputs=contourURL,http://localhost:8080/wps/grid.txtInput:grid.txt
Output: CloudSlice (Top)
9
NetCDF ingest added to WCS
10
GMU Web Coverage Service
WCS is a standard interface that supports the networked access to multi-dimensional and multi-temporal geospatial data
WCS provides intact geospatial data products encoded in HDFEOS, NITF, GeoTIFF, and netCDF (soon) to meet the requirements of client-side rendering, multi-source integration and analysis, and inputs to scientific models and other clients beyond simple viewers.
WCS Operations include: GetCapabilities DescribeCoverage GetCoverage
Current repository: serving 1199 products and 24GB data for all four test cases acquired from U of Wisc.
11
Standard Process to Access Data through WCS
• WCS 1.1.0 server : http://data.laits.gmu.edu/cgi-bin/wcs110• Use example: http://data.laits.gmu.edu/pli/reqnote.htm• One route to access data –
– getCapabilities (find coverage)->describeCoverage( describe coverage)->getCoverage(get coverage data)
http://data.laits.gmu.edu/cgi-bin/wcs110?service=WCS&request=getCapabilities&version=1.1.0
NETCDF:"/Data/G12IR04D20011009221100.nc":Band4_TEMP
http://data.laits.gmu.edu/cgi-bin/wcs110?service=WCS&request=DescribeCoverage&version=1.1.0&identifier=NETCDF:"/Data/G12IR04D20011009221100.nc":Band4_TEMP
http://data.laits.gmu.edu/cgi-bin/wcs110?service=WCS&version=1.1.0&request=GetCoverage&identifier=NETCDF:"/Data/G12IR04D20011009221100.nc":Band4_TEMP&format=image/geotiff&BoundingBox=-780000,-150000,1070000,1587300,urn:ogc:def:crs:EPSG::6371229
12
WCS Client Function
.
13
Use WCS Client APIgmu.csiss.wcsclient.goes.getTIFF tiffwcs = new getTIFF(“data.laits.gmu.edu/cgi-bin/wcs110”,“WCS”, “1.1.0”,“NETCDF:\"LEOIR31D20040525043000.nc\":Band31_TEMP”,“Gtiff”,“0,0,0,0,ogc:def:crs:OGC:0.0:imageCRS”,“”); int imageH=tiffwcs.getImageW();int imageW=tiffwcs.getImageH();float[] imageData = new float[imageW*imageH];imageData = tiffwcs.getDataFromImage();
Data on WCS server
imageData=
14
SwathScaled integers
1354x2030
Enhanced-V on 25 May 2004 0430 UTC (TERRA overpass)MOD021KM.A2004146.0430.005.2007018113431.hdf
1. Select & download from GSFC LAADS Website to desktop 2. Subset to binary using hdf2bin utility for display
MODIS Prototype Test Results
Sink hole
Enhanced-V peak
Detected Peaks (overshooting tops)1. Contour maximum (cloud top)
1. Orange – single level2. Red – multiple levels
2. Contour Integral max & min drawn to top1. Blue – modes (max)2. Green – nodes (min )
Swath subset Grid 30x30
Bow front
Detected Sinks (holes in anvil)1. Contour minimum (sink)
1. Orange – single level2. Red – multiple levels
2. Contour Integral max & min1. Blue – modes2. Green - nodes
15
Detected Sinks (holes)
Detected Peaks (tops)1. Contour maximum (cloud top)
1. Orange – single level2. Red – multiple levels
2. Contour Integral max & min1. Drawn from2. Blue – modes (max)3. Green – nodes (min)
Enhanced-V feature:
MODIS Prototype Contour Integral Results
Albers Equal AreaBlack Body Temperature withMB Enhancement 2243x2367
1. URL request to GMU WCS for subset on grid2. Downloaded from WCS to desktop for display
MODIS Detected Peak Events (cloud tops) from WCS Alberts Equal Area Grid
y = 18.101Ln(x) + 1960.2
1950
1960
1970
1980
1990
2000
2010
2020
2030
2040
2050
2060
2070
0 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150
area (~1KM pixels)
brightness (tens of degrees K)
0
1
2
3
4
5
6
7
8
Log. (0)
Gulf Coast
Center peak - Cloud top event #0Bow shape - Cloud top event #1
Cloud #
Subset 1km Grid 40x60
16
NetCDF Integrated Data Viewer (from UCAR) From Modis L1b Band 31 reformated to NetCDF CF metadata in LEOIR31D20040525043000.nc by U of Wisconsin McIDAS
Enhanced-V peak
Sink hole
Bow front
MODIS Test Results
17
Technical Status: Case Studies
• Translated priority GOES and MODIS IR product metadata and coordinate system information into NetCDF Climate and Forecast standard and passed through to WCS– Assigned metadata, including domain corner lat/lon and sub-
satellite point– Remapped MODIS and GOES to Albers Conic Equal Area 1 km
grid using VisAD tools and software (U of Wisc SSEC CIMSS)– Adding AVHRR navigation capability (U of Wisc SSEC)– Considered alternate data and software options with GSFC
GOES Project and with NOAA CLASS involving significantly more development effort
• Selected MODIS and GOES IR rapid scan for detecting the Enhanced V and the onset of convection
18
Technical Status:Pearson Correlation Matrices
• Developed fabricated matrix to best represent what the enhanced-V feature looks like in the imagery• Three different matrices were developed – based on results of quantitative parameters of enhanced-V features from 2003 and 2004enhanced-V seasons (total of 450 cases looked at) (Brunner et al. 2007)• GIF images of fabricated enhanced-Vs were created in Paint Shop Proand Jython code was used to create matrix of brightness temperaturevalues from GIF image (assigned each RGB color value in GIF imageto a brightness temperature)• Mean/Median Enhanced-V Fabricated Matrix (30X30 pixel matrix MOD 3A)
• Maximum Enhanced-V Fabricated Matrix (50X50 pixel matrix MOD 3B)
• Minimum Enhanced-V Fabricated Matrix (15X15 pixel matrix MOD 3C)
19
Mean/Median Enhanced-V Fabricated Matrix (30X30 pixels)
TMIN (coldest cloud top temperature) – 201 KTMAX (warmest cloud top temperature) – 217 KTDIFF (warm-cold couplet) – 16 KDIST (distance between warm and cold location) – 10 KMDISTARMS (averaged distance of both V-arms) – 36 KMANGLEARMS (angle between both V-arms) – 75 Degrees
20
Enhanced-V Algorithm Inputs
• McIDAS AREA file of Low Earth Orbit (LEO) satellite data (currently set up to input 150X150 pixel region), given a line/elem value in image as upper left point of desired region (line/elem/brightness temperature value at each pixel in region is input)
• Enhanced-V Fabricated Matrix of brightness temperatures (have three different versions), ASCII file of brightness temperatures in matrix
21
Enhanced-V Algorithm Statistical Correlation Code: Output
* Algorithm takes 150X150 pixel region and steps through this region one pixel at a time while comparing 10.7 micron brightness temperatures to a fabricated enhanced-V matrix of brightness temperatures (looks for a similar pattern in brightness temperature values)
* ASCII output file of line/elem/correlation value at every pixel
* ASCII filtered output file of line/elem/correlation value of all pixels that exceed a certain correlation value threshold (such as >= 0.5 for example)
22
LEO Enhanced-V Proof of Concept Test Cases
• Three Advanced Very High Resolution Radiometer (AVHRR) enhanced-V cases were selected to test the enhanced-V statistical correlation algorithm
* Case 1: 25 May 2004 0424 UTC over Oklahoma* Case 2: 10 May 2004 0042 UTC over South Dakota/Nebraska/Iowa border* Case 3: 6 May 2003 2218 UTC over northeastern Oklahoma
• The enhanced-V cases are East-West oriented enhanced-Vs and theenhanced-V fabricated matrices are East-West oriented (“simple” case)
• In addition, a null case was used to test the enhanced-V algorithm for thunderstorms (no enhanced-V features though) over southwest Kansas on 10 May 2004 at 0042 UTC; Desired outcome is no detects of enhanced-V features for the null case
23
Enhanced-V Test Case #1 - 25 May 2004, 0424 UTC
24
Summary Table of Results (3 cases, 3 enhanced-V matrices, 3 statistical correlation value thresholds; total of 27 runs)
Case # Matrix
MOD
Statistical
Corr.
threshold
EV
Detected
(Y or N)
False
Detect
(Y or N)
Statistical
Corr.
threshold
EV
Detected
(Y or N)
False
Detect
(Y or N)
Statistical
Corr.
threshold
EV
Detected
(Y or N)
False
Detect
(Y or N)
1 3A (30x30)
0.5 N N 0.4 N Y 0.3 Y Y
1 3B (50x50)
0.5 N N 0.4 Y N 0.3 Y N
1 3C (15x15)
0.5 N N 0.4 N Y 0.3 N Y
2 3A (30x30)
0.5 N Y 0.4 N Y 0.3 N Y
2 3B (50x50)
0.5 N N 0.4 N N 0.3 N N
2 3C (15x15)
0.5 N N 0.4 N N 0.3 Y Y
3 3A (30x30)
0.5 Y N 0.4 Y Y 0.3 Y Y
3 3B (50x50)
0.5 N N 0.4 N N 0.3 Y N
3 3C (15x15)
0.5 N N 0.4 N N 0.3 Y/N Y
25
Case 3: MOD3A: 0.5 Run
• (+) Enhanced-V detected correctly
26
Case 1: MOD3A: 0.5/0.4/0.3 Runs
0.5 Run – No enhanced-V detected0.4 Run – False detect in anvil region0.3 Run – False detect in anvil and clear skyregions; Enhanced-V detected correctly aswell
27
Case 2: MOD3A: 0.5/0.4/0.3 Runs
False detects of enhanced-V in anvil region inall runs; the number of false detects increasesas the statistical correlation value thresholddecreases from 0.5 to 0.3
28
Summary Table of Results (1 null case, 3 enhanced-V matrices, 3 statistical correlation value thresholds; total of 9 runs)
Case # Matrix
MOD
Statistical
Corr.
threshold
False
Detect
(Y or N)
Statistical
Corr.
threshold
False
Detect
(Y or N)
Statistical
Corr.
threshold
False
Detect
(Y or N)
Null 3A (30x30)
0.5 N 0.4 Y 0.3 Y
Null 3B (50x50)
0.5 N 0.4 Y 0.3 Y
Null 3C (15x15)
0.5 N 0.4 N 0.3 Y
• For Null Case using MOD3A matrix; false detects of enhanced-V found in anvil region for statistical correlation thresholds of 0.4 and 0.3
2D Array BT(6.7)2D Array BT(10.7)UL Image Line/Elem
BT(6.7) – BT(10.7)To Isolate Overshooting Top Pixels
STEP 1
2D Array BT(6.7) – BT(10.7)2D Array BT(10.7)2D Array Image Line Values2D Array Image Elem Values
STEP 2
Thermal Couplet AnalysisFor Each Identified Overshooting Top Pixel[BT(6.7) – BT(10.7) ≥ 4K];Distance And Temperature DifferenceThreshold Checks Performed:Distance ≤ 25kmBT(10.7) Difference ≥ 12K And ≤ 35KBT(6.7) – BT(10.7) ≥ -5K Of Potential Warm Pixel
Additionally, Angle Orientation Of Detected Thermal Couplet Is Calculated.
1D Array Thermal Couplet Image Line Locations(Use Overshooting Top Pixel Location)
1D Array Thermal Couplet Image Elem Locations(Use Overshooting Top Pixel Location)
1D Array Thermal Couplet Values1D Array Angle Orientations Of Thermal Couplets
Enhanced-V Statistical Correlation AlgorithmSearch For Enhanced-V Features Around Thermal Couplet Regions:Orient Enhanced-V Fabricated Matrix In Direction Of Thermal Couplet Angle OrientationSearch 50x50 Pixel Box Around Overshooting Top Pixel Location
STEP 3
Image Line/Elem Locations Of Identified Enhanced-V PixelsStatistical Correlation Values Of Identified Enhanced-V Pixels
30
MODIS Test Cases for Revised Enhanced-V Algorithm/Pre-
Processing (Using Methodology in Process Flow Chart on Previous Slide)
Case 1: 25 May 2004 0430 UTC (TERRA overpass)* Enhanced-V over OklahomaCase 2: 7 April 2006 1702 UTC (TERRA overpass)* Numerous enhanced-Vs over Tennessee and KentuckyCase 3: 7 April 2006 1842 UTC (AQUA overpass)* Numerous enhanced-Vs over Tennessee and KentuckyCase 4: 12 October 2001 overpasses* Enhanced-Vs over southern Plains, during GOES-12 SRSO Test period Case 5: 24 April 2007 overpasses* Enhanced-V over southwest Texas that was associated with Eagle Pass killer tornadoCase 6: 9 October 2001 overpasses* Enhanced-Vs over southern and central Plains, during GOES-12 SRSO Test periodCase 7: 10 May 2006 overpasses* Numerous enhanced-Vs over Texas and Gulf Coast, during GOES-12 RSO
31
Upcoming Conference Presentations/Publications
• An oral presentation will be given at the upcoming Joint EUMETSAT/AMS Satellite Conference in Amsterdam on the enhanced-V detection algorithm work
• Peer-reviewed paper entitled “A Quantitative Analysis of the Enhanced-V Feature in Relation to Severe Weather” has been accepted for publication in Weather and Forecasting
32
GOES-12 CI Nowcasting Using 1-Min Resolution Data
Mean cloud top cooling rate for small and towering cumulus clouds are computed over .1 x .1 degree boxes over a 5-minute period
This allows cloud motion during this image sequence to be disregarded, allowing us to better identify developing cumulus clouds with fewer false alarms than the previously described method
30 MINS LATER
33
Cooperation and Follow-on
• The objective overshooting top and enhanced-V algorithm may satisfy future GOES-R Advanced Baseline Imager (ABI) requirement
• Thunderstorm feature detection is complementing NASA Advanced Satellite Aviation-weather Product (ASAP) research toward improving airline safety through use of satellite data
• Overshooting top detection will also provide possible signal of turbulent atmosphere which can be integrated into the NOAA Aviation Weather Center Graphical Turbulence Guidance (GTG) next generation product, used operationally by NOAA and FAA
34
Milestone StatusMilestone
Months after Task Start Status
Data sources and interface review January-07 completed in MarchDesign and architect the virtual sensor platform February-07 completed in March
Modify WCS server to work with GOES data and in-situ reports March-07
completed in April, revised to use standard projection (Alberts Equal Area)
Event detection data model design review April-07
conducted through a series of telecons and meetings at GMU and GSFC, completed in March
Install and configure virtual sensor platform June-07
software modifications completed in May, hardware upgrades delayed until after server relocation at GMU - in August
Integrate prototype detection and tracking algorithms with GOES, MODIS sample data July-07 started, no issuesDevelop automated algorithms for Enhance V September-07 started, on scheduleAssess convection initiation detection skills September-07 started, on scheduleFinish tests on GEO and LEO case study datasets November-07
35
Cost and Schedule Status• One Year funding: $382,913• Direct for NASA GSFC PI• GMU/U of Wisc Grants, NSSL agreement
• Commitments $316k Jan 2007• Cumulative Obligations: $316k Mar 2007
• The project is currently spending below the plan due to accumulation of short delays:– Setup grants and awards– Data source/format– Temporary loss of the lead engineer at GMU– Location of GMU processing facility– $20K equipment purchase delayed until August
36
Cost and Schedule StatusEvent Detector Cost Phasing
$0
$50,000
$100,000
$150,000
$200,000
$250,000
$300,000
$350,000
Dec-06 Jan-07 Feb-07 Mar-07 Apr-07 May-07 Jun-07
Labor
Equipment
Cum. Cost Plan
Funding
Cum. Actual (est.)