Phenomenology Metric Development for SAR Scene Modeling Tools

Patricia A. Ryan, Kelce S. Wilson

Air Force Research Laboratory, Wright-Patterson AFB, OH 45433

ABSTRACT

Synthetic Aperture Radar (SAR) image modeling tools are of high interest to Automatic Target Recognition (ATR)algorithm evaluation because they allow the testing of ATRs over a wider range of extended operating conditions (EOCs).Typical EOCs include target aspect, target configuration, target obscuration, and background terrain variations. Since thephenomenology fidelity of the synthetic prediction techniques is critical for ATR evaluation, metric development forcomplex scene prediction is needed for accurate ATR performance estimation. An image domain hybrid prediction techniqueinvolves the insertion of a synthetic target chip into a measured image background. Targets in terrain scenes will bepredicted and compared with similar measured data scenarios. Shadow region histograms and terrain region histograms willbe used to develop some first generation metrics for phenomenology validation of hybrid SAR prediction techniques.

Keywords: Synthetic Aperture Radar (SAR) Modeling, Validation Metrics

1. HYBRID SAR PREDICTION TECHNIQUE OVERVIEW

Figure 1 shows an image domain technique for inserting a synthetic target chip into a measured background image developedby DEMACO, Inc. [1]. The Xpatch signature prediction code is used off-line to predict a three dimensional scattering centerrepresentation ofthe target signature from a high fidelity target geometry model [2]. The 3D scattering centers are generatedfor the case of a target on a flat ground plane to insure that target/ground interactions are included in the prediction. Apredicted SAR image of the target is reconstructed from the 3D scattering center representation, and a target/shadow mask isgenerated from an on-line ray trace of a low resolution version of the 3D target geometry. The on-line SAR prediction of thetarget is superimposed on the target/shadow mask, and mask statistics are applied to the inserted synthetic target chip. Sensoreffects including noise, an approximate point spread function, and errors due to motion compensation and waveformcorrelation are applied when inserting the chip into the measured background. Tool inputs include the sensor noise floor anda level for the integrated sidelobe ratio (ISLR) used to simulate motion compensation and waveform correlation effects.

2. METRIC TOOL DEVELOPMENT

Figure 2 shows the planned transition for SAR image prediction tools for ATR Performance evaluation applications. The firststep in model validation is to develop metrics of comparison for assessing the quality of the data produced by the syntheticprediction technique. Although visual assessments of data quality are valuable, the development of metrics related to thephenomenology allows a more automated comparison of measured and synthetic data. The automated comparison isnecessary for large data sets encompassing the wide range of aspect angles typically needed to fully validate the modelingprocess. As synthetic predictions are calibrated, model process validation techniques concentrate on metric basedcomparisons of calibrated measured and synthetic data and visually comparing a geometric model with photographs anddesign drawings.

Ideally, phenomenology based validation techniques are complete enough to insure model performance in an ATR.However, as signature modeling techniques typically do not model radar sensor effects including noise, sensor positionuncertainty, and sensor motion, an ATR specific validation is usually required to insure that the modeled data simulates themeasured data through the entire system process. In addition, the information processing techniques used in some ATRsoften removes or distorts information content from signature data. An ATR specific validation compares synthetic andmeasured data products used by the ATR. Usually, the final ATR specific validation involves training the ATR with syntheticand measured data, and comparing confusion matrix results when the ATR is tested with measured data.

Modeling processes included in the hybrid prediction technique include the creation of a CAD geometry model of the target,Xpatch 3D ISAR prediction, scattering center extraction, and the insertion process. The insertion process includes on-line 2DSAR reconstruction from scattering centers, target/shadow mask generation and the approximate modeling of sensor effectsfrom measured image header parameters. As other efforts are exploring ground target CAD geometry and 2D SAR

Part of the SPIE Conference on Algorithms for Synthetic Aperture Radar Imagery VI582 Orlando, Florida • April 1999

SPIE Vol. 3721 • 0277-786X/99/$1 0.00

reconstruction from scattering centers, this effort focuses on validating the simulated shadow region. The shadow region wasdetermined to be of most interest initially for assessing the target insertion process because ringing and multi-bounce fromthe target and sensor effects are most evident in this region.

ModelingTool

Development

ModelATRValidation

ATR PerformanceEvaluationDatabaseDevelopment

Figure 2: Modeling Tool Transition to ATR Performance Evaluation Applications

A measured target chip is shown in Figure 3. A segmentation algorithm was developed to identify the target region, targetboundary, and shadow region in the image chips. The input to the algorithm is a threshold and known point in the shadowregion. As shown in Figure 3. the algorithm traverses the shadow regions in four directions. The algorithm startsby steppingin one direction. If an adjacent point is less than the known point and the threshold, it is declared an inner region point. If itis greater than the known point or the threshold, it is declared a possible border point. A boundary region is declared whentwo successive possible border points are identified. Both points are declared border points. The algorithm then steps inanother direction. As two successive points that are greater than the input threshold are required to declare a border point,pixels with returns greater than the input threshold are possible within the resulting defined shadow region.

583

Low ResGeometry

Ray TracedTarg Mask

High ResGeometry

MaskStatstics

SensorEffects

3D Scattering PredictedCenters SAR image

Figure 1: Image Domain Target Chip Insertion for Hybrid SAR Scene Prediction

Hybrid image

ModelingProcess

hdation—0

This iterative process results in the definition of a region boundary. a punctured regionand some interior unknown points as

shown in Figure 4. The punctured region capability models the effects of target multi-bounce as part of the target region so

that the sensor effects modeling in the shadow region can be better isolated. Low level regions in the image caused by target

self-shadowing are included in the shadow region. The sensor effects are approximated overthe entire inserted chip. Thus.

the simulated processing should also be compared to the actual measured data in the low return regions of the image. rather

than just the area defined by the chip insertion process as a calculated shadow mask region. On the second pass of the

segmentation algorithm, interior points that are not adjacent to a punctured region boundary are reclassified as interior points

as shown in Figure 4.

First Pass

Figure 3: TargeLShadow Segmentation

PuncturedRegion

InteriorPoints

ShadowRegion

584

Figure 4: Target/Shadow Region Segmentation Algorithm

TargetRegion

Second Pass

3. METRIC DEVELOPMENT

Figure 5 shows the two threshold criteria used in segmenting the measured and synthetic shadow regions. The first choice ofthe threshold criterion is to use the same input threshold level in dB for segmenting the target/shadow region for both themeasured and synthetic target chips. To help in automating threshold selection, a program was developed to iterate overpossible thresholds and compute the size ofthe measured target chip shadow region in pixels. A plateau region was expectedto help identify the threshold, however in most cases, the plateau region was very small or non-existent. Thus, the initialchoice of a threshold is a manual process. The second input threshold criterion holds the number of measured and hybridshadow pixels equal while allowing different input threshold values.

Four targets at various aspect angles were used in the metric development experiment. Figure 6 shows sample imagery,threshold selection curves, segmented shadow regions and shadow region histograms for a representative measured andsynthetic inserted target chip. The measured target chip is shown in Figure 6a, and the inserted target chip is shown in Figure6b. Figure 6c shows the size of the shadow region in pixels as a function of input threshold level for both the measured andsynthetic chip. The shadow region closest to the target image includes target multi-bounce effects in addition to sensor noiseand impulse response effects. The input threshold for shadow region segmentation is chosen to isolate sensor noise andimpulse response effects approximated in the target chip insertion process, and the measured curve in Figure 6c is used toselect the input threshold used in the shadow region segmentation algorithm.

An input threshold of—17.2 dB was used in the shadow region segmentation algorithm to obtain the measured and syntheticshadow regions shown in Figures 6d and 6f. The measured shadow region includes 665 pixels, and the synthetic shadowregion includes 742 pixels. The size of the synthetic shadow region compared to the size of the measured shadow region inpixels for the same input threshold level is a metric that lends insight into the accuracy of the target/shadow region maskgeneration process. Since the outer edges of the shadow region include terrain scattering, this metric also lends insight intothe accuracy of the sensor effects processing approximation used to model the blurring of the synthetic shadow region intothe terrain.

MeasuredShadowRegion

Hybrid Shadow RegionInput T_meas = T_syn

Hybrid Shadow RegionNP_meas = NP_syn

Figure 5: Threshold criteria used to defme shadow region

585

D) MeasuredShadow Region

E) Synthetic ShadowRegion: Tm — Ts

U)a)>(

00a)-oEz

F) Synthetic ShadowRegion: NPm NPs

(1) MeasuredShado Histogram

11) Synthetic Shado'.histogram Tni — Ts

I) Synthetic ShadovHistogram: NPm NPs

586

Figure 6: Sample image chips. threshold selection criteria, shadow regions and shadow region histograms.

A) Measured B) SyntheticTarget ('hip Target Chip

-18-16Threshold (dB)

C) ThresholdSelection

-20

12001000800600400200

C,)

a)

0a)

Ez

I� 1(X)a)xa-o 5()a)0E

0-25 -2CC

Pixel Level (dB)

1(X)

50

—1 0

I (X)

So

0)

-25

Pixel Level (dB) Pixel Level (dB)

Table 1: Shadow Region Histogram Statistics

MeasuredSynthetic

Input Tm = Ts = -17.2 dBNP = 742

SyntheticInput Ts = -17.9 dBNPm =NPs = 665

Mean -18.0dB -18.2dB -18.3dB

Std. Dev 1.44 1.5 1.42

The histograms for the measured and synthetic shadow regions resulting from using an input threshold of—17.2 dB are shownin Figures 6g and 6h, and the mean and standard deviation are shown in Table 1 .Because the shadow segmentation algorithmis more robust than a constant threshold, the shadow region will include pixels with returns greater than the input thresholdused for shadow segmentation. The mean and standard deviation of the shadow region are metrics that measure the accuracyof the sensor noise approximation used in the insertion process. To a lesser extent, these statistics also measure the accuracyof the sensor effects processing approximations used to model the blurring of the synthetic shadow region.

The similarity of the shadow region histogram shapes is another metric that can be used to assess the insertion process. Forthis example the measured and synthetic shadow region histograms shown in Figure 6g and 6h are most different for thelower pixel returns. The synthetic shadow region histogram includes a higher number ofthe lower return pixels.

The synthetic shadow region shown in Figure 6f was obtained by varying the input threshold to the segmentation algorithmuntil the number of pixels in the shadow region was equal to the number of pixels in the measured shadow region shown inFigure 6c. The input threshold used to segment the equal pixel count shadow region was —17.9 dB. As this is the input to thesegmentation algorithm, it does not guarantee that all pixels in the shadow region will have returns less than —17.9 dB. Forthe measured/synthetic case shown, the shadow region statistics did not show significant sensitivity to the input thresholdchoice used in the shadow segmentation algorithm.

The histograms for the measured and synthetic shadow regions resulting from using a pixel count of 665 are shown inFigures 6g and 6i, and the mean and standard deviation are shown in Table 1 . The mean and standard deviation of theshadow region are metrics that measure the accuracy of the sensor noise approximation used in the insertion process. To alesser extent, these statistics also measure the accuracy of the sensor effect modeling approximations used to model theblurring of the synthetic shadow region. The similarity of the shadow region histogram shapes is another metric that can beused to assess the insertion process. For this example the measured and synthetic shadow region histograms shown in Figure6g and 61 are most different for the lower pixel returns, and the synthetic shadow region includes a higher number of thelower return pixel.

4. CONCLUSIONS AND RECOMMENDATIONS

Image domain target insertion techniques are inherently limited by the knowledge of the sensor effects processingapproximations. Sensor noise parameters are often found in image headers or data dictionaries, however complex sensorimpulse response functions are not commonly distributed with measured imagery data sets. As complex imagery is acommon form of distributing SAR data, insertion techniques that can approximate sensor effects are of great interest. As themodeling of sensor effects is most evident in the shadow region, quantitative comparisons of image returns in the shadowregion lend insight into the accuracy ofthe insertion modeling process.

As a simple threshold was not adequate for isolating the shadow region, a robust shadow region segmentation algorithm wasdeveloped. Preliminary metric development experiments show that the algorithm can be used to segment measured andsynthetic shadow regions for quantitative comparisons. The choice of an input threshold to the segmentation algorithm ischosen manually. Additional investigations to automate the input threshold are needed to automate the shadow regionsegmentation tool. The segmentation tool can be used to segment the target region and background region for further metricdevelopment and validation studies.

587

The first and second order statistics metrics allow the accuracy of the noise model coupled with the accuracy of sensor effectmodeling approximations including motion compensation, point spread function, and waveform correlation effects to beassessed. A Chi Square Test may be a useful metric for assessing the similarity between measured and synthetic shadowregion histograms. Additional metrics that decouple the sensor noise modeling from the general sensor effects should beexplored.

This metric development experiment used the default input parameters to the SAR insertion tool. Additional parameterstudies varying the input parameters used to produce the synthetic target chips are required to completely decouple the noisemodel effects from the sensor processing model effects. The tools and metrics developed applied to an input parameter studywill be used to fully validate the tool. Additional tool development to support the tool validation is in progress.

5. ACKNOWLEDGEMENTS

The image domain synthetic chip insertion tool development was sponsored by the OSD Office of the Director, DefenseResearch and Engineering (DDRE). The work was supported under their project for Automatic Target RecognitionAssessment Technology.

6. REFERENCES

1. Sullivan, D., D. Andersh, T. Courtney, N. Buesing, and P. Jones, "Development of SAR Scene Modeling Tools for ATRPerformance Evaluation", To be published.

2. Bhalla, R. , H. Ling, J. Moore, D. J. Andersh, S. W. Lee, and J. Hughes, "3D Scattering Center Representation ofComplex Targets Using the Shooting and Bouncing Ray Technique: A Review, " IEEE Antennas and PropagationMagazine, Vol. 40, No. 5, October 1998, pp. 30 —39.

588