INSERTION OF SYNTHETIC FEATURES IN SAR CCD IMAGERY

Eric Turner ∗

University of California, BerkeleyBerkeley, California 94720 USA

elturner@eecs.berkeley.edu

Rhonda D. Phillips, Carol Chiang, Miriam Cha

MIT Lincoln LaboratoryLexington, Massachusetts 02420 USA

{rhonda.phillips,chiang,miriam.cha}@ll.mit.edu

Keywords: Put Keywords here. A list of no more than 5keywords or keyword phrases will identify your paper inindices and databases. Do not use the words ‘computer’,‘simulations’, ‘model’, or ‘modeling’ since these are all as-sumed. Use text of about 10 points in size, but never smallerthan 8 points. Use boldface for the word ‘Keywords’ thenregular text for the keywords themselves as in the followingexample. Example keywords: Discrete event simulation,DEVS, environmental science, decision support systems

AbstractSynthetic Aperture Radar Coherent Change Detection (SARCCD) images are highly sensitive change maps produced bymultiple SAR images. The noise and characteristics of SARand CCD result in SAR CCD images being challenging to in-terpret, making human analyst training essential. Simulatinga SAR and CCD dataset with realistic noise is an unsolvedproblem, and collecting real data with a wide variety of sce-narios is challenging and error prone.

In this paper, we introduce two algorithms to insert tiretracks and footprints (simulated activities) into existing, non-simulated CCD images. In the first approach, we directlyinsert synthetic tracks that are modeled to characterize theappearance of tire tracks and footprints into existing CCDimages. In the second approach, we use a phase-shift tech-nique prior to CCD formation, providing more realistic ac-tivity signatures at an increased computational complexity.The position of the tire tracks and footprints are determinedusing a simulation software application such as Joint Semi-Automated Force (JSAF), allowing this SAR CCD simula-tion capability to be integrated into a larger overall system. Inthis paper, we provide a full description of the algorithms andthe integration into a software system, and we provide experi-mental results showing the simulated CCD image output withcomparisons between the two approaches.

∗This work is sponsored by the United States Air Force under Air ForceContract FA8721-05-C-0002. Opinions, interpretations, conclusions and rec-ommendations are those of the author and are not necessarily endorsed by theUnited States Government.

1. INTRODUCTIONSynthetic Aperture Radar Coherent Change Detection

(SAR CCD) is a sensitive change detector capable of detect-ing small disturbances in scenes such as tire-tracks and smallobjects. Applications for SAR CCD include search and res-cue operations and activity monitoring. The CCD capabilityis unique from other monitoring sensors in that it is forensicin nature (detects activities that occurred in between two datacollections), and therefore does not require data collectionduring the activity of interest. CCD is also unique comparedwith typical change detectors because phase history is used inaddition to magnitude, enabling the detection of changes thatare on the order of the radar wavelength rather than the imageresolution.

The process of collecting and forming CCD images is ex-pensive and difficult. There is no guarantee that resultingimages will be high quality, and there is no guarantee thatscripted activities (such as driving vehicles through the sceneto generate tracks) will be visible in CCD. Although mod-els exist to simulate SAR images, there are no existing mod-els for generating CCD images with interesting features. Thecontribution of this paper is to introduce a method for in-serting interesting features into existing CCD images, and toevaluate the features based on similarity to real data. Section2 provides background on SAR CCD processing. Section 3details the different algorithmic approaches to inserting syn-thetic signatures into CCD. Section 4 provides analysis toshow that synthetic tracks are similar to authentic tracks, andSection 5 concludes the paper.

2. BACKGROUNDSAR CCD uses a traditional interferometry technique that

utilizes phase information of two (or more) SAR images col-lected from nearly identical geometries to detect small (sub-wavelength) changes in ground surface height between col-lections. These small height differences result in a decorrela-tion or low coherence in the areas where changes occur be-tween images. This sensitive detector is capable of detectingsmall ground disturbances such as tire tracks. Change is de-

1

tected in areas of low coherence between the two SAR im-ages.

The formation of a CCD image begins with the collectionand formation of two or more SAR images from identicalcollection geometries (using identical flight paths) [1]. Aut-ofocus is a typical processing step in the formation of SARimages, and care should be taken to autofocus both SAR im-ages together so that pixel intensities for the same featureswill match [2]. In order to perform accurate change detection,both images should be registered with sub pixel accuracy asdescribed in [2]. As collection geometries are not identicalin practice, aperture trimming in Fourier space is necessaryto correct slight phase differences caused by differences incollection angles [2]. Once these non-trivial processing stepsoccur, the coherence between two complex pixels in two SARimages can be estimated using the complex correlation coeffi-cient in a neighborhood around the pixels [1]. In this manner,the coherence is calculated for each set of overlapping pixels,and the CCD image is formed.

3. TRACK INSERTION ALGORITHMSThis Section describes two different approaches for insert-

ing synthetic tracks/targets into CCD images. The first ap-proach requires only a CCD image, while the second ap-proach requires the SAR images used to form a particularCCD image. The first approach uses a model to describe theappearance of tracks in a CCD image. Once the model is de-fined, it can be used to generate new tracks by inserting lowcoherence pixels in the pattern of a track. The disadvantageof this approach is that the tracks may not statistically matchthe rest of the image without carefully setting model param-eters. The second approach randomly alters the phase of oneSAR image to match the geospatial pattern of a track. Whenthe two SAR images are processed to form CCD, the tracknaturally appears and statistically matches the rest of the im-age. The disadvantage of this approach is that both complexSAR images are required. The two algorithms are describedin detail below.

3.1. Description of Post-CCD AlgorithmGiven a raster image γ ∈ [0,1]m×n that represents the result

of calculating the Coherent Change Detection (CCD) of twoSynthetic Aperture Radar (SAR) images, along with a list ofpixel locations, the goal is to insert a realistic synthetic trackat those pixel locations in the original image. The followingdescribes the model used to characterize the appearance oftire-tracks and footprints in SAR CCD images. The imagesused in this report were collected using a Boeing CompactKu-band radar.

3.1.1. Tire-tracksObserving Figure 1, tire-tracks in CCD imagery have lower

coherence than the surrounding terrain, and appear to varyrandomly in intensity. This variance can be explained by en-vironmental properties, such as background coherence, ge-ometry of the vehicle, nature of the terrain (hard or soft soil),as well as radar characteristics.

Figure 1. Example tire tracks in a SAR CCD image.

Our goal is to parameterize the appearance of tire-tracks,providing a realistic model without unnecessary complexity.Upon inspection, one can characterize the pattern of a tire-track with a sequence of rectangles in-line with the path ori-entation (see Figure 2). By determining the probability dis-tributions of all parameters that determine the placement andgeometry of these rectangles (or “blocks”), one can then cre-ate new tracks with the same characteristics.

Figure 2. Close-up of tiretracks in SAR CCD image (left)with outline of each block (right)

Given the assumption that there are two parallel tire-trackswith separation s (measured in pixels), Figure 3 represents allinformation necessary to define the position of a track blockgiven the position of the previous block. Table 1 defines the

set of parameters along with example distributions for eachparameter of CCD images with a pixel resolution of 0.15 me-ters/pixel. These parameters were chosen by experimentationfor a specific radar geometry set.

Note that the intensity of each block is not represented bya constant-height rectangle, but rather is initially representedas a pyramid with an offset peak. The location of this peakwithin the pyramid is represented by the parameters mx andmy. Note that this peak will represent a maximum intensityvalue.

Using the above parameters, the pixel intensities of an in-dividual tire-track can be generated in a blank image (back-ground of zero intensity). Note that the track componentsmust be sheared and rotated to match the specified pathscurvilinear profile. Once the resulting image is created, itcan be passed through a rudimentary low-pass filter to cre-ate more realistic block geometry.

Note that the average coherence along a track is lower thanthe local average image coherence, even in locations that donot contain a noticeable block. This effect can be modeled byadding additional layers of blocks attenuated by some factor.Thus two separate tracks are generated for each path, wherethe second track is scaled, and the results are added together.The intensity-scaling factor for the second set of blocks usedin the above example is 0.4.

At this stage, a track with positive intensities exists on abackground of zero intensity. Note that the track so far is in-dependent of the coherence in the original image. In order tomodel environmental effects on the tire-tracks, this image canbe scaled element-wise by the original CCD image, γ. The re-sult is that areas of low coherence in the CCD image are lessaffected by the track disturbance. Finally, the track image issubtracted from γ, giving the modified CCD image. If we letTi ∈ [0,1]m×n be defined as a single layer of blocks, and h bedefined as a low-pass filter, then the modified CCD image, γ′,can be defined as:

γ′ = γ − γ ×

((T1 +0.4 T2)∗h

)(1)

Where × represents element-wise multiplication, and ∗represents two-dimensional convolution. Figure 4 shows theresult of this process, where each of the Random Variables inTable 1 are modeled with Uniform distributions.

3.1.2. FootprintsFootprints can be formed using the same model as above,

with modified distributions for each parameter. Also note thatthe parameters W and s are not necessary in this case. Table 2is an example set of parameters for footprints. Again, the ex-ample values given were chosen experimentally.

Note that footprints are much fainter that tire-tracks. Fig-ure 5 shows an image with three sets of footprints. The top

Figure 4. Artificial track (with parameters specified in Ta-ble 1) inserted into Figure 1

Figure 5. Real footprints (top two horizontal lines) with ar-tificial footprints (bottom line)

two are real footprints, while the bottom is a synthetic trackthat is an example of the above parameter set.

3.2. Description of Pre-CCD AlgorithmA CCD image is computed from two existing SAR images,

S1,S2 ∈ Cm×n. The earlier image S1 is often referred to asthe “reference” image while S2, which was taken after somechange occurred, is referred to as the “mission” image. Thecoherence of a pair of complex SAR images is defined as:

γ(x,y) =E[S1(x,y)S2(x,y)∗

]√

E[| S1(x,y) |2

]E[| S2(x,y) |2

] (2)

Where E[...] represents statistical expected value. Since theCCD algorithm is performed on a finite-sized image, an esti-mator for the expected value is used,

γ̂(x,y) =ΣN

k=1S1(xk,yk)S2(xk,yk)∗√(

ΣNk=1 | S1(xk,yk) |2

)(ΣN

k=1 | S2(xk,yk) |2)

(3)Note that the numerator effectively performs an inner prod-

uct on a sub-region of the two respective regions, whilethe denominator normalizes by the magnitude of these sub-regions. Since the inner product is proportional to the mag-nitude of each sub-image, the resulting value is primarily de-pendent on the relative phase between the two images. Thusif one wanted to artificially reduce the coherence of a partic-ular region, one might naı̈vely perform a uniform phase-shift

Figure 3. Definition of track parameters

Table 1. Tire-Track Parameters and Example DistributionsParameter Description Mean VarianceW Width of track 5 (pixels) n/as Distance between tiretracks 2 (pixels) n/ad Along-track distance between blocks 4 (pixels) 12 (pixels2)x Cross-track offset of block 8 (pixels) 5.33 (pixels2)l Length of block along track 6 (pixels) 1.33 (pixels2)w Cross-track width of block 8 (pixels) 5.33 (pixels2)m Maximum intensity of block (measured from zero intensity) 0.3 (intensity value) 0.0133mx The along-track location of the peak of block intensity 0.8 (percentage of l) 0.03my The cross-track location of the peak of block intensity 0.5 (percentage of w) 0.00333

Table 2. Footprint Parameters and Example DistributionsParameter Description Mean Varianced Along-track distance between blocks 15 (pixels) 40.33 (pixels2)x Cross-track offset of block 0 (pixels) 1.33 (pixels2)l Length of block along track 4.5 (pixels) 2.083 (pixels2)w Cross-track width of block 4.5 (pixels) 2.083 (pixels2)m Maximum intensity of block (measured from zero intensity) 0.2 1.267mx The along-track location of the peak of block intensity 0.8 (percentage of l) 0.03my The cross-track location of the peak of block intensity 0.5 (percentage of w) 0.00333

of the values in that region in the mission SAR image. Thisapproach is not an effective way to adjust the magnitude of γ.Consider the following example:

3.2.1. Constant Phase-Shift ExampleAssume the simple case where the mission and reference

images are perfectly aligned: ∀x,y : S1(x,y) = S2(x,y). Thismeans the CCD image has a magnitude of 1 at every pixel,indicating perfect coherence. The goal is to artificially lowerthe coherence in a particular region by shifting the phase ofthat region by some angle, ∆θ.

Define S1(x,y) = |S1(x,y)|ei∠(S1(x,y)) and S2(x,y) =|S2(x,y)|ei∠(S2(x,y)). For a pixel (x,y) contained in this mod-ified region (sufficiently far from the edge of the region), it

can be shown:

γ̂(x,y)=ΣN

k=1 |S1(xk,yk)|ei∠(S1(xk,yk)) |S2(xk,yk)|ei(∆θ−∠(S2(xk,yk)))√(ΣN

k=1 | S1(xk,yk) |2)(

ΣNk=1 | S2(xk,yk) |2

)(4)

Since ∀k ∈ [1,N] : S1(xk,yk) = S2(xk,yk):

γ̂(xk,yk) =ΣN

k=1 |S1(xk,yk)|2ei∆θ

ΣNk=1 | S1(xk,yk) |2

= ei∆θ (5)

Thus the result still has a magnitude of 1. Since the phaseof the coherence is not typically used, this technique is not aneffective way to artificially insert disturbances. This method

will reduce the coherence in the image, but only due to edgeeffects along the region of disturbance.

3.2.2. Algorithm DescriptionTo achieve realistic disturbances in CCD using a phase-

shift approach, a random variable is added to the phaseof each pixel desired instead of a deterministic phase-shiftwithin the selected region of the SAR mission image. Byvarying the phase shift pixel-by-pixel, deconstructive interfer-ence is introduced in the summation in the numerator of (3),while the value of the denominator remains constant, effec-tively reducing the coherence. In practice, additive Gaussianwhite noise was chosen for this algorithm in order to mimicnatural disturbances within the SAR image. By adjusting thevariance of this noise, the intensity of the artificial track canbe adjusted, allowing flexible control of image editing.

The following is an example MatLab function that will in-ject a set of tire tracks into a mission SAR image, prior toCCD formation. It assumes the existence of a create mask()function that specifies which pixels are covered by the tire-track.

3.2.3. Example usageFigure 6 compares the results of the above approach, as

well as a constant phase shift. Note that the vertical lines onthe left-hand side were generated using Gaussian noise of in-creasing variances and the right-hand side lines were gener-ated using a constant phase shift with increasing shift angles.The center of each horizontal track has high coherence, withlow coherence around the edges. When adjusting the phase ofthe modified region by a constant amount, only edge effectsalong that region appear in the resultant CCD magnitude.

Figure 6. Artificial insertion by Gaussian phase shift (left)and deterministic constant phase shift (right)

Selecting an intensity that is appropriate for the image, onecan mimic the appearance of tire-tracks. In Figure 7, thereare two sets of tire-tracks. The left-hand set exists in the orig-inal image, while the right-hand set is artificially insertedinto the image in post-processing using the above technique.

These artificial tracks display the same pattern as those cre-ated using the post-CCD algorithm described in Section 3.1,even though the phase mask does not contain any informationabout “blocks”.

Figure 7. Original tire-tracks (left) verses artificial tire-tracks (right)

4. ANALYSISThere exists a variety of ways to determine the correctness

of synthetic tracks. The goal of this paper is to mimic the pat-tern of actual tracks in CCD images. The purpose of the fol-lowing analysis is to measure this correctness of the synthetictrack injection algorithms described above. The simplest formof such an analysis is visual inspection. A more quantita-tive method is to determine the statistics of synthetic tracksand compare those values to the statistics of actual tracks.One of the primary motivations behind developing a modelfor synthetic tracks is to enhance track-finding algorithm de-velopment. Thus, another method of analysis is to use exist-ing track-finding techniques on injected synthetic tracks, thencompare the rate of detection to the same algorithms’ perfor-mance on actual tracks. The following describes the results ofthese two approaches on the synthetic injection techniques.

4.1. Statistical ApproachBy comparing the relevent statistics of authentic and syn-

thetic tracks, one can determine whether both types of trackshave the same characteristics. Tracks must be separated fromthe rest of the image before extracting statistical information,so that background clutter is not considered part of the track.This separation is performed spatially by tracing across the

track and keeping all pixels that fall within a certain distancefrom the track center. For synthetic tracks, this tracing is ac-complished using the defining path of the track. For actualtracks, the tracing is placed either by hand or by a line-findingalgorithm.

Figure 8. Straightening a track from a trace.

Figure 8 shows the case where the image within 4 pixels toeither side is kept as part of the track. This track is detectedusing a line-finding algorithm, and the width of four pixelsallows for the full track to be represented (a single tire from avehicle), while limiting the amount of background clutter inthe trace.

Since this approach linearizes each track, it can be treatedas a discrete stochastic process. The following analysis con-siders the sample mean and standard deviation for each trackobtained. These parameters are sufficient to show a separa-tion between tracks and non-track regions as well as show thesimilarities between real and synthetic tracks.

Figure 9. Plotting mean and standard deviation of acquiredtracks and non-track regions

As a test set, 1642 potential tracks were obtained fromtwo different SAR CCD processing chains. These potentialtracks include authentic tire-tracks and footprints, synthetic

tire-tracks and footprints, as well as false positives includingrandom regions of high and low convergence, as well as linearartifacts that occurred during SAR processing.

Figure 9 plots the mean and standard deviation of the in-tensity of these data graphically. Note that since all valueswithin the CCD image are within the range [0,1], there existsa maximum standard deviation associated with each mean.For a given mean µ, a track whose pixels are composed ofsamples of a Bernoulli distribution with success probabilityp = µ will yield this maximum standard deviation, which isshown in Figure 9 with a dotted line.

Each point on this figure represents the statistics for a po-tential track. While any point below the dotted line could rep-resent a track in a SAR CCD image, note that the tracks ob-tained only occupy a small portion of this region. This clus-tering indicates properties of the CCD image, such as contrastand average coherence. The red points represent false classifi-cations of linear artifacts in the imagery that were not tracks,while the blue points represent actual tire tracks.

This plot shows that falsely classified linear features have alower mean intensity than actual tracks. The black and greenpoints represent synthetic tire-tracks and footprints, respec-tively, which have been inserted into the CCD imagery. Thecluster of synthetic tire-tracks overlaps with actual tire-tracksfound, and the cluster of synthetic footprints lies with highermean intensity on the same curve. This is to be expected,since footprints have a lighter average intensity. Thus this pairof statistics indicates that the described synthetic tracks havethe same statistical properties as authentic tracks. Since theseare the statistical parameters that most easily distinguish be-tween authentic tracks and common non-track linear artifacts,the similarity of the values for synthetic and authentic trackscomfirms that the synthetically injected tracks maintain thesame statistical properties as authentic tracks. Thus the tech-niques for modeling these tracks prove effective.

4.2. Track-Finding Algorithm AnalysisSince an application of synthetic track insertion is to create

realistic training data for algorithms whose input is SAR CCDimagery, one potential metric of quality for synthetic tracks istheir rate of detection by existing line-finding algorithms. Ifthe rate at which a sample track is detected is equivalent tothe rate of detection of authentic tracks, then synthetic andauthentic tracks are indistinguishable in the context of thesealgorithms.

The line detection algorithm used in this analysis was de-veloped by Lincoln Laboratory, and the algorithm segmentseach image into 250×250 pixel (corresponding to 25×25 m)subregions. In each region, a hypothesis test is conducted todetermine whether a line exists at angle θ and offset d, testing10,000 possible pairs of these values.

Several test images that contain existing tracks were used

for this analysis. Artificial tracks were inserted into theseimages. The shape and intensity of the inserted tracks ismatched to the existing tracks. Since the line detection al-gorithm used assumes straight linear features, the curvatureof a track greatly affects its detectability. Figure 10 highlightsa situation where the synthetic track has a curvature of zero,which is much easier to detect than the authentic track eventhough it is laid on top of a region of lower coherence. Fig-ure 11 shows an example where a synthetic track has beeninserted in order to mimic the curvature characteristics of anauthentic track.

Figure 10. Example input to line-finding analysis. Authentictrack is in lower left, while synthetic track is inserted acrossupper-right. Note that the synthetic track has a lower curva-ture value, which makes it more detectable to a linear feature-finding algorithm.

Figure 11. In this input to the line-finding analysis, the syn-thetic track (featured below the original track) was formed tomimic the curvature and intensity of the original track.

The test set of images used has synthetic tracks that matchthe curvature of existing tracks. In order to provide a scorefor this analysis, the line detection algorithm is run on the testimages, and each line that corresponds to a track (authenticor synthetic) is retained. For each authentic track in the testset, there exists a synthetic track that mimics it, creating Npairs of these tracks in total. Thus, for synthetic-authentic pairk ∈ [1,N], let sk = 1 if the synthetic track was tagged, andsk = 0 if it was missed. Similarly, let ak = 1 if the authentictrack was tagged, and ak = 0 if it was missed. Thus a scoringmetric is:

score =1N

N

∑k=1

(sk−ak) (6)

A score close to zero is desired. Scores close to±1 indicatethat the track finder is biased towards either synthetic or au-thentic tracks. Note that this scoring metric will return zero if,for one pair, only the synthetic track is found, and for another,only the authentic track is found. This is intentional, since itcorrects for the effects of local background disturbances thatcould cause only one track to be detected, even though theyhave the same curvature or average background intensity.

For the 73 track pairs tested, the resultant score was -0.0411. The overall track recall rate was 19.8%, where 24.7%of the all tracks considered were in areas of low coherence,and the total percentage of track pairs that satisfied ak = skwas 71.2%. This analysis indicates that the synthetic tracksclosely resemble authentic tracks.

5. CONCLUSIONThis paper demonstrates two techniques used to insert syn-

thetically generated tracks into existing SAR CCD images.Both of these techniques are verified using statistical com-parisons between synthetic tracks and existing tracks. Addi-tionally, linear detection algorithms were used to compare thedetectability of each type of track. This analysis resulted inconfirmation that synthetic and authentic tracks are equallydetectable.

Several areas discussed in this paper can be the focus of fu-ture work. The pre-CCD track insertion algorithm introducedin Section 3.2 generates features by adding stationary Gaus-sian noise to existing phase values in the mission SAR im-age. This initial approach yielded realistic results, but a morecomplex insertion technique may yield an even more accu-rate product. Additionally, further statistics could be consid-ered in the analysis of these insertion techniques, in additionto the mean and variance of the intensity of tracks. Consider-ing relational statistics, such as the auto-correlation functionof a track’s intensity, may affirm measured parameters suchas block spacing in the insertion algorithms.

Finally, note that a goal for the development of these algo-rithms is to generate publically available SAR CCD datasets.There exist SAR CCD data in the public domain, yet thesesets contain very limited instances of footprints or tire tracksand do not include groundtruth position data [3]. In conjunc-tion with the algorithms described in this paper, the existingimages in these sets can be used to create training data foralgorithm development.

REFERENCES[1] C.V. Jakowatz, Jr, D.E. Wahl, P.H. Eichel, D.C. Ghiglia,

and P.A. Thompson. Spotlight-mode Synthetic Aperture

Radar: A Signal Processing Approach, chapter 5. KluwerAcademic Publishers, Norwell, MA, 1996.

[2] A.W. Doerry. SAR data collection and processingrequirements for high quality coherent change detec-tion - art. no. 694706. Radio Sensor Technology XII,6947:94706–94706, 2008.

[3] S.M. Scarborough, L. Gorham, M.J. Minardi, U.K. Ma-jumder, M.G. Judge, and L. Moore. A challenge problemfor sar change detection and data compression. SPIE,2010.

BiographyIf space permits, include a brief biography of no more than

300 words for each author at the end of the article to give itgreater impact and validity for the audience.

Listing 1. Example insertion function. Note mask is a boolean matrix.f u n c t i o n outimage = inject_track_in_SAR(cimg, pts_list, W, s, t)

% outimage = inject_track_in_SAR(cimg, pts_list, W, s, [t])%% Given a complex SAR image, a list of track points,

5 % and the width of the track, will return an edited% image that contains these tracks.%% arguments:%

10 % cimg - complex SAR image%% pts_list - list of pixel row,col values% (e.g. [r1,c1;r2,c2;...])%

15 % W - width of tires, in pixels%% s - spacing between tires, in pixels%% t - intensity of track (how dark it is)

20 % Default is 1.%% return value:%% outimage - complex image that contains track

25

i f (nargin == 4)t = 1;

end

30 imgdim = s i z e(cimg);mask = create_mask(imgdim, pts_list, W, s);

%outimage = exp(1i .* mask .* t .* (pi/8)% .* randn(size(cimg))) .* cimg;

35 outimage = exp(1i .* mask .* t .* (pi/8)) .* cimg;

end