melbourne 2002 heiko schröder srikanthan thambipillai ian mcloughlin bertil schmidt wu jigang...

Melbourne 2002

Heiko Schröder

Srikanthan ThambipillaiIan McLoughlin

Bertil SchmidtWu Jigang

Imrich VrtoOndrej Sykora

Tanja Vladimirova

Fault tolerant high performance computing on-board small satellites

Melbourne 2002

1000 nmRGB 2500 nm

Multispectral

Melbourne 2002

CHRIS

Multispectral

Melbourne 2002

Hyperspectral

Melbourne 2002

Hyperspectral

Melbourne 2002

Hyperspectral

Melbourne 2002

Fault tolerant On-board computing

10 km/s 1 image/s 100 Mbit/image 4000 s/orbit 400 Gbit/orbit download: 4 Gbit/orbit

On-board image analysis andcompression

800 km

Singapore100 x output if useful/useless<=1/100

100 x value

Melbourne 2002

Methods currently used

shadow-processorsmajorityvoting

Byzantine systems ASTRIUM, deep space

Melbourne 2002

1 CAN2 CANs •Industrial spec.

•mil-spec.•radiation tolerant•radiation hardened

386 is modern

Melbourne 2002

Our aim: High performance via COTS16 processors (+ spares) off-the-shelfconnected via afault tolerant reconfigurable network

In X-SAT restricted to image processing

Mesh/torus

Melbourne 2002

processorsfault

tolerantmesh

on-board

Melbourne 2002

switch

current communication

FPGA

ctrlh/vo/er/w

Instructionsto PEs

link to PE

Melbourne 2002

spares

C3 -- torus

spares

Replacement algorithm exists for up to 4 faults.Reconfiguration software runs on FPGA.Could be repaired within << 1sec.

Melbourne 2002

ctrlh/vo/er/w

Instructionsto PEs

Diagnosticset switches

Melbourne 2002

Available data (320 images) – search task

Oil slicks, forest fires, red tide, settlements, …Efficiency of the system: useful output / useful input <=1

Randomselection

E=Q=1/64

Output

Algorithms:•Compression•Classification•Segmentation

Melbourne 2002

Compressionratio (CR=4loss-less)

Segmentation gain (SG=16, 1/16 of a useful image is useful)

Classification gain(CG=5, 1 in 5 images contain useful information)

E=1/16Q*CR

E=1/64Q

E=5/16Q*CR*CG

E=5/64Q*CG

E=5/4Q*CG*SG

The satellite efficiency cube

Not likely

LOSSY=60E=5/2

E=5Q*CR*CG*SG

Melbourne 2002

1 2 3 4 5 6 7 8

9 10 11 12 13 14 15 16

17 18 19 20 21 22 23 24

25 26 27 28 29 30 31 32

33 34 35 36 37 38 39 40

41 42 43 44 45 46 47 48

49 50 51 52 53 54 55 56

57 58 59 60 61 62 63 64

LL1+2+3+4

HL1+3-2-4

LH1+2-3-4

HH1+4-2-3

1 2

3 4

LL HL

LH HH

LL HL

LH HH

LL HL

LH HH

LL HL

LH HH

LL HL

LH HH

LL HL

LH HH

LL HL

LH HH

LL HL

LH HH

LL HL

LH HH

LL HL

LH HH

LL HL

LH HH

LL HL

LH HH

LL HL

LH HH

LL HL

LH HH

LL HL

LH HH

LL HL

LH HH

LL LL

LL LL

LL LL

LL LL

LL LL

LL LL

LL LL

LL LL

LH LH

LH LH

LH LH

LH LH

LH LH

LH LH

LH LH

LH LH

HL HL

HL HL

HL HL

HL HL

HL HL

HL HL

HL HL

HL HL

HH HH

HH HH

HH HH

HH HH

HH HH

HH HH

HH HH

HH HH

1 2 3 4 5 6 7 89 10 11 12 13 14 15 16

17 18 19 20 21 22 23 2425 26 27 28 29 30 31 3233 34 35 36 37 38 39 4041 42 43 44 45 46 47 4849 50 51 52 53 54 55 5657 58 59 60 61 62 63 64

L1+2

H1-2

L3+4

H3-4

Invertible!+ /2 - /2

Melbourne 2002

33+34+41+4249+50+57+58

-35+36+43+4451+52+59+60

HL1 HL1 HL1 HL1

HL1 HL1 HL1 HL1

HL1 HL1 HL1 HL1

HL1 HL1 HL1 HL1

HH1HH1HH1HH1

HH1HH1HH1HH1

HH1HH1HH1HH1

HH1HH1HH1HH1

LH1 LH1 LH1 LH1

LH1 LH1 LH1 LH1

LH1 LH1 LH1 LH1

LH1 LH1 LH1 LH1

LL1 LL1 LL1 LL1

LL1 LL1 LL1 LL1

LL1 LL1 LL1 LL1

LL1 LL1 LL1 LL1

7+8+15+16

23+24+31+32

38+40+47+48

55+56+63+64

5+6+13+14

21+22+29+30

37+38+24+46

53+54+61+62

3+4+11+12

19+20+27+28

35+36+43+44

51+52+59+60

1+2+9+10

17+18+25+26

33+34+41+42

49+50+57+58

1 2 3 4 5 6 7 89 10 11 12 13 14 15 16

17 18 19 20 21 22 23 2425 26 27 28 29 30 31 3233 34 35 36 37 38 39 4041 42 43 44 45 46 47 4849 50 51 52 53 54 55 5657 58 59 60 61 62 63 64

LL2

LH2

LH2

LH2

LH2

HL2

HL2

HH2

HH2

HL2

HL2

HH2

HH2

LL2

LL2 LL2LH3

HL3

HH3

LL3

33+42-34-41

35+44-36-43

49+58-50-57

51+60-52-59

LL1+2+3+4

HL1+3-2-4

LH1+2-3-4

HH1+4-2-3

1 2

3 4

L1+2

H1-2

L3+4

H3-4

1+…+64

1..4,9..12,17-20,25-2833-36,41-44,49-52,57-60

-5-8,13-16,21-24,29-32

37-40,45-48,53-56,61-64

1+…+32-

33+…+64

1-4,9-12,17-20,25-2837-40,45-48,53-56,61-64

-5-8,13-16,21-24,29-32

33-36,41-44,49-52,57-60

Melbourne 2002

1+2+9+103+4+11+12

17+18+25+2619+20+27+28

5+6+13+147+8+15+16

21+22+29+3023+24+31+32

33+34+41+4235+36+43+4449+50+57+5851+52+59+60

37+38+45+4638+40+47+4853+54+61+6255+56+63+64

1+2+9+103+4+11+12

-17+18+25+2619+20+27+28

5+6+13+147+8+15+16

-21+22+29+3023+24+31+32

33+34+41+4235+36+43+44

-49+50+57+5851+52+59+60

37+38+24+4638+40+47+48

-53+54+61+6255+56+63+64

1+2+9+10+17+18+25+26

-3+4+11+12

19+20+27+28

5+6+13+1421+22+29+30

-7+8+15+16

23+24+31+32

33+34+41+4249+50+57+58

-35+36+43+4451+52+59+60

37+38+24+4653+54+61+62

-38+40+47+4855+56+63+64

1+2+9+1017+18+25+26

-3+4+11+12

19+20+27+28

5+6+13+1421+22+29+30

-7+8+15+16

23+24+31+32

33+34+41+4249+50+57+58

-35+36+43+4451+52+59+60

37+38+24+4653+54+61+62

-38+40+47+4855+56+63+64

LL2 HL2

LH2 HH2

Zero-tree

Melbourne 2002

Main ideas of zero tree encoding:•When the parent is small the children are small•If a root of a tree is smaller than a given threshold,

and all descendants are too,then only the root needs to be encoded

•Many values of the result of the wavelet transform are small,as they are differences of neighbors.

Melbourne 2002

Melbourne 2002

How to find areas of interestImage classificationIn real-time

?

Melbourne 2002

Thresholding

Melbourne 2002

Mathematical morphologyMathematical morphology

erosion

dilation

erosion

edge detection, thinning, noise removal, enlarging

Structural elementreference point

Melbourne 2002

ThresholdingMM-segmentation

Melbourne 2002

skeletonsskeletons

Histograms

Melbourne 2002

Red square skeletonRed square skeleton

1

1

0

3

0

0

0

1

6

Melbourne 2002

new = min{W,NW,N}+1

one-sweep algorithm to produce the red square skeleton:

Melbourne 2002

Rough SegmentationRough Segmentation

• Threshold

• Noise removal

• Red square

• frame

Melbourne 2002

MM-Hough TransformMM-Hough Transform

reference pointerosion

m d

d

m

a dot leads to one addition if there is a matching point

Melbourne 2002

• Higher contrast

• More flexibility– Lines of given thickness

– Dashed lines

– Lines of given length

– Lines of given orientation

– Other curves

Lines at 90 degrees

Melbourne 2002

Melbourne 2002

60 sec420km

1min420km

1min maneuver420km

30sec210km

Single image19 bands

Image sequence3 bands

130sec 19 bands910km

Image sequence19 bands

60 sec420km

Investigative mode

Melbourne 2002

60 sec420km

60 sec420 km

45 sec315 km

7 min2900 km3 bands

5 min1800 km19 bands

Follow coast3 bands

?Change band selection

Search mode

Melbourne 2002

2000 km

High-performance Computer network

Real-time image analysis•Classification•Segmentation•compression

Intelligent search

Maximize the efficiency of the satellite!

200 km

1 min

??

??

Thank you!

Melbourne 2002

… an arrary of SHARCs to provides throughput 160 Mb/s.… 2.5 billion floating point operations per second. … first demonstration of real-time image processing in space.

image cube froma 30 km wide swath of Korea’s coastline.(Launch: 2001?)

Nemo