Pa

ge

1

Ve

rsio

n:

13

.04

.20

11

G

erh

ard

.pa

ar@

joa

nn

eu

m.a

t

PRoViScout

FP7-PRoViScout – Planetary Robotics Vision Scout Gerhard Paar(1), Mark Woods (2) & the PRoViScout Team

(1) Joanneum Research, Graz, Austria

(2) Scisys Ltd., Bristol, UK

11th ASTRA Conference, ESTEC, The Netherlands,

April 12-14, 2011

Pa

ge

2

Ve

rsio

n:

13

.04

.20

11

G

erh

ard

.pa

ar@

joa

nn

eu

m.a

t

PRoViScout

FP7-SPACE Call 2008 (ITT): The Objectives

— Call Topic: „Space Exploration―

— Optimum preparation of scientific payloads

— Effective scientific exploitation of future space missions data

— Enhance effectiveness & productivity in usage of data,

including archived data

— Improve capabilities to move, to select and collect and finally

return samples to Earth

— Increase public awareness

— Complementary to and in close co-operation with ESA

Pa

ge

3

Ve

rsio

n:

13

.04

.20

11

G

erh

ard

.pa

ar@

joa

nn

eu

m.a

t

PRoViScout

Heritage: EUROBOT, ExoMars, PRoVisG, CREST, MER – 12 Partners

MSSL/UCL: Exobiology & WALI

DLR: Geology

JR: 3D Vision & PR

AU: Robotics & Platform / IP

JPL: Autonomous Science on MER

SciSys: Autonomous Scientist & SW Framework

CTU: Vision & www

GMV: Navigation

CSEM: Zoom 3D-TOF & RGB Cam

TRS: Simulation

UoS: Decision Module

ULeic: Geology

Science

Pa

ge

4

Ve

rsio

n:

13

.04

.20

11

G

erh

ard

.pa

ar@

joa

nn

eu

m.a

t

PRoViScout

Objectives: Efficient Scout Mission

© Scisys

Pa

ge

5

Ve

rsio

n:

13

.04

.20

11

G

erh

ard

.pa

ar@

joa

nn

eu

m.a

t

PRoViScout

Target2: Sample Fetching Rover

©

See http://www.youtube.com/watch?v=uCQw5dcqUaI

Pa

ge

6

Ve

rsio

n:

13

.04

.20

11

G

erh

ard

.pa

ar@

joa

nn

eu

m.a

t

PRoViScout

Where to go for Optimum Result

Identify / select targets

and reach the best one !

Pa

ge

7

Ve

rsio

n:

13

.04

.20

11

G

erh

ard

.pa

ar@

joa

nn

eu

m.a

t

PRoViScout

Technical Approach: PRoViSC

!!

Planetary Robotics Vision Scouting Chain

Planetary Robotics

Vision Monitoring

& PRoViM

Pa

ge

8

Ve

rsio

n:

13

.04

.20

11

G

erh

ard

.pa

ar@

joa

nn

eu

m.a

t

PRoViScout

Work Packages

PROVISC &

On-Board System

WP 4

Science Target

Selection

WP 3

Consolidation

WP 2

Simulation & Testing

WP 6

Exploitation

WP 7

Www.proviscout.eu

Spin Off

Science

Expertise

Missions

Expertise

Tools

PROVIM

WP 5

Publication

Workshops

Field Test

Vision Scientists

Planetary Scientists

Management

WP 1

Meetings,

Technical & Scientific Mgt

EC

Financial Reporting

214 PM

Funding:

• 50%... Industry

• 75%... R&D

• 100 %... Management

• 100 %... Dissemination

30 Months

April 2010 – Sept 2012

Pa

ge

9

Ve

rsio

n:

13

.04

.20

11

G

erh

ard

.pa

ar@

joa

nn

eu

m.a

t

PRoViScout

Planetary Science Aspects

— Site selection & Assessment

— Criteria

• Accessibility

• Science Richness

• (Weather) Safety

• Traversability

• Lack of vegetation

— Clarach Bay, Ceredigion

— Morocco

— Tenerife

— Science Requirements

— Training Image Definition & Masks

© Univ. Leicester

© Univ. Leicester

Pa

ge

10

V

ers

ion

: 1

3.0

4.2

01

1

Ge

rha

rd.p

aa

r@jo

an

ne

um

.at

PRoViScout

Technical Approach: Platforms & Sensors

© Aberystwyth University

© Aberystwyth University

Pa

ge

11

V

ers

ion

: 1

3.0

4.2

01

1

Ge

rha

rd.p

aa

r@jo

an

ne

um

.at

PRoViScout

© CSEM / Zurich

Innovation in Space Robotics: Zoomed3D-TOF & RGB

— 20W optical peak power @ 15 MHz Modulation

— Distance range: 1m – 10m

— FoV (Diagonal) 7° (zoomed) – 40° (Wide Angle)

— Real-Time Mapping

— Full Sunlight resolution (st.dev)

0.01 m@ 1m (full field)

0.2 m@ 5m (full field)

0.003 m@ 1m (full zoom)

0.02 m@ 5m (full zoom)

— RGB 1 Mpix camera (IR filter, optional)

— Common beam path of TOF & RGB

— Drawbacks: Dimensions / Weight (& Eye Safety)

Pa

ge

12

V

ers

ion

: 1

3.0

4.2

01

1

Ge

rha

rd.p

aa

r@jo

an

ne

um

.at

PRoViScout

Software Components

© Scisys

Problem:

High effort into

integration of

heterogeneous

components

Pa

ge

13

V

ers

ion

: 1

3.0

4.2

01

1

Ge

rha

rd.p

aa

r@jo

an

ne

um

.at

PRoViScout

3D Vision Processing Chain

Ortho image DEM

Hazard map Slope map

Path Planning

Pa

ge

14

V

ers

ion

: 1

3.0

4.2

01

1

Ge

rha

rd.p

aa

r@jo

an

ne

um

.at

PRoViScout

Example: 3D Vision Server

Pa

ge

15

V

ers

ion

: 1

3.0

4.2

01

1

Ge

rha

rd.p

aa

r@jo

an

ne

um

.at

PRoViScout

Navigation

© GMV, 2011

All rights reserved

Pa

ge

16

V

ers

ion

: 1

3.0

4.2

01

1

Ge

rha

rd.p

aa

r@jo

an

ne

um

.at

PRoViScout

Classification Scheme 1 Feature Extraction (SIFT) Classifier (NN)

Representative Test images using the same classifier:

Pa

ge

17

V

ers

ion

: 1

3.0

4.2

01

1

Ge

rha

rd.p

aa

r@jo

an

ne

um

.at

PRoViScout

Training Accumulation ROIs

SIFT / SURF Hits

Pa

ge

18

V

ers

ion

: 1

3.0

4.2

01

1

Ge

rha

rd.p

aa

r@jo

an

ne

um

.at

PRoViScout

Science Assessment Interface Function Description Parameters (in/out) Constraints Data Size

(Large/Small) Required By

(Called from)

Determine

science targets

in 3D (*)

Controls the detection of

science targets in imagery.

Provides the high level

Interface to other modules.

RGB image [in];

3D science target

coordinates (X,Y,Z of

type double),

science target

classes, weights

(double) [out]

Large bandwidth for

reading [in] images

and the scientific

database description.

Little bandwidth for

output [out]

Executive

Determine Local

Interest

Candidates

Inspects an RGB image to

determine particular cues

for the detection of science

targets in 2d image space,

assigning weights prop. to

each candidate.

RGB Image [in]

Local Interest

Candidates (2D

positions in image

and priorities) [out]

Candidate coordinates

within image dimension,

priorities 0..100 (0 low)

Internal (*)

Determine

Region of

interest (ROI) for

science targets

Weighted integration of

local interest candidates in

order to make a decision for

a (2D) segmentation into

science target(s). The

integration step will utilize

the scientific database for

matching candidates.

Local Interest

Candidates (2D

positions in image

and priorities) [in]

Requests on science

database (description of

features, etc.). Calls the

Target Localization

Module finally to return

3D position of the

determined scientific

target.

Internal (*)

Multiple cues for the Decision Module

Pa

ge

19

V

ers

ion

: 1

3.0

4.2

01

1

Ge

rha

rd.p

aa

r@jo

an

ne

um

.at

PRoViScout

Multi – Cue Information & Scale

Context…

Hit of

Scientific

Interest

Pa

ge

20

V

ers

ion

: 1

3.0

4.2

01

1

Ge

rha

rd.p

aa

r@jo

an

ne

um

.at

PRoViScout

Multi – Cue Information & Scale

ROI Context…

Pa

ge

21

V

ers

ion

: 1

3.0

4.2

01

1

Ge

rha

rd.p

aa

r@jo

an

ne

um

.at

PRoViScout

Multi – Cue Information & Scale

ROI…

Pa

ge

22

V

ers

ion

: 1

3.0

4.2

01

1

Ge

rha

rd.p

aa

r@jo

an

ne

um

.at

PRoViScout

Multi – Cue Information & Scale

Target Context…

Pa

ge

23

V

ers

ion

: 1

3.0

4.2

01

1

Ge

rha

rd.p

aa

r@jo

an

ne

um

.at

PRoViScout

Multi – Cue Information & Scale

Target

& 3D Target

Pa

ge

24

V

ers

ion

: 1

3.0

4.2

01

1

Ge

rha

rd.p

aa

r@jo

an

ne

um

.at

PRoViScout

Technical Approach: PRoViM - To understand system behavior

©: all Trasys

Heritage:

3D-ROV

Pa

ge

25

V

ers

ion

: 1

3.0

4.2

01

1

Ge

rha

rd.p

aa

r@jo

an

ne

um

.at

PRoViScout

PRoViM – to – www

Window monitor

Window monitor

Window monitor

Screen-shot memory

(file system, RAM, …)

www server

© Trasys; CTU Prague

Pa

ge

26

V

ers

ion

: 1

3.0

4.2

01

1

Ge

rha

rd.p

aa

r@jo

an

ne

um

.at

PRoViScout

Status as from April 2011 (M13)

— Design & 70% Prototype components finished

— 3D-TOF & RGB to be assembled

— Science Requirements finished

— Test / training data & samples forthcoming

— Aerobots mapping test successful

— Field Assessment in Tenerife June 2011

— Hints welcome !

— First SW Integration Campaign June 2011

— HW Components Integration Campaign October 2011

— Field Test Summer 2012 @ Tenerife

Pa

ge

27

V

ers

ion

: 1

3.0

4.2

01

1

Ge

rha

rd.p

aa

r@jo

an

ne

um

.at

PRoViScout

Other (&) Innovative Aspects (Selection) — 3D-TOF & Zoom & RGB & Space Application

— WALI (Wide Angle Laser Imager) as Flourescent Organics cue

— SpaceMaster Programme Link

(Students – into – Project)

— Combination Science / Navigation

Decision System

— Aerobot & Rover Combination (Offline ?)

— Science & System Integrated

— Towards MSR

— Towards Planetary Science Agent Scenarios

— Geologists – in – the – loop

— Compare with Human Spaceflight (Costs vs. Performance Issues)

Pa

ge

28

V

ers

ion

: 1

3.0

4.2

01

1

Ge

rha

rd.p

aa

r@jo

an

ne

um

.at

PRoViScout

Related Presentations @ ASTRA — M. Woods: High-Level Autonomy

Wed 10:25, Session 1A

— A. Medina: Heritage EUROBOT

Thu 12:10, Session 6A

— L. Tyler: Aerobot

Thu 14:00, Session 7A

www.proviscout.eu

© SciSys

© gmv

© Aberystwyth University