project public report - fp7-icarus · information networks has been recognised by the fp7-security...

PROJECT PUBLIC REPORT

Grant Agreement number 285417

Call identifier SEC 2011.4.2-2

Project Acronym ICARUS

Project title Integrated Components for Assisted Rescue and

andUnmanned Search operations Unmanned Search operations

Funding Scheme Collaborative Project

Project Starting date February 01, 2012

Project Duration 48 months

Project Coordinator Royal Military Academy – Geert De Cubber

[email protected] Issue V7.0

Name, title and organisation of the scientific

representative of the project's coordinator Dr. Geert De Cubber, Royal Military Academy

(RMA)

Coordinator Telephone Number +32(0)2-44-14106

Coordinator E-Mail Address [email protected]

Project website address http://www.fp7-icarus.eu/

Document validated by RMA – Daniela Doroftei

Dissemination Level* PU

(*)

PU = Public.

PP = Restricted to other programme participants (including the Commission Services).

RE = Restricted to a group specified by the consortium (including the Commission Services).

CO = Confidential, only for members of the consortium (including the Commission Services).

http://www.fp7-icarus.eu/

Project Public Report v7.0

Table of Contents

1. ..... Executive Summary of the project ......................................................................................... 5

2. ..... Summary description of ICARUS project context and objectives ............................................. 6

3. ..... ICARUS main Scientific and technological results foregrounds ............................................. 11

3.1. User-centered design requirements ..................................................................................... 11

3.2. Sensing ................................................................................................................................... 13

3.3. Unmanned Aerial Systems .................................................................................................... 15

3.4. Unmanned Ground Vehicles ................................................................................................. 17

3.5. Unmanned Maritime Platforms ............................................................................................ 19

3.6. Heterogeneous teams and network centric operations ....................................................... 21

3.7. Communication ..................................................................................................................... 24

3.8. Command and Control .......................................................................................................... 27

3.9. Training and Support ............................................................................................................. 29

3.10. Validation of the ICARUS system in an earthquake response scenario ............................ 32

3.11. Validation of the ICARUS system in a marine incident response scenario ....................... 34

4. ..... ICARUS Impact ................................................................................................................... 36

4.1. Impact on the general society and wider societal implications ............................................ 36

4.2. Economic Impact ................................................................................................................... 38

4.3. Main dissemination activities ................................................................................................ 39

4.4. Exploitation of results ............................................................................................................ 42

5. ..... Website ............................................................................................................................. 43


Acronyms & Definitions

AR Augmented Reality

C2I Command, Control and Intelligence

C4I Command, Control, Communications, Computers and Intelligence

CAN Controller area network

CMRE Centre for Maritime Research and Experimentation

COP Common Operational Picture

CSAP Communication strategy and action plan

DARIUS Deployable SAR Integrated Chain with Unmanned Systems

DOW Description of Work

EC European Commission

EM-DAT Emergency Events Disasters Database (http://www.em-dat.net/ )

EPFL École Polytechnique Fédérale de Lausanne

ESRIF European Security Research and Innovation Forum

EU European Union

EUB End-Users Board

EURON EUropean RObotics research Network

FINROC Framework for INtelligent RObot Control

FP Framework Programme

FPGA Field-programmable gate array

GIS Geographic Information System

GPS Global Positioning System

HMI Human Machine Interface

ICD Interface Control Definition

IMU Inertial Measurement Unit

INSARAG International Search and Rescue Advisory Group

I2C Inter-Integrated Circuit

IP Internet Protocol

IPR Intellectual Property Rights

IR Infrared

JAUS Joint Architecture for Unmanned Systems

KoM Kick-off Meeting

LAN Local Area Network

LOS Line of Sight

LUGV Large Unmanned Ground Vehicle

MANET Mobile ad hoc network

NATO North Atlantic Treaty Organisation

MPCS Mission Planning and Coordination System

MSAR Marine Search and Rescue

MST Management Support Team

NETD Noise equivalent temperature difference

NIC Network interface controller

http://www.em-dat.net/


NIST National Institute of Standards and Technology

OCHA Office for the Coordination of Humanitarian Affairs

OROCOS Open RObot Control Software

OSOCC On Site Operations Coordination Centre

OSVSD Operational Service Validation Scenario Definition Document

PC Personal Computer

PCB Printed circuit board

PID Proportional–integral–derivative

PMB Project Management Board

PMP Project Management Plan

PVC Polyvinyl chloride

QCD Quantum Cascade Detector

QoS Quality of Service

QSTRR Qualitative Spatio-Temporal Representation and Reasoning

RC Remote Control

RC2 Robot Command and Control

REA Research Executive Agency

RGB Red, Green, Blue

ROIC Readout Integrated Circuits

ROS Robot Operating System

SAB Security Advisory Board

SAR Search And Rescue

SRAD System Requirements and Architecture Definition Document

STAB Scientific and Technical Advisory Board

STANAG Standardisation Agreement

SUGV Small Unmanned Ground Vehicle

TEC Thermoelectric Cooling

TBD To be determined

UAS Unmanned Aerial System

UGV Unmanned Ground Vehicle

UHF Ultra High Frequency

URD User Requirements Document

USA United States of America

USAR Urban Search and Rescue

USB Universal Serial Bus

USV Unmanned Surface Vehicle

UTRO Universal transfer of rights and obligations

UV Unmanned Vehicle

VHF Very High Frequency

VR Virtual Reality

WIMAX Worldwide Interoperability for Microwave Access

WLAN Wireless local area network


1. EXECUTIVE SUMMARY OF THE PROJECT

Recent dramatic events such as the earthquakes in Nepal and Tohoku, typhoon Haiyan or the many

floods in Europe have shown that local civil authorities and emergency services have difficulties with

adequately managing crises. The result is that these crises lead to major disruption of the whole local

society. On top of the cost in human lives, these crises also result in financial consequences, which

are often extremely difficult to overcome by the affected countries. The goal of ICARUS is to decrease

the total cost of a major crisis. In order to attain this goal, the ICARUS project proposes to equip first

responders with a comprehensive and integrated set of unmanned search and rescue tools, to

increase the situational awareness of human crisis managers, such that more work can be done in a

shorter amount of time. The importance of combining such cognitive robotic systems with on-line

information networks has been recognised by the FP7-Security call 2011.4.2-2, which is the topic

addressed by ICARUS.

As every crisis is different, it is impossible to provide one solution which fits all needs. Therefore, the

ICARUS project will concentrate on developing components or building blocks that can be directly

used by the crisis managers when arriving on the field. Furthermore, the project aims to provide an

integrated proof-of-concept solution, to be evaluated by a board of expert end-users that can ensure

that operational needs are addressed.

The ICARUS project deals with the development of a set of integrated components to assist search

and rescue teams in dealing with the difficult and dangerous, but life-saving task of finding human

survivors. The ICARUS tools consist of assistive unmanned air, ground and sea vehicles, equipped

with victim detection sensors. The unmanned vehicles collaborate as a coordinated team,

communicating via ad hoc cognitive radio networking. To ensure optimal human-robot collaboration,

these tools are seamlessly integrated into the C4I equipment of the human crisis managers and a set

of training and support tools is provided to them to learn to use the ICARUS system.

The ICARUS project does not only focus on the development of tools and services, but also on the

integration of these novel tools into the standard operating procedures of the end-users. Indeed, in

many cases these integration issues, procedural incompatibilities or absence of legal framework are

the main bottlenecks impeding a successful deployment in practical operations and not pure

technological issues. ICARUS therefore concentrates also on placing novel technological tools into the

hands of the end users, thereby driving the acceptance and practical use of these tools.


2. SUMMARY DESCRIPTION OF ICARUS PROJECT CONTEXT AND OBJECTIVES

In the event of large crises, a primordial task of the fire and rescue services is the search for human

survivors on the incident site. This is a complex and dangerous task, which - too often - leads to loss

of lives among the human crisis managers themselves. The introduction of unmanned search and

rescue devices can offer a valuable tool to save human lives and to speed up the search and rescue

process. Therefore, ICARUS concentrates on the development of unmanned search and rescue

technologies for detecting, locating and rescuing humans. In this context, there is a vast literature on

research efforts towards the development of unmanned search and rescue (SAR) tools, notably in the

context of EU-sponsored projects. This research effort stands in contrast to the practical reality in the

field, where unmanned search and rescue tools have great difficulty finding their way to the end-

users. Notable bottlenecks in the practical applicability of unmanned search and rescue tools are:

o Slow deployment time of the current generation of unmanned SAR tools

o Limited autonomy and self-sustainability of the current generation of unmanned SAR tools, both

from a point of view of the robot intelligence and from an energy and mobility perspective

o Limited collaboration between unmanned SAR devices

o Insufficient integration of the current generation of unmanned SAR tools in the C4I equipment

used by fire and rescue services

o Insufficient support and training are available for the end-users to learn to use the unmanned

tools

o Problems of interoperability of (unmanned SAR) equipment when multi-national crisis

management teams need to collaborate on an incident site

The ICARUS project addresses these issues, aiming to bridge the gap between the research

community and end-users, as described in section 3.1. The core objective of the ICARUS project is to

develop robots which have the primary task of gathering data. The unmanned SAR devices are

foreseen to be the first explorers of the area, as well as in situ supporters to act as safeguards to

human personnel. In order not to increase the cognitive load of the human crisis managers, the

unmanned SAR devices will be designed to navigate individually or cooperatively and to follow high-

level instructions from the base station. The robots connect wirelessly to the base station and to

each other, using a wireless self-organising cognitive network of mobile communication nodes which

adapts to the terrain. The unmanned SAR devices are equipped with sensors that detect the

presence of humans and will also be equipped with a wide array of other types of sensors. At the

base station, the data is processed and combined with geographical information, thus enhancing the

situational awareness of the personnel leading the operation with in-situ processed data that can

improve decision-making. All this information will be integrated in existing information systems, used

by the forces involved in the operations. In line with the current bottlenecks, as stated above, eight

main objectives are defined for the ICARUS project. These objectives address the operational needs

of rescue and civil protection services and are defined and evaluated by the end-users using two

main demonstration scenarios (an earthquake response and a marine incident), as described in

sections 3.10 and 3.11.


Objective 1: Development of a light sensor capable of detecting human beings

The first objective is the development of a small light-weight camera system capable of detecting

human survivors. These prototype cameras will have a resolution of 8x8 pixels arranged in a small

array of 2x2 single chips. They will be based on novel and very promising Quantum Cascade Detector

(QCD) technology. The latter will allow the manufacture of highly sensitive, low noise, narrow band

IR detectors with a detection wavelength of 8 µm. This ultra-sensitive, but relatively low-resolution

QCD camera will be complemented by a commercial high-resolution lower-sensitivity micro-

bolometer camera. Minimal levels of weight (500 g), dimensions (12x12x6 cm) and total power

consumption (5 W) are being targeted. Image and video processing algorithms for detecting human

survivors will be developed and combined to obtain sufficient detection performance. Data fusion

methods will be applied to images coming from different cameras, resulting in different detection

algorithms. The main results of this activity are described in section 3.2.

Objective 2: Development of cooperative Unmanned Aerial System (UAS) tools for unmanned SAR

UAS platforms will be given a crucial role by acting as quick deployment assets in the field to provide

valuable information to enhance situational awareness in support of the assessment of crisis

managers, as well as to enable tactical planning and decision-making. This aerial infrastructure will

also provide continuous support to coordinators and operators in the field, complementing the UGV

and USV solutions. UAS platforms will be equipped with sensors tailored to SAR requirements,

including the QCD camera and victim detection algorithms, allowing for the localisation and tracking

of victims. In order to meet the end user demands, complementary platforms are proposed. A small

long-endurance solar aeroplane is meant to provide the highest view at a maximum height of 300m,

as allowed by national legislation, and therefore enabling the mapping functionality and initial victim

search. Payload other than small cameras is limited, but operation times span up to a day. With

shorter range and endurance, but closer to the ground and the victims, two rotary wing systems are

to be deployed. A Quadrotor with a size of 1m and a maximum payload of 1kg will be used for

delivery tasks outdoors and observation. A slightly smaller multicopter will be used for indoor people

search. Consequently, on-board autonomous functionalities will be developed to decrease the

operator workload and increase the operational efficiency in the overall C4I system. The main results

of this activity are described in section 3.3.

Objective 3: Development of cooperative Unmanned Ground Vehicle (UGV) tools for unmanned

SAR

The ICARUS project considers the production of the two types of robotic systems, using existing base

platforms:

o The Large UGV (LUGV) that shall be part of the ICARUS project shall serve as a platform fulfilling

several central tasks. After being deployed close to the site of an emergency, it shall move in a

semi-autonomous way in a potentially hazardous and unknown environment. The LUGV can act

as a mobile sensor platform, gathering a large amount of precise data is necessary for (semi-)


autonomous navigation in challenging environments as well as for the support of emergency

teams. The UGV shall also be used to transport the small UGV and shall be equipped with a

powerful manipulator that can be used to clear the vehicle’s path from obstacles like small

debris.

o A smaller UGV shall gather more information about areas which the LUGV cannot reach. The

small UGV will be able to enter in collapsed buildings to search for human victims, an extremely

dangerous but also life-saving task. The SUGV will be equipped with a propulsion system allowing

it to manoeuvre in highly unstructured environments like collapsed buildings.

The main results of this activity are described in section 3.4.

Objective 4: Development of cooperative Unmanned Surface Vehicle (USV) tools for unmanned

SAR

This project proposes two main lines of work in order to address the identified demands. On one

hand the project will present the instrumentation of a survival capsule to allow its motion towards

survivors at the surface. On another hand the project will undertake the adaptation of a medium size

USV for search and rescue operations. Existing survival capsules that usually inflate when deployed

allow survivors to climb aboard providing extra floatation and thermal insulation. The incorporation

of power generation capabilities, a minimal set of instruments, basic communication equipment, and

motion capabilities on board these capsules, will increase the lifesaving capabilities of such devices

allowing their use in scenarios with reduced accessibility for other search and rescue services. USVs,

as unmanned systems, allow remote human intervention under severe environmental conditions

without putting additional people at risk. They have therefore a large potential for SAR operations at

sea, especially under bad weather conditions with low visibility. The main results of this activity are

described in section 3.5.

Objective 5: Heterogeneous robot collaboration between Unmanned Search and Rescue devices

This objective is focused on a key enabling technology concept for the safe integration of

autonomous platforms into search and rescue operations: the heterogeneous network. The project

specifically addresses the intrinsic capabilities and characteristics of a given platform, and how these

characteristics are communicated, understood, and exploited by the rest of the SAR system

(including human teams, infrastructures, and other autonomous vehicles within the ICARUS

integration concept). The present objective therefore addresses the integration of heterogeneous

teams into a single, unified, interoperable system through establishing and demonstrating the

interactions and use cases of different vehicle types. The application of search and rescue influences

the definition and interactions of the network, and this project objective addresses the

interoperability challenges and the robust definition and specification of tasks, and roles and

responsibilities between the autonomous capacity of the heterogeneous team and the mission-level

tasking and supervision of the C2I system in network-centric operations. The main results of this

activity are described in section 3.6.


Objective 6: Development of a self-organising cognitive wireless communication network, ensuring

network interoperability

ICARUS will develop a network infrastructure which adapts to and, at the same time, takes advantage

of the peculiarities of the posed SAR scenarios. A holistic approach will be followed, reusing state-of-

the art solutions conveniently and focusing investigation on unsolved challenging issues:

o Mobile and wireless ad-hoc communications in combined land-air-sea environments with

robotic and human actors, supporting both Line-of-Sight (LOS) and non-LOS scenarios.

o Self-coordination and optimisation of spectrum resources by using cross-layer cognitive radio

techniques maximising network usability and minimising interferences.

o Self-managed network able to adapt to varying and extreme conditions by using power-

efficient, failure-resilient protocols (e.g. active routing, data-replication, store-and-forward)

and convenient guidance of robotic network nodes with specific communication capabilities.

o Flexible security scheme providing granular encryption, integrity and authentication.

o A harmonised management and control overlay on top of a highly robust waveform, able to

encompass several data-link technologies (WLAN, GSM) ensuring interoperability.

The main results of this activity are described in section 3.7.

Objective 7: Integration of Unmanned Search and Rescue tools in the command and control

systems of the Human Search and Rescue forces

ICARUS aims at developing (robot) platform independent monitoring and control capabilities that will

be able to handle, process and integrate a wide variety of data flows coming from sources such as

the robotic platforms’ sensors, human beings (bystanders) in the field, GIS displaying a priori

knowledge about the intervention field, etc. The resulting information and knowledge will primarily

be exploited at the command and control application level, in order to effectively provide human

operators with a high level of awareness allowing them to lead the robotic activities in a coordinated

way with humans on field activities. As a noticeable feature, the command and control centre will

provide a haptic tele-presence workstation allowing real-time control of haptic compliant robotic

arms. The command and control system will be designed to promote interoperability of the

controlled systems, as well as aiming for seamless integration into existing infrastructure and

applications used by first responders. The main results of this activity are described in section 3.8.

Objective 8: Development of a training and support system for the developed Unmanned Search

and Rescue Tools for the Human Search and Rescue teams

In the ICARUS project several types of unmanned vehicles will be used, so from a training point of

view the main objective is to deliver software tools that can simulate such a system. Different types

of simulation (ground, air, water) will be developed and integrated to perform complex training of

future ICARUS operators. The training tool will be capable of simulating predefined scenarios where

virtual robots would send sensor data to the Command and Control Component operated by rescue

services so that they can assess the simulated emergency and act accordingly. Furthermore,

scenarios could be recorded from past events and then re-run for training purposes by using this

tool. The Command and Control Component for support rescue services will integrate all sources of


spatial information such as maps of the affected area, satellite images and sensor data coming from

the unmanned robots in order to provide a situation snapshot to the rescue team and thus facilitate

decision-making. The interactive human-machine interface that uses semantic information to

operate robots will be used for rescue operations. The Command and Control Component will equip

rescue teams with ICARUS robots. Control decisions will be coordinated and supervised and

therefore tasks will be executed with decreased risk. The main results of this activity are described in

section 3.9.


3. ICARUS MAIN SCIENTIFIC AND TECHNOLOGICAL RESULTS FOREGROUNDS

3.1. USER-CENTERED DESIGN REQUIREMENTS

ICARUS is an end-user driven project, aiming to bridge the gap between the end users and the

research community. This is a key differentiating factor and focus point of the ICARUS project.

Indeed, it has no sense to invest in the development of high-tech tools when these tools are not

useable by the end users on the terrain in practice. It has been the main focus of ICARUS to always

let the end users drive the developments, by letting them define the user requirements, involving

them in the tool development process and certainly let them assess the final resulting products in

operational validation scenarios. Furthermore, it has to be taken into account that the practical use

on the terrain of the ICARUS tools stands or falls with the acceptance of these tools by the end users.

Therefore, we have gone through great efforts to put these tools in the hands of the end-users,

explaining them the advantages of the systems and – maybe even more importantly – also pointing

out the disadvantages and the limitations.

A major testimony of the statement that ICARUS is very committed to end user validation, driving

acceptance and to the integration of the unmanned tools into the standard operating procedures of

relief workers was the intervention with an unmanned system in a real crisis theatre, in response to

the floods in Bosnia - Herzegovina. In spring 2014, Bosnia and Herzegovina and Serbia were hit hard

by catastrophic massive flooding, leading to at least 53 deaths and affecting millions of people. The

EU Civil Protection Mechanism was activated due to the catastrophic crisis and (among many others)

the Belgian state offered help by sending in the B-FAST team. Along with the B-FAST team, ICARUS

partner RMA sent an expert in robotics and an Unmanned Aerial System (UAS) in order to assist the

team for task such as damage assessment, dike breach detection, mapping, aerial inspection and for

re-localizing the many explosive remnants of war which had been displaced due to the landslides and

which created an extremely dangerous situation for the local population and the relief workers. To

our knowledge, this was the first ever international deployment of an unmanned aerial system by an

official state-run USAR team (so, not an NGO, relief organization, manufacturer, technology center,

..) in another country. The mission was highly successful, providing assistance on the crisis sites not

only for several international relief

teams (B-FAST, THW, …), but also for

the Bosnian Mine Action Centre. As

we were during the mission tightly

integrated with these end-users and

their procedures, this provided a

deep insight in their requirements,

procedures, indica-ting also the

bottlenecks towards the integration

of unmanned systems in the

standard operating procedures of

the SAR workers.

One of the most important ICARUS

actions related to the end users ICARUS deployed with B-FAST in Bosnia


relations was the compilation of the user requirements. A dual approach was followed specifically

tailored towards the USAR and MSAR users. The SAR end-user community was prospected to select

key players that would be willing to collaborate, thereby using multiple methods of information

gathering: membership of the end-user board, online surveys, personal interviews with key

stakeholders and – most importantly - involvement and feedback from end users during operational

tests of prototype ICARUS systems. The result of all these activities culminated in the compilation of

a user requirements document defining clearly the needs of urban and marine SAR teams related to

unmanned tools. This document is of high value as it can also be used an input to policy for the

introduction of unmanned tools is SAR operations. Both UN-INSARAG and EU-DG-ECHO have

therefore already shown their interest in this work.

The information from the end-users was transformed into an architectural solution, thereby

specifying the requirements of all systems involved in ICARUS. To assess the performance of the final

systems against these user and system requirements, operational validation scenarios were defined

for all ICARUS components. The approach followed for conceptualizing the validation scenarios is

inspired by the approach followed by the National Institute for Standards and Technology (NIST). In

this context, each of the validation scenarios consists of three aspects: a detailed scenario, a

capability score sheet and a score sheet for the different metrics (Key Performance Indicators). This

makes it possible to qualitatively and quantitatively evaluate the performance of the different tools

during the demonstrations, following standardised procedures as defined in an operational service

validation scenarios definition document. This is again a very important document, with a high

potential impact as all unmanned tools which would in the future be integrated in SAR operations

will be required to be tested, validated and certified using agreed procedures and this document

spreads a basis for doing this work. For this reason, the work done in this context has received

interest from UN-INSARAG, EU-DG-ECHO and the

Japanese government.

ICARUS made a point of deeply committing end-

users with its activities. As a testimony to this

statement it can be mentioned that the final land

demonstration was attended by around 100 key

stakeholders. ICARUS was also recognized as an

important player by end-user organisations, which

led to invitations for high level events, such as the

UN World Conference in Disaster Risk Reduction in

Sendai, Japan and the yearly UN-INSARAG team leaders meetings and the 2015 INSARAG Global

Meeting. Also on more policy related matters, ICARUS was able to make an impact, as ICARUS had

the honour to be invited by DG-ECHO to set up an outdoor demonstration during the EU Civil

Protection Forum, where Mr Christos Stylianides, EU Commissioner for Humanitarian Aid and Crisis

Management inspected the ICARUS tools during the first-ever legal flight of an RPAS in the EU capital

(organized by ICARUS).

In collaboration with the CRASAR team in the USA, a set of best practices towards end-users was

compiled based on lessons learnt from real operations and technological innovations obtained. These

best practices provide a quick reference guide for governments agencies and NGO’s who want to

incorporate unmanned tools in their operations.

http://www.fp7-icarus.eu/sites/fp7-icarus.eu/files/imagecache/thumb800/pic12.jpg


3.2. SENSING

One of the most critical and important tasks for crisis management in case of a large-scale disaster is

the search for human survivors. To maximize the chances of survival, victims have to be fount as fast

and efficient as possible. Icarus has proven that unmanned vehicles equipped with advanced sensing

technology are an excellent choice for this task, which have numerous advantages compared to

classical human search and rescue teams. An integral part of the overall concept is optical sensing

technology, which has to be combined with fast data acquisition, processing, and software for human

detection. The partners in the Sensing Workpackage addressed the major challenges for hard and

software systems for victim detection on unmanned vehicles.

In a first step, a detailed evaluation of the requirements for optical systems for victim detection was

performed. User cases were considered and a high level architecture was developed. An analysis and

comparison of commercial visible and infrared cameras was provided.

For many systems, commercial visible and infrared cameras are sufficient. Mechatronics integration

was performed and the Icarus Common Sensing and Processing Unit was developed. For airborne

detection, the best solution is the Visual Inertial (VI) – Sensor. It has been successfully integrated into

three of the ICARUS UAS fleet, namely the endurance airplane, the outdoors Quadrotor, and the

Multicopter.

The ICARUS Common Sensing and Processing Unit is based on two commercial visible light cameras for stereo-vision and

one FLIR camera for mid-infrared vision. It was integrated into several ICARUS platforms

Commercial mid-infrared cameras have limited detection speed and it is difficult to achieve low noise

levels. The partners in the Sensing Workpackage investigated if these challenges can be overcome

using another approach for semiconductor mid-infrared cameras. They developed the first quantum

cascade detector (QCD) pixel arrays. In combination with the commercial FLIR camera that operates

in a different spectral region, this will enable differentiation between black-body radiation sources

and other heat sources, allowing separating humans from other hot objects.

In a first step, the work package partners theoretically investigated photonic cascade semiconductors

and developed novel simulation tools for photonic semiconductor structures. These simulations were

the foundation for optimized designs, which were successfully produced in semiconductor growth

facilities in Austria. The WP partners achieved a major break-through in quantum-cascade detector

technology. The semiconductor chips represent now the current state of the art: they have an order


of magnitude higher responsivity than previously achieved for QCDs. The ICARUS partners developed

the first quantum cascade pixel array detector that was ever manufactured. They achieved a full

processing cycle for the QCD detector, starting from the semiconductor growth and finishing with the

final detector.

The WP partners achieved a scientific breakthrough in photonic semiconductors that have high

visibility and make QCDs highly attractive for numerous novel applications. Icarus results enabled a

new approach for a sensing device that was honoured with the Photonics21 Student Innovation

Award.

From semiconductor chip to final hermetically sealed detector

An environment hostile to electronics and harsh conditions during the field of applications is a major

challenge for the detector housing, which greatly affects the camera systems in addition to high

mechanical loads on the vehicles. To meet these requirements, the WP partners developed an

appropriate integration and packaging technology, which is fully hermetically sealed and supports

operation at environmental temperatures.

Moreover, the partners showed a proof-of-principle experiment for remote CO2 detection,

demonstrating that systems for survivor detection based on this technology are in principle feasible.

Another important task in the Sensing Workpackage was the development of control, acquisition,

and detection software. Control software addressing individual detector elements for processing on

the ICARUS host platform was successfully developed. Efficient methods for stereovision and camera

calibration were implemented and software development for human detection based on data from

all camera types was finished.


Thermal images with detected victim positions

3D dense map from Marche-en-Famenne provided by UAS.

ICARUS UAS fleet. Top: AtlantikSolar from ETH. Bottom left: AROT from EURECAT. Bottom right: Skybotix indoor multirotor

3.3. UNMANNED AERIAL SYSTEMS

The ICARUS unmanned aerial systems (UAS) consist of a team of aerial robots. Their main objectives

is to provide fast and reliable aerial information to the search and rescue (SAR) teams during their

planning and mission phase improving the time to rescue and the efficient allocation of the SAR

resources.

Three different robots are developed and

adapted for assistance of the SAR teams. They

all equip the same visual-inertial sensor

developed for the UAS. This sensor suite can

synchronise visual and thermal images

together with IMU data which can be used for

further map generation.

The solar powered fixed-wing UAS

AtlantikSolar has improved endurance

capabilities to stay airborne for multiple days

due to the integrated solar cells on the wings.

This was shown with a successful flight

demonstration to break a new world record of

over 81 hours continuous flight by travelling 2316km in July 2015 in Switzerland. The integrated

autopilot can effectively track a given waypoint path while copying with strong winds and is capable

of autonomous landing. Thus, it can autonomously execute an inspection mission while covering a

large area or simply stay airborne and act as a communication relay with its integrated WiFi antenna.

It only needs high-level supervision over the C2I while controlling the UAS.

The equipped payload consists of a visual and thermal camera to provide aerial imagery. Together

with its fast deployment time, the fixed-wing UAS is an effective mean to gather first visual

information from the disaster area. After returning from its mission, the stored imagery are

downloaded to the ground station where the imagery is processed to build a dense 3D map of the

environment.

Furthermore, victims can be

detected using the combined

visual and thermal images. A

notification is send to the C2I

system such that the SAR

teams are aware of potential

victim locations.

The visual information

gathered form the fixed-wing

UAS are taken from a height

of approximately 200 m


Detailed 3D map provided by the AROT UAS

Generated map for collision avoidance.

above ground, thus it cannot provide information of specific disaster side details. Thus, by arriving at

the disaster side, the SAR teams can deploy the rotary-wing UAS AROT. The SAR teams can use the

AROT UAS to gather information about object with a high rate of detail and precision. Its capability to

hover enables it to fly at close proximity to interesting object.

The AROT UAS is equipped with both visual

and thermal cameras. A high power

processor can calculate a detailed map of

the object online and send it to the C2I. The

UGVs and SAR teams can use this

information to plan their further actions. The

integrated victim detection algorithm can

automatically detect victims, to verify the

victims found by the fixed-wing UAS and to

detect missed ones. Close inspection of the

victims can give some visual feedback of

their health. Thy UAS can track the victims until help is arriving or it can use the integrated delivery

mechanism to drop a lightweight emergency kit or a water bottle to help the victims directly.

The UAS provides direct visual feedback for the pilot to control the UAS. Its versatile control provides

different types of user interaction with the pilot ranging from complete manual control, position hold

to complete waypoint following and autonomous start and landing from the C2I depending on the

mission objective.

The indoor multirotor UAS is capable of providing first-hand information of potentially collapsed

indoor environments. The UAS has a small footprint, which allows easy access into building by small

holes or open windows. Protection units can protect the propeller from unintentional hits with the

surroundings. It has the advantage to fly and explore obstructed buildings, which are hardly to be

trespassed by UGVs and too dangerous for being accessed by people. Its focus relies on providing a

live feed from its visual and thermal cameras to the SAR teams.

In order to fly in such cluttered environments, the UAS has a high autonomy. It can used the stereo

camera set to identify online the obstacles in the rooms. It generates an occupancy grid map and

localise itself within it. With this map, it is aware of potential obstacles and autonomously avoids

them. It can be tele-operated over

visual feedback to fly from room to

room. Further, it can communicate

and be operated over the C2I

system and send the visual and

thermal streams. Using the live

imagery, the SAR teams can thus

search for victims inside the building

and gain insight of the building and

its structural stability.


3.4. UNMANNED GROUND VEHICLES

Two UGVs were employed in ground operations, the SUGV and LUGV (Small and Large Unmanned

Ground Vehicle).

The LUGV was used in the project as tele-

operated heavy duty ground robot.

The main intended task for this machine is to

grant access to the SAR wherever the way is

obstructed by debris.

During the deployment phase LUGV is able to

follow a path composed by multiple waypoints,

localizing itself using the GPS and other

navigation sensors.

A pair of laser scanners mounted both forward

and backward and a stereo-camera allow proper environment sensing with the aim to avoid eventual

obstacles and create a navigation map.

The safety during navigation is provided by a safety chain consisting of four emergency stop buttons

on the machine (one per each side) and a wireless emergency stop control. Whenever one of the

button is pressed or the communication with the wireless control is lost for some reason the robot

motion is completely stopped and broken, as well as the arm.

A gas sensor was mounted during the final demonstration to alarm SAR operators in case of leakage

of hazardous gases.

The LUGV demonstrated to be able to break concrete walls and to remove debris obstructing access

to destroyed buildings. These tasks were performed mounting either a jackhammer or a gripper on

its hydraulic arm, with the operator controlling the manipulator using a remote control. In test phase

such arm was proven to be controlled by the exoskeleton from WP320.

Mounting instead a box on the arm it is relatively easy to transport SUGV and deploy it on top of the

roof of a building whose access from ground level is not possible.

LUGV deploying SUGV on a roof LUGV removing debris

The SUGV has a configuration that is similar to LUGV but its intended scope is different. Its compact

dimensions make it suitable for entering level buildings, exploring and looking for victims.


With a camera mounted on the arm’s

gripper it is able to extend visual

feedback to small holes and

inaccessible corners.

Equipped with caterpillars, it can drive

over rubble, stairs and uneven ground.

Beside the basic telemetry sensors it is

possible to notice two small laser

scanners on the sides and a Kinect2

device in the front, such sensors

provide a 3D point cloud that is used

for mapping and navigation.

In a similar way as LUGV, a two-dimensional map is created on-board and used for safe navigation.

The collision avoidance software helps the operator to drive safely in hazardous environment

without hitting obstacles. Such feature becomes critical when the delay in communication makes

difficult a precise motion control.

Cameras are mounted on the gripper, on the forward and backward side. A CO2 sensor provides

measurements about CO2 level in the air.

After locating a possible victim a voice communication between the latter and the SAR operator is

possible via the speaker and microphone available on SUGV. Basic instructions can be given to

reassure the victim and an emergency kit or water bottle can be carried.

SUGV was connected to the C2I framework and controlled remotely by the operator using a joypad

and the feedback from cameras and other sensors. The C2I operator has several information about

the robot such as its GPS position, inclination, battery level, CO2 readings and manipulator joints

values. The motion can be controlled manually with the joypad or semi-autonomously sending a list

of GPS waypoints that the robot has to reach in sequence.

The communication with the C2I was done using an improved wireless infrastructure. For this reason,

a communication box and several omnidirectional antennas were mounted on top.

An important feature realized within the project

was the haptic control of the manipulator using

the exoskeleton provided by WP320. Standing in

front of the C2I control station, the operator was

able to open a door from the handle using only

the visual feedback from the manipulator

camera and a 3D model of the arm position.

To help the operator in opening doors and

manipulating objects a laser pointer has been

mounted on the gripper. Such pointer projects a

light pattern on to the object to be grasped,

basing on the pattern size and shape the

operator can estimate the distance between the

gripper and the object.

SUGV arm controlled by exoskeleton


3.5. UNMANNED MARITIME PLATFORMS

Development of a robotized survival capsule

The robotized survival capsule is a robotic surface platform that carries an uninflated life raft and is

able to inflate it close to survivors. Due to its characteristics, the capsule can approach survivors

without endangering them and automatically inflate the life raft close to them. It constitutes,

therefore, the last element in the ICARUS integrated toolkit for maritime search and rescue.

This capsule can perform autonomous operations or be remotely operated from a control station,

through a radio link. It is equipped with several sensors including a video camera to gather

information about the victims state.

Robotized capsule with life raft before inflation (left) and after inflation (right).

The capsule is 1.5m long and 0.5m wide and weights 20kg. It has a payload capacity exceeding 15kg,

allowing for the transportation of a 4 people life raft. It is propelled by a fully protected water jet

system powered by an electric engine. It can reach a top speed of 5 knots and can operate for 20min

at a 3knots, resulting in a maximum range of 2km.

Development of capsule deployment system

The capsule deployment system is a mechanical structure that can the adapted to different

unmanned surface vessels (USVs) in order to carry robotized capsules to remote areas and launch

them near detected victims. It was designed in a modular way so that it can be used to transport one

or more capsules and be installed on USVs with different characteristics. To increase flexibility and

simplify installation procedures, these structures also include an electronics box that is responsible

for the release of the capsules via a radio command.


Deployment system installed on U-Ranger USV (left) and ROAZ II USV (right).

USV Sensing and Perception

The following results were obtained:

Algorithms for the detection of victims on the water using visible and thermal cameras and

geo-referencing of their position, allowing for detections at distances up to 200 m;

Algorithms for detection of obstacles on the water combining information from multiple

sensors, including radar and multilayer laser scanner.

USV Behavior sets

Main achievements:

Development and implementation of high level behaviors on USVs endowing them with

increased autonomy allowing remote operators or mission supervisors to focus on payload

data (victim detection, situation awareness) rather than in the operation of the USVs

themselves; these behaviors include loitering, waypoint tracking, obstacle avoidance, and

dynamic reference tracking;

Development and implementation of reactive collision avoidance modes on USVs in accordance to

COLREGs (International Regulations for Preventing Collisions at Sea), increasing the safety of USV

operation and contributing


3.6. HETEROGENEOUS TEAMS AND NETWORK CENTRIC OPERATIONS

The ICARUS project involves a team of assistive unmanned air, ground and sea vehicles. In order to

support the human crisis managers, they must collaborate as a coordinated, seamlessly-integrated

team in the single Command and Control Station (C2I) in the field.

A heterogeneous fleet is the one composed by elements of different types such as the ICARUS team

that includes up to nine different types of vehicles. Each of these robots has been developed by a

different ICARUS partner, using their own designs, development frameworks and middlewares. Thus,

a strong effort had to be devoted to their integration as a single team, which was the responsibility of

this work package.

The ultimate objective was to achieve systems interoperability, which can be understood as the

ability of robots to operate in synergy to the execution of assigned missions and, therefore, enables

diverse teams to work together, sharing data, intelligence and resources. ICARUS has proposed the

adaptation of all the vehicles to a single standard external interface as a method to ensure

interoperability. Each single team kept using their own tools inside their systems as long as the

interaction with the rest of the team was ruled by an interoperability standard. This approach

provided a common framework for the development of the unmanned assets, minimizing the

integration time and costs by avoiding custom implementations.

Our strategy in terms of interoperability was to build upon existing body of work in the field, avoiding

duplicating and re-inventing proven technology. During the initial steps of the work, the most

relevant multi-domain interoperability protocols for unmanned systems were identified and

evaluated against the ICARUS requirements and foreseen scenarios. As an outcome of this analysis,

the ICARUS standard interface for interoperability is heavily based on the Joint Architecture for

Unmanned Systems (JAUS). Gaps identified during the analysis were filled by extending the protocol

with the required functionality.

All this functionality is provided to the robot manufacturers as a software library referred to as

ICARUS interoperability layer. Just by instantiating a software module named JAUSRobot, the vehicles

automatically become compliant with the standard. This module acts as a bridge between their

internal and external worlds. This interoperability layer is also responsible for the integration of the

ICARUS communication network and the Command and Control Station on each individual platform.


Robot’s adaptation strategy

The validation of these developments were organized as a series of operations involving different

combinations of pairs of air, ground and sea vehicles during the integration trials carried out in 2014.

The aim was to demonstrate the achievements in terms of system interoperability. Some examples

of the multi-robot collaboration experimented during ICARUS are described here and illustrated in

the images below:

Multi-stage aerial reconnaissance, mapping and victim search: a fixed-wing UAV provides an

initial assessment of the disaster area to identify the critical sectors, followed by an

outdoors multirotors UAV providing a close-up look at a single sector.

Multi-domain indoors victims search: a small ground robot and a small indoors multirotors

cooperate in search for survivors inside a building.

Multi-domain victims search in water: a fixed-wing, a multirotor and surface vehicles

cooperate searching for survivors in the water.


Examples of collaborative mapping and victim search missions:

Multi-UAV operating outdoors (top left), Ground+Aerial operating indoors (top right), Aerial+Sea missions (bottom)

After this mid-project validations by pairs, three full-team validations were performed during the

final project demonstrations:

the maritime trials and demonstration in Alfeite, Lisbon (Portugal) in July 2015,

the land trials and demonstration in Marche-en-Famenne (Belgium) in August 2015

and the participation in the euRathlon competition in September 2015 where the project

received the Best Multi-Robot Coordination Award by the IEEE Robotics and Automation

Society (RAS)

Together, these large-scale operational exercises complete the validation of the ICARUS

interoperability standard interface. Therefore, ICARUS as a project has demonstrated multi-domain

multi-robot heterogeneous interoperability in realistic Search and Rescue operations.


3.7. COMMUNICATION

ICARUS Tactical Communications Network

Communications (COM) have become a key concern in large crisis events which involve numerous

organisations, human responders and an increasing amount of unmanned systems which offer

precious but bandwidth-hungry situational awareness capabilities. The ICARUS team in charge of

COM provisioning —lead by INTEGRASYS with contributions from RMA and QUOBIS— has designed,

implemented and tested in real-life conditions an integrated multi-radio tactical network able to fulfil

the new demands of cooperating high-tech search and rescue teams acting in incident spots. The

ICARUS network offers interoperable and reliable communications with particular consideration of

cooperative unmanned air, sea and land vehicles

During the final project demonstrations conducted at the Almada Camp of the Portuguese Navy and

the Roi Albert Camp of the Belgium Army, the ICARUS network and associated tools have provided

significant value for mission commanders along different mission phases. First, as a powerful

deployment planning tool; and second, as a network management and optimisation tool able to

seamlessly connect all robots telemetry and tele-control capabilities to the ICARUS C2I stations,

mitigating eventual coverage and throughput shortcomings arising during operations.

ICARUS COM working in the Land Demonstration

Communications interoperability and performance optimisation

A real-time management and control middleware (COMMW) has been the key piece enabling

interoperable and resilient tactical communications in the ICARUS scenario of crisis response

operations covering air/sea/land portable and mobile nodes.

The COMMW stack has been designed with fast deployment, interoperability and performance-

based, real-time self-management capabilities in mind. It uses standard datalink technologies (ETSI

DMR and IEEE 802.11x) to transparently offer an unified communication capability. Applications are

provided with flexible end-to-end connectivity and data transfer patterns (including raw transport of

application data over the link layer) along with high-granularity Quality of Service; hiding low-level

details of underlying datalinks. One of the application interfacing options supported in the COMMW

is the JAUS robotic middleware used in ICARUS.

A combination of both centralized and peer-to-peer algorithms provide adaptive and harmonised

handling of radio channels using a cognitive spectrum approach as well as link-layer and network-

layer addressing, routing and capacity management functions; allowing for rapid deployments under


unknown spectrum occupancy conditions, harsh propagation environments, large throughput

demands and varying platforms constrains.

The COMMW has been implemented on open, Linux-based embedded computing platforms with

proper kernel and user-space extensions enabling an overall optimisation of the network stack;

including the queuing components present in the system data path which may largely affect

throughput and latency of applications.

Regarding WLAN datalinks, the COMMW handles a full-mesh network where key system parameters

affecting the overall network performance particularly range and throughput are controlled and

optimised in real-time according to predefined and reconfigurable operator policies. These

parameters refer to three distinct areas: i) radio link, ii) CSMA/EDCA protocols and ii) mesh routing

protocols

Left: SUGV COM box and set of antennas; Right: ICARUS DMR hardware transceiver

The DMR datalink technology provides coverage over long ranges (typically beyond 5 km in open

areas) and can handle both voice and low-rate data. The so-called soft-DMR modem implemented in

ICARUS enables adaptation of key transmission parameters (e.g., coding rate, delivery mode, channel

access mode or power) on a per- flow basis, accounting for traffic type (data or voice) and required

delay and reliability. Furthermore, a node discovery service and a capacity management protocol

(allowing allocation of throughput levels per node) were implemented to strength the networking

aspects of DMR. All these characteristics make the soft-DMR well suited for networked tactical and

mission critical applications.

The COMMW framework seamlessly integrates and jointly manages both WLAN and DMR links

described above according to dynamic mission requirements. During implementation, ICARUS has

made use of HW/SW mass-market technologies thoroughly engineered for professional performance

exploiting unlicensed spectrum in UHF, 2.4Ghz and 5Ghz bands. In real safety-critical operations,

access to radio spectrum with proper EIRP limits must be guaranteed to ensure required throughput

and operation in long ranges or harsh propagation scenarios such as rubble or indoor. The COMMW

includes by-design specific provisions to ease integration of new datalink technologies and extend

operation to new frequency bands, by adapting the cognitive radio functions to implement any

required spectrum access rules. Existing 802.11 COTS professional transceivers which can tuned to

operate in any band up to 6Ghz will allow to readily reuse all of the COMMW/COMCON 802.11

capabilities in low-frequency spectrum particularly suitable and eventually protected for Public

Protection and Disaster Relief (PPDR) applications. In the migration phase towards

commercialisation, the team is also working on the integration of LTE services; either commercial (if


available on crisis location) or PPDR-specific (e.g. operating in the 700MHz) to be used as a

complementary incident-spot capacity as an interconnection means between distant incident-spots.

While low-layer LTE functions would be out of control of ICARUS COM reducing optimisation

possibilities, the framework is already able to evaluate in real-time the throughput and latency

offered by external networks, which would be used to manage the available capacity as a whole.

Communication management tools for mission operators

The ICARUS COMMW provides a convenient set of tools for mission network operators for easy

configuration of COMMW nodes based on capacity allocation targets for both locally-generated and

relayed traffic; differentiating among individual application flows and supporting latency, reliability

and security requirements in addition to throughput. Operators are provided with a set utilities for

guidance on setting the different configuration parameters. Such settings can be changed

dynamically during the mission execution.

Furthermore, a rich graphical environment called COM console (COMCON) has been developed to

support planning, supervision and optimisation of the integrated multi-radio tactical network,

combining simulation features with real-time Monitoring & Control capabilities.

At planning phase, the COMCON accurately characterizes COM components, propagation

environments, RF interference and vehicles platforms in order to assess global network performance

over wide operation areas; as well as the performance of individual terminals along given mission

routes. This allows in particular to take proper decisions on radio bands and channels; antennas

pattern/polarization and transceiver features for every node in the network. Furthermore, the

eventual need and location of network relays can be assessed. The tool includes in particular

UHF/2.4GHz/5GHz propagation models for indoor, rubble and sea environments; as well as protocol

models of 802.11 mesh networks enabling informed planning of CSMA-related parameters and

reliable estimation of throughput performance.

ICARUS COM Console used in mission planning and in mission operations

At operations phase, the COMCON features a centralized monitoring of all key parameters affecting

the network performance, allowing to mitigate coverage and throughput problems by timely

reconfiguration and eventual reallocation of nodes. Some optimisation actions those of limited

scope are performed automatically by the COMMW itself, while some others those of wider

scope will require human operator intervention to decide the best solution given the current

mission conditions.


3.8. COMMAND AND CONTROL

The Command, Control and Intelligence (C2I) system is the central system for generating joint Search

and Rescue (SAR) mission plans consisting of humans and unmanned systems, commanding and

monitoring these assets during field operations. The C2I system consists of the following sub-

systems:

Mission Planning and Coordinating System (MPCS)

Robot Command and Control center (RC2)

Portable Exoskeleton

Mobile first responder device

The MPCS is a browser based utility that gathers mission functionalities by facilitating central mission

planners to specify missions over an intuitive graphical interface. Maps hosted within a local

Geographic Information System (GIS) forms the basis of this SAR mission planning system. A set of

tools are provided to support mission planners to author SAR mission tasks based on specified

objectives, the assembly analysis of data collected from the mission sections (Common Operational

Picture – COP), visualization or rendering of these data by users and high level monitoring of mission

execution. The MPCS is primarily connected to the SAR first responders – essentially embodied as

RC2s and to external global crisis data sources such as MapAction and GDACS.

The RC2 is a standalone software application that provides the primary field operator user interface

and includes all the tools necessary to monitor and control the multi-domain robots (UAV, UGV and

USV) simultaneously. The RC2 hardware is designed for outdoor use, keeping in line with the end

user requirements. The system consists of a rugged laptop within a portable ruggedized case

integrated with secondary displays, joystick controllers, power supply and wireless communication


antennas. Initial tasks are to synchronize mission plans and external crisis data, and deploy the RC2

to the relevant sector designated by the MPCS. The RC2 provides user interfaces and data critical for

the command and control of multiple, heterogeneous robots and allows first responders with mobile

devices to receive the latest mission updates and sectors maps. Robot specific sensor data (cameras,

3D point clouds, robot health etc.) acquired at the RC2 through the ICARUS mesh communication

framework can be visualized, geotagged and fused into the embedded map client which is connected

to a local GIS repository. Robot specific commands and operations are dynamically configured and

made available to the RC2 operator.

The wearable exoskeleton is composed of a 7 degrees of freedom arm (from the shoulder to the

wrist). It is mainly based on rapid prototyping process (laser sintering) with Alumide (composite

aluminium and polyamide) and PA-GF (glass fiber reinforced polyamide). The exoskeleton allows the

RC2 operator to control a robot arm mounted on a ground robot intuitively and accurately,

augmented with force feedback. It interfaces with the RC2 providing high fidelity haptic rendering

consisting of a 3D visual interface of the robot model and its immediate environment.

Using a mobile device, first responders

can push and pull messages, photos

and position information over the

network to the RC2. The mobile

application has been deployed on an

Android device. All mobile devices

connect over Wifi to the GIS in the RC2

within its vicinity. A SAR responder can

insert geotagged images and messages

and access latest map updates,

geotagged messages, robot positons

and information from other SAR

responders.


3.9. TRAINING AND SUPPORT

Mobile Data Center

The ICARUS HPC (High Performance Computing) solution is based on a Server Supermicro RTG-RZ-

1240I-NVK2 shown in the figure below. This server supports NVIDIA GRID and CUDA Technology and

provides a parallel programming solution for processing large data sets. This server makes it possible

that data from many sources (UxV) can be efficiently integrated and visualised over Ethernet. The

server was embedded into a ruggedized chassis. The chassis can by carried by two people. The

chassis protects the server from vibrations and mechanical stress. It is extended by a mobile display-

keyboard-mouse component for on-site server management purposes. It is a fully integrated and

autonomous solution. It requires 2kV AC power. The communication system is based on a WiFi

router that can establish a local network within a range of 25m. In the case of access to Ethernet, the

server is connected directly to the network. It can also be integrated with a regular Data Centre.

HPC solution for ICARUS project is based on Server Supermicro RTG-RZ-1240I-NVK2.

Unmanned Mobile Mapping System (UMMS)

The UMMS for the ICARUS project can be equipped with different 3D sensors. The first configuration

is composed of a 3D Z+F Imager 5010 laser measurement system. In this configuration, the system

has a scanning range of 170m with an accuracy of ca.1mm. The platform is also prepared to work

with other, custom-made, scanning systems of different range and accuracy.

Unmanned Mobile Mapping System


Mission Planning Control Station

The MPCS (Mission Planning Control Station) is a crucial part of the Remote Command and Control

Station (RC2) that is dedicated to the initial planning of the search and rescue mission. The MPCS is

connected to the RC2’s of unmanned platforms and shares a common interface with the former.

Mission plans created in the MPCS can be passed directly to the operator console. The MPCS training

concentrates on familiarizing the user with the MPCS interface and functionalities. It consists of a

user manual describing in detail each of the elements of MPCS and a set of simple exercises to

familiarize the user with the tools. These exercises can also be used to verify the readiness of the

trainee.

ICARUS Support System

The ICARUS support system is dedicated to handling all data gathered during the course of a rescue

operation. The system consists of software for data preparation, matching processing and

visualization as well as interface tools that allow the user to modify and better explore the dataset.

During the evaluation phase, all this software was tested and adjusted to better suit the end user

needs. The main functionalities of support system that were evaluated are:

Raw 3D scan matching and creating 3D models.

3D point clouds processing and classification.

Visualization of 3D data.

Working with 3D data.

ICARUS Training Tools

These tools include UGV, UAV and USV simulators dedicated to operators’ training. ICARUS vehicles

are simulated in virtual environments that can be modelled based on real measurements based on

input from the Unmanned Mobile Mapping System.

UGV training tool.


USV training tools.

UAV training tools


3.10. VALIDATION OF THE ICARUS SYSTEM IN AN EARTHQUAKE RESPONSE SCENARIO

The ICARUS project considers two main demonstrations to validate the tools developed during the

whole duration of the project: a land demonstration, simulating an earthquake in an urban

environment and a marine demonstration, simulating a shipwreck in coastal waters. Here, we discuss

the first demonstration, where the Belgian First Aid and Support Team (ICARUS-partner B-FAST)

intervened in response to a simulated earthquake, helped by the ICARUS tools developed within the

project. This event took place at the military base of Marche-en-Famenne, Belgium and had as main

purposes to validate each of the individual ICARUS tools, but also – and more importantly – validate

the performance of these tools as an integrated system and their integration into the standard

operating procedures of the end-users which were supposed to use these novel technological tools.

The public demonstration was attended by around 100 stakeholders and presented over the course

of a full day 6 distinct operational scenarios:

1. C4I Integration

The purpose of this scenario is to demonstrate the

integration of ICARUS tools into the procedures of the SAR

workers. In practice, this means that the ICARUS command

and control tools need to be able to integrate in the On-Site

Operations and Commance Centre (OSOCC), pull in data from

registered sources and produce data in standardised

formats. For this purpose, an OSOCC was set up and the

different interoperability aspects were validated.

2. Mission Planning

During this phase of the operation, the mission planner at

the OSOCC assigns sectors and tasks to SAR teams. He does

this by fusing information from different data sources (GIS

maps, UAS data). Specific for this scenario is that the ICARUS

endurance aeroplane is used by the mission planner as a tool

for increasing his situational awareness, by mapping the

crisis area and transmitting real-time data to the command

station.

3. Deployment

During this phase, the USAR teams move towards and deploy

into a sector assigned by the mission planner via the C2I. The

main purpose of this scenario is to test the (rapid)

deployment capabilities and the integration of the

communication and C2I system. Another purpose of this

scenario is to test the network and C2I management

capabilities when confronted with dynamic team and

resource allocations.


4. SAR Operations on a semi-demolished apartment building

During this phase, the USAR team, helped by the UGV and UAV systems, rescues victims trapped in a

semi-demolished apartment building. The main purpose of this scenario is to test the assessment,

search and rescue capabilities of the outdoor rotorcraft (support for situational awareness, mapping,

victim search, collaboration with canine victim search teams, rescue kit delivery) and the large UGV

(debris clearance, breaching an entry to blocked structures, executing shoring operations) and their

collaborative operation.

5. SAR Operations on a semi-demolished school building

During this scenario, The USAR team, helped by the UGV and UAV systems, rescues victims trapped

in a semi-demolished school building. The main purpose of this scenario is to test the assessment,

search and rescue capabilities of the small UGV and the indoor rotorcraft (semi-autonomous indoor

flight) related to indoor victim search and their collaborative operation mode.

6. SAR Operations on a semi-demolished CBRN warehouse

During this operation, the USAR team, helped by the UGV and UAV systems, rescues victims trapped

in a semi-demolished warehouse building. The main purpose of this scenario is to test the

assessment, search and rescue capabilities of the small and large UGV (deployment of the small UGV

by the LUGV, manoeuvring in small spaces, executing grasping operations with the help of the

exoskeleton) and the indoor and outdoor rotorcraft and their collaborative operation mode.


3.11. VALIDATION OF THE ICARUS SYSTEM IN A MARINE INCIDENT RESPONSE SCENARIO

Demonstration of multiple robotic operations in realistic SAR scenario

The sea demonstration involved fixed wing and rotary aerial robots and also unmanned surface

vessels and robotized capsules. A realistic scenario was organized simulating an accident on a

ferryboat with victims in the water.

Ferryboat used in sea demonstration.

The aerial robotic platforms were used to survey the accident area spotting and localizing victims on

the water, as well as tracking their drift due to currents.

Aerial platforms performing area surveys.

Unmanned surface vehicles were used to perform close range surveys as well as deploy robotized

capsules close to the victims on the water.

Unmanned surface vehicles carrying capsules: ROAZ II (left), U-Ranger (right).


Multiple robots in operation (left) and victims on life raft (right).

Demonstration of C2I and communication tools in realistic SAR scenario

The ICARUS C2I and communication tools were used during the demonstration to operate the robotic

platforms and to provide relevant data about the accident, as well as locations of victims to the

operators. In particular, the use of a common communication and interface framework to

simultaneously operate multiple and heterogeneous platforms was one of the greatest achievements

of the project.

Increased awareness of unmanned systems capabilities in SAR operations

The sea demonstration included the visit of several authorities, some of them coordinating maritime

search and rescue organizations. Official presentations of the ICARUS project were conducted,

attracting the Portuguese media (national radios and television channels), resulting in several

interviews and short reports about the project, contributing to an increased awareness of the

usefulness of robotic platforms in search and rescue operations.

Establishment of field trials procedures for requirement validation

The established procedures included several ways to validate requirements and also to assess the

capabilities of individual platforms and of their coordinated use.

All demonstration activities were based on a storyboard describing in the accident and identifying

the roles of the different tools. Such storyboard was used to define sets of individual and integrated

experiments during which it also possible to validate requirements and gather data for making

performance analysis of each system or set of systems.

System validation also included the assessment of the experiments by experts external to the project

research team.


4. ICARUS IMPACT

4.1. IMPACT ON THE GENERAL SOCIETY AND WIDER SOCIETAL IMPLICATIONS

The main objective of ICARUS is to increase the safety and security of the citizens by introducing

novel unmanned tools in the operational toolkit of search and rescue workers. ICARUS is a research

project, so the main expected impacts will be more visible over the longer term, but due to a

diversified mix in the project of higher and lower TRL activities, the project has also achieved an

important impact on short term. These impacts are described more in detail in the following table:

Denomination ICARUS Impact Societal Implication

Short

term

Impacts

on the

field

(within

the

lifetime

of the

project)

Wider general

societal

acceptance of the

use of RPAS

ICARUS has performed multiple

demonstrations (e.g. the EU CP

Forum where the first legal flight in

the EU capital was performed) and

conferences (not less than 7 RPAS

conferences were organized in the

scope of ICARUS) open to the

general public to showcase the good

use of these tools for disaster relief

and civil protection operations.

The insertion of RPAS in civil

airspace stands or falls with the

public acceptance of these tools.

The ICARUS actions are inspired

by the will to drive this public

acceptance.

Wide acceptance

in the SAR

community on

the use of RPAS,

including bringing

these novel

unmanned tools

to the field.

ICARUS has worked together with

the end users, incorporating them in

our requirements definition,

developments, validation and

demonstration activities such that

they could experience the

advantages of the systems first

hand. Moreover, during the Bosnia

mission (see below), the RPAS tools

were brought into a real crisis area.

End users are now convinced

about the advantages of the

unmanned tools (certainly the

RPAS, which they want to deploy

as soon as possible). This will

help to speed up rescue

operations (and make them more

safe for the SAR workers) and

save lives.

Wider acceptance

among policy

makers on the

use of RPAS for

crisis

management


demonstrations (e.g. the EU CP

Forum where the first legal flight in

the EU capital was performed) and

conferences (not less than 7 RPAS

conferences were organized in the

scope of ICARUS) to convince policy

makers about the advantages of

unmanned SAR tools.

Policy makers seem now

convinced about the advantages

of the unmanned tools (certainly

the RPAS), which will drive to

define the policies and

procedures for introducing these

tools into the crisis management

framework.

Direct assistance

of search and

rescue teams to

During the Spring floods in 2014 in

the Balkans, an ICARUS RPAS

assisted the international relief

By helping with the operations of

the international relief teams,

ICARUS also helped the local


help the local

population during

flood relief

operations

teams for tasks such as dyke breach

detection, optimization of the

position of the water pumps, area

prioritization, structural inspection

and situational awareness.

population in Bosnia. ICARUS was

officially praised and thanked for

its operations by the Bosnian

Ministry of Security.

Direct assistance

of demining

teams to help in

the assessment of

suspected

hazardous areas

During the Spring floods in 2014 in

the Balkans, an ICARUS RPAS

assisted the local demining teams for

re-locating the many landmines

which had shifted position.

By helping with the operations of

the demining teams, ICARUS also

helped the local community in

Bosnia.

Mid-term

Expected

Impacts

on the

field

(within 4

years

after the

end of

the

project)

Wide acceptance

in the SAR

community on

the use of

unmanned

ground and

marine vehicles,

including bringing

these novel

unmanned tools

to the field.

ICARUS has worked together with

the end users, incorporating them in

our requirements definition,

developments, validation and

demonstration activities such that

they could experience the

advantages of the systems first

hand. This should lead to the

effective use of unmanned ground

and marine SAR vehicles in the near

future.

End users are now getting


of the unmanned tools, even

though there are still some

robustness and deployment

issues to be tackled for ground

and marine vehicles. Finally, this

will help to speed up rescue

operations (and make them more

safe for the SAR workers) and

save lives.

Wide acceptance

among policy

makers on the

use of unmanned

ground and

marine vehicles

for crisis

management


demonstrations to convince policy

makers about the advantages of

unmanned SAR tools.

Policy makers seem now


of the unmanned tools, which

will drive to define the policies

and procedures for introducing

these tools into the crisis

management framework.

Improved

interoperability

and SAR

equipment

ICARUS has developed a set of

interoperable tools which seamlessly

work together with a unique

command and control system.

The use of these tools on the

field will help to speed up SAR

operations and save lives.

Improved

command &

control and

support system

for planning and

management

ICARUS developed a unique

command and control system which

is capable of controlling a range of

interoperable assets. Support tools

are provided enabling the mana-

gement of complex SAR operations.




Improved training

curricula for SAR

workers,

ICARUS has developed training tools

which allow end users to learn how

to operate the different ICARUS

Training is paramount, as SAR

workers will not use tools they

are not trained for. The impact of


including

unmanned tools.

tools in a safe virtual environment. the training tools is therefore the

wider use of the unmanned tools

Long-

term

Expected

Impacts

on the

field

(4 years

after the

project

end)

Collaborative and

cooperative

operation of

multiple

unmanned tools

in a crisis area

ICARUS has developed collaborative

robotic agents. However, we must

be realistic that the simultaneous

use of multiple collaborative

unmanned systems in the same crisis

theatre will still require that SAR

workers first get used to using

singular systems.




4.2. ECONOMIC IMPACT

Globally serious urban problems of mass destruction, whether caused by nature or by man, such as

earthquakes, floods, wars and even terrorist attacks occur. These events bring unfortunately high

number of victims, and under these grave circumstances the collaboration of SAR teams is required.

Members of these rescue agencies perform a difficult task, exposing their own lives at great risk to

rescue the victims.

On account of ICARUS technological advances in the area of robotics comes an application of great

global interest for its humanitarian nature. Robots can make disasters go away faster because they

can replace manual actions with automated and remotes tasks. This means that intervention speeds

up which increases the possibility of life-saving victims. Once tasks of search and rescue are

complete, then recovery groups can enter the affected area and restore utilities, repair roads, etc.

The reduction of the initial response phase by just 1 day reduces the overall time through the

reconstruction and recovery phase by up to 1,000 days. Note that there were economic losses of 110

billion dollars, thus using robotic machinery, given its high cost, the economy would recover more

quickly.


4.3. MAIN DISSEMINATION ACTIVITIES

Dissemination activities were divided in three parallel main phases, each of these phases being

dependent upon the maturity of the project at a specific point in time.

The objective of the first phase (RP1 to RP4) of communication activities was to convey the

message that a new Robotic SAR project, ICARUS, had started. This phase was carried out by

attending conferences and events as well as through direct contact with stakeholders. In

addition, this phase has been running alongside the other two phases in order to maximise

the engagement of potential targets that might have been be interested in the project. As an

example, the Project partners delivered 25 presentations and lectures during RP1 which is

the highest number as compared to RP2 (11), RP3 (11) and RP4 (16).

The second phase (RP2 to RP4) was carried out to present the results and achievements of

research and development activities undertaken in the frame of the ICARUS project,

therefore targeting technical experts in priority.

Finally, when ICARUS outcomes were mature enough (RP3 to RP4), communication efforts

focused on the end users, the decision-makers and budget holders to ensure the

sustainability of the project outputs in the long-term. During this last phase, the project put

the emphasis on the exploitation of the ICARUS results as well as on the interoperability and

the integration of the different ICARUS unmanned platforms. In addition, two large-scale

field demonstrations were organised by the Project partners during the last year of the

Project.

Dissemination activities performed during the Project timeline included:

Development of a project website. The website (http://www.fp7-icarus.eu/) was made

available in May 2012 and constantly updated so as to include new website sections and

information on the latest Project development (see the new project results section). It was

used as an important dissemination channel to inform the various communication audiences

about the Project status and the results of the research. The website will be maintained a

minimum of one year after the end of the Project.

From April 2012 to December 2015, the statistics are (source: Google Analytics):

o Sessions: 26.292 sessions (548 per month)

o Unique visitors: 17.557 (365 per month)

o New visitors: 67%

o Average number of new visitors per month: 365

o Average visit duration: 0:02:49


http://www.fp7-icarus.eu/project-results


Project documentation. Dissemination material produced and regularly updated by the

project include:

o 2 Project leaflets regularly updated (general public + technical level)

o presentation poster (updated during the Project)

o roll-up (updated during the Project)

o power point template (for partners only)

o pop-up stand (or exhibition booth) used for major events

o Newsletters

Facebook and twitter pages (RP3) were part of the ICARUS “push campaign” to improve

Project visibility during the last year. This page contained all the latest news published in the

project website plus “shared” news from relevant institutions. As an example, the

announcement of successful ICARUS participant to euRathlon reached 1.000 people on

Facebook.

Image Bank was an online repository where images related to the Project and to Search and

Rescue were stored for quick access by the partners and authorised stakeholders interested

in the Project. The Image Bank included 28 albums containing 1,276 pictures in total.

Dissemination database. A contact database (369 entries) of relevant communication

targets has been developed (M9) and regularly updated.

Participation in events. In total, the Project partners participated in 149 dissemination

events since the beginning of the project such as conferences, workshops, field

demonstrations, etc. This number includes 63 lectures and presentations.

These events were a useful platform to reach the audience and provided for networking

opportunities. During the last communication phase (RP3 to RP4), particular attention was

paid to the events allowing field-trials in order to demonstrate the added value of the

ICARUS toolbox. For instance, the Project partners were invited to multiple conferences,

workshops and field demonstrations including:

The European Civil Protection Forum 2015 (Brussels, May 2015) organized by the

EC’s DG ECHO;

The Eurathlon 2015 robotics challenge (Piombino, September 2015) where several

ICARUS partners won multiple prizes and awards;

The United Nation’s International Research and Rescue Advisory Group (INSARAG)

Global Meeting (Abu Dhabi, October 2015);

The RPAS workshop for civil protection experts (Brussels, January 2016).


Publications. The Project produced a total of 102 publications including 12 (12%)

publications during the first year, 19 (18%) publications during the second year, 17 (17%)

publications during the fourth year and 54 (53%) publications in the last year.

Publication of news release on the website. In total, the Project published 86 news items

since April 2012.

Production of videos. In total, the project produced 10 high-quality videos available in a

dedicated media section on the website. The updated version of the ICARUS project

presentation video (3:26) has reached 4.388 views on Youtube. In aggregate, the ICARUS

videos have reached 13.555 views. In addition, the EU Commission (ECHO) has posted the

video of on the official EU Civil Protection website.

ICARUS Maritime Demo. On the 9th and 10th of July 2015, ICARUS Project partners simulated

a maritime crisis management scenario at the Navy Base of Alfeite (Almada) in Portugal. A

video of the event is available here.

ICARUS Final Land Demo. On the 4th of September 2015, the final field trials of the Land

robotic and C2 tools developed within the FP7 ICARUS Project took place on the training

grounds of the Belgian First Aid and Support Team (ICARUS partner B-FAST) in Marche-en-

Famenne, Belgium. In total, the event attracted approximately 100 participants from 9

Countries (Europe + USA). A video of the event is available here.

First Legal Drone Flight performed in Brussels. On 7 May 2015, and within the framework of

the EU Civil Protection Forum, several ICARUS Partners performed a dedicated Remotely

Piloted Aircraft System (RPAS) outdoor demonstration which was the first-ever legal RPAS

flight in Brussels, raising particular interest from the Belgian media. A 2-pages article was

published in “Le Soir”, a leading national French speaking newspaper (see here).

Media coverage – blogs & newspapers. Communication activities undertaken during the

Project can be assessed on the basis of its impact in the media. In total, 32 articles about

ICARUS have been published in newspapers (online and offline) including major

(inter)national newspapers.

Media coverage – television. The ORF (Österreichischer Rundfunk) Austrian Television aired

a 45 minutes documentary on Robotic Search and Rescue that features many of the

developments of the ICARUS Project, including interviews of several ICARUS partners. ORF

came to Berchtesgaden, Germany to film the EURATHLON event in which several ICARUS

partners participated. In addition, the Radio Télévision Belge Francophone – RTBF - produced

a documentary on ICARUS that was aired multiple times in a scientific national television

show.

https://www.youtube.com/watch?v=1q3UbeAKpOQ

https://www.youtube.com/watch?v=1q3UbeAKpOQ

https://youtu.be/XDJ23PQedYQ

https://youtu.be/jetP2wWH8AM

http://www.lesoir.be/946029/article/actualite/belgique/2015-07-25/quand-drones-belges-se-muent-en-secouristes

http://www.atube.at/alpha/index.php?p=beitrag&i=164&format


4.4. EXPLOITATION OF RESULTS

The ICARUS project is the result of merging the capacities and abilities of entities at different stages

of the SAR value chain. The ambitious objective and scope of the project has made necessary the

participation of a large number of beneficiaries in order to reach the expected results in different

fields. Each beneficiary has already contributed to a wide range of developments based on their

extensive expertise as part of their core business activity.

On account of ICARUS, 50 results have been identified with the aim of improving existing SAR

technologies, in particular, features that enhance UAVs, UGVs, USVs and the provision of a novel

Command & Control platform.

Many of these results are in a pre-commercial stage and are expected to be launch onto the market

by 2016. Thus a collection of innovative tools have been conceived for Urban and Maritime SAR

operations such as:

Unmanned Maritime Capsule

Heavy Unmanned Ground Vehicle

Training and Support system

Rapid deployment unmanned aerial vehicle team for assisting international relief teams

Rapid mapping tools, combining data from aerial and ground based assets

X-LINK middleware software for tactical communications

GEOBEAM tool for tactical communications planning and management

Multi- Domain Robot Command and Control Station (MDRC2)

Lightweight and Integrated platform for Flight Technologies (LIFT)

Icarus GIS Virtual Machine

Integrated Module

Further information is available on the website at http://www.fp7-icarus.eu/exploitation

Visitors can also contact developers through the project website in order to facilitate their

commercialization.

http://www.fp7-icarus.eu/exploitation


5. WEBSITE

The ICARUS website can be found on the following address: http://www.fp7-icarus.eu/

The website (see screenshot below) consists of 8 main sections:

The homepage shows the latest news and a quick overview of the project objectives

The Project Overview gives a more detailed description of the project and its partners

The Search and Rescue section introduces the problems related to this subject to the readers

A list of publications can be found on the Publications section

A News section lists all ICARUS-related news

A Links section brings readers in contact with partner organisations, projects and institutes

The Contact section gives the contact details of the project responsibles for different

requests

A Project Results section showcases the exploitable products coming forth out of ICARUS


project public report - fp7-icarus · information networks has been recognised by the fp7-security...

Documents