rfcs project - telerescuer
TRANSCRIPT
TeleRescuer Project RFC-CT-2014-00002
TELERESCUER SYSTEM FOR VIRTUAL TELEPORTATION OF RESCUER FOR INSPECTING COAL MINE
AREAS AFFECTED BY CATASTROPHIC EVENTS
T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2
2
SCHEDULE
1. Introduction
2. Work Packages and Bar Chart
3. Budget information
4. Deliverables
5. Detailed identification of needs, formulating requirements (WP1)
6. Research into the UV (WP 2)
7. Research into virtual teleportation technology… (WP3)
8. Dissemination of results (WP5)
9. Management and Coordination (WP6)
10. Conclusions
T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2
3
INTRODUCTION GENERAL IDEA
TeleRescuer is an innovative system for inspecting coal mine roadways, especially those affected by catastrophes such as fire, explosion of methane or coal dust, and the others.
The system allows virtual teleportation of a mining rescuer to those areas of a coal mine, in which he could not remain due to hazards for life or health
T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2
4
INTRODUCTION MAIN PARTS
T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2
5
WORK PACKAGES AND BAR CHART
T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2
6
WORK PACKAGES AND BAR CHART WP1-WP3
Added to initial bar chart in official ammendment no 1 (accepted by EC)
T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2
7
WORK PACKAGES AND BAR CHART WP4-WP6
T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2
8
DELIVERABLES ACHIEVED DELIVERABLES Deliverable number
Deliverable name Foreseen finalisation date
Real finalisation date
Form Location
D1.1. Formal specification of requirements
M6 31.12.2014 Written document
Appendix 2 to the first annual report
S1.1. A decision as to whether the system has to be ATEX-ready/ATEX-certified
M6 31.12.2014 Written document (minutes from the meeting)
Appendix 3 to the first annual report
D3.1 A report on rescuers’ knowledge acquisition and representation
M6 31.12.2014 Written document
Appendix 4 to the first annual report
D3.3 A report on the simulations of the operations of rescuers’ in a hazardous area of the coal mine with augmented reality elements
M18 Written document
Appendix 2 to the mid-term report
D5.1 A TeleRescuer project official webpage
M3 30.09.2014 Website Appendix 5 to the first annual report and internet (www.telerescuer. Polsl.pl)
D5.3 Articles about the obtained results in scientific and industry journals
M12-M36 - Published papers
List of papers presented in section 2 (Project overview table)
D5.4 A scientific seminar presenting the theoretical results achieved to-date
M18 23.09.2015 Meeting -
D6.1 A Consortium Agreement M3 07.07.2014 Written document
Appendix 6 to the first annual report
D6.2 A first annual report M9 (initial deadline: M15)
31.03.2015 Written document
CIRCABC
D6.3 Mid-term financial and technical reports
M21 Written document
CIRCABC
T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2
9
DELIVERABLES FUTURE DELIVERABLES
Deliverable number
Deliverable name Foreseen finalisation date
D2.1. A prototype of a mechatronic platform of the UV M24
D2.2. A prototype of a communication system and a sensory system M27
D2.3. A prototype of a control system M27
D2.4. A prototype system for building maps M27
D2.5 A method and a prototype system for the autonomous operation of the UV in a known environment
M27
D3.2 A prototype of an effective human-machine interface for virtual teleportation M19
D3.4 A prototype of the training simulator M30
D4.1 A report on tests of the system and its components M36
D5.2 A brochure about the TeleRescuer project (as an electronic pdf file and in print) M33
D5.5 A promotional seminar intended for the potential recipients of the project’s results M35
D6.4 A second annual report M27
D6.5 Final technical and financial reports M36+9
T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2
10
DETAILED IDENTIFICATION OF NEEDS, FORMULATING REQUIREMENTS (WP1)
The identification of the needs has been carried out in close collaboration with the Central Mining Rescue Station (CMRS). Results of those activities have been summarized, reported and converted into formal requirements.
Requirements have been specified for each subsystem of the UV, including:
• Robot platform: Platform’s mobility, dimensions, weight, protection level, operating time and range; Required external mechanical equipment and it’s parameters;
• Communication system: Optical fiber communication; Wireless communication with motes;
• Sensory system: Internal sensors: orientation sensors, robot state sensors, protection sensors; External sensors: gas sensors (CH4, CO, CO2, O2), air velocity senor and also
requirements on their placement; Cameras: quantity , type and optimal placement;
• Control system with autonomous operation: Autonomous operation modes and conditions for autonomous operation;
T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2
11
RESEARCH INTO THE UV (WP2) MECHATRONIC SUBSYSTEM OF THE UV (T2.1)
Mechatronic subsystem is composed of: Base robotic platform:
• High mobility tracked platform with independent tracks’ flippers
• Designed to meet ATEX standards requirements
• Capable of being used in various missions due to different configurations of external equipment
External equipment:
• Cylinder arm with sensors, cameras and lights
• Laser scanner • Mote dispenser • Fiber unwinder
T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2
12
Managed 2 microcontrollers:
I. 1st C: Arduino Uno + Arduino Ethernet Shield Connections:
• SPI: Voltage sensor (fuel gauge) to control batteries;
• SPI: Ethernet shield; • I2C: Inertial Measurement Unit (IMU) - MPU-
6050; • UART RS-485:
II. 2nd C: PIC Microcontroller series 30GP or 33MC from Microchip Sensor control module (outside the cylinder) managing gas sensors: • Methane (min. 0 ÷ 20% V/V) • Mass flow (0 ÷ 20 m/s) • O2 (0 ÷ 25%) • CO (0 ÷ 10000 ppm) • CO2 (0 ÷ 5% vol) • Humidity/Temperature (0÷100% /-20÷+60ºC)
• Free pins: 4 PWM pins (illumination), 4 I/O digital pins (relay, …)
Arduino Uno + Eth shield
MPU-6050
SENSORS
SENSORY SYSTEM (T2.2)
RESEARCH INTO THE UV (WP2)
T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2
13
Constraints: • Reduced space for all the camera electronics (130 x 110 x 160 mm box) • Need to work under fast Ethernet protocol (Not Giga because of the possible danger caused by a laser: IEC-
68070-28) • Ability to change online the video bandwidth output of each camera • Real time low latency video compression • On-off capabilities for each camera • PoE with the lower power consumption as possible • At least one thermal sensor for smoke • Illumination will be outside the safety box so it must be ATEX ready
Visible light cameras: NC353L Elphel model
2 fish-eye (180º): at front and at rear • Lens • 5Mpix color sensor board model 0353-00-17 • IP fast Ethernet camera main board 10353
1 stereoscopic camera (2 lenses) • Additional Synchronization board for the stereo rig model 10359
Thermal Camera: FLIR AX8 With converter to Ethernet cable
Illumination: Adaro Tecnologia M1 enclosure Own electronics (PWM, omnidirectional LEDs)
VISION SYSTEM (T2.2)
RESEARCH INTO THE UV (WP2)
T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2
14
Spool of Sedi-Ati
Possible Secondary Switches
WIRED COMMUNICATION SYSTEM (T2.2)
RESEARCH INTO THE UV (WP2)
Initial Possibility: • Own solution; • A simple unwinder (with an Optical Fiber Rotatory Joint – OFRJ of 2 channels) + secondary system to
wind the optical fiber once the robot is outside;
Problems: • OFRJ: Much expensive; • Dimensions of a reel with strong optical cable;
Solution: Spool for unmanned ground vehicles with not much strong cable;
100Base-FX or 1000Base-FX? • 1000Base-FX is not easy to atexize; • 1000Base-FX normally uses lasers: difficult to satisfy the regulation IEC-68070-28; • 100Mbps is enough for cameras with an acceptable resolution;
3 switches needed: 1) Primary station (in the safe place):
• 1 Optical Fiber port (to the robot); • 1 Ethernet port: ATEX Ethernet (80 m máx.) for the first mote; • 1 port for the operator;
2) Main switch (in the chassis): 4 ports (PC board, 1st mote, secondary switch, optical (for the O.F.)); 3) Secondary Switch (in the cylinder):
• Small (112 mm x 155 mm x 134 mm OR 106 mm x 160 mm x 134 mm); • Managed; • Able to bear high temperatures; • PoE ports (4 at least for cameras); • 8 ports (4 cameras, 1 C, 1 primary switch, 1 relay, 1 free);
T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2
15
Requirements: • Able to work in hazardous
environment; • Size constraints; • Range distance; • Robust;
Technical requirements: • Ad-hoc network: just forwarding
elements, no routers, no Aps; • Self-organizing network: Same LAN,
pre-configured IP addresses;
Technologies: 1) UWB-Decawave: Data rate (6.8 Mb/s
máx. Non effective!!! ; Max 10m ). Not optimized for communications;
2) IEEE 802.11 (Atexized Raspberry Pi);
ATEX-ready Batteries: • Lithium iron phosphate batteries
(LiFePO4)
WIRELESS COMMUNICATION SYSTEM (T2.2)
RESEARCH INTO THE UV (WP2)
First prototype
T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2
16
RESEARCH INTO THE UV (WP2) CONTROL SYSTEM (T2.3)
Mote #n
Chassis – inner space
4x RoboteQ FBL 2360- Track + Track Arm motors- Absolute sensor tilt of arms (connected to RoboteQ AI)- 2 x IRC + Hall (connected to RoboteQ)
PC Board
Vision (2D/3D)- 1 IR, 2x for stereo (common Eth.channel), 2 x fish = 5 cameras with 4 IP interface
UC3M
Cylinder Control System - Communication with external Sensor Control System (RS485) and MCS (Ethernet)- Cameras power ON/OFF- Lighting – front and rear- Voltage of power supply measuring, - IMU – Tilting x,y,z,..- for info about head tilt – artificial horizon for operator
UC3M
Ethernet Switch3D Laser scanner
12V, 20W
end
cap
26+2 TemperatureSensors (DALLAS)
Mote #02
Mote #01
(CH4, CO, CO2, O2, temperature outer, humidity,.....
Sensor Control System
CPU
CH4, CO, CO2, O2, temperature outer, humidity,.....
Metalic Ethernet (4 wire)
Optical serial communication (2 fibres)
Metalic CAN bus (2 wire)
Metalic serial communication + power(2 + 2 wires)
Metalic 1wire bus (3 wire)
Optical Fibre Interface
USB/CAN interface
USB/1Wire Interface
Main IMU
Autonomy Lock
Button
Central Stop
Button
Metalic USB (4 wire)
end
cap
end
cap
end
cap IMU Configuration
USB Reserve
RS485 Reserve
BatteryManagementSystem
CAN
CAN
DI ENABLE
USB (2xRS232)
USB
USB
USB
RS485
USB
ETH
ETH ETH ETH
1WI
Optical Interface
Optical Ethernet (1 - 2 fibres)
ETH
ETHETHETH
Optic Fibre ReelUC3M
ETH
ETH
RS48
5
RS23
2/RS
485
DC/DC converter
Ethernet switch
Main Switch
Flameproof enclosure bushing
ETHETH
UC3M
UC3M
SUT
SUT
Optical Fibre Interface +
ENABLE control (VŠB)
UC3M
SUT
SUT
SUT
SUT
VSB
VSB
SUT
SUT SUT
UC3MFinal Assebling + cabling: SUT, VSB
UC3M
SUT, UC3M
MOTE Relaser
SUT
Wireless comunication
module (UC3M)
RoboteQSensor arm Tilting, – Sens.armCylinder, Methane arm 3 axes= 1 drive, 1 motor,2x solenoid (1 clutch, 1 brake)2 x Absolute sensors – yes SUT
VPwrCtrl
(ENABLE)
RS232
Methane sensor on methane arm
Main Control System and its Connectivity with All Subsystems
T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2
17
RESEARCH INTO THE UV (WP2) METHODOLOGIES FOR BUILDING MAPS (T2.4)
Laser Range Finder (Sick LMS111) + Positioning unit; Methane sensor SC-CH4 with ATEX switches off 3D LRF,
when methane concentration exceed the limit; A few methods were programmed for improvement of
visualization and coloring output with additional information;
Prototype tested in Królowa Luiza Coal Mine (October 2015): no failures were encountered;
T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2
18
RESEARCH INTO THE UV (WP2) AUTONOMOUS OPERATION OF THE UV (T2.5)
• Localization in a (not) known environment, • Path planning in a known environment, • Movement realization according to a
planned path in a constantly changing environment
Main challenges
Growing uncertainty over time
Reduced uncertainty thanks to map matching
Based on R. Siegwart (ETH Zurich)
When autonomy will be used?
T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2
19
RESEARCH INTO THE UV (WP2) AUTONOMOUS OPERATION OF THE UV (T2.5)
Tests in simulated environments Tests of inertial navigation
Robot localization based on 3D scans
T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2
20
RESEARCH INTO VIRTUAL TELEPORTATION TECHNOLOGY… (WP3) OPERATOR STATION
In the operator station the following technolgies are used:
3 monitors (144 Hz)
3D technology
Nvidia 3D vision
T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2
21
RESEARCH INTO VIRTUAL TELEPORTATION TECHNOLOGY… (WP3) OPERATOR STATION - SOFTWARE STRUCTURE
T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2
22
DISSEMINATION OF RESULTS (WP5) RESULTS ACHIEVED (1/2)
The official TeleRescuer logo;
TeleRescuer website: www.telerescuer.eu (D5.1);
Template for multimedia presentations and official documents;
T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2
23
DISSEMINATION OF RESULTS (WP5) RESULTS ACHIEVED (2/2)
A brochure on the main project’s findings - with the use of augmented reality techniques (D5.2);
A scientific seminar presenting the theoretical results (D5.4): Seminar on 23th of September, 2015 (Gliwice,
Poland); Seminar on 9th March, 2016 (Gliwice, Poland); Webinars/teleconferences (in average one per
month);
Scientific papers: 10 publications) (D5.3);
T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2
24
DISSEMINATION OF RESULTS (WP5) PUBLICATIONS IN 2014-2015 (D5.3)
1. W. Moczulski, K. Cyran, P. Novak, A. Rodriguez, M. Januszka, TeleRescuer - a concept of a system for teleimmersion of a rescuer to areas of coal mines affected by catastrophes. Abstract, 28 November 2014 (full paper will be published in: Proc. of the Institute of Vehicles, Warsaw University of Technology in 2015).
2. Smart Autonomous Mobile Systems for Exploring the Unknown. Poster presented at the 1st PERASPERA Workshop, Noordwijkerhout, The Netherlands, 11-12 February 2015.
3. P. Novák, J. Babjak, T. Kot, W. Moczulski, Control System of the Mobile Robot TELERESCUER. Proc. of the Optirob Conference, Bucharest, Romania, 27-30 June, 2015.
4. D. Myszor, W. Moczulski, K. Cyran: Innowacyjny interfejs ratownika umożliwiający wirtualną teleportację w celu inspekcji wyrobiska kopalni dotkniętego katastrofą (in Polish). Proc. of the 42nd Symposium on Technical Diagnostics (Abstracts), Silesian University of Technology, Faculty of Transport, Wisła, Poland, 02-06 March, 2015.
5. Novák P., Babjak J., Moczulski W. Control System of the Mobile Robot TELERESCUER. Applied Mechanics and Materials. 2015, vol. 772, pp. 466-470, doi : 10. 4028, ISSN : 1662-7482.
6. Novák P., Babjak J., Kot T., Olivka P. Exploration Mobile Robot for Coal Mines. In Modelling and Simulation for Autonomous Systems. International Workshop, MESAS 2015, Prague, Czech Republic, April 29-30, 2015, 209-215, ISBN 978-3-319-22383-4.
7. Kot T., Novák P., Babjak J. Virtual Operator Station for Teleoperated Mobile Robots. In Modelling and Simulation for Autonomous Systems. International Workshop, MESAS 2015, Prague, Czech Republic, April 29-30, 2015, 144-153, ISBN 978-3-319-22383-4.
8. Timofiejczuk A., Adamczyk M., Bagiński M., Golicz P., Wymagania dla robotów uczestniczących w akcjach ratowniczych w podziemnych kopalniach węgla kamiennego. Mechanizacja, automatyzacja i robotyzacja w górnictwie. Monografia. Krzysztof Krauze (Red.). Centrum Badań i Dozoru Górnictwa Podziemnego w Lędzinach, Katedra Maszyn Górniczych, Przeróbczych i Transportowych AGH w Krakowie. Lędziny: Centrum Badań i Dozoru Górnictwa Podziemnego, 2015, s. 59-65
9. Timofiejczuk A., Adamczyk M., Mura G., Nocoń M., Moczulski W., Układ mobilny specjalistycznego robota do inspekcji wyrobisk kopalnianych dotkniętych katastrofą. Mechanizacja, automatyzacja i robotyzacja w górnictwie. Monografia. Krzysztof Krauze (Red.). Centrum Badań i Dozoru Górnictwa Podziemnego w Lędzinach, Katedra Maszyn Górniczych, Przeróbczych i Transportowych AGH w Krakowie. Lędziny: Centrum Badań i Dozoru Górnictwa Podziemnego, 2015, s. 66-74
10. Mura G., Adamczyk M., Nocoń M.. Numerical simulation of mobility of miners rescue robot. 13th Conference on Dynamical Systems Theory and Applications. DSTA 2015, Łódź, December 7-10, 2015, Poland. Eds. J. Awrejcewicz, M. Kaźmierczak, P. Olejnik, J. Mrozowski. Łódź: Wydaw. Politechniki Łódzkiej, 2015, s. 222
T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2
25
MANAGEMENT AND COORDINATION (WP6)
Collaboration with the use of knowledge management system: MOBIROB platform on the base of OpenKM;
At least 20 technical meetings since the start of the project: including videoconferences and live meetings;
4 official meetings: kick-off meeting (Gliwice, July 2014), first annual meeting (Madrid, March 2015), half-year meeting in 2015 (Ostrava, July 2015), mid-term meeting (Gliwice, January 2016);
All actions in TeleRescuer project related to WP6 have been strongly supported by the Project Management Centre at the Silesian University of Technology
T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2
26
CONCLUSIONS PROBLEMS ENCOUNTERED AND CORRECTIVE ACTIONS
Due to financial issues at AITEMIN, this partner had to leave the consortium, with effect on the 31st of March 2015.
The withdrawal of AITEMIN was organized in an orderly manner.
A suitable replacement partner, with capacity to carry out the work initially allocated to AITEMIN was proposed, and accepted by the consortium.
AITEMIN will cooperate with the new partner in order to achieve a seamless transition.
Instead of AITEMIN the consortium included two new beneficiaries: UC3M and KOPEX.
KOPEX represents mining experience and knowledge necessary for completing the project.
T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2
27
CONCLUSIONS
All tasks in the framework of the TeleRescuer project are carried out in accordance with the schedule and budget: All tasks from WP1 have been
finished;
Tasks from WP2, WP5 and WP6 have been started and are still in progress.;
Tasks from WP4 are planned in the future;
All planned objectives, deliverables and milestones have been achieved.
Since the aproval of the amendment has taken quite a long time, the Consortium will apply for the extension of the project.
T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2
28
CONCLUSIONS FUTURE WORK
Regarding to WP2: Mechatronic subsystem of the UV; Communication system and sensory system; Control system; System for map building; System responsible for autonomous operation of UV in a known
environment; Regarding to WP3:
Software and hardware of the human-machine interface (HMI) for virtual teleportation technology;
Software and hardware for training simulator;
Regarding to WP4: Plan of validation tests.
In the next one year period the following results will be achieved:
TeleRescuer Project RFC-CT-2014-00002
Silesian University of Technology Coordinator
TeleRescuer Project Office
D.Sc. (habil) Anna TIMOFIEJCZUK Silesian University of Technology
Faculty of Mechanical Engineering
Vice Dean for Organisation and Development
Project Coordinator
Akademicka Street 2A 44-100 Gliwice
Konarskiego Street 18a
44-100 Gliwice Phone: +48 32 237 24 26
Fax: +48 32 237 13 60 e-mail: [email protected]
www.telerescuer.eu
THANK YOU FOR ATTENTION