handheld combat support tools utilising iot technologies ... · operators. introducing...

6
978-1-5386-4980-0/19/$31.00 ©2019 European Union Handheld combat support tools utilising IoT technologies and data fusion algorithms as reconnaissance and surveillance platforms. Mariusz Chmielewski Cybernetics Faculty Military University of Technology Warsaw, Poland [email protected] Marcin Kukiełka Cybernetics Faculty Military University of Technology Warsaw, Poland [email protected] Paweł Pieczonka Cybernetics Faculty Military University of Technology Warsaw, Poland [email protected] Tomasz Gutowski Cybernetics Faculty Military University of Technology Warsaw, Poland [email protected] This paper documents the idea and construction details of mobile support tools used for research in constructing innovative tools supporting selected range of reconnaissance and surveillance activities performed using handhelds and integrated or built-in sensors. In this research a mobile application such as SAME or mCOP serves as a communication and decision support node, exchanging and fusing data gathered from sensors (other handhelds), perimeter sensors, UAVs. The data gathering process is supplemented by the operator-soldier with additional information increasing data reliability. The task of recon data recognition, aggregation and deconfliction, in presented research, is performed by developed algorithms and operators. Introducing human-in-the-loop serves as complex- case solution and mechanisms for extending reconnaissance recognition features in the system, where an operator annotates gathered data and is able to aggregate, supplement, describe battlespace elements such as equipment, personnel, activities (tasks) and in result engaged units. The research contains also a mobile system architecture description, demonstrating proof-of- concept and implementation assumptions which led to deployment of highly specialised, service-based tactical support mobile application mCOP and its upgraded version SAME. Such applications serve as operational picture managers fusing data from command centre and available scenario participants. Both prototypes serve also as technology demonstrators for testing GIS-based decision support and augmented reality features indirectly supporting research on the efficiency of mobile tools introduction to support soldiers and low level commanders. Keywords— military tools, mobile application, IoT, data fusion, sensors, mobile application, reconnaissance I. INTRODUCTION IoT technologies provide many opportunities for supporting military operations especially in terms of reconnaissance or surveillance nodes. In majority cases documented cases of IoT deployments are connected to development of specialised remote, mobile sensor networks supporting surveilling and investigating given battlespace regions providing object detection and recognition. Concealed, secure, mobile and handheld tools carried everywhere by the operating soldiers and civilians, can provide up-to-date important surveillance and recon data. Such features increase the speed and accuracy of battlespace perception thus improving command system decision superiority. Moreover application of tactical support tools integrated with battlespace sensors (possibly developed as IoT based systems) deliver new capabilities for C4ISR – military command and control systems. The statement is direct research thesis, which have been constructed and proven during the development and trial deployment phases of mCOP and SAME applications. Both smartphone applications have been tested during the demonstrations and live exercises exposing the solutions to scientific and practical characteristics of proposed algorithms and functionalities. The verification process aggregated several procedures for testing effectiveness of: recon sensors data fusion, tactical and topographical orientation, terrain calculations as well as data sharing effectiveness. One of crucial concepts implemented and validated during trials, has been connected to designing, an IoT technology scenario implementations aimed directly at supporting military operations (in this case reconnaissance and surveillance). These scenarios engaged sensor systems (UAVs, surveillance towers, perimeter sensors, etc.) with handheld technology applications. Presented work describes proof-of-concept application and a long-term study of applying mobile decision support tools to increase the efficiency of decision process using unreliable and uncertain recon data. Elements of prepared methods include integration with sensor data sources performing filtering and data fusion in order to automatically input data into system allowing users to alter and refine data manually. The application works as data hub, which collects data, fuses combat situation elements data and in case of conflicts or information gaps, it allows users to supplement information storing such actions as metadata distributed in command network. Sensor data fusion capabilities are mainly connected with object’s location and classification, which are main problems for recon and surveillance tasks. To demonstrate possible, alternative IoT applications, integrated with SAME software, a lower level, wearable sensor data processing tasks have been presented, involving health state monitoring. As an research outcome a set of experiences and usage concept of SAME and mCOP will be presented, testing innovative concepts of combining new technologies: handhelds, wearables, environment sensors, unmanned vehicles and other IoTs. 219

Upload: others

Post on 13-Aug-2020

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Handheld Combat Support Tools Utilising IoT Technologies ... · operators. Introducing human-in-the-loop serves as complex-case solution and mechanisms for extending reconnaissance

978-1-5386-4980-0/19/$31.00 ©2019 European Union

Handheld combat support tools utilising IoT technologies and data fusion algorithms as reconnaissance and surveillance platforms.

Mariusz Chmielewski Cybernetics Faculty

Military University of Technology Warsaw, Poland

[email protected]

Marcin Kukiełka Cybernetics Faculty

Military University of Technology Warsaw, Poland

[email protected]

Paweł Pieczonka Cybernetics Faculty

Military University of Technology Warsaw, Poland

[email protected]

Tomasz Gutowski Cybernetics Faculty

Military University of Technology Warsaw, Poland

[email protected]

This paper documents the idea and construction details of mobile support tools used for research in constructing innovative tools supporting selected range of reconnaissance and surveillance activities performed using handhelds and integrated or built-in sensors. In this research a mobile application such as SAME or mCOP serves as a communication and decision support node, exchanging and fusing data gathered from sensors (other handhelds), perimeter sensors, UAVs. The data gathering process is supplemented by the operator-soldier with additional information increasing data reliability. The task of recon data recognition, aggregation and deconfliction, in presented research, is performed by developed algorithms and operators. Introducing human-in-the-loop serves as complex-case solution and mechanisms for extending reconnaissance recognition features in the system, where an operator annotates gathered data and is able to aggregate, supplement, describe battlespace elements such as equipment, personnel, activities (tasks) and in result engaged units. The research contains also a mobile system architecture description, demonstrating proof-of-concept and implementation assumptions which led to deployment of highly specialised, service-based tactical support mobile application mCOP and its upgraded version SAME. Such applications serve as operational picture managers fusing data from command centre and available scenario participants. Both prototypes serve also as technology demonstrators for testing GIS-based decision support and augmented reality features indirectly supporting research on the efficiency of mobile tools introduction to support soldiers and low level commanders.

Keywords— military tools, mobile application, IoT, data fusion, sensors, mobile application, reconnaissance

I. INTRODUCTION

IoT technologies provide many opportunities for supporting military operations especially in terms of reconnaissance or surveillance nodes. In majority cases documented cases of IoT deployments are connected to development of specialised remote, mobile sensor networks supporting surveilling and investigating given battlespace regions providing object detection and recognition. Concealed, secure, mobile and handheld tools carried everywhere by the operating soldiers and civilians, can provide up-to-date important surveillance and recon data.

Such features increase the speed and accuracy of battlespace perception thus improving command system decision superiority. Moreover application of tactical support tools integrated with battlespace sensors (possibly developed as IoT based systems) deliver new capabilities for C4ISR – military command and control systems. The statement is direct research thesis, which have been constructed and proven during the development and trial deployment phases of mCOP and SAME applications. Both smartphone applications have been tested during the demonstrations and live exercises exposing the solutions to scientific and practical characteristics of proposed algorithms and functionalities. The verification process aggregated several procedures for testing effectiveness of: recon sensors data fusion, tactical and topographical orientation, terrain calculations as well as data sharing effectiveness. One of crucial concepts implemented and validated during trials, has been connected to designing, an IoT technology scenario implementations aimed directly at supporting military operations (in this case reconnaissance and surveillance). These scenarios engaged sensor systems (UAVs, surveillance towers, perimeter sensors, etc.) with handheld technology applications. Presented work describes proof-of-concept application and a long-term study of applying mobile decision support tools to increase the efficiency of decision process using unreliable and uncertain recon data. Elements of prepared methods include integration with sensor data sources performing filtering and data fusion in order to automatically input data into system allowing users to alter and refine data manually. The application works as data hub, which collects data, fuses combat situation elements data and in case of conflicts or information gaps, it allows users to supplement information storing such actions as metadata distributed in command network. Sensor data fusion capabilities are mainly connected with object’s location and classification, which are main problems for recon and surveillance tasks. To demonstrate possible, alternative IoT applications, integrated with SAME software, a lower level, wearable sensor data processing tasks have been presented, involving health state monitoring. As an research outcome a set of experiences and usage concept of SAME and mCOP will be presented, testing innovative concepts of combining new technologies: handhelds, wearables, environment sensors, unmanned vehicles and other IoTs.

219

Page 2: Handheld Combat Support Tools Utilising IoT Technologies ... · operators. Introducing human-in-the-loop serves as complex-case solution and mechanisms for extending reconnaissance

II. CONCEPT OF DEVELOPED IOMT PLATFORM

Gathered IoT sensor data can be processed by data fusion algorithms which make the produced common operating picture more genuine.The platform itself is a multi-agent environment, where some of the agents are sensors and some act as a recipients. Client applications scan available networks searching for compatible sensors and subscribes as event receiver. Each detected change is then broadcasted to subscribed clients. Platform enables also peer-to-peer communication between different nodes. This approach allowed to implement a number of features basing on significant node capabilities. When discovered node introduce itself as a client application it is able to conduct live-chatting with another client application, send a MEDEVAC request or e.g. stream a video. If found node describes itself as a sensor then the application can gather and process the given data. There are couple of types of sensors including video-streams (CCTV[12] cameras), air condition (humidity, temperate etc.), location sensors or soldier’s mental and physical health monitors.

Fig. 1. SAME map (main) view demonstrating Blue Force Tracking

module and current individual status with downloaded (altered) tactical scenario. Fused data from user’s input and command situation awarenes.

The main concept was to build a mobile tool that will improve the effectivity of single soldier and provide squad monitoring capabilities for commanders. The assessment of utilization of such tool has been conducted by implementation of significant operational scenario.

III. IMPLEMENTATION OF IOMT OPERATIONAL SCENARIO

When discussing the usage of combat supporting tools utilizing IoT in military reconnaissance and surveillance operations it is necessary to define how, when and under what circumstances the tools would be used. It is advised to create an operational scenario for each operation type and

functionality that IoT’s are able to provide. The scenario can, not only present how the tool can be used, but also define some restrictions, exceptions and help testing the tool in a combat environment. The structure of developed scenarios for regarding IoMT is not common in accessible sources and often is restricted to basic metadata regarding the author, producer, scenario system and an unstructured scenario description. It is crucial to view scenarios as opportunities to demonstrate how technology can satisfy required operational capabilities. To make the descriptions of scenarios more universal and complete it was decided to base the structure on the commonly used use case specification. Using the established structure for IoMT operational scenarios, building and developing scenarios can result in better understanding and testing of IoMT in operational usage. Table I presents an operational scenario for Blue Force Tracking service [19].

TABLE I. USE CASE SCENARIO OF TECHNOLOGY IMPLEMENTING BLUE FORCE TRACKING SERVICE

Attribute Value - description Scenario code S-RC-BFT-01 Scenario name

Monitoring the status and the location of battlefield objects – Blue Force Tracking for mobile systems

Operation type

Kinetic

Operation class

Reconnaissance, all classes indirectly

Used technology

Methods of localization of battlefield objects (GPS) and data (signal) distribution techniques under reduced communication bandwidth. Technologies are required to consider situations with full electromagnetic interference.

Technological and functional complexity of the scenario

Components of the technology exist, providing communication secrecy becomes problematic, especially object localization in closed areas, where GPS signal is restricted or not available. System should be capable of localizing and monitoring the objects and their status in a condition of signal interference. Considered technologies should include tactical mobile terminal systems that would aid data distribution in short distances.

Technological durability

High technological durability. No need for additional maintenance processes, applied technologies are renewable and do not need to be complemented.

Main flow of the scenario

Objects, based on data supplied by satellites and alternative sources of localization, determine their position in open area, buildings and in urban terrains. The transmission of data regarding localization and identification is carried out through available tactical communication channels and ad-hoc channels build based on mobile devices or mobile network access points (IoT).

Alternative flows of the scenario

Localization data can be complemented with data of battlefield objects on the communication path. Apart from that the BFT data of own forces elements should be supplemented with reconnaissance data regarding enemy forces and other units on the battlefield. The data are transferred to decision-making body can be complemented with soldier’s individual details (identification data, personal data, medical and physiological status evaluated based on the data from biomedical sensors). It is assumed that there will be a lot of data transfer of single. Data transferred to the BFT service will be complemented with serial data, configuration, armament parameters and the status of supplies.

Exceptional situations

The technology must provide communication channels that are immune to interference and BFT data propagation algorithms. It is assumed that functioning of the system can be suspended and hidden temporarily, to make sure that the EM signatures are not revealed.

Action: add units, recon data, waypoints,

actions-tasks, symbols

Menu: cmd module, tactical chat, GIS manager, augmented view, recon stream, silent mode, settings symbols

Situation: observer (pos.-orient.) tactical elements (symbology APP-6A) units, equipment, warfare, tasks

Status: location, messages pending, network status: -Wireless, -Tactical Radio, -No Comm, -Silence

220

Page 3: Handheld Combat Support Tools Utilising IoT Technologies ... · operators. Introducing human-in-the-loop serves as complex-case solution and mechanisms for extending reconnaissance

Preconditions for operational usage

The usage of individual tactical terminals, equipment of soldiers with reconnaissance data reporting tools, equipment with localization terminals, biomedical sensors. BFT data reporting can be done using the tactical network, multiprotocol mobile devices communication or networks provided by a group of drones.

Supported branches of service

Land Forces, Special Forces, Territorial Defence Forces

Cooperating systems

Battlefield Management Systems, C2, C3I, C4ISR, Dismounted Soldier Solution

IV. BATTLESPACE AND HEALTH STATE SENSING

Methods for sensing and diagnostics applied in reconnaissance and surveillance require the possibility to utilise them in many scenarios – for the purpose of presented research we will concentrate on force tracking systems BFT [3] and derivatives of such services e.g. soldier health state monitoring, as well as supporting combat triage [2],[1]. The latter approach is to apply these methods to probe and examine the battlespace environment of the area of operation and possibly enemy forces. In order to monitor forces and warfare, a state of an individual soldier can be inspected therefore, a set of dedicated sensors should be considered: inertial sensors, thermometer, photopletysmography, electromyography, electrocardiograph. Lower blood temperature would indicate and possible blood loss. Using PPG [7] might enhance blood loss detection, it allows to keep track of the changes in the blood oxygen saturation. Carrying out reconnaissance of the battlefield can be done using many tools proper for different situations. Field intrusion sensors disposed at the battlefield can report any kind of entry into the restricted zone. This makes it possible to track the movement of enemy units. Sensors can track motion or can report situations of break through the border of the selected area. It can be achieved by using infrared motion detectors, electromagnetic radars or perimeter fibre optic sensors [9]. Depending on the operational needs appropriate sensors, tailored to situation should be used for reconnaissance and surveillance. If the recon process is carried out in difficult conditions and can be planned, that take longer to implement and provide higher weather resistance are advised like fibre optic cable sensors or microwave sensors.

V. INTEGRATED SENSOR PLATFORMS

Presented sensing methods have found their implementations in many devices created both for civil and military usage. They can truly increase the reliability and the quality of reconnaissance allowing the commander to know the condition of troops and of the surrounding area. These methods are implemented in sensors that are a part of IoMT. Using the sensors makes it possible to constantly monitor the changes in the values of measured parameters that are indicators of the condition of a soldier. A mobile application receives the data from the sensors and using dedicated algorithms analyses and detects undesirable situations. Integrating signals received from dynamic and diverse set of sensors makes the reasoning results more reliable. This is the reason why it is necessary to use IoMT on the battlefield. Sensor platforms that allow diagnosing and monitoring of the soldiers are often presented as wearables (bands, chest or arm straps, caps, vests) that do not affect the motor functions of the owner. To demonstrate biomedical reasoning capabilities for soldier monitoring MYO armband, has been selected. It is

built up of 8 surface EMG sensors, capable of measuring muscle activity, it is also equipped with an inertial measurement units, enabling movement tracking.

Fig. 2. Gesture recognition multi-sensor – MYO - utilised to perform actigraphy and myography during combat – to identify seizures, assess

movement intensity and fatige.

Based on the biomedical data received from the sensors, a set of health state events can be identified such as chemical warfare poisoning, nervous system and muscle failures, fractures, seizures, internal bleeding. The recognition of undesirable states of soldiers should be reported directly to the commander through the tactical wireless network. The default SDK for the armband supports recognizing certain gestures but it raw EMG and IMU data can be also accessed, which can be later analysed to detect indicated states. Another utilised sensor - wearable band is Microsoft Band. It is equipped with PPG sensor, accelerometer, gyroscope, GPS, ambient light sensor, temperature sensor, UV sensor, capacitive sensor, galvanic skin response sensor and a barometer. Such a wide range of sensors allows to precisely monitor the state of soldiers and partially environment. The dedicated SDK provides access to raw sensor data which can be analysed and described conditions can be detected.

Fig. 3. Multi-sensor armband (wearable) Microsoft Band 2 – utilised in soldier health state monitoring integrated with SAME.

The band is environment resistant, and tuned to sport usage. Unfortunately it has a very short battery life due to the energy consumption caused by the sensor activity and screen lighting, fully charged battery lasts for 48 hours of normal usage [22]. Sensor platforms that allow users to detect intrusions into the protected territory are widely used in the security systems. For indoor usage PIR motion detectors are usually preferred due to low prices and sufficient detection range. They are provided by plenty of manufactures that specialize in security systems.

sEMG segments [x5] modules forming band (detachable)

MYO Sensor - processing segment with inertial and sEMG channel

sEMG segment with battery module

Barometer

UV sensor

Heart Rate Sensor

PPG sensor

GSR sensor

GPS - Glonass geopositioning

sensor Actigraphy sensor 9DoF inertial

221

Page 4: Handheld Combat Support Tools Utilising IoT Technologies ... · operators. Introducing human-in-the-loop serves as complex-case solution and mechanisms for extending reconnaissance

VI. SENSOR DATA FUSION METHODS

During the evaluation of correctness of application’s behaviour a few problems were encountered. One of them is tracking multiple targets by one sensor in cluttered environment (when targets are close to each other). Some of gathered information can be unavailable to read or just be noise, while other sensors can produce appropriate values. That was the main reason why data fusion algorithms need to be introduced. The data fusion can be conducted on three different levels: data association, state estimation and decision fusion [13]. The first one is most applicable for proposed platform because of the fact that the client application has access mostly to raw sensor data. One of the most commonly used algorithms in such situations is k-Nearest-Neighbour algorithm, which has been adopted in case of recon data fusion. The algorithm was tested using two types of metrics – Euclidean and Mahalanobis. In simulated cluttered environment the algorithm produced many pairs with the same probability therefore obtained results in several cases contained classification errors .

TABLE II. AGGREGATED AVERAGE PROBABILITIES OF ACQUIRING PROPER DATA USING SELECTED DATA FUSION ALGORITHM

Amount of targets to track

kNN (k = 3) Probabilistic data fusion

Euclidean Mahalanobis

5 82% 79% 76%

20 36% 39% 66%

50 13% 9% 49%

Another implemented approach was probabilistic data

association algorithm. Its aim is to attribute probability of association for each hypothesis from correct target measurement [16].

Fig. 4. Fused tactical situation – (left) raw sensor data identyfing unkown objects (units) – (right) aggregated and recognised hostile units infantry and

anti-armour unit

Correct measurement refers to observations that were gathered in certain moment of time in validity gate. This gate is considered as a centre of predicted measurements of target and is used to choose a set of basic parameters. That method gives good results when tracking targets which do not do sudden changes in patterns of movement. Table IV. demonstrates the results of experiment that was conducted in order to assess the effectiveness of usage of significant data fusion algorithm. The task was to determine proper location of detected objects. There were six sensors providing location for each of the targets. The objects have been given routes to follow. The simulation environment was sampling the real and detected positions every 30 seconds in ten minutes long period. For each sample every data fusion algorithm was applied and the correctness of position calculated. Correct position was considered when the distance from real location to calculated was less than 25 meters. All units were deployed and moving within 4 km2 area. The results contains the average probabilities of correct assessments of units position using chosen data fusion algorithm. Implementation of these methods were integrated within SAME mobile application, working as a (recon) client node for platform described in the paper.

VII. SAME FEATURES AND REQUIREMENTS

Military operations, which scenario is described above, can be supported by mobile handheld COP, reconnaissance and awareness tools.

Fig. 5. Application GIS layers, datasources and map entity management functionality – delivering configuration and tactical filtering capabilities

In case these systems are being powered by external data source, fusion of data can be used in order to provide to soldier detailed information about tactical situations. External data source can be system of IoT platforms, which aggregate data about environment via various sensors. IoT platforms can broadcast data by many network channels. IoT's information can be received, parsed, analysed and displayed by combat support tools, like mobile application. SAME - Situation

Data fusion – aggregated and grouped equipment forming two hostile units (from doctrinal templates)

Initial sensor data (seismic and video surveillance) – 3 unknown objects (equipment, soldiers preliminary grouped into units)

User-Observer Unit

Supporting friendly armored company unit

Map composition and management component

Tactical scenario elements – grouped depending on element’s

Area signs, activity and task symbols – implemented as vector images (editable)

Server based - tile layers and Web Map Service layers presenting GIS data and

Weather and video surveillance sensor data sources integrated for rendering

222

Page 5: Handheld Combat Support Tools Utilising IoT Technologies ... · operators. Introducing human-in-the-loop serves as complex-case solution and mechanisms for extending reconnaissance

Awareness Mobile Environment is a mobile application for handheld devices, based on Android operating system. The application is dedicated for tactical level commanders, and it integrates with C4ISR systems. It can be used as battle management system, which delivers solution for improved situation awareness. SAME allows users to work in reduced tactical network bandwidth. Applications first source of data are sensors of mobile device such as GPS, accelerometer, magnetometer, gyroscope and camera. Second source of data are external services which deliver data about operation environment and battlefield situation. The application is designed and implemented with regard to tactical usage of smartphones (gloves and ragged equipment), which delivers fast and ergonomic mechanisms for scenario modifications and server-based scenario download from standardised external sources and exchange protocols (JC3IEDM, NFFI, TSO). The most important set of requirements considered: individuals should operate on common operating picture, and use NATO data and symbols standards in application, which also provides tactical chat with various communication protocols. Application should integrate with C4ISR systems. One of important feature is radio silence support and support for IoT data acquisition and fusion.

Fig. 6. High-level component model presenting SAME platform

dependent model application constructs

Application should detect threat, analyse potential of units on the battlefield and synchronise data in real time. Mobile system should be able to receive and broadcast video and voice via encrypted protocols. Application uses WMS[11] standard for storage and displaying map data and overlays (both locally cached or networked accessed) Battlefield data, friendly and enemy units which are displayed on common operating picture are generated in NATO APP6-A standard. Applications uses NATO Vector Graphics 2.0 protocols. Tactical chat is based on two selectable communication protocols; MQTT[13] and XMPP[13]. User can create chat with one or more persons and see if they are connected in one network. XMPP protocol is also used to receive data from IoT

data providers. The application has been connected with weather condition data provider. Application supports urban operations, by using data received from CCTV or UAV images as well. In order to receive real time broadcast application uses RTSP protocol, which provides current information of battlefield situation. Besides receiving data from external video providers, application also uses RTSP protocol to video and voice chat in common network. SAME application allows users to connect and exchange data with other COP applications and systems, like ATAK via data exchange standards described above.

VIII. SAME ARCHITECTURE AND CONSTRUCTION ASSUMPTIONS

Implementation of application was conducted with established requirements, based on modern warfare operational scenarios. SAME as mobile application is composed of 3 main modules, which cooperates with each other. Communication module represents methods of text, voice and video exchange with other users. This module uses few protocols in order to provide connection to various systems and platforms.

Fig. 7. SAME logical perspective model demonstrating integration

responsibilities between tactical C4ISR system services (WMS, FFI, Chat) and selected sensor systems [IoMT] CCTV, UAV, etc.

223

Page 6: Handheld Combat Support Tools Utilising IoT Technologies ... · operators. Introducing human-in-the-loop serves as complex-case solution and mechanisms for extending reconnaissance

The module needs external services to provide communication in common network. Tactical situation visualisation module is responsible for displaying received data on map and HUD (available in augmented reality mode). In order to meet NATO’s requirements commercial software needs to meet also NATO - STANAG 4677 [23] requirements for mobile (DSS – Dismounted Soldier Solution requirements), which have also been implemented within presented software making the application tuned with combat capabilities. The module also allows the user to report units, accident or defined event and supports blue force tracking. Data fusion and analyser module is responsible for calculating threat and evaluating potential of forces on the battlefield. This module uses artificial intelligence such as deep learning, and neuron network to analyse received data. More data from various sources application receive, the more precise analysis result is.Detailed diagram placed below, contains described in previous chapter services and data sources which provide necessary data for mobile application.

IX. CONLUSIONS

Modern battlefield requires utilization of state-of-the-art techniques and technologies in order to obtain decision superiority through application of mobile tools and intelligent sensors. Features of IoT components provide new opportunities for stationary and mobile systems especially in the military domain. Large scale data availability comes with many problems including bandwidth requirements, data inconsistency and noise, which need to be solved through application of intelligent solutions deployed as elements of C4ISR systems or supplementary tools such as individual, specialised mobile applications. Such an approach has been demonstrated in the paper fusing experiences from development of mobile decision support applications and software components aimed at integration with reconnaissance sensor platforms implemented as IoMTs. The research also verified the most efficient set of data fusion methods showing their applicability and capabilities including kNN classifier and probabilistic data fusion. The proposed set of solutions presents novel approach, turning handhelds and mobile applications into functional recon nodes. The nodes are capable of processing sensor data and supplementing it with human-in-the-loop knowledge and interactions. The system proved to utilize various kinds of sensors starting from in-device components, through wearables, perimeter sensors and finishing with specialised sensor network systems. Serving as integration hub provides distributed reliable architecture for reconnaissance mobile tools deployed at the lowest levels of command. Constructed tactical software architecture enables the applications to integrate according to sensor’s capabilities published in form of metadata. Based on such configuration the system is capable to acquire and interpret sensor data streams or datasets and preliminary select adequate analytical algorithms for battlespace state evaluation (e.g. recon video footage from CCTV - perimetry monitoring or soldier’s actigraphy and HR data for health state or fatigue monitoring).

ACKNOWLEDGMENT

This work was developed under NATO Tide Hackathon 2018 and partially supported by a project EU NEC DEMO demonstrating NEC capabilities of Polish Armed Forces.

REFERENCES [1] M. Chmielewski, W. Kulas, M. Kukiełka, D. Frąszczak, D.

Bugajewski, J. Kędzior, D. Rainko, P. Stąpor, “Development of operational picture in DSS using distributed SOA based environment, tactical networks and handhelds”, Information Systems Architecture and Technology: Selected Aspects of Communication and Computational Systems, Wroclaw, 2014.

[2] M. Dyk, M. Chmielewski, A. Najgebauer, “Combat triage support using the Internet of Military Things”, In: Computer Science and Information Systems (FedCSIS), FedCSIS, Federated Conference on. IEEE, p. 835-842, 2017.

[3] M. Chmielewski, M. Kukiełka, "Applications of RFID technology in dismounted soldier solution systems–study of mCOP system capabilities", MATEC Web of Conferences, vol. 76, EDP Sciences, p. 04014, 2016.

[4] M. Chmielewski, M. Kukiełka, D. Frąszczak, D. Bugajewski, “Military and Crisis Management Decision Support Tools for Situation Awareness Development Using Sensor Data Fusion”, In International Conference on Information Systems Architecture and Technology, Springer, Cham, pp. 189-199, 2017.

[5] M. Héder, “From NASA to EU: the evolution of the TRL scale in Public Sector Innovation”, In The Innovation Journal: The Public Sector Innovation Journal, Volume 22(2), 2017.

[6] Fahreddin Sadikoglua, Cemal Kavalcioglub, Berk Dagmanc, "Electromyogram (EMG) signal detection, classification of EMG signals and diagnosis of neuropathy muscle disease”.

[7] Mohamed Elgendi, “On the Analysis of Fingertip Photoplethysmogram Signals” In Current Cardiology Reviews, 2012.

[8] Hugo F. Posada-Quintero, “Electrodermal Activity: What it can Contribute to the Assessment of the Autonomic Nervous System”, 2016.

[9] M. Szustakowski, W. Ciurapiński, N. Pałka, M. Życzkowski, “Recent Development of Fibre Optic Sensors for Perimeter Security”, 2002.

[10] G. Yatman, S. Üzumcü, A. Pahsa & A. A. Mert, “Intrusion detection sensors used by electronic security systems for critical facilities and infrastructures: a review”, 2015.

[11] Phil Yang , Ying Cao & John Evans, “Web Map Server Performance and Client Design Principles”

[12] C Norris, G Armstrong, “The maximum surveillance society: The rise of CCTV”

[13] Muneer Bani Yassein, Mohammed Q. Shatnawi, Dua' Al-zoubi, "Application layer protocols for the Internet of Things: A survey"

[14] H. Schulzrinne, A. Rao, R. Lanphier, Real Time Streaming Protocol (RTSP)

[15] D. L. Hall, J. Llinas, “An introduction to multisensor data fusion”, Proceedings of the IEEE, vol. 85, 1997

[16] Y. Bar-Shalom, E. Tse, “Tracking in a cluttered environment with probabilistic data fusion”, Automatica, vol. 11, 1975

[17] D. L. Hall, J. Llinas, “Handbook of Multisensor Data Fusion”, CRC Press, 2001

[18] H. F. Durrant-Whyte, “Sensor models and multisensor integration”, International Journal of Robotics Research, vol. 7, 1988

[19] Project for the Polish Ministry of Defense codenamed „MIoT” MON No. 522/2017/DA (02 170). Projekt realizowany na rzecz MON Nr 522/2017/DA (02 170) o kryptonimie „MIoT” .

[20] Webpage: https://www.pilz.com/en-GB/eshop/00106002197040/PSE Nopt-Light-curtains (Access: 2018-12-05).

[21] Webpage: https://fibersensys.com/15-products/114-microwave-sensor (Access: 2018-12-05).

[22] Webpage: https://support.microsoft.com/en-us/hub/4294906/microsof t-band-help (Access: 2018-12-05).

[23] NATO - STANAG 4677 - Dismounted Soldier Systems Standards and Protocols for Command, Control, Communications and Computers (C4) Interoperability (DSS C4 Interoperability), 2014

224