risk management file€¦ · 2.1 overview of iso 14971 the iso 14971 defines a framework for the...

(Page intentionally blank)

FP7-611223

Project co-funded by the European Commission under the

Information and Communication Technologies (ICT) 7th

Framework Programme

Risk Management File

Application of ISO 14971 in WELCOME project software stack

Document Identifier: RM0.1 Due Date: N/A

Delivery Date: N/A Classification: Confidential

Editors: Pantelis Natsiavas, Document Version: 1.0

WELCOME Wearable Sensing and Smart Cloud Computing for Integrated Care to COPD Patients with

Comorbidities

Contract Start Date: 1st November 2013 Contract Duration: 48 months

Project Partners: EXODUS (GR), CSEM (CH), KINGSTON (UK), AUTH (GR), INVENTYA (UK), CAU (DE), ROYAL COLLEGE OF SUR (IR), SMARTEX (IT), CIRO+ B.V. (NL), Kristronics GmbH (DE), UNIVERSADE DE COIM (PT), Croydon Health Servi (UK)

RM0.1 – Risk Management of 38

(Page intentionally blank)


Contributors

Name Organization

Pantelis NATSIAVAS AUTH

Ioanna CHOUVARDA AUTH

Rui Pedro PAIVA COIMBRA

Nada PHILIP KINGSTON

Drishty SONBARTH KINGSTON

Dimitris KARAMITROS EXUS

Peer Reviewers

Name Organization

Andreas RAPTOPOULOS EXUS

Revision History

Version Date Modifications

0.1 24/09/2016 Initial draft

1.0 31/10/2016 Integrated updates from all partners


Executive Summary

This document contains the application of the WELCOME project’s risk management policy, following

the ISO 14971 standard.


Table of Contents

Executive Summary .................................................................................................................................. 4

1 Introduction ..................................................................................................................................... 6

2 Risk Management in WELCOME project .......................................................................................... 7

2.1 Overview of ISO 14971............................................................................................................ 7

2.2 Risk Management policy ......................................................................................................... 8

2.3 Risk Analysis, Evaluation and Control ..................................................................................... 9

3 Risk Table ....................................................................................................................................... 11

Bibliography ............................................................................................................................................ 37

Abbreviations ......................................................................................................................................... 38


1 Introduction

The WELCOME project’s design and implementation process has included constant risk management

procedures which are based on the guidelines of ISO 14971 [1], as it has been reported in D1.1, D1.3

and D3.3. The two ISO 14971 key requirements that have been adopted in the project’s design phase

are:

1. The classification of the respective medical device

The consortium investigated the classification of the WELCOME software stack and decided

that it is of Class IIa according to EU’s Directive 93/42/EEC.

2. The main requirement is the maintenance of one central document, the Risk Management

File, where the risks are documented, quantified and mitigation steps are recorded.

The project’s Risk Management File is kept as an internal (non-public) document and is

iteratively reviewed and updated.

It should be noted that the certification of the risk management approach followed is considered out

of the project’s DoW and is done as an extra task in order to increase the project’s exploitation

potential. This document is complementary to D3.3 due in M36.

The structure of this document is organized as follows: Section 2 describes the WELCOME project’s

consortium risk management policy (e.g. acceptance polity etc.) while section 3 contains the updated

risk management table.


2 Risk Management in WELCOME project

In this section, we briefly describe the ISO 14971. Furthermore, we provide details regarding the Risk

Management policy defined by the consortium.

2.1 Overview of ISO 14971

The ISO 14971 defines a framework for the systematic risk management on the development and use

of medical devices. The standard provides a definition of specific terminology used in such processes

and suggests methods to initially quantify and finally minimize the risks of the device usage. The risk

management process as defined in the ISO 14971 is shown in Figure 1.

Figure 1: ISO 14971 [1] schematic representation of the risk management process

Two key-points of ISO 14971’s rationale are the following:


1. The concept of risk is defined as follows: “It is accepted that the concept of risk has two

components: a) the probability of occurrence of harm; b) the consequences of that harm, that

is, how severe it might be.”

2. While the above definition of risk is clearly based on the quantification of the risk’s

propability of occurrence, it is clear that this should not prevent decision making: “The

decision to use a medical device in the context of a particular clinical procedure requires the

residual risks to be balanced against the anticipated benefits of the procedure”. Therefore,

even if the residual risk of a specific procedure cannot be credibly quantified, the procedure

should still be applied if we are certain that the benefit of the procedure is bigger than the

risk.

Some of the key concept definitions used in ISO 14971 standard are shown In Table 1:

Table 1: Main risk management concept definitions

Concept Definition according to ISO 14971

Risk combination of the probability of occurrence of harm and the severity of that harm

Risk Analysis systematic use of available information to identify hazards and to estimate the risk

Risk Assessment overall process comprising a risk analysis and a risk evaluation

Risk Control process in which decisions are made and measures implemented by which risks are reduced to, or maintained within, specified levels

Risk Estimation process used to assign values to the probability of occurrence of harm and the severity of that harm

Risk Evaluation process of comparing the estimated risk against given risk criteria to determine the acceptability of the risk

Risk Management systematic application of management policies, procedures and practices to the tasks of analysing, evaluating, controlling and monitoring risk

Severity measure of the possible consequences of a hazard

2.2 Risk Management policy

WELCOME project consortium has clearly committed on applying a coherent risk management

process to minimize safety risks and enhance future commercialization possibilities. Therefore, a risk

management policy has been defined on top of the ISO 14971 principles. The key-points of the

defined Risk Management policy are:

ISO 14971 is oriented towards physical medical devices and not medical software. For

example, the questions in Annex C and the examples of hazards identified of Annexes E are

typically used to identify medical device risks and they only partially address software issues.

Therefore, the consortium decided to adopt these as indicative guides but the WELCOME

Risk Management process should not restrict on them.

Adoption of the ISO 14971’s suggestion of adopting a unified Risk Management File

describing the project’s Risk Management process. Risk Management File is iteratively

reviewed and updated by the project’s technical team and the project’s coordinator.


Adoption of a specific risk analysis, evaluation and control methodology, defining specific

thresholds based on which risks are managed. The rationale of the ISO 14971 Annex D has

been adopted and explained in detail in the current section (definition of severity levels,

definition of probability levels etc.).

The most tangible outcome of the risk management process is the Risk Table in Section 3. The Risk

Table contains details regarding the quantification of the various risks, mitigation steps, residual risk

etc. Furthermore, it identifies the Work Package (WP) that the respective risk is tackled in, in order to

facilitate their management.

2.3 Risk Analysis, Evaluation and Control

Following the ISO 14971’s rationale, the risks are evaluated by estimating the severity of harm, the

probability of occurrence of the hazard and on the detectability of the hazardous situation. Typically,

the probabilities cannot be quantified credibly and therefore they are organized in levels of

estimation. These levels are adapted on the WELCOME Software Stack context and are an essential

part of the risk management process. They are presented in the following tables.

It should be noted that these risks when applying risk management typically refer to harm to patients,

health care professionals or environment. However, taking into account the research nature of the

WELCOME project which is conducted as a blind study, these risks are inherently negligible.

Therefore, our risk management also refers to technical risks.

Table 2: Severity of potential harm

Level score

Level Description

1 Negligible Benign, no potential harm

2 Minor Could potentially lead to system or patient difficulties, inconvenience or temporary discomfort

3 Moderate Compromise in the system function hindering the therapeutic or diagnostic procedure but results are still adequate

4 Major The system hinders or misguides the therapeutic or diagnostic procedure

5 Catastrophic Results in a major defect of the therapeutic or diagnostic procedure

Table 3: Probability of harm occurrence

Level score

Level Description Guiding probability range

1 Improbable No known occurrences on similar products or processes

Failure rate less than 0.01%

2 Remote Relatively few failures Failure rate less than 0.1%

3 Occasional Occasional failures Failure rate between 0,1 and 1%

4 Probable Repeated failures Failure rate between 1 and 10%

5 Frequent Failure is almost inevitable to occur Failure rate over 10%


The severity of the possible harm and the respective probability of harm occurrence are self-

explanatory. Regarding detectability, it is defined as the ability to detect a failure before it causes

harm.

Table 4: Detectability of hazardous situation

Level score

Level Description Guiding probability range

1 Almost certain

Control will almost certainly detect the existence of a defect products or processes

Detectability rate equal or larger than 99.9%

2 High Controls have a good chance of detecting the existence of a defect

Detectability rate between 60 and 99.9%

3 Moderate Controls may detect the existence of a defect

Detectability rate between 30 and 60%

4 Low Controls have a poor chance of detecting the existence of a defect

Detectability rate between 1 and 30%

5 Non

detectable Controls will not or cannot detect the existence of a defect

Detectability rate smaller than 1%

In order to evaluate a risk in a quantified manner, the Risk Priority Number (RPN) metric is used,

calculated by the various levels scores using the formula:

RPN = Severity x Probability x Detectability

Each risk is accepted or reduced according to the next table’s thresholds:

Table 5: Acceptability rules

RPN Level Description Guiding probability range

RPN>=45 Unacceptable risk Risk too severe to be accepted and must be reduced through

mitigation measures or risk control

45>RPN>=15

Significant risk The risks in this range are deemed acceptable, only if the reduction is not technically possible. If the risk may be reduced, then a mitigation measure or a risk control shall be implemented. Risk can be acceptable only with a favourable Risk/Benefit ratio.

15>RPN>=1 Acceptable risk Risk is acceptable. It represents the low risk area where a

mitigation measure would not result in risk reduction.

3 Risk Table

This section contains the risk table depicting the identified risks, the mitigation steps and the overall evaluation of the residual risk. Due to document layout issues and for

easier maintenance, this table is normally kept and updated in an independent excel file.

Table 6: WELCOME project risk table

ID System

component Risk

description Harm

Related Work

Package Seve

rity

Initial Risk Mitigation

Pro

bab

ility

De

tect

abili

ty

RP

N

Actions taken

Pro

bab

ility

De

tect

abili

ty

Re

sid

ual

RP

N

Residual risk


ID System

component Risk

description Harm

Related Work

Package Seve

rity


Pro

bab

ility

De

tect

abili

ty

RP

N

Actions taken

Pro

bab

ility

De

tect

abili

ty

Re

sid

ual

RP

N

Residual risk

1 Feature Extraction Server

Not enough or adequate data for training

Low accuracy of feature extraction algorithms

WP3, WP5

4 4 2 32

Data collected at various locations in Coimbra and Greece, with multimorbid patients, to generate adequate training data. A dataset collected from 30 volunteers was acquired. Based on those acquisitions a dataset with 20 volunteers (17 patients and 3 healthy volunteers) was used to test the crackles and wheeze detection. The respiratory sounds of six patients contain wheezes or wheezes and crackles. The healthy subjects exhibit normal respiratory sounds, while another set of three patients had only crackle manifestations. A dataset with at least 30 patients (with COPD and with manifestation of wheezes and/or crackles) and 15 healthy subjects should be acquired using the vest.

2 2 16

There is always space for addition of training data towards improvement of algorithms. However, the risk of inadequate data is considered minimized.


Bad data acquisition in training data

Low accuracy of feature

WP3, WP5

3 4 2 24

Careful data acquisition and annotation performed. All data were annotated by experts and double-checked.

1 2 6 Practically none.


ID System

component Risk

description Harm

Related Work

Package Seve

rity


Pro

bab

ility

De

tect

abili

ty

RP

N

Actions taken

Pro

bab

ility

De

tect

abili

ty

Re

sid

ual

RP

N

Residual risk

extraction algorithms


Cough data acquisition not representative


WP3, WP5

2 4 3 24 Data were acquired from COPD patients

2 2 8

Data were from controlled environment and maybe results differ from data acquired at home


Insufficient algorithm accuracy, e.g., confusion between different lung sounds (and background sounds)


WP3, WP5

3 4 3 36

Train the classification algorithm with a sufficiently broad range of sound classes. In the case of a multi-channel context, we should investigate exploiting source separation. For the crackles event detector we measure a detectability value and a positive predictive value equal to 84±22 % and 78±17%, For the wheezes event detector we measure

2 2 12 Results may differ from data acquired at home


ID System

component Risk

description Harm

Related Work

Package Seve

rity


Pro

bab

ility

De

tect

abili

ty

RP

N

Actions taken

Pro

bab

ility

De

tect

abili

ty

Re

sid

ual

RP

N

Residual risk

a detectability value and a positive predictive value equal to 79±28 % and 90±10%, respectively


Noisy data in testing

Low accuracy of feature extraction algorithms which could decrease DSS performance

WP3, WP5

4 3 3 36

Algorithms developed with noise robustness and tested with realistic data before use. Rules are based not only on signals but also on reported data through questionnaires, and aggregation of simple signs.

2 3 24 Results may differ from data acquired at home


ID System

component Risk

description Harm

Related Work

Package Seve

rity


Pro

bab

ility

De

tect

abili

ty

RP

N

Actions taken

Pro

bab

ility

De

tect

abili

ty

Re

sid

ual

RP

N

Residual risk


High computational cost

Unfeasible processing time

WP3, WP5

4 4 1 16

We evaluate the trade-off between algorithm complexity and accuracy. For example, we could eliminate unnecessary extracted audio features. We conducted performance testing to estimate the order of the execution time needed for each feature extraction processing algorithm and estimated the needed cloud resources. As a fallback mitigation strategy we could buy more cloud resources to decrease delay. In the plenary meeting in Coimbra, the FES development team presented a quantified performance analysis of each algorithm. It seems that the performance of every algorithm implemented or integrated until now is acceptable.

2 1 8

The remaining risk has to do with feature extraction algorithms that have been recently implemented/integrated as they are considered somewhat unstable regarding performance (EIT processing algorithms).


ID System

component Risk

description Harm

Related Work

Package Seve

rity


Pro

bab

ility

De

tect

abili

ty

RP

N

Actions taken

Pro

bab

ility

De

tect

abili

ty

Re

sid

ual

RP

N

Residual risk

7 Decision Support System

Rules inadequately expressed

Loss of alerting and difficulty in maintenance

WP3, WP5

3 2 2 12

We updated the DSS rationale with an aggregation strategy of simple rules which would depict the patient’s overall status. This would improve the patient status presented to the clinicians even in case when rules expressed in a complex manner would not fire. Furthermore, testing has been conducted with simulated data before actual use in order to test the ability of rules to fire.

1 2 6

DSS testing with real data during integration phase will practically show if the rules are adequately expressed.

8 Decision Support System

Frequent updating of rules

This might (under conditions) lead to some down time.

WP3, WP5

1 1 4 4 The DSS is designed to accept rules as part of the DSS ontology and it is external to the engine.

1 4 4 Practically none

9 Storage Engine

Requirements for entities that are in

A complex model which

WP3, WP5

2 1 1 2

We will try to take advantage of ontology engineering best practices to avoid pitfalls. E.g. new entities will be dropped and we will proceed with



ID System

component Risk

description Harm

Related Work

Package Seve

rity


Pro

bab

ility

De

tect

abili

ty

RP

N

Actions taken

Pro

bab

ility

De

tect

abili

ty

Re

sid

ual

RP

N

Residual risk

conflict or overlapping with existing ones.

could be very hard to maintain.

existing ones and local entities where possible. Until now, the feedback from the users of the model is very positive. It seems that the model covers most of their needs.

10 Storage Engine

Unstable complex model, needs for frequent updates

Inconsistency / System often down for upgrade

WP3, WP5

2 2 1 4

Exhaustive tests have been conducted. SE has been tested in local development server and in cloud environment too, including file storage capabilities.


11 Communication API

Frequent loss of communication

Reduced speed of response

WP3, WP5, WP6

4 3 2 24

The mitigation steps are based on data compression, repeating strategy, informing user etc. Risk has been considered during the overall data communication design and implementation. For example, data compression is an option for all clients and warning on-screen messages will be presented to the application user. Furthermore, exhaustive tests have been conducted using local development

2 2 16

We cannot really be sure until the system is tested under real conditions. The dependency on the real network capacity still exists. We cannot guarantee that there will be no loss of communication.


ID System

component Risk

description Harm

Related Work

Package Seve

rity


Pro

bab

ility

De

tect

abili

ty

RP

N

Actions taken

Pro

bab

ility

De

tect

abili

ty

Re

sid

ual

RP

N

Residual risk

servers and cloud infrastructure, with satisfactory results.

12 External Sources Connector

Highly complex API regarding environmental data retrieval

Delays in development or not able to retrieve environmental data

WP3, WP5

1 3 1 3

We have conducted a thorough search in order to find alternatives for environmental data providing web service, in order to reduce dependency on specific external data providers. We currently use the ones we estimate as more credible and easy to use and keep the others as a fallback solutions.


13 Orchestrator Agent

Low time performance

Delays in the execution of the main workflow of information of the OA would cause delays in

WP3, WP5

4 3 1 12

We use jBPM as a workflow engine which is widely used and supported by a major Java EE vendor (Red Hat). The lack of real multithreading has been overcome with the use of custom java code which allows the multiple run of jBPM workflows. We tested the multiple parallel run of jBPM workflows using unit tests



ID System

component Risk

description Harm

Related Work

Package Seve

rity


Pro

bab

ility

De

tect

abili

ty

RP

N

Actions taken

Pro

bab

ility

De

tect

abili

ty

Re

sid

ual

RP

N

Residual risk

the overall system.


EIT image reconstruction not feasible

Bad quality of produced EIT images which would consequently lead to bad feature extraction results based on EIT images.

WP3, WP5

3 4 2 24

We have already tested the most widely accepted EIT reconstruction algorithms against our selected electrode topology. We used simulation models in order to balance the lack of real data

4 2 24

Since we do not yet have real data from a real vest, there is a high uncertainty regarding our ability to reconstruct useful EIT images.


ID System

component Risk

description Harm

Related Work

Package Seve

rity


Pro

bab

ility

De

tect

abili

ty

RP

N

Actions taken

Pro

bab

ility

De

tect

abili

ty

Re

sid

ual

RP

N

Residual risk

15

Web applications (App for HCP, Clinical administration application)

Too complex information presentation

An example consequence could be the loss of prioritization for HCPs actions and potential delays

WP6 3 2 2 12

User acceptance tests will follow that will provide feedback on the user interface usability. Based on that feedback, the user interface may be redesigned to accommodate the user comments.

2 2 12

As far as user acceptance tests are not completed, the risk still remains.

16


A requested resource is not available on the WELCOME Cloud

An uncomprehensible error output shown to the user leading to frustration.

WP6 2 3 4 24

The application is configured to redirect to an error page, along with a message that provides information on the error that occurred



ID System

component Risk

description Harm

Related Work

Package Seve

rity


Pro

bab

ility

De

tect

abili

ty

RP

N

Actions taken

Pro

bab

ility

De

tect

abili

ty

Re

sid

ual

RP

N

Residual risk

17


Loss of connectivity to the Organisation Hub

The user cannot be authorized and therefore no operation could be executed

WP5, WP6

5 3 3 45 Warning on-screen messages will be presented to the application user

3 3 45

We cannot really be sure until the system is tested under real conditions. The dependency on the real network dependencies under real setup still exists. We cannot guarantee that there will be no loss of communication.

18


Network related communication delays

The user could get confused and frustrated

WP6 1 3 3 9

Progress dialogs or status messages will inform the user of the application if a network related process takes longer than usual to be completed

1 3 3



ID System

component Risk

description Harm

Related Work

Package Seve

rity


Pro

bab

ility

De

tect

abili

ty

RP

N

Actions taken

Pro

bab

ility

De

tect

abili

ty

Re

sid

ual

RP

N

Residual risk

19 App for HCP

Vitals data are too large to be handled all at once.

This could lead to delays on the user interface or frequent failures.

WP6 4 3 1 12

The vitals related data will be retrieved from the WELCOME Cloud in smaller parts and in an asynchronous mode, so that the application operation is not obstructed in any way. Additionally, progress bars are used to indicate the background loading of data. Finally, progressive loading of data while the HCP scrolls through the graphs could facilitate the smoother data handling process by the application.


20 App for patient and carer

Information and guidance misleading for patient/carer

Problems in patient treatment

WP3, WP6

4 3 3 36

The risk has been identified in D3.1 and analyzed in WP6. The mitigation includes user’s acceptance testing and proper design of the respective web forms.

3 3 36

As far as user acceptance tests are not completed, the risk still remains.


ID System

component Risk

description Harm

Related Work

Package Seve

rity


Pro

bab

ility

De

tect

abili

ty

RP

N

Actions taken

Pro

bab

ility

De

tect

abili

ty

Re

sid

ual

RP

N

Residual risk

21 Patient hub

Loss of communication between the patient hub and the vest

Data is lost

WP4, WP6

3 4 4 48

The vest transfers all the data when it is charging. It does not delete them until they are successfully transferred.

2 4 24

Until the full integration is conducted, the residual risk cannot be really estimated.

22 Patient hub

Loss of communication between the medical devices and the patient hub

Data is lost and inconvenience could be caused to the patient

WP6 3 4 3 36 Medical devices store the readings. Therefore, they can be recovered using a retry policy.

4 1 12


23 Patient hub

Loss of communication between the patient hub and the cloud

Data is lost and inconvenience could be caused to the HCP

WP6 3 4 3 36 The data are locally stored in the patient hub and will be resent following a retry policy.

4 3 36



ID System

component Risk

description Harm

Related Work

Package Seve

rity


Pro

bab

ility

De

tect

abili

ty

RP

N

Actions taken

Pro

bab

ility

De

tect

abili

ty

Re

sid

ual

RP

N

Residual risk

24 Patient hub

Due to programming error medical data is misprocessed or changed

DSS is affected and false alarms are propagated to HCPs

WP6 4 2 4 32

Exhaustive unit and integration testing is conducted. HCPs can recognize a result as incorrect for reasons such as: the result is physiologically impossible; the result is inconsistent with the patient’s overall clinical status.

2 2 16


25 All cloud services

Request overload

Reduced speed of data processing and response which could cause inconvenience to the end users or drop overall system's performa

WP3, WP5, WP6

4 3 2 24

Orchestrator agent has been designed as a multi-threading application. Furthermore, quantitative performance tests for FES have been conducted to define upper limits and SE performance has also been verified. As a fallback solution, we increase computing resources if necessary. System logging mechanisms have also been established to prevent such failures.

3 1 12



ID System

component Risk

description Harm

Related Work

Package Seve

rity


Pro

bab

ility

De

tect

abili

ty

RP

N

Actions taken

Pro

bab

ility

De

tect

abili

ty

Re

sid

ual

RP

N

Residual risk

nce

26 All cloud services

Unable to acquire suitable PaaS infrastructure that matches our needs.

We will set-up the components in using IaaS cloud resources. This means that we would have to manually maintain required components (application server, triple

WP5, WP6

2 2 1 4

We have confirmed that Java EE PaaS containers are available through the major commercial cloud service providers, including our selected provider (Amazon Web Services). We also confirmed that there is the capability of hosting an out of the box virtuoso (RDF triple store) on the cloud.

1 1 2

Until the integration test is conducted, there is still some risk as some of the provided PaaS might not be fully working as expected.


ID System

component Risk

description Harm

Related Work

Package Seve

rity


Pro

bab

ility

De

tect

abili

ty

RP

N

Actions taken

Pro

bab

ility

De

tect

abili

ty

Re

sid

ual

RP

N

Residual risk

store etc.)

27

Organization hub and Communications API

Poor communication performance in the Organisation hub and WELCOME Cloud authorisation integration

Execution of user authorisation validation requests for every incoming request to the Cloud from the User applications may cause delays in the overall communication

WP5, WP6

3 2 2 12

The use of a user token caching mechanism has been established at the WELCOME Cloud API side instead of per-request user authorization validation. In case of poor communication performance under real working conditions, a caching mechanism for user tokens (with appropriate expiration periods) will be available in the WELCOME Cloud API, so that user authorization validation requests are not executed in the case of cached user Oauth2 tokens.

1 2 6

The network capacity dependencies still exist. We cannot guarantee that even the adoption of the mitigation strategy in an environment with poor communication infrastructure may not cause problems.


ID System

component Risk

description Harm

Related Work

Package Seve

rity


Pro

bab

ility

De

tect

abili

ty

RP

N

Actions taken

Pro

bab

ility

De

tect

abili

ty

Re

sid

ual

RP

N

Residual risk

and responsiveness of the system

28 All system components

Lack of maintainability

The system would not facilitate error detection, changes, updates etc.

WP3, WP5, WP6

3 3 1 9

We use design patterns like Singleton, Repositories etc. which lead to less and easier to maintain code. We also make heavy use of logging in order to facilitate debugging under special circumstance. Furthermore, each team developing software uses a source versioning framework (SVN, Git etc.). Moreover, having informal reviews of the code by other team members is something that has proven very helpful in practice.

1 1 3

We are pretty confident that there is no residual risk of maintainability mostly because of our internal reviews of code by other team members.


ID System

component Risk

description Harm

Related Work

Package Seve

rity


Pro

bab

ility

De

tect

abili

ty

RP

N

Actions taken

Pro

bab

ility

De

tect

abili

ty

Re

sid

ual

RP

N

Residual risk


Overengineering

We could end up developing everything from scratch, including functionality that has been developed by others. This woule inevitably introduce new errors etc.

WP5, WP6

2 2 4 16

We decided to implement the system being as modular as possible, following well known software design patterns and using widely accepted frameworks. For instance, we use slf4j for logging, junit for testing, spring for dependency injection, jBPM for workflow management, Apache Jena for RDF handling etc.



ID System

component Risk

description Harm

Related Work

Package Seve

rity


Pro

bab

ility

De

tect

abili

ty

RP

N

Actions taken

Pro

bab

ility

De

tect

abili

ty

Re

sid

ual

RP

N

Residual risk


Scalability and performance

The system could end-up to be a monolith which is hard to scale. This could heavily affect performance and end-user experience.

WP3, WP5, WP6

3 1 3 9

We decided to use cloud infrastructure, properly modularize the design, identify possible performance bottlenecks as early as possible, run performance tests and modify the design accordingly. For example, we identified that FES could be the most significant performance bottleneck. Performance tests on the existing endpoints have quantified the needed cloud resources and the capability of requests’ service. Another bottleneck that we identified is the triple store used as backend in our Storage Engine. We performed benchmarking tests against it in order to confirm its capability of hosting the WELCOME project data.



ID System

component Risk

description Harm

Related Work

Package Seve

rity


Pro

bab

ility

De

tect

abili

ty

RP

N

Actions taken

Pro

bab

ility

De

tect

abili

ty

Re

sid

ual

RP

N

Residual risk

31 Cloud services

Proper selection of cloud provider

Making a wrong selection of cloud provider could cost money, time and have effects on our compliance with regulations

WP5, WP6

3 2 2 12 We have selected Amazon Web Services as one of the most widely used cloud services providers.

1 2 6

Cannot safely be evaluated before the completion of integration testing.


ID System

component Risk

description Harm

Related Work

Package Seve

rity


Pro

bab

ility

De

tect

abili

ty

RP

N

Actions taken

Pro

bab

ility

De

tect

abili

ty

Re

sid

ual

RP

N

Residual risk

32 Cloud services

Failure to complie with EU regulations on our cloud services

Failing to be compliant with EU regulations could hinder approval of clinical trials (bioethics committees etc.) and a possible future CE marking process

WP3, WP5, WP6

4 3 3 36

We have adopted the standards ISO 14971 and IEC 62304. We have also used all the respective EU directives for data privacy for guidance during the overall platform's design.

1 3 12

This risk is consider minimized given the project's context and scope.


ID System

component Risk

description Harm

Related Work

Package Seve

rity


Pro

bab

ility

De

tect

abili

ty

RP

N

Actions taken

Pro

bab

ility

De

tect

abili

ty

Re

sid

ual

RP

N

Residual risk

33 Cloud services

Cloud provider lock in

Being locked in one cloud provider’s services could be dangerous in case of service termination or cost increase

WP5, WP6

2 1 1 2

In our market search we keep track of alternatives and in some cases we investigate to develop services on IaaS level rather use them as ready PaaS. The use of widely accepted open source technologies practically minimizes this risk.


34 External Sources Connector

Dependency on external service to retrieve weather data

Not able to retrieve weather information

WP3, WP5

1 2 1 2

We have conducted a thorough search to find alternatives web services as data providers and these are kept as fallback solutions.

1 1 1

Until the ESC has been thoroughly tested under real working conditions, the risk remains


ID System

component Risk

description Harm

Related Work

Package Seve

rity


Pro

bab

ility

De

tect

abili

ty

RP

N

Actions taken

Pro

bab

ility

De

tect

abili

ty

Re

sid

ual

RP

N

Residual risk

35 Overall system

Due to unexpected delays and schedule changes the integration testing period is too tight.

Not able to conduct full integration test and therefore discovering bugs and misfunctions in the first clinical trial setting.

WP7, WP5, WP6

4 3 4 48

The design and development of the different components was well monitored and actions were taken in case of expected delay in components delivery. E.g. If there is a delay in delivering the vest. Integration of vest was taking into account during the design stage and hence type and structure of the possible generated data from the vest were identified and other subsystems were designed accordingly. The Vest API was designed and developed early and simulated vest data were used to integrate to the patient hub for performance testing. We are already under heavy integration test procedures and try to finalize a convenient schedule.

3 4 48

Despite the fact that resources have been committed by all partners this risk still remains.


ID System

component Risk

description Harm

Related Work

Package Seve

rity


Pro

bab

ility

De

tect

abili

ty

RP

N

Actions taken

Pro

bab

ility

De

tect

abili

ty

Re

sid

ual

RP

N

Residual risk


Too many events set up in a day for a patient or informal carer

This could lead to delays on the user interface while loading all events in diary module.

WP3, WP6

2 3 2 12

A loading icon/spinner will indicate the user that some requests have been made and all events will eventually be displayed on the calendar.



Unfilled questionnaires are submitted accidently by patients or informal carers

Patients or informal carer can get frustrated if they have accidently sent a questionnaire without filling it

WP3, WP6

4 3 1 12

A warning message is displayed to the patient or informal carer to ask them to fill up all questions in a questionnaire before submitting it. The questionnaire will not be posted to the cloud unless all questions have been answered by the user.



ID System

component Risk

description Harm

Related Work

Package Seve

rity


Pro

bab

ility

De

tect

abili

ty

RP

N

Actions taken

Pro

bab

ility

De

tect

abili

ty

Re

sid

ual

RP

N

Residual risk

up completely.


Loss of connectivity to the Organisational Hub

The patient or informal carer cannot be authorised and therefore they cannot access the user application

WP5 WP6

4 3 3 36

Warning on-screen messages will be presented to the application user and the system is brought in a stable state.

3 3 36



ID System

component Risk

description Harm

Related Work

Package Seve

rity


Pro

bab

ility

De

tect

abili

ty

RP

N

Actions taken

Pro

bab

ility

De

tect

abili

ty

Re

sid

ual

RP

N

Residual risk

39 All components

Mismatch between the developed product and the requirements is intended to meet

Sub-systems cannot be integrated together

WP7, WP6, WP5

5 2 5 75

Unit testing and validation for each sub-system were performed prior to integration and results are reported in D4.2, D5.1, D5.2 and D6.1.

1 5 25

Despite the fact that resources have been committed by all partners this risk still remains, until full integration test is conducted.

40 All components

Delay in communicating changes by developers to any components

Sub-system cannot be integrated together

WP7, WP6, WP5

5 2 5 75

Integration infrastructure and repository is in place to communicate changes and to control the components released version for integration.

1 5 25

Despite the fact that resources have been committed by all partners this risk still remains, until full integration test is conducted.

4 Conclusions

This document describes the identified risks at current stage of the project (M36) while integration is

still ongoing. The risk management table will keep being updated based on the results of the

integration process and the pilots’ outcomes.

Bibliography

[1] International Standards Organization, “ISO 14971 (Medical devices — Application of risk

management to medical devices”, http://www.iso.org/iso/catalogue_detail?csnumber=38193

http://www.iso.org/iso/catalogue_detail?csnumber=38193


Abbreviations

COPD: Chronic Obstructive Pulmonary Disease

DSS: Decision Support System

FES: Feature Extraction Server

ISO: International Standards Organization

OA: Orchestrator Agent

RPN: Risk Priority Number

WP: Work Package

risk management file€¦ · 2.1 overview of iso 14971 the iso 14971 defines a framework for the...

Documents