d7.1.2-test and evaluation report for salus …...fp7-287800 salus salus-fp7-287800• d7.1.2 •...

FP7-287800 SALUS

SALUS-FP7-287800• D7.1.2 • Version 1.0, dated October 01, 2013 1

SALUS “Scalable, Standard based Interoperability Framework for

Sustainable Proactive Post Market Safety Studies”

SPECIFIC TARGETED RESEARCH PROJECT PRIORITY Objective ICT-2011.5.3b) Tools and environments enabling the re-use of electronic health records

SALUS D7.1.2 Test and Evaluation Report for SALUS Components –R1

Due Date: September 30, 2013 Actual Submission Date: October 08, 2013 Project Dates: Project Start Date : February 01, 2012

Project End Date : January 31, 2015 Project Duration : 36 months

Deliverable Leader: SRDC

Project co-funded by the European Commission within the Seventh Framework Programme (2007-2013)

Dissemination Level

PU Public X PP Restricted to other programme participants (including the Commission Services) RE Restricted to a group specified by the consortium (including the Commission Services) CO Confidential, only for members of the consortium (including the Commission Services)

FP7-287800 SALUS


Document History: Version Date Changes From Review

V0.1 May 03, 2013 Initial Document SRDC All Partners

V0.2 Sep 10, 2013 Updated requirement traceability matrix INSERM SRDC

V0.3 Sep 12, 2013 Updated tests for IRT INSERM SRDC

V0.4 Sep 18, 2013 Updated tests for TIL OFFIS SRDC

V0.5 Sep 20, 2013 Updated tests for ANT OFFIS SRDC

V0.6 Sep 24, 2013 Updated tests for SIL AGFA SRDC

V0.7 Sep 25, 2013 Updated tests for CSCT and general document review

SRDC SRDC

V0.8 Sep 26, 2013 Updated requirement traceability matrix OFFIS SRDC

V0.9 Sep 26, 2013 Updated some use case test scenarios of SIL AGFA SRDC

V0.10 Oct 1, 2013 Updated tests for CDE Repository and general document review

SRDC SRDC

V0.11 Oct 1, 2013 Updated tests for deidentification and pseudonymization services

SRDC SRDC

V1.0 Oct 7, 2013 Final improvements SRDC All Partners

Contributors (Benef.) Gokce Banu Laleci Erturkmen (SRDC), Mustafa Yuksel (SRDC), Ali Anil Sinaci

(SRDC), Suat Gonul (SRDC), Frerk Muller (OFFIS), Tobias Krahn (OFFIS), Gunnar Declerck (INSERM), Sajjad Hussain (INSERM), Kristof Depraetere (AGFA), Giovanni Mels (AGFA)

Responsible Author Suat Gonul Email [email protected]

Beneficiary SRDC Phone +90 3122101763

FP7-287800 SALUS


SALUS Consortium Contacts:

Beneficiary Name Phone Fax E-Mail SRDC Gokce Banu Laleci

Erturkmen +90-312-2101763 +90(312)2101837 [email protected]

EUROREC Georges De Moor +32-9-2101161 +32-9-3313350 [email protected] UMC Niklas Norén +4618656060 +46 18 65 60 80 [email protected] OFFIS Wilfried Thoben

+49-441-9722131

+49-441-9722111

[email protected]

AGFA Dirk Colaert +32-3-4448408 +32 3 444 8401 [email protected] ERS Gerard Freriks +31 620347088 +31 847371789 [email protected] LISPA Davide Rovera +390239331605 +39 02 39331207 [email protected] INSERM Marie-Christine Jaulent +33142346983 +33153109201 marie-

[email protected] TUD Peter Schwarz +49 351 458 2715 +49 351 458 7319 Peter.Schwarz@uniklinikum-

dresden.de ROCHE Jamie Robinson +41-61-687 9433 +41 61 68 88412 [email protected]

FP7-287800 SALUS


EXECUTIVE SUMMARY The SALUS evaluation and testing framework follows the standard process defined on the evaluation reference model and guide ISO/IEC CD 25040[1] of the SQuaRE series of standards. In D7.1.1 – “Functional and Non-functional Evaluation Criteria for SALUS Components” the principles of SALUS evaluation and testing framework is presented. This Deliverable, D7.1.2 Test and Evaluation Report for SALUS Components –R1 reports the results of the first iteration of the tests for the initial releases of SALUS Prototypes developed within the scope of WP4, WP5 and WP6. This deliverable is the first version of the deliverables in which the evaluation reports of the SALUS software components will be presented. In this version, there will be evaluation reports for only a subset of the SALUS software components which are developed for the first prototype of the project. Furthermore, some of the use case tests specified in the D7.1.1 will not be included in this deliverable in case the source code for those use cases has not been implemented yet. The complete set of use case tests will be presented in the next version of this deliverable i.e. the D7.1.3 - Test and Evaluation Report for SALUS Components –R2. Regarding the evaluation methods to be applied, in this deliverable, since the implementation is not finalized yet, we will be reporting only a subset of the evaluation methods which were specified in the D7.1.1 namely:

• Use case tests • Fault tolerance analysis • Unit tests • Integration tests

Another task that has been done in the scope of this deliverable is to update the requirement traceability matrix, which is located at the end of this deliverable” according to the latest status of the requirements.

FP7-287800 SALUS


TABLE OF CONTENTS Executive Summary ............................................................................................................................................ 4!Table of contents ................................................................................................................................................ 5!1! PURPOSE .................................................................................................................................................... 7!2! REFERENCE DOCUMENTS ..................................................................................................................... 7!

2.1! Definitions and Acronyms .................................................................................................................... 7!3! SALUS Evaluation Procedure .................................................................................................................... 10!

3.1! Evaluation of the first SALUS Prototype ........................................................................................... 11!3.1.1! Realizing and Reporting the Tests ............................................................................. 12!

4! Common Data Element Repository ............................................................................................................ 14!4.1! EM1 - Functionality: Functional Test Cases and EM4- Reliability: Fault tolerance analysis ............ 14!

4.1.1! TC-4.2-N1 Import a new Domain Model to CDE Repository .................................. 14!4.1.2! TC-4.2-N2 Annotate a Domain Model with CDEs in CDE Repository ................... 17!4.1.3! TC-4.2-N3 Annotate the CDEs with extraction specifications from Content Models20!4.1.4! TC-4.2-3 Search CDEs .............................................................................................. 22!4.1.5! TC-4.2-4 Browse/View CDEs ................................................................................... 24!4.1.6! TC-4.2- N4 Edit existing CDEs ................................................................................. 26!4.1.7! TC-4.2- N5 Add new CDEs ...................................................................................... 29!4.1.8! TC-4.2- N6 Retrieve Extraction Specifications of CDEs .......................................... 30!

4.2! EM2 - Functionality: Integration Tests ............................................................................................... 32!4.3! EM3 - Functionality: Unit Tests ......................................................................................................... 33!

5! SEMANTIC SERVICES ............................................................................................................................ 34!5.1! EM1 – Functionality: Functional Test Cases, EM4 – Reliability: Fault Tolerance Analysis ............. 34!5.2! EM2 – Functionality Integration Tests ............................................................................................... 34!5.3! EM3 – Unit Tests ................................................................................................................................ 36!

6! Semantic Interoperability Layer ................................................................................................................. 36!6.1! EM1 - Functionality: Functional Test Cases, EM4 - Reliability: Fault tolerance analysis ................ 36!

6.1.1! TC-4.3- N1 Create Content Model Ontologies ......................................................... 36!6.1.2! TC-4.3-2(U) Map Content Model Ontologies to SALUS Common Information Model (CIM) Ontology 39!6.1.3! TC-4.3-3 Include a Terminology System to the SALUS Semantic Resource Set .... 41!6.1.4! TC-4.3-4 (U) Import Mappings between Two Terminology Systems ...................... 43!6.1.5! TC-4.4-1 (U) Retrieve a Clinical Data Instance ........................................................ 44!6.1.6! TC-4.4-2 (U) Convert to SALUS Semantic Data Representation ............................. 46!6.1.7! TC-4.4-3 Export SALUS Semantic Data Representation ......................................... 48!6.1.8! TC-4.4-5 (U) Query in SIL ........................................................................................ 51!6.1.9! TC-4.4-6 (U) Query a SALUS Semantic Data Representation ................................. 52!6.1.10! TC-4.4-8 (U) Select Eligible Patients ...................................................................... 54!6.1.11! TC-4.4-11 (N) Select Single Patient ........................................................................ 56!

6.2! EM2 - Functionality: Integration Tests ............................................................................................... 58!6.2.1! Integration Tests for SIL-DS LISPA ......................................................................... 58!

6.3! EM3 - Functionality: Unit Tests ......................................................................................................... 59!6.3.1! Unit Tests for EHR RDF Service .............................................................................. 59!6.3.2! Unit Tests for SIL-DS (TUD) Service ....................................................................... 59!6.3.3! Unit Tests for Content Converter .............................................................................. 65!6.3.4! Unit Tests for Content Formatter ............................................................................... 66!

7! Technical Interoperability Layer ................................................................................................................ 67!7.1! EM1 - Functionality: Functional Test Cases and EM4 - Reliability: Fault tolerance analysis ........... 67!

7.1.1! TC-5.2-2-2 Querying data source through TIDSQS ................................................. 67!7.2! EM2 - Functionality: Integration Tests ............................................................................................... 71!7.3! EM3 - Functionality: Unit Tests ......................................................................................................... 72!

7.3.1! Unit Tests for LISPA Connector ............................................................................... 72!7.3.2! Unit Tests for QED-EXT Module ............................................................................. 74!

8! ICSR Reporting Tool ................................................................................................................................. 74!

FP7-287800 SALUS


8.1! EM1 - Functionality: Functional Test Cases and EM4 - Reliability: Fault tolerance analysis ........... 74!8.1.1! TC-5.3-1 Reporting an ADE Using ICSR Reporting Tool ....................................... 75!8.1.2! TC-5.3-2 Recording an ADE Notified by the ADE Notification Tool to be Reported Later 78!8.1.3! TC-5.3-3 Accessing Previously Sent and Waiting to be Reported ICSRs ................ 80!8.1.4! TC-5.3-4 Updating and Sending an ICSR Reported in a Previous Session .............. 81!


9! ADE Notification Tool ............................................................................................................................... 84!9.1! EM1 - Functionality: Functional Test Cases and EM4-Reliability: Fault tolerance analysis ............. 84!

9.1.1! TC-6.1-1 ADE Detection ........................................................................................... 84!9.1.2! TC-6.1-2 Semi-automatic ADE Detection ................................................................ 88!


10! De-Identification and Pseudonymization Toolset .................................................................................... 92!10.1! EM1 - Functionality: Functional Test Cases and EM4- Reliability: Fault tolerance analysis .......... 92!

10.1.1! TC-5.4-1.1N De-identify Patient Summary ............................................................ 92!10.1.2! TC-5.4-1.2N De-identify E2B Individual Case Safety Report ................................ 94!10.1.3! TC-5.4-3 Re-identify a Pseudonym ......................................................................... 97!

10.2! EM2 - Functionality: Integration Tests ........................................................................................... 100!10.3! EM3 - Functionality: Unit Tests ..................................................................................................... 101!

11! Case Series Characterization Tool ......................................................................................................... 101!11.1! EM1 - Functionality: Functional Test Cases and EM4-Reliability: Fault tolerance analysis ......... 101!

11.1.1! TC-6.2-1 Case Series Characterization Analysis on Population Data ................... 101!11.2! EM2 - Functionality: Integration Tests ........................................................................................... 106!11.3! EM3 - Functionality: Unit Tests ..................................................................................................... 107!

12! Conclusions ............................................................................................................................................ 107!13! References .............................................................................................................................................. 108!APPENDIX 1.! Assesment of REQUIREMENTS TRACEABILITY MATRIX ........................................ 109!

FP7-287800 SALUS


1 PURPOSE

The purpose of this Deliverable is to report the results of the first iteration of the tests for the initial releases of SALUS components developed within the scope of WP4, WP5 and WP6, following the principles of SALUS evaluation and testing framework defined in D7.1.1- Functional and Non-functional Evaluation Criteria for SALUS Components. In the scope of this deliverable, we will also keep track of the updated requirements considering the latest advancements on the project progress to keep the deliverables consistent with each other. The changes on the requirements will be provided as an input to the “D3.3.2 – Revised Requirement Specification & Conceptual Design of the SALUS Architecture”. Based on the requirement updates, we have also updated the test cases defined in the D7.1.1.

2 REFERENCE DOCUMENTS

The following documents were used or referenced in the development of this document:

• ISO/IEC CD 25040: Software engineering - Software product Quality Requirements and Evaluation

(SQuaRE) - Evaluation reference model and guide, Switzerland: International Organization for

Standardization.

• ISO/IEC 9126-1: Software Engineering-Software product quality-Part 1 : Quality model. Geneva,

Switzerland: International Organization for Standardization .

• SALUS Deliverable D7.1.1 - Functional and Non-functional Evaluation Criteria for SALUS

Components • SALUS Deliverable 3.3.1- Requirement Specification of the SALUS Architecture

2.1 Definitions and Acronyms

Table 1 - List of Abbreviations and Acronyms

Abbreviation/ Acronym DEFINITION

A Answer ANT ADE Notification Tool

C Component CF Code Fragments CL Check List

CSC Case Series Characterization EM Evaluation Module

EMA Efficiency Metric of the Architecture EMC Efficiency Metric ET Execution time F Fault

FMA Functionality Metric of the Architecture FMC Functionality Metric

I Installations IEC International Electrotechnical Commission IFF ICSR form filling IFM ICSR form management IRT ICSR Reporting Tool ISO International Organization for Standardization

FP7-287800 SALUS


Table 2 - Definitions

Term DESCRIPTION data collection of values assigned to base measures, derived measures and/or indicators decision criteria thresholds, targets, or patterns used to determine the need for action or further

investigation, or to describe the level of confidence in a given result derived measure measure that is defined as a function of two or more values of base measures developer individual or organisation that performs development activities (including

requirements analysis, design, testing through acceptance) during the software life cycle process

end user individual person who ultimately benefits from the outcomes of the system entity object that is to be characterised by measuring its attributes evaluation method

procedure describing actions to be performed by the evaluator in order to obtain results for the specified measurement applied to the specified product components or on the product as a whole

evaluation module

package of evaluation technology for measuring software quality characteristics, sub characteristics or attributes

evaluation tool an instrument that can be used during evaluation to collect data, to perform interpretation of data or to automate part of the evaluation

evaluator individual or organisation that performs an evaluation failure termination of the ability of a product to perform a required function or its

inability to perform within previously specified limits fault incorrect step, process or data definition in a computer program functional requirement

requirement that specifies a function that a system or system component must be able to perform

indicator measure that provides an estimate or evaluation of specified attributes derived from a model with respect to defined information needs

measure (noun) variable to which a value is assigned as the result of measurement measure (verb) make a measurement measurement set of operations having the object of determining a value of a measure process system of activities, which use resources to transform inputs into outputs quality model defined set of characteristics, and of relationships between them, which provides a

framework for specifying quality requirements and evaluating quality

JETM Java™ Execution Time Measurement MMA Maintainability Metric of the Architecture MMC Maintainability Metric MTBF Mean Time Between Failure

N Number OP Operating System PH Patient History

PMA Portability Metric of the Architecture PMC Portability Metric RMA Reliability Metric of the Architecture RMC Reliability Metric SDC Standard Deviation Coefficient SIL Semantic Interoperability Layer TAS Temporal Association Screening TC Test Case TIL Technical Interoperability Layer TPD Temporal Pattern Discovery UMA Usability Metric of the Architecture UMC Usability Metric UT Unit Test

FP7-287800 SALUS


requirements expression of a perceived need that something be accomplished or realized software product set of computer programs, procedures, and possibly associated documentation and

data software product evaluation

technical operation that consists of producing an assessment of one or more characteristics of a software product according to a specified procedure

software quality capability of software product to satisfy stated and implied needs when used under specified conditions

software quality characteristic

category of software quality attributes that bears on software quality

software quality evaluation

systematic examination of the extent to which a software product is capable of satisfying stated and implied needs

system a combination of interacting elements organised to achieve one or more stated purposes

user individual or organisation that uses the system to perform a specific function value number or category assigned to an attribute of an entity by making a measurement verification confirmation, through the provision of objective evidence, that specified

requirements have been fulfilled

FP7-287800 SALUS


3 SALUS EVALUATION PROCEDURE

The evaluation procedure for the SALUS Project comprises a number of steps which will divide the evaluation into discrete activities ranging from the specification of the evaluation requirements to the performing the necessary actions for the evaluation and reporting the results. These activities have already been described in the D7.1.1. In this chapter, there will be a short summary regarding to the SALUS evaluation procedure. While determining the evaluation requirements, we have grouped the requirements as a set of quality characteristics defined in the ISO/IEC 9126 standard [2]. Different evaluation levels are assigned to the software characteristics considering the usage and environment of the software products e.g. safety, economic, security and environmental aspects. On the other hand, an evaluation level defines the depth or thoroughness of the evaluation in terms of evaluation techniques to be applied and evaluation results to be achieved. The table below, where D is the lowest level and A is the highest level, shows the level assignments to the quality characteristics considering the aspects mentioned before. These decisions have been explained in detail in D7.1.1. Characteristics Safety aspects Economy aspects Security aspects Environment related

aspects Functionality Level B Level D Level B Level D Reliability Level D Level D Level C Level D Usability Level D Level D Level D Level D Efficiency Level D Level D Level D Level D Maintainability Level D Level D Level D Level D Portability Level D Level D Level D Level D

Table 3 - Quality Characteristics versus Evaluation Levels

The design of the evaluation contains attaching evaluation techniques to software characteristics and levels as well as attaching evaluation modules to evaluation techniques. In the following table for each quality characteristics, a list of evaluation techniques that are selected to be applied according to the evaluation levels selected is presented. Please note that these methods are selected based on the guidelines provided by ISO/IEC 25040.

Quality Characteristic

Level Evaluation Techniques

Functionality B Functional Testing (Through use case and integration tests) Component Testing (Through unit tests)

Reliability C Fault tolerance analysis

Usability D User Interface Inspection Efficiency D Execution Time Measurement Maintainability D Inspection of documents (check lists) Portability D Analysis of installation

Table 4 - Evaluation techniques to implement in SALUS

Among the selected quality characteristics, only the first two characteristics from the Table 4 will be considered in the scope of this deliverable i.e. the functionality and reliability characteristics. To evaluate them, the following evaluation modules are attached to these characteristics. Details of modules can be found in the D7.1.1 and summartized on the next section.

FP7-287800 SALUS


• EM1 - Functionality: Use Case Tests • EM2 - Functionality: Integration Tests • EM3 - Functionality: Unit Tests • EM4 - Reliability: Fault tolerance analysis

3.1 Evaluation of the first SALUS Prototype The first release of some of theSALUS components has been realized within the first sixteen months of the project. These are:

• Common Data Element Repository • Semantic Interoperability Layer • Technical Interoperability Layer • ICSR Reporting Tool • ADE Notification Tool • De-Identification and Pseudonymization Toolset • Case Series Characterization Tool

Each component’s evaluation is executed based on the following functional evaluation modules: use case tests, integration tests and unit tests.

• Use case tests are derived from the Use Case Specifications provided in Requirements Specification of SALUS Project (D3.3.1). Each component's use case is divided into the operations that take place during the execution, based on the use case activity diagram. Scenarios then are built: each one representing a sequence of the operations based on the test case input data. For each scenario the expected result is determined. After performing the evaluation on each test case, the percentage of executions, the outcome matching the expected result is reported for each scenario. In Appendix 2 of D7.1.1, Test Case Specifications for each component are provided. In this deliverable, a subset of these Test Specifications have been run to test the first release of SALUS Prototypes; i.e. as the first releases do not address the full requirements set in D3.3.1, only the test cases that are covered by the first prototypes have been run and reported in this deliverable.

• Integrations tests are intended to test Web Services or RESTful services of SALUS components through the HTTP protocol. These tests ensure that web applications of SALUS system expose their services successfully to the outside world and process the incoming requests successfully. Furthermore, by developing such tests, we will be sure that SALUS components can communicate seamlessly through their HTTP based services. Either interactions between SALUS components via HTTP protocol or RESTful / Web services of SALUS components can be tested by integration tests.

• Unit tests are standalone tests that verify if the component’s function(s) under test meet the

requirements. For SALUS unit tests are developed and run for the code involved in each component. The percentages of successful unit cases for each unit are to be reported. In this deliverable the results of the unit tests run for the first release of SALUS Prototypes are reported. The source code coverage with unit tests should be as much as possible; however API methods i.e. public methods that can be used by other clients are given priority to be tested.

In addition to this, Fault Tolerance Tests have also been run for the first releases by testing the system through erroneous input cases through the Fault Tolerance Test Cases defined in D7.1.1. In this Deliverable, the results of all these test are reported as separate sections for each of the SALUS component developed in the first prototype as Sections 4-Section 11. In D7.1.1, tests for the non-functional evaluation modules for evaluating Reliability, Usability, Efficiency, Maintainability and Portability have also been defined, and which will be executed through fault tolerance

FP7-287800 SALUS


analysis, user interface analysis, execution time measurement, inspection of documentation, and analysis of software installation procedures. For the evaluation of the first release of SALUS prototype, we have not addressed these test cases. These will be run for the final releases of SALUS components, and the results will be reported in Deliverable D7.1.3 - Test and Evaluation Report for SALUS Components - R2. Also, the complete set of use case tests will not be included in this first version of deliverable, since the implementation is not finalized yet. While some of the use cases will not be included as a whole, some of them will be included even if a partial implementation regarding those use cases has been done. For the latter, if there are some detailed test cases for which the implementation is not ready, the test result will be stated as Not Applicable (N/A).

3.1.1 Realizing and Reporting the Tests

Realization of the use case tests are done by manually applying the operations that are constructing the use cases being tested. There are basic and alternative flows which are combined in different combinations to create different scenarios to test the use cases thoroughly. These scenarios are run several times with different variables. Each run is named as test case. Several test cases are defined in the Section 4 for all components of the SALUS system. Results of these test cases are reported with a description.

Integration and unit tests are run automatically when the SALUS code base is compiled. Throughout the SALUS Project, we use the Maven framework to compile the code base. When the project is compiled with the mvn install command, all the integration and unit tests are run automatically and reports of these tests are put into the surefire-reports directory which is found in the build directory of each particular maven module. Furthermore, it is possible to generate more human readable reports using the mvn surefire-report:report command. As a result of this command, reports are generated in html format within the site directory which is also located in the build directory of a particular maven module. An example for surefire report can be seen in XML format below: <?xml version="1.0" encoding="UTF-8"?><testrun name="eu.salusproject.til.iheProfiles.qedext.webservice.QedWebServiceTest" tests="2" started="2" failures="0" errors="0" ignored="0"> <testsuite name="eu.salusproject.til.iheProfiles.qedext.webservice.QedWebServiceTest" time="19.431"> <testcase name="sendQedQueryAndRetrieveResponse" classname="eu.salusproject.til.iheProfiles.qedext.webservice.QedWebServiceTest" time="0.22"/> <testcase name="sendHqmfQueryAndRetrieveResponse" classname="eu.salusproject.til.iheProfiles.qedext.webservice.QedWebServiceTest" time="0.214"/> </testsuite> </testrun> An example for surefire reports can be seen in HTML view below:

FP7-287800 SALUS


FP7-287800 SALUS


4 COMMON DATA ELEMENT REPOSITORY

4.1 EM1 - Functionality: Functional Test Cases and EM4- Reliability: Fault tolerance analysis

4.1.1 TC-4.2-N1 Import a new Domain Model to CDE Repository

Description This use case is for importing a Domain Model (or data dictionary in a domain) (e.g.

BRIDG Model, HITSP C154 Data Dictionary, OMOP Common Model, HL7 Green CDA

Model) to the Common Data Element (CDE) Repository. A domain model can be in

various formats: including a comma separated Model (CSV), UML Model, a database

model expressed in Data Definition Language, an XSD model or a local domain ontology

(e.g. data definition ontology).

The final objective is to identify the CDEs in these domain models and save it as a new

domain model to the CDE repository; annotate the CDEs identified in these new domain

models added to the CDE repository with the existing CDEs in the repository (UC-4.2-N2

Annotate a Domain Model with CDEs in CDE Repository). Parent

- Included sub-use cases -

Extended sub-use cases -

Scope CDE Repository

Actor(s) CDE Repository User (Domain Expert)

Goal Importing a new domain to enrich Common Data Element Set maintained within the CDE

Repository. Trigger

This use case is initiated by a CDE Repository User (Domain Expert) when a new domain

model needs to be added to the CDE Repository to extend the CDE Set if necessary. Frequency

Whenever a new domain needs to be added to the CDE Repository, approximately

once/twice a year. Minimal Post conditions CDE Repository informs the user (Domain Expert) about the result of the import

operation. Success Post-Conditions The Domain Model is successfully imported to the CDE Repository and ready to be

annotated with the existing CDEs in the CDE Repository

FP7-287800 SALUS


4.1.1.1 Use-Case (B)asic flows

#ID Description

B1 Domain Expert intends to import a new domain model and points to the location where the domain model

resides

B2 CDE Repository retrieves the model, parses it and imports this content as a new Domain Model consisting of

new CDEs in the CDE Repository.

B3 Domain Expert views the result, if necessary modifies the exported CDEs and confirms the import operation.

4.1.1.2 Use-Case (A)lternative flows

#ID Description

A1 CDE Repository is unable to access or parse the domain model. CDE Repository informs the Domain Expert

about the problem and present guidance to solve the problem.

A2 The tool cannot save the imported content model to the repository since the repository is temporarily

unavailable, the system affected should either retry later to communicate with the repository or send back an

error message

4.1.1.3 Fault Cases These will be used for executing the test scenarios for fault tolerance test:

#ID Fault Fault Tolerance Requirement Description

FT1 Inaccessible domain model CDE Repository is unable to access the domain model. CDE

Repository informs the Domain Expert about the problem and

present guidance to solve the problem.

FT2 Unsupported Domain Model Format CDE Repository is unable to parse the domain model. CDE

Repository informs the Domain Expert about the problem and

present guidance to solve the problem.

FP7-287800 SALUS


4.1.1.4 (S)cenarios to be tested

#ID Flows

S1 B1 B2 B3

S2 B1 A1

S3 B1 B2 A2

4.1.1.5 Execution (Functionality and Fault Tolerance Tests) Results

#ID Scenario

Variables Expected

Result

Domain

Model Type

Fault

TC1 S1 DDL None B3

TC2 S1 CSV None B3

TC3 S2 CSV FT1 A1

TC4 S2 DDL FT1 A1

TC5 S3 DDL FT2 A2

TC6 S3 CSV FT2 A2

Domain Model Type

4.1.1.6 Results

FP7-287800 SALUS


ID Test Case Execution Description Result

TC1 OMOP is selected to import. CDE Repository parses corresponding DDL and

imported OMOP as new Domain Model consisting of new CDEs in the CDE

Repository. Domain Expert is able to view the result and modify the exported

CDEs.

SUCCESS

TC2 SALUS CIM is selected to import. CDE Repository parses corresponding CSV

and imported SALUS CIM as new Domain Model consisting of new CDEs in the

CDE Repository. Domain Expert is able to view the result and modify the

exportedCDEs.

SUCCESS

TC3 OMOP is selected to import. CDE Repository cannot parse corresponding DDL.

An error message is displayed to Domain Expert.

SUCCESS

TC4 SALUS CIM is selected to import. CDE Repository cannot parse corresponding

CSV. An error message is displayed to Domain Expert.

SUCCESS

TC5 OMOP is selected to import. CDE Repository is not reachable. An error message

is displayed to Domain Expert.

SUCCESS

TC6 SALUS CIM is selected to import. CDE Repository is not reachable. An error

message is displayed to Domain Expert.

SUCCESS

4.1.2 TC-4.2-N2 Annotate a Domain Model with CDEs in CDE Repository

Description The CDE Repository supports a mechanism to upload new Domain Models to the CDE

Repository. The purpose is to further populate the CDE Repository by new CDEs, and link the

newly extracted CDEs to the existing CDEs in SALUS CDE Repository paving the way to

Semantic interoperability. Parent

- Included sub-use cases UC-4.2-N1: Import a new Domain Model to CDE Repository

UC-4.2-4 Browse/View CDEs Extended sub-use cases -



Goal The goal of this use case is to annotate the newly extracted CDEs with the available CDEs in

SALUS Core Ontology Trigger

This use case is initiated by a CDE Repository User (Domain Expert) when a new Domain

Model needs to be added to CDE Repository. Frequency

Whenever a new Domain Model is added to CDE Repository, approximately once/twice a year Minimal Post conditions CDE Repository informs the Domain Expert about the result of the annotation operation

Success Post-Conditions The existing CDEs corresponding to the newly extracted CDEs in the new Domain Model have

been identified and the new CDEs have been annotated with these CDEs.

FP7-287800 SALUS



#ID Description

B1 The candidate CDEs in the new Domain Model are identified

B2 The already existing matching CDEs to the candidate CDEs are searched from the CDE Repository [UC-4.2-3

Search CDEs]

B3 The matching CDEs are presented to the user

B4 The user checks the matched CDEs and selects the most appropriate one

B5 A link between the selected CDEs are created in reference to well defined KOS ontologies like

skos:exactMatch, skos:closeMatch


#ID Description

A1 Domain Expert cancels the annotation operation.



FT1 CDE Repository is down The tool cannot save the annotations/mappings to the repository since

the repository is temporarily unavailable, the system affected should

either retry later to communicate with the repository or send back an

error message.

FP7-287800 SALUS



4.1.2.5 #ID 4.1.2.6 Flows

S1 B1 B2 B3 B4 B5

S2 B1 B2 B3 B4 A1

S3 B1 B2 B3 A1

4.1.2.7 Execution (Functionality and Fault Tolerance Tests)

#ID Scenario

Variables Expected

Result

Cancel Fault

TC1 S1 No None B5

TC2 S1 No FT1 FT1

TC3 S2 Yes None A1

TC4 S3 Yes None A1

4.1.2.8 Results


TC1 The new Data Model is identified. Domain Expert searches a CDE in existing

CDEs. The results are displayed and the expert selects one of them and an

appropriate relationship. The user sends the form and the relation is defined.

SUCCESS



appropriate relationship. The user sends the form but the server is unreachable. An

error message is supposed to be displayed but it is not.

FAIL



appropriate relationship. The user cancelled the operation and returned the

previous page.

SUCCESS


CDEs. The results are displayed. The user cancelled the operation and returned the

previous page.

SUCCESS

FP7-287800 SALUS


4.1.3 TC-4.2-N3 Annotate the CDEs with extraction specifications from Content Models

Description The CDE Repository supports a mechanism to link the CDEs in CDE repository to the implementation dependent Content Models in care and research domains like ASTM/HL7 CCD, EN 13606 EHR Extracts, OMOP database, Content Model Ontologies (e.g. data definition ontology) through extraction specifications. These Extraction Specifications are scripts (such as XPATH, SPARQL, SQL) that can be used to directly retrieve the data element indicated by the CDE definition from the medical content available as Content Models. Extraction specifications to the ontological representation of SALUS CDE Set, i.e. SALUS Common Information Model Ontology can also be defined as SPARQL queries.

Parent

- Included sub-use cases

UC-4.2-N1: Import a new Domain Model to CDE Repository UC-4.2-4 Browse/View CDEs

Extended sub-use cases - Scope

CDE Repository Actor(s)

CDE Repository User (Domain Expert) Goal

The goal of this use case is link the CDEs with implementation dependent Content Models through Extraction Specifications.

Trigger This use case is initiated by a CDE Repository User (Domain Expert) when a new Domain Model needs to be added to CDE Repository or a new Content Model is needed to be associated with the available CDEs in the repository

Frequency Whenever a new Domain Model is added to CDE Repository or a new Content Model is needed to be associated with the available CDEs in the repository, approximately once/twice a year

Minimal Post conditions CDE Repository informs the Domain Expert about the result of the association operation Success Post-Conditions

The existing CDEs have been successfully linked with the implementation dependent Content Models through Extraction Specifications..

FP7-287800 SALUS



#ID Description

B1 The source CDEs in the CDE Repository are identified through the UC-4.2-4 Browse/View CDEs use case.

B2 The user selects a Content Model to which an extraction specification will be added for the selected CDE

B3 The user specifies the Extraction specification script either as XPATH (in case of XML based content models) or as SPARQL (in case of Ontology based content models ) or as SQL (in case of database based content models )

B4 A link between the selected CDEs and the selected Content Models are created


#ID Description

A1 Domain Expert cancels the association operation.


FP7-287800 SALUS



FT1 CDE Repository is down The tool cannot save the associations to the repository since the repository is temporarily unavailable, the system affected should either retry later to communicate with the repository or send back an error message.


#ID Flows

S1 B1 B2 B3 B4

S2 B1 B2 B3 A1


#ID Scenario

Variables Expected

Result

Cancel Fault

TC1 S1 No None B4

TC2 S1 No FT1 FT1

TC3 S2 Yes None A1

4.1.3.6 Results


TC1 The user browses and selects a Content Model to which an extraction specification

will be added for the selected CDE. The user specifies the extraction specification

and adds it to the Content Model. A link between the selected CDEs and the

selected Content Model is created.

SUCCESS



and adds it to the Content Model. Since the repository is down, the extraction

specification cannot be saved but no error message is displayed.

FAIL



and cancels the operation. The user returns the previous page.

SUCCESS

4.1.4 TC-4.2-3 Search CDEs

Description CDE Repository User (Domain Expert) can search the CDE Repository for CDEs matching the

search criteria.

FP7-287800 SALUS


Parent -

Included sub-use cases -


Scope

CDE Repository Actor(s)

CDE Repository User (Domain Expert) Goal

Finding the CDEs in the CDE Repository matching the search criteria. Trigger

The CDE Repository User can trigger this action from the CDE Repository Menu Frequency

• Whenever a new Content Model is to be added to SALUS Semantic Interoperability

Framework, approximately once/twice a year

• Whenever initiated by the CDE Repository User Minimal Post conditions The user is informed about the status of the search operation.

Success Post-Conditions Matching CDEs are discovered and presented to the CDE Repository User


#ID Description

B1 The CDE Repository User specifies the search criteria, which could be keywords, selected terminology codes,

or a candidate CDE

B2 Based on the specified criteria, CDE Repository is searched and the matching CDEs are displayed to the user


#ID Description

A1 Based on the specified criteria, CDE Repository is searched and no matching CDEs are found. Appropriate

message is shown to the Domain Expert to repeat the search with different search criteria.



FT1 CDE Repository is down The tool cannot search through the repository since the repository is

temporarily unavailable, the system affected should either retry later

to communicate with the repository or send back an error message.

FP7-287800 SALUS



#ID Flows

S1 B1 B2

S2 B1 A1


#ID Scenario

Variables Expected

Result

Cancel Fault

TC1 S1 No None B2

TC2 S1 No FT1 FT1

TC3 S2 No None A1

TC4 S2 No FT1 FT1

4.1.4.6 Results


TC1 The Domain Expert defined the search criteria and made the search. The results of

the search are displayed.

SUCCESS

TC2 The Domain Expert defined the search criteria and made the search. The repository

is down so an error message indicating that the server is unreachable is supposed

to be displayed. However, an inappropriate error message is displayed.

FAIL

TC3 The Domain Expert defined the search criteria and made the search. However, no

result is found. An error message is displayed.

SUCCESS

TC4 The Domain Expert defined the search criteria and made the search. The repository

is down so an error message, indicating that the server is unreachable, is supposed

to be displayed. However, an inappropriate error message is displayed.

FAIL

4.1.5 TC-4.2-4 Browse/View CDEs

Description CDE Repository User can browse the CDE Repository, and view the details of a selected CDE

Parent -


Extended sub-use cases UC-4.2-3 Search CDEs

FP7-287800 SALUS




Goal Browsing CDE Repository and viewing the details of a selected CDE

Trigger • The CDE Repository User can trigger this action from the CDE Repository Menu

• After searching a CDE through Search CDEs. Frequency

Whenever initiated by the CDE Repository User Minimal Post conditions The user is informed about the status of the search operation.

Success Post-Conditions The CDE Repository User can successfully browse through different categories, and select and

view the detailed attributes of the selected CDEs through CDE Repository Interface


#ID Description

B1 The CDE Repository User can search a specific CDE through “UC-4.2-3 Search CDEs” or Browse through the

categories available in the CDE Repository

B2 The CDE Repository User Interface displays the detailed attributes of the selected CDE. The CDE Repository

User can filter the attributes to be displayed as a configuration.

B3 If required, the CDE Repository User can navigate through the links between the CDEs (such as parent CDE)


#ID Description

A1 Domain Expert cancels the operation while the system is in progress of browsing. The system returns to home

browser screen.


FP7-287800 SALUS



FT1 CDE Repository is down The tool cannot browse through the repository since the repository is




#ID Flows

S1 B1 B2 B3

S2 B1 B2 A1


#ID Scenario

Variables Expected

Result

Cancel Fault

TC1 S1 No None B3

TC2 S1 No FT1 FT1

TC3 S2 Yes None A1

TC4 S2 Yes FT1 FT1

4.1.5.6 Results


TC1 The user browses through CDEs and is able to display their detailed attributes. The

user can navigate through the parent CDE of selected CDE.

SUCCESS

TC2 The user tried to browse CDEs but the repository is down. An error message is

displayed but it is not appropriate.

FAIL

TC3 The user browses through CDEs and is able to display their detailed attributes. The

user closed the details and returned the previous page.

SUCCESS

TC4 The user tried to browse CDEs but the repository is down. An error message is

displayed but it is not appropriate.

FAIL

4.1.6 TC-4.2- N4 Edit existing CDEs

Description CDE Repository User can edit and update the CDEs in the CDE Repository

Parent -


FP7-287800 SALUS


Extended sub-use cases UC-4.2-3 Search CDEs



Goal Browsing CDE Repository and viewing the details of a selected, and update the details of CDE if

necessary Trigger

• The CDE Repository User can trigger this action from the CDE Repository Menu

• After searching a CDE through Search CDEs. Frequency

Whenever initiated by the CDE Repository User Minimal Post conditions The user is informed about the status of the update operation.

Success Post-Conditions The CDE Repository User can successfully update the selected attributes of the selected CDE

through CDE Repository Interface


#ID Description

B1 The CDE Repository User can search a specific CDE through “UC-4.2-3 Search CDEs” or Browse through

the categories available in the CDE Repository

B2 The CDE Repository User Interface displays the detailed attributes of the selected CDE.

B3 If required, the CDE Repository User can update the allowed attributes of a CDE (such as Value Domain,

Property Name) through the interfaces provided


#ID Description

A1 Domain Expert cancels the operation. The system returns to home browser screen.


FP7-287800 SALUS



FT1 CDE Repository is down The tool cannot update the repository since the repository is




#ID Flows

S1 B1 B2 B3

S2 B1 B2 A1

S3 B1 B2 B3 A1


#ID Scenario

Variables Expected

Result

Cancel Fault

TC1 S1 No None B3

TC2 S1 No FT1 FT1

TC3 S2 Yes None A1

TC4 S3 Yes None A1

4.1.6.6 Results


TC1 The user browses through the repository and selects a CDE to display the details.

The user changes the attributes of the selected CDE.

SUCCESS


The user changes the attributes of the selected CDE. The repository is down and an

error message is displayed.

SUCCESS


The user cancels the operation and returns the previous page.

SUCCESS


The user changes the attributes of the selected CDE and cancels the operation. The

user is returned to previous page.

SUCCESS

FP7-287800 SALUS


4.1.7 TC-4.2- N5 Add new CDEs

Description CDE Repository User can add a new CDE to a selected Domain Model in the CDE Repository

Parent -





Goal Adding a new CDE definition to a selected Domain Model manually

Trigger The CDE Repository User can trigger this action from the CDE Repository Menu

Frequency Whenever initiated by the CDE Repository User

Minimal Post conditions The user is informed about the status of the create operation.

Success Post-Conditions The CDE Repository User can successfully create a new CDE definition through CDE

Repository Interface


#ID Description

B1 The CDE Repository User selects a Domain Model available in the CDE Repository

B2 The CDE Repository User adds a new CDE definition to the selected model by filling in the detailed attributes

of the CDE.

B3 The CDE Repository User saves the new CDE definition.


#ID Description

A1 Domain Expert cancels the operation. The system returns to home browser screen.


FP7-287800 SALUS



FT1 CDE Repository is down The tool cannot save to the repository since the repository is




#ID Flows

S1 B1 B2 B3

S2 B1 B2 A1


#ID Scenario

Variables Expected

Result

Cancel Fault

TC1 S1 No None B3

TC2 S1 No FT1 FT1

TC3 S2 Yes None A1

4.1.7.6 Results


TC1 The user selects a data model and fills the form to add a new CDE. The form is

sent and the new CDE definition is saved.

SUCCESS

TC2 The user selects a data model and fills the form to add a new CDE. The form is

sent but the server is down. An error message is displayed.

SUCCESS

TC3 The user selects a data model and fills the form to add a new CDE. The user

cancels the operation and returned to previous page.

SUCCESS

4.1.8 TC-4.2- N6 Retrieve Extraction Specifications of CDEs

Description An external system actor (Metadata Requestor) can retrieve the extraction specification of

a selected CDE for a selected Content Model through the Rest Service interfaces supported Parent -

FP7-287800 SALUS


Included sub-use cases - Extended sub-use cases -


Actor(s) Metadata Requestor

Goal Retrieving the extraction specification of a selected CDE for a selected Content Model

Trigger Metadata Requestor can trigger this action by sending a “Retrieve Metadata” request to the

CDE Repository Frequency

Whenever initiated by the Metadata Requestor Minimal Post conditions The Metadata Requestor is informed about the status of the “Retrieve Metadata” request Success Post-Conditions The Metadata Requestor can successfully retrieve the extraction specification of a selected

CDE for a selected Content Model


#ID Description

B1 The Metadata Requestor sends a “Retrieve Metadata” request to the CDE repository indicating the Domain

Model and the identifier of the CDE

B2 The CDE Repository processes this request and prepares the “Retrieve Metadata Response” message

including the set of “Extraction specifications” available for this CDE in the CDE Repository.

B3 The Metadata Request parses the “Retrieve Metadata Response” message and retrieves the extraction

specification for the selected Content Model


None



FT1 CDE not available The CDE queried is not available in the CDE Repository. The CDE

Repository should send a fault message

FT2 Request Message Error The “Retrieve Metadata Request” Message cannot be parsed. The

CDE Repository should send a fault message


#ID Flows

S1 B1 B2 B3

FP7-287800 SALUS



#ID Scenario

Variables Expected

Result

Fault

TC1 S1 None B3

TC2 S1 FT1 FT1

TC3 S1 FT2 FT2

4.1.8.6 Results


TC1 “Retrieve Metadata Request” is sent to CDE repository indicating the Domain

Model and the identifier of the CDE. The CDE Repository processes this request

and prepares the “Retrieve Metadata Response” message including the set of

“Extraction specifications” available for this CDE in the CDE Repository. The

Metadata Request parses the “Retrieve Metadata Response” message and retrieves

the extraction specification for the selected Content Model.

SUCCESS


Model and the identifier of the CDE. CDE is not available and CDE repository

sends a response with empty body.

FAIL


Model and the identifier of the CDE. CDE cannot parse the request and returns a

fault message.

SUCCESS

4.2 EM2 - Functionality: Integration Tests Currently the CDE Repository has not been integrated with any of the SALUS components, and hence integration tests have not been executed. In the second release of the SALUS prototype, it is planned to integrate the CDE Repository Component with the Post Marketing Safety Study Tool (to be used in ROCHE Pilot). The integration tests will be included in D7.1.3.

FP7-287800 SALUS


4.3 EM3 - Functionality: Unit Tests

FP7-287800 SALUS


5 SEMANTIC SERVICES

5.1 EM1 – Functionality: Functional Test Cases, EM4 – Reliability: Fault Tolerance Analysis

Semantic Services are composed of components that organize the flow among several SALUS components.So, they are already used in the use cases and so in the test cases as intermediate components. As a result, we have not defined any test cases specific to these components.

5.2 EM2 – Functionality Integration Tests

FP7-287800 SALUS


FP7-287800 SALUS


5.3 EM3 – Unit Tests

6 SEMANTIC INTEROPERABILITY LAYER

6.1 EM1 - Functionality: Functional Test Cases, EM4 - Reliability: Fault tolerance analysis

6.1.1 TC-4.3- N1 Create Content Model Ontologies

Description

This use case aims to serve to the creation of SALUS Semantic Resource Set, by creating

the Content Model Ontologies for the required Content Models in SALUS Pilot

applications. Currently four types have been identified:

• Creating a Content Model Ontology for TUD ORBIS Database Schema

• Creating a Content Model Ontology for HL7 CDA Model

• Creating a Content Model Ontology for OMOP CDM

• Creating a Content Model Ontology for EN 13606 EHR Extract Model Parent -

FP7-287800 SALUS


Scope SALUS Semantic Resource Set Actor(s) SALUS member

Goal Creating the Content Model Ontologies for the required Content Models in SALUS Pilot

application Trigger - Frequency These ontologies will be created once for each such content model. Preconditions Content Model has a formal description such as DDL, XSD Minimal Post-Conditions Content Model Ontologies are created Success Post-Conditions Content Model Ontologies are created


#ID Description

B1 SALUS member selects the Content Model

B2 Based on the type of the selected Content Model different mechanisms are used to create the Content Model

Ontology

B3 The Content Model Ontology is added to SALUS Semantic Resource Set

6.1.1.2 Use-Case (A)lternative flows None

6.1.1.3 Fault Cases None

FP7-287800 SALUS



#ID Flows

S1 B1 B2 B3

6.1.1.5 Execution

#ID Scenario

Variables Expected

Result

Content Model

Type

Fault

TC1 S1 TUD ORBIS

DB

None B3

TC2 S1 HL7 CDA None B3

TC3 S1 OMOP CDM None B3

TC4 S1 EN 13606

EHR Extract

None B3

6.1.1.6 Results


TC1 Creation of content model ontology for the TUD ORBIS Database

Schema. For the TUD ORBIS Database Schema, there is no support

for this. There is a tool to extract an ontology based on the database

schema. However due to the complexity and size of the schema,

manual corrections are needed on the resulting ontology. This tool is

not developed by the ACA team. Overall process is not automated as

it will be done once.

SUCCESS

TC2 XSD Schema is parsed and transformed into RDF format by the

Ontmalizer tool developed in the scope of SALUS. SUCCESS

TC3 The OMOP content model ontology has been created manually by

representing the fields contained in the database schema as

ontological elements.

SUCCESS

TC4 We do not have the content model ontology for EN 13606 EHR

Extract Model yet. It is planned to be completed in the next prototype

phase.

N/A

FP7-287800 SALUS


6.1.2 TC-4.3-2(U) Map Content Model Ontologies to SALUS Common Information Model (CIM) Ontology

Description

This use case aims to serve to the creation of SALUS Semantic Resource Set, by creating the

mapping rules between the Content Model Ontologies and SALUS CIM Ontology (which is the

semantic representation of SALUS CDEs). Having the Content Models already associated with

the CDEs in SALUS CDE Repository through the “UC-4.2- N2 Annotate a Domain Model with

CDEs in CDE Repository” use case would help the Domain Expert while defining the mappings. Parent - Scope SALUS Semantic Resource Set Actor(s) Domain Expert

Goal Creating the mapping rules between Content Model Ontologies and SALUS CIM Ontology, to

extend SALUS Semantic Resource Set Trigger The Domain Expert can trigger this action.

Frequency Mapping definitions between Content Model Ontologies are created only once. However, the

actual mapping operation occurs whenever initiated by the Domain Expert

Preconditions 1. SALUS CIM Ontology is created

2. Content Model Ontologies have already been created and saved through the “UC-4.3-N1

Create Content Model Ontologies” use case Minimal Post-Conditions Domain Expert is informed about the results of the mapping operation

Success Post-Conditions

Mapping rules between Content Model Ontologies and SALUS CIM Ontology are created and

saved to extend SALUS Semantic Resource Set


#ID Description

B1 Domain Expert selects the Content Model Ontology

B2 Domain Expert defines mapping rules from the selected Content Model Ontology class to class(es) of the

target SALUS CIM Ontology as N3 rules. While defining the mappings, the Domain Expert can optionally use

“UC-4.2-4 Browse/View CDEs” use case to see the associations between Content Models and the CDEs in

SALUS CIM

B3 Domain Expert repeats step 3 for each Content Model Ontology class that he wants to define mapping to the

SALUS CIM Ontology.

B4 Domain Expert reviews and confirms the overall mapping.

B5 Mapping rules are tested with sample data sets.


FP7-287800 SALUS




FT1 Mapping not correct Domain expert checks the system with sample datasets, mapping

result is not correct. The operation is reviewed starting from B2.


#ID Flows

S1 B1 B2 B3 B4 B5

6.1.2.5 Execution

#ID Scenario

Variables Expected

Result

Fault

TC1 S1 None B7

TC2 S1 FT1 FT1

6.1.2.6 Results


TC1 Conversion of content model ontologies to the SALUS CIM ontology. AGFA has

written tests to convert TUD ORBIS ontology to first version of SALUS CIM;

SRDC has written tests to convert CCD-CDA and QEDExt ontologies to first

version of SALUS CIM. Conversion rules are applied in a modular way based on

different entities such as condition, medication, etc. using the EYE tool of AGFA.

Correct results are obtained.

SUCCESS

TC2 Conversion of content model ontologies to the SALUS CIM ontology. AGFA has

written tests to convert TUD ORBIS ontology to first version of SALUS CIM;

SRDC has written tests to convert CCD-CDA and QEDExt ontologies to first

version of SALUS CIM. Incomplete or wrong conversion rules are applied in a

modular way based on different entities such as condition, medication, etc. using

the EYE tool of AGFA. Wrong results are obtained.

SUCCESS

FP7-287800 SALUS


6.1.3 TC-4.3-3 Include a Terminology System to the SALUS Semantic Resource Set

Description

This use case is for including a Terminology System (a subset, or the complete content), which

did not exist in the Semantic resource set of SALUS previously. A terminology system will be

represented as an ontology to extend the SALUS Semantic Resource Set. The terminology

system to be imported can be an international one, or proprietary. Parent - Scope SALUS Semantic Resource Set Actor(s) SALUS member Goal Enriching the SALUS Semantic Resource Set with Terminology Systems

Trigger This use case is initiated when a new Terminology System needs to be added to extend the

SALUS Semantic Resource Set. Frequency Whenever a new Terminology System needs to be added, approximately once/twice a year. Preconditions - Minimal Post-Conditions SIL informs the SALUS member about the result of the import operation.


The terminology system is successfully imported to the SALUS Semantic Resource Set, aligned

with the SALUS Core Ontology and ready for mapping with other available terminology

systems.


#ID Description

B1 A new terminology system is intended to be included in the SALUS system. Either a subset or the complete content of a terminology system can be included.

B2 SALUS member points to the location where terminology system to be imported resides.

B3 The system retrieves the actual content of the terminology system and if it is not represented already as an ontology, imports this content as a new ontology and includes it to the SALUS Semantic Resource Set.

B4 (Optional) Terminology Expert fine tunes the generated terminology ontology (e.g. change of a designation, adding/modifying a semantic relationship between two coded terms).

B5 Terminology Expert views the result and confirms the operation.


#ID Description

A1 The Terminology Expert references a public terminology repository.

A2 The Terminology Expert references a local terminology file.


FP7-287800 SALUS



#ID Flows

S1 B1 B2 A1 B3 B4 B5

S2 B1 B2 A1 B3 B5

S3 B1 B2 A2 B3 B4 B5

S4 B1 B2 A2 B3 B5

6.1.3.5 Execution

#ID Scenario

Variables Expected

Result

Fault

TC1 S1 None B5

TC3 S2 None B5

TC5 S3 None B5

TC7 S4 None B5

6.1.3.6 Results


TC1 Terminology systems such as ICD10, ICD9, Snomed CT are loaded into the

SALUS terminology server backed by a Virtuoso triple store from local files. The

loading operation is done by manually and this is not an automatized process.

However, we have not done any tuning on the loaded terminology systems.

N/A

TC2 Terminology systems such as ICD10, ICD9, Snomed CT are loaded into the

SALUS terminology server backed by a Virtuoso triple store from local files. The

loading operation is done by manually and this is not an automatized process.

SUCCESS

TC3 Terminology systems are not parsed from a remote location. N/A

TC4 Terminology systems are not parsed from a remote location. N/A

FP7-287800 SALUS


6.1.4 TC-4.3-4 (U) Import Mappings between Two Terminology Systems

Description This use case is for importing the mappings between the codes of two terminology systems

included in the SALUS Semantic Resource Set. Parent - Scope SALUS Semantic Resource Set Actor(s) SALUS member

Goal Defining procedures to import the terminology system mappings, which are developed in the

scope of external projects such as OntoADR, Crossmap, OMOP, etc.

Trigger This use case is initiated by a SALUS member after a new Terminology System has been added

to the SALUS Semantic Resources Set. Frequency Once after a new Terminology System is added to SALUS Semantic Resources Set.

Preconditions The terminology system in question should be included through “UC-4.3-3 Include a

Terminology System to the SALUS Semantic Resource Set” use case. Minimal Post-Conditions The system informs the SALUS member about the result of the mapping operation.


Mappings between codes from different terminology systems are successfully saved, and ready

to be used while doing translation of clinical content or running semantic queries on top of the

SIL.


#ID Description

B1 SALUS member finds external projects providing mappings between two terminology systems which are

included in the SALUS Semantic Resource Set.

B2 SALUS member defines a method automatizing the mapping extraction from the found projects which can be

used within the scope of SALUS.

B3 SALUS member runs the method and get the mappings in RDF format.

B4 SALUS member loads the mappings in RDF format to the SALUS Semantic Resource Set.




FT1 Inefficient mapping extraction

method

The mapping extraction method is inefficient for some reason and it

takes too much time to complete. The operation is reviewed starting

from B2.

FP7-287800 SALUS



#ID Flows

S1 B1 B2 B3 B4

6.1.4.5 Execution

#ID Scenario

Variables Expected

Result

Fault

TC1 S1 None B4

TC2 S2 FT1 FT1

6.1.4.6 Results


TC1 Mappings between terminology systems are found from project such as OntoADR,

OMOP, Crossmap. Mapping extraction methods were defined and run. The

resultant RDF mappings were added into the SALUS Semantic Resource Set to be

used during the terminology reasoning operations. However, these set of

operations are done manually and they are not automatized at all.

SUCCESS

TC2 Mappings between terminology systems are found from project such as OntoADR,

OMOP, Crossmap. Mapping extraction methods were defined and run. However,

the running time of these methods were too much. The main reason of excessive

run time was the size of the output, since the hierarchical structure of the codes in

the terminology systems are considered while producing the mappings to be used

in SALUS project. In such cases, we terminated the execution of terminology

mapping process, amended our methods and restarted execution.

SUCCESS

6.1.5 TC-4.4-1 (U) Retrieve a Clinical Data Instance

Description This use case is for retrieving a clinical data instance and transforming it from its native format to

a semantic representation (to an ontological instance) using a content model ontology. The

content model ontology is a semantic modeling of the native format. Parent - Scope Semantic Interoperability Layer

Actor(s) Data Source (EHR data source, Lab Data Source, Data Warehouse), Technical Interoperability

Layer (TIL)

FP7-287800 SALUS


Goal Retrieving a clinical data instance. Create a faithful one-to-one formalization of the originating

native data instance using the respective content model ontology.

Trigger

The system requests clinical data from an EHR Data Source or Lab Data Source or Data

Warehouse. Alternatively, if this use-case is used in the scope of a subscription based use-case,

then the clinical data is not requested (pulled) from the data source, instead, the data source

pushes the clinical data into this use-case through the related use-case (e.g. UC-5.1-2 Subscribe

to Data Source). Frequency Whenever a request arrives to retrieve a clinical data instance.

Preconditions The system is configured with queries to fetch the underlying native data instances from the Data

Sources.

The query is expressed in content model ontology of the Data Source. Minimal Post-Conditions N/A Success Post-Conditions Clinical data from the Data Source is available in the content model ontology of the Data Source.


#ID Description

B1 The Query is translated to the native query format of the Data Source.

B2 The system makes a connection to the Data Source. The system sends a request for data to the data source in

the native query format of the data source.

B3 Based on the type of the data source, data source can be connected by invoking the “UC-5.2-2 Query Data

Source” Use case (supported through Technical Interoperability Layer).

B4 The data source returns requested clinical data in the native format of the Data Source.

B5 The system converts the clinical data to a semantic representation using the data sources respective content

model ontology.




FT1 Native query unavailable The Query cannot be translated to the native query format The system returns an error message stating that the Query cannot be

translated to the native query format.

FT2 Data Source unavailable The Data Source is not available for executing the query on. The system returns an error message stating that the Data Source is

not available for executing the query on.

FP7-287800 SALUS



#ID Flows

S1 B1 B2 B4 B5

S2 B1 B2 B3 B4 B5

6.1.5.5 Execution

#ID Scenario

Variables Expected

Result

Fault

TC1 S1 None B5

TC2 S1 FT1 FT1

TC3 S1 FT2 FT2

TC4 S2 None B5

6.1.5.6 Results


TC1 Valid query is passed to SALUS SIL, it is translated into SPARQL queries, patient

data is obtained from the ORBIS SPARQL endpoint and the retrieved data is

translated into the SALUS CIM.

SUCCESS

TC2 Invalid query is passed to SALUS SIL to obtain the patient data from the EHR

source. The system returns a bad request response.

SUCCESS

TC3 Valid query is passed to SALUS SIL, however, since the ORBIS system is not

available, the system returns an error indicating that ORBIS system is not

available.

SUCCESS

TC4 Valid query is passed to SALUS SIL, it is passed to TIL, the data is retrieved from

the TIL and the retrieved data is translated into the SALUS CIM.

SUCCESS

6.1.6 TC-4.4-2 (U) Convert to SALUS Semantic Data Representation

Description This use case retrieves a clinical data instance and creates a semantic model from it, represented

in the SALUS Core ontology. Parent - Scope Semantic Interoperability Layer Actor(s) -

FP7-287800 SALUS


Goal The clinical data is represented in the SALUS core ontology.

Trigger The system requests clinical data from a Data Source (EHR data source or lab data source or

Data warehouse). Called after the execution of “UC-4.4-1 (U) Retrieve a Clinical Data Instance”

use case Frequency Whenever a request arrives

Preconditions

The system is configured with mapping rules which map the source content model ontology to

the SALUS core ontology.

(Optional) The system is configured with terminology mappings from the reference

terminologies used by the data source to the SALUS core ontology. Minimal Post-Conditions N/A


The clinical data instance is transformed to the SALUS core ontology and is made available in

the system.


#ID Description

B1 The system retrieves the clinical data instances by invoking use case “UC-4.4-1 (U) Retrieve a Clinical Data

Instance”

B2 The system uses the mapping rules to align the clinical data instance with the SALUS core ontology.

B3 The semantic model generated in previous steps is available in the system.




FT1 Terminology mapping unavailable The terminology mapping from the local terminology system to the

SALUS core ontology is unavailable.

The system returns an error message stating that the mapping from

the local terminology system to the SALUS core ontology is

unavailable.

FP7-287800 SALUS



#ID Flows

S1 B1 B2 B4

6.1.6.5 Execution

#ID Scenario

Variables Expected

Result

Fault

TC1 S1 None B3

TC2 S1 FT1 FT1

6.1.6.6 Results


TC1 Resolve /pavo entity SUCCESS

TC2 Resolve /pavo entity for non-existent domain SUCCESS

6.1.7 TC-4.4-3 Export SALUS Semantic Data Representation

Description This use case transforms a clinical data instance, represented in the SALUS Core ontology, to

data represented in a Content Model Ontology. Parent - Scope Semantic Interoperability Layer Actor(s) - Goal The clinical data is represented in the desired content model ontology.

Trigger Can be called during the execution of use case “UC-4.4-6 Query a SALUS Semantic Data

Representation” Frequency Whenever a request arrives

Preconditions

The system is configured with mapping rules which map the SALUS core ontology to the desired

target content model ontology.

(Optional) The system is configured with terminology mappings from the SALUS core ontology

to the desired target terminologies. Minimal Post-Conditions N/A

Success Post- The clinical data instance is transformed to the desired content model ontology.

FP7-287800 SALUS


Conditions


#ID Description

B1 The input of this use case is a clinical data instance in the SALUS core ontology and the desired target

content model ontology.

B2 The system uses the mapping rules to align the clinical data instance with the desired target content model

ontology.

B3 (Optional) The system uses the terminology mapping rules to map clinical data instances to the desired

terminology code values.




FT1 Ontology mappings unavailable The ontology mapping from the SALUS core ontology to the target

content model ontology is unavailable. The system returns a meaningful error message indicating the

unavailable mapping.

FT2 Concept mapping unavailable Mapping for a concept from EHR data terminology system to the

desired terminology system is not available. The system returns a meaningful error message indicating the

unavailable mapping.

FP7-287800 SALUS



#ID Flows

S1 B1 B2

S2 B1 B2 B3

6.1.7.5 Execution

#ID Scenario

Variables Expected

Result

Fault

TC1 S1 None B2

TC2 S1 FT1 FT1

TC3 S2 None B3

TC4 S2 FT1 FT1

TC5 S2 FT2 FT2

6.1.7.6 Results


TC1 A patient data sample in SALUS CIM format is transformed into the target domain

ontology e.g. OMOP

SUCCESS

TC2 A patient data sample in SALUS CIM format is intended to be transformed into

the target domain ontology e.g. OMOP. However, the ontology mappings from

SALUS CIM to OMOP ontology cannot be found. System gives the necessary

error message.

Till now, this test has been executed via the command line by specifying the

ontology mappings manually.

SUCCESS

TC3 During the semantic data export operation, we have not considered the

terminology mapping yet. Currently we keep the local codes at this step, and when

needed by client applications terminology mapping is issued by Safety Analysis

Query Manager component.

N/A

FP7-287800 SALUS


6.1.8 TC-4.4-5 (U) Query in SIL

Description This use case facilitates querying one or more data source over SALUS Interoperability

Layer. Parent Scope Semantic Interoperability Layer

Actor(s) Any SALUS System Actor (such as Technical Interoperability Layer, ICSR Reporting

Tool) Goal Select, combine and project clinical data from one or more semantic models.

Trigger The system wants to execute a query on Data Sources through Semantic Interoperability

Layer Frequency Preconditions N/A Minimal Post-Conditions N/A Success Post-Conditions The system successfully returns the requested data.


#ID Description

B1 If the query is forwarded through “UC-5.2-1 Query SIL through TIL” use case, the query in native format is

mapped to SALUS semantic data representation format by invoking the “UC-4.4-7 Translate Query to

SALUS Semantic Data Representation” use case

B2 The request may be either for a population of patients or for a single patient 1. In case a population of data is requested, the eligibility query is executed by invoking the “UC-4.4-8 (U)

Select Eligible Patients” use case. 2. In case a single patient is requested, the request is handled by the “UC-4.4-11 (N) Select Single Patient”

use case.



FP7-287800 SALUS



#ID Flows

S1 B1 B2

S2 B2

6.1.8.5 Execution

#ID Scenario

Variables Expected

Result

Query Type Fault

TC1 S1 Eligibility None B2

TC2 S1 Patient id None B2

TC3 S2 Eligibility None B2

TC4 S2 Patient id None B2

Query Type

6.1.8.6 Results


TC1 Querying SIL through TIL is not needed till now, so it is not implemented yet. N/A

TC2 Querying SIL through TIL is not needed till now, so it is not implemented yet. N/A

TC3 Several tests executed for getting the patient the data through SIL directly an

eligibility query

SUCCESS

TC4 Several tests executed for getting the patient the data through SIL directly using a

patient id

SUCCESS

6.1.9 TC-4.4-6 (U) Query a SALUS Semantic Data Representation

Description This use case performs a semantic query on one or more SALUS semantic data representations. Parent UC-4.4-5 Query in SIL Scope Semantic Interoperability Layer Actor(s) Goal Select, combine and project clinical data from one or more semantic models.

FP7-287800 SALUS


Trigger The system wants to execute a semantic query on SALUS data representations Frequency Preconditions N/A Minimal Post-Conditions N/A Success Post-Conditions The system successfully returns the requested data.


#ID Description

B1 If the query results are not present in the system (as a result of a previous subscription or query), they are

retrieved from the Data Sources by invoking use case “UC-4.4-2 (U) Convert to SALUS Semantic Data

Representation” for each instance.

B2 The resulting semantic data representations are queried

B3 (Optional) Obtained data in the previous step is converted into SALUS core ontology model through the

“UC-4.4-2 (U) Convert to SALUS Semantic Data Representation” use case

6.1.9.2 Use-Case (A)lternative flows None..



FT1 Query failure The retrieval of the data from the data sources fails. The system returns an error message stating that retrieval of the data

from the data sources failed.


#ID Flows

S1 B1 B2 B3

6.1.9.5 Execution

#ID Scenario

Variables Expected

Result

Fault

TC1 S1 None B3

TC2 S1 FT1 FT1

FP7-287800 SALUS


6.1.9.6 Results


TC1 POST /generic-endpoint/sparql

default-graph-uri: entity-uri

query: select * where { ?s ?p ?o }

SUCCESS

TC2 POST /generic-endpoint/sparql

default-graph-uri: non-existant-entity-uri

query: select * where { ?s ?p ?o }

SUCCESS

6.1.10 TC-4.4-8 (U) Select Eligible Patients

Description This use case executes a semantic patient eligibilty criteria query represented in the SALUS core

ontology on the SIL, to retrieve eligible patients EHR data. Parent Scope Semantic Interoperability Layer Actor(s) Case Series Characterization Tool Goal Patients satisfying the eligibility criteria are identified. Trigger The Case Series Characterization tool queries the SIL Frequency

Preconditions

The system is configured with terminology mappings from the SALUS core ontology to the

desired target terminologies.

The system is configured with query templates, mapping eligibility criteria to semantic query

patterns, expressed in the SALUS core ontology. Minimal Post-Conditions N/A


The patients, which satisfy the eligibility criteria, are identified and their EHR data is returned to

the caller.

FP7-287800 SALUS



#ID Description

B1 The system uses the terminology mappings to map the coded query parameters to the code values, used by the

data source.

B2 The system uses the query templates to map the eligibility criteria to semantic query patterns, expressed in the

SALUS core ontology and using the coded values found in B1, The system determines the semantic models

which need to be queried.

B3 The system invokes UC-4.4-6 “Query a SALUS Semantic Data Representation” with the query generated in

B2, on the models found in B2.

B4 For each patient found in the result of B3, the patients EHR data are retrieved by invoking UC-4.4-2 “Convert

to SALUS Semantic Data Representation”.




FT1 Terminology mapping unavailable The mapping of a terminology code present in the query to the

terminology code system used by the data source is not available..

The system returns an error message stating that the mapping of a

terminology code to the terminology code system used by the data

source is not available.

FT2 Unsupported query The query template for mapping an eligibility criterion present in the

query is not available.

The system returns an error message stating that the eligibility

criteria could not be translated into a query pattern.

FP7-287800 SALUS



#ID Flows

S1 B1 B2 B3 B4

6.1.10.5 Execution

#ID Scenario

Variables Expected

Result

TC1 S1 none B3

TC2 S1 FT1 FT1

TC3 S1 FT2 FT2

6.1.10.6 Results


TC1 POST valid-eligibility-query to

/silds/ eligiblepatients

SUCCESS

TC2 POST valid-eligibility-query to

/silds/ eligiblepatients containing non-existant terminology code SUCCESS

TC3 POST eligibility-query containing a non-mapped criterion to

/silds/ eligiblepatients SUCCESS

6.1.11 TC-4.4-11 (N) Select Single Patient

Description This use case executes a query represented in the SALUS core ontology on the SIL, to retrieve

the clinical data of a single patient. Parent Scope Semantic Interoperability Layer Actor(s) Safety Analysis Tools Goal Returning the patient data based on the given patient identifier Trigger The Case Series Characterization tool queries the SIL Frequency

Preconditions The system is configured with terminology mappings from the SALUS core ontology to the

desired target terminologies.

The system is configured with query templates, mapping eligibility criteria to semantic query

FP7-287800 SALUS


patterns, expressed in the SALUS core ontology. Minimal Post-Conditions N/A


The patients, which satisfy the eligibility criteria, are identified and their EHR data is returned to

the caller.


#ID Description

B1 The patient id is represented with a query in SALUS core ontology format to be executed in the on the EHR

source

B2 The system queries the clinical data through the “UC-4.4-1 (U) Retrieve a Clinical Data Instance” with the

query generated in the previous step

B3 Retrieved patient data is transformed into the desired format through the “UC-4.4-2 (U) Convert to SALUS

Semantic Data Representation” use case


6.1.11.3 Fault Cases None.


#ID Flows

S1 B1 B2 B3

6.1.11.5 Execution

#ID Scenario

Variables Expected

Result

TC1 S1 none B3

6.1.11.6 Results


TC1 Patient data for given patient identifier is retrieved through the SIL. SUCCESS

FP7-287800 SALUS


6.2 EM2 - Functionality: Integration Tests

6.2.1 Integration Tests for SIL-DS LISPA

FP7-287800 SALUS



6.3.1 Unit Tests for EHR RDF Service

6.3.1.1 Surefire Report Summary tr.com.srdc.ontmalizer

6.3.2 Unit Tests for SIL-DS (TUD) Service

6.3.2.1 Surefire Report Summary com.agfa.aca.argus com.agfa.aca.argus is the service responsible for retrieving clinical data instances, using queries expressed in the data sources content model ontology and for creating a semantic model from it, represented in the SALUS Core ontology set.

FP7-287800 SALUS


FP7-287800 SALUS


6.3.2.2 Surefire Report Summary com.agfa.aca.pavo com.agfa.aca.pavo is responsible for generating entities, represented in the SALUS Core ontology set. It uses entity parameters to instantiate the content model query templates, used to query the datasource through com.agfa.aca.argus

FP7-287800 SALUS


6.3.2.3 Surefire Report Summary aca-salus-cscqc aca-salus-cscqc is responsible for translating patient eligibility criteria represented in Turtle, to SPARQL queries on the semantic data entities generarted by com.agfa.aca.pavo.

FP7-287800 SALUS


FP7-287800 SALUS


6.3.3 Unit Tests for Content Converter

6.3.3.1 Surefire Report Summary com.agfa.aca.eye

FP7-287800 SALUS


6.3.4 Unit Tests for Content Formatter

6.3.4.1 Surefire Report Summary eu.salusproject.sil.formatter

FP7-287800 SALUS


7 TECHNICAL INTEROPERABILITY LAYER

7.1 EM1 - Functionality: Functional Test Cases and EM4 - Reliability: Fault tolerance analysis

7.1.1 TC-5.2-2-2 Querying data source through TIDSQS

Description

This use case allows any SALUS system service to query the underlying EHR system for

existing data using SALUS semantic interoperability layer.

This TC is focusing on TIDSQS Parent

- Included sub-use cases NA

Extended sub-use cases NA

Scope Any scenario

FP7-287800 SALUS


Actor(s) Semantic Interoperability Layer (SIL), Query Interface (QI), SALUS service, Semantic

Interface, Electronic Health Record (EHR). Goal

Querying the EHR for existing data. Trigger

This use case is initiated by a SALUS system service which needs specific data. This could

be e.g. the ICSR reporting tool or the ADE Notification Tool. Frequency

Runs whenever a query is initiated from a SALUS service. Minimal Post conditions Empty query response is delivered to SALUS service querying the data

Success Post-Conditions Complete set of queried data is delivered after translation within the SIL


#ID Description

B1 EHR RDF service initiates a query through the Query Interface (QI) for retrieving new or changed data of a

specific patient

B2 TIQSDS extracts the payload carrying the medical data and initiates communication of IHE QED to the EHR

B3 TIDSQS sends the query payload by using IHE QED extension to the EHR system

B4 TIDSQS successfully retrieves query results from the EHR and sends the result back to EHR RDF service


#ID Description

A1 The EHR system is not available due to a network issue. Therefore the communication cannot be established

and query can be made. This will lead to an incomplete query and will be notified to the EHR RDF service

A2 The query template send to the EHR cannot be understand by the EHR system and leads to an exception

which will result in an incomplete query notified to the EHR RDF service


FP7-287800 SALUS



FT1 Incorrect payload The payload defined for the query at the SALUS core system can for

some reason not be understood by the EHR system. If the payload is

a valid message of IHE QED this will just lead to an empty response.

In case of a not valid payload an exception will be thrown.

FT2 Query issue in TIDSQS due to

network problems

The query to be sent through TIDSQS failed for some reason. This

can be for example a missing network connection. This will lead to

an exception within the JAVA code and therefore notify the sender

about the problem. This not expected to be the case for SALUS

pilots as this communication will be localhost here.


#ID Flows

S1 B1 B2 B3 B4

S2 B1 B2 A1

S3 B1 B2 B3 A2


#ID Scenario

Variables Expected

Result

Content Model Type Fault

TC1 S1 IHE QED standard

query

None B4

TC2 S1 IHE QED by EN13606

query

None B4

TC3 S2 IHE QED by invalid

query

FT2 A1

TC4 S3 IHE QED standard

query

FT1 A2

FP7-287800 SALUS


7.1.1.6 Results


TC1 1. An IHE QED standard query was send to the TIDSQS from ICSR tool

2. The query was checked by TIDSQS and send through IHE QED to the

remote site.

3. TIDSQS Data Repository receives the message and forwards it to the

LISPA connector

4. Results were captured from the database and put into a valid IHE QED

response message

5. The result was send back to TIDSQS Data Consumer

6. The data was handed over to ICSR tool

SUCCESS

TC2 EN 13606 Implementation not finished yet

N/A

TC3 1. ICSR tool and TIDSQS Data Consumer started, but TILDSQS not started.

2. A query was performed and an exception thrown

SUCCESS

TC4 1. ICSR tool sends a query which can not be answered by LISPA connector

as such information is not available (Lab results)

2. The query was checked by TIDSQS and send through IHE QED to the

remote site.

3. TIDSQS Data Repository receives the message and forwards it to the

LISPA connector

4. An empty response message is created

5. The empty result was send back

6. The empty message was send to the ICSR tool

SUCCESS

FP7-287800 SALUS



FP7-287800 SALUS



7.3.1 Unit Tests for LISPA Connector

FP7-287800 SALUS


FP7-287800 SALUS


7.3.2 Unit Tests for QED-EXT Module

8 ICSR REPORTING TOOL

8.1 EM1 - Functionality: Functional Test Cases and EM4 - Reliability: Fault tolerance analysis

FP7-287800 SALUS


8.1.1 TC-5.3-1 Reporting an ADE Using ICSR Reporting Tool

Description This use case is for reporting a suspected ADE to the local/international regulatory authorities by

reusing the information available in EHR Data Source and in a pseudonymized way. Parent

- Included sub-use cases UC-.4.4-5 Query in SIL


Scope

ICSR Reporting Tool (IRT) Actor(s)

Health Professional (HP), EHR Data Source, PharmacovigilanceCenter (Local Regulatory

Authority, International Regulatory Authority), ADE Notification Tool (ANT), SIL, de-

identification/pseudonymisation service. Goal

Reporting of ADEs to the local regulatory authority or an international Pharmacovigilance center

after filling the ICSR form by reusing the available information in EHR System. Trigger

This use case is initiated (1) by the ADE Notification Tool after (a) an ADE is detected

automatically and (b) the HP validates the tool’s proposition and (c) decides to report the

suspected ADE now (alternatively he can decide to report the suspected ADE later). Or it is

initiated (2) by the HP himself if (a) he detects an ADE on the basis of his own expertise and (b)

decides to report it Frequency

Runs whenever an ADE is detected to be reported to the Local/International Regulatory

Authority. Minimal Post conditions 1. IRT presents the status of ICSR Reporting process.

2. If the ICSR reporting procedure is interrupted (because of a computer crash for example), a

backup process makes it possible to continue the reporting where it was interrupted.

3. ICSR report is Pseudonymized through a reversible pseudonymization algorithm. Success Post-Conditions ICSR in E2B XML Format has successfully been sent electronically to the Pharmacovigilance

Center, or has been printed to be signed and sent using paper mail.

FP7-287800 SALUS



#ID Description

B1 IRT is triggered by the ADE Notification Tool (through the “UC-6.1-1 ADE Detection” use case) or by the HP

himself.

B2 IRT queries the EHR System through SIL-DS service calling to collect medical summary of the patient with

the given PID by invoking the “UC-4.4-5 Query in SIL” use case.

As a result, EHR is queried and the collected medical summary is exported to the ICSR (through the included

use cases, “UC-4.4-3 Export SALUS Semantic Data Representation” and “UC-4.4-4 Export a Clinical Data

Instance”).

B3 IRT presents the pre-filled ICSR as a form to HP on the dedicated GUI.

B4 HP using the provided interface fills manually the missing fields. Once mandatory fields are completed, the

“send the report” and “print the report” buttons are activated on the GUI.

B5 HP selects the “send the report” function.

B6 IRT generates the report in the mandated format, sends it to de-identification/pseudonymisation service, which

transmits it after being de-identified/pseudonymized to the Pharmacovigilance Center.

B7 The Pharmacovigilance Center sends back to the de-identification/pseudonymisation service a message

confirming that the ICSR file has been received. The message is sent by de-identification/pseudonymisation

service to IRT which displays it to the user.

B8 The ICSR file gets saved in a local repository and becomes accessible for later consultation and update by the

HP if necessary. (The IRT invokes the ICSR Local storage service to save and load sent ICSRs.)


#ID Description

A1 For some reason, IRT is unable to query the EHR System to prepopulate the reporting form (step B2 of the

basic flow). The IRT displays an error message.

A2 The HP cancels the reporting procedure after the pre-filled ICSR form has been displayed on the GUI (step

B3 of the basic flow).

A3 The HP prints the completed ICSR form to send a paper copy using mail. He does not use the GUI function

dedicated to send electronically the ICSR (step B5 of the basic flow).


FP7-287800 SALUS



FT1 Failure to access EHR data For some reason, IRT is unable to query the EHR System through

SIL service calling to prepopulate the reporting form (step B2 of the

basic flow). The IRT displays an error message.

FT2 No confirmation message For some reason, the Pharmacovigilance Center does not send a

confirmation message to the IRT (step B7 of the basic flow).


#ID Flows

S1 B1 B2 B3 B4 B5 B6 B7 B8

S1’ B1 B2 B3 B4 B5 B6 X B8

S2 B1 B2 A1

S3 B1 B2 B3 A2

S4 B1 B2 B3 B4 A3


#ID Scenario

Variables Expected

Result EHR data accessible

Local Triplestore accessible

HP cancels reporting

Fault

TC1 S1 Yes Yes No None B8

TC2 S1’ Yes Yes No FT2 B6

TC3 S2 No Yes No FT1 A1

TC4 S3 Yes Yes Yes None A2

TC5 S4 Yes Yes No None A3

8.1.1.6 Results


TC1 Test made manually based on a patient summary n3 file with fake data.

Integration with SIL-DS was tested separately.

B6 and B7 cannot be tested because integration of IRT and de-

identification/pseudonymisation service is not achieved yet.

SUCCESS for

B1-B5 and

B8

TC2 S1’ cannot be tested because integration of IRT and de- N/A

FP7-287800 SALUS


identification/pseudonymisation service is not achieved yet, so no

confirmation message can be received (for FT2). TC3

S2 cannot be tested because the mechanism to display an error message

when IRT is unable to query the EHR System through SIL-DS to

prepopulate the reporting form (FT1) is not implemented yet.

N/A

TC4 S3 cannot be tested because the mechanism to cancel an ICSR is not

implemented yet. N/A

TC5 S4 cannot be tested because the mechanism to print an ICSR (A3) is not

implemented yet. N/A

8.1.2 TC-5.3-2 Recording an ADE Notified by the ADE Notification Tool to be Reported Later

Remark: The use case UC-5.3-2 (Recording an ADE Notified by the ADE Notification Tool to be Reported Later) has been substantially changed compare to previous descriptions, because the functionality enabling the user to postpone the reporting of an identified ADE is now supported by the SAVE option of IRT main GUI.

Description This use case is for postponing the completing and sending of an ICSR using the SAVE

functionality of IRT main GUI. Parent

- Included sub-use cases UC-5.3-1


Scope

ICSR Reporting Tool (IRT) Actor(s)

Health Professional (HP), ADE Notification Tool (ANT). Goal

Saving a partially filled ICSR form to be finalized and sent later. Trigger

This use case is initiated by the HP when he decides to finalize and send later an ICSR. Frequency

Runs whenever an ICSR is created and the HP decides to finalize and send it later. Minimal Post conditions IRT ICSR storage component informs the HP about the result of the ICSR storage

operation. Success Post-Conditions A partially filled ICSR E2B compliant xml file is created and saved in a local repository

and is accessible by IRT for later finalization.

FP7-287800 SALUS



#ID Description

B1 IRT is triggered by the ADE Notification Tool (through the “UC-6.1-1 ADE Detection” use case) or by the

HP himself.

B2 IRT queries the EHR System through SIL-DS service calling to collect medical summary of the patient with

the given PID by invoking the “UC-4.4-5 Query in SIL” use case.

As a result, EHR is queried and the collected medical summary is exported to the ICSR (through the included

use cases, “UC-4.4-3 Export SALUS Semantic Data Representation” and “UC-4.4-4 Export a Clinical Data

Instance”).

B3 IRT presents the pre-filled ICSR as a form to HP on the dedicated GUI. The user selects the SAVE option on

the IRT GUI.

B4 As a result, the prepopulated ICSR gets saved in a local repository (the IRT invokes the ICSR Local storage

service to save the ICSR) and becomes accessible by IRT for later checking, finalization and reporting.


#ID Description

A1 The IRT cannot save the ICSR because the Local storage service is temporarily unavailable (step B4 of the

basic flow). The IRT should either retry later to communicate with the Local storage service or display an

error message.



#ID Flows

S1 B1 B2 B3 B4

S2 B1 B2 B3 A1


FT1 Local storage service unavailable The IRT cannot save the ICSR since the Local storage service is

temporarily unavailable. The IRT should either retry later to

communicate with the Local storage service or display an error

message.

FP7-287800 SALUS



#ID Scenario

Variables Expected




Fault


TC2 S2 Yes Yes No FT1 A1

8.1.2.6 Results


TC1 Test made manually based on a patient summary n3 file with fake data. Integration

with SIL-DS was tested separately.

SUCCESS

TC2 S5 cannot be tested because the mechanism to check if the Local storage service is

available (step B8 of the basic flow) is not implemented yet. N/A

8.1.3 TC-5.3-3 Accessing Previously Sent and Waiting to be Reported ICSRs

Description This use case is for accessing previously sent and waiting to be reported ICSRs. ICSR xml files

are recorded in the local system after been sent to the regulatory authorities. The HP can access

them through a dedicated interface of the IRT. A function “See the previously sent and waiting to

be reported ICSRs” is available. Parent

- Included sub-use cases -


Scope ICSR Reporting Tool (IRT)

Actor(s) Health Professional (HP).

Goal Allowing the HP to access previously sent and waiting to be reported ICSRs through a dedicated

interface of the IRT. Trigger

This use case is initiated by the HP when activating the IRT function “See the previously sent

and waiting to be reported ICSRs”. Frequency

Runs whenever the HP decides to access previously sent and waiting to be reported ICSRs. Minimal Post conditions If access to previously sent and waiting to be reported ICSRs is not possible, a message is

displayed to the HP explaining that the access operation has encountered a problem. Success Post-Conditions The list of previously sent and waiting to be reported ICSR files with their edition date and

references of related patient (name, PID) is displayed to the HP through a dedicated interface.

The content of the files is viewable upon request.

FP7-287800 SALUS



#ID Description

B1 The HP selects the option “Form manager” available on the IRT GUI.

B2 IRT browses the database where the previously sent and waiting to be reported ICSRs files are described.

B3 A table listing previously sent and waiting to be reported ICSRs is displayed on the IRT GUI (with the ICSR

ID, edition date, references of the related patient: name and PID, status of the report: completed or waiting to

be reported, etc.).




#ID Flows

S1 B1 B2 B3


#ID Scenario

Variables Expected




Fault


8.1.3.6 Results


TC1 Test made manually based on several ICSR forms with different status (sent,

waiting to be completed, nullified).

SUCCESS

8.1.4 TC-5.3-4 Updating and Sending an ICSR Reported in a Previous Session

Description This use case is for updating a previously sent ICSR. If the HP has additional information

on an ADE described in an already reported ICSR, or has forgotten to mention something

in this ICSR, he can decide to update the ICSR. A dedicated functionality “Update the

ICSR” is available in the IRT interface displaying previously sent and waiting to be

reported ICSRs. Parent

-

FP7-287800 SALUS


Included sub-use cases UC-5.3-3 Accessing previously sent and waiting to be reported ICSRs


Scope ICSR Reporting Tool

Actor(s) Health Professional (HP), Pharmacovigilance Center (Local Regulatory Authority,

International Regulatory Authority), SIL Goal

Allowing the HP to update a previously reported ICSR through a dedicated interface of the

IRT and send the updated ICSR to the local regulatory authority or an international

Pharmacovigilance center. Trigger

This use case is initiated by the HP when activating the function “Update the ICSR” for a

previously reported ICSR displayed in the IRT interface. Frequency

Runs whenever the HP decides to update a previously reported ICSR file needing to be

modified or completed. Minimal Post conditions 1. The status of ICSR updating procedure is presented on IRT.

2. If the ICSR updating procedure is interrupted (because of a computer crash for

example), a backup process makes it possible to continue the updating where it was

interrupted.

3. ICSR report is Pseudonymized through a reversible pseudonymization algorithm. Success Post-Conditions 1. The updated ICSR in E2B XML Format has successfully been sent to the Local

Regulatory Authority and/or International Regulatory Body.

2. A versioning system implemented in the IRT allows distinguishing the first ICSR

version sent to regulatory authorities with the new one, and relating the two files to the

same ADE case.

FP7-287800 SALUS



#ID Description

B1 The HP selects the function “Update the ICSR” available on the form manager (by invoking the “UC-5.3-3

Accessing Previously Sent and Waiting to be Reported ICSRs” use case).

B2 IRT opens the previously sent ICSR file stored in the local repository and updates automatically the E2B data

elements used to indicate that a new ICSR is realized for the same case.

B3 IRT presents the pre-filled ICSR as a form to HP on the dedicated GUI.

B4 HP using the provided interface updates the fields needing to be modified or completed. HP selects the “send

the report” function.

B5 IRT generates the report in the mandated format, sends it to de-identification/pseudonymisation service,

which transmits it after being de-identified/pseudonymized to the Pharmacovigilance Center.

B6 The Pharmacovigilance Center sends back to the de-identification/pseudonymisation service a message

confirming that the ICSR file has been received. The message is sent by de-identification/pseudonymisation

service to IRT which displays it to the user.

B7 The ICSR file gets saved in a local repository under a name indicating an updating process and becomes

accessible for later consultation and further update by the HP if necessary. (The IRT invokes the ICSR Local

storage service to save and load sent ICSRs.)

8.1.4.2 Use-Case (A)lternative flows, Fault Cases, (S)cenarios to be tested, Execution (Functionality and Fault Tolerance Tests)

Except for B1 and B2, for which no (A)lternative flows and Fault Cases are requested, UC-5.3-4 and UC-5.3-1 have the same flow of actions. The testing procedure applicable for TC-5.3-4 is thus already assumed for the biggest part by TC-5.3-1. Only the basic flow of action must be tested.

8.1.4.3 Results


TC1 Test made manually based on a fake ICSR form that was not previously sent to

regulatory authorities, but whose was registered as “submitted” in form manager.

SUCCESS

8.2 EM2 - Functionality: Integration Tests None.

8.3 EM3 - Functionality: Unit Tests None.

FP7-287800 SALUS


9 ADE NOTIFICATION TOOL

9.1 EM1 - Functionality: Functional Test Cases and EM4-Reliability: Fault tolerance analysis

9.1.1 TC-6.1-1 ADE Detection

Description This use case is for enabling the ADE detection.

Parent -

Included sub-use cases ADE detection rule management


Scope ADE Notification Tool

Actor(s) ICSR Reporting Tool, Clinical Information System, Physician A, SALUS Semantic

Interoperability Layer (SIL) Goal

An ADE is detected, either manually by the physician or semi-automatically by intelligent

ADE detection algorithms. Trigger

This use case is initiated by new incoming data in an EHR-Session that is relevant for the

ADE Detection (for example: new medical event, changes in medication or new lab

results) or manually by the Physician A who manually detected an event as an ADE. Frequency

Whenever new relevant data is added to an EHR or the Physician A manually detects an

event as an ADE. Minimal Post conditions -

Success Post conditions 1. An ADE is detected and in the case of an unknown ADR it is forwarded to the ICSR

Reporting Tool.

2. A new ADE detection rule was derived or a proposal (for example in form of more

precise values used within the detection rules) for improving the ADE detection is visible

for the physicians.

FP7-287800 SALUS



#ID Description

B1 An ADE is detected (semi)-automatically or manually by a HP.

B2 Newly found ADRs are forwarded to the ICSR Tool.

B3 The ADE Detection Rules are updated.


#ID Description

A1 No ADE is detected.

A2 A new proposal for adding an ADE Detection Rule is denied by other HPs.

A3 The tool cannot connect to the rules repository for using/manipulating the ADE Detection Rules since the

repository is temporarily unavailable, the system affected should either retry later to communicate with the

repository or send back an error message.

A4 The tool cannot connect to the ICSR Tool. The affected system should either retry later to communicate with

the repository or send back an error message.



FT1 Inaccessible rules repository The ADE Notification Tool is unable to access the ADE Detection

Rules repository, informs the user about the problem and presents

guidance to solve the problem.

FT2 No connection-possibility to the

ICSR Tool

The ADE Notification Tool is unable to send an ADE to the ICSR

Tool, informs the user about the problem and presents guidance to

solve the problem.

FP7-287800 SALUS



#ID Flows

S1 B1 B2 B3

S2 B1 B2 A2

S3 A1

S4 B1 B3

S5 B1 B2 A3

S6 B1 B3 A4


#ID Scenario

Variables Expected

Result

Detection type Fault

TC1 S1 manual None B3

TC2 S1 (semi)-

automatical

None B3

TC3 S2 Manual/semi-

automatical

None A2

TC4 S2 Manual/semi-

automatical

FT2 A4

TC5 S3 - None A1

TC6 S4 Manual/semi-

automatical

None B3

TC7 S4 Manual/semi-

automatical

FT1 A3

TC8 S5 Manual/semi-

automatical

FT1 A3

TC9 S5 Manual/semi-

automatical

FT2 A4

TC10 S6 Manual/semi- FT2 A3

FP7-287800 SALUS


automatical

9.1.1.6 Results


TC1 A currently unknown ADE is created manually and prepared for the transport to

the IRT, The transport to a REST-Service is tested locally and ready for

integration.

SUCCESS

TC2 A patient with a currently unknown ADE is created manually and put into the

detetion algorithm. The detection algorithm detects that ADE automatically and

prepares the transport to the IRT. The transport to a REST-Service is tested locally

and ready for integration.

SUCCESS

TC3 Pleaes note that the Rule Management Use Case is not in scope of the 1st

prototype, so this TC at this stage of the project can be tested the same way as

TC1.

N/A

TC4 Pleaes note that the Rule Management Use Case is not in scope of the 1st

prototype, so this TC at this stage of the project can be tested the same way as

TC2.

N/A

TC5 A patient with some diagnoses and medications is created manually and put into

the detetion algorithm. The detection algorithm doesn’t detect anything atypical.

No alarm and no notification is produced.

SUCCESS

TC6 A patient with an ADE is created manually and put into the detection algorithm.

After the ADE was found, the Detection Rules are updated, The ADE is visible in

the GUI and ready to be examined by the physician in detail later.

SUCCESS


After the ADE was found, the detection rule repository, which needs to be

updated, is not accessable because it is not present. The user gets informed and

guidance is presented.

SUCCESS



prepares the transport to the IRT. The transport to a REST-Service is tested locally

and ready for integration. Afterwards, the detection rules need to be updated, but

the repository is not accessible, because it is not present. The user gets informed

and guidance is presented.

SUCCESS



prepares the transport to the IRT. If the IRT cannot be accessed due to internet

connection problems or something like this, the user gets informed and guidance is

presented.

SUCCESS

FP7-287800 SALUS



The detection algorithm detects that ADE. The ADE will not be reported.

Afterwards, the detection rules need to be updated, but the repository is not

accessible, because it is not present. The user gets informed and guidance is

presented.

SUCCESS

9.1.2 TC-6.1-2 Semi-automatic ADE Detection

Description This use case is for supporting the physician to detect ADEs and to propose new ADE

detection rules based on statistical methods and intelligent/adaptive detection algorithms. Parent

ADE detection Included sub-use cases ADE detection rule management


Scope ADE Notification Tool

Actor(s) ICSR Reporting Tool, Clinical Information System, Physician A, SALUS Semantic

Interoperability Layer (SIL), ADE Notification Tool Goal

An ADE is detected automatically, confirmed/rejected by the physician and in the case of

an unknown ADR forwarded to the ADE Reporting Tool. Finally, if necessary, the ADE

detection rules are updated. Trigger

This use case is initiated by new incoming data in an EHR-Session that is relevant for the

ADE Detection (for example: new medical event, changes in medication or new lab

results). Frequency

Whenever new relevant data is added to an EHR. Minimal Post conditions -

Success Post conditions 1. An ADE is found and in the case of an unknown ADR it is forwarded to the ICSR

Reporting Tool.

2. A proposal (for example in form of more precise values used within the detection rules)

for improving the ADE detection is visible for the physicians.

FP7-287800 SALUS



#ID Description

B1 Incoming EHR data is checked via the SALUS SIL, whether the new data is relevant for ADE Detection.

B2 New data is checked for known/unknown patterns.

B3 An alert is produced to warn the HP about an suspected ADE to be confirmed/rejected.

B4 The answer from the HP is stored and a new proposal for an ADE Detection Rule is visible for other HPs.


#ID Description

A1 New EHR data is not relevant for ADE Detection.

A2 No known/unknown pattern was found.

A3 The statistical methods cannot be applied to check the data for unknown patterns due to a missing EHR

database. A discreet hint is displayed in the ADE Notification Tool Main GUI



FT1 EHR Database is not available The ADE Notification Tool needs to calculate some measures on the

background database to detect unknown patterns. In the case that the

EHR Database is not available, the detection of unknown patterns is

not possible.

FP7-287800 SALUS



#ID Flows

S1 B1 B2 B3 B4

S2 B1 B2 A3

S3 B1 A1

S4 B1 B2 A2


#ID Scenario

Variables Expected

Result

Physicians’ decision for

suspected ADE

Fault

TC1 S1 Confirmation/rejection None B4

TC2 S2 - FT1 A3

TC3 S3 - None A1

TC4 S4 - None A2

9.1.2.6 Results


TC1 A diagnose is manually created and added to a patient object, that contains

medication information. The detection algorithm detects an ADE and produces an

alert alert in the GUI. Now the physician can decide wheather this is an ADE or

not. For test cases, we propose he confirms that event being an ADE. Please note

that the Rule Management Use Case is not in scope of the 1st prototype, so this TC

is finished before the aspirated result “B4”.

SUCCESS

TC2 A diagnose is manually created and added to a patient object. The connection to

the EHR system is turned off. A hint is visible in the main GUI indicating that

there is no connection to the clinical information system.

SUCCESS

TC3 Address information of a patient is put into the detection algorithm. Address

information is not relevant for ADE detection. Nothing is analysed and no

alert/notification is produced.

SUCCESS

TC4 A diagnose is manually created and added to a patient object, that contains

medication information. The detection algorithm doesn’t detect anything. No

alert/notification is produced.

SUCCESS

FP7-287800 SALUS




FP7-287800 SALUS


10 DE-IDENTIFICATION AND PSEUDONYMIZATION

TOOLSET

10.1 EM1 - Functionality: Functional Test Cases and EM4- Reliability: Fault tolerance analysis

10.1.1 TC-5.4-1.1N De-identify Patient Summary

10.1.1.1 Use Case Template

Description

This use case is for removing key personal identification information from a clinical data

instance (patient summary) which is in the format of SALUS Common Information Model

(CIM). SALUS CIM is an ontology, therefore the patient summary conforming to it is an

instance of SALUS CIM ontology. The identification information to be

removed/generalized/pseudoymized can belong to both the patients and also healthcare

professionals. De-identification is realized by several De-identification Methods based on a

configuration that describes how to de-identify which fields in the patient summary.

Pseudonymization is one of the de-identification methods. If the configuration indicates that a

field in the patient summary should be pseudonymized, then the related de-identification method

is invoked automatically to handle the pseudonymization. Parent NA Scope De-identification Service Actor(s) De-identification Service, SALUS SIL, SALUS PHT

Goal Removing/Generalizing/Pseudonymizing key personal identification information from a clinical

data instance

Trigger Request for patient summary from an EHR System / Data Warehouse (through SALUS SIL) for

clinical research purposes. Frequency For each clinical data transfer/feed through SALUS SIL.

Preconditions

1. The clinical data instance to be de-identified is compiled in SALUS Common Information

Model that is recognized by the SALUS SIL, and hence the De-identification Service.

2. The de-identification algorithms/methods are already implemented in the De-identification

Service.

3. De-identification Service and EHR System / Data Warehouse reside in the same healthcare

settings for security and privacy related reasons (care zone).

4. EHR System / Data Warehouse established a secure communication with the De-

identification Service. Minimal Post-Conditions NA


De-identification service sends the de-identified patient summary directly to the date requester,

Patient History Tool.

FP7-287800 SALUS



#ID Description

B1 EHR System / Data Warehouse sends the patient summary to the De-identification Service through SIL. The

patient summary is an instance of the SALUS CIM ontology.

B2 De-identification service receives the patient summary. If no configuration is passed together with the patient

summary, then the default configuration for the de-identification of the patient summaries within SALUS is

applied.

B3 Based on the configuration, the de-identification methods are applied on the patient summary.

B4 De-identification Service sends the de-identified patient summary directly to the Patient History Tool

according to the Pommerenning Approach that SALUS Security-Privacy related tools have been designed

upon.


#ID Description

A1 De-identification service cannot parse the submitted patient summary. It is not in the correct format of

SALUS CIM instance.

A2 De-identification service cannot send the patient summary to the Patient History Tool. PHT is informed about

the problem and an indicating log is created in the log base of SALUS components.



FT1 Unrecognizable format De-identification Service does not recognize the format of the patient

summary. De-identification Service informs PHT about the problem.

FT2 Fail to send back De-identification Service cannot send the de-identified patient

summary back to the requester – Patient History Tool.

FP7-287800 SALUS



#ID Flows

S1 B1 B2 B3 B4

S2 B1 B2 A1

S3 B1 B2 B3 A2

10.1.1.6 Execution

#ID Scenario

Variables Expected

Result

Fault

TC1 S1 None B4

TC2 S2 FT1 A1

TC3 S3 FT2 A2

10.1.1.7 Results


TC1 A sample patient summary conforming to the SALUS CIM has been submitted to

the De-identification service upon the request coming from PHT. De-identified

patient summary has been presented in the PHT.

SUCCESS

TC2 Any non-conformant patient summary does not lead to errors in the execution

because it is an ontological instance. If the requested data cannot be found in the

patient summary, simply it cannot be presented in PHT.

N/A

TC3 Call-back URL of PHT is closed after data request. Since no data arrives in PHT,

no patient data can be presented.

SUCCESS

10.1.2 TC-5.4-1.2N De-identify E2B Individual Case Safety Report


Description

This use case is for removing key personal identification information from an E2B

Individual Case Safety Report (ICSR). Such identification information to be removed can

belong to both the patients and also healthcare professionals. De-identification is realized

by several De-identification Methods based on a configuration that describes how to de-

identify which fields in the ICSR. Pseudonymization is one of the de-identification

FP7-287800 SALUS


methods. If the configuration indicates that a field in the ICSR should be pseudonymized,

then the related de-identification method is invoked automatically to handle the

pseudonymization.

De-identification of E2B ICSRs is persistent within the De-identification Services. That is,

a researcher from research zone, can later access to the details of the patients reported with

ICSRs through the Patient History Tool. Pseudnoyms and corresponding patient identifiers

are persisted within the care zone for later re-identification. Parent NA Scope NA Actor(s) De-identification Service, IRT

Goal Removing/Generalizing/Pseudonymizing key personal identification information from an

ICSR. Trigger A request to report an Adverse Event through ICSR Reporting Tool (IRT). Frequency For each Adverse Event reporting through IRT.

Preconditions

1. The ICSR to be de-identified is compiled in E2B format that is recognized by the De-

identification Service.

2. The de-identification algorithms are already implemented in the De-identification

Service.

3. De-identification Service and IRT reside in the same healthcare settings for security

and privacy related reasons (care zone).

4. IRT established a secure communication with the De-identification Service.

Minimal Post-Conditions

De-identification Service informs IRT about the result of the de-identification operation.


ICSR receivers which are indicated inside the ICSRs successfully receives the de-

identified E2B reports from the De-identification Service.

FP7-287800 SALUS



#ID Description

B1 IRT sends the E2B ICSR to the De-identification Service.

B2 De-identification service receives the E2B ICSR. If no configuration is passed together with the patient

summary, then the default configuration for the de-identification of the E2B based ICSRs within SALUS is

applied.

B3 Based on the configuration, the de-identification methods are applied on the ICSR.

B4 De-identification Service sends the de-identified ICSR directly to its recipients according to the

Pommerenning Approach that SALUS Security-Privacy related tools have been designed upon. This is an e-

mail message.


#ID Description

A1 De-identification Service cannot parse the submitted E2B ICSR.

A2 De-identification service cannot send the E2B report to the recipient e-mail addresses. IRT is informed about

the problem and an indicating log is created in the log base of SALUS components.



FT1 Unrecognizable format De-identification Service does not recognize the format of the patient

summary. De-identification Service informs IRT about the problem.

FT2 E-mail failure De-identification Service cannot send the reports to the e-mail

address(es) indicated inside the ICSRs.

FP7-287800 SALUS



#ID Flows

S1 B1 B2 B3 B4

S2 B1 A1

S3 B1 B2 B3 A2

10.1.2.6 Execution

#ID Scenario

Variables Expected

Result

Fault

TC1 S1 None B4

TC2 S2 FT1 A1

TC3 S3 FT2 A2

10.1.2.7 Results


TC1 A sample ICSR instance conforming to the E2B model has been submitted to the

De-identification service by manually invoking the service. De-identified patient

summary has been sent to the e-mail address given in the sample ICSR.

SUCCESS

TC2 A non-conformant ICSR is created by deleting a mandatory field in the report.

Since XSD validation failed, error messages are returned by the De-identification

service.

SUCCESS

TC3 A non-existent e-mail address is given in the sample ICSR, De-identification

performed sending the e-mail; however since the error message is received later

from the SMTP server, de-identification service could not return the error code

synchronously.

N/A

10.1.3 TC-5.4-3 Re-identify a Pseudonym


Description This use case is for re-identifying a pseudonym (PSN); i.e. replacing the PSN corresponding

Patient Identifier (PID) for a PSN in a query/notification about a specific patient. In some cases,

it is necessary to do re-identification, for instance for tracing case reports to their corresponding

FP7-287800 SALUS


patient records for providing detailed information on extended parts of the underlying medical

history of the patient. In those ICSRs, there are E2B specific report numbers which are specific to

individual case reports. Apart from that, there may be a request from the research zone to see the

summaries of the patients after doing an analysis (i.e. Case Series Characterization). This can be

done by requesting the patient summaries by providing the PSNs sent with the research analyses

results. Parent - Scope Security and Privacy Services Actor(s) E2B Pseudonymization Service, PHT Goal Retrieving the corresponding PID for a PSN

Trigger Request by the Safety Analyst in the regulatory authority for tracing a case report to its

corresponding patient records for getting more detailed information or presenting a notification

about a specific patient back to Data Source.

Frequency Whenever clinical review of ADEs is required/or notification about a specific patient back to

Data Source needs to be sent.

Preconditions

1. The pseudonym that is available within a query/notification is previously generated by the

same Pseudonymization Service within the UC-5.4-1.1N De-identify patient summary or in

UC-5.4-1.2N De-identify E2B Individual Case Safety Report.

2. The reversible encryption methods for re-identification are already implemented in the

Pseudonymization Service.

3. Data Requester/Data Target established a secure communication with the Pseudonymization

Service.

4. Pseudonymization Service implements an effective sender authentication and authorization

that prevents a trial encryption attack.

Minimal Post-Conditions

Pseudonymization Service informs the Data Requester/Data Target about the result of the re-

identification operation. Success Post-Conditions Pseudonymization Service successfully creates the PID corresponding to the PSN.

FP7-287800 SALUS



#ID Description

B1 Data Requester/Data Target sends the query/notification about a specific patient data including the

pseudonym (PSN) to the Pseudonymization Service.

B2 Pseudonymization Service looks for the corresponding Patient Identifier (PID) for the PSN and returns the

PID.


#ID Description

A1 Pseudonymization Service is not able to find the corresponding PID for the provided PSN. Pseudonymization

Service informs the Data Requester/Data Target about the problem with a report (e.g. no corresponding PID

exists), and requests it to retry the operation.



FT1 Unable to find PID Pseudonymization Service is not able to generate the corresponding

PID for the provided PSN. Pseudonymization Service informs the

Data Requester/Data Target about the problem with a report (e.g. no

corresponding PID exists), and requests it to retry the operation.


#ID Flows

S1 B1 B2

S2 B1 A1

10.1.3.6 Execution

#ID Scenario

Variables Expected

Result

Fault

TC1 S1 None B2

TC2 S2 FT1 A1

FP7-287800 SALUS


10.1.3.7 Results


TC1 A valid PSN is provided to the service and the corresponding PID is returned. SUCCESS

TC2 An invalid PSN is provided and NULL is returned by the service. SUCCESS


FP7-287800 SALUS



11 CASE SERIES CHARACTERIZATION TOOL

11.1 EM1 - Functionality: Functional Test Cases and EM4-Reliability: Fault tolerance analysis

11.1.1 TC-6.2-1 Case Series Characterization Analysis on Population Data

Description This use case consists of querying the underlying EHR sources with eligibility criteria for

background and foreground populations; executing statistical analysis on top of the retrieved

FP7-287800 SALUS


patient population according to the configurations and risk factors defined by the safety

analyst. Parent

-

Included sub-use cases UC-4.4-5 Query in SIL


Scope

Case Series Characterization Tool Actor(s)

Safety Analyst, Data Source, Technical Interoperability Layer (TIL), Semantic

Interoperability Layer (SIL), Semantic Services Goal

Case series characterization analysis on top of the patient data to be obtained from the EHR

sources connected to the SALUS system Trigger

This use case is initiated by the safety analyst in the Pharmacovigilance Center. Frequency

Runs whenever the safety analyst wants to conduct a case series characterization analysis

based on population data. Minimal post conditions Case Series Characterization Tool runs without an error

Success post conditions Case Series Characterization Tool presents the results to the safety analyst in compatible with

the demands of the safety analyst.

FP7-287800 SALUS



#ID Description

B1 Safety analyst defines background and foreground queries; risk factors and statistical configurations

B2 Case Series Characterization Tool delegates the queries to Safety Analysis Query Manager to query the Data

Sources through the “UC-4.4-5 Query in SIL” use case

B3 Result data set from the Data Source(s) is returned to the Safety Analysis Query Manager through SIL.

B4 Additional means provided by the Safety Analysis Query Manager characterize the cases and contrast them to

a background population on top of the retrieved data sets results and results are delegated to the Case Series

Characterization Tool

B5 Case Series Characterization Tool presents the results to the safety analyst.


#ID Description

A1 Safety Analysis Tool is unable to access the underlying Data Source(s). Case Series Characterization Tool

informs the Safety Analyst about the problem and presents guidance to solve it.

A2 Safety Analysis Query Manager is unable to apply the statistical analysis rules on top of the patient data

retrieved from the underlying Data Source(s).

A3 Empty result is returned by the Data Source(s), since there were no records matching the queries. Case Series

Characterization Tool informs the safety analyst about this fact.



FT1 Inaccessible Data Source(s) Case Series Characterization Tool is unable to access the underlying

Data Source(s). Safety Analysis Tool informs the Safety Analyst

about the problem and presents guidance to solve it.

FT2 Analysis process is unable to

conclude successfully

The means provided by the Safety Analysis Query Manager is not

able to present the results of the analysis process run on top of

retrieved data sets, to the safety analyst. Case Series Characterization

Tool informs the safety analyst about the problem and presents

guidance to solve it.

FP7-287800 SALUS



#ID Flows

S1 B1 B2 B3 B4 B5

S2 B1 B2 A1

S3 B1 B2 B3 A2

S4 B1 B2 B3 B4 A3


#ID Scenario

Variables Expected

Result

Content Model Type Fault

TC1 S1 DS: Semantic Aware None B5

TC2 S1 DS: Not Semantic

Aware

None B5

TC3 S1 DS: Both None B5

TC4 S2 DS: Both FT1 A1

TC5 S3 DS: Both FT2 A2

TC6 S4 DS: Both None A3

DS: Data Source

11.1.1.6 Results


TC1 A complete case series characterization cycle is executed. Population data is

obtained based on the defined eligibility criteria from the database keeping the

TUD data. Statistical analysis is done on the population according to the defined

configurations. Results are presented to the analyst successfully.

SUCCESS

TC2 A complete case series characterization cycle is executed. Population data is

obtained based on the defined eligibility criteria from the database keeping the

LISPA data. Statistical analysis is done on the population according to the

defined configurations. Results are presented to the analyst successfully.

SUCCESS

TC3 A complete case series characterization cycle is executed. Population data is SUCCESS

FP7-287800 SALUS


obtained based on the defined eligibility criteria from the databases keeping both

TUD and LISPA data. Statistical analysis is done on the population according to

the defined configurations. Results are presented to the analyst successfully TC4

LISPA and TUD sources are queried for the defined eligibility criteria. Both of

those systems were not available. CSCT produced the expected error message.

SUCCESS

TC5 LISPA and TUD sources are queried for the defined eligibility criteria. Some

patient data was obtained from these sources. However, statistical analysis rules

could not be applied on the retrieved data. Necessary log messages were created.

SUCCESS

TC6 LISPA and TUD sources are queried for the defined eligibility criteria. However,

there were no suitable patients for the provided criteria. CSCT produced the

expected message.

SUCCESS

FP7-287800 SALUS



FP7-287800 SALUS



12 CONCLUSIONS

In this deliverable, the evaluation reports for the SALUS components developed in the scope of the first prototype of the SALUS project are presented. Specifically, considering the state of the implementation, results are reported for the following evaluation modules, which are already defined in the D7.1.1 - Functional and Non-functional Evaluation Criteria for SALUS Components:

• Use case tests • Fault tolerance analysis • Unit tests • Integration tests

The remaining evaluation modules will be reported in the next version of this deliverable i.e. the D7.1.3 Test and Evaluation Report for SALUS Components – R2. According to the reports, the following statistical information has been obtained: Success Fail

FP7-287800 SALUS


Use cases tests without fault cases 53 0 Use cases tests with fault cases 25 6 Integration tests 37 0 Unit tests 255 13

Table 5 - Statistics for the Test Reports

When the table Success/Fail table is investigated, it can be see that there are a number of failing fault tolerance tests and unit tests. The reason for failure of some of the fault tolerance tests is that the error handling mechanism, which has incomplete or missing checkpoints in the current implementation. During the rest of the implementation, we will resolve those issues. The reason for failing unit tests is that there is a missing configuration while running the tests for the EYE framework. We will resolve this problem during the rest of the implementation as well. Please note that in this version of the deliverable, use cases are tested only for the SALUS components for which an implementation is done in the scope of the first prototype. So, the complete set of the use case tests defined in the D7.1.1 is not included in this deliverable. The remaining ones will be reported again in the D7.1.3. Furthermore, considering the current implementation of the SALUS components in the first prototype, the status of the requirements were also updated in the Requirement Traceability Matrix.

13 REFERENCES

[1] ISO/IEC CD 25040: Software engineering - Software product Quality Requirements and Evaluation (SQuaRE)

- Evaluation reference model and guide, Switzerland: International Organization for Standardization.

[2] ISO/IEC 9126-1: Software Engineering-Software product quality-Part 1 : Quality model. Geneva, Switzerland:

International Organization for Standardization

[3] Maven is a software project management and comprehension tool, http://maven.apache.org/

FP7-287800 SALUS


APPENDIX 1. ASSESMENT OF REQUIREMENTS TRACEABILITY MATRIX

This section consists of a summary of all requirements collected from D3.3.1 and that will continue to be updated during the course of the SALUS development and evaluation. These requirements need to be on this deliverable since some of them can change since the publication of D3.3.1 and the updated requirements are reported here. For clearness of arrangement the requirement are separated in the different categories:

! Functional (coming from uses cases)

! Interface (related to the Graphic user Interfaces, and System Interfaces)

! Non functional

! Data Depending on the category of the requirements a set of attributes will be associated to each requirement. The significant attributes are:

! RE-ID: A unique identifier for the requirement. It will be composed of three parts:

“R”+”Type of requirement – “+“enumerator”

! Requirement text: The explanation of the requirement.

! Liability: The liability of the requirement. Shall is a “must” requirement, Should is an

“optional” requirement, and Obsolete identifies a requirement that is not necessary any more.

! Status: The status of the requirement, if it is proposed, validated, rejected, designed,

implemented, or tested.

FP7-287800 SALUS


FP7-287800 SALUS

SALUS-FP7-287800• D7.1.2 • Version 1.0, dated October 01, 2013 of 132

Functional Requirements

RE-ID Requirement text Liability Status

Requirements for Common Data Element (CDE) Repository

FR-42-1

CDE Repository shall implement mechanisms for authenticating and authorizing Domain Experts

Shall Tested

FR-42-2 CDE Repository should implement mechanisms to parse different domain models such as HITSP C154, CDISC SDTM, CDASH, OMOP CDM Should Tested

FR-42-3 CDE Repository should implement mechanisms to import different domain models such as HITSP C154, CDISC SDTM, CDASH, OMOP CDM

Should Tested

FR-42-4 CDE Repository shall provide feedback to the Domain Expert about the result of the import operations

Shall Tested

FR-42-5 CDE Repository shall present error message when it is unable to access or parse the file content to be imported Shall Tested

FR-42-6 CDE Repository should provide mechanisms to enable the Domain Expert to choose from the discovered matching CDEs to add semantic links with the selected CDEs

Shall Tested

FP7-287800 SALUS


FR-42-7 CDE Repository shall provide mechanisms to annotate the CDEs with extraction specifications from Content Models ( which could be HL7 CCD, or SALUS CIM Ontology) Shall Tested

FR-42-8 CDE Repository shall provide mechanisms to curate new CDEs based on candidate CDEs Shall

Proposed

FR-42-9 CDE Repository shall implement mechanisms to search CDEs through selected search criteria Shall Tested

FR-42-10 Similarity search and several filtering mechanisms should be available for the use of the Domain Expert Shall Proposed

FR-42-11 CDE Repository shall implement mechanisms to let the Domain Expert browse, filter and view the CDEs Should

Tested

FR-42-12 CDE Repository shall implement mechanisms to edit existing CDEs Should

Tested

FR-42-13 CDE Repository shall implement mechanisms to add new CDEs Shall Tested

FR-42-14 CDE Repository shall implement mechanisms to retrieve the extraction specification of a selected CDE for a selected Content Model through the Rest Service interfaces supported

Shall Tested

Requirements for SALUS Semantic Interoperability Layer

FR-4.3-1

System shall enable mechanisms to define mapping Content Model Ontologies to SALUS CIM Ontology Shall

Tested (For OMOP)

FR-4.3-2 System shall enables mechanisms to add the mapping definitions and rules to the SALUS Semantic Resource Set Shall

Tested

FP7-287800 SALUS


FR-4.3-3 System shall enable mechanisms for adding a Terminology System (a subset, or the complete content), which did not exist in the knowledge base of the SIL previously, as an ontology to extend the SALUS Semantic Resource Set Should

Tested

FR-4.3-4 System shall implement mechanisms to import mappings between codes from different terminology systems Shall

Tested

FR-4.3-5 System shall implement mechanisms for searching terminology system codes by code and by designation Shall

Tested

FR-4.3-6 System shall implement mechanisms for retrieving the mappings between terminology systems Shall

Tested

FR-4.3-7 System shall provide error messages when it fails to access terminology server Shall

Tested

FR-4.4-1 SIL shall implement a service for translating an XML instance to the ontological representation of the clinical data Shall

Tested

FR-4.4-2 SIL shall implement a mechanism to translate the ontological representation of the clinical data instance (yet, it is an instance of a Content Model Ontology) to an ontological instance in SALUS Core Ontology, according to the previously defined mapping definitions and rules

Shall Tested

FR-4.4-3 SIL shall implement a mechanism to retrieve the clinical data instances in SALUS Core Ontology format from Data Sources Shall

Tested

FR-4.4-4 SIL shall provide an error message when the native representation format of the original clinical data instance is not recognized Shall

Tested

FR-4.4-5 SIL shall generate an error report when the necessary mapping definitions and rules are not available Shall

Tested

FR-4.5-6 SIL shall implement mechanisms for translation of the clinical data instance represented in SALUS Core Ontology to the corresponding ontological representation of the target content model ontology according to the previously defined mapping definitions and rules

Shall Proposed

FP7-287800 SALUS


Requirements for Technical Interoperability Layer Part 1: Subscription Based Interoperability Profiles and Open Source Toolsets

FR-5.1-1 TIL shall provide standardized interface for subscribing, unsubscribing and retrieving data from Data Source(s)

Shall

Tested

FR-5.1-2 TIL shall provide two level of interfaces: (1) subscription interfaces to be used by Data Requester system actors like ADE Notification Tool to subscribe to Data Source(s) through SIL; (2)subscription interfaces to be implemented on top of Data Source(s) to be used by SIL to communicate with the Data Source(s). 1. Obselete

2. Shall

1. N/A 2. Tested

FR-5.1-3 SIL shall decide whether to use semantic interfaces or TIL to query data from Data Source Shall

Tested

FR-5.1-4 Using TIL existing standards for subscription based data query shall be used as e.g. IHE Care Management profile, as these are available in most of the clinical information systems and therefore allow SALUS to be compatible to most EHR systems.

Shall

Tested

FR-5.1-5 No patient identifying data shall be stored during query of data. Shall

Tested

Requirements for Technical Interoperability Layer Part 2: Query Based Interoperability Profiles and Open Source Toolsets

FR-5.2-1 TIL shall support standardized interfaces to retrieve data from Data Source(s) Shall Tested

FP7-287800 SALUS


FR-5.2-2 TIL shall provide two level of interfaces: (1) query interfaces to be used by Data Requester system actors like ICSR Reporting Tool to query Data Source(s) through SIL; (2) query interfaces to be implemented on top of Data Source(s) to be used by SIL to communicate with the Data Source(s).

1. Obselete 2. Shall

1. N/A 2. Tested

FR-5.2-3 SIL shall decide whether to use semantic interface or TIL to query data from Data Source. Shall Tested

FR-5.2-4 Using TIL existing standards for data querying shall be used as e.g. IHE Query for Existing Data profile, as these are available in most of the clinical information systems and therefore allow SALUS to be compatible to most EHR systems. Shall Tested

FR-5.2-5 No patient identifying data shall be stored during query of data. Shall Tested

Requirements for ICSR Reporting Tool

FR-5.3-1 IRT should be able to be integrated and launched from the EHR system.

Shall

Proposed

FR-5.3-2 ANT should implement mechanisms to trigger automatically the creation of a new ICSR form by the IRT when an ADE is detected and validated by the HP. Shall

Proposed

FR-5.3-3 IRT should implement mechanisms to trigger manually from the EHR system’s interface (via an option displayed in a menu of the EHR system’s interface) the creation of a new ICSR form. Shall

Proposed

FP7-287800 SALUS


FR-5.3-4 IRT should implement mechanisms to call SIL-DS service for retrieving the medical summary of a patient identified by a PID. Shall

Proposed

FR-5.3-5 SIL should implement mechanisms to translate the received MEDICAL DATA to the common semantic model of SALUS Obselete

N/A

FR-5.3-6 IRT should implement mechanisms to convert patient data received from SIL-DS to the syntax of E2B XML. Shall Proposed

FR-5.3-7 IRT should implement mechanisms to present the pre-filled ICSR as a html form to the end user. Shall Proposed

FR-5.3-8 IRT should implement mechanisms to check if mandatory data fields are correctly filled and format of the data entered is correct (date, etc.), before sending the ICSR form.

Shall Proposed

FR-5.3-9 SALUS platform should provide Pseudonymization Services. shall Proposed

FR-5.3-10 SIL should implement interfaces to communicate with the Pseudonymization Service to send the ICSR form identified by the PID of interest before sending it to IRT.

Obselete N/A

FR-5.3-11 IRT should implement mechanisms to ensure that ICSR form has been pseudonymized before sending it to the regulatory authorities. Shall Proposed

FR-5.3-12 De-identifcation/Pseudonymisation service should implement mechanisms and interfaces to send the pseudonymized ICSR form to the Local/International Regulatory Authority. Shall

Proposed

FR-5.3-13 IRT should implement mechanisms to print the ICSR form in case the HP wants to send a paper version to the Local/International Regulatory Authority (in LISPA case, where the physical signature of HP is needed for validation by regulatory authority).

Should Proposed

FR-5.3-14 IRT should implement mechanisms for receiving message from the Regulatory Authority confirming that the ICSR form has been electronically received.

Should Proposed

FP7-287800 SALUS


FR-5.3-15 EHR system should be configured to allow IRT to record ICSR files in a local repository. Should Proposed

FR-5.3-16 EHR system should be configured to allow IRT to access and update ICSR files recorded in a local repository during a previous session.

Should Proposed

FR-5.3-17 IRT should implement mechanisms for browsing the local repository where are stored previously sent and waiting to be reported ICSRs files (via a database storing information describing the ICSRs files having been created in the repository: Sender’s (case) safety report unique identifier, edition date, references of the related patient: name and PID, status of the report: completed or waiting to be reported, etc.).

Should Proposed

FR-5.3-18 IRT should implement mechanisms for duplicating previously sent ICSR files stored in the local repository and updating automatically the E2B data elements used to indicate that a new ICSR is realized for the same case.

Obselete N/A

FR-5.3-19 IRT should implement a versioning system to distinguish first ICSR versions sent to regulatory authorities with new ones, and relating the two files to the same ADE case.

Should Proposed

FR-5.3-20 IRT should implement a backup process making it possible to continue the reporting where it was interrupted in case of ICSR reporting interruption (because of a computer crash for example).

May Proposed

FR-5.3-21 IRT should implement a saving process making it possible for the user to save the ICSR form partially filled and continue later the reporting process.

Should Proposed

FR-5.3-22 When reporting an ADE, IRT should allow the HP to browse the already reported ADE by the same HP, who previously reported the same kind of ADE either for the same patient or some other patient.

May Proposed

Requirements for SALUS Security and Privacy Services

FR-5.4-1 SALUS shall provide a De-identification Service for removing key personal identification information from a clinical data instance. Clinical data instance can be either a patient summary as a SALUS CIM ontology format. Shall

Tested

De-identification Service shall be configurable, that is, it shall be possible to dictate what kind of de-identification methods will be applied on configured data fields in the clinical data instance. Shall

Tested

FP7-287800 SALUS


De-identification Service shall provide a default configuration for E2B Deidentification and CIM Deidentification separately. Shall

Tested

FR-5.4-2 A Pseudonymization method shall be provided within the De-identification service for replacing the Patient Identifier (PID) that exists in a clinical data instance, which can be an ICSR or a patient summary formalized with the SALUS Common Information Model ontology, with a pseudonym (PSN).

Shall Tested (except 13606)

FR-5.4-3 De-identification Service should implement mechanisms for generating a PSN given a PID by two-way and/or one-way cryptography algorithms based on the requirement of the use case. Should

Tested

FR-5.4-4 De-identification Service shall implement mechanisms for forwarding the pseudonymized clinical data to Data Requester/Data Target Shall

Tested

FR-5.4-5 De-identification Service shall notify the data sender about the result of the pseudonymization operation Shall Tested

FR-5.4-6 De-identification Service shall inform the the Data Requester about the problem with an error report if De-identification Service is not able to generate PSN for the provided PID.

Shall Tested

FR-5.4-7 De-identification Service shall implement mechanisms for re-identifying a pseudonym (PSN); i.e. replacing the PSN corresponding to the Patient Identifier (PID) in a query/notification about a specific patient that needs to be sent back to Data Source.

Shall Tested

FR-5.4-8 De-identification Service shall inform the Data Requester/Data Target about the problem with a report when Pseudonymization method is not able to generate the corresponding PID for the provided PSN

Shall Tested

FR-5.4-9 SALUS shall provide an Audit Repository to log Audit records sent by SALUS system actors that implement IHE Secure Node actor in compliance to IHE ATNA Profile through ITI-20 transactions

Shall Proposed

FR-5.4-10 Secure Node shall send ITI-20 Record Audit Event transaction to Audit Repository. Shall Proposed

FR-5.4-11 Audit Repository shall implement mechanisms to store and maintain audit records received from Secure Nodes Shall Proposed

FP7-287800 SALUS


FR-5.4-12 Audit Repository shall inform the Secure Node about the problem with an error report when it is not able to process the audit log it receives

Shall Proposed

FR-5.4-13 SALUS shall provide a Certificate Authority for assigning certificates to all of the SALUS system actors (When it is not appropriate to use the existing certificates of the SALUS parties)

Shall Proposed

FR-5.4-14 All SALUS system actors shall be mutually authenticated with the assigned certificates Shall Proposed

FR-5.4-15 All communications among the SALUS system actors should be secured using the assigned certificates. Should

Proposed

FR-5.4-16 All SALUS system actors with an interface should be integrated with the Single Sign-on mechanisms of the host organization (like Hospitals, National Networks, Regulatory Authorities, Research Institutes, Pharmaceutical Companies) Should

Proposed

Requirements for ADE Notification Tool

FR-6.1-1 The EHR System should implement mechanisms to launch the ADE Notification Tool. Should

Proposed

FR-6.1-2 When new data is added to an EHR, the ADE Notification Tool has to distinguish between “relevant data for ADE detection” and “irrelevant data for ADE detection”.

Should Tested

FR-6.1-3 Known ADE detection rules have to be integrated. Should Tested

FR-6.1-4 Statistical methods have to be implemented to derive new ADE detection rules. Should Tested

FR-6.1-5 The ADE Notification Tool should implement a graphical user interface. Should Tested

FP7-287800 SALUS


FR-6.1-6 The ADE detection rules have to be visible to the physicians (for example to select one of them for quality improvement). Should Tested

FR-6.1-7

The ADE Notification Tool should allow the physicians to add new ADE detection rules manually. Should Tested

FR-6.1-8 The ADE Notification Tool should allow the physicians to change existing ADE detection rules manually (for example the parameters of lab results within a rule).

Should Tested

FR-6.1-9 The ADE Notification Tool should allow the physicians to delete existing ADE detection rules manually. Should Tested

FR-6.1-10 Changes concerning the ADE detection rule base (removing/adding/changing a rule) should be visible for other physicians before they are applied.

Should Proposed

FR-6.1-11 The ADE Notification Tool should provide mechanisms for a physician to confirm/reject the changes on the ADE detection rule base made by other physicians.

Should Proposed

FR-6.1-12 The ADE Notification Tool should provide the possibility for the physician to justify his decision (for example changing a rule) in form of a free-text field.

Should Proposed

FR-6.1-13 The ADE Notification Tool should store the physician’s decision on alerts (an ADE is present or not) to evaluate the quality of each ADE detection rule.

Should Proposed

FR-6.1-14 When a new ADE detection rule is proposed by a physician or derived by the ADE detection, the relevant parameters/information has to be forwarded to the SALUS ADE Notification Tool expert who implements the new detection rule.

Should Proposed

FR-6.1-15 The ADE Notification Tool should allow the physician to manually decide that an ADE is present. Should Tested

FR-6.1-16 When the ADE Notification Tool produces an alert this should be presented to the physician accordingly via the graphical user interface.

Should Tested

FP7-287800 SALUS


FR-6.1-17 In the case that the ADE Notification Tool detects a new ADR, this should be forwarded to the ICSR Reporting Tool. Should Tested

Requirements for Safety Analysis Tools

FR-6.2-1 The CSC Tool should be a web based system to support access from different locations Should

Tested

FR-6.2-2 The CSC Tool shall use an authentication system to validate that the Safety Analyst can access the interface Shall

Tested

FR-6.2-3 The user should be able to specify the statistics that should be visible in the interface Should

Tested

FR-6.2-4 The user shall be able to specify the inclusion criteria both for the cases and the comparator population Shall

Tested

FR-6.2-5 There shall be a mechanism to translate the inclusion/exclusion choices for the cases and background population specified in queries into query templates used in the SIL Shall

Tested

FR-6.2-6 There should be a mechanism that ensures that only valid query templates are created Should

Tested

FR-6.2-7 The query results shall be presented to the user in the CSC Tool interface in the data model preferred by the Safety analyst in a pseudonymized way. Shall

Tested

FR-6.2-8 The patients in the case group shall be possible to transfer into the Patient History Tool Shall

Proposed

FR-6.2-9 Optional: The patients in the background population group should be possible to transfer into the Patient History Tool May

Proposed

FP7-287800 SALUS


FR-6.2-11 The TPC Tool should be a web based system to support access from different locations Should

Proposed

FR-6.2-12 The TPC Tool shall use an authentication system to validate that the Safety Analyst can access the interface Shall Proposed

FR-6.2-13 The user shall be able to specify the inclusion criteria both for the cases and the comparator population Shall Proposed

FR-6.2-14 There shall be a mechanism to translate the inclusion/exclusion choices for the cases and background population specified in queries into query templates used in the SIL

Shall Proposed

FR-6.2-15 There should be a mechanism that ensures that only valid query templates are created Should Proposed

FR-6.2-16 The results from the query should be presented graphically to the user Should Proposed

FR-6.2-17 The EA Tool should be a web based system to support access from different locations Should Proposed

FR-6.2-18 The EA Tool shall use an authentication system to validate that the Safety Analyst can access the interface Shall Proposed

FR-6.2-19 The user shall be able to specify the inclusion criteria both for the cases and the comparator population Shall Proposed

FR-6.2-20 There shall be a mechanism to translate the inclusion/exclusion choices for the cases and background population specified in queries into query templates used in the SIL

Shall Proposed

FR-6.2-21 There should be a mechanism that ensures that only valid query templates are created Should Proposed

FP7-287800 SALUS


FR-6.2-22 The Analyst shall be able to run the selected signal detection algorithms on top of the Analysis Repository to identify emerging patterns of temporal association between drug prescriptions and medical events for detailed clinical review

Shall Proposed

FR-6.2-23 When necessary use cases “UC-6.2-1 Query Population Data for Safety Analysis” and “UC-6.2-4 Visualizing Patient History” are triggered.

Should Proposed

FR-6.2-24 The Patient History Tool should be a web based system to support access from different locations Should Tested

FR-6.2-25 The Patient History Tool shall use an authentication system to validate that the user can access the interface Shall Tested

FR-6.2-26 The Patient History Tool shall retrieve data from the Safety Analysis Repository Shall Proposed

Interfaces Requirements

RE-ID Requirement text Liability

Status

Requirements of CDE Repository

IR -4.2-1 CDE Repository GUI shall provide interfaces to help and inform the user about the authentication and authorization processes Shall Tested IR -4.2-2 CDE Repository GUI shall provide interfaces to import new domain models Shall Tested

IR -4.2-3 CDE Repository GUI shall present error messages to inform the Domain Expert when the CDE Repository is not accessible Should Tested

IR -4.2-4 CDE Repository GUI shall provide interfaces to annotate the CDEs by providing links to existing CDEs Shall Tested

IR -4.2-5 CDE Repository GUI shall provide interfaces to add extraction specifications to the CDEs Shall Tested

IR -4.2-6 CDE Repository GUI shall provide interfaces to search CDEs Shall Tested

IR -4.2-7 CDE Repository GUI shall provide interfaces to filter CDEs Shall Proposed

FP7-287800 SALUS


1 This specific requirement, as other of the same kind, will depend on whether SALUS team will be able and decide to change the local EHR system in the course of the project, for instance to add a button “Report an ADE”, as described here. One can hypothesize that modifying the EHR system will be very difficult, and consequently recommend technical solutions not requiring such modifications. The solution described by the present requirement is of that kind: the button is displayed on top of the EHR interface (type “floating window”), but it is not included inside the EHR interface.

As explained in SALUS D8.1.1, section 4.1.2.2, it will be discussed and decided whether the IRT shall be a standalone Web based interface or can be integrated to the existing systems used by the Physicians.

See also the “Open issues” of this section.

IR -4.2-8 CDE Repository GUI shall provide interfaces to browse CDEs Shall Tested

IR -4.2-9 CDE Repository GUI should provide interfaces to display the detailed attributes of the selected CDE Shall Tested

Requirements for ICSR Reporting Tool IR -5.3-1 IRT should implement on top of the EHR system’s interface the following buttons giving access to several functionalities for ICSR

reporting: a. “Report an ADE” to trigger manually the creation of a new ICSR form1 ; b. “See the previously sent and the waiting to be reported ICSRs” to access IRT interface displaying the list of

previously reported ICSR files and waiting to be reported ICSR files for the patient (given his PID).

Should

Proposed

IR -5.3-2 The IRT graphical user interface (GUI) displaying previously sent and waiting to be reported ICSR : a. should be able to display the list of previously sent and of the waiting to be reported ICSR files with their

edition date, references of related patient (name, PID), status of the report: completed or waiting to be reported, and possibly other information;

b. should make available a function/button “Update the ICSR” for previously sent ICSR files of that list; c. should make available a function/button “Finalize and report the ICSR” for ICSR files recorded as “to be

reported later” during a previous session of that list

Should

Proposed

IR -5.3-3 The IRT GUI for filling the ICSR form should consist of fixed sets of data fields and editable field-texts corresponding to the ICSR E2B required fields. Should Proposed

IR -5.3-4 The IRT GUI should distinguish mandatory and optional data fields. Should Proposed

IR -5.3-5 The IRT GUI should have an editable textbox for storing any additional comments and remarks from the HP. Should Proposed

IR -5.3-6 The IRT GUI should have a “Report” button for sending the filled ICSR form to the Local/International Regulatory Authority (after pseudonymization process). Until all the mandatory fields are not filled and validated by HP, the “Report” button remains inactive. Once all the mandatory fields are filled and validated by HP, the “Report” button becomes active for sending the ICSR form.

Should Proposed

IR -5.3-7 In case mandatory data fields are not correctly filled or format of the data entered is not correct (date, etc.), the ICSR form is not sent, and IRT GUI asks the user to correct the wrong entries. Should Proposed

IR -5.3-8 The IRT GUI should have a button to print the ICSR form in case the HP wants to send a paper version to the Local/International Regulatory Authority.

Should Proposed

FP7-287800 SALUS


IR -5.3-9 The IRT GUI should implement an interface component to inform the user that the ICSR form has been electronically received by the Regulatory Authority.

Should Proposed

IR -5.3-10 OPTIONAL. IRT GUI should implement interface components proposing the user to continue the reporting process where it was interrupted in case of ICSR reporting interruption.

May Proposed

IR -5.3-11 OPTIONAL. IRT GUI should implement interface components allowing the loading of a saved partially filled ICSR form to continue the reporting process where it was voluntarily stopped

May Proposed

Requirements for SALUS Security and Privacy Services IR -5.4-1 Audit Repository shall provide interfaces to authorized users to view Audit logs in the repository Shall Proposed

IR -5.4-2 Audit Repository shall provide interfaces to authorized users to sort and filter Audit logs in the repository Shall Proposed

Requirements for ADE Notification Tool IR -6.1-1 An access to the local EHR database storing clinical data of the patient should be available. Should Tested

IR -6.1-2 An access to all local and available EHR files should be available for the statistical methods of the ADE Notification Tool through TIL.

Should Proposed

IR -6.1-3 The new detected ADRs should be forwarded to the ICSR Reporting Tool. Should Tested

Requirements for Safety Analysis Tools IR -6.2-1 CSC Tool should use the user authentication system already created in SALUS Should Tested

IR -6.2-1 CSC Tool should implement functionality to specify the input to create the query template Should Tested

IR -6.2-2 Should include the possibility to select cases and background population Should Tested

IR -6.2-3 CSC Tool should include a graphical interface that presents the results from the queries back to the user Should Tested

IR -6.2-4 TPC Tool should use the user authentication system already created in SALUS Should Proposed

IR -6.2-5 TPC Tool should implement functionality to specify the input to create the query template: a. Should include the possibility to select cases and background population

Should Proposed

IR -6.2-6 TPC Tool should include a graphical interface that presents the results from the queries back to the user Should Proposed

IR -6.2-7 Optional: A “Save” button to store results for future review May Proposed

IR -6.2-8 Safety Analyst can logged in to the SALUS Analyst Interface Should Proposed

IR -6.2-9 The Analyst can specify the specifications of the medical datasets to be collected using the SALUS Analyst Interface: The sections of the EHR extracts of interest are specified and the eligibility criteria specifying the patient population of interest are specified

Should Proposed

IR -6.2-10 The user shall be able to input a patient identifier to see the patient history Shall Proposed

IR -6.2-11 Optional: The user shall be able to sort the results based on dates (like start date of medication) and type of event (Medication, Diagnosis and Lab Value)

May Proposed

FP7-287800 SALUS


Data Requirements

RE-ID Requirement text Liability Status

Requirements of CDE Repository

DR-4.2-1 The curated CDEs should be represented in an ontology conforming to ISO/IEC 11179 Metadata Registry/Repository Model.

Should Tested

Requirements of SALUS Semantic Interoperability Layer

DR-4.3-1 The mappings/alignments among source and target ontologies should be represented in a machine processable manner, e.g. as SPARQL constructs or N3 rules. Should

Tested

DR-4.3-2 The alignment rules between Terminology System Ontology and the SALUS Core Ontology should be represented in a machine processable manner, e.g. as SPARQL constructs or N3 rules. Should

Tested

DR-4.3-3 The mappings between different terminology system codes should be represented in a machine processable manner Should Tested

DR-4.3-4 The mapping information between two terminology systems to be provided by the Terminology Expert, or to be retrieved from external terminology repository should be represented in a machine processable format

Should Tested

DR-4.3-5 The external domain ontology to be included to the SALUS Semantic Resource Set shall be in one of the well accepted representation standards like RDF, OWL, N3. Shall

Tested


FP7-287800 SALUS


2 See ANNEX section for a precise description of data elements required by E2B(R2) specifications. 3 The degree of mapping coverage between the two data model (for instance expressed in percentage of E2B ICSR data elements for which a direct equivalent is described in the data elements of the data model used by EHR system) that must be achieved for the ICSR prepopulation function to be efficient remains to be quantified. However a basis requirement is that ICSR data elements that are mandatory for E2B have such a mapping. The same remark applies for terminology mappings.

DR-5.1-1 Data queried from Data Source should be presented in a standardized way like CDA or CCD document type, to be processable by SALUS ontology. Should Proposed


DR-5.2-1 Data queried from Data Source should be presented in a standardized way like CDA or CCD document type, to be processable by SALUS ontology.

Should Tested


DR-5.3-1 An access to the local EHR database storing clinical data of the patient should be available2. Should

Proposed

DR-5.3-1 SIL shall be configured to reconcile the Data Model and the terminology systems used by the ICSR Reporting Tool (in compliance with the E2B guideline) and the EHR System3. Shall

Proposed

DR-5.3-1 OPTIONAL. Proof-tree generated by the reasoner (i.e. justification trace) for a detected ADE in an EHR under the given ADE rules. May

Proposed


DR-5.4-1 Audit Records shall comply with the IHE Audit Trail format. Shall Proposed

FP7-287800 SALUS



DR-6.1-1 A threshold for every ADE detection rule should be implemented and controlled so that it is possible to find ADE detection rules that always produce wrong alerts Should Tested

DR-6.1-2

The following data fields are necessary to get from the EHR-System: 1. - gender: must have 2. - age: must have 3. - lab results: must have 4. - diagnoses: must have (but we need more than just the current diagnosis) 5. - medications (drug, dosage and unit): must have

- symptoms/emergency: must have

Should Tested

DR-6.1-3

OPTIONAL. The following data fields would be nice to get from the EHR-System: 1. weight: nice to have 2. - drug allergies: nice to have 3. - intolerances: nice to have 4. - family history: nice to have 5. - past medical history: nice to have 6. - encounters/episodes of care/stays: nice to have 7. - pregnancy: nice to have 8. - vital signs measurement: nice to have 9. - procedures: nice to have

May Tested


DR-6.2-1 It is assumed that a Data warehouse has already established by the participating healthcare institute. SALUS will support tools to create this DWH if it does not exist.

Should Proposed

Non Functional Requirements

FP7-287800 SALUS


RE-ID Requirement text Referred use case

Liability Status

Requirements of CDE Repository NFR-4.2-1 Processing time Requirements:

i. The import operation for uploading new UML Models/Content Models/Local Domain Ontologies must be realized in a reasonable time.

ii. The annotation of Content Models must be realized in a reasonable time. iii. Searching matching CDEs should be realized within a few seconds

Should

Tested (Import can take a couple of minutes according to the number of CDEs in the Content Model to be imported.

NFR-4.2-2 Space Requirements: (i) A reasonable amount of RAM shall be available for annotating Content Model Ontologies with Terminology System codes and available CDEs. (ii) A reasonable amount of disk space shall be available for storing CDEs in CDE Repository.

Shall Tested

Requirements of SALUS Semantic Interoperability Layer

NFR-4.4-1 Processing time Requirements: a. The operation for retrieving terminology systems from external resources must be realized in a reasonable time b. The operation for uploading external ontologies must be realized in a reasonable time c. Retrieving clinical data instance to the SIL must be realized in a reasonable time d. Translating clinical data instance in source format to target format must be realized in a reasonable time e. Running a semantic query on top of the SIL must be realized in a reasonable time

Shall Proposed

NFR-4.4-1 A reasonable amount of RAM shall be required for SIL. Shall Proposed


FP7-287800 SALUS


NFR-5.1-1 The result of a query should be available within an acceptable time for the physician.

Should Proposed


NFR-5.2-1 The result of a query should be available within an acceptable time for the physician.

Should Proposed


NFR-5.3-1 Processing time Requirements: Once the creation of a new ICSR is triggered, the ICSR pre-population process must be realized in a reasonable time. Time to be decided with end users.

Shall Proposed

NFR-5.3-2 User Requirements: When reporting an ADE through the IRT, the tool should allow the HP to see justification behind the ADE to be reported.

Should Proposed

NFR-5.3-3 Space Requirements: A reasonable hard-disk space shall be required to store the history/log of all ADEs reported by each HP. Shall Proposed


NFR-5.4-1 Processing time Requirements: a. De-identification of a single clinical dataset must be realized in a reasonable time (less than a minute). b. Re-identification should be realized within a few seconds c. Pseudonymization should be realized within a few seconds

Should Tested

NFR-5.4-2 Space Requirements: (i) A reasonable amount of disk space shall be required for the server to host Audit Repository (ii) A reasonable amount of RAM shall be required for the server to host De-identification/Pseudonymization services.

Shall De-identification/Pseudonymization Tested


NFR-6.1-1 The graphical user interface of the ADE Notification Tool should be easy to understand and intuitive to work with due to the fact that the HP usually are no computer experts.

Should Tested

NFR-6.1-2 The ADEs should be detected within an appropriate time. Should Tested

FP7-287800 SALUS



NFR-6.2-1 Processing time Requirements: a. Once the query templates are sent the results should be presented to the user within a reasonable time. Time to be

decided. b. Long running queries should not be stopped due to “time-outs” in the interface

Should Proposed

NFR-6.2-2 Processing time Requirements:

a. The ad hoc selection of cases and background population will probably make for quite long execution times and as long as the save functionality exists that won’t be a problem.

Should Proposed

NFR-6.2-3 Processing time Requirement: a. Since a user usually analyses multiple patient histories for a specific issue the processing time is important for

effective patient review. No more than a couple of seconds can be used for each patient

Should Proposed

FP7-287800 SALUS


d7.1.2-test and evaluation report for salus …...fp7-287800 salus salus-fp7-287800• d7.1.2 •...

Documents