d5.4 specification for functional ecrf and...

TRANSFoRm FP7-‐247787 D5.4 Specification for functional eCRF and DSS

1

Translational Research and Patient Safety in Europe

D5.4 Specification for functional eCRF and DSS

Work Package Number: WP5 Work Package Title: Interfaces: User and Software Services

Nature of Deliverable: Other

Dissemination Level: Public

Version: v1.0

Delivery Date From Annex 1: M51 (May 2014)

Principal Authors: Olga Kostopoulou (KCL), Talya Porat (KCL), Lei

Zhao (UW), Sarah N. Lim Choi Keung (UW),

Theodoros N. Arvanitis (UW)

Contributing Authors: Samhar Mahmoud (KCL), Piotr Bródka (WRuT),

Andrzej Misiaszek (WRuT), Wlodzimierz Tuligłowicz

(WRUT), J.-F. Ethier (INSERM), Anna Andreasson

(KI)

Partner Institutions:

This project has received funding from the European Union’s Seventh Framework Programme for research, technological development and demonstration under grant agreement no 247787 [TRANSFoRm].


2

Table of Contents List of Figures ............................................................................................................ 3

List of Tables ............................................................................................................. 4

Abbreviations ............................................................................................................. 5

Executive Summary .................................................................................................. 6

1. Specification for functional DSS ....................................................................... 8

1.1 Introduction ...................................................................................................... 8

1.2 User Interface Requirements and Recommendations ..................................... 8

1.3 DDSS Design .................................................................................................. 9

1.4 Usability Evaluation of the “dummy” prototype .............................................. 10

1.5 Summary ....................................................................................................... 13

1.6 References .................................................................................................... 14

2. Specification of a functional eCRF to be integrated with an EHR ................ 15

2.1 Overview ........................................................................................................ 15

2.2 Semantic-aware eCRF Modelling .................................................................. 18

2.3 GORD Evaluation Study Specification .......................................................... 39

2.4 Conclusion ..................................................................................................... 47

2.5 References .................................................................................................... 48

Appendix 1: User Requirements Elicitation (manuscript) ................................... 50

Appendix 2: DDSS User Guide ............................................................................... 67

Appendix 3: GORD-SDM.xml .................................................................................. 78

Appendix 4: Study Data Collection Forms ............................................................ 79

Appendix 4A CROMs ............................................................................................. 79

Appendix 4B PROMs ............................................................................................. 91


3

List of Figures

Figure 1: GORD study timeline [2, Figure 2] ............................................................. 17

Figure 2. CDISC suite of standards [9] ...................................................................... 20

Figure 3. Structure of the XML file ............................................................................. 21

Figure 4. Example of Treatment Epoch element for the GORD study ...................... 22

Figure 5. Example of Continuous Arm element for the GORD study ........................ 22

Figure 6. Example of Treatment cell for the GORD study ......................................... 23

Figure 7. Example of the Follow-up segment for the GORD study ........................... 23

Figure 8. Examples of the criteria check and study statistics collection activities in the

GORD study ....................................................................................................... 23

Figure 9. Structural elements of the GORD study ..................................................... 24

Figure 10. Activities with workflow-specific roles in the GORD study ........................ 25

Figure 11. Examples of Transition and Switch elements for the eligibility criteria check

activity ................................................................................................................ 25

Figure 12. Example of a Trigger element in the GORD study ................................... 26

Figure 13. Timing constraint for the first visit PROM collection in the continuous arm

........................................................................................................................... 26

Figure 14. ODM file attributes for the GORD study ................................................... 28

Figure 15. StudyEventDef element for unscheduled visits in the on-demand arm .... 29

Figure 16. FormRef element for an event visit CROM .............................................. 29

Figure 17. ItemGroupDef element for the item group for weight ............................... 30

Figure 18. ItemDef element for the weight item ......................................................... 31

Figure 19. Link between ODM and the Reference Model via archetypes ................. 33

Figure 20. Part of the data extraction query for weight .............................................. 34


4

Figure 21. Archetype for weight - definition ............................................................... 34

Figure 22. Archetype for weight - ontology definitions ............................................... 35

Figure 23. Archetype for weight - ontology term bindings ......................................... 35

Figure 24. Archetype-based query model – main elements ...................................... 37

Figure 25. Query model - expressions ...................................................................... 38

Figure 26. GORD evaluation study workflow ............................................................. 40

Figure 27. Randomisation blocking definition ............................................................ 46

Figure 28. Study parameter – planned number of recruited subjects ........................ 47

List of Tables

Table 1. EHR data elements for CROM form pre-population .................................... 45

TABLE 2: Excerpt from the SAR analysis of a think-aloud protocol of GP 9 (cancer

diagnosis study) ................................................................................................. 56

TABLE 3: Excerpt from the SAR analysis of a CDM protocol of GP 8 (intuition study)

........................................................................................................................... 57

TABLE 4: Cognitive requirements of the diagnostic process .................................... 58


5

Abbreviations CDIM Clinical Data Integration Model

CRIM Clinical Research Information Model

CROM Clinician Reported Outcome Measures

DNC Data Node Connector

DDSS Diagnostic Decision Support System

DSS Decision Support System

EC Eligibility Criteria

eCRF electronic Case Report Form

EHR Electronic Health Record

GORD Gastro-oesophageal reflux disease

GP General Practitioner / General Physician

ODM Operational Data Model

PRM Protocol Representation Model

PROM Patient Reported Outcome Measures

RfE Reason for Encounter

SDB Study Database

SDM Study Design Model

SDTM Study Data Tabulation Model

SF-12 Short Form 12

TAM Technical Acceptance Model

TSS TRANSFoRm Study System

VarQs Form variables expressed using the TRANSFoRm query model


6

Executive Summary This document delivers the functional specifications for eCRF and DSS TRANSFoRm

software configurations. It is composed of two sections. Section 1 focuses on the

DSS and provides:

(1) a summary of the diagnostic decision support system (DDSS) design

requirements and recommendations

(2) a short description on the proposed user interface following the requirements and

recommendations and

(3) main findings of the preliminary prototype usability evaluation.

We are currently updating the design of the prototype according to the usability

findings and we will continue to evaluate and update the DDSS using participatory

design methods. A manuscript on the user requirements elicitation methods and

design recommendations is included in Appendix 1: User Requirements Elicitation

(manuscript) and is currently being prepared for submission to a peer-reviewed

journal). a detailed DDSS user guide is included in Appendix 2: DDSS User Guide.

Section 2 of this deliverable describes the outcomes of WT 5.2a Specification for a

functional eCRF to be integrated with an EHR. It reports the development of the

specification and demonstration of a functional electronic Case Report Form,

designed to enable the collection of semantically-controlled data from within an EHR.

The eCRF model described specifies the way eCRF forms are constructed, and how

the eCRF data is made semantically interoperable with clinical data from within an

EHR. The specification of the eCRF model is described using the example of the

TRANSFoRm GORD validation study.

Following the model-based approach in TRANSFoRm, the eCRF model works within

the scope of the Clinical Research Information Model (CRIM), D6.2. The model

describes the lifecycle of the study data collection process, including activities such

as informed consent and randomisation, as defined in the study protocol. It also

supports data collection using various methods (mobile, web interface, software

within the EHR system), in various languages, and is in a format that can be

deployed within an EHR system (e.g. XML format).

The specification also shows how the CDISC Standard Operational Data Model

(ODM) is extended to enable eCRF data to be semantically interoperable with EHR


7

data. This is mainly to enable EHR data to be extracted and pre-populate eCRFs

where applicable. The deliverable demonstrates how archetypes representing clinical

data elements, linked to the Clinical Data Integration Model (CDIM) ontology (D6.3),

are bound to eCRF form elements to maintain the semantic meaning throughout. The

extended ODM document that is exchanged between the Study System and an EHR

system will include queries describing how eCRF form elements can be extracted

from the EHR system.


8

1. Specification for functional DSS

1.1 Introduction

A major part of TRANSFoRm is the design and development of a functional prototype

for a diagnostic decision support system (DDSS) that will be integrated with the

Electronic Health Record (EHR), for aiding general physicians in the diagnosis of

three major reasons for encounter: chest pain, abdominal pain and dyspnoea. In the

future, the DDSS can be expanded to include other reasons for encounter, though

this is beyond the scope of TRANSFoRm. This document summarises the user

interface requirements, design and evaluation of the DDSS prototype. For further

information, please refer to the appendices.

1.2 User Interface Requirements and Recommendations

To elicit user requirements for the design of the DDSS, we conducted in situ

observations and interviews of 8 family physicians consulting with patients. We also

used data collected in other recent studies of Dr Olga Kostopoulou. Specifically, we

analysed 34 think-aloud protocols of 17 physicians diagnosing computerised

patients, and 24 interview protocols of 18 family physicians describing past cases of

intuitive diagnosis. Transcripts were coded using the Situation Assessment Record

method (SAR; Hoffman et al., 1998). The result of the analysis was a Decision

Requirements Table (See Appendix 1: User Requirements Elicitation (manuscript))

that guided the user interface design recommendations for the DDSS. Below are the

main design recommendations:

• Highlight important patient information existing on the health record, e.g. risk

factors.

• Display possible diagnoses at the start of the clinical encounter, based on

existing information in the health record and the current reason for encounter

(RfE).

• Enable users to select a diagnosis from the list and check what features

(symptoms and signs) are relevant (i.e. can impact on the likelihood of that

diagnosis).

• Enable users to indicate both the presence and absence of symptoms and

signs.


9

• Facilitate data insertion and coding during the consultation.

• Propose specific and relevant examinations and investigations that could be

preformed in order to differentiate between the diagnoses on the list.

• Alert physicians after they give a diagnosis but have failed to check any

symptoms and signs of serious or urgent possible diagnoses.

See Appendix 1: User Requirements Elicitation (manuscript) for the manuscript in

preparation that describes the user requirements elicitation methods and user

interface design recommendations.

See DELIVERABLE 4.1 for the technical and functional aspects of the DDSS.

1.3 DDSS Design

Based on the above requirements and recommendations we tried several,

preliminary user interface designs of the DDSS. We presented and discussed these

at TRANSFoRm face-to-face meetings and with GPs from the department of primary

care of King’s College London. On the basis of these discussions, a design was

adopted and a “dummy” prototype was developed and evaluated with 10 experienced

GPs (see 1.4 Usability Evaluation of the “dummy” prototype).


10

Appendix 2: DDSS User Guide presents the DDSS User Guide that explains the

DDSS functions and how GPs can use it. For the final evaluation study of the fully-

integrated and functioning DDSS prototype (WT3), we have an agreement with

Vision EHR (www.inps4.co.uk/vision) to integrate the DDSS into their system, though

the DDSS could be integrated with any EHR system.

1.4 Usability Evaluation of the “dummy” prototype

For the usability evaluation, a “dummy” interactive prototype of the DDSS was

developed. The prototype is “dummy” in the sense that it is not yet integrated with

Vision EHR or the evidence database. Hence, the selections (e.g., RfE, signs,

symptoms, examinations, and diagnosis) and the suggested diagnoses list are pre-

defined and static (i.e. do not change according to user input). See the DDSS Interactive Demo demonstrating the prototype functionality.

1.4.1 Method

We evaluated the “dummy” prototype with 10 experienced GPs (6 male, mean 13.4

years in general practice, range 4 to 23 years). Our aim was to evaluate the three

usability goals: effectiveness (can users use the tool?), efficiency (how well can they

use it?) and satisfaction with the tool (ISO 9241-11). Each evaluation lasted up to 1.5

hours. Using the DDSS prototype, participants were asked to diagnose two scenarios

of young women presenting with abdominal pain. The researcher simulated the two

patients but did not interrupt participants in any other way. She observed and noted

the participants’ workflow, selections on the screen, the information that they entered

in the record, errors in the recording of information and other key incidents, and

comments that they made.

At the end of each scenario and at the end of the evaluation, the researcher

interviewed the participants. The interview addressed participants’ overall

impressions of the diagnostic system and specifically asked what they liked or

disliked about it, what features they found useful, and to name situations where they

thought that the system would be most effective. The interview also asked what

additional functionality or improvements they would like and whether they would like

to use this tool in the future. Finally, the researcher sought opinions on a specific

design idea: a ‘premature closure’ alert, which would be triggered after a diagnosis is

entered and only if the GP has not checked any of the symptoms and signs of


11

serious or urgent possible conditions.

1.4.2 Main findings

1.4.2.1 Effectiveness

All participants managed to enter the patient’s main RfE at the beginning of the

consultation and the main signs, symptoms and examination results during the

consultation. All participants viewed the list of suggested diagnoses and interacted

with it (i.e., clicked on a diagnosis).

1.4.2.2 Efficiency

The diagnostic system was very easy to learn and use appropriately. All participants

managed to use it after 3 minutes of explanation and approximately 2 minutes of

system trial. No errors were performed using the tool, except for one participant who

made it quit by mistake (clicked the ‘Done’ button before entering all the patient

information).

1.4.2.3 Satisfaction

Perceived Effectiveness

Participants were enthusiastic about the system. They thought it could be very

effective and help them in the diagnostic process. Particularly, they thought it could

expand the number of hypotheses they consider and prevent them for missing

important information. “I didn’t think about Pelvic Inflammatory Disease, however I

saw it on the list, so I wanted to rule it out” (GP 2).

They liked the idea that the system suggests possible diagnoses based on

information from the patient’s health record and enables physicians to view main

signs and symptoms relating to a specific diagnosis. “Since I thought she has

bacterial infection, I clicked on it and ticked all the signs, very helpful” (physician 7).

The ‘premature closure’ alert was presented and demonstrated to participants at the

end of the evaluation, asking their opinion about its possible effectiveness. All

participants thought that it could be very useful, both for helping them not to miss

serious diagnoses and also for legal reasons (making sure they ask appropriate

questions for serious or urgent diagnoses). “Excellent idea, just by itself its great”

(physician 2); “This tool makes me think about serious diagnoses that I might miss”

(physician 6); “For legal reasons its great, because you can show that you thought


12

about these serious things, I would really like it” (physician 4).

Participants raised two main concerns relating to the DDSS effectiveness:

1. Most participants said they would like to use the system in relatively complex,

rather than straightforward cases. “I think in the Appendicitis case the tool is

not very useful, it will be in more complex cases, like the Crohn’s scenario”

(physician 2). In addition, they think that the three reasons for encounter the

system currently supports (abdominal pain, chest pain, dyspnoea) are

effective, but they would like to elaborate the use of the tool to other conditions

as well, such as: headaches, leg pains, cough, anaemia, and weight loss.

2. Two GPs raised a concern that the patients may not like the idea that the GP

uses such a tool: “Patients won’t like GPs to use it” (physician 4). However,

most participants were not concerned about this, even one of the GPs that

raised the concern gave a solution: “I will answer in the same way as I answer

when they ask me why I use GP notebook, I say: ‘I am just double checking or

looking at the latest guidelines’, and now I can say: ‘I just want to make sure I

am not missing any serious diagnoses’, and then its OK, patients won’t mind”

(physician 2).

Perceived Efficiency

Participants felt that the system was very easy to use and they had no difficulties

learning to use it and entering the information while interacting with the simulated

patient (i.e., researcher).

Specifically, participants liked the opportunity to record absence of symptoms and

signs as well as presence and to have a summary of all the information that they

gathered, including examination results: “The system focuses me on the task of

diagnosing, I know where I am and what I have to do, what I already asked and what

I still need to check, sometimes in my work there are so many distractions that I don’t

remember why the patient even came” (physician 3).

Participants understand the importance of coding information and thought that this

system could motivate them to code more than they currently do. The main concern

is that coding information into the DDSS may be time consuming, and therefore ways

to improve data coding and insertion should be considered.


13

1.4.3 Proposed changes

In view of the above findings, two main changes are proposed to the DDSS

prototype:

1. Add a confirmation dialog to prevent users from closing the DDSS by mistake

(e.g., when clicking the ‘Done’ button or the ‘X’ icon).

2. Facilitate data insertion and coding to decrease interaction time in the

following ways:

• Support use of the keyboard (in addition to the mouse) throughout the

system, to reduce interaction time,

• Facilitate entry of examination results by adding designated fields close to

the selected examination (e.g., select ‘temperature’ and enter the value).

• If the user selected a specific examination (e.g., examination of abdomen

or chest examination), the system will display a group of test results that

are relevant for the examination. For example, if the user selects:

“examination of abdomen”, the system will provide relevant examination

results (e.g., normal, tenderness, guarding, rebound, etc.) and the user will

just tick ‘Yes’ or ‘No’. Most participants in the usability evaluation coded

“examination of abdomen”, but wrote the examination results in free text.

This solution will facilitate coding the examination results and will provide

the DDSS with important information that can be used to differentiate

between the possible diagnoses.

• In future versions, as proposed by us and by one of the participants,

patients may enter the RfE and answer questions about symptoms, while

waiting for the physician. The DDSS could then take this information into

account when suggesting possible diagnoses. This solution can reduce the

amount of time the GP interacts with the system.

1.5 Summary

Section 1 summarises the design process of the DDSS from the elicitation of user

requirements to the design of a “dummy” prototype and its preliminary usability

evaluation. Initial findings are promising and suggest that the system has the

potential of being both effective and efficient.

Iterative design is important and we will continue to update and evaluate the DDSS,

and plan more usability evaluations with GPs once the prototype will be integrated to


14

Vision EHR and to the evidence database.

1.6 References

1. Hoffman, R. R., Crandall, B. W., & Shadbolt, N. R. (1998). Use of the critical

decision method to elicit expert knowledge: A case study in cognitive task

analysis methodology. Human Factors, 40(2), 254-276.

2. ISO 9241-11. Ergonomic requirements for office work with visual display terminals

(VDTs) Part 11: guidance on usability. International Organization for

Standardization (1998).

(additional references in the Appendix 1 and 2)


15

2. Specification of a functional eCRF to be integrated with an EHR

2.1 Overview

A core output of the TRANSFoRm project is the specification and demonstration of a

‘functional’ electronic Case Report Form (eCRF) [1, p. 21, WT5.2a]. The eCRF is

designed to enable the collection of semantically-controlled and standardised data

from within an EHR system.

In the context of TRANSFoRm, the eCRF specification needs to satisfy the following

requirements:

• The eCRF specification will support the collection of semantically–controlled

data. Primary care patient data in European EHR systems are generally coded

with specific clinical coding schemes such as ICPC [2], ATC [3], etc. The

eCRF specification should enable the collection of EHR data coded in different

coding schemes from various EHR systems in different countries.

• The development of the eCRF specification will follow a model-based

approach in compliance with the project philosophy of TRANSFoRm. An eCRF

model will be developed within the framework established by Clinical

Research Information Model (CRIM) [4].

• The eCRF model will enable semantic interoperability between the study

system and the EHR system, and support automated pre-population of EHR

data into study case report forms.

• The eCRF model will support the collection of data through a variety of

technologies: web interface, mobile application, as well as data collection

software embedded in an EHR system.

• The eCRF specification will support international languages in addition to

English. So when deployed in different countries, the case report forms will

present native languages to users.

• The eCRF model will cover the whole lifecycle of study data collection, and

include support for various study activities such as eligible patient identification

and recruitment, informed consent, study id allocation, randomisation, adverse

event monitoring, etc.

• The specification should have appropriate representations (such as XML) for


16

deployment within an EHR system.

• The eCRF model will be closely aligned to existing standards produced by

CDISC [5] and will be taken to CDISC for incorporation into standards. This is

essential to enable interoperability between TRANSFoRm and Clinical Trial

Data management and electronic remote data capture systems.

The eCRF model will be validated by TRANSFoRm GORD use case (D1.1). An

instance specification is developed for the GORD evaluation study. This study

specification will be tested in the TRANSFoRm federated infrastructure which

manages the deployment of the functional eCRFs in EHR systems (D7.1).

The main aim of this work is to make the eCRF forms and the data collected on them

semantically interoperable with data from EHR systems. EHR data can be used to

pre-populate eCRFs. In the following sections, we will recapitulate the GORD use

case briefly and then describe in detail our eCRF modelling methodology in section

2.2 Semantic-aware eCRF Modelling. The GORD use case will be used

throughout that section as one illustrative example to elaborate the technical details.

Finally, section 2.3 GORD Evaluation Study Specification details how the model of

GORD study is constructed by applying the modelling methodology presented in

section 2.2 Semantic-aware eCRF Modelling.

2.1.2 GORD Use Case The gastro-oesophageal reflux disease (GORD) use case was introduced in

deliverable D1.1[6]. This research use case is being used as a representative

randomised control trial. GORD is a spectrum of disorders mainly caused by the

retrograde flow of gastric acid from the stomach into the oesophagus. The main

symptoms are heartburn and acid regurgitation. GORD is treated using proton pump

inhibitors (PPI), H2-blockers or antacids. As many patients still have symptoms in

spite of PPI treatment, continuous versus on-demand PPI use regarding symptom

severity and quality of life is being compared in this study.

GORD Evaluation Study

The aim of the evaluation study is to: (1) determine the effectiveness in terms of

symptom control and quality of life (QoL) and event-initiated assessment of QoL and

symptoms in patients with GORD, randomised to either continuous or on-demand

use of PPI; and (2) to investigate whether using an electronic system to recruit study


17

participants and to collect data increases the recruitment rate in primary care clinical

studies [7], [8].

In this clustered randomised controlled trial, practices will be randomised to either

using standard trial methods (paper version) or using TRANSFoRm’s electronic tools.

In the TRANSFoRm supported arm, the GORD study will have the following timeline

(Figure 1) and data collection workflow:

Figure 1: GORD study timeline [2, Figure 2]

• Visit 1: The patient attends the primary care practice, either already taking

PPI, or as a new reflux case with a diagnosis of GORD or a symptom of

GORD. At this point, the patient will be flagged as potentially eligible for the

study. The general practitioner (GP) checks the eligibility criteria and gives

study information if the patient is eligible. If the patient consents to take part in

the study, the patient is randomised to either continuous or on-demand PPI.

o CROMs are completed by the GP in the eCRF.

o GP is informed of the randomisation outcome.

o PROMs are completed by the patient via their smartphone (via Android

or iOS mobile application) or computer (via web browser and web

application).

• Visit 2: After 8 weeks, the patient returns for a follow-up visit. CROMs and

PROMs are completed by the GP and patient respectively.

• Event: If the patient visits the practice between Visit 1 and Visit 2, CROMs for

event visits are completed by the GP.


18

• At the end of the study, participants and GPs will be asked to answer some

question on the acceptance of the TRANSFoRm system (TAM survey).

Study Data Collection Forms

All the forms that will be used for data collection (CROMs and PROMs) are available

for reference in Appendix 4: Study Data Collection Forms

• Appendix 4A CROMs Paper Version electronic Case Report Forms (CROMs)

• Appendix 4B PROMs Patient Reported Outcome Measures (PROMs)

2.2 Semantic-aware eCRF Modelling

2.2.1 Modelling Philosophy

The development of eCRF specification follows a model-based approach in

compliance with the project philosophy of TRANSFoRm. The modelling process is

based on those standard information models widely accepted in the health

informatics domain. Following is a short summary of the modelling process.

• Study timeline is modelled by CDSIC SDM.

• Case report forms are modelled by CDISC ODM.

• Clinical information available in EHR systems can be used to answer

questions in the case report forms. Automated data collection from EHR

systems requires computable definitions of those clinical data elements. The

openEHR archetype approach is used as the main mechanism to model EHR

data.

• Based on the archetype approach, a query model is developed to enable

formulation of eligibility criteria and data extraction requests, which can be

interpreted and executed by EHR systems.

• CDISC ODM is extended to embed data extraction query requests, so the

eCRFs can be pre-populated using the EHR data when they are deployed into

an EHR system.

EHR systems across Europe use heterogeneous systems to record and manage

clinical data in primary care. Clinical concepts can be semantically interoperable via

common ontologies, and mappable medical vocabularies, as has been done in the

TRANSFoRm project with the CDIM ontology and the TRANSFoRm vocabulary


19

service. In TRANSFoRm, the clinical data from EHR systems and data warehouses

need to be additionally semantically interoperable with other systems, such as the

study system, in particular the eCRF forms and the data that are collected within

them, that are common to both the clinical and research domains. eCRFs are

commonly structured according to standards, such as CDISC. There is a need to

make data elements that are common to both domains semantically interoperable.

This is to ensure that data can be extracted from EHR systems to populate eCRFs,

and also to report eCRF data back into the EHR systems to assist medical

practitioners.

In TRANSFoRm, this semantic interoperability between the clinical and research

domains is enabled by semantically extending the CDISC ODM standard to maintain

the meaning of clinical data elements between systems crossing these two domains,

as will be described in the rest of this document.

The CDISC foundational standards form the basis of a suite of standards supporting

the clinical research process from protocol through data collection, data

management, data analysis and reporting [9]. They represent interest areas that are

common across research studies, such as demographics, medication history,

medical history, adverse events. The foundational standards are shown at the top of

Figure 2.


20

Figure 2. CDISC suite of standards [9]

The file format that will be used to exchange eCRF data between the TRANSFoRm

Study System and the EHR systems is SDM as an XML document. The document

can be schematically depicted as in Figure 3 and the following sections will describe

each section in more detail.


21

Figure 3. Structure of the XML file

2.2.2 CDISC SDM

The clinical research study protocol is the plan that describes the study’s objectives,

methodology, statistical considerations, and the organisation of the study [10]. This

plan includes the design of the study, which includes the arm descriptions, the

schedule of activities, the eligibility criteria and summary information [11], [12].

Several CDISC standards represent aspects of the study design, but do not specify

the study design completely. For instance, the Operational Data Model (ODM)

represents the metadata for the data collected in the study, but does not describe the

planned timing of the study events (ODM is described further in Section 2.2.3

CDISC ODM). The Study Data Tabulation Model (SDTM) includes trial design

datasets, but only pertains to the visits, which are only part of the activity schedule.

As for the Protocol Representation Model (PRM), it is a conceptual model that

includes the study design, but has no specification details.

The CDISC Study Design Model (SDM) has been developed to specify the study

design. It extends the core ODM and consists of the following sub-components that

model the design of the study, not its execution. The SDM is modelled in XML.

• Structure: epochs, arms, cells, segments, activities

• Workflow: decision points, branches

• Timing: when activities should happen

Metadata definitions

Study design

Queries and archetypes


22

2.2.2.1 Structural Elements

Structural elements define the overall structure of the study. These are illustrated in

Figure 9, which describes parts of the GORD study. The full study design

specification for the GORD study is given in Appendix 3: GORD-SDM.xml.

• Epoch: a sequential block of time for a study, e.g. the GORD study has two

epochs for treatment and follow-up.

Figure 4. Example of Treatment Epoch element for the GORD study

• Arm: defines a study arm. The GORD study has two arms for continuous and

on-demand PPI.

Figure 5. Example of Continuous Arm element for the GORD study

• CellDef: a study cell represents the intersection of an epoch and an arm. For

example, the left study cell represents the treatment epoch, which occurs

before randomisation into continuous or on-demand arms.


23

Figure 6. Example of Treatment cell for the GORD study

• SegmentDef: a segment has a set of activities. The continuous follow-up

segment depicts the activities involved in the follow-up epoch of the

continuous arm for the GORD study.

Figure 7. Example of the Follow-up segment for the GORD study

• ActivityDef: an activity represents a point in the study at which a specific

action is to be taken. There are 8 activities in the treatment segment depicted

in Figure 9, such as trial start, inclusion/exclusion criteria check and

randomisation (Section 2.3.4).

Figure 8. Examples of the criteria check and study statistics collection activities in the GORD study


24

Figure 9. Structural elements of the GORD study

2.2.2.2 Workflow Elements

Workflow elements specify the possible participant paths through a study. They are

defined in separately from structural elements in the XML file, but they reference

objects defined in the Structure section of the document [11]. The main types of

workflow elements are as follows:

• Activities with workflow-specific roles: StudyStart, StudyFinish, PathCanFinish

• Entry and exit criteria for activities: EntryCriteria, ExitCriteria, Criterion

• Workflow paths and branching: Transition, Switch, Trigger

In the following sections, we describe the key workflow elements as they are used for

the GORD study.

• StudyStart: This identifies the single point of entry into the study for a

participant, and the associated activity can be used as an anchor for other

activities relative to the study start.

• StudyFinish: This identifies the single point at which the participant’s path

through the study has finished.

• PathCanFinish: This element allows the specification of activities for which

Treatment Epoch Follow Up Epoch

Continuous Arm

On-‐demand Arm

Trial Start Informed Consent

Study Statistics Criteria Check

Participant Account Randomise

First visit CROM

RandomiseOutcome

Final Visit CROM

Follow Up Segment

Columns shown as large rectangles are epochs

Rows marked by arrows are arms

Rectangles with bold outlines are study cells

Rectangles within the study cells are segments

Rectangles within the segments are activities

LEGEND

Final Visit PROM

Event Visit CROM

First Visit PROM


25

the workflow can provide a transition from one activity to another.

Figure 10. Activities with workflow-specific roles in the GORD study

• Transition: This element specifies a set of branches that could be followed

after an activity is completed. Transition elements are associated with

activities only. A Transition references the originating activity via the

SourceActivityOID attribute.

• Switch: Each Transition element contains one Switch element, which defines

a set of TransitionDestination elements. A TransitionDestination element

points to target activity and also references the ODM ConditionDef element,

used for workflow transition conditions. The TransitionDefault element

specifies the target activity that must take place in this transition.

Figure 11. Examples of Transition and Switch elements for the eligibility criteria check activity


26

• Trigger: Trigger elements are used to specify contingencies for unplanned

events. They denote a divergence to a new path within the referenced

structural element. In the example below, an alarm is triggered during the

follow-up segment of the continuous arm when an event visit happens. In this

transition, CROMs must be collected during this visit.

Figure 12. Example of a Trigger element in the GORD study

2.2.2.3 Timing Constraints

Timing constraints are defined in their specific Timing section in the SDM-XML

document. They determine constraints on activities and workflow transitions. Some of

the core concepts of the timing constraint are now described, with reference to one

example in the GORD study: The first visit PROM collection in the continuous arm

should take place 1 day after the outcome of the participant randomisation is known.

It is allowed to occur 1 day before or 3 days after the target time.

Figure 13. Timing constraint for the first visit PROM collection in the continuous arm


27

• Absolute and relative timing constraints: Absolute constraints limit when an

activity can take place, while relative timing constraints allow the specification

of the timing of two activities relative to one another. In Figure 13, the

randomisation outcome and the first visit PROM collection in the continuous

arm are activities relative to each other.

• Timing windows: An ideal time for the elapsed time between activities and

workflow transitions can be specified, as well as durations before or after this

ideal time. In the example (Figure 13), the pre-window duration is 1 day, while

the post-window duration is 3 days.

• Timepoint granularity: The options available are PY, PM, PD, PTH, PTM, PTS,

meaning that the activity timepoint can happen at any time in that year, month,

day, hour, minute or second respectively. In the example, the timepoint

granularity of PD means that the PROM collection can happen at any time in

that day, and is still within 1 day after the end of the first visit, when the

randomisation outcome is known to the GP.

• Timing types: The timing type describes the relationship of one activity relative

to another, such as StartToStart, StartToFinish, FinishToStart and

FinishToFinish. In the example, the timing type is FinishToStart, showing that

the PROM collection (subsequent activity) should start after the randomisation

outcome activity (preceding activity) is completed.

2.2.3 CDISC ODM

The Operational Data Model (ODM) is a vendor-neutral, platform independent format

for the interchange and archive of clinical study data [13]. The format is especially

suitable when multiple systems and data sources are involved. The clinical research

environment has existing clinical data management systems (CDMS) and many

CDMSs already support the ODM format. In this regard, within the TRANSFoRm

project, the ODM format is being used for the interchange of clinical research data.

ODM is also compatible with CDISC SDTM, which is a tabular structure.

The ODM models the study data as consisting of several types of entities:

• Item: Individual clinical data item, such as weight measurement.

• Item group: Closely related items are collected together into item groups.

• Form: A form collects a set of logically and temporally related information and

represents a page in a CRF on screen or on paper.


28

• Study event: A series of forms are collected as part of a study event.

A study protocol may be designed to have a number of planned visits by study

participants, and each visit can correspond to one or more study events. In the ODM

XML document, the metadata entities StudyEventDef, FormDef, ItemGroupDef and

ItemDef describe the types of entities that are allowed in the study.

Relevant attributes that are used in the GORD study are described below and Figure

14 shows an extract from the study definition. The GORD study file is a snapshot and

contains metadata:

• StudyOID: Clinical data entities can be identified with internal or external keys.

The StudyOID uniquely identifies a study.

• FileType: This can be Snapshot (current state of the database with no

information of the state over time) or Transactional (shows both current state

and prior states of the database).

• Granularity: This indicates the breadth of information in the document, such as

All (any type of data and metadata), Metadata (only metadata), AllClinicalData

(clinical data only).

Figure 14. ODM file attributes for the GORD study

Metadata Elements

• StudyEventDef: This element is a collection of forms that form part of

scheduled events (planned visits), unscheduled events (unplanned, e.g. study


29

termination due to serious adverse event) or common study events (e.g.

adverse events).

Figure 15 shows the definition for the unscheduled visits of participants in the

on-demand arm, when CROMs need to be collected. The Repeating attribute

indicates that this event can occur repeatedly for the participant. This study

event has only one form with reference FD.EVENT_VISIT_CROM.

Figure 15. StudyEventDef element for unscheduled visits in the on-demand arm

• FormDef: This element describes a form that is used in a study. For example,

Figure 16 shows the definition of the Event Visit CROM form, which was

referenced in Figure 15. This form does not occur repeatedly within the study

event, as indicated by the Repeating attribute. This form has references to 13

item groups.

Figure 16. FormRef element for an event visit CROM


30

• ItemGroupDef: This element describes an item group that can occur in a

study. Figure 17 shows the weight item group definition. It is a non-repeating

group that has 2 items: the weight value and the weight unit, referenced by the

ItemRef element. The transform:Query element will be discussed in Section. It

is related to how the data is extracted from the EHR source.

Figure 17. ItemGroupDef element for the item group for weight

• ItemDef: This element describes a type of item that occurs within a study. It

can have properties, such as data type, range, and codelist restrictions. Figure

18 shows the definition of the weight value item. It is a floating point number

with 2 decimal points. The Question element shows the text that prompts a

user to provide the data for this item. The RangeCheck element constrains the

value to > 0, with an ErrorMessage element to display an error when the range

check is violated.

The transform:CdimBinding element at the end of ItemDef (line 15) will be

described in Section 2.2.4.1 Semantic extension to ODM. This links the weight

item to the ontology of clinical primary care domain.


31

Figure 18. ItemDef element for the weight item

2.2.4 Detailed Clinical Modelling Approach and Archetypes

In the previous section, we described how clinical research studies are defined. In

TRANSFoRm, the reuse of routinely collected clinical data will be achieved by pre-

populating eCRFs with available clinical data directly from EHR systems. For this to

be possible, the clinical data needs to be semantically interoperable between the

eCRF (TRANSFoRm study system) and the EHR system. We adopt a two-level

modelling approach [14]–[16] to separate out the more stable domain information

from the various schema implemented by the heterogeneous data sources [17].

Detailed Clinical Models (DCM) organise health information by combining knowledge,

data element specification, relationships between elements, and terminology into

information models that allow deployment in different technical formats [18], [19].

DCM enables semantic interoperability by formalising or standardising clinical data

elements which are modelled independently of their technical implementations. The

data elements and models can then be applied in various technical contexts, such as

EHR, messaging, data warehouses and clinical decision support systems.

Within the TRANSFoRm project, the two-level modelling approach of DCM is

depicted on the first level as an information model, the Clinical Research Information

Model (CRIM) [4], which defines the workflow and data requirements of the clinical

research task, combined with the Clinical Data Integration Model (CDIM) [20], [21],

an ontology of clinical primary care domain that captures the structural and semantic


32

variability of data representations across data sources. This separation of the

information model from the reference ontology has been previously described by

Smith and Ceusters [22]. At the second level, archetypes are used to constrain the

domain concepts and specify the implementation aspects of the data elements within

EHR systems or patient registries. We use the Archetype Definition Language (ADL)

to define the constraints and combine them with CDIM concepts in specifying the

appropriate data types and range values. The two-level modelling approach, using

the concept of archetype for detailed clinical content modelling, has been adopted by

ISO/CEN 13606 [23], [24]. Based on the OpenEHR framework [25], this approach

makes it possible to separate specific clinical content from the software

implementation. The technical design of the software is driven by the first level

information model which specifies the generic information structure of the domain.

The archetype defines the data elements that are required by specific application

contexts e.g. different clinical studies.

2.2.4.1 Semantic extension to ODM

A study participant’s clinical data space is represented by a generic reference model.

The reference model is based on TRANSFoRm Clinical Research Information Model

(CRIM), which is based on the Biomedical Research Integrated Domain Group

(BRIDG) model of the semantics of protocol-driven clinical research [26]. Data

elements retrieved from EHR sources are considered as

PerformedObservationResult class (BRIDG class). PerformedObservationResult is

extended so its attribute is of type Element (openEHR class). The tabular structure

PerformedObservationResult – Element is compatible with CDISC SDTM and ODM

forms. Figure 19 shows how ODM can be semantically extended via the

ItemGroupDef and ItemDef elements to be interoperable with EHR clinical data that

are represented in CRIM reference model, with a binding to the CDIM ontology.


33

Figure 19. Link between ODM and the Reference Model via archetypes

For example, in Figure 18, we mentioned the transform:CdimBinding element for

form item Weight. This element is a TRANSFoRm extension to ODM, for semantic

interoperability. The weight concept is identified in the CDIM ontology as concept

CDIM_000068. This binding enables to maintain the link between the form item and

the weight value after it is retrieved from the EHR system for pre-population.

In Figure 17, the last element in the item group definition for Weight is a

transform:Query element, which is also a TRANSFoRm extension to ODM:

This query ID refers to a data extraction request for weight value, unit and time of

measurement, as shown in Figure 20, indicated by the three CdimConcept elements.


34

Figure 20. Part of the data extraction query for weight

The request for data extraction of the three weight-related data items is accompanied

by constraints that specify how the data should be filtered. These are specified in the

Archetype section of the query, as shown in Figure 21 (definition), Figure 22

(ontology term definitions) and Figure 23 (ontology term bindings).

Figure 21. Archetype for weight - definition


35

Figure 22. Archetype for weight - ontology definitions

Figure 23. Archetype for weight - ontology term bindings

2.2.5 Archetype-based Query Model for Eligible Patient Identification and Patient Data Extraction

The archetype approach establishes the semantic foundation for describing clinical

data elements, and enables the semantic interoperability across different EHR

systems. Based on the archetypes, a query model is developed to facilitate query

formulation for eligible patient identification and patient data extraction.


36

As described in Figure 24, the query model allows formulating three different types of

query requests and their responses, including

-‐ EligibleSubjectCountRequest, requesting for counts of eligible subjects. Its

response returns the count of found eligible subjects. -‐ FlagEligibleSubjectRequest, flagging eligible subjects. Its response returns

the count of flagged subjects. -‐ DataExtractionRequest, retrieving data from the source for the flagged

subject(s). Its response returns the requested data.

All these query requests can embed query criteria. The query criteria allow using

archetypes for clinical data specifications, and allow doing arithmetic calculations and

calling functions (Figure 25). Furthermore, complex Boolean logic and temporal

constraints can be constructed in the query criteria. The actual queries formulated by

the model are encoded in xml format. Figure 20 - Figure 23 describe an example of

DataExtractionRequest which requests for the patient weight using the TRANSFoRm

archetype weight as the data element specification.


37

Figure 24. Archetype-based query model – main elements


38

Figure 25. Query model - expressions


39

2.3 GORD Evaluation Study Specification

An instance specification for the GORD evaluation study is developed by applying

the modelling methodology and all the models described in previous sections. The

study is encoded in CDISC SDM with a number of TRANSFoRm extensions to cover

the missing functions such as form pre-population, randomisation, mobile interface

rendering, and alarm symptom monitoring. In the GORD SDM XML, all the

TRANSFoRm extensions are defined under the namespace transform and are

denoted in the form of <transform:XXX>.

2.3.1 Study Workflow

The GORD evaluation study is a single-blind, multi-centre, parallel-arm, cluster

randomised controlled trial to assess the effectiveness of the TRANSFoRm tools in

recruiting patients into a study in primary care and obtaining complete case data. The

complete workflow is depicted using a flowchart in Figure 26. The study has two

arms: on-demand or continuous prescribed use of PPI. Participants and clinicians will

not be blinded as it is the dosage regimen that differs between groups and both

patients and clinicians will see the prescribed dosage. However, the analysis of the

study will be blinded. The study will span 32 weeks - 24 for the recruitment of

participants and 8 weeks for follow-up.

Following CDISC SDM, the study is designed to comprise two epochs (Figure 4):

• Treatment epoch – participant is randomised and PPI is prescribed.

• Follow-up epoch – participant is followed up with PROMs and CROMs.

The intersection of the two epochs with the two arms produces three cells (Figure 6),

each of which contains one segment:

• Treatment cell represents the part of study (i.e. treatment epoch) with only

one path, which is same for both arms. Treatment cell contains the treatment

segment.

• On-demand follow-up cell represents the follow-up epoch for the on-demand

prescribed use of PPI arm and contains the on-demand follow-up segment.


40

• Continuous follow-up cell represents the follow-up epoch for the continuous

prescribed use of PPI arm and contains the continuous follow-up segment.

Figure 26. GORD evaluation study workflow

Figure 9 presents the full picture of the study structure, composed of epochs, arms,

cells and segments. Examples are presented in Figure 4 - Figure 7. Segments are

the basic building blocks of study design. A segment usually specifies a combination


41

of planned observations and interventions, which may or may not involve treatment,

during a period of time. A segment includes one or more activities.

In the design specification of the GORD study, the treatment segment contains all

the activities that will be carried out during the patient’s first visit (visit 1 week 0) to

the primary health care centre (PHC), including:

• Trial start: the patient will be flagged as a potentially eligible subject if the

patient presents related diagnosis, symptoms or prescriptions. The trial start

activity embeds a flagging patient query which will trigger the study when the

patient is flagged.

• Check inclusion/exclusion criteria: after the patient is flagged and the study

is triggered, the GP is asked to check the inclusion and exclusion criteria.

• Collect study statistics: if the patient is not eligible, anonymized information

of age and gender is saved as study statistics.

• Informed consent: if the patient is eligible, the GP gives the patient the study

information.

• Randomisation: if the patient consents, the GP confirms the age and gender.

This information is used to allocate patients to randomisation blocks in the

randomization. After completing this point, the patient is randomized (but the

outcome is hidden from the GP until the end of the consultation when the dose

is prescribed to avoid biasing the GP).

• Create study account for the patient: at this point an ID, user name and

password is created for the study participant.

• First visit CROM: the GP checks the prefilled information and adds missing

information in the CROM.

• Randomisation outcome: when the CROMs are completed, the GP will be

informed of the randomization outcome and can prescribe the appropriate

dose. The participant will receive information about when and how to fill out

the PROMs. The GP/PHC books a follow up appointment with the participants

8 weeks later. After this point, the study enters the follow-up period and is

running in the two parallel arms.

The on-demand follow-up segment contains the activities that will be carried out

during the follow-up period for the on-demand arm, which include:

• First visit on-demand PROM: The PROMs can be filled out on the patient’s


42

smart phone or on a computer with Internet access and should be completed

within 3 days of visit 1. The patient will receive reminders 2 days, 3 days and 4

days after visit 1.

• Final visit on-demand CROM: when the participant returns for the follow-up

visit (visit 2) after 8 weeks, the CROMs are activated and the GP checks out

prefilled fields and completes any missing information.

• Final visit on-demand PROM: the participant completes the PROMs within 3

days after visit 2. Similarly, the patient will receive reminders 2 days, 3 days

and 4 days after visit 2.

• Event visit on-demand CROM: if the participant visits the PHC between visit

1 and visit 2, CROMs for event visits are activated. The recording of event

visits between visit 1 and visit 2 is to make sure that the number of PHC

contacts doesn’t differ between the treatment regimes.

The continuous follow-up segment contains similar activities to those of the on-

demand follow-up segment, whereas the activities are specifically associated with the

continuous arm.

All the activities described above are defined in the study SDM XML as

<sdm:ActivityDef >. One example has been presented in Figure 8. The execution

order between these activities is defined in the SDM workflow section

<sdm:Workflow>. One example is presented in Figure 11. The timing constraints are

defined in the SDM timing section <sdm:Timing>. One example has been presented

in Figure 13. Note that SDM XML allows only explicit timing constraints between

activities not study events. The 8-week constraint between visit 1 and 2 is therefore

defined between the last activity of visit 1 (Randomisation Outcome) and the first

activity of visit 2 (Final Visit CROM) for either arm.

2.3.2 Case Report Forms

The GORD study data consists of two parts: care-reported outcome measures

(CROMs) and patient-reported outcome measures (PROMs). CROM represents the

category of all types of objectively reported care outcomes, collected at the

physician’s office including data from the patient’s electronic health record, such as

BMI, blood pressure, laboratory data. PROM represents the category of all types of

subjective reported outcomes such as health-related quality of life, treatment


43

satisfaction, subjective health status and subjective symptoms.

All study parameters (CROMs, PROMs) are assessed with electronic questionnaires.

No invasive procedures, physiological investigations, clinical or laboratory tests will

be performed as part of this study. GPs report care reported outcome measures

(including age, gender, BMI, smoking, PPI consumption and medical history) at the

time of inclusion, in case of events and at the 8 week follow-up. Participants complete

patient reported outcome measures (including demographics, GORD symptoms,

health related quality of life, self-rated health and use of GORD medication) through

mobile phone or web at the time of inclusion and after 8 weeks. Appendix 2 includes

paper versions CRF for both CROM and PROM.

Based on the study plan and paper versions CROM and PROM, the SDM XML

encoded CROM forms for visit 1, visit 2 and event visit, PROM forms for the on-

demand arm visit 1 & 2, and PROM forms for the continuous arm visit 1 & 2. CROM

forms are all same for both arms, whereas PROM forms are different between the

two arms. CROMs will be deployed into an EHR system and triggered from GP

desktop, while PROMs can be accessed through mobile phone or web form and can

be used by the patient at any location (e.g. from home). Because CROMs and

PROMs are technically managed and rendered by different software components, a

TRANSFoRm extension <transform:FormType> is introduced into <FormDef> to

indicate how this form should be processed. Examples of <FormDef> and its internal

elements are presented in Figure 16 - Figure 18.

In addition to the CROM and PROM forms, the SDM XML includes a number of other

forms as they are used in various study activities, including inclusion/exclusion

criteria form, study statistics form, randomisation form to capture input to the

randomisation service, and randomisation outcome form to inform GPs about the

randomisation result.

2.3.3 Flagging Patients and Form Pre-population

When a patient visits the primary care centre for any complaint, the patient will be

flagged by the system as a potentially eligible patient for the GORD study if

1. There is a GORD diagnosis or a diagnosis of symptom of GORD i.e. heartburn

or acid regurgitation already in the EHR


44

OR

2. The GP enters a GORD diagnosis or diagnosis of symptom of GORD i.e.

heartburn or acid regurgitation during the consultation

OR

3. A PPI has been prescribed or is prescribed.

AND

The patient is aged between 18 and 65 years.

These criteria are formulated in a FlagEligibleSubjectRequest query. This flagging

query is embedded in the Trial Start activity and will trigger the execution of the

GORD study once the patient is flagged.

As described in section 2.3.2, CROM forms are deployed and triggered within an

EHR system. When a CROM form is triggered, the form is either presented in a

stand-alone application on the GP’s desktop, such as a web browser, or embedded

with the EHR user interface. Certain data items can be retrieved from the GP’s EHR

system, and can be used to pre-populate related questions in the form. Those pre-

populated fields however have to be manually checked (and potentially edited) by the

GP before the form is submitted. The following table Table 1) summarises the data

elements which are expected to be available from GP’s EHR system and so can be

used to pre-populate relevant GORD CROM forms.

Table 1 lists the data elements, their archetypes and the visits (marked by ‘X’) at

which they will be collected. Some data items are collected only at the time of first

visit, while some others have to be collected every time. The data elements are listed

together with their identifying terminology codes, where prescriptions use ATC codes

and diagnosis use ICD-10 codes. Data extraction queries for retrieving these data

elements are formulated as DataExtractionQueryRequest for each of them. All the

queries are defined in the <transform:Queries> section in the SDM XML, and are

linked to the corresponding <ItemGroupDef> elements in the related ODM forms

through UUID, following the approach elaborated in Section 2.2.4.1 Semantic

extension to ODM. Figure 16 - Figure 23 have presented the CROM question, the

query, and their linkage, using the data element weight as an example.


45

Table 1. EHR data elements for CROM form pre-population

Data Element Archetype Visit 1 Visit 2 Event Visit

Birth year (YYYY) Date of Birth X

Gender Gender X

Height Height X

Weight Weight X X X

PPI use (A02BC) Prescription X

Antacids (A02A)/alginate (A02BX13)

Prescription X X X

Esophagitis (K20) Diagnosis X X X

Barrett’s oesophagus (K22.7) Diagnosis X X

2.3.4 Randomisation

Study participants will be randomized to either on-demand or continuous use of PPI

for 8 weeks. Patients randomized to on-demand use will be prescribed PPI in the

presence of symptoms but no more than 40 mg omeprazole per day. Patients

randomized to continuous use of PPI will be prescribed 20 mg omeprazole per day.

This will be a block randomization containing 4 blocks based on age (50 years and

below, and above 50 years) and gender. The study will not be blinded as it is the

dosage regimen that differs between groups so there won’t be a need to break the

randomization code.

The input parameters to the block randomisation function (i.e. age and gender) are

captured by the Randomisation Form which will be triggered by the randomisation

activity at patient’s first visit. Collected data are sent to the TRANSFoRm Study

System which assigns the patient to a block and applies the randomisation function.

The outcome is hidden from the GP until the end of the consultation when the dose is

prescribed. The outcome is then informed through the Randomisation Outcome

Form.

The TRANSFoRm randomisation function requires proper configuration of the

blocking variables in the study definition. In the case of the GORD study, the blocking


46

variables are year of birth and gender. Because the current version of SDM XML

does not cover randomisation design, the blocking variables have to be specified in a

TRANSFoRm extension element <transform:ParticipantSegmentVariables>. As

shown in Figure 27, the variables year of birth and gender are linked to the

corresponding fields in the Randomisation Form through the ItemOID, and their

range definitions specify the group arrangement. Another parameter required by the

randomisation function is the estimated number of recruited participants. The number

is used to initialise the random allocation sequence. This parameter is defined as a

<sdm:Parameter> in the <sdm:Summary> section (Figure 28).

Figure 27. Randomisation blocking definition


47

Figure 28. Study parameter – planned number of recruited subjects

2.4 Conclusion

In Section 2. Specification of a functional eCRF to be integrated with an EHR, we

described how a functional eCRF is made semantically interoperable with an EHR

system to enable the pre-population of eCRFs with available primary clinical data. A

two-level modelling approach has been adopted, with the first level being guided by

the reference model for the clinical research domain (CRIM), and on the second

level, archetypes are used to constrain the reference model. Clinical data elements

have a binding to the CDIM ontology that describes the primary care clinical domain.

To maintain the semantic meaning when using eCRFs, the CDISC ODM standard

has been extended to allow for archetypes to further define form elements and

queries to EHR systems to extract specific clinical data items.


48

2.5 References

[1] TRANSFoRm Consortium, “TRANSFoRm Annex I - Description of Work, Version 4.” 08-Nov-2013.

[2] World Health Organization, “International Classification of Primary Care, Second edition (ICPC-2),” WHO. [Online]. Available: http://www.who.int/classifications/icd/adaptations/icpc2/en/. [Accessed: 21-May-2014].

[3] WHO Collaborating Centre for Drug Statistics Methodology, “ATC Structure and principles.” [Online]. Available: http://www.whocc.no/atc/structure_and_principles/. [Accessed: 21-May-2014].

[4] W. Kuchinke, T. Karakoyun, and C. Ohmann, “Clinical Research Information Model,” Project Deliverable TRANSFoRm Deliverable D6.2, V1.0, May 2012.

[5] Clinical Data Interchange Standards Consortium, “CDISC.” [Online]. Available: http://www.cdisc.org/. [Accessed: 21-May-2014].

[6] P. Leysen, H. Bastiaens, P. van Royen, L. Agreus, and A. N. Andreasson, “D1.1 Detailed Use Cases,” TRANSFoRm Project Deliverable, Feb. 2011.

[7] A. N. Andreasson, R. A. Verheij, and K. Hek, “The effect of on demand versus continuous use of proton pump inhibitors on reflux symptoms, quality of life and self-rated health in patients with gastro-oesophageal reflux disease,” Evaluation Study Research Ethics Protocol Protocol ID KIANANDR140701 (draft), Apr. 2014.

[8] N. Mastellos, K. Huckvale, M. Larsen, J. Car, A. N. Andreasson, and L. Agreus, “Protocol for evaluation studies for the GORD use case.” Mar-2014.

[9] Clinical Data Interchange Standards Consortium (CDISC), “CDISC Foundation Standards.” [Online]. Available: http://www.cdisc.org/foundation-standards. [Accessed: 23-Apr-2014].

[10] Clinical Data Interchange Standards Consortium (CDISC), “Representation Model for Planning and Design of Medical Research Protocol.” [Online]. Available: http://www.cdisc.org/protocol. [Accessed: 23-Apr-2014].

[11] Clinical Data Interchange Standards Consortium (CDISC), “CDISC Study Design Model in XML (SDM-XML) Version 1.0,” 2011.

[12] Clinical Data Interchange Standards Consortium (CDISC), “Study/Trial Design Model: Study Design in XML Version 1.0.” [Online]. Available: http://www.cdisc.org/study-trial-design. [Accessed: 23-Apr-2014].

[13] Clinical Data Interchange Standards Consortium (CDISC), “Operational Data Model.” [Online]. Available: http://www.cdisc.org/odm. [Accessed: 24-Apr-2014].

[14] A. L. Rector, W. A. Nowlan, S. Kay, C. A. Goble, and T. J. Howkins, “A framework for modelling the electronic medical record,” Methods Inf. Med., vol. 32, no. 2, pp. 109–119, Apr. 1993.

[15] S. B. Johnson, “Generic data modeling for clinical repositories.,” J. Am. Med. Inform. Assoc., vol. 3, no. 5, pp. 328–339, 1996.

[16] T. Beale, “Archetypes: Constraint-based domain models for future-proof information systems,” 2002. [Online]. Available: http://www.openehr.org/files/resources/publications/archetypes/archetypes_beale


49

_oopsla_2002.pdf. [17] S. N. Lim Choi Keung, L. Zhao, J. Rossiter, M. McGilchrist, F. Culross, J.-F.

Ethier, A. Burgun, R. Verheij, N. Khan, A. Taweel, V. Curcin, B. Delaney, and T. N. Arvanitis, “Detailed Clinical Modelling Approach to Data Extraction from Heterogeneous Data Sources for Clinical Research,” in 2014 Summit on Clinical Research Informatics, San Francisco, USA, 2014.

[18] W. Goossen, A. Goossen-Baremans, and M. van der Zel, “Detailed Clinical Models: A Review,” Healthc. Inform. Res., vol. 16, no. 4, pp. 201–214, Dec. 2010.

[19] W. T. F. Goossen and A. Goossen-Baremans, “Bridging the HL7 template - 13606 archetype gap with detailed clinical models,” Stud. Health Technol. Inform., vol. 160, no. Pt 2, pp. 932–936, 2010.

[20] J.-F. Ethier, O. Dameron, V. Curcin, M. M. McGilchrist, R. A. Verheij, T. N. Arvanitis, A. Taweel, B. C. Delaney, and A. Burgun, “A unified structural/terminological interoperability framework based on LexEVS: application to TRANSFoRm,” J. Am. Med. Inform. Assoc., Apr. 2013.

[21] J.-F. Ethier, M. M. McGilchrist, A. Burgun, and F. Sullivan, “D6.3 Data Integration Models,” TRANSFoRm Deliverable, May 2013.

[22] B. Smith and W. Ceusters, “HL7 RIM: an incoherent standard,” Stud. Health Technol. Inform., vol. 124, pp. 133–138, 2006.

[23] EN 13606 Association, “The CEN/ISO EN13606 standard.” [Online]. Available: http://www.en13606.org/the-ceniso-en13606-standard. [Accessed: 25-Apr-2014].

[24] P. Muñoz, J. D. Trigo, I. Martínez, A. Muñoz, J. Escayola, and J. García, “The ISO/EN 13606 Standard for the Interoperable Exchange of Electronic Health Records,” J. Healthc. Eng., vol. 2, no. 1, pp. 1–24, 2011.

[25] openEHR Foundation, “openEHR.” [Online]. Available: http://www.openehr.org. [Accessed: 25-Apr-2014].

[26] BRIDG Founding Stakeholders, “BRIDG Model.” [Online]. Available: http://www.bridgmodel.org/. [Accessed: 25-Apr-2014].


50

Appendix 1: User Requirements Elicitation (manuscript)

Eliciting user decision requirements for designing computerised diagnostic support for GPs

Talya Porat, Amanda Woolley, Olga Kostopoulou

Department of Primary Care and Public Health Sciences, School of Medicine, King’s College London

Correspondence: Olga Kostopoulou, King’s College London, School of Medicine, Department of Primary Care and Public Health Sciences, Capital House, 42 Weston Street, London SE1 3QD, UK

E-mail: [email protected] Tel: +44 (0)20 7848 6757 Fax: +44 (0)20 7848 6620


51

ABSTRACT

Background: Despite its 40-year history, computerised diagnostic support is not

commonly used in medicine.

Aim: To identify user decision requirements for the design of a computerised diagnostic support tool for GPs.

Design and Setting We employed the decision-centred design framework of cognitive systems engineering. All data were elicited from GPs in London and pertained to consultations with patients, either real or simulated.

Methods: To collect background information about the domain, we conducted in situ observations and interviews with 8 GPs and performed a hierarchical task analysis of the diagnostic task. To identify key components of expert decision making, we used cognitive task analysis on 58 existing verbal protocols: 17 GPs thinking aloud while seeking information to diagnose cases on computer and 18 GPs describing cases that they diagnosed intuitively.

Results: To support the decision requirements of the diagnostic process that we identified, we propose six solutions for the design of computerised tools: (1) Suggest possible diagnoses to consider early on in the consultation before hypothesis testing starts. (2) Provide checklist of important, diagnostic symptoms and signs for the suggested diagnoses. (3) Make salient important information in the patient’s record. (4) Suggest examinations and investigations. (5) Alert when GPs have not checked for serious or urgent diagnoses. (6) Facilitate coding and data entry.

Conclusion: Studies employing multiple human factors techniques and data types

are rare. Our approach enabled us to propose interface design solutions that go

beyond existing ‘differential diagnosis generators’, aiming to improve acceptance of

the resulting tool by GPs.

Keywords: General Practice; Clinical Decision Support Systems; Decision making;

Diagnostic Errors


52

HOW THIS FITS IN

We provide a new perspective on the design of diagnostic support beyond the well-

known differential-diagnosis generators. We emphasise the need for early and

interactive support, coupled with facilitated data entry and coding throughout the

consultation. The support tool should make the diagnostic task more efficient and

less error-prone and should reduce the GPs’ burden of coding. This would result in a

richer and easily searchable health record and increase the tool’s perceived benefits

and likelihood of use by GPs in their everyday practice.


53

INTRODUCTION

Diagnostic errors are common, harmful to patients and costly. They are the cause of

preventable mortality and morbidity and the commonest cause of litigation against

GPs in both the UK and USA.1 2 Computerised diagnostic support has been one of

the strategies proposed for the reduction of diagnostic error.3 However, more than 40

years after De Dombal’s AAPHelp,4 the use of computerised diagnostic support is not

widespread.

One large roadblock to acceptance of diagnostic support is lack of integration with

the physician’s workflow and the electronic patient record (eHR).5-7 Commercial

diagnostic tools, such as DXplain™, Isabel, and SimulConsult, are stand-alone

applications, requiring physicians to switch from their eHR to the tool and add

information twice, costing precious time. Moreover, physicians must first decide to

consult the system, though they may not recognise when they need advice.8 Finally,

these systems are designed to be used after the physician has collected as much

information as possible from the patient, which the system will need in order to

provide diagnostic suggestions (‘differential diagnosis generators’). Thus, advice

comes late in the diagnostic process, and the information that physicians enter into

the system depends on the initial hypotheses considered. “The system’s advice, and

thus its potential value, depends on how users can convey to the DSS their personal

understanding of a case by selectively entering clinical findings…”.9 A recent

experimental study found that presenting GPs with differential diagnoses to consider

early on in the consultation, before they start testing hypotheses, significantly

improved diagnostic accuracy over control.10 Taking all this evidence into account,

we aimed to elicit decision requirements for the design of a diagnostic tool for GPs

that is being developed as part of the TRANSFoRm project (transformproject.eu).

To elicit requirements, we employed a multi-method approach encompassing a range

of human factors techniques and data sources.

Below, we present the methods and associated results, while the table of user

requirements and associated design solutions is presented in Appendix 1.


54

METHODS AND ASSOCIATED RESULTS

We employed a decision-centred design (DCD) framework of cognitive systems

engineering that advocates focusing on difficult key decisions and non-routine

situations, where errors may lead to injury and/or death.11 It is therefore suitable for

designing support for medical diagnosis. DCD uses cognitive task analysis (CTA)

techniques to identify the key decisions.12 It then translates these into requirements

that guide the design of solutions for supporting decision making in challenging

situations, assuming that the routine requirements will be incorporated along the way.

The DCD framework includes five stages. Here, we describe the first three stages:

preparation, knowledge elicitation, and analysis and representation.

Stage 1: Preparation

The preparation stage seeks to gather background material about the domain that

will inform about the nature and range of the tasks involved and identify cognitively

complex task elements. Preparation started with reviewing an existing hierarchical

task analysis (HTA) of the primary care consultation.13 HTA models tasks as

hierarchies of goals and sub-goals, with plans that show how sub-goals should be

carried out.14

We aimed to refine the parts of the HTA that related to diagnosis. For this purpose,

we observed 8 GPs (5 male, mean 8.6 years in general practice, SD 6) consulting

with their patients and offered them recompense at the standard clinical hourly rates.

The researcher (TP) sat in the consulting room and unobtrusively observed the

clinical interaction, taking notes. A total of 104 clinical encounters (23.5 hours) were

observed. The researcher noted how the GPs used their eHR, for which tasks and at

which stage in the workflow. Notes from all observations were compared to gain a

better understanding of the information requirements of the diagnostic task.

GPs were then interviewed about their diagnostic strategies in the cases observed

earlier, past diagnostic errors and suggestions how diagnostic support could help

avoid these errors. On the basis of the observations and follow-up interviews, we

refined and expanded the diagnostic component of the HTA (Figure 1).


55

0. Provide consultation with GP

PLAN 0: Before starting surgery do 1, 2, 3 and 4 in any order and combination. If patient has arrived and when ready to receive patient - 5. When patient comes into the consultation room – 6 – 7 – 8. If do not know or cannot remember patient’s information go back to 4. Throughout the consultation do 11. If decided on course of action do 9 and 10. If further information regarding present visit needs to be obtained – 7. If next patient arrived and when ready to receive patient – 5 and carry on as for previous patient.

1. Review the list of appointments

2. Check whether first patient has arrived (system indication)

3. Open patient health record

4. Get familiar with patient’s clinical history

5. Call patient in

6. Establish why patient has come

7. Obtain new information PLAN 7:

Do 7.1. As appropriate and if available – 7.2 and 7.3, in any order and combination. 7.1 Obtain new information from patient PLAN 7.1 Do any 7.1.1 to 7.1.4 in any order and combination, as appropriate.

7.1.1. Obtain information verbally 7.1.2. Obtain information via observation 7.1.3. Obtain information via physical examination 7.1.4. Obtain information via near-patient testing (e.g., dipstick urine analysis)

7.2 Review relevant pathology results 7.3 Review relevant clinical correspondence

8. Decide on course of action

9. Safety net

10. Carry out next steps

11. Record consultation

Figure 1: An extract from the refined HTA about the clinical consultation, with associated plans and goals. For illustration purposes, only goal 7.1 is re-described.

Stage 2: Knowledge elicitation

The knowledge elicitation stage uses cognitive task analysis (CTA) methods to elicit

critical incidents and key components of expert decision making. For this purpose, we

analysed existing think-aloud protocols of GPs diagnosing patient cases presented

on computer and interview protocols of GPs describing past cases of intuitive


56

diagnoses.

1. Think-aloud protocols

We used data from an ongoing study by the last author that investigates how GPs

deal with early presentations of cancer (lung, colorectal, and myeloma). Participating

GPs viewed a series of patient cases on computer. After some initial information

about the patient and the reason for encounter, GPs requested further information in

order to diagnose. They could take a history and request results of physical

examinations and laboratory tests. A researcher provided responses from a

predetermined list. Furthermore, participants were instructed to think out loud.15 We

used the first 34 think-aloud protocols from this study: 11 pertained to lung cancer, 11

to myeloma and 12 to colorectal cancer.

We used the situation assessment record (SAR) method to analyse the protocols.16

In SAR, the timeline for an event specifies the points at which the expert engaged in

situation assessment and decision making. Each of the events may be defined in

terms of cues (information items), expectations, hypotheses, strategies, goals and

decision points (See Table 1 for an excerpt from the SAR analysis. The scenario

features the first visit of a patient with early myeloma).

TABLE 2: Excerpt from the SAR analysis of a think-aloud protocol of GP 9 (cancer diagnosis study)

Situation Assessment 1 Cues 67, female, a bit on the heavy side, has a history of blood

pressure, hypertension and arthritis. She’s on medication for her blood pressure and she’s been seen with a back problem relatively recently. Referred for physio and note that she attends quite frequently. Seems to be well but is holding her back and does seem to be in pain. So she’s talking about having back pain. She’s taken cocodemol and the pain hasn’t got better.

Expectations I’m assuming that she’s wanting better pain relief. She is already taking co-codamol, she’s had the physio.

Goal Ask about the physio / explore the reason for encounter

Decision point 1 “Did you feel that the physiotherapy made any difference or whether the treatment is still ongoing?”

Situation Assessment 2 Cues “I've seen the physio twice now. She recommended some

exercises to do at home. I try to do them every day. They keep me active but I am not sure how much they are helping. I have some follow up sessions booked”.

Hypotheses 1 and 2 “So it may be that she needs to be encouraged to be a little bit more patient or I’m thinking about disc prolapse”

Goal Differentiate between hypotheses 1 and 2


57

Decision point 2 “Has the pain got any worse?”

Situation Assessment 3 Cues “After the pain started it has been the same constant aching

most days.” Expectations “I think she’s only had two sessions of the physio, she’s only

been doing the exercises at home for a little while. I’m thinking she probably just needs to be a bit more patient”.

Hypothesis 2 Mechanical pain (“So it does sound like it’s mechanical low back pain”).

Goal Increase medication to control the pain

Decision point 3 “Ask her how much of the co-codemol she’s taking and if she’s getting any problems with it such as constipation”.

Situation Assessment 4 Cues “I have been taking the co-codamol tablets, as I was told,

mostly 8 per day. They take the edge off the pain but it doesn't go completely”.

Hypothesis 2 Mechanical low back pain Goal “She’s taking up to the maximum dose so I don’t particularly

want to change anything there”.

Decision point 4 “She should continue with her exercises and give the physio a bit more chance, so I’ll ask her to come back if it doesn’t settle and to continue with her physio”.

2. Interview protocols based on the Critical Decision Method

We also used data from a completed study by the second and third authors, where

GPs were interviewed about patient cases that they believed to have diagnosed by

intuition.17 We used these data in order to gain an understanding of cognitive

processes during intuitive decision making in addition to the more analytical

reasoning approach of the think-aloud protocols. At the interviews, the researchers

prompted the GPs systematically, following the Critical Decision Method (CDM) that

has been used in numerous domains to investigate the cognitive components of

proficient performance.18 They thus collected 24 CDM protocols from 18 GPs, which

we re-analysed using SAR, to enable comparison with the analysis of the think-aloud

protocols (See Table 2 for an excerpt from the SAR analysis. The scenario features a

patient with ovarian cancer).

TABLE 3: Excerpt from the SAR analysis of a CDM protocol of GP 8 (intuition study)

Situation Assessment 1 Cues 67, female, not happy with previous consultation, felt unwell,

tired with no energy, dizzy, loss of appetite, weight loss (a stone over a year), overweight, ex-smoker, previous tests were all normal. Non-diagnostic cues: “A very, very frequent attender… and has multiple social and psychological problems. And she wears you out…”


58

Expectations She lost a stone in weight over the last year, less concerning since the weight loss was over a long period of time (“but if you lost that sort of weight over a month or two or three, then you’re going to be worried”), in addition, “she is a biggish lady, and therefore a stone didn’t feel that much in that sense. But she’d never lost weight before. And not many 67 year olds go on a diet. I mean, young women and men do, but not a 67 year old”.

Decision point 1 Perform a full examination

Situation Assessment 2 Cues Abdomen normal, chest normal (full examination normal) Expectations “I think my guts told me that there wasn’t anything too much

wrong with this lady” Hypothesis 1 Seemingly nothing is wrong, possibly irritated bowel Goal Confirm nothing is wrong with blood tests

Decision point 2 Perform screen tests (to look at all options): blood tests

(ESR, kidney and liver, sugar, thyroid, FBC) and chest x-ray.

Situation Assessment 3 Cues Blood tests normal, chest x-ray normal Expectations Given problem with previous consultation best to follow

through to avoid upset: “I did pick up a feeling in her voice that she was unhappy with the previous consultation with my partners, and I thought by following her through and making sure that nothing went wrong, as it were, that we might avoid any complaint or upset. So that was really why I brought her back”.

Hypothesis 2 Nothing serious Goal Arrange a follow-up

Decision point 3 Arrange an appointment in two weeks

Stage 3: Analysis and representation

The HTA and the analyses of these different data types enabled us to elicit cognitive

requirements (Table 3) for which we propose user interface design solutions. The

proposed solutions are summarised in a decision requirements table (Appendix A),

as suggested by the DCD framework.

TABLE 4: Cognitive requirements of the diagnostic process

1 Retrieval and integration of information 1.1 Retrieval of relevant and important information from the health record 1.2 Integration of the patient’s reason for encounter with relevant information from the

health record 2 Diagnostic hypotheses generation 3 Hypothesis testing and refinement (including decisions about what and how much

information to elicit) 3.1 What questions to ask the patient 3.2 If and what examinations to perform 3.3 If and what investigations to perform or order


59

3.4 How to interpret the information elicited 3.5 When to stop eliciting information (define a stopping rule) 4 Deciding on a course of action

1. Retrieval and integration of information

GPs must retrieve and integrate information from different sources to build an

understanding of the patient’s condition. They scan the patient’s health record

looking for significant information, either by browsing in a structured way (problems,

medications, previous encounters) or by actively filtering information (display only

high priority’ problems). Important information may be missed, if it is not well-

presented and emphasised in the record or due to time constraints and distractions:

“I missed once a cancer case. It was a woman in her 50s coming with a headache, I

missed the information that when she was young she had cancer, it was in the record

but at the very bottom, I didn’t scroll down” (GP3, post-observation interview). In

addition, GPs may have preliminary strong assumptions from reading information in

the eHR, even before starting the consultation: “I thought, he’s going to be anxious, I

almost, you know, if you’d said, ‘What’s this man going to come in with?’ I would say,

‘Anxiety, definitely’. Because I mean, the last four times it had been about anxiety…”

(GP 5, CDM protocols, the patient was later diagnosed with heart disease).

Errors in initial information integration of the patient’s reason for encounter with

information from the health record can arise if the GP is guided by strong

assumptions: “Well, in terms of, she’s obviously attending frequently, she’s a

hypertensive, she’s mildly obese…She’s got chronic back problems, probably osteo-

related and probably associated with her extra weight…So my advice…she needs to

do a bit more herself, perhaps get more fitter with activities” (GP 6, think-aloud

protocols, early presentation of myeloma).

Interface design solution

A diagnostic tool should make salient important and relevant patient information in

the record, e.g., risk factors, relevant family history, and chronic diseases, thereby

increasing the GP’s ‘situation awareness’.19

2. Diagnostic hypotheses generation

GPs generate a small number of diagnostic hypotheses early in the consultation,

which guide the diagnostic process and can determine its outcome.20 A pre-existing


60

disease, a colleague’s opinion, situational factors (e.g., time of the year or a previous

patient), and assumptions about the patient (e.g., frequent consulter) may install a

leading hypothesis at the exclusion of other alternatives. “I think I was agreeing with

the earlier doctor, who saw her a week earlier, that maybe it was a sprain” … “I think

it was that feeling of…she comes here often, and she’s quite anxious because her

husband left her recently and she was all alone and she’s struggling. And she wants

reassurance that everything is doing okay” (GP 1, CDM protocols, missed foot

fracture).


A diagnostic tool should display a list of possible diagnoses by integrating information

from the eHR with the reason for encounter. This aims to reduce narrow focus on one

diagnosis developing early in the encounter. The following information should be

taken into account:

• Demographic data (age, gender, etc.)

• Risk factors (smoking, alcohol, family history, BMI, etc.)

• Chronic diseases (e.g., asthma, diabetes)

• Recent examination and investigation results (e.g., last 3-6 months)

Following the results of a recent experimental study15, the tool will display the list of

possible diagnoses as soon as the GP enters a reason for encounter. This aims to

expand the hypothesis space and enable GPs to consider other possibilities, before

‘sticking’ to a leading hypothesis at the exclusion of others.

3. Hypothesis testing

As the problem space is large, information gathering would be endless, was it not

hypothesis-driven and guided by a stopping rule.21 In addition to deciding what

questions to ask, examinations to perform and investigations to order, GPs have to

decide when to stop gathering information and proceed to diagnosis and/or

management. Physicians seeing the same patient can differ greatly in their

diagnostic approach, as illustrated in the following example from the think-aloud

protocols. At the first patient presentation, GP3 asked the patient 18 questions,

performed 4 examinations and ordered 8 investigations before deciding to refer the

patient to hospital: “So it’s becoming a bit more, looking like this lady may

unfortunately have myeloma, which would fit with this persistent worsening back


61

pain, mild anaemia and raised globulins and urine proteins. … So I’m referring her to

the haematologist and I’m going to ask as an urgent...”

GP9 seeing the same patient asked only 3 questions and told her to come back if the

back pain persisted. At the second patient presentation (with prolonged and

worsening symptoms), GP9 ordered a single investigation (X-ray of the back) and

upon discovering that it was normal, the GP decided to prescribe pain relief: “Am I

concerned that there’s something that we’re missing or should I just try her for a bit

longer with better analgesia? So I think, given that we haven’t tried better analgesia, I

think that’s the next thing that I would do. So I’d stick with my diagnosis for now and

increase her analgesia.”

This example illustrates two factors in the diagnostic process that may lead to error:

first, that asking too few questions (presumably driven by a single hypothesis) may

lead to misdiagnosis, as important information will not be discovered. Second, that

once the physician adopts an interpretation, it may prove resistant to change, despite

discovering new information that is inconsistent with that interpretation.


In addition to presenting a list of diagnostic suggestions, the tool should enable users

to click on a suggested diagnosis and view the important features (symptoms and

signs) that can change the likelihood of the diagnosis. Users can check for these

features in the patient and tick either ‘Yes’ or ‘No’ to indicate their presence or

absence. The eHR will be automatically updated and so will the list of suggested

diagnose, if appropriate. For example, the order of the diagnoses may change

according to their updated likelihood, and diagnoses may be added or removed. The

tool should also propose examinations and investigations that could differentiate

between the suggested diagnoses. In this way, GPs are likely to elicit and consider

more information.

Finally, to avoid missing life-threatening or urgent conditions, the tool should alert

GPs who conclude the diagnostic process without checking any of the symptoms and

signs of these conditions.

4. Decide on course of action

Errors in management decisions can stem directly from misdiagnosis. They can also

occur if the physician does not safety net for serious possibilities and only manages

for what he/she considers to be the most likely diagnosis. Finally, errors may occur


62

due to insufficient knowledge about the most appropriate way to manage a specific

disease.


When the GP enters a diagnosis, he/she should be able to view guidelines with direct

links to forms for referral, investigations and prescribing, in relation to that diagnosis.

DISCUSSION

Summary: Using multiple methods and types of data, we elicited cognitive

requirements of the diagnostic task and proposed specific user interface design

solutions for a diagnostic tool. The tool should adhere to known design requirements

for workflow and eHR integration and suggest diagnoses for GPs to consider early in

the process, so that narrow focus on a single hypothesis is lessened. The tool should

have at its heart the reason for encounter and should facilitate data coding and

insertion. It should suggest symptoms and signs that are important for the relevant

diagnoses. It should highlight significant information in the eHR and alert GPs if they

have not checked for life-threatening or urgent conditions. These features could

enhance the tool’s usefulness and acceptability.

Strengths and limitations: The strength of our work resides in its use of multiple

methods of data collection and analysis (observations, interviews, HTA, CTA, and

SAR) to elicit cognitive requirements of the diagnostic task. This has not been done

before for the development of diagnostic support.

Another point of strength is our use of different data types that covered the spectrum

of diagnostic reasoning from the intuitive to the analytical. This ensures that the

requirements elicited and design solutions proposed are relevant to and can support

the different modes of clinical thinking.17 22 The use of multiple data sources is novel

in the requirements elicitations literature.

A limitation of this study is by focusing on the diagnostic task and tailoring design

solutions to the diagnostic tool, we disregarded the interaction of the diagnostic task

with other non-diagnostic tasks performed by the GP during consultation (e.g.,

prescribing medication).

This could have an effect on the overall user experience with the diagnostic tool, and

should be evaluated.


63

Comparison with existing literature: We are not aware of any diagnostic support

systems for healthcare that were designed on the basis of a detailed requirements

elicitation process, which may partly explain their lack of widespread adoption.

Implications for research and/or practice: The RfE is the trigger of the proposed

tool. Entering a RfE at the start of the consultation may require GPs to change the

way that they interact with their computer. For some, this change would be relatively

small. However, GPs differ greatly in the timing and amount of information that they

record; some were observed to record only the consultation outcome and only after

the patient has left the room. For these GPs, using the tool would be a challenge.

Another potential problem is the multiple RfEs that patients often present with, in

which case the GPs will need to decide which is the main one to record.

Funding: Financial support for this study was provided in part by a grant from the

European Commission under the 7th Framework Programme, Grant Agreement

Number 247787 for “Translational Research and Patient Safety in Europe”

(TRANSFoRm - www.transformproject.eu). Financial support for the study that

elicited the think-aloud protocols was provided by CRUK to Olga Kostopoulou, under

the NAEDI scheme (ref. C33754/A12222). Financial support for the study that elicited

the CDM interview protocols was provided by a departmental PhD studentship to

Amanda Woolley.

Ethical approval: Ethical approvals were obtained from the North West London

Research Ethics Committee 2 (10/H0720/50) and West London Research Ethics

Committee 2 (11/LO/0079).

Competing interests: The authors have no competing interests to declare.

Acknowledgements: Ellen Wright and Thomas Round helped with piloting the follow

up interviews, recruiting other GPs for observations and interviewing and providing

clinical advice.

References

1. Silk N. What went wrong in 1000 negligence claims. Health Care Risk Report, November 2000:13-16.

2. Gandhi TK, Kachalia A, Thomas EJ, et al. Missed and Delayed Diagnoses in the Ambulatory Setting: A Study of Closed Malpractice Claims. Ann Intern Med 2006;145(7):488-96.


64

3. Graber ML, Kissam S, Payne VL, et al. Cognitive interventions to reduce diagnostic error: a narrative review. Bmj Qual Saf 2012;21(7):535-57.

4. de Dombal FT, Leaper DJ, Staniland JR, et al. Computer-aided diagnosis of acute abdominal pain. BMJ 1972;2:9-13.

5. Shibl R, Lawley M, Debuse J. Factors influencing decision support system acceptance. Decis Support Syst 2013;54(2):953-61.

6. Kawamoto K, Houlihan CA, Balas EA, et al. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ 2005;330(7494):765.

7. Garg AX, Adhikari NKJ, McDonald H, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes - A systematic review. JAMA 2005;293(10):1223-38.

8. Friedman CP, Gatti GG, Franz TM, et al. Do Physicians Know When Their Diagnoses Are Correct? Implications for Decision Support and Error Reduction. J Gen Intern Med 2005;20(4):334-39.

9. Friedman CP, Elstein AS, Wolf FM, et al. Enhancement of Clinicians' Diagnostic Reasoning by Computer-Based Consultation: A Multisite Study of 2 Systems. JAMA 1999;282(19):1851-56.

10. Kostopoulou O, Rosen A, Round T, et al. Early diagnostic suggestions improve the diagnostic accuracy of Family Physicians: a randomized controlled trial (under review).

11. Militello LG, Klein G. Decision centered design. In: Lee J, Kirlik A, eds. Oxford Handbook of Cognitive Engineering, 2013.

12. Chipman SF, Schraagen JM, Shalin VL. Introduction to cognitive task analysis. In: Schraagen JM, Chipman SF, Shalin VL, eds. Cognitive task analysis. Mahwah, NJ: Lawrence Erlbaum Associates, 2000:3–23.

13. Kostopoulou O. From cognition to the system: developing a multilevel taxonomy of patient safety in general practice. Ergonomics 2006;49(5-6):486-502.

14. Shepherd A. Hierarchical Task Analysis. London: Taylor & Francis, 2001. 15. Ericsson KA, Moxley JH. Thinking aloud protocols: Concurrent verbalizations of thinking

during performance on tasks involving decision making. In: Schulte-Mecklenbeck M, Kühberger A, Ranyard R, eds. A handbook of process tracing methods for decision research: A critical review and user's guide. Hove: Psychology Press, 2011.

16. Hoffman RR, Crandall B, Shadbolt N. Use of the Critical Decision Method to Elicit Expert Knowledge: A Case Study in the Methodology of Cognitive Task Analysis. Human Factors 1998;40(2):254-76.

17. Woolley A, Kostopoulou O. Clinical intuition in family medicine: more than first impressions. Ann Fam Med 2013;11(1):60-6.

18. Klein GA, Calderwood R, Macgregor D. Critical Decision Method for Eliciting Knowledge. Ieee T Syst Man Cyb 1989;19(3):462-72.

19. Stanton NA, Chambers PRG, Piggott J. Situational awareness and safety. Saf Sci 2001;39:189–204.

20. Elstein AS, Shulman LS, Sprafka SA. Medical Problem Solving: A Ten-Year Retrospective. Evaluation and the Health Professions 1990;13(1):5-36.

21. Elstein AS, Shulman LS, Sprafka SA. Medical problem solving: An analysis of clinical reasoning. Cambridge, MA: Harvard University Press, 1978.

22. Evans JST, Stanovich KE. Dual-Process Theories of Higher Cognition: Advancing the Debate. Perspect Psychol Sci 2013;8(3):223-41.


65

Appendix – Cognitive Requirements Table

Cognitive requirement Difficulty/potential errors

How is it done? Critical cues Design solutions

1. Retrieval and integration of information (current information in the eHR with the RfE)

Important information in the eHR may be missed. Early assumptions may bias the interpretation of the RfE.

Before patient enters, GPs scan the eHR. GPs try to listen to the RfE without interrupting.

Previous consultations, risk factors, chronic diseases, current medications, recent test results

1. Make critical cues in the eHR more salient. Use easily recognisable icons and combined displays.

2. Diagnostic hypotheses generation

Focusing narrowly on a hypothesis.

GPs may think in terms of general categories (serious/not serious, urgent/not urgent). They may also consciously try to think of and exclude serious but rare conditions.

As above Suggest possible diagnoses by integrating information from the eHR with the RfE.

3. Hypothesis testing and refinement

Difficulty integrating new information that is not consistent with leading hypothesis.

See below.

Working hypothesis, patient input,

History taking, physical examination, current and past investigations

Provide an easy interface for entering coded symptoms and signs.

Update the list of suggested diagnoses according to coded input of symptoms and signs.

3.1. What questions to ask the patient?

GPs may omit to check for important symptoms and signs of serious but rarer diseases.

Asking about alarm symptoms to rule out serious diagnoses.

As above. Enable users to click on suggested diagnoses and view important features (that impact on diagnostic likelihood).

Propose examinations & investigations that could differentiate between the suggested diagnoses.


66

Cognitive requirement Difficulty/potential errors

How is it done? Critical cues Design solutions

3.2. If and what investigations to perform or order?

Not investigating appropriately (ordering too few or too many investigations).

Knowing the diagnostic value of investigations.

As above.

3.3. How to interpret the information elicited

Abnormal results may be normalised or dismissed.

Patient attributes, age, gender, risk factors.

Contextualise abnormal or borderline results according to patient’s age, RfE and possible diagnoses.

3.4. When to stop eliciting information (define a stopping rule)

Too little information may be elicited. Too much information may be elicited in the absence of guiding hypotheses

Stop searching, once a satisfactory explanation is reached and the most serious alternatives have been explored

3. Alert when GP enters a diagnosis without having checked symptoms and signs of life-threatening or urgent conditions

4. Deciding on a course of action Inappropriate

management due to misdiagnosis or lack of awareness about managing a specific condition

After the user enters a diagnosis, display guidelines with direct links to forms for referral, investigations and prescribing.


67

Appendix 2: DDSS User Guide

Overview

This document describes the functionality of the diagnostic decision support system

(DDSS) from your (the GP) point of view. The diagnostic decision support system is

a diagnostic tool integrated with the electronic health records (eHR) system to

support you in your diagnostic process during consultation. Currently the diagnostic

tool is integrated with Vision eHR, but it could integrate with any electronic health

records system. This is the first developed prototype, please take into consideration,

that it includes the main functionality, but the design and some additions/changes

may apply in future versions.

Functionality

The DDSS enables you to:

• View a dynamic possible diagnoses list for a specific patient based on the information in the patient’s electronic health record and his/her reason for encounter.

• Select a diagnosis from the list and check what features (symptoms and signs) are relevant, and can change the likelihood of that diagnosis.

• Code in an easy and intuitive way patient’s signs, symptoms and examinations. You can code negative symptoms (e.g., no blood in stool), in addition to positive ones.

• View recommended examinations and investigations that could be preformed in order to differentiate between the diagnoses on the list.

• View the reason/s why a specific diagnosis is on the ‘possible diagnoses’ list. • Select a diagnosis.

Workflow

In the following we will describe the diagnostic process when using the diagnostic

tool.

1. View patient information

As done today, open the patient’s electronic health record in Vision (Figure 1).


68

Figure 1 – A simulated patient’s health record in Vision eHR

2. Evoke the diagnostic tool and select RfE

If the patient’s reason for encounter (RfE) is diagnostic (not administrative, e.g.,

requesting a repeated prescription), click the ‘New consultation’ button from the eHR

toolbar (Figure 2), a dialog requesting you to enter the patient’s main reason for

encounter will be displayed (Figure 3) on top of the patient’s health record. You can

always navigate from and to the patient’s health record.

Figure 2: New Consultation button is added to Vision toolbar (first button on the left)

Currently the diagnostic tool supports three reasons for encounter (RfEs): abdominal

pain, chest pain and dyspnoea. The RfE field in the RfE dialog is an auto-complete

field. Once you start typing, the relevant options will be displayed (see Figure 3).

Evoking the ‘continue’ button will close the RfE dialog and open the main diagnostic

screen (figure 5). You can minimize or enlarge the screen by clicking the relevant

icons on the right, and you can close the screen by clicking the X icon.

Figure 3: Reason for encounter (RfE) dialog


69

3. Diagnostic support

The main diagnostic screen contains the following zones and components: screen

caption; significant patient information; patient’s RfE; signs, symptoms and

examinations list; general notes/comments; possible diagnoses list; and operation

button (Figure 4).

Figure 4: diagnostic screen – application shell

Figure 5: main diagnostic screen

3.1 Screen caption

The screen caption contains the patient name, gender, and DoB. You can minimize

or enlarge the screen by clicking the relevant icons on the right. You can close the


70

screen by clicking the X icon. A message asking if you are sure that you want to exit

the diagnostic support will be displayed.

Figure 6: Screen caption

3.2 Significant patient information

On the top left side of the screen, significant information about the patient, which will

be taken from the health record such as risk factors (e.g., relevant family history,

smoking and alcohol usage, previous cancer), and chronic diseases (e.g., diabetes,

asthma), will be displayed. Hovering on a cue will display more information (e.g.,

hovering on ‘smoking’ will display a tooltip: ‘10 packages a day’).

Figure 7: Significant patient information area

3.3 Patient’s RfE

The reason for encounter that you selected in the previous screen (e.g., abdominal

pain) is now displayed on top of this screen. You can edit the reason for encounter by

clicking ‘Edit’. The RfE dialog will open and enable you to change the reason for

encounter (Figure 8).

Figure 8: The RfE dialog is displayed after clicking ‘Edit’.

3.4 Signs, symptoms and examinations list


71

In this part of the screen, you can enter coded patient’s signs, symptoms and

examination results. This field is an auto-complete field, once you start typing, the

relevant options will be displayed (see Figure 9). It is important to note that we

filtered the list of all possible Read Codes, to a reasonable size, enabling you to

select frequent used and relevant signs and symptoms, without the need to search

from an enormous database. For example, instead of including the 21 Read Codes

that are related to a pregnant woman (e.g., wife pregnant, partner pregnant, pregnant

IUD failure), we display only one Read Code: Patient pregnant.

Figure 9: Entering patient’s signs, symptoms and examination results.

You can enter information using the mouse or the keyboard. Using the keyboard, you

can start typing the desired symptom, sign or examination (e.g., ‘vomi’ for vomiting),

navigate to the desired value using the arrows (e.g., ‘Diarrhoea and vomiting’), and

press the ‘enter’ key to select the value. Pressing the ‘enter’ key again will evoke the

‘Add’ button and display a small dialog enabling you to write notes about the specific

sign, symptom or examination (Figure 10). After writing the notes, pressing the ‘enter’

key again (evoking the ‘OK’ button), will display the sign, symptom or examination in

the list (see Figure 11).


72

Figure 10: Evoking the “Add” button displays the Notes dialog.

Figure 11: Evoking the ‘OK’ button from the notes dialog adds the selected value (e.g., vomiting) to the list.

For each sign and symptom you can indicate whether it is present or absent, by

ticking the ‘Yes’ or ‘No’ checkboxes (‘Yes’ is the default value). For example, you can

select the symptom ‘blood in vomit’ and indicate ‘yes’ if it exists and ‘No’ if it doesn’t

(see Figure 12).


73

Figure 12: ‘No blood in vomit’ symptom was added to the list (the ‘No’ value is checked).

Each time you enter a new value (sign, symptom or examination result), if it is

relevant to a specific diagnosis or several diagnoses, the possible diagnoses list on

the right side will update accordingly.

You can remove a value by clicking the red X icon on the left side of the value.

3.5 General notes

In this text box (Figure 13) you can enter general comments or notes concerning the

consultation, which are not related to a specific sign or symptom.

Figure 13: General notes/comments text field.

3.6 Possible diagnoses list

On the right side of the screen, a list displaying the possible diagnoses for a specific

patient, ranked according to likelihood, is displayed (Figures 12, 14). The list of

possible diagnoses, is based on the information from the patient’s electronic health

record such as: demographic data (age, gender, etc.), risk factors (smoking, alcohol,

previous cancer, BP, etc.), previous diagnoses, chronic health problems (e.g.,

asthma, diabetics), and recent investigation results (e.g., relevant blood tests) and

the patient’s reason for encounter you entered in the first step. The list updates

automatically as you gather and enter information to the signs, symptoms and


74

examinations list.

Figure 14: Possible diagnose list (the list is ranked according to likelihood).

For each diagnosis on the list you can see the reasons this diagnosis is displayed on

the list (for example: female, over 60 years old, etc.), by hovering on the diagnosis.

Figure 15: Hovering on a diagnosis will display the reasons this diagnosis is on the list

You can select a diagnosis from the list and check what signs and symptoms are

relevant and can change the likelihood of that diagnosis (see Figure 16). For

example clicking on ‘Crohn’s disease’, will display: diarrhoea, weight loss, anal

fissures, mouth ulcers, etc.. For each symptom you can tick ‘Yes’ or ‘No’, the

symptoms you checked will be added to the ‘signs, symptoms and examinations’ list

(see Figure 16). Clicking on the diagnosis again will update the possible diagnoses

list according to the new information you entered.


75

Figure 16: Clicking on a diagnosis will display the main symptoms that can confirm it or rule it out

It is important to note that only ‘reason for encounter’ is a mandatory field. However,

entering patient’s signs, symptoms and examinations is recommended in order for

the system to be more efficient in its diagnoses support.

4. Decide on a diagnosis

Clicking ‘Done’ from the main diagnostic screen (Figure 16) will display the ‘Working

diagnosis’ dialog (Figure 17). In this screen you can select the working diagnosis for

the patient and add notes.

The diagnosis field is an auto-complete field (similar to the RfE field), once you start

typing, the relevant options will be displayed. You can interact with this dialog using

the mouse or the keyboard. Using the keyboard, you can start typing the desired

diagnosis (e.g., ‘’appen’ – for appendicitis), select the desired value using the arrows,

and press the ‘enter’ key to select the desired value. You can add notes to the

diagnosis, such as differentials and/or management plans.


76

Figure 17: Working diagnosis dialog

5. Transfer the information

Clicking the ‘OK’ button will close the diagnostic tool and transfer all the information

you entered to the patient’s health record in vision (see figure 18).

Now you can proceed with the consultation: prescribe medications, refer to diagnostic

tests, etc.

Figure 18: patient’s health record containing the information transferred for the diagnostic tool

During the consultation, you can go back and forth from the diagnostic tool to the

New Added Information


77

patient health record in Vision.

6. Premature closure alert

The premature alert is displayed only after you give a diagnosis but failed to check at

least one of the life-threatening diseases’ symptoms and signs. The system will

display the alert once you close the DDSS (click the ‘Done’ button).

The screen will display only the serious diagnoses (life-threatening ones, see Figure

19 for a preliminary design). You can decide to go back to the main diagnostic screen

(click ‘reconsider’) and check the diseases’ relevant symptoms (see Figure 16) or you

may ignore the alert by clicking the ‘Ignore’ button.

Figure 19: Premature closure alert (preliminary design)


78

Appendix 3: GORD-SDM.xml

GORD-SDM.xml (including archetypes and data extraction requests)

Note: the development of this SDM specification for the GORD evaluation study is

based on the GORD study protocol and is based on the technology requirements of

the underpinning eCRF infrastructure for study execution and semantic

interoperability with EHRs. The current version of SDM as submitted is a reflection of

the up-to-date status of the GORD protocol and the latest development of

TRANSFoRm eCRF infrastructure. New versions will be delivered regularly following

any change in the GORD protocol and new advance in the development of the

TRANSFoRm eCRF infrastructure.


79

Appendix 4: Study Data Collection Forms

Appendix 4A CROMs


80

1. PAPER VERSION ELECTRONIC CASE REPORT FORM

1.1. First visit

An eligible person is one who consults a GP because of symptoms of GORD (the main reason for the visit can be another problem). Reasons for the index visit can include : GORD diagnosis, mention of acid regurgitation and/or heartburn, GORD complications or a treatment decision (prescription) in a previously diagnosed person.

All participants must have at least one typical reflux symptom (presence of troublesome heartburn and/or acid regurgitation).

All participants must be aged between 18 and 65 years

Checklist for inclusion.

Yes No

Age 18 or above, but not more than 65 years

GORD diagnosis

Previous PPI user

Can complete questionnaires

Has E-mail address (TRANSFoRm arm)

Checklist for exclusion.

No Yes

Barrett’s oesophagus


81

Severe oesophagitis LA C-D1

Continuous use of NSAID/aspirin

Prophylactic PPI use to reduce the risk of ulcers in persons being treated with NSAIDs

PPI treatment to heal an ulcer induced by NSAID treatment in the last 6 months

PPI treatment for H. pylori eradication in the last 6 months

Duodenal or gastric ulcers in the last 6 months

Myocardial infarction during the last 6 months.

Cancer any location

Severe disorders other than GORD with negative impact on quality of

life

Severe disorders other than GORD with negative impact on quality of

life as judged by GP

Signs of upper gastrointestinal bleeding (e.g B-Hb, F-Hb, melaena

(black blood in stool), haematemesis (vomiting blood)

Unintentional weight loss as judged by GP or BMI<19

Pain behind the breastbone (ICD-10: R07; Dolores tracheae et

thoracis) not due to heartburn/reflux

Nausea with vomiting (ICD-10: R11)

1 • Grade C: mucosal breaks that extend between the tops of two or more mucosal folds, but which

involve <75% of the mucosal circumference

Grade D: mucosal breaks which involve ≥75% of the mucosal circumference


82

Dysphagia (ICD-10: R13) / difficulty swallowing

Pregnancy

If the patient exhibits any of the exclusion criteria above, no further data will be collected.


83

Randomisation

Has the patient received written and oral information about the aim and the content of the

study? Yes/No

Has the patient given informed consent to participate in the study? Yes/No

Confirm this information

Sex: male/female

Year of birth (YYYY): __________________

ID: ________________________

Date:

Cause for visit:

-‐ Acute symptoms/disease -‐ Check-up/routine review/repeat prescription review -‐ Medical certificate -‐ Other

Height (cm): _________________

Weight (kg): _________________

PPI use:

-‐ Current PPI user -‐ Former PPI user


84

For current PPI user, type of PPI:

- omeprazole 20mg

- lansoprazole 15mg

- pantoprazole 20mg

_______ doses per day

_______ doses per week

Total of ______ doses over last 4 weeks

Has the patient consumed H2-blockers the last week?

-‐ Yes/no o If yes, continuous or on demand? o Type: _____________________

Has the patient consumed antacids/alginate the last week?

-‐ Yes/no o If yes, continuous or on demand? o Type: ______________________

Previous relevant diagnoses: ___________________________________

Present relevant diagnoses: ____________________________________

Has the patient had an upper endoscopy? Yes/no

If yes, when? _________________

Diagnoses from endoscopy:

-‐ Esophagitis? Yes/no.


85

o If yes, Los-Angeles grade A-B2: _____________ /unknown/other classification (histological):______________

-‐ Other changes to the oesophageus. Yes/no/unknown. o If yes, please specify: __________________

Helicobacter pylori status known? Yes/no

-‐ If yes, what methodology was used? o Direct (at endoscopy: CLO-test, histology, culture) Positive/Negative test

outcome o Indirect (serology, urea breath test, F-Hp) Positive/negative test outcome

How many days has the patient been on sick leave during the last 3 months? Yes/no

If yes, how many days?

Does the patient currently smoke?

-‐ Yes/No -‐ If yes, number of cigarettes/day?

How does the GP rate the patient’s general health status?

-‐ Very good -‐ Quite good -‐ Neither good nor poor -‐ Quite poor -‐ Very poor

Outcome of randomisation

2 • Grade A: one or more mucosal breaks no longer than 5 mm, none of which extends between the

tops of the mucosal folds.

• Grade B: one or more mucosal breaks more than 5 mm long, none of which extends between the

tops of two mucosal folds.


86

-‐ On demand -‐ Continuous


87

1.2. Event visit

ID: ________________________

Date: ______________________

Cause for visit:

-‐ Acute symptoms/disease -‐ Check-up/routine review/repeat prescription review -‐ Medical certificate -‐ Other

Weight (kg): _________________

How many doses of prescribed PPI (20 mg omeprazole) has the patient consumed during the last week?


Total of ______ doses per last week

Other reflux drugs taken the last week? Yes/no

If yes, which

-‐ PPI other than the prescribed PPI Yes/no. If yes, type: ____________ -‐ H2-blockers Yes/no. If yes, type: ____________ -‐ Antacida/alginate Yes/no. If yes, type: ____________

Present diagnoses: ____________________________________

Has the patient had an upper endoscopy since the last visit? Yes/no

If yes, when? _________________


-‐ Esophagitis? Yes/no.


88

o If yes, Los-Angeles grade A-D: _____________ /unknown/other classification: ______________

-‐ Barrett’s? Yes/no o If yes, histology confirmed intestinal metaplasia yes/no/unknown

-‐ Other changes to the oesophageus. Yes/no. o If yes, please specify: __________________

Has the patient been on sick leave since the last visit? Yes/no

If yes, how many days? ____

Does the patient currently smoke? Yes/No

If yes, number of cigarettes/day?

Have the patient experienced any alarm symptoms since the last visit?

-‐ Yes/No -‐ If yes, which?

o gastrointestinal bleeding o unintentional weight loss o difficulty swallowing o persistent vomiting o anaemia o (palpable) epigastric mass o other:________




89

1.3. Final visit

ID: ________________________

Date: ______________________

Weight (kg): _________________

How many doses of prescribed PPI (20 mg omeprazole) has the patient consumed during the last week?


Total of ______ doses per last week

Other reflux drugs taken the last week? Yes/no

If yes, which

-‐ PPI other than the prescribed PPI Yes/no. If yes, type: ____________ -‐ H2-blockers Yes/no. If yes, type: ____________ -‐ Antacida/alginate Yes/no. If yes, type: ____________

Present diagnoses: ____________________________________

Has the patient had an upper endoscopy since the last visit? Yes/no

If yes, when? _________________


-‐ Esophagitis? Yes/no. o If yes, Los-Angeles grade A-D: _____________ /unknown/other classification:

_____________ -‐ Barrett’s? Yes/no

o If yes, histology confirmed intestinal metaplasia yes/no/unknown -‐ Other changes to the oesophageus. Yes/no.

o If yes, please specify: __________________


90

Have the patient experienced any alarm symptoms since the last visit?

-‐ Yes/No -‐ If yes, which?

o gastrointestinal bleeding o unintentional weight loss o difficulty swallowing o persistent vomiting o anaemia o (palpable) epigastric mass o other:________

Has the patient been on sick leave since the last visit? Yes/no

If yes, how many days? ____




91

Appendix 4B PROMs


92

PATIENT REPORTED OUTCOME MEASURES

ID: _____________

Date: ______________

How many persons (excluding yourself) currently live in your household? ____ persons

How would you describe your current occupation or employment status?

(More than one answer possible)

- Employed (including civil service) - Self employed or family business - Student - Looking for a job (unemployed) - Unable to work due to illness or disability - Retired - Mainly homemaker (including looking after children etc)

What is the highest level of education that you achieved?

- No qualifications/ pre-primary/ primary school

- Secondary school (CSEs, GCSEs or equivalent)

- Further secondary education (AS levels, A-levels and NVQ level 3+, or equivalent)/ Degree or equivalent professional qualification/ Post-graduate qualification (Masters and doctoral level programmes)

Do you currently smoke? Yes/no

If yes, how many cigarettes per day? ________

How would you rate your general health status?

- Very good - Good - Neither good nor poor - Poor - Very poor


93

<<<<<INSERT RDQ>>>>>

Consider your reflux symptoms over the past week.

How often have you had difficulty getting a good night’s sleep because of your symptoms?

Daily Often Sometimes Never How often have your symptoms prevented you from eating or drinking any of the foods you like?

Daily Often Sometimes Never How frequently have your symptoms kept you from being fully productive in your job or daily activities?

Daily Often Sometimes Never

<<<<<INSERT SF-12v2>>>>>>

<<<visit 1 only>>>>>

When did you take PPI for the first time?

- Last month - Last year - Last five years - More than 5 years ago

<<<Patient randomized to on-demand PPI use:>>>

Have you taken PPI the last week? Yes/no

If yes, how often did you take PPI last week?

1 day/2 days/3 days/4 days/5 days/6 days/7 days


94

<<<Patient randomized to continuous PPI use:>>>

Have you forgotten to take your PPI the last week? Yes/no

If yes, many days did you forget to take your PPI last week?

1 day/2 days/3 days/4 days/5 days/6 days/7 days

<<<all:>>>

How many days the last 4 weeks (28 days) have you taken PPI? _______ days

Did you ever take more than 1 dose PPI in one day? Yes/no

If yes, how many doses did you take in total the last 28 days? _____ doses

Have you the last week consumed H2 blockers (drug examples)? Yes/No

If yes, how often? Daily/5-6 days/3-4 days/1-2days

Have you the last week consumed antacia or alginate (drug examples)? Yes/no

If yes, how often? Daily/5-6 days/3-4 days/1-2days

Are you content with your current treatment regime? Yes/no

Have you been on sick-leave the last month? Yes/no

If yes, for how long?

1 day/more than 1 day but less than a week/1-2 weeks/3-4 weeks/fulltime

Are you suffering from?

- Unintentional weight loss (yes/no) - difficulties swallowing (yes/no) - Anemia (yes/no)


95

<TRANSFoRM arm only>

Do you have any information you would like to share with your GP? (For urgent information, please contact your primary care centre directly)

d5.4 specification for functional ecrf and...

Documents