Declaration of Innsbruck

Jan Talmon - Maastricht University

Elske Ammenwerth - UMIT

Content

Reasons for paying attention to evaluation Introduction Some key observations and

recommendations Current activities

Reasons for concern

Scientific Building an evidence base

Managerial Support decisions

Introduction 1-2

Interest in evaluation since early 90-ties Mostly decision support Sydpol, COMAC-BME, KAVAS, Eurodiabeta

Growing interest in evaluation of Information Systems mid 90-ies ATIM, VATAM

Lack of agreed upon evaluation frameworks

Introduction 2-2

Working groups of IMIA and EFMI HISEVAL meeting Innsbruck April 2003

3 activities: GEP-HI, STARE-HI, EP-HI Declaration of Innsbruck

Declaration - Preamble

Evaluation of Health Information Systems is morally imperative Intended to improve functioning of health

professionals Has impact on patient care Benefits and potential harms need to be known

Declaration - Definitions

An HIS is more than the hard- & software It includes also all stakeholders that are affected

by the use of the system Evaluation is measuring properties of a HIS

Informs a decision to be made Evaluation of HIS has to deal with actors, the

artefacts and their interactions.

Observations

Evaluation generates information Should be relevant for the decision at hand

Evaluation supports reflective practice It is part of a professional attitude In line with engineering ethics

Evaluation is challenging Each study requires careful design, guided by the decision

to be taken. Detailed guidelines are hardly to be given Evaluation is not free

It takes time, is costly and require planning from the onset of a project

Recommendations 1-3

Evaluation is an ethical imperative Part of code of ethics of IMIA (in context of EPD)

Evaluation should be free of pressure There should be no pressure as to what the desired

outcome of the study is

Evaluation studies should be grounded on theory and rigorous approaches Only validated methods and techniques should be used. No reinvention of the wheel or use of non-validated tools

Recommendations 2-3

Reports on evaluation methods and methodical studies are encouraged Not only the study should be reported, but there should be

a reflection on the approach taken and the lessons learned Creation of an evidence base

Guidelines for good evaluation practice should be developed

Guidelines for reporting of studies should be developed CONSORT, STARD

Recommendations 3-3

Evaluation should be promoted by centers of excellence Evaluation requires specific skills. Bringing those skills and

expertise together will improve the quality of the studies done

Evaluation networks should be established The field can only develop through (international)

collaboration An open access repository of evaluation resources

is needed Not only scientific reports, but also tools, questionnaires,

experiences.

Recommendations Proposed additions

Industries should include evaluation in their offers

An accreditation/certification body is required

Current activities

GEP-HI: Good evaluation practice in Health Informatics Pirkko Nykänen Tampere

STARE-HI: Standards for reporting of evaluations in Health Informatics Jan Talmon Maastricht

EP-HI: Evaluation Portal for Health Informatics Gerrit Boers Maastricht