Page 1

PROSIT Publication Guidance

PROSIT Technical Document

Version 1.1

20.04.2012

Edited by: Jens Attumalil and Wendelin Schramm

Editors of previous versions:

Version 1.0: Adel Al-Ayyash, Katharina Chruscz, Franziska Herget, Lennart Koester, Klaus Kurtiak,

Monika Pobiruchin, Serife Soenmez, Sinthu Vijayanathan, and Wendelin Schramm

http://www.prosit.de

Page 2

Preamble

The PROSIT Publication Guidance (PPG) has been written to help PROSIT community members to

create scientific and evidence-based health-economic publications that fulfil a defined minimum

level of quality. Approaches as well as proper methods are explained in detail in this guide.

It’s main goal is to provide the readers with a set of tools and a clear path to follow so they can either

contribute valuable content or information to the PROSIT project or enable them to assess the

quality of information that is already part of the project.

The guidance initially relied on the Drummond and Jefferson paper “Guidelines for authors and peer

reviewers of economic submissions to the BMJ” (1). In later years this document was step-by-step

enhanced to match new requirements such as the German IQWIG method paper (2, 3).

First, the section Preparation guides the choice of methods adequate to investigate defined research

questions.

The following section Performing a health-economic study describes the methods of a good

modelling study.

Then the section How to structure and write a publication deals with the requirements of a HE

publication or reporting of HE results.

We do hope that readers find this guidance though short and in telegram style helpful to their work.

Heilbronn, July 2012

Wendelin Schramm, Jens Attumalil

How to cite this guidance:

PROSIT Disease Modelling Community. PROSIT Publication Guidance Version 1.1 [monograph on the

Internet]. Heilbronn: Heilbronn University; 2012 [cited 201X XYmonth XYday]. Available from:

http://www.prosit.de/index.php/Prosit_Publication_Guidance

License:

This document is published under the GNU Free Documentation License (FDL).

PROSIT© is a registered trademark.

Page 3

Contents

Preamble ................................................................................................................................................. 2

Preparation .............................................................................................................................................. 4

Research question ............................................................................................................................... 4

Study types and questions posed .................................................................................................... 4

Selection of alternatives .................................................................................................................. 5

Searching for evidence ........................................................................................................................ 5

Search for trials and reviews ........................................................................................................... 5

Costs .................................................................................................................................................... 6

Performing a health economic study ...................................................................................................... 9

How to structure and write your publication ........................................................................................ 11

References ............................................................................................................................................. 16

Page 4

Preparation

Research question

When formulating the research question it is necessary to meet certain criteria:

The question should be economically important.

The research question or hypothesis should be phrased in a way that takes both cost and

outcome into consideration.

The hypothesis of the economic evaluation should also be stated clearly.

The question should indicate the viewpoint of the economic evaluation (e.g. societal,

sicknessfund, patient etc.).

Study types and questions posed

Depending on the research question different types of HE studies are adequate means for the inves-

tigation.

Study type Measurement of benefits Question posed

Cost minimisation

analysis

Benefits found to be equivalent Which is the most efficient way of

achieving a given goal

(or objective)?

Cost effectiveness

analysis

Natural units (e.g. life years gained) What is the most efficient way of

spending a given budget?

Cost-utility analysis Healthy years (e.g. quality adjusted life

years (QALY), healthy years

equivalents)

See cost effectiveness analysis

Cost-benefit

analysis

Monetary terms Should a given goal (or objective)

be pursued to a greater or lesser

extent?

Table source: Drummond, Jefferson, BMJ 1996 (1)

The form of an evaluation should always be stated and a clear justification given for choosing thi

distinct type of analysis.

Page 5

Selection of alternatives

When comparing two treatments it should clearly be stated why the alternative was chosen,

furthermore the alternative should either be the most widely used treatment or the treatment that

provides best cost effectiveness. If current common practice is "doing nothing" then a comparison

with placebo should be conducted as it may open up the possibility of comparing the study with

other studies that used placebo as comparator.

The chosen alternatives should be described in sufficient detail to provide the reader with enough

information to relate the information on costs and outcomes to the alternative courses of action. In

addition, the chosen alternative must be designed to help to get as close a measure as possible of the

opportunity cost of the new treatment (1). Decision trees or other decision analytic techniques can

help the reader to follow the paths the treatments might take and provide them with information

about costs and outcomes associated with each path. Information about probabilities, cost, and

outcomes linked to paths of each treatment will help the reader to make their own decisions about

each treatment.

Searching for evidence

Economic evaluations are based on effectiveness data which can be derived from either single clinical

trials, registries or, and this is frequently necessary, from an overview of several clinical trials (meta-

analysis).

Depending on which source the evaluation relies, different questions arise and several aspects have

to be considered.

Search for trials and reviews

A good and reliable source for systematic reviews is the Cochrane Collaboration: The Cochrane

Library hosts all systematic reviews executed under the conditions and rules of the Cochrane

Collaboration.

Clinical trials, i.e. their results, are often published in renowned journals. We recommend searching

for such publications in databases like:

Pubmed/MEDLINE

EMBASE

Database of Abstracts of Reviews of Effects (DARE)

Current Controlled Trials

NHS Economic Evaluation Database

Health Technology Assessment Database

For the German health care system:

Page 6

IQWIG systematic reports

Federal Health Monitoring System (also English pages!)

Databases of the German Institute of Medical Documentation and Information DIMDI (for

the German health care system)

Be aware that for the purpose of disease modelling results have to be reported in a manner that

allows for instance deriving transition probabilities over time. Not necessarily do publications and

reviews support these requirements which means that occasionally highest level information has to

be neglected for a modelling study. Before sorting out high level evidence contacting the authors

directly and a question for additional information could be helpful.

Grading data

If possible only rely on clinical trials which are randomized, double-blinded and controlled. This type

of study design is called a gold standard and ensures high internal validity.

In order to judge the quality for publications it might be useful to allocate the Jadad Score (4) as first

orientation.

The PROSIT community grades according to the Scottish Intercollegiate Guidelines Network (SIGN)

methodology.

The GRADE network provides a similar high level alternative and is used by the Cochrane

Collaboration.

The study protocol of a clinical trial should report the design and result of the study, method of

allocation of subjects, effect size with confidence intervals, whether it is an intention-to-treat

analysis or an evaluable cohort analysis. If it is a trial with placebo control take into account to

mention suitable active comparators.

Reviews should report details about the method of synthesis or meta-analysis, search strategy,

criteria for inclusion or exclusion of studies.

Costs

What should be considered?

Costing involves estimating the amount of allocated resources (2) used and their prices (unit costs)

for example, days in hospital at a defined DRG base-case rate. Both aspects should be reported

separately.

All clinical procedures of a patient should be reported. But not all procedures take place in a regular

clinical care (efficacy) therefore it should be considered whether all procedures in a clinical trial are

distinctive of normal clinical practice (effectiveness).

Page 7

Estimates of resource quantities should be based on data of real patients collected from medical

records. Regarding the German health care system estimates of resources can be either based upon

Microcosting or Top-down-approach (5). The prices of the resources can be obtained from the

finance departments of particular institutions or from national statistics but it should be considered

that charges or fees can differ from real costs. The evaluation should comment on the extent of

which using charges may bias their estimates.

In most health economic studies only direct and indirect costs are considered, but other view points

are availbale depending on the economic perspective pursued in the HE study:

Direct costs

Direct costs refer to materials, diagnostics, treatments, pharmaceuticals, expenses related to the

production of a product/service or in this case to the diagnosis/treatment/follow-up of a health

condition. Direct cost can be summed up easily.

Indirect costs

Indirect costs are costs not linked “directly” to the medical treatment or disease like loss of income,

absenteeism and prospective treatment costs. This type of costs can’t be measures as easily as direct

cost.

Less frequently used costs

Intangible costs: This type of costs is the hardest type of costs to quantify because it consists of costs

which are caused by subjective phenomena such as physical pain, adverse effects and other types of

individual feelings. These phenomena have to traslated into costs for instance by determing a

willingness-to-pay for avoiding certain outcomes and adverse effects.

Opportunity costs: The cost of an alternative that must be forgone in order to pursue a certain

action. Put another way, the costs (and benefits) you could have received by taking an alternative

action.

Friction costs: In health care systems friction costs are sometimes used in the societal or employer

prespectives in the context of productivity losses caused by the necessary replacement of staff that

has become sick. This may comprise direct and indirect costs. Loss in work force due to illness adds

additional costs to the loss in work productivity.

In Economics: the direct and indirect costs associated with the execution of financial transactions.

Friction costs include the commissions and fees, interest rates, research time (opportunity costs),

loan origination fees, tax implications and the time value of money associated with the transaction1.

Where to search for prices of resources in Germany?

For medications ( www.rote-liste.de )

o The Rote Liste is a list of medicines in Germany and is published by the Red List

GmbH. It contains brief information on medicinal products marketed in Germany. It

is aimed at medical and pharmaceutical professionals (doctors, pharmacists,

1 http://www.investopedia.com/terms/f/frictioncost.asp#axzz1nxgBlJnb

Page 8

hospitals, etc.) with the purpose to inform them of the commercially available

preparations. It is published annually updated as a book since 1933.

G-DRG-System ( www.g-drg.de )

o German Diagnosis Related Groups, short G-DRG system. For the remuneration of

general hospital services was introduced to German hospitals in accordance with §

17b hospital financing law (KHG), an integrated, performance-based and flat-rate

compensation system. This is based on the G-DRG system (German-Diagnosis

Related Groups) system, whereby each inpatient case is remunerated by an

appropriate DRG fee.

Fees for physicians in the Statutory Health Care System ( http://www.kbv.de/8156.html, in

German)

Fees for physicians privately insured( http://www.e-bis.de/goae/defaultFrame.htm, in

German )

Currency conversion should be based on real purchasing power, rather than financial exchange rates,

which fluctuate according to money market changes.

Selection of statistical tests

The statistical tests which are used should match the data set they are used on as there are a lot of

tests which differ in target size (e.g. variance or average). The chosen test should be justified for

better traceability. This verifies the usability of the test. Normally the choice should be made before

collecting data, a minimum sample size is given by the choice of the test. As this is a complex process,

it is recommended to contact a biometrician if necessary.

The details of the statistical tests should be given as well as the confidence intervals of the observed

variables.

Page 9

Performing a health economic study

Evaluation Outcomes

An explanation about every used (comparative) outcome should be given.

Common outcomes fall into one of three categories:

o the medical events (e.g. myocardial infarction, numerb of amputations etc.), the

quality-of-life (“QALY'S”)

o the costs like direct and indirect costs as well as the "willingness to pay".

o Combined outcomes (medical and economic) like costs per life-year gained (CLYG)

If intermediate results are used as final outcomes discuss their trustworthiness.

Modelling

What is a model?

There are several definitions for the term "model". One provided by the former EU project HARMET 2

has been: “a disease model describes and simulates on the computer different courses of a disease

considering possible intervention strategies and their costs”. Such definitions have in common, that a

model may not be a perfect copy but a simplified representation of reality. The extent of this

simplification depends on the needs of those, who want to use the model. Basically there are no

good disease models, but some of them may be helpful for instance to draw conclusions or to make

decisions.

Why modelling?

In most cases economic data that were collected in clinical trials, cannot be used alone for a fully

informed decision. As a rule of thumbs all events/outcomes and all costs should be assessed in a HE

study. Thereby is the modelling of the outcome amount an essential component of evaluation. In

addition, modelling techniques loow it to expand an evaluation beyond what was observed.

What demands should be met on a modelling?

One of the main requirement is that the model is explicit and clear. To ensure this, the following

factors are essential:

full transparency with clearly described and well-founded model inputs and assumptions

sufficient depth for adequate representation of the modelled diseases with its associated

cost and the corresponding measures

flexibility to calculate multiple scenarios with varying constellations and adoption settings

possibility of determining the uncertainty in the predicted cost

use of relevant data

2 HARMET = The Harmonisation by Consensus of the Methodology for Economic Evaluation of Health Care Technologies in the European Union. Project no. BMH1-CT94-1252.

Page 10

Handling uncertainty

To figure out the influence of uncertainty on the results a sensitivity analysis should take place. By

varying input variables by +/- 10% PROSIT Markov models can show the influence of varying input on

the modelling outcomes. Extreme value calculations can help to test the robustness of the model

over the complete range of possible model inputs.

Discounting

Should the data be discounted?

All costs and effects hould be presented in discounted and nominal (i.e. undiscounted) values. The

Hannoveraner consensus statement recomends a discount rate of 5% (6) for Germany. Wheras the

IQWIG Institute demands sensitivity analyses with the discount rates of 0%, 3%, 7% and 10%.

The used rates should be justified.

What about the influences of inflation?

If inflation data is used, mention why. The inflation rate for the health care system is published by

the Ministry of Health and/or the national statistics office3.

3 For Germany: http://www.destatis.de

Page 11

How to structure and write your publication

Report contents

Health-economic publications and reports follow a different line of argumentation as for instance

when reporting a prospective clinical trial.

We suggest following structure of a health-economic publication:

Introduction and objectives:

o Introductions should be short and arresting, and they should tell the reader why you

have undertaken the study. The introduction is what motivates the audience to read

a paper and the first sentence is particularly important. The introduction should be

concise and include the key papers in that field of research. Introduction should

provide a context or background for the study (i.e., the nature of the problem and its

significance). State the specific purpose or research objective, or hypothesis tested

by the study. The research objective is often more sharply focused when stated as a

question.

Definitions

o Each thematic area has its own professional vocabulary. Therefore it is necessary to

explain concepts and definitions in advance.

Limitations

o Not every question can be answered fully, so that responses can be interdisciplinary.

Each question is under a certain limitation in terms of the appropriate response.

Methods: Patient characteristics, Cost data set, clinical data etc.

Health-economic study type and study design

Economic perspective

Model used for the analyses

o This section should be written in great detail of what was done giving a sufficiently

clear overview, so that others could reproduce the study. The guiding principle

should be clarity about how and why a study was done in a particular way. Strike a

balance between brevity and completeness, but the reader must be able to assess

the following:

What type of study was performed? (i.e. was it a randomized controlled

clinical trial? How was randomization done etc.?)

How many subjects (or samples) were included?

Who were the subjects?

Where did the subjects come from?

What were the inclusion and exclusion criteria?

What intervention (if any) was offered?

How long was the follow-up (if relevant)?

What was the response rate (for surveys/questionnaires)?

What outcomes were measured and how?

Page 12

What statistical tests were used?

Scenarios investigated

Data sources and data handling (incl. synthesis)

Underlying assumptions (if any)

Results: Medical outcomes, Economic outcomes, combined health-economic outcomes (e.g.

ICERs, CLYG), other outcomes

o Results are the core of the paper. The results should be presented in a clear,

accurate and concise format. There should be no redundancy. Be sure to include

basic descriptive data. The text should tell the story and the tables give the evidence

while the figures illustrate the highlights. Present your results in logical sequence in

the text, tables, and illustrations, giving the main or most important findings first. Do

not repeat in the text all the data given in the tables or illustrations.

Validation (Sensitivity analyses, extreme value analysis)

Influence of discounting

Discussion / Limitation

o The key questions here are a What and a How and a Where. What were the

strengths and weaknesses of your study? Then how do your findings fit in with work

published by others? Finally, where is this line of research going next? This is when

you face the challenge of deciding whether your original hypothesis is still standing.

The discussion aims to summarize your work and put it into perspective. In the era of

evidence-based medicine (EBM), this section should include the interpretation of

results within the available body of systematically collected and evaluated

knowledge. It is important to acknowledge potential limitations of the study — no

article is perfect, but equally these limitations should not be ignored. Always

remember to comment on the clinical significance of your findings. Do not repeat in

detail data or other material given in the Introduction or the Results section.

Common mistakes include citing as first study in the world, emphasizing the

strengths of the study more than its weaknesses, reiterating & inflating selected

results, going beyond the evidence and drawing unjustified conclusions.

Conclusion

o The concluding paragraphs of 2-3 lines mentioning the principal findings & their

relevance makes up the conclusion of the study. Link it with the aims of the study. It

is useful to present these as bullet points as this provides maximum impact.

Unjustified conclusions not supported by the data should never be made

Study report

In order to meet the standard outlined in this document the study should concern itself only with

relevant and comparable topics. At the beginning of the study it should be explained why the

research question was chosen and how it is important to the key potential decision makers. When

mentioning the key potential decision makers it should also be made clear which point of view the

Page 13

study takes, e.g. health care system or health insurance, and why this point of view was chosen for

the study in order to make it apparent to the reader if the study can be of importance to him or her.

Lastly, it should be explained how the costs used for calculation are made up.

Transparency

Data of one economic study is closely bound to a specific setting. Decision makers need to generalize

results or transform them so that it suits their point of view. Transparency of data helps alleviate this

process.

Economic evaluations should be executed under conservative assumptions. Report results of

sensitivity analysis which help the readers assess the robustness of study conclusions.

Furthermore every conclusion should be exclusively drawn from published data, which enables the

reader to understand and verify it.

Report on disaggregated or aggregated data?

At first report components of costs or benefit (direct costs, indirect costs, life years gained,

improvement in quality of life) in a disaggregated form. Afterwards combine them in appropriate

indices or ratios.

Readers should be able to calculate other ratios which might fit better regarding their respective

purpose or point of view.

Reporting about cost data

Estimates of resources used and their prices have to be reported separately to help the reader judge

their relevance to his/her setting. Beside of this, the costs of all procedures done to a patient during

a clinical trial should be included because these costs could possibly influence outcome.

The dates of both the estimates of resource quantities and prices should be recorded, along with

details of any adjustments to a more recent price level. Adjustments to the inflation of preceding

years, modifications to show the relevant perspective as well as discounting is to be reported (2).

Rating data

If possible only rely on clinical trials which are randomized, double-blinded and controlled. This type

of study design is called gold standard and ensures highest internal validity.

In order to judge the quality for publications it might be useful to allocate the Jadad score.

The study protocol of a clinical trial should report the design and result of the study, method of

allocation of subjects, effect size with confidence intervals, whether it is an intention-to-treat

analysis or an evaluable cohort analysis. If it is a trial with placebo control take into account to

mention suitable active comparators.

Page 14

Overviews should report details about the method of synthesis or meta-analysis, search strategy,

criteria for inclusion or exclusion of studies.

At the end of the publication / report provide the reader with information how you assessed the

sources used in this publication. For example give the full table of Jadad scores you obtained from

your sources.

Missing data

If there is no available source (study, overview) it is justifiable to use other sources like expert

opinion. To collect opinions of several experts a delphi panel could be the method of choice.

Naturally information gained under such conditions is not as reliable as outcomes from a clinical trial

for example.

Maybe not every information was reported in the publication. Contact the author to gain additional

knowledge and information you might need.

There is also the possibility that the clinical trial was too insignificant or maybe the result was not

positive or neutral (p-Value > alpha value). Those trials are likely to be never published in journals

although their data might be essential for an economic evaluation. This phenomenon is called

publication bias. Funnel plots help to assess whether there is publication bias or not.

Missing data can be acquired by modelling.

Regardless which kind of source is used in an economic evaluation it should be considered that any

limitations which weaken the assessment of effectiveness (e.g. clinical trials, overview) deteriorate

any economic evaluation.

Information about tests used

The result depends on the Statistical tests used. That means a sample can be interpreted in different

ways in which all of them look evident. So it is important to justify the Selection of Statistical Tests.

Especially if approximated tests are used. Strengths and Weak Points of the Test should be alluded,

also resulting consequences for the sample and the Research Question. Reporting results does not

mean reporting the final conclusion. For better understanding it is helpful the present all

intermediate results. During this the reader can follow the chain of evidence by himself. Confidence

intervals around the main variables should be specified (5).

Comparison

Authors often compare their cost effectiveness ratios with other studies using league tables.

Those rankings should be treated with caution.

Page 15

Firstly, differences might arise from different methodological approaches rather than real

differences in intervention.

Secondly, take into consideration how the cost effectiveness or cost-utility ratio of each

figure in the league was generated; maybe one intervention was compared to placebo

control. Another intervention was compared to current gold standard in healthcare.

Summarising it, only comparisons with health care interventions should be made which are quite

similar in setting and study design.

Page 16

References

1. Drummond MF, Jefferson TO. Guidelines for authors and peer reviewers of economic submissions to the BMJ. The BMJ Economic Evaluation Working Party. BMJ. 1996;313(7052):275-83. Epub 1996/08/03. 2. (IQWiG) IfQuWiG. Entwurf einer Methodik für die Bewertung von Verhältnissen zwischen Nutzen und Kosten im System der deutschen gesetzlichen Krankenversicherung. Cologne2009. p. 74. 3. Schwalm A, Danner M, Seidl A, Volz F, Dintsios CM, Gerber A. [IQWiG's methods for the cost-benefit assessment : Comparison with an international reference scenario]. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz. 2010;53(6):615-22. Epub 2010/05/08. Wo steht die Kosten-Nutzen-Bewertung des IQWiG : Abgleich mit einem internationalen Referenzszenario? 4. Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Controlled clinical trials. 1996;17(1):1-12. Epub 1996/02/01. 5. (IQWiG) IfQuWiG. Technischer Anhang Kostenbestimmung. 2008. 6. J.-M. Graf von der Schulenburg WG, F. Jost, N. Klusen, M. Kubin, R. Leidl, T. Mittendorf, H. Rebscher, O. Schöffski, C. Vauth, T. Volmer, S. Wahler, J. Wasem, C. Weber. Deutsche Empfehlungen zur gesundheitsökonomischen Evaluation - dritte und aktualisierte Fassung des Hannoveraner Konsens. 2007.