coursebook episode research training - …...student hands in his/her project plan and registration...
TRANSCRIPT
Coursebook
EPISODE RESEARCH
TRAINING
CKO-9 course
and
Research Training Internship
October – 2014
Plu-nr: xx-xxx
3
CONTENTS 3
PREFACE 4
1 CHOOSING YOUR RESEARCH PROJECT 5
Procedure 5
Selecting your project 6
Supervision during the Research Training Internship (RTI) 7
2 ORGANISATION EPISODE RESEARCH TRAINING 9
Coordination and learning objectives 9
Central Clinical Education (CKO-9) 9
Research Training Internship 10
Personal learning objectives 10
Assessment
Central Clinical Education (CKO-9)
Research Training Internship
Attendance and absence (CKO-9)
Evaluation
10
10
10
11
12
3 TWO-WEEK CKO-9 COURSE 13
Structure and learning components 14
Lecture 1: Profile of CKO-9 15
Module 1: Rationale of the research 17
Module 2: First appraisal of the literature 19
Module 3: Specifying the research question 22
Seminar 1: A first outline of your research 24
Module 4: Pathways to efficient study design 25
Module 5: Measurement of key variables 27
Module 6: Considering the study subjects 30
Seminar 2: Designing your data collection 33
Lecture 2: Philosophy of science influencing your research 34
Module 7: Societal and ethical justification 36
Module 8: Designing the data analysis 39
Module 9: Organisation and management summary 46
Seminar 3: Practical organisation and numerical thoughts 49
Computer lab: SPSS practicalities 50
Module 10: Reporting your Plan of Work and management summary 54
Seminar 4: Sales and communication 57
CKO-9 Evaluation form 58
4 TWELVE-WEEK RESEARCH TRAINING INTERNSHIP 60
General information 60
Supervision 61
Writing your research report 63
APPENDICES 66
Registration form 67
Template Project Outline and tips 69
Two examples
Assessment protocols
72
76
4
PREFACE
Scientific research is driven by scientific curiosity, by professional and social ambition, and – certainly in
today’s medicine – by evidence-based motives as well as by serendipity (i.e. making a fortunate discovery
while looking for something else). So, there are many factors that influence the health care people receive
following or during a visit to the doctor.
It often starts with discoveries made by scientists working in laboratories. Once scientists or medical-
specialists spot the potential of the scientists’ work, they begin developing it further, with clear goals in
mind; to help to cure or prevent disease. In some cases medical breakthroughs arise following a researcher’s
study of the spread of a disease between people or from the question whether it is desirable to implement a
new treatment.
Medical-specialists have to deal with this continuous development in the field of medical knowledge. They
have to keep up and anticipate on new insights and advanced technical possibilities. One of the important
skills required in this process of advancement of knowledge is a critical attitude towards research practice.
Furthermore, medical specialist themselves conduct experiments to aid and support these developments in
scientific knowledge.
Medical-students have to familiarize themselves with this always changing and exciting world of scientific
research. This acquaintance starts in the first-year Medical Professional Training course (MPV-1) and
continuous in other MPV, 5BOSA8 and O courses in the bachelor’s phase. Their theoretical scientific
training ends in the master’s phase in Episode Research Training / Episode Onderzoeksstage in which they
choose an Research Training Internship and prepare this internship in CKO-9. This episode can be located
in the final stages of their master’s phase, when they have already gained experience at various clinical
departments or at the beginning, prior to their clinical internships, in case of an waiting period.
We hope that the CKO-9 course and the Research Training Internship that forms part of this Episode
increases your enthusiasm for scientific work and form the kick off for continuing a research career in this
exciting and important medical field following your graduation.
5
1 CHOOSING YOUR RESEARCH PROJECT
There is a wide range of research projects open to students when preparing for their Research Training
Internship (RTI). These can be found in any department of the Radboud University Medical Centre
(Radboudumc) or in many institutions elsewhere in the Netherlands and abroad. These may be projects in
clinical specializations within or outside the hospital, in basic science subjects, in meta-medical domains, or
in the behavioural sciences. In each and every case, the RTI must meet the following quality requirements:
- the internship plan must be approved by the Research Training Committee (RTC);
- the internship will be pre-trained in the CKO-9 course;
- the host organization for the internship must guarantee host supervision quality;
- the internship plan must make clear that the proposed time frame can be met and can be completed
with a report;
- a senior member of staff, holding a doctorate degree and in the employment of the Radboudumc,
must declare him/herself to be prepared to take on final responsibility for supervision and assessment
(the Radboudumc final supervisor).
Students are strongly encouraged to apply for an RTI abroad as this is in line with the University’s pursuit
of internationalization goals. A great many of medical faculty staff have contacts with leading research
institutes elsewhere in the world. They are often prepared to assist motivated students in the organisation of
an internship project abroad. These internships, however, are subject to the same quality requirements as
internal RTIs or other external RTI’s.
Students must be aware that RTIs abroad require more preparation and additional effort; often this is a
preparatory period of at least one year. The internationalization officer of the Radboudumc Education
Institute has been specially appointed to support students in preparing for these training programmes
abroad.
Procedure Before students embark on their RTI, they have submitted a project plan for approval by the RTC. The plan
has to be handed in at the Student Information Point (StIP) at OIO/OSZ. Forms for internship project plans
can be obtained from the OIO/OSZ desk or can be downloaded from www.umcn.nl/student. Students have
to include information about supervision arrangements, and they need to submit a concise project outline of
preferable max 2 A4 (background, research question, study design, impact, plan of work, feasability,
literature). A template for the concise project outline has been included in the Appendices.
Students hand in their completed project plan and the (StIP) “Registration form” (see Appendices) for the
CKO-9 course and the Research Training. On this registration form, which has to be signed by both the
student and the UMC final supervisor, both declare that they are acquainted with the Episode-9 Course
Book and that there is sufficient opportunity for achieving the general learning objectives of the internship
in the 12-week period available. This then forms the first quality control step in internship projects.
Registration form and the project plan must be submitted before the first day of the period preceding the
block period in which students wish to enrol in the CKO-9 training period.
Research Training Committee (RTC)
The chair of the Research Training Committee: Professor J.P.H. Drenth
- phone 024-3616999
- email [email protected]
The managing director is drs. C.C.M. Janssen
The Committee consists of 4 members, all of whom are qualified principle investigators and represent
scientific disciplines ranging from applied medicine (causal and predictive research) to (molecular) life
sciences.
6
Subsequently, the RTC reviews the project plan and provides commentary as she finds the project outline
lacks information or is of insufficient quality. Also, before approving an internship, the RTC verifies
whether the conditions necessary for supervision have been met. Without this approval, students cannot
embark upon the CKO-9 course nor on their internship. Please note: In case accreditation by Ethics
Committee is necessary and NOT already approved, the RTI can’t be approved by the RTC. Students can
monitor the progress of the approval procedure via their Student Web Dossier. After formal approval and
sign off of the plan by the RTC and the students have been registered for the CKO-9 course, the quality
control process has been completed.
Please note: students may only embark on their RTI after they have completed the CKO-9 course.
If two students together embark on an internship, both students must submit their own (and hence not
identical) project proposal and registration form.
In the case of an internships not in Radboud University Medical Centre (abroad or external), written
confirmation from the host institution is required, specifying who will supervise the student at the host
institution
Selecting your project First of all, students need to decide on the country, city, institution, and discipline in which they mean to
take the RTI. Subsequently, they need to investigate the practicalities and verify whether their ideas are
feasible.
When selecting an RTI, there are two routes for students to take: either they make a choice from the list of
projects available on the web or they take the initiative to arrange an internship themselves. These two
routes are described below.
Project proposals initiated by scientific departments
On the Radboudimc website students can find descriptions or suggestions for RTI projects.
Members of staff in scientific departments often frame project proposals that link to the research interests
of the department concerned. These proposals sometimes are ready made and allow students to make an
immediate start. More commonly, however, project proposals have an open character and describe the
research theme or research methodology in broad terms. In this case, it is up to students and their
supervisors to formulate a concrete internship plan that is feasible in a 12-week period.
Project proposals initiated by students
Students are at liberty to look for or formulate their own RTI position and find an appropriate host for
internship and supervision. A common procedure is for students who are interested in a particular field to
contact a department or a researcher to discuss the option for an RTI. These then jointly formulate an
internship plan, which may match the framework of an ongoing research interest. Then the student needs to
arrange host-based supervision which meets the regulations for supervision and assessment. Students then
contact the Radboudumc supervisor, presenting an outline of the activities proposed at the host organization
and supervision arrangements made at the host institution. The Radboudumc supervisor then decides
whether he/she is prepared to take on final responsibility for the internship. On acceptance, the student
informs the host supervisor.
SELECTING A RESEARCH PROJECT FROM THE WEB-LIST
Student selects a project from the web-list and contacts the organization that offers an internship (often
Radboudumc)
Both host (or daily) supervisor and Radboudumc final supervisor confirm or refute students’
placement.
Student hands in his/her project plan and registration form (see Appendices) to StIP.
Student checks whether the internship proposal has been approved by the RTC and when he/she is
enrolled in the CKO-9 course.
7
Supervision during the Research Training Internship (RTI) A senior member of staff, holding a PhD and being an employee of Radboudumc, has final responsibility
for the RTI. This also applies to internships taken outside Radboudumc. This Radboudumc final supervisor
is also responsible for approving the assessments on the first assessment form. This form, therefore, must
also be co-signed by the Radboudumc final supervisor before being submitted to the RTC.
Internal internship
RTI performed at Radboudumc must be supervised by a Radboudumc member of staff holding a doctorate.
In everyday practice, non-PhD staff can also be involved in supervisory activities. In these cases a senior
member of staff holding a doctorate (the Radboudumc final supervisor) will formally be in charge of
project’s supervision and is responsible for signing the project proposal in advance, and/or signed approval
of the daily supervisor’s assessments on the first assessment protocol.
The (daily) supervisor holds regular supervisory meetings with students and provides day-to-day project
supervision. The internship student takes the initiative for scheduling supervisory meetings. Should any
problems occur, the Radboudumc final supervisor is the person for students to turn to and discuss issues
regarding the discharge of their research tasks at the department. If this does not help to improve matters or
if the daily and final supervisor are one and the same person, students can contact the RTC.
External internship
In the case of external internships (either in the Netherlands or abroad), supervision at the host institution
needs to be guaranteed. The Radbboudumc final supervisor who was responsible for initial project approval
also has the final responsibility for supervision and makes sure that supervision at the host institution is
practised in the right way. He/she is also responsible for signed approval of the host supervisor’s
assessments on the first assessment protocol.
Prior to the start of an external internship the Radboudumc supervisor needs to be convinced of the
likelihood of the internship being completed successfully. In preliminary contacts, attention needs to be
paid to the suitability of the internship topic, its feasibility, host supervision arrangements, and to
assessment procedures. The host institution agrees to provide the required supervision. In the case of an
internship abroad, travel, accommodation, and organizational issues involved need to be taken care of.
The host supervisor at the internship institution is responsible for supervision throughout the internship and
holds regular supervisory meetings with the student. The daily supervision need not necessarily be provided
by the host supervisor; they can delegate qualified replacements. The internship student takes the initiative
for scheduling supervisory meetings. In addition, the host supervisor is responsible for assessing the
internship, both its final product and that the process leading to it. Should any problems occur, the host
supervisor is also the person for students to turn to and discuss issues regarding the discharge of their
DEVELOPING YOUR OWN INTERNSHIP
Student contacts the host organization to discuss options (can also be Radboudumc).
Student presents a draft version of activities and arranges host-based supervision at the host organization
of their choice.
Student contacts a Radboudumc final supervisor.
The Radboudumc final supervisor decides whether he/she is prepared to take final responsibility for the
internship and makes arrangements with the host tutor.
Student contacts the host organization.
Host tutor confirms whether students can or cannot be accepted for an internship.
Student hands in his/her project plan and registration form (see Appendices) to StIP
Student checks whether the internship proposal has been approved or not by the RTC and when he/she is
enrolled in the CKO-9 course.
8
research tasks at the department. If this does not help to improve matters, students should first contact their
Radboudumc final supervisor. If no solution is found, students can contact the RTC.
9
2 ORGANISATION EPISODE RESEARCH TRAINING
Episode Research Training consists of two parts:
- The Central Clinical Education CKO-9 course ‘Professionalising your approach to Medical Research’ and,
- A twelve week Research Training Programme.
Learning Objectives Central Clinical Education (CKO-9)
An inventory of the scientific topics of the Nijmegen research internships shows that the majority of these
projects can be classified as applied medical research. These studies provide quantitative evidence on the
effect of medical interventions, and on the diagnosis, aetiology and prognosis of disease. The remainder are
mechanism studies (how illnesses develop) and health care evaluations.
The CKO-9 course deals with the aims, methodology and procedures of conducting scientific research. Up
to this point in the curriculum, students have engaged with other people’s scientific work, but, in their 12-
week RTI, based on the Plan of Work created in this course, they conduct their own first scientific study.
Central learning objectives
CKO-9 is a two-week basic training course preceding the internship on medical research. After completing
the training course students are able to conduct their internship following the proposed Plan of Work.
Key Features
The course teaches you to:
- relate current medical research (literature) to your research project
- motivate the rationale of your research and address the main research question
- follow pathways resulting in an efficient study design
- understand common study designs, meta-analysis and qualitative studies
- define key variables and how to measure them
- consider the number of subjects / materials / tissues needed
- organize the data collection
- analyse data (computer lab with SPSS)
- organise, manage and report a Plan of Work.
Coordination
The responsibility for the entire episode programme and its execution lies with Episode Research
Training / Episode Onderzoeksstage Coordinator professor A.L.M. Verbeek
- phone 024 -3613102
- email [email protected]
The final responsibility for the Central Clinical Education (CKO-9) lies with Executive Coordinator
dr. J.D.M. Otten
- phone 024-3616979
- email [email protected]
The final responsibility for the Research Training lies with the chair of the Research Training
Committee professor J.P.H. Drenth
- phone 024-3616999
- email [email protected]
10
Research Training Internship
The main objective of the RTI is that students will enhance the scientific understanding they gained in the
Bachelor’s cycle and in CKO parts of the Master’s cycle by deepening, widening, and applying their
understanding to more specific and more complex situations. The internship should be considered an
educational or learning activity, which is also true for preparing the research report: the internship is an
exercise in performing research, not a test of researcher competence. This does not preclude certain students
from actually achieving a more or less independent researcher level.
General learning objectives
The RTI comprises the following general learning objectives. Students can:
1. specify the motive for the study, the interests it serves, and the medical scientific context in which it
fits.(= CKO-9)
2. draft a definitive research proposal on the basis of the following skills:
a. They can retrieve, critically read, and systematically review relevant literature.
b. They can give greater precision or depth to the research question on the basis of the literature.
c. They can formulate a study design that fits the definitive research question.
3. organize and execute the proposed study.
4. systematically analyse the data they have collected.
5. provide clear descriptions of their results and neat summaries in tables and figures.
6. point out errors of measurement and other limitations in the data they collected.
7. critically reflect on the study design and the results.
8. produce a written report that conforms to requirements:
a. In terms of content (justification of the study design, clear description, analysis, critical reflection).
b. In terms of form (carefully edited, trim layout, clear language, references).
9. deliver a concise oral presentation of research results to the department where the study was performed
and engage in productive discussion of its results.
Personal learning objectives of the Research Training Internship In addition to the general learning objectives, there may be specific and personal learning objectives
relating to a particular type of internship: in one type of internship students will need to compose a
questionnaire for data collection purposes, and in another type, they may need to learn how to use a pipette;
in one, they will need to recruit test subjects, in another, they will be working with experimental animals or
with laboratory samples that have already been collected. Sometimes, they need to write an application to
the Central Committee on Research Involving Human Subjects, and sometimes, this will not be necessary.
This shows how a range of personal learning objectives may be required on a case-by-case basis. Students
have to include three specific learning objectives in their Plan of Work. It is important that students and
their supervisors, in advance, agree both the general learning objectives as well as the personal learning
objectives: which specific skills must have been developed by students upon completing their internship?
Assessment Central Clinical Education (CKO-9) course
During the CKO-9 course which lasts two weeks, students prepare a Plan of Work for their own RTI. This
written plan is the final product of the CKO-9 course, to be assessed by the CKO-9 course lecturer on a
pass-fail basis. In case of a fail, the course coordinator will ask students to revise their plan; only when
students have produced the desired result can they officially embark on their RTI.
As additional course requirement, students must have discussed their Plan of Work with their supervisors,
and supervisors must have approved the plan as feasible and as suitable for achieving the internship
objectives. Also the student’s presence and dedication during the scheduled instruction activities, including
the running of the quick tests have to be satisfactorily fulfilled.
Research Training Internship
The RTI will be assessed by means of two assessment protocols. This has been done to validate the
reliability of the assessment procedure. If the first assessment is biased owing to process aspects – for
11
example the project-based collaboration between student and supervisor – the independent assessment of
the written report by a second assessor may redress the balance.
First assessment protocol
The first assessment protocol (see Appendices) is used by the internship daily or host supervisor. If this
supervisor is not the same person as the Radboudumc final supervisor, the latter must express his/her
agreement with the assessments of the former. Especially for foreign internships, adaptation of grades is
often required, because foreign grading systems can deviate substantially from the Dutch system. The
assessment on the first assessment protocol of the Radboudumc final supervisor is decisive. In all cases, the
Radboudumc final supervisor has to sign the protocol.
The first assessment protocol distinguishes between assessment of students’ overall performance (process
aspects) and their achievements (learning outcomes), i.e. the degree to which they have accomplished the
general internship learning objectives. These general learning objectives bear on aspects like the research
rationale, study design, organization and execution, data analysis, discussion of results, and the written
research report. The protocol also records specific internship learning objectives, which, however, do not
contribute towards the final grade.
Second assessment protocol
An independent second assessor will be asked to assess the RTI report (product aspects) by means of a
second assessment protocol. If the first assessment should be biased owing to process aspects – for example
the project-based collaboration between students and supervisors – the independent assessment of the
written report by a second assessor may redress the balance. As the written report (product) may be
considered an account of the students’ learning, it may indirectly and implicitly reveal something about the
general learning objectives of the internship. The report can be assessed objectively by this independent
second assessor as he/she is unaware of students’ and supervisors’ day-to-day conduct throughout the
internship. The protocol used in assessing the report deals with aspects of both content and form (see
Appendices).
Final grade of the episode
As soon as the assessments have been recorded in the protocols concerned, the final grade of the RTI is
determined by the chairperson or a member of the RTC who examines the protocols. Only after a member
of the committee has co-signed the final assessment form, will the final grade be valid.
The final grade of the internship is calculated as follows: the overall performance grade + the general
learning objectives grade + 2 x the written report grade / 4. Each of these separate grades must be a pass for
students to pass their internship. The RTC chairperson acts as the examiner on behalf of the Examining
Board. If the assessments in the first protocol are not validated by the Radboudumc final supervisor or if the
grades on both protocols are too far apart (with a gap of three or more), it is up to the chairperson to take a
decision and decide on measures to be taken, if any. The final result is registered in OSIRIS the student
information system.
In conformity with examination regulations, this episode is given a single grade. The final grade is the same
as the final grade awarded to the RTI. The grade will only be valid if students have passed their CKO.
The above is a paraphrase of the assessment regulations for Episode Research Training / Episode
Onderzoeksstage. The full text of these episode assessment regulations can be found on the Episode
Research Training BlackBoard page.
Attendance and absence (CKO-9) As participation in all scheduled instruction activities at CKO-9 (lectures, seminars and computer lab) in
this course is compulsory, absence will be registered. If students should be unable to attend compulsory
course components, they must inform the course coordinator (via BlackBoard) who will then decide
whether their absence is permitted or not. If a student’s absence is legitimate, the coordinator will assess
whether lost ground needs to be caught up by means of an assignment. If students are absent without leave
or show lack of commitment, they will fail the CKO-9 course.
12
Evaluation The “Instituut Wetenschappelijk Onderwijs en Opleidingen” (IWOO, the Scientific Education Institute)
has, in collaboration with the IOWO, developed a questionnaire to evaluate all components of the clinical
master’s phase. After having finished the RTI and submitting it to StIP, the student receives an
automatically generated email after one week to evaluate both the CKO-9 course and the RTI using our
web-based questionnaire. Students can open this questionnaire in BlackBoard via “Evaluatie / Esurvey” or
directly through http://www.iowo.nl/evaluatie_episode_9.
It will be greatly appreciated if this evaluation is completed immediately after receiving the email.
13
3 TWO-WEEK CKO-9 COURSE
Professionalising your Approach to
Medical Research
Over the last four years, almost 800 medical students have participated in and successfully
completed the CKO-9 course.
Their feedback:
‘I’m glad I took this intensive 2 week crash course in doing research. I wanted to learn the
basics quickly before I started my research internship.’
‘Looking back at the CKO-9 course, I notice that I learned a lot. In the beginning, I honestly had no clue how to start a research project. Now I have a clear plan of approach’ ‘I have more ideas about the possibilities and limitations of my research project. Now it feels
like a personal piece of work.’
‘We learned to make a Plan of Work for a scientific research project. This will be very useful in the near future when I decide to do more scientific research elsewhere.’
You are now nearing your choice of clinical specialism. The CKO-9 course has been designed to
demonstrate how medical research will guide and support you as a practicing specialist. Research
forms the basis for creating new knowledge, which in turn leads to new or improved medical
practice. By experiencing a professional approach to research, we hope that, in the future, a
number of you will continue to be active researchers alongside your daily work in your chosen
specialism.
14
Structure and Learning Components
Main Objective CKO-9
After completing this course you are ready to conduct your medical research internship following the
proposed Plan of Work. To get the most out of your internship we have developed ten modules, which
together with lectures, four work groups and one computer lab on SPSS biostatistics, form the learning
components of the research methodology course. Each of the learning components is described in detail
below.
Blended learning: The CKO-9 way of working
In CKO-9 we stimulate the use of peer review, which you will become familiar with in the first lecture.
This formula is used in all workgroups and is strongly recommended, although not compulsory, for all
modules by working in couples.
The strength of peer review will become clear as you interact with your immediate fellow students and
when preparing the workgroups by means of the modules. Peer review through interaction with the rest of
the class and the lecturer in the workgroup will re-enforce the process of critical reflection. This will help
you when deciding on the various aspects of scientific research and will eventually improve the products
you have to make in the assignments, leading to a reasoned plan of approach for your RTI.
The assignments start with reading chapters in the book of Zielhuis et al., which provide the theoretical
background and are a lead-in to more practical skills necessary for fulfilling the assignments. The course
comprises an interactive mix of teaching and learning: two interactive lectures, ten study assignments, four
work groups including peer and intervision elements, one computer lab session, and individual coaching &
support throughout. These have been designed to help you independently understand and apply the
principles and methods of predictive and causal research. At the end of the second week, students have to
defend their written Plan of Work. The course text is in English, whereas the course itself will be delivered
in Dutch.
COMPULSORY TEXTBOOKS Zielhuis GA, Heydendael PHJM, Maltha JC, van Riel PLCM. Handleiding Medisch-wetenschappelijk
Onderzoek. Zesde herziene druk. Amsterdam: Elsevier Gezondheidszorg, 2010, pp. 220.
Petrie A, Sabin C. Medical Statistics at a Glance. Oxford: Blackwell Publishing Ltd, 2009, 3rd
edt.
Fletcher RW, Fletcher SW. Clinical Epidemiology: The Essentials. Philadelphia: Lippincott Williams
& Wilkins, 2005, 4th
edt.
Team of teachers
Course coordinator Central Clinical Education (CKO-9) dr. J.D.M. Otten
- phone 024-3616979
- email [email protected]
Teachers:
Dr. J.A.A.M. van Dijck (department for Health Evidence)
Dr. J. Fransen (dep. of Rheumatology)
Dr. M.M..H.J. van Gelder (department for Health Evidence)
Drs. H. Groenewoud (department for Health Evidence)
Dr. R.J. Melis (dep. of Geriatrics)
Prof. dr. A.L.M. Verbeek (department for Health Evidence)
15
Lecture 1: PROFILE OF CKO-9
What’s it all about? In this kick-off lecture we get acquainted with each other and consider the aim of the RTI: namely to
acquire knowledge. Before starting the internship, you have to develop a reasoned Plan of Work. How? In
this lecture, we present a structured approach that will convert your enthusiasm and curiosity for your
research project into an informative summary accompanied by an appealing title. In the next two weeks,
you will refresh your general research skills in pairs and peer-review your work in seminar groups. Finally
you will write and defend your own Plan of Work.
INTRODUCING YOUR RESEARCH SUBJECT The main part of the exercise lasts about twenty minutes (for groups of two to three students) and
gives you the chance to find out a little about yourself and your partner through some 'what-will-the-
project-be-about' questions. You work together to develop a précis of your research project as well as
creating an appealing and appropriate title. You can present both in class, and you can give each other
(positive) feedback on your proposals.
Simple questions to help 'break the ice' are, for example, those about your research project, its title, the
medical topic, where it’s conducted, the institute, or what your interests are.
After twenty minutes, one or two of you will be requested to present your project: title, type of
research, and most importantly, the précis.
It is also important to think about the kind of research type (see next page)
Tips!
- Demonstrate the value of your research to others.
- Make your title active.
- Show that you are committed to your research project.
Learning objective
Students can give a lively and interesting précis of their internship project.
Précis: (Longman Dictionary) Fr, a shortened form of a speech or piece of writing, giving only the
main points.
ELECTIVE TEXTBOOKS ON SPSS: de Vocht A. Basishandboek SPSS 20.Eerste druk. Utrecht: Bijleveld Press, 2012, pp. 255.
QUALITATIVE STUDIES: Lucassen PLBJ, Olde Hartman TC. Kwalitatief onderzoek. Praktische
methoden voor de medische praktijk. Bohn Satfleu van Loghem. Houten 2007, pp.
META-ANALYSIS: Egger m, Smith GD, Altman DG. Systematic reviews in health care. Meta-
analysis in context. London: BMJ Publishing Group, 2001, 2nd
edt.
16
Type of research
Medical research is defined by its object and its methodology. It can be divided into four basic and distinct
categories, each with a different kind of study object, design, data collection, analysis and reporting.
- The first type is diagnosis (e.g. developing new tests). This kind of study can be presented, for instance,
with outcome measures like Sensitivity & Specificity and analysed by using a ROC-analysis.
- The second is treatment (comprising medical intervention in general and health care research). A
survival analysis is useful – for example presented as the median of 5-years survival etc.
- The third type is aetiology (pathogenesis, how does disease develop). A specific tool in this case is for
instance the use of a case-control design, presented with odds ratios.
- The fourth addresses prognosis of disease (how does disease progress, what’s the mechanism). Making
a Clinical prediction rule is for instance the possible solution here, with estimation of the risk for a
certain outcome.
Thanks to developments in techniques in biomedicine, new categories are continuously being added to this
list. For instance, the internist Alvan Feinstein added ‘clinimetrics’, but in essence this is a tool, technique
or measurement instrument and not a special medical research category. Biomarkers, computer-aided
cytological testing, measurement research (correlation), imaging, and genetic measurements have also been
added. Some methodologists even have added EBM, meta-analysis, biostatistics, bioinformatics, library
science, web-based questionnaires and qualitative studies to the list of research methods. In essence, all of
these new methods can be brought back to one of the four research topics described above.
Question
Does this taxonomy of types of medical research objects and research methods or techniques make any
sense to you? What kind of research would you like to get involved in? We’d love to hear where your
research fits in.
Collocation of medical domains with methodology
Which methodology in B is commonly applied in the medical domains of A? Match a methodology in B to
a type of research methodology in A. There may be more than one possibility. For instance i with 1.3 etc.
If you don’t recognize some of the items, use Wikipedia as a quick and dirty method and take this for
granted. (See also in Fletcher, Clinical Epidemiology, The Essentials Table 12.1 Matching the Strongest
Research designs to Clinical Questions).
A
1 diagnosis
2 medical intervention:
treatment, prevention
3 etiology, pathogenesis
4 prognosis
B
i. qualitative study
ii. evidence-based medicine
iii. clinimetrics
iv. biochemistry
v. clinical psychology
vi. pilot study.
vii. polygenic approach
viii. survey
17
Précis: The temptations of chocolate
Based on scientific research, consumption of flavenoid containing products like green tea,
red wine, etc. seem to be associated with a substantial reduction in the risk of cardio-
metabolic disorders. There are studies suggesting that consumption of dark chocolate,
containing flavonole, has a positive influence on human health in particular by
antihypertensive, anti-inflammatory, activation of nitric oxide and other beneficial effects.
We want to carry out a study to evaluate whether an intervention of daily consumption of
dark chocolate will lower the blood pressure in normotensive persons.
Module 1: RATIONALE OF THE RESEARCH
Is my research worth doing? There must be a valid reason for launching a scientific study: a question that has been raised by clinical
practice, (applied) scientific researchers, or society and that needs to be addressed. Both for yourself and for
all other players in practice, it is vital that the motivation of why the research should take place is stated
clearly and concisely (remember the word précis). If you can do this, you will find that you will be able to
communicate the content and importance of your research to peers.
Learning Objective
Students are able to demonstrate that their research is worthwhile and relevant. The information should
include: what their project is about, why it should be conducted (the motivation, either from a scientific or
societal viewpoint), why and for whom this medical knowledge is important, and what the expected impact
of the research outcome will be.
Assignment
1. You are requested to study the textbook by Zielhuis et al, Chapters 1 and 2 until step 4 on page 33.
2. In order to prepare for the assignment, take Quick Test 1, which can be found on Blackboard. The
test contains thirteen multiple choice questions. It highlights, at an introductory level, concepts like
rationale, relevance and reflections on doing research, as well as type of medical research.
All 10 Modules of the CKO-9 course will succeed with a Quick Tests after reading chapters from
Zielhuis et al.
3. After reading the text from Zielhuis et al, you should be able to answer questions a – d.
a. Which function does medical research have?
b. Research methodology consists of four types of criteria. What do ‘transparency’, ‘validity’,
‘generalizability’ and ‘precision’ refer to?
c. Read the twelve steps in the tracking of medical research and think about which step best
fits the 10 modular learning components of CKO-9 (see Table of Contents, Chapter 3, and
all learning objectives of the ten Modules).
d. What’s the difference between your ‘rationale of research’ and ‘rational research’? Why is
your rationale rational?
4. Look at the title (4 words) and the précis (86 words) about chocolate (in the box below) and answer
the questions e- g.
e. Is the title appropriate? Why, why not?
f. What will this study be about?
18
g. Is there a rationale to do this research? To get going we will show you the so-called BMJ
boxes. So far we know, the British Medical Journal, BMJ was the first journal where
authors are required to accompany their article with some statements about the study.
Please see the examples below.
5. Now it’s time for your research. Write an appropriate title for your research in 15 words. Complete
your précis in 75 words. Remember, a précis should cover relevance and content of your research
altogether. Then, answer the question “What’s the rationale of your research?”
To start up with the rationale, make a BMJ-box for your study, providing 3 statements about
“What your study will (probably) add”. Write your rationale in 220-260 words. These three
elements will form part of your Plan of Work (to be completed in Module 10).
Tips!
Tip 1 - Before you start to write, you should spend at least 15 minutes planning your writing. Your
answers should be well organised with clear linking between sentences and paragraphs. Remember
from your Academic Writing Course: your written work should have a clear beginning, middle and
an end, an appropriate style and layout, and be written legibly as you will be assessed by the target
reader(s) of your Plan of Work.
The rationale has to be convincing in order to write a powerful Plan of Work. Information should
be relevant and your rationale should be concise: “Less is more”, or “Kort is Prachtig”, or “In der
Beschränkung zeigt sich der Meister” or “La limitación se muestra por el maestro” etc.
Tip 2 - Start writing your Plan of Work now. This take-home message was launched by Professor
Jan Vandenbroucke in 1989. Read his text in Vandenbroucke JP . Een recept voor een klinisch-
wetenschappelijk leeronderzoek. Ned Tijdschr Geneeskd 1989; 133: 34-8 (especially pages 37-38
“Vroeg opschrijven”.
Tip 3 - Another Tip and Take-home message! If you have problems writing good English
sentences, please take a look at the Academic Phrase bank
(http://www.phrasebank.manchester.ac.uk/index.htm).
WHAT THIS STUDY ADDS (chocolate and blood pressure) Consumption of dark chocolate, containing cocoa and a richness of flavonoles, will have a
beneficial effect on blood pressure by lowering the systolic and diastolic pressure with 5-10 mm
Hg.
The diminishing effect for consumption of milk chocolate (more cocoa butter and less cocoa) and
white chocolate (no cocoa) indicate a dose-response relation of cocoa in the reduction of blood
pressure.
19
Module 2: FIRST APPRAISAL OF THE LITERATURE What have the ‘others’ been doing?
In the previous module, you discussed the motive or rationale of your research project and internship. A
literature search is a logical consequence of developing your research motive: you need to understand what
others are doing and what the state-of-the-art is in this field.
Learning objective
By doing a first literature search, students are able to select the three most important papers which are
related to their project. They can present the ‘state-of-the-art’ of the research problem and can criticise these
papers.
Assignment
1. First, read Chapter 4 on “Literatuur en literatuuronderzoek” from the textbook by Zielhuis et al.
which focuses on the words and phrases in the box below.
2. After reading Chapter 4 from Zielhuis et al, you should complete the Quick Test No. 2
Here another five questions a – e about doing a literature search.
a. If you want to speed up your literature search, is Google of benefit? Why? Why not?
b. PICO is especially connected with one of the four types of medical activities. Which?
c. As in any other science, researchers like to structure their publications. In evidence-based
medicine these are registered in primary, secondary and tertiary publications. Give examples of
each kind of these papers.
d. Will the Science Citation Index be of value to you?
e. To become familiar with your research object (and if Dutch is your (near) native language),
try:
“Google zoeken: ntvg.nl - Nederlands Tijdschrift voor Geneeskunde| - Uitgebreid zoeken”.
3. We would now like to learn something about your ability to search for literature, to raid libraries
and delve into electronic archives when doing research. So, select the three most relevant articles
in relation to the rationale of your study and download the papers. Describe your search strategy,
and make a reference list in the style of the references in the box below.
For the three references you have selected, extract the relevant information for your research and
make a well-ordered table. Tabulate information like study units or population, design, sample
size, measurement instruments, determinant, outcome and, importantly, all notes you made on
strong and weak aspects of the studies.
Example: Temptations of Chocolate on next page
Critical appraisal – MeSH – NTvG uitgebreid zoeken – State-of-the-art medical knowledge
20
...
4. To get going we will first show you the BMJ box “What is already known on this topic” on our
chocolate-blood pressure item. Then you are requested to make a box like this for your own study.
Search for evidence – Between May and August 2011 we carried out a search for the most recently
published studies to identify the main elements for our research question on the influence of dark
chocolate on blood pressure. We used two databases for the literature search: PubMed and Web of
Knowledge. The terms chocolate, chocolate* [Title/Abstract], cocoa [MesH], cocoa* [Title/Abstract]
were applied in combination with blood pressure [MeSH], blood pressure* [Title/Abstract].The
bibliographies of selected references were also searched. More than 40 articles were found from the start
of 2003 to mid 2011, of which 5 met the specified inclusion criteria: [add your specifications here]
normotensive healthy persons.
Eventually, we selected three articles for the first critical appraisal in which we extracted technical
information on populations, study design, numbers of subjects, measure instruments, etc. The selection
was based on [add your selection criteria here] two systematic reviews and one study on working of
polyphenoles which was excluded in the meta analysis.
In Table 1, we made an overview of the main findings and notes on my literature appraisal.
Table 1. Title [part of it]
No. Author, yr Design Main findings Criticisms Recommendations
own research
1. Ried, 2010 Meta-analysis …………………… …………… ……………………
…………………… …………… ……………………
…………………… …………… ……………………
2. etc.
References
1. Ried K, Sullivan T, Fakler P, Frank OR, Stocks NP. Does chocolate reduce blood pressure? A meta-
analysis. BMC Med. 2010;8:39.
2. Desch S, Schmidt J, Kobler D, Sonnabend M, Eitel I, Sareban M, Rahimi K, Schuler G, Thiele H.
Effect of cocoa products on blood pressure: systematic review and meta-analysis. Am J Hypertens.
2010;23: 97-103.
3. Cienfuegos-Jovellanos E, Quiñones Mdel M, Muguerza B, Moulay L, Miguel M, Aleixandre A.
Antihypertensive effect of a polyphenol-rich cocoa powder industrially processed to preserve the
original flavonoids of the cocoa beans. J Agric Food Chem. 2009;57:6156-62.
WHAT IS ALREADY KNOWN ON THIS TOPIC
Studies on Kuna Indians reveal that high cacao intake is correlated with low prevalence in cardio
metabolic disorders. Migration and changes in diet suggest cacao as an explanation for this
phenomenon and not a genetic predetermination.
High intake of cacao polyphenols lowers the risk of cardio-metabolic disorders. Some fruit (berries grapes
etc.), red wine, green tea and cacao products are Ployphenol rich.
In vitro research suggests that cacao in dark chocolate has an influence on vascular endothelial function
and activation of nitric oxide.
21
5. Finally, guided by the table you made under instruction 3 (which in this stage is based on the three
articles you have selected), write your critical appraisal (i.e. strong and weak points inclusive) in
220-260 words in an appropriate scientific style.
Start with state-of-the-art evidence (step 4, Chapter 4, textbook by Zielhuis et al). Include all
information which is worthwhile and relevant for your research! It will help you when formulating
your research objective and research question (Module 3).
Tip!
For search strategies, see BlackBoard Hulp bij literatuuronderzoek with MeSH-terms. In case of a
systematic review, meta-analysis, qualitative study or any problems with your search, you may
contact our Medical Library as described on BlackBoard, or ask your teachers.
22
Module 3: SPECIFYING THE RESEARCH QUESTION What exactly will I be doing?
In this module, we will deal with the link between the state-of-the-art for the literature and the rationale of
your study which will result in your Research Question. An idea for scientific research does not just ‘jump
out of the blue’; it is often generated by an observation or by the results of previous research. This leads to a
preliminary formulation of the Research Question, which is sometimes converted into a hypothesis and
tested by the researcher with observations in practice. This is called ‘empirical’ research. Based on the
formulation of your Research Question, you determine what the best form of research is: will it be a pilot
study, a qualitative study, or a full-blown quantitative research programme?
Learning objective
Students can translate the scientific objective into one (yes, 1!) well-defined Research Question and/or
testable hypothesis. Furthermore, they can relate the Research Question to its outcome, its determinant, the
timing of determinant and outcome and the relevant domain.
Assignment
1. Today’s theory revisits some texts you have read before.But firstly, please consider page 35,
section “De beperking van de vraagstelling” in JP Vandenbroucke’s Een recept voor een klinisch-
wetenschappelijk leeronderzoek. Ned Tijdschr Geneeskd 1989; 133: 34-8.
Reread Chapter 2, step 1-3 in Zielhuis et al.
2. Complete Quick Test 3 that consists of thirteen questions on the theme of specifying a Research
Question.
Now, focus on the 11-word Research Question “Does consumption of chocolate have influence
on IMT and blood pressure?” Apply the seven assessment criteria (Table 2.2 in Zielhuis et al.)
to the Research Question in order to demonstrate the (in)-appropriateness of the scientific
question by answering questions a - c.
a. What does the RQ demonstrate about the criteria of unambiguity, singularity, simplicity
and formulation?
b. Is the RQ scientifically relevant? How do you know this research is worthwhile doing
(what did you say in module 1)?
c. Are determinant and outcome specified?
3. Read the extract below which is about particularistic (or factual knowledge) and abstract
knowledge. It has been paraphrased from Bouter – van Dongen – Zielhuis’ textbook on
Theoretical Epidemiology (2009).
d. Does the RQ on chocolate and blood pressure address particularistic or abstract
knowledge?
e. What do you think about (determinant-outcome) timing and domain?
PARTICULARISTIC VERSUS ABSTRACT KNOWLEDGE
Research questions describe issues that either refer to particularistic or abstract knowledge.
Particularistic research intends to affirm propositions about a specified relation between determinants
and outcomes under given circumstances of time and place. Most of these studies are used to map
health conditions (disease frequency in relation to, e.g., age, gender, SES), use of medicines in health
surveys, and evaluations or bench marking of medical interventions. (See next page)
23
4. Application task - Look at the Précis you made in Module 1. What is your domain? Does it
concern a specific target population or an abstract general situation?
Conceptualize and write your Research Question in an appropriate style and number of words.
5. You will need all the products of the tasks in Modules 1-3 in Seminar 1. Are you happy with them?
Did you succeed in writing the outline?
Seminar 1 is a small class where you and your fellow medical students together with a specialised
lecturer / researcher will be meeting to peer-review your rationale-based Research Question and to
improve your approach to developing your Plan of Work. Read the main text for Seminar 1
carefully to get a general idea of the programme.
This kind of description is of value to support policy makers, doctors in their daily medical decision
making, and to direct future research. An example of particularistic research is an investigation of a
Salmonella outbreak in a nursing home.
In contrast, abstract knowledge is devoid of content that is time-place dependent. Abstract knowledge
is expected to be generalizable to all human-kind (even to other species). With this kind of research
we aim to study general, abstract, scientific knowledge on (causal) determinant-outcome associations
or disease mechanisms, which are seldom descriptive ‘by nature’. Think about the question whether
alcohol (abuse) is a causal factor for liver cirrhosis. The causal link between alcohol abuse and liver
cirrhosis holds true throughout the world; it is not scientifically restricted to De Achterhoek, the
Netherlands in 1977, or the Castro-Castro Prison in Peru in 2011.
Once a link like this has been scientifically established, next steps in medical research will be aimed
at gaining further insights into types of alcohol abuse, types of users, and perhaps type of cirrhosis as
well.
NB: For particularistic research problems, the specific features (species, time and place) of the study
must be reflected in the research question. For abstract research problems, these features are usually
not part of the Research Question, but described in the design part of the study. Adapted from: Bouter LM, van Dongen MCJM, Zielhuis GA. Epidemiologisch onderzoek. Opzet en interpretatie. Houten:
Bohn Stafleu van Loghum; 2010: pp 81-2.
24
Seminar 1: A FIRST OUTLINE OF YOUR RESEARCH
Is it relevant?
To stress the focus of your research topic, you are going to sum up rationale of your study and state-of-the-
art of the literature in so-called British Medical Journal (BMJ)-boxes: What is already known? And ‘what
will my study add?
In addition, the most important issue in designing your research is to come up with a crystal clear Research
Question (RQ). Your RQ must meet all the requirements of scientifically feasibility completeness,
uniformity, singularity, scientific relevance, simplicity, and correct formulation. In the peer review session
of this seminar meeting you will be asked if your RQ meets these requirements.
Learning objective
Students can outline their proposed study conceptually and defend its relevance to fellow students, leading
to an appropriate research question (RQ).
Based on their RQ, the students are able to look ahead to designing the data collection, the analysis and its
provisional embedding in a societal and ethical context. These will be discussed in Modules 4-8.
Workgroup process
1. First we start with a moment for reflection: What have you learnt most from Modules 1-3 on
Rationale, Literature and the Research Question? Gather all your learning experiences, pieces of
information, and evidence from the assignments so far. Remember Vandenbroucke’s adage “Vroeg
opschrijven”. In how far have you succeeded in writing the outline (background or rationale, aim
of the study etc.) of your Plan of Work (major issue of Module 10)?
2. Now we will evaluate your experience with the quick tests and the modules you have taken so far.
Do you understand the theoretical components? This evaluation will be supported by a
presentation on the effects of chocolate intake on blood pressure in adults.
3. Then, in small groups of 3 students, review the outline of your study using the précis you made in
Module 1. Highlight the state-of-the-art of your research topic (which is a product of Module 2)
answering the question “What is already known on this topic” in three concise statements. Then,
present the rationale of your research topic (also a product of Module 1) by means of three
statements in a separate box entitled “What does my study add”. Keep both preliminary BMJ-
boxes in stock for Module 10 when, amongst all the other issues, your Plan of Work approaches its
final stage.
4. Then still within the peer review format, scrutinize your research question based on the criteria in
Table 2.2 in Zielhuis’ textbook. You now have to submit your RQ to the lecturer. You will receive
feedback in the following seminar.
5. During the last part of the Seminar 1 the group will prepare for the following modules on
designing your data collection.
25
Module 4: PATHWAYS TO EFFICIENT STUDY DESIGN Designing the yellow brick road
The next three modules are on data collection. Data collection has two forms, namely collecting in the
conceptual dimension and in the operational dimension. The conceptual dimension is about how to design
your data collection efficiently and is addressed in Module 4. The operational dimension of data collection
concerns the study subjects to be measured; this is addressed in Modules 5 and 6.
In Seminar 1 you finalized your Research Question. During this process, you have unwittingly thought
through the research typology and methodology. Types of methodology are pilot studies, descriptive
studies, and explorative and hypothesis testing. To harvest scientific information for your Research
Question you have to shape or form the data to be collected. A template can help you to gather the
empirical facts. There are two standard templates: a survey-like study design and an experimental
investigation. Both forms are applied to help you ‘mould’ your Research Question to arrive at a
representative, valid, reproducible and reliable answer for your target reader.
Learning objective
Students demonstrate that their study makes sense; that it will produce a valid answer to the Research
Question. They are able to do so by selecting the optimal study design which will allow them to effectively
obtain empirical data from the study domain.
Assignment
1. In what way will your study conceptually be set up? The title of Module 4 “Pathways to Efficient
Study Design” suggests two ways: the practical and the nominal way. In a practical pathway you
ask the question, “How can I efficiently harvest the information?” In the nominal pathway you
choose a study design from a list of types such as experimental studies, observational studies, in
vitro studies, etc.
Now read Chapter 2 (steps 4 - 8) and Chapter 3 (Vormen van onderzoek) of Zielhuis’ textbook.
2. To become further acquainted with the topic, match the concepts in the box below to the thirteen
questions in Quick Test 4.
3. Please, answer the extra questions a – f.
a. Describing the overall design consists of five elements. Which?
b. How do you sketch or model a study design? See for example the sketch for the chocolate
study on the next page. What are the essential elements?
c. What’s the difference between an experiment and a descriptive / survey-like study?
d. To what extent is tissue engineering a type of research?
e. Do you happen to know what, in medicine, is meant by a translational study?
f. Assess the appropriateness of the designs chosen for the question whether chocolate intake
does lower blood pressure? See the textbox below with three researchers and their proposals.
Design (noun) - Design (verb) – Experiment - Pilot – ‘Trial and Error’
26
4. Based on the Précis and Research Question of your project, select a suitable design (use Zielhuis,
Step 4, p. 33 and Chapter 3) and sketch the study model (Fig. 2.1 and 2.2 in Zielhuis, p. 37).
Write and motivate the design of your study project in 75 words, as shown in Zielhuis’ Examples
2.14 – 2.20.
Tip!
Still puzzled and fascinated about your study design? Then read: Beurskens AJHM, de Vet HCW,
Kant IJ. Dwalingen in de methodologie. Pilotonderzoeken: zin en onzin. Ned Tijdschr Geneeskd
1998; 142: 2142-5.
For matching the strongest research designs to Clinical Questions the box below may help.
Temptations of chocolate
Researcher1
(Born in Belgium and now working at University Hospital Maastricht)
Study design: Survey, comparison systolic and diastolic blood pressure in medical students in
Brussels and Maastricht.
Researcher 2
(Researcher appreciating the article of Grassi et al. Am J Clin Nutr 2005;81:611– 4 and decided on
a “me too” study)
Study design: Cross-over design, comparison of blood pressure (systolic and diastolic) before and
after consumption of dark chocolate.
Researcher 3
(PhD student, just followed a course on randomised clinical trials)
Study design: Randomised trial, random allocation of different volumes of cacao to a group of
medical students.
Question Design
Diagnosis Survey study
Prevalence Survey study
Incidence Cohort study
Risk Cohort study
Case control study
Prognosis Cohort study
Treatment Clinical trial
Prevention Clinical trial
Cause Cohort study
Case control study
27
Module 5: MEASUREMENT OF KEY VARIABLES
What am I measuring?
Medical research is founded on observations: measurements. To perform these measurements, one needs
instruments. Measuring instruments include not only things like rulers or devices but also questionnaires,
visual inspections, chemical or laboratory analyses, and registrations. For both dependent, independent
variables and for all other variables, measuring instruments are necessary. But can you really rely on your
instruments? What happens if the reliability is not guaranteed?
Learning objective
Students can describe how they measure the key variables of the Research Question, and which instruments
they will use. By doing so, the students show that they understand the (potential) magnitude of
measurement errors and their effect on the research result.
Assignment
1. Now you’re on the verge of taking the fifth Quick Self-test. Quick Test 5 addresses seemingly
unrelated topics, but they are all about with variation and measurement. They are about the
constructs: ‘variable’, ‘random vs. systematic measurement error’ and ‘inter-/intra-rater
variability’.
2. Module 5 theory focuses on the qualities of measurement instruments. Read Chapter 7 in Zielhuis’
book. Then answer questions a – e.
a. Is ‘taking medical history’ a type of measurement instrument? What about the accuracy? And,
how can you learn to quantify its accuracy?
b. Have you ever worked with depression scales? Or the APGAR test? How many items were
involved? Do you agree or disagree with the number of items needed to accurately measure
somebody’s health status: The more, the better?
c. Describe two types of measurement research.
d. What’s the purpose of the kappa statistic?
e. Is measurement research an issue in your project? If so, see the textbox below.
TO AGREE TO DISAGREE ON MEASUREMENTS
- De Vet HCW, Beurskens AJHM. Reproduceerbaarheid van metingen. Ned Tijdschr Geneeskd
1998; 142: 2040-3.
- Pouwer F, van der Ploeg HM, Bramsen I. Bias door vragenlijsten. Ned Tijdschr Geneeskd
1998; 142: 1556-8.
- Van Breukelen GJP, Berger MPF. Hoe te komen tot een gewogen totaalscore van
afzonderlijke scores op een vragenlijst. Ned Tijdschr Geneeskd 2000; 144: 1580-4.
- Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of
clinical measurement. Lancet 1986; i: 307-10.
- Petrie A, Sabin C. Medical Statistics at a Glance. Oxford: Blackwell Publishing Ltd, 2005, 2nd
edt. Chapter 39: Assessing agreement.
- Fletcher RW, Fletcher SW. Clinical Epidemiology: The Essentials. Philadelphia: Lippincott
Williams & Wilkins, 2005, 4th edt. Chapter 2: Abnormality.
28
3. At least three different kinds of variables can be distinguished in every study (see Figure 2.1 in
Zielhuis et al, Chapter 2, page 37). Look how these have been registered in Table 1, with four
aspects A – D for each kind of variable.
Table 1. Example - Register and order the main variables of your research. Three examples are
presented.
CENTRAL / KEY
VARIABLE
DETERMINANT OUTCOME FURTHER
CHARACTERISTICS
(confounding factor,
intermediate, or domain
variable)
VARIABLE*
e.g. AGE e.g. BLOOD PRESSURE e.g. PHYSICAL FITNESS
A - Conceptual scale
B - Measurement scale
C - Measurement
instrument
D - Sensitivity to
measurement error
A - young, middle aged, old
B - year of age
C – questionnaire D - not at issue.
A - hypertension, yes/no
B - systolic blood pressure, mmHg
C - automated Sphygmo type® D – reliability being tested yet.
A - poor, borderline, sufficient, good, excellent
B - predicted VO2max,
ml/min/kg
C - Åstrand-Rhyming sub maximal cycle exercise test D - reproducibility study published by JT Groothuis et al. (in press)
*) Something which represents something (such as temperature) which can vary in quantity or size
For your study, list the key variables when measuring determinant, outcome and further relevant
characteristics in Table 2. Complete the list with useful information regarding question marks I-V.
I. Indicate the likelihood (low, medium, high) of the occurrence of random measurement
errors.
II. And what about the occurrence of systematic errors?
III. What makes you think that the likelihood of making measurement errors will be great or
small?
IV. What can you do to reduce the potential of making serious errors of measurement?
V. How do you judge the impact of the errors on the study result?
29
Table 2. My project - Main variables of my research project.
CENTRAL / KEY CONCEPT DETERMINANT OUTCOME FURTHER
CHARACTERISTICS
(confounding factor,
intermediate, domain)
MAIN VARIABLE
A - Conceptual scale
B - Measurement scale
C - Measurement instrument
D - Sensitivity to measurement error
I - random measurement error
II - systematic errors
III - likelihood of occurrences of
measurement errors
IV - how to restrict errors
V - impact on study
30
Module 6: CONSIDERING THE STUDY SUBJECTS Who and what do I need?
In Module 4 you decided what your study population would be: humans, fruit flies, or (bio)materials. Now
you will focus on different aspects of this population. For instance, how many subjects do you need for your
study? This question relates to the accuracy with which you want to estimate the outcome measures of your
research object.
Consider study populations or research subjects in a broad perspective, for instance, “the people to be
interviewed” in case of a qualitative research, or a specific group of articles about a subject in case of a
systematic review or meta-analysis.
Learning objective
Students can qualify their choice of research subjects (representative vs. selective) for the study. They show
how they determine the number of subjects to be used and how they will be recruited. Additionally, they
can demonstrate the effect of sample size on the confidence interval of an effect parameter.
Assignment
1. The theory in this module is preceded by Quick Test 6, which consists of 13 two/three-option
multiple-choice questions on the concepts below. Which conceptual qualities are you confident of?
2. Developing your skills – As the CKO-9 course develops, you are becoming familiar with the fact
that you always have to motivate your decisions at all stages of your research. Today’s stage is
about study subjects and sample size. How to be precise with minimal effort? Before reading about
formulas for sample size calculation in Zielhuis, let’s take a look at the reasoning behind these in
the next boxes.
1 study unit
2 numbers needed to study
3 random sample
4 representativeness
5 selective study population
6 type of sampling
A population represents the entire group of individuals in whom we are interested for the value of a
certain parameter. We usually cannot study the entire population, so we have to take a sample instead.
After measuring a number (size = n) of the subjects, we can calculate the mean. To have an idea about
the variability of our measurements, we can take many repeated samples of size n from the population
and look at the distribution of all the means (see Chapter 10 in Petrie and Sabin, Medical Statistics at a
Glance). This variability of the distribution of the mean is known as the Standard Error of the Mean
(SEM) and can be calculated by SEM = SD/√n in which SD is the standard deviation of the sample
and n the sample size.
With this SEM we can calculate the 95% Confidence Interval (CI) of the mean. In a formula: 95% CI:
[sample mean - 1.96 * SEM, sample mean + 1.96 * SEM].
This factor of 1.96 (Zα/2) is because, in the theoretical normal distribution, numbers more than 1.96
standard deviations away from their mean occur with a frequency of less than 2.5 % (= α/2).
See Example 1 below for some practice with SEMs and calculating 95% CIs.
31
The temptations of Chocolate Example 1
Study design: ecological study, comparison of systolic and diastolic blood pressure amongst
medical students in the cities of Brussels and Maastricht.
Findings: The mean systolic blood pressure was 130 mm Hg (SD 10) for 16 medical students in
Brussels. In Maastricht, for the same number of students, the mean systolic blood pressure was
135 (SD 10).
Now we can calculate the SEM: 10/√16 = 2.5 mmHg for both groups. The 95% CI for the
Brussels group is about [130 – 5 , 130 + 5] = [125 - 135]. For Maastricht the outcome is 135 mm
Hg, 95% CI [130 - 140].
Further questions:
- Notice, there is an overlap between the 95% CIs of the mean blood pressure in Brussels
and in Maastricht. What is your conclusion? Are both means significantly different?
- Can you discern the difference between the means in the two cities by calculating the
95% CIs in case 100 medical students were measured for each group? What is your
conclusion now?
Of course the number of subjects also influences the precision of a proportion (e.g. the sensitivity
of a test (%). The calculation of the standard error (here of the Proportion, SEP) is somewhat
different. The formula for this SEP = √ (p*(100%-p)/n, where p is the proportion, 1-p is the
complement of p and n the sample size.
For calculating the 95% CI of the proportion the same formula can be used: [proportion - 1.96 *
SEP, proportion + 1.96 * SEP].See example 2 for calculating SEPs and 95% CIs.
Example 2
Study design: before/after design, comparison of blood pressure
(systolic and diastolic) before and after consumption of dark
chocolate in 15 subjects.
Findings: See figure for the systolic blood pressure before and after
two week’s consumption of dark chocolate. In 13 of the 15 medical
students (87%) the systolic blood pressure showed a decrease of
more than 3 mm Hg.
Questions:
- How ‘precise’ is our result of 87% of students with reduced
blood pressure?
Calculating the SEP: (√ (p*(1-p)/n = √ (87 %*( 100 - 87% / n= 8.7
Now the 95% CI: CI= (87% - 1.96*8.7%, 87% + 1.96*8.7%) = (70% - 100%).
Is this a reasonable outcome for you? Why not fill out 104% as the upper boundary?
- We studied another group of students in which white chocolate was prescribed.
Suppose we found a reduced blood pressure in 53% of the students i.e. 8 out of 15 or
53 out of 100 students. Is this outcome (53%) for white chocolate different from dark
chocolate in a study group of 15 students? And in case in both groups 100 students
were studied?
3. For further theoretical considerations, read Chapter 6 ‘steekproefomvang en powerberekening’ in
Zielhuis’ textbook. This time there are no questions to answer, just mark all sections which are
directly beneficial for your research and can easily be applied. If you face difficulties with the
theory, discuss your difficulties in Seminar 2.
32
4. In this task you are going to write around 250 words on the number of units needed in your study,
and how they will be recruited. The task focus is on the reasoning behind the number, rather than
the fact that your supervisor told you that you needed ‘so many subjects’. Do you really need to
work with the whole group of study units, or should you give priority to a sample - random or
selective? To help you, look back at the articles you selected in Module 2 on ‘literature’. What
numbers did they use and how were these motivated?
Requirement!
In your Plan of Work you will need a numerical motivation for your study size. What are the three
main qualitative reasons behind reaching this point? (Think about time, effort, and your role in the
research process).
Tips!
In your Bachelor of Medicine period you studied biostatistics using the book by Petrie & Sabin
“Medical Statistics at a Glance”. In the 2nd
edition (Oxford: Blackwell Publishing Ltd, 2005),
Chapters 10 and 11 focus on quantitative population parameters like proportions and means. As
you may have noticed, the precision of these estimates are of value for your numerical
considerations on the number of subjects needed. Refresh your knowledge on 95% CI, (confidence
intervals).
Some like to cast their Research Question in terms of hypothesis testing: Is the outcome (mean or
proportion) in group A different from that in group B? T-tests are commonly applied when
comparing means; Chi-square tests [pronounce ‘chi’ as ‘kai’] are commonly applied when
comparing proportions. For details of hypothesis testing and these statistical tests, consider Petri &
Sabin.
5. Finally, prepare well for Seminar 2. Skim the main text of Seminar 2 for a general understanding
of the theory and as preparation for peer review.
33
Seminar 2: DESIGNING YOUR DATA COLLECTION
Learning objective
Students can outline their study design to fellow students and justify the choices for the study units, key
variables and measurement instruments. Furthermore, they can underpin the number of subjects needed and
can present their expected research outcome using a graph and/or table.
Seminar way of working
As in Seminar 1, Seminar 2 includes three parts: review of the theory from Modules 4-6, peer review on
‘collecting the right data’, and reflection on the next modules.
1. First things first: are there any questions left after completing the Quick Tests and Modules 4-6?
And having received all kinds of feedback on your RQ, is everything crystal clear to you and your
peers? How many study subjects or units form part of your project?
2. During the peer review, which forms the main part of this seminar, you will discuss your choices
of measurement instruments. Working in small groups, present your précis and discuss all items
you have entered in Table 1 on the measurement of the main variables. Consider, choose and
decide on the instruments which you think fit your RQ best.
Note: Your decisions have bearing on the cost, time, effort and organisation of your further
research preparations. In addition, your data analysis is at stake!
Then, having decided on the most important determinant and outcome, take the plunge and sketch
the association (recall Module 4 assignment 3 of the study model)! What numbers do you expect in
your research, based on previous Modules?
Following this, design an n x m table, representing the association between outcome and
determinant. Sketch a Figure that illustrates your (expected) outcome. Don’t forget to give your
Table and Figure a title and the appropriate legend.
3. During the last part of Seminar 2, we will ask you to think about where you are heading in this
course. This is more than – ‘I’m just here to follow the course’: we would like you to consider
aspects like:
- points you like
- points that you might do differently
- the way we cooperate with peers
- the learning and teaching materials.
It is essential for all of you to look for ways that you can change / improve / vary your own
learning and benefit from experiences.
4. After this moment of reflection on what you have done so far in this pre-internship preparation
course, we will look forward to what will happen in the second week. Read the textbox below and your CKO-9 course book.
Lecture 2: Philosophy of science; it’s influence on your research
Module 7: Societal and ethical justification Module 8: Designing the data analysis Module 9: Organisation and management summary
Seminar 3: Practical organisation and numerical thoughts
Computer lab: SPSS practicalities
Module 10: Reporting your Plan of Work and management summary
Seminar 4: Sales and communication
34
Lecture 2: PHILOSOPHY OF SCIENCE: INFLUENCING YOUR
RESEARCH
Thinking about things
In this second lecture we will attempt to place your research project in the perspective of epistemology
[Epistemol, what? Read the following textbox]. This is the field of philosophy that deals with aspects of
how to learn about the object of research, thus the nature, scope and the limitations of scientific knowledge.
Do you think that your research will result in the right answer (and... what is right?) and, is the right answer
that what we want to know? And what powers influence and manipulate the choices to be made in your
research?
Learning objective
Students can position their study in the field of epistemology and can evaluate the various influences on the
dynamics of science on the outcomes of their own research.
This interactive lecture on the theme of acquiring medical knowledge consists of three parts. Firstly,
students (co-assistants) refresh their knowledge and understanding of philosophical perspectives on
scientific medical knowledge.
Secondly, basic understanding is acquired of the actual influences on the dynamics of science (conceptual,
academic, economic, political)
Thirdly, students will reflect on the role of (the latest) medical knowledge for future medical practice as a
medical professional.
Epistemology
Epistemology originates from the Greek words epistèmè and logos, meaning knowledge,
science, and study of, respectively. It is the branch of philosophy concerned with the nature and
scope (limitations) of knowledge. It addresses the questions:
What is knowledge?
How is knowledge acquired?
How do we know what we know?
And is it all true?
Much of the debate in this field has focused on analyzing the nature of knowledge and how it
relates to connected notions such as truth, belief, and justification. It also deals with the means
of production of knowledge, as well as skepticism about different knowledge claims.
Many dictionary definitions may give the impression that epistemology is closely related to
critical thinking or, why not, critical appraisal: "the study or a theory of the nature and grounds
of knowledge especially with reference to its limits and validity" (Merriam-Webster's Online
Dictionary, 11th Edition).
35
To refresh your memory, review the textbox on schools of philosophy relevant to medical sciences:
Logical positivism, 1910 - 1950 Logical positivism (also known as logical empiricism and
neo-positivism) is a stream of philosophy that combines
empiricism—the idea that observational evidence is
indispensable for knowledge—with a version of rationalism
incorporating mathematical and logico-linguistic constructs
and deductions of epistemology.
Critical rationalism, 1950 - 1975 Critical rationalists hold that scientific theories, and any
other claims to knowledge, can and should be rationally
criticized, and (if they have empirical content) can and
should be subjected to tests that may falsify them. Thus
claims to knowledge may be contrastively, normatively
evaluated. They are either falsifiable and thus empirical (in a
very broad sense), or not falsifiable and thus non-empirical.
Paradigm theory, 1975 - 1990 A paradigm shift (or revolutionary science) is, according to
Thomas Kuhn in his influential book The Structure of
Scientific Revolutions (1962), a change in the basic
assumptions, or paradigms, within the ruling theory of
science. It is in contrast to his idea of normal science.
Social constructivism, 1990 - 2005 Social constructionism and social constructivism are
sociological theories of knowledge that consider how social
phenomena or objects of consciousness develop in social
contexts. A social construction (social construct) is a
concept or practice that is the construct (or artefact) of a
particular group.
New pragmatism, 2005 – 2015 The central claim here is that organization of scientific
studies needs to be fundamentally reshaped. This change is
needed to provide room for ethics and to increase the
relevance of research in society and clinical practice. It is
argued that the new pragmatism provides critical resources
for this change. Pragmatism is a particularly helpful
approach to define the use of scientific knowledge in the
social reality while it avoids entrenched epistemological
distinctions that marginalize ethics and make research less
useful.
To give your own study more flavour:
- Are there issues of integrity at stake in your project or the way in which it will be carried out?
- Are the rivalling groups that deny the relevance of your research goals?
- Does your research aim at the discovery of truth or knowledge, or merely the likelihood that things
are true?
- Does your approach endeavour to be objective?
- Nota bene: Scientific knowledge is often deducted from subjects, which means that it is inherently
subjective.
- How is your project organized? Who is ultimately responsible?
- Which interests from economics, society and politics are involved?
- What is your ambition with your project?
- Should society invest in your project?
- Are the possible clinical results for helping patient theoretical, practical or ideological?
36
Module 7: SOCIETAL AND ETHICAL JUSTIFICATION Am I doing the right thing?
The choice whether to finance a specific research project depends on the preferences of the funding agency.
Media and interest groups can exert considerable social pressure to prioritize particular topics. Once the
topic and the Research Question of a study have been decided on, interested parties have to refrain from
interfering in the following steps of the research process until the study has been completed (no conflict of
interests).
Furthermore, conducting research involving humans is subject to increasingly complex legal requirements.
This means that you, as a clinical researcher, are not only expected to be an expert in the field of research
but are also expected to be fully informed about legislation and arrangements with supporting departments
in order to carry out the research. All research has to be conducted in accordance with the Law for the
Protection of Personal Data (WBP) and the Law for Agreement on Medical Treatment (WGBO). The
manner in which these regulations are incorporated can be found in the Code of Conduct. In case of clinical
research with medicinal products, the Good Clinical Practice guideline was developed to protect the
(human) rights of a subject in a clinical trial. It also provides assurance regarding the safety and efficacy of
the newly developed drugs. Similar requirements have been made for non-human research.
Learning objective
Students can argue to what extent their research proposal requires societal and moral justification (in view
of legislation, Good Clinical Practice, and accreditation by the Ethics Committee). Furthermore, they can
demonstrate that they have accounted for privacy matters applicable to the study subjects.
Assignment
1. Whether or not you are dealing with humans, animals or biomaterials, it’s always good to have a
clear idea about the human rights regarding the data and information you need for your research.
Start reading Chapter 10 Bescherming van Onderzoeksdeelnemers in Zielhuis’ research manual.
Don’t forget to visit the websites on Codes Goed Gedrag, Goed Gebruik (see www.federa.org or
Zielhuis pp 210-211for some other sites).
2. Quick Test 7 introduces the changes in science, society, philosophy and ethics over the last
decades. It looks at the concepts of:
- conflict of interest
- informed consent
- proportionality
- subsidiarity
- and abbreviations like CMO, METC, and WMO.
3. Please, address the following issues a and b.
a. We would like you to reflect on your own or to talk to your partner about the email message
in the textbox. It’s about efficient information exchange. Discuss whether it is a correct way of
working and why. Otherwise, what might be an alternative solution?
37
b. Do research proposals in general, but also research internships, need CMO accordance or
approval by relevant authorities? Read the next internship proposal (for a ‘me-too’ study) and
decide whether it requires approval from a relevant review committee.
Proposal: Development of an innovative WalkingTest
Gordon Cooper developed his well-known fitness test for military personnel half a century ago: the
Cooper-test, running for 12 minutes. In older adults a walking test is more appropriate. The elderly,
especially, vary in life expectancy and physical fitness. Both aspects should be considered when
setting goals for healthy ageing. In the last few decades, a number of papers were published about
walking or gait speed tests, although most of the time, gold-standard tests were lacking.
The purpose of this research internship is to develop a new test covering a 400m long-distance walk,
to be conducted outdoors. The 6-minute Åstrand-rhyming sub maximal cycle exercise test provides the
gold-standard physical condition.
The setting, environmental conditions, number of subjects (age, gender, and multi-morbidity),
recruitment, physical exams and lab are all up to you, as you are the clinimetric data-analyst.
So, will your innovative walking test prove to be a useful predictor of physical fitness for all of our
55+ adults?
3. During each module, we will be asking you to think about what the theory means for your research
project:
- So, is your research ‘CMO-compulsory’?
- Does it concern biomaterials and/or humans; and, in what way?
- Will it use medical records, questionnaires, Internet information?
Email: subject information
Forward: list of records
Hello Hans,
I’m sorry, I have to cancel tomorrow’s delivery. No – I haven’t run off on holiday – but today our
management team announced unexpectedly that we will be starting tomorrow instead of next week
with the replacement of our lab department. That means I have scanned and digitized all test outcomes
and linked these with information from the medical records. I’ve filed them electronically for you and
ordered the records by patient name – so you will be able to screen for the right patient for your
research.
My apologies for canceling at such short notice!
Kind regards, Julia
PS: Our ICT department can generate the most up-to-date copy of the list, if convenient.
Based on “Gait speed and survival in older adults” by Stephanie Studenski, MD, MPH, et al. JAMA 2011; 305: 50-58.
RESEARCH INTERNSHIP
PROPOSAL
38
- Who’s the owner of the collected data? How are these data protected?
- Do you anticipate any conflict of interest? Why? And why not?
Make notes on these issues. These notes are not only for you, the trainee, but also for your peers, team
members, and supervisor. You will have to compile them in your Plan of Work (Module 10).
39
Module 8: DESIGNING YOUR DATA ANALYSIS
Getting the right result!
Every scientific question can be cast in a general research model. These models consist of many kinds of
variables. The determinant and outcome variables are the most important when answering the research
question. Many research questions in medical science are about an association between these variables. All
variables must be (made) measurable, before appropriate measuring instruments are selected to carry out the
measurements (Module 5 on measurements of key variables). In order to collect the data, you first need to
design the data file with an accompanying codebook. Designing and performing data analysis are tasks
preferably done together with your supervisor (and sometimes with experts in statistics) to preclude the
possibility of you getting completely lost in your data file. Wrong interpretation of the output, might lead to
contradictory interpretation of the results of the analyses. If you have designed your research well, the data
analysis will provide the answer to your research question. But please note: it is a great shame whenever the
following quote by PWA Willems (Utrecht University) becomes applicable to your study: “Unfortunately,
only after having analysed the data, has it become clear how the research should have been designed.”
Learning objective
In their Research Question, students can distinguish the key variables of the association to be studied and
convert the variables to a codebook. Based on this, they can design the crude data analysis aiming to answer
the Research Question. Students can indicate how to use a statistical package like SPSS for data entry, data
management, data analysis, and result presentation and interpretation.
Important: This module is a large one, comprising six parts!
Instruction
1. Read Chapter 8 in Zielhuis’ textbook and choose the topics which best fit your Research Question:
- measurement scale of your outcome and determinant variable
- codebook terminology
- statistical analysis, especially estimating parameters and their precision
and,
- is hypothesis testing part of your research project?
For further consideration: have a look at the book by Petrie & Sabin “Medical Statistics at a
Glance”, 3rd
edition (Oxford: Blackwell Publishing Ltd, 2009). Are Chapters 20, 24, 26 and 44
relevant to your study?
2. Quick Test 8 looks at the basics of statistical analysis describing the data: the ‘average’ and the
‘spread’, displaying the data graphically, but most and foremost, the interpretation of the
population on parameters and their standard errors
3. Collocation of medical research domains, methodology and data-analysis:
In extension to a former task (Lecture 1), please match a domain of research (A), and a study
method (B) to a type of data analysis (C). There may be more than one possibility. For instance 1
with c, d etc.
If you don’t recognize some of the items, please use Wikipedia as a quick and dirty method to gain
an understanding of the contents.
40
Return to Module 2 and the review of the three papers on your subject: what kind of data analysis
did the authors use?
4. To give you an even more “A-ha” feeling on data analysis, you are going to read about the topic of
graft loss following kidney transplantation and the influence of the ‘heart-beating donor’.
5.
Study the codebook on the dataset NIERTRANSPLANT.SAV concerning 1228 patients.
Using SPSS as statistical software, we analysed the data that have been made anonymous
and which were made available to us by the courtesy of Professor Andries Hoitsma,
transplantation internist at the University Medical Centre in Nijmegen. Look at the outcomes
carefully and answer the questions.
Précis - Kidney transplantation is the current treatment of choice in patients with end-stage renal
disease. Substantial changes have occurred (new immunosuppression agents, new selection criteria
for donors and recipients in the intervention procedures).
We want to study to what extent outcomes like graft loss and patient survival still are dependent on
donor source, i.e. living-donor grafts or cadaveric grafts / (non-) heart beating donor organ? This
research question is relevant for the current and future availability of donors.
A 1. diagnosis
2. medical intervention:
3. treatment, prevention
4. etiology, pathogenesis, polymorphisms
5. prognosis
C
a. observation series
b. survival analysis
c. t-test
d. ROC-analysis
e. proportional hazard modelling
f. logistic regression analysis
g. chi-square test
h. patient interview
i. ANCOVA
j. focus group interview
k. longitudinal data-analysis
B
i. qualitative study
ii. evidence-based medicine
iii. clinimetrics
iv. biochemistry
v. clinical psychology
vi. pilot study.
vii. polygenic approach
viii. survey
ix. medical ethics
x. social medicine
Medical research domains (A), study methods (B) and data-analyses (C) the last 500 CKO-9
students put forward in their Plan of Work in the period 2009-2011
41
Codebook on kidney transplantation and graft loss
VARIABLE
NAME
VARIABLE
LABEL
VALUE LABELS
ID identification number
unique number ranging
from 1863 to 110465
GEND gender
1 = male
2 = female
AGE age at transplantation 18 – 76 yr
S_BD systolic blood pressure before
transplantation
80 – 240 mmHg
“ . “ is missing value
DONOR heart beating donor organ
(HBD)
1 = heart beating donor
2 = non-heart beating
GL graft loss 1 = yes
0 = no
TIME_TO_GL time to graft loss or censored 1 – 21 yr
Applying SPSS statements such as: Analyze – Descriptive statistics – Frequencies / Descriptives /
Explore / Crosstabs, resulted in several outcome tables and graphs (below) on domain variables. Study
these descriptives and then answer questions a – c.
a. What is the prevalence rate of hypertension, defined as S_BD > 180 mmHg?
b. What are the mean age and systolic blood pressure of the recipients? How many digits (and what
unit) are appropriate?
c. What kind of information on the correlation between age and systolic blood pressure can you defer
from the output? (Table II, Figure I)
Table 1. Systolic blood pressure before transplantation of 1228 recipients
Systolic blood pressure before transplantation (mmHg) Frequency Percent Valid Percent
Cumulative Percent
Valid 80-110 115 120 125 130 135 140 145 150 154 155 156 158 160 165 170 173 175 180 185
39 5
40 11 60 11 79 13
100 1
13 1 1
85 5
73 1 4
40 1
3.2 .4
3.3 .9
4.9 .9
6.4 1.1 8.1 .1
1.1 .1 .1
6.9 .4
5.9 .1 .3
3.3 .1
6.0 .8
6.2 1.7 9.3 1.7
12.2 2.0
15.5 .2
2.0 .2 .2
13.2 .8
11.3 .2 .6
6.2 .2
6.0 6.8
13.0 14.7 24.0 25.7 37.9 39.9 55.4 55.6 57.6 57.7 57.9 71.1 71.8 83.1 83.3 83.9 90.1 90.2
42
Table II. Correlation and regression between systolic blood pressure and age among 1228 recipients.
** Correlation is significant at the 0.01 level (2-tailed)
a
Dependent Variable: Systolic blood pressure before transplantation
In Figure I the results are shown of linear regression of S_BP as a function of AGE.
Answer question d.
d. Can you set up the regression line: S_BP = B0 + B1 x AGE? What increase in systolic blood
(mmHg) pressure do you expect in case a person becomes 1 year older?
Figure I. Correlation and regression between systolic blood pressure and age
190 192-240
Total
Missing System Total
29 34
646
582 1228
2.4 5.3
52.6
47.4 100.0
4.5 5.4
100.0
94.7 100.0
Mean Std, Deviation
N Correlations
Age at
transplantation
Systolic blood pressure before transplantation
Age at transplantation
45.99
12.993
1228
Pearson Correlation Sig. (2-tailed) N
1
1228
.160** .000 646
Systolic blood pressure before transplantation
151.96 25.016 646 Sig. (2-tailed) N
.160** .000 646
1
646
Regression Coefficientsa
(linear regression) Model
Unstandardized Coefficients
B Std. Error t Sig.
1 (Constant) Age at transplantation
137.476 .313
3.656 .076
37.602 4.111
.000
.000
43
6. Now, regarding our key Research Question: What is your prior belief of graft loss rate in type of
transplantation, which is operationalised in HBD (heart beating donor transplantation) vs. non-
HBD patients.
How many recipients sustain graft loss, and what %, of all the recipients is this? Is this different
for type of donor (HBD vs. non-HBD patients)?
Has your hypothesis about the effect of HBD-donor been confirmed or refuted by Table III and
Figure II, respectively?
e. What confounding factors would you like to address? Would you consider systolic blood
pressure as a confounder in the relationship of HBD-transplantation and graft loss? What does
Table IV tell you in this respect?
For the statistical analysis, we quoted SPSS statements like:
Analyze – Descriptive statistics – Crosstabs, mark in ‘Cells’ option ‘Percentages’:
Row, Column and Total and in ‘Statistics’: Chi-square.
Analyze – Compare means, followed by Independent - Samples – T Test
Analyze – Survival – Kaplan-Meier, with Options ... , as Plots, Survival and thick
under Compare factor for log-rank.)
Table III. Heart beating donor intervention and the effect on graft loss.
Heart beating donor organ (HBD) Graft loss (GL)
No Yes Total
No Count Expected count % within HBD % within GL % of Total
403 439.4
39.7% 76.6% 33.2%
612 575.6
60.3% 88.8% 50.4%
1015 1015.0
100.0% 83.5% 83.5%
Yes Count Expected count % within HBD % within GL % of Total
123 86.6
61.5% 23.4% 10.1%
77 113.4
38.5% 11.2%
6.3%
200 200.0
100.0% 16.5% 16.5%
Total Count Expected count % within HBD % within GL % of Total
526 526.0
43.3% 100.0% 43.3%
689 689.0
56.7% 100.0% 56.7%
1215 1215.0
100.0% 100.0% 100.0%
44
Figure II. Graft loss according to kind of kidney transplantation from (non-)heart beating
donor.
Table IV. T-test statistics on the association of systolic blood pressure before transplantation and type of kidney donor.
HBD N Mean Std. Deviation Std. Error Mean
Systolic Blood Pressure non_HBDonor 537 153.08 25.417 1.097
HBDonor 108 146.53 22.334 2.149
t-test for Equality of Means
95% Confidence
Interval of the Difference
F
Sig. t df
Sig. (2-
tailed)
Mean
Difference
Std. Error
Difference Lower Upper
Systolic
Blood
Pressure
Equal variances
assumed
2.591 .108 2.491 643 .013 6.550 2.629 1.388 11.713
Equal variances
not assumed
2.715 167.729 .007 6.550 2.413 1.787 11.314
HBD-donor n= 200
non-HBD n= 1015
45
7. Back to your own research proposal: What is your basic analysis of the Research Question?
Sketch the key Table and the key Figure. Concerning the Table, is the outcome presented in rows
or columns? What about the determinant?
Do the same for the Figure: which variable is on the X-axis and which on the Y-axis?
Complete your draft Table and Figure with appropriate title and legends and 95% CIs or statistical
test outcomes, whatever is most convenient.
NB: you will need this work on draft tables and figures for peer review in Seminar 3 and the SPSS
computer lab afterwards.
Tip 1!
For statistical problems during your internship, first of all mobilize co-workers in your host
department. In case of major problems about your data analysis, you may use the email address for
statistical consultation by RUNMC experts; this is posted on BlackBoard.
Tip 2!
To display tables and figures, see section Resultaten (pp167-9) in Chapter 9 Publiceren of
Zielhuis’ textbook and Chapter 4 in Petrie & Sabin’s Medical statistics at a glance.
Tip 3!
Which language is used in your text, tables and figures? Use a dot as delimiter if you use English
(2½ -> 2.5) and a comma in case of Dutch (2½ -> 2,5).
46
Module 9: ORGANISATION AND MANAGEMENT SUMMARY
Justifying your case!
We can imagine that you would like to perform your study alone. Usually, however, you will be working in
a team. In a team, all members have their own tasks and roles and their own planning, timetables, and
priorities. In addition, other important facilities like the availability of a workplace, a personal computer,
and in case of research abroad, accommodation can be very important.
More directly connected to your research is the question, ‘is your study feasible?’ This means, is your
research affordable in time and money, is there enough technical expertise, and is it possible to include the
estimated number of subjects. Is it possible to use measuring instruments and are there others users?
All these and more are part of planning your research organization.
Learning objective
Students can discern the main players related to their research project: scientific support, information
collection, collaboration and use of measurement instruments. They can also demonstrate the feasibility of
achieving their proposed deadlines in the given time span of three months.
Assignment
N.B.: today’s assignment consists of six parts (Quick Test excluded).
1. Nothing to read this time, but Quick Test 9 requires you to take a deeper look at organising your
progress. You will think about communication – logistics – piloting – time-planning.
2. You now have less than a few days before you start your internship. Are you prepared? Is your
supervisor prepared?
Search the department you are going to on the Internet. Who is who? And what have they published
recently? What about your supervisor, his/her (major) articles (impact factors) and H-index / citation
report? For the latter, go to the “ISI Web of Knowledge” (via University Library), fill out the name of
your supervisor, select the right articles and choose “Analyze Results” or “Create Citation Report”.
Plan an email to send to your supervisor saying (once again) who you are, anticipating potential
starting points and that you are looking forward to the internship. Don’t forget to mention the CKO-9
pre-training track: that you will forward your Plan of Work to them within one week.
3. So far, you have worked on setting goals and objectives. Now it is time to organise your actions into a
plan. Actions set into a time-frame make up your plan. Usually, we see plans like the one in the table
below:
47
Time schedule research internship
Week 1 Complete preparations. Contact department MDs
Weeks 2-6 Enrolment. Input case record forms. Follow-up
Weeks 7-9 Measurements. Input findings
Week 10 Data analysis
Weeks 11-12 Writing paper. Presentation. Completion Internship
Can you be a bit more specific for your own plan? Put also the writing of some of the parts of your
report / draft article on the time table. Make sure for yourself and for us, CKO-9 teachers that your
plan is workable. Who are your team members? Make sure you have organised your actions in a
logical and practical order. Can you accomplish the actions you have set for yourself in the time
frame allotted?
4. Time-management is one thing. Self-management is another thing. How will you approach your
research internship further?
In textbooks on self-management, so-called SWOT analyses are advised. In SWOT you have to
pinpoint your own Strengths (S) and Weaknesses (W) and confront these with the Opportunities
(O) and Threats (T) emerging from your environment (trainee post). Read the Table and fill out
your SWOT items. What issues of your approach deserve your special attention during your
research internship?
No. MAIN POINTS
MY SWOTs
1 Manage your time
S:
W:
O:
T:
...
...
...
...
2 Team work, cooperation with
subjects
S:
W:
O:
T:
...
...
...
...
3 Approach to end result and
products of your internship:
- What do you have to do?
- What standard of work is
expected?
S:
W:
O:
T:
...
...
...
...
4 Presentations / written texts S:
W:
O:
T:
...
...
...
...
5 Fulfilling your own learning
objectives (see below, task 5)
S:
W:
O:
T:
...
...
...
...
5. The main objective of your research internship is ‘learning by doing’ - how to conduct research.
Nine general objectives have been formulated in that regard (see this course book Episode
Research Training, Chapter 2, Learning Objectives).
48
But you also have to master three specific (personal) research objectives (See p. 10 “Personal
learning objectives of the research training internship”). But first in the textbox below, you are
going to read about four different medical students and their personal learning objectives.
6. Which students erroneously mention general learning objectives, instead of personal objectives?
Master Student in 2011 Personal Learning Objectives
A. H. Critically read literature
Analyse the collected data
Produce an article
B. I. i. Speaking Spanish
ii. Improving interview techniques
iii. Team building
iv. Back-packing through the country-side
C. J. - Gain experience in setting up and conducting
qualitative research
- Develop interview techniques further
- Practice writing a good report
D. K. 1. Competences in Pubmed
2. Skills in statistical testing and SPSS
3. Developmental experiences in writing scientific
papers
4. Visiting my first scientific congress
Write your three personal learning objectives. These personal learning objectives are more specific
goals for your internship. You have to put time and effort into this - beyond your plan scheduled
above (assignment 3). Make clear how, which activities and courses you will take, to achieve these
goals. So make sure your host supervisor agrees with you and supports you.
7. Finally, have a quick look at Seminar 3, tomorrow. It’s all about Practical organization, and …
some numerical thoughts. You will work on the latter in a computer lab on SPSS, which is
scheduled following Seminar 3. But first, review Modules 7 – 9.
49
Seminar 3: PRACTICAL ORGANISATION AND NUMERICAL
THOUGHTS
Get organized!
In this seminar, you will reflect on the approach, planning, and organisation of your own internship.
Independent of your internship supervisor or the actual research supervisor, you have to, in advance,
formulate all the points that deserve special attention regarding the management of your internship.
Management largely comes down to self-management. What are your well- and less-developed self-
management competences? The other aspect for reflection is the design of your data analysis. How will you
conduct your basic statistical analysis and what is the expected outcome, and how will it be visualized into
a figure / graph?
Learning objective
Students can demonstrate their firm grasp of the anticipated organisational aspects of their study and can
specify time constraints and the relevant team players. Additionally, they can make a blueprint of the
descriptive and inferential data analyses as presented in the result section of a scientific article.
Method of working (instructions will follow in the workgroup seminar)
In the peer review part of the seminar, you will review the design of your data analysis. In order to do so,
please bring along the final productions of the Module 8 home assignment; these include drafts and
printouts dealing with practical issues involved in data analysis, and sketches of the final Table and Figure.
This seminar concludes with the important subject of your own organizational skills for managing your
internship. You will focus on the products of Module 9 and the tables completed.
50
Computer lab: SPSS PRACTICALITIES (translation under construction)
Are my results significant?
Now things really get going! You enter your data into the computer, check the quality of the data files for
all separate variables, make descriptive statistics, and, finally, develop statistical data analyses of the
association that is to be studied. Until quite recently, this was done by noting down measurements followed
by some simple arithmetic, but now statistical software packages are commonly used everywhere. In the
CKO-9 course, we use SPSS.
Do you remember your first SPSS data analysis in the Bachelor phase? Before using the package, you may
benefit from a (re-)introduction to this software at the level of codebook, data entry, elementary procedures,
recall for the help function, and simple output.
Learning objective
Students are able to process and analyse their own virtual data file or make use of the presented examples,
resulting in (simple) tables and graphs.
Interactive statistical lab
In the1st hour, a demonstration is given of how to build a data base with records and different kind of
variables, how to do simple descriptive analyses, and how to compare means and proportions. We will then
briefly touch on how to transform variables into other variables and to use different windows in SPSS.
During the 2nd
hour you will complete the ‘muesli’ exercise: how does muesli intake affect mental
concentration tests in some volunteers? (Creating a data base, data entry, descriptive analysis of all
variables, comparing pre- and post- measurements, confounding and or subgroup analyse).
CKO-9 COO-SPSS:
Data entry and simple statistical analyses “Muesli study”
Basic skills in SPSS
Précis
Some resident physician underestimates the importance of well-balanced
diets. The internships are long and aggravating, and there is simply not
enough time to eat. Even though they are aware that diet has an important
effect on their ability to concentrate and capacity to handle swiftly, their diet
suffers from both ‘too much’ and ‘too little’ – resulting in a negative
influence on physical and mental strain. We hypothesize that breakfast with
muesli will facilitate slower carbohydrate uptake and have a positive
influence on the physician’s reaction time.
We will study this by giving a muesli diet to 20 resident physicians for a
period of 4 weeks. Their reaction time will be assessed before and after this
period.
The next table shows the data of 20 participants.
51
Table 1: Data from the muesli study
ID Gender Total reaction time Total reaction time Birth date
(before) (after)
1 male 78 75 12.01.85
2 female 90 81 06.05.88
3 female 68 65 05.11.83
4 female 77 70 07.11.88
5 female 78 77 21.12.83
6 male 81 76 25.05.89
7 female 73 72 01.03.85
8 female 57 54 04.08.89
9 male 55 52 19.03.83
10 female 74 69 22.05.84
11 male 63 58 10.04.85
12 male 61 58 23.04.81
13 male 80 75 31.01.89
14 female 82 77 13.09.83
15 male 72 65 17.07.86
16 male 65 55 15.06.88
17 male 90 79 16.12.83
18 female 68 65 05.11.87
19 female 77 68 07.11.88
20 male 81 70 25.02.89
Code book
Variable label Variable name Value range
Identification number ID 1-20
Gender Gender 1=male, 2=female
Reaction time (RT) before muesli diet (sec). RTbefore 55-90
Reaction time after 4 weeks muesli (sec.) RTafter 52-81
Birth date Birth 23.4.1981 – 25.5.1989
Variables, yet to make (see under 5):
Difference in reaction time (RTbefore – RTafter)
in seconds
Diff
Age at 1.1.2010 (years) Age
Data entry:
1. Enter the Table 1 data directly in the data editor (Data view-window).
Tip: Start to define the variables in the Variable view-window. For help about the different types of
variables see “help>>topics”.
2. Save the file via <file> <save as> with file name “Muesli_ZSO9.SAV”.
It is now possible to conduct some simple analyses.
Understanding the data:
3. Briefly describe the variables in the data file.
Step 1: Click on <analyze>.Search for the analysis you want to perform (to analyze you can use
<help> <statistics coach>): Use <Descriptive Statistics> and next <Frequencies> and
<Descriptives>.
52
Step 2: Click on <paste> to save the syntax. The commands for the analysis will automatically be
pasted into the syntax window. (Clicking on <OK> will run the analysis immediately without
saving the commands in the Syntax window. Saving your analyses in the Syntax window with
<paste> provides a journal of all the steps you have taken.
Step 3: Next, go to the Syntax window and select the command just pasted. Click on <run> or on
►Now the data will be analyzed and the results will be written to the Output window.
Step 4: Click on the Output window and study the results.
Step 5: Start a new analysis: Go to step 1 and choose another analysis from the drop-down menu.
- How many male and female participants are available from the file?
- What is the mean reaction time before the start of the muesli? And after 4 weeks eating muesli for
breakfast? Are these results different?
4. Try also <Analyze>, <Descriptive Statistics>, <Explore>. This procedure gives you different
outcome measurements and produces graphs simultaneously.
Making a new variable:
5. Make a new variable in which the difference between reaction time before and after the muesli diet
is calculated (variable name ‘Diff’) and the age at the start of the study (variable name ‘Age’), that
is the 1st of January 2010 (see box)
Step 1: Click on <Transform>. Use <Compute variable>. Add the new name of the variable in
‘Target variable’. Choose the desired calculation by entering the numeric expression in the box
(see below).
Step 2: Save the calculation in the Syntax (<Paste>, see assignment 3) or use OK directly.
Step 3: Behold the results in the Data view-window.
Variable Numeric expression
Difference in reaction time:
RTbefore – RTafter
Age: DATEDIFF(DATE.DMY(01,01,2010),birth,"years")
(see function group-> date arithmetic, datediff and date creation)
Consider the age. What is the mean age? Is this okay?
What is the mean difference in reaction time before and after the 4 weeks muesli diet?
Data-analysis:
6. In assessment 3 step 5 you compared the mean reaction times at the start of the study and after the
diet. Is this a significant difference? The null hypothesis is no difference (i.e. mean Diff = 0).
Use <Analyze>, <Compare Means>. Which procedure do you prefer?
The unpaired T-test ( <Independent-Samples T Test> ) or the paired T-test ( < Paired-Samples T
Test> )?
What is the mean change in reaction time? What is the 95% Confidence interval? Is null (‘0’)
included? What does the outcome of the P-value suggest?
53
7. Now we want to know whether the reaction time will be different between male and female
participants (effect modification).
Step 1: Click on <Analyze>, <Compare Means>, <Independent-Samples T Test>.
Step 2: Use ‘Diff’ for Test variable and ‘Gender’ for Grouping variable. Define Groups and click
on OK.
What is the difference in reaction time between males and females? Is the difference null? What is
the 95% Confidence interval? Is ‘0’ included? What can you conclude from the P-value?
8. Visualise the reaction time at the start of the study for males and females in one graph. Use the
<Graphs> menu. Which type of graphs are useful (try BAR-chart, BOX-plot, ERRORBAR and
HISTOGRAM)? Which is the most suitable?
Visualise the coherence between the reaction time before and after the muesli diet. Use a
SCATTER-plot. Can you draw a regression line through the scatter plot (click twice on the plot
and use options)?
Here ends the computer lab SPSS practicalities. If you have any questions about the analysis of your own
study, don’t hesitate and ask the SPSS lecturer
54
Module 10: REPORTING YOUR PLAN OF WORK AND
MANAGEMENT SUMMARY
Convincing your readers!
After a fortnight of preparation and anticipation, you now have reached the final and most comprehensive
assignment. Why are you actually doing this internship and why have you chosen to work on this specific
research topic? How will you organise the whole project? You will complete a Plan of Work, parts of which
you have already worked on during your previous assignments. With this final Plan of Work, you will not
just do yourself and your supervisor a favour, but you have also taken the first steps towards producing your
final internship report and a possible scientific publication. The CKO-9 lecturing team really hope that you
achieve this last target: publication!
Learning objective
Students can write the final Plan of Work for their internship. Furthermore, they can summarize the
relevance of the anticipated results, and can deliberate on the strengths and anticipated weaknesses of their
proposed study at a scientific and societal level.
Assignment
1. Make the Quick Test. In addition, to warm you up on the art of good communication resulting in
your Plan of Work, here are some additional questions:
- What is the maximum acceptable number of slides for your 10-minutes PowerPoint
presentation?
- What is, in general, the number of words allotted for an abstract (e.g. for NTvG) and how
many for a scientific article ( e.g. the New England Journal of Medicine). How many
words are you allowed for your internship report?
- What are “Authorship-responsibility / Copyright transfer forms”?
- Does your type of research require a special form of reporting (CONSORT, MOOSE
STARD, STROBE etc)? (See Zielhuis, Chapter 9)
2. In this Module you complete your Plan of Work. As in any other piece of written work, it should
be preceded by a summary. This summary, however, will differ from the summary of a scientific
paper. Firstly, think about whom are you writing the Plan of Work for? Secondly, start with
background information. But thirdly, and most importantly, address logistics and practicalities in
the plan of investigation.
Write a management summary in 500 words:
- title and author … your name
- background …. rationale, what’s already known
- purpose … what will this study add, and research question
- setting
- most important part of the management summary: the plan of investigation (for
example enrolment of subjects or materials, numbers, measurement tools, data collection,
human subjects protection, data reporting, data analysis)
- expected results and relevance, impact.
55
3. Completing your Plan of Work
What have you learned from all the modules? Have you made informative and additional notes in
the seminars? The aim of the final module is to complete your Plan of Work. You can use the
template presented in the box as guide for writing your Plan of Work.
But, please note, this is not virtual ‘run through’ of all the previous modules, it’s about getting a
clear view of the end result of CKO-9: a report of your Plan of Work. Your report will let your
research internship supervisor see how you are trying to interest fellow researchers and ‘sell’ the
project. Make sure you cover all the points required, and write enough detail to fulfil the task. Take
care of an appropriate style and layout. But, most importantly, show a well-ordered structure in
what you present the logical development of ideas, facts and expectations.
For example, the chapter on ‘research question’ should not be found ‘just anywhere’ in your plan,
nor should it contain just one statement like “Does chocolate lower blood pressure in hypertensive
patients?” Even a research question on medical chocolate intervention should be introduced by a
strong opening, motivating the decisions regarding the domain, determinant and outcome.
For this task, you have to produce a single piece of writing, of 10 – 15 (maximum) sheets of A4 (in
professional English or Dutch).
4. This part is not an assignment, but describes how we will assess your Plan of Work. It may,
therefore, facilitate completion of your final report.
Assessment - Your performance at CKO-9 is assessed at two levels which score your ability to
write a Plan of Work: A = sufficient / pass and B = insufficient / fail.
The score is based on the criteria for style (see left hand box below) and content (see right hand
box) and the requirements presented in Chapter 2 of this course book.
For the assessment of the content every item of your Plan of Work is scored on a scale ranging
from -, ±, +, which is short for absent or weak, borderline, and good. The scores are assigned to
your text, graphs, formulas and, more importantly, the structure within and between the chapters.
Template for writing your Plan of work
Front page
Management Summary
Introduction with rationale, literature (search, overview (table), critical appraisal
and reference list) and a paragraph specifying the Research question
Proposed plan of investigation with study design, study model including key
variables, study population and number of subjects and measurement instruments
Societal and ethical justification
Proposed data analysis with main table and figure
Organization with time table and SWOT analysis
Personal learning objectives
For some examples of well-done Plans of Work see BlackBoard
56
5. Prepare Seminar 4 well. Skim its text for a general understanding of the peer review process and
the task. NB: The 500-word management summary of your Plan of Work is pivotal for Seminar 4.
Print three hard copies and bring them to Seminar 4 for your peers.
Plan of Work:
Your Content Checklist
Title and front page information
Management summary
Rationale
Literature
Research Question
Study design
Study population
Measurements
Data analysis
Organization
Personal objectives
Plan of Work:
Your Writing Checklist
Have you included information about all
the previous module assignments?
Have you used your own words and
avoided copying from the specific
instruction text?
Is your chapter structure sufficiently clear
to show a good control of content and form
(opening , middle and closing section)
Are the front page requirements met? (title,
extension, ‘Plan of Work’, author, student
nr., creation date, emails
student/supervisor, department, calendar
period)
Have you included the management
summary following the front page?
Did you avoid redundant information,
grammar, spelling and other errors that
might cause confusion?
57
Seminar 4: SALES AND COMMUNICATION
Selling your story
Using the management summary of their Plan of Work, students can reflect on the strong and weak points
of their own study and communicate these to fellow students. They should ask critical questions about the
relevance of the research and the quality of the proposed work.
Learning objective
Students can summarize their study and present the aspects that make their research project worthwhile.
Method of working
In this seminar, we start by handing out the management summaries (Module 10) for peer review in small
groups. Then, the seminar group are given an imaginary budget of € 1,000,000 to be spent on a maximum
of 2 research projects that appear to its members to be most worth funding.
In the final part of the seminar, students reflect on the research process in its entirety. What makes a
research project:
- Useful? (for scientific knowledge - for clinical and health policy - for future research directions)
- Good, according to what you have learned about research questions
- Good, according to what you have learned about study design / data acquisition
- Feasible
- Clear and unambiguous.
Evaluation
This course has been developed especially for you, the student. We need to know what you think .. in order
to improve and develop the course further. Some of the verbal feedback given during the course has already
been included in parts of the seminars, however we would like to ask you to spend a few minutes filling in
the form below (distributed during the seminar) and handing it in. It is anonymous, .. unless we discuss
your feedback during the last five minutes of this CKO-9 course, before you leave to start your internship
next Monday.
58
CKO-9 Evaluation Form Calendar year: ... Month: ...
1. Was the content and nature of the course sufficiently clear after the first lecture (HC1)?
1 2 3 4 5
No absolutely not O O O O O Yes completely
Comment.....................................................................................................................................................
2. Did the course meet your expectations?
No absolutely not O O O O O Yes completely
Comment.....................................................................................................................................................
3. Was the time you had to invest in the course in accordance with your expectations?
No absolutely not O O O O O Yes completely
Comment.....................................................................................................................................................
4. What is your overall judgement of the teacher’s performance?
Very bad O O O O O Very good
Comment.....................................................................................................................................................
5. What is your overall judgement of the teaching materials used (module guide, books, powerpoint presentations etc?)
Very bad O O O O O Very good
Comment.....................................................................................................................................................
6. How useful was the course to you?
Not at all O O O O O Very useful
Comment.....................................................................................................................................................
7. If CKO-9 was a voluntary course, would you recommend the course to others?
Not at all O O O O O Very much
Comment.....................................................................................................................................................
59
8. Were the Quick Tests useful studying Textbook and Modules?
Absolutely not O O O O O Very useful
Comment.....................................................................................................................................................
9. Did you appreciate working in peer review groups?
Not at all O O O O O Very well
Comment.....................................................................................................................................................
10. How do you rate this course?
Very bad 1 2 2 4 5 6 7 8 9 10 Very good
11. Please give three positive aspects of CKO-9 and three aspects that will have to improve
................................................................................................................................................................
................................................................................................................................................................
................................................................................................................................................................
................................................................................................................................................................
................................................................................................................................................................
................................................................................................................................................................
60
4 TWELVE-WEEK RESEARCH TRAINING
General information Besides practical professional training, scientific education is an important component of a medical study
programme, involving gaining knowledge of both scientific methods and a series of skills, like an
investigative and inquisitive mindset, the power of precise formulation, and a wide-ranging orientation
towards disciplines other than your own. These elements jointly foster the development of a scientific
attitude.
Having received scientific training in the Bachelor’s cycle (MPV, optional courses, and 5BOSA8) and in
the Master’s cycle (CKO and CKO-9), students round off their scientific training in the Research Training
Internship (RTI). This internship can be located in the final stages of their master’s phase - generally in the
final six months - when they have already gained experience at various clinical departments. However,
when students are on a residency waiting list, they may opt to take their RTI prior to CKO-1.
In all cases, students must contact the host organization well before the start of the internship to regarding
the actual date of commencement and other important matters.
It is important that students submit major changes of the approved project plan to the Research Training
Committee (RTC) confirmed by the Radboudumc final supervisor. In case of doubt (whether changes are
minor or major), submission to the RTC is advisable to prevent problems with the assessment in the end.
The Research Training takes 12 consecutive full-time weeks, a period the medical faculty considers a
minimum requirement for achieving the set objectives. The internship must be completed within this period,
including the completion of the report and assessment by the supervisor. If students should wish to pursue
their research projects on a voluntary basis after this period, they are at liberty to do so if their research
interests do not interfere with their residency or their exams. The number of credits awarded to the RTI will
be the same in all cases.
Extended RTI
There is an option for students to take an extended RTI, in which the internship is extended by a maximum
of three months and replaces their optional residency. If students indicate in advance they like to take an
extended RTI that does not replace their optional residency but that is additional to it, this can be recorded
on their degree certificate.
If students wish to take an extended RTI (replacing their optional residency and with a maximum duration
of three months), they must supply a motivation indicating how the extended length of the internship will be
expressed in the final product. This motivation must be signed by the Radboudumc final supervisor.
Obviously, if students want to take an extended internship that is additional to the regular curriculum, they
must also provide a motivation in advance. Exemption in special cases.
The scientific internship is a compulsory exam component for all students. However, for those who have
performed scientific work at a level that clearly meets the requirements made during the RTI, there is an
exemption option. The main requirement for exemption is that students should have produced an approved
dissertation leading to a doctoraal(master’s) examination on a topic or subject that is relevant to medicine.
Coordination
The final responsibility for the Research Training lies with the chair of the Research Training
Committee professor J.P.H. Drenth
- phone 024-3616999
- email [email protected]
61
Granting exemptions is the exclusive privilege of the Examining Board in Medicine, on the advice of the
Research Training Committee (RTC). An application for exemption addressed to the Examining Board in
Medicine must be made in writing, stating reasons and submitting proper evidence (report and assessment).
Supervision Supervision Protocol
It is self-evident that, in supervision, matters of content need to be dealt with. These might relate to topics
such as inevitable constraints to the study with a view to the time available, making content choices,
mastering the necessary research skills, the availability of materials, etc. Time devoted to supervision by
supervisors may vary, but a minimum time investment of 30 to 40 hours must be expected. In the initial and
concluding stages, a great deal of supervision time will be required, with less in the middle stages.
In supervision of final projects, there are several moments when students meet up with their host supervisor.
It is up to students to take the initiative for arranging supervisory meetings. In addition, students must also
make sure that their supervisor receives the relevant information in writing several days prior to the
meeting, in order for him/her to be able to prepare for the meeting properly. After the meeting has taken
place, students write a concise report of the major observations, conclusions, and appointments. This report
is a memory aid for both students and supervisors. In the case of an internship outside the Radboudumc,
moreover, students should also send a copy of the report to the Radboudumc final supervisor to keep
him/her informed of the students’ activities at the host institution.
The following delineation of six supervisory meetings, needless to say, is not meant to be rigidly
prescribed. This description is to convince students that they not only may expect to receive regular
supervision but also that students themselves are expected to carefully plan and prepare supervisory
meetings.
Introductory meeting
During their CKO-9 course, students have contacted their internship supervisor to make sure he/she agrees
on the process of research topic development during the CKO-9 course. This contact may lead to an
introductory meeting. If this is impossible, an introductory meeting at a later point in time is inevitable.
During this introductory meeting, the following points need to be raised:
- organizational matters, such as contracts, the Radboudumc supervisor, day-to-day supervision at the
host institution, facilities at the host institution, and time frame. These have obviously been arranged
in advance, but need to be checked during this meeting;
- the final products that are to be handed in (report, paper, oral presentation) and any possible
additional details, such as the language of the report;
- supervision: what arrangements have been made?;
- important information on the day-to-day state of affairs at the host institution;
- information about the research topic that is important for students to have in advance. In summary:
preparing the topic.
First meeting: the research plan (general learning objectives 1 and 2)
In the very early stages of the internship, the provisional research plan that was the final product of the
CKO-9 course needs to be finalized into a finished design. First of all, students need to reconsider the
motive of their study. What is its relevance? In which medical scientific context does the study fit? If
necessary, students need to trace, critically read, and systematically review additional literature. They re-
examine and improve their research question and work towards a fitting study design. This stage is all about
writing a sound and detailed research plan, which includes a time frame. Students hand in this plan to their
supervisor several days before their supervisory meeting is scheduled.
Topics for discussion in this meeting may include:
- the plan’s strong and weak points;
- unfeasible points and suggestions for improving these;
- appointments.
Students write a concise report of this meeting.
62
Second meeting: making the study design more concrete (general learning objectives 2 and 3)
The study design must now obtain its final shape. Choices made now need to be justified. New choices
must be made on the operationalization of the major variables.
Several days before the meeting is scheduled, students hand in the required information to their supervisor
in writing. Topics for discussion at this meeting may include:
- the fit between the study design and the research question; is the design complete?;
- the suitability and quality of the measuring instruments that are to be used;
- a data collection strategy;
- organizational aspects of data collection;
- appointments.
Students write a concise report of this meeting.
Third meeting: data analysis and description of results (general learning objectives 4, 5, and 6)
Several days before the supervisory meeting is scheduled, students hand in a partial report including data
and results. Topics for discussion at this meeting may include:
- data analysis;
- results description;
- appointments.
Students write a concise report of this meeting.
Fourth meeting: first draft of the research report (general learning objectives 7 and 8)
Several days before the meeting is scheduled, students hand in a first draft of their report. Topics for
discussion at this meeting may include:
- the structure of the report;
- the contents of the report;
- the results and statistics;
- the discussion, conclusions drawn, critical reflection on design and results;
- language: clarity, unambiguity, reasoning, and style.
Students write a concise report of this meeting.
Fifth meeting: second draft of the research report (general learning objectives 7, 8, and 9)
Several days before the meeting is scheduled, students hand in the second draft of their report, having
processed any remarks made in the previous meeting. Topics for discussion at this meeting may include:
- the processing of remarks made in the previous meeting;
- extra attention now to be paid to the discussion, the conclusion, and putting conclusions in a wider
perspective;
- agreements about finishing the report;
- agreements about the oral presentation to be given at the host institution.
Students write a concise report of this meeting.
Sixth meeting: final meeting (general learning objectives 7, 8, and 9)
Before this final meeting, the supervisor has examined the final draft of the research report. The oral
presentation of the study at the host institution has been given. This final meeting deals with:
- discussion of the oral presentation;
- assessment of the internship, report, and oral presentation;
- evaluation of the internship;
- any further positive criticisms relating to the internship, supervision, etc;
- the future: paper or follow-up study.
When the final meeting has taken place, students themselves make sure that the Radboudumc final
supervisor – if this is not the same person as the (daily or host) supervisor – is presented with a copy of the
report and the completed assessment protocol and expresses his/her (non-) agreement with the assessments
of the (daily or host) supervisor. In case of non-agreement the Radboudumc final supervisor adjusts the
grade and motivates the grade change. Especially for foreign internships, adaptation of grades is often
required, because foreign grading systems can deviate substantially from the Dutch system.
Finally, students must hand in the completed assessment protocol and two binded copies of the research
report to the StIP at the Radboudumc.
The RTC does not play a direct role in the actual supervision of RTI; it has a conditional and monitoring
role. By making sure that expedient procedures are in place and by spot-checking these procedures, the
63
committee aims to make sure that proper student supervision during the RTI is guaranteed. If there are any
problems during the internship, the committee is available for mediation and, if necessary, correction.
Writing your research report All students finalize their Research Training with a written report, preferable in English. This report may be
one of two kinds: it may either be a research report or a draft paper. In the case of a twin internship, both
students must each produce their own individual written report, as they also submitted two individual
project proposals.
The results of scientific research are commonly reported in journals. Writing and publishing a paper
requires proficiency in precise and concise formulation. Reporting the results of the RTI is therefore an
excellent opportunity to practise this skill. This is why students are expected to produce a concise report,
whether they choose to hand in a draft paper or a research report.
Research report
Students need to bear in mind that an exhaustive or wordy report causes reader exasperation and generally
does not contribute to a favourable assessment. The maximum length of the text is 5,000 words (appendices
and reference list included). The number of words must be stated on the title page.
The RTI report may be written in Dutch or English.
The RTI report must include the following elements:
- a full title page (title, author, supervisor(s), department, internship period, duration of internship,
date of the report);
- structured summary (aim, method, results, discussion) not exceeding one page;
- contents including page numbers;
- orderly structure (see below) (example: background, research question, method, statistical analysis,
results, discussion, putting your own results in a wider perspective);
- tables and figures, if any (always provided with a number and title);
- footnotes and references.
There are several ways to structure a report. An example is given below.
- Summary
This is a brief but complete description of all important elements (one page at the most). It includes
the aim, place where the study was performed, research design, methods, results, and conclusion.
- Contents
- Preface
- Chapter 1: Introduction
The introduction outlines the motive for the study. It also details the wider framework by briefly
summarising the literature relevant to the issue. The introduction ends with the formulation of the
research question.
- Chapter 2: Research question
This chapter expands the research question, paying clear attention to bottlenecks, preconditions, and
any additional questions. This is also where the concepts are defined.
- Chapter 3: Method
This chapter describes exactly how the study was designed. It deals with the following issues:
ADDITIONAL TEXTBOOK A very useful guide to performing medical research is:
Overbeke AJPM, Gijn J van, Walvoort HC. Publiceren in biomedische tijdschriften. Een praktische
handleiding. Houten: Bohn, Stafleu & Van Loghum, 1999, pp. 220.
64
study design;
population or materials investigated;
techniques and instruments;
interventions or more general determinants;
method of analysis.
- Chapter 4: Results
The results are presented in this chapter.
- Chapter 5: Discussion
This chapter deals with the following issues:
linking the results to the research question;
any methodological limitations in the design or the execution;
discussion of the results in the perspective of the literature;
any recommendations.
- References
References must be represented consistently in line with journal regulations (See Zielhuis et al. p.
170).
Example 1:
Zielhuis G.A., Heydendael P.H.J.M., Maltha J.C., Riel P.L.C.M. van (2010). Handleiding medisch-
wetenschappelijk onderzoek. Utrecht: Wetenschappelijk uitgeverij Bunge.
Example 2:
Taylor S.E. Hospital patient behavior: reactance, helplessness or control? Journal of Social Issue
(1979); 35: 156_184.
- Appendices
For example: research instruments, letters, additional tables and graphics, glossary, etc.
A draft paper
If students choose to write a draft paper as their final report, their contribution is likely to be part of a team
product, as there often are multiple authors. To make sure that the students’ contribution is acceptable,
students are required to be either the 1st or the 2
nd author of the draft paper. The RTC prefers a (first) draft
article over a final one. Not only because a final one usually requires more time than there is available (see
deadline), but also because a draft version provides more insight into what students themselves are able to
c.q. what they have learned. Also, in addition to the draft article, the RTC calls for a separate process
description (max 2 A4) where students describe which research activities they have conducted themselves
and how this went.
The draft paper must be structured in accordance with a structure that is common for research papers.
Maximum length is about 3,000 words (author guidelines for the aimed journal is decisive). The number of
words must be stated on the title page.
Handing in the report
After completion of the internship, the final report (either the research report or the draft paper plus the
process description) must be handed in to the following people as soon as possible (at the very latest after 3
months):
1. the (daily or host supervisor. This person assesses the students’ performance on the basis of the first
assessment protocol and, therefore, includes the general learning objectives in his or her assessment
and, hence, also the quality of the report as well (general learning objective 8);
2. the Radboudumc final supervisor. This may be – but need not be – the same person as the one above.
In any case, the Radboudumc final supervisor must always be presented with a copy of the report.
This Radboudumc final supervisor must also indicate whether he/she accepts the assessments of the
(daily or host) supervisor. In case of non-agreement the Radboudumc final supervisor adjusts the
grade and motivates the grade change. Especially for foreign internships, adaptation of grades is
often required, because foreign grading systems can deviate substantially from the Dutch system.
The assessment on the first assessment protocol of the Radboudumc final supervisor is decisive. In
all cases, the Radboudumc final supervisor has to sign the protocol.
3. StIP (two binded copies). Two copies of the internship report AND the completed first assessment
protocol must immediately be handed in to StIP. These two copies should be binded! The StIP will
then forward one copy of the report to a second assessor appointed by the RTC. The second copy
will be kept on file in the library. With a view to their possible graduation date, students are advised
65
to bear in mind that assessment by this second assessor may take a number of weeks. Students are
therefore advised to hand in their internship report at least six weeks before the graduation date of
their choice. Students with a final grade of 8 or above may qualify for Faculty or University awards
(see next section).
Faculty and university awards
In order to promote students’ motivation, Radboud University and Radboudumc award a number of prizes
for the best internship reports or master theses in an academic year.
a. The Radboud University Student Prize (€ 1000) is awarded by the University to one student in each
faculty. This prize is awarded on the recommendation of the Radboudumc’s Board of Directors.
b. Faculty student prizes of € 350 each (one for medicine, one for dentistry, and one for biomedical
health sciences) may be awarded by the Radboudumc’s Board of Directors on the recommendation
of the Commission Master Awards.
c. Faculty incentive prizes of € 125 each (one for each study programme) may be awarded by the
Board of Directors on the recommendation of the Commission Master Awards.
Award candidates can be registered with the Commission Master Awards by the internship supervisor
and/or the Radboudumc final supervisor. For this purpose, they should submit a written motivation of their
registration, including two copies of the reports concerned.
The Commission Master Awards assesses all registrations and advises the University Council on the
awards. The Commission consists of eight members (with doctoral degrees) of the scientific staff of the
Faculty of Medical Sciences, selected from departments that are involved in the three study programmes.
66
APPENDICES
Registration form: page 67
Template Project Outline and tips page 69
Two examples page 72
1ste
Assessment protocol page 76
2nd
Assessment protocol page 80
67
68
EXPLANATION AND INSTRUCTIONS CKO9
In CKO9, you will be preparing your internship by expanding your research question, methodology, statistics, etc. The internship always follows on to CKO9 but need not follow immediately.
The CKO9 course will structurally be offered in weeks 3 and 4 of each period.
Enrolment in CKO9 is conditional upon: 1) your having completed your residency or your waiting for CKO1v to start while already having
been assigned to a residency group; and 2) your having arranged an internship position; and 3) your internship proposal having been approved.
Enrolment in CKO9 is also conditional upon your signing up well ahead of time; this means your registration must be in on the 1st day of the period preceding the CKO9 period. For example: if you wish to enrol in CKO9 in weeks 3 and 4 of period 2, your registration form must be in on day 1 of period 1 at the very latest.
There is the possibility of doing an extended research internship (4, 8, or 12 week extension). If you wish to receive additional credits for the extended internship, please indicate this on your registration form. If you wish use these credits to compensate one or several optional residencies in Episode 10, you should also indicate this separately. Once launched, internships cannot be changed into extended internships, and any additional internship time will not earn you extra credits.
GENERAL LEARNING OBJECTIVES of the research internship 1. Students can specify the motive for their study, the interest it serves, and the medical scientific
context it fits into. 2. Students can propose a definitive study design on the basis of the following skills:
2A - Students can retrieve, critically read, and systematically review relevant literature; 2B - Students can fine-tune/elaborate the research question on the basis of the literature;
2C - Students can frame a study design that fits the definitive research question. 3. Students can organize and carry out the proposed study. 4. Students can systematically analyse the collected data. 5. Students can lucidly describe the results and neatly summarize them in tables and figures. 6. Students can indicate errors of measurement and other limitations in the collected data. 7. Students can reflect critically on their study design and results. 8. Students can produce a written research report that is in line with the requirements made upon
such reports. 9. Students can orally present the outcomes of their study to the department where the study was
performed and are able to engage in fruitful discussion of these outcomes.
PERSONAL LEARNING OBJECTIVES of the research internship In addition to general learning objectives, there may be specific personal learning objectives relating to a particular type of internship: in one type of internship, students will need to compose a questionnaire for data collection purposes, and, in another type, they will need to recruit test subjects. A range of specific learning objectives may be called for on a case-by-case basis. It is important for students and their supervisors to agree in advance about the specific learning objectives: which specific skills must have been developed by students upon completing their internship?
COMPLETING THIS EXTRA INFORMATION IS OBLIGATORY (more choices possible). Internship is based on: □ (Only) existing material
□ Gathering new material
Accreditation by Ethics Committee is: □ Necessary □ Not necessary
□ Already approved Activities to be performed by student:
□ Acquisition of study subjects
□ Taking measurements on study subjects (patients, animals etc.)
□ Taking measurements on (bio)material
□ Taking interviews
□ Encoding interviews
□ Learning specific (lab) skills
□ Attending policlinic / operations with regard to the study
□ Preparing an accreditation by the Ethics Committee
□ Analysing already prepaired data
□ Building own data files
□ Simple statistic analyses (descriptive, T-test etc.)
□ Advanced statistic analyses supported by a statistician (e.g. multivariate analysis etc.)
69
Template Project Outline - Research Training
Internship (preferable 2A4 max, incl. timetable)
Title
Background of the research (based on 1-2 key publications
Research question / Hypothesis
Study design / study population / number of subjects / variables (including primary
outcome)
Rationale (Why is the research important? / How the knowledge can be of use? /
What is the scientific/clinical/societal impact)?
Detailed plan of work: specification of students’ activities and completed timetable (*
see at bottom)
Feasibility? Any problems anticipated? Which?
Reference list of literature (1-2 key publications)
Tips at page 26 - 27
Examples at page 28 -31
Tip A
Tip B1-B2
Tip C
Tip D1-D8
Tip E
Tip F
Tip G
Tip H
70
* TIME TABLE Research Training Internship (Do not include CKO9!)
Please mark with a cross (X) the applicable activities and, if necessery, expend the table with
more relevant activities (and weeks).
Week 1 2 3 4 5 6 7 8 9 10 11 12
Study of literature
Technical instruction /
practise technique /
learning specific lab
skills
Preparations in
advance, logistics (A)*
Taking measurements
(B)*
Data analyses /
statistics
(C)*
Write introduction
Write method
Write results
Write discussion
Prepare presentation
Presentation to host
department
Miscellaneous :
Miscellaneous :
* A Please specify A (for instance: Draw sample, invite/recruite subjects etc)
* B Please specify B (for instance: Distract data from statuses, taking interviews, other
measurements on study subjects, measurements on (bio)material etc)
71
TIPS ON WRITING THE PROJECT OUTLINE
Title Tip A: Give succinct, short catchy title
Background Tip B1: Set project in 3-4 sentences, describe the field and provide
the information that led up to the research question
Tip B2: Funnel background information further, focus on the unmet
needs and describe which elements are worth to study
Research question Tip C: Fix project and frame the most important research question, if
applicable, in the format of an hypothesis that can be tested
Study design Tip D1: Label the type of study
Tip D2: Describe accurately main characteristics of study population
Tip D3: Describe major inclusion and major exclusion criteria
Tip D4: Give accurate description of the control population
Tip D5: Describe precisely the most important outcome measure you
want to assess
Tip D6: Then define the secondary outcome measures
Tip D7: Describe the rationale for the size of the study
Tip D8: If you already know which statistical analyses you are going
to do, please describe
Rationale Tip E: Why in the first place should you start this study? Is there
some overarching importance to do so? What is the significance of
this study proposal?
Plan of work Tip F: Put into words the specific activities you are going to carry
out and also complete the time table
Feasibility Tip G: Show you really have thought about the project and whether
it is feasible to do. Explain possible difficulties that you expect to meet
and how you think to overcome these
Literature Tip H: 1-2 key publications
72
Example 1 - Project Outline Research Training Internship Department: Vascular Surgery, Catharina Hospital Eindhoven
Title Gait analysis in patients with peripheral arterial disease.
Background Patients with peripheral arterial disease (PAD) often present with intermittent claudication. This typical
pain in the leg musculature during ambulation decreases in rest. Patients with PAD show reduced strength
in the lower extremities, lower physical activity levels, limited walking performance and reduced health-
related quality of life.[1,2]
Previous research described reduced step length and decreased walking speed
during over ground walking measures after ischemic pain was induced on a tread-mill.[2-4]
Sufficient gait analysis during the development of ischemic pain on a tread-mill is lacking. Since tread-mill
walking consumes significant more energy compared to over ground walking,[5]
continuous measurement
on a tread-mill could monitor specific changes during walking in similar circumstances as supervised
exercise therapy. The Optogait is an advanced instrument for quantifying spatiotemporal gait parameters
and allows for continuous measurements during treadmill walking. It has already been used in healthy
adults and patients with a total knee replacement.[6]
Proper gait analysis could contribute to the development
of improved treatment regimes.
Research question/hypothesis Do patients with PAD have a shorter step length compared to healthy controls?.
Design We will perform a prospective case control study including 200 patients with PAD and 200 healthy
controls. Patients diagnosed PAD (Fontaine-score of II and ankle-brachial index of <0,9) will be recruited
from our outpatient department. Patients will be excluded if they have comorbidities that are expected to
interfere with their walking performance, including amputation, osteoarthritis, pulmonary disease, heart
failure or use of walking aids. A total of 200 healthy controls (ankle-brachial index >0.9) without any
significant comorbidities interfering with their walking performance (aforementioned) will be recruited.
Our primary outcome is step length as assessed with Optogait. Secondary outcome measures are walk cycle
time, stance time, swing time (in which the foot is suspended and proceeds in the air), stride length, cadence
(steps/min), walking speed, single and double support phase.
We expect that patients with PAD will have a 8% shorter step length and a sample size of 200 patients per
arm is needed to detect a significant difference between the both groups (power: 80%; α: 0.05).
Rationale By providing data on gait pattern characteristics we will be able to develop an optimized treatment protocol
for patients with PAD to improve overall walking performance. Ultimately, this will lead to an
improvement of health-related quality of life.
Plan of work 1. Patients will be recruited from the vascular surgery outpatient clinic of Catharina Hospital in Eindhoven.
2. After their visit at the outpatient clinic, patients will undergo gait analysis on a tread-mill in a test
laboratory in the same hospital. Patients will be transported by wheelchair or asked to rest at least twenty
minutes after their walk through the hospital before they are tested.
3. Gait analysis is performed and the patients will be asked to walk as far as possible on their preferable
walking speed. Patients are instructed to inform the researcher about the onset of pain. The Optogait
monitor will be used to measure the data mentioned above.
4. The same gait analysis will be performed on 30 healthy subjects, after PAD is excluded by a normal
ankle-brachial index.
5. Outcome measurements will be analyzed using SPSS.
See also timetable
Feasibility This research internship is part of a larger study (PhD thesis). The study has been approved by the Ethical
Board of the Radboud University Medical Center. I will be responsible for the measurement of 30 patients
and 30 controls. That makes it possible for me to finish this project within 12 weeks time. Identifying
controls could be difficult as we need to randomly select healthy individuals from the population, matched
73
for possible confounders. Currently we are discussing the use of random volunteers from our personal inner
circle versus partners of patients.
Reference list 1. Altered gait profile in subjects with peripheral arterial disease. Gardner AW, Forrester L, Smith
GV. s.l. : Vascular Medicine, 2001, Vols. 6:31-34.
2. Relationship between temporal-spatial gait parameters, gait kinematics, walking performance,
exercise capacity, and physical activity level in peripheral arterial disease. Crowther RG, Spinks
WL, Leicht AS, Quigley F, Golledge J. s.l. : Journal of Vascular Surgery, 2007, Vols. 45: 1172-
1178.
3. Peripheral arterial disease affects kinematics during walking. Celis R, Pipinos II, Scott-Pandorf
MM, Myers SA, Stergiou N, Johanning JM. s.l. : Journal of Vascular Surgery, 2009, Vols. 49:127-
132.
4. Peripheral arterial disease affects the frequency response of ground reaction forces during walking.
McGrath D, Judkins TN, Pipinos II, Johanning JM, Myers SA. s.l. : Clinical Biomechanics, 2012,
Vols. 27: 1058-1063.
5. Kinematic, kinetic and metabolic parameters of treadmill versus overground walking in healthy
older adults. Parvataneni K, Ploeg L, Olney SJ, Brouwer B. s.l. : Clinical Biomechanics, 2009,
Vols. 24:96-100.
6. Validity of the Optogait photoelectric system for the assessment of spatiotemporal gait parameters.
Lienhard K, Schneider D, Maffiuletti NA. s.l. : Medical engineering & physics, 2013, Vols. vol:35
iss:4 pg 500-504.
7. Exercise Training for Claudication. Stewart KJ, Hiatt WR, Regensteiner JG, Hirsch AT. s.l. : New
England Journal of Medicine, 2002, Vols. 347(24):1941-51.
TIMETABLE Weeks
1 2 3 4 5 6 7 8 9 10 11 12
Project Finishing literature study X X Technical instruction/practice
technique/learning lab skills
X X
Preparations, logistics, testing of equipment
(specify under A.)
X X
Taking measurements (specify under B.) X X X X X X X Statistical analysis X X X X Interpreting results X X
Writing study report Writing introduction of study report X X X
Writing methods X X X X
Writing results X X X
Writing discussion X X X
Presenting results Prepare oral presentation X X
Oral presentation at (host) department X
Miscellaneous / other ……
……
A. Specification of study preparations: Installation Optogait and tread-mill, writing research
protocol en patient information for approvement METC, start preparations for including patients at
outpatient department, test analyses on tread-mill
B. Specification of study measurements: Patient recruitment, examination of gait pattern of patients
and controls following a standardized research protocol, data extraction to SPSS/Excel.
74
Example 2 - Project Outline Research Training Internship Department: Fysiology, Radboudumc, Nijmegen
Title Influence of an (in)active lifestyle on vascular endothelial function in men with and without a history of
myocardial infarction.
Background of the research Worldwide cardiovascular diseases (CVD) are still a growing health problem and the leading cause of death.
In 2012, CVD accounted for 17.3 million deaths. This number is expected to grow to 23.6 million deaths in
2030. One third to half of the CVD consists of coronary heart diseases1. A
recent paper in The Lancet
revealed that physical inactivity has overtaken smoking as the principal cause of cardiovascular death
worldwide2. Although exercise seems to be a powerful tool to prevent CVD, not all cardio-protective effects
of exercise are understood. Analyses of the literature support several potential mechanisms for the cardio-
protective effects of exercise by an improved endothelial function3. However, a previous study revealed that
healthy active individuals are not immune to CVD4. If differences are present in endothelial function between
athletes with and without a myocardial infarction, it might provide new insight for a non-response to the
cardio-protective effects of exercise. Therefore, the purpose of this study is to examine the differences in
endothelial function in athletes with a history of myocardial infarction, healthy athletes, sedentary individuals
with a history of myocardial infarction, and healthy sedentary individuals. I hypothesize that the healthy
athletes will have a better endothelial function compared to the athletes with a myocardial infarction. In
addition, I expect that athletes with a myocardial infarction and inactive subjects without a myocardial
infarction will have a comparable endothelial function.
Research question (or hypothesis) What are the differences in endothelial function between athletes with a history of myocardial infarction, healthy athletes, sedentary individuals with a history of myocardial infarction, and healthy sedentary individuals?
Study design Study design: Cross-sectional observational study.
Study population: 40 male subjects, aged > 40 years. Based on physical lifestyle and history of myocardial
infarction they will be divided into four groups (Table 1).
Athletes Sedentary individuals
History of myocardial infarction Group A Group C
Healthy Group B Group D
Table 1: Schematic overview of the groups in the proposed study
Power: Based on anticipated differences in %FMD between study groups (active vs. inactive) of 3.5% with a
SD of 2.4,5,6
a power of 90% and alpha 5% significance level, we calculated that 2*((1.65+1.28)
2*2.4
2)/3.5
2)=8 subjects per group should be included. We will include 10 subjects per study
group to prevent problems in case subjects quit the experiment. Variables: Endothelial function of the brachial artery as measured by the flow-mediated dilation (FMD).
Study methods: The endothelial function of the brachial artery will be examined by inflating a blood
pressure cuff around the forearm, distal from the imaged artery. Using echo-Doppler, the brachial artery
baseline diameter and blood flow will be assessed. Subsequently, the blood pressure cuff will be inflated to
220 mmHg to block arterial inflow. After 5 minutes, the blood pressure cuff is released and hyperemic blood
flow and changes in arterial diameter and flow will be assessed. These responses are commonly referred to
as flow-mediated dilatation and represent a nitric oxide-mediated endothelium-dependent vasodilatation in
the brachial and artery.
Primary outcome: To determine the brachial endothelial function measured by FMD of athletes with a
history of myocardial infarction (A), healthy athletes (B), sedentary individuals with a history of myocardial
infarction (C), and healthy sedentary individuals (D).
Analysis: Statistical analysis will be performed by two tailed ANOVA analysis and subsequent contrast
analysis via Bonferroni.
75
Rationale The findings of this study will enhance our understanding of potential underlying mechanisms of
cardiovascular disease and myocardial infarction and the role of physical activity in prevention of myocardial
infarction.
Feasibility This is an ongoing study, in which three groups have been measured (A, B and D). During my internship, I
will include physically inactive subjects with a history of a myocardial infarction (group C) and I will screen
subjects to determine eligibility for the study. Although this is a common group of subjects, some difficulties
and motivational problems to complete the study might occur. After the measurements of group B, I will
analyze the data of all four groups and write a paper including all four groups. Since the Echo-Doppler
measurements will be conducted by an experienced sonographer I will assist during measurements. I will
also interpret the results.
Time table Research Training Internship
Week 1 2 3 4 5 6 7 8 9 10 11 12
Study of literature X X
Learn how to assist and interpret measurements X
Preparations in advance (A) X X
Taking measurements (B) X X X X
Data analyses/statistics X X
Write introduction X X
Write method X X
Write results X X
Write discussion X X X
Prepare presentation X X
Presentation to host department X X
Finish research report X
*A: Invite/recruit subjects *B: The flow mediated dilatation (FMD) of the brachial artery.
References 1. Laslett LJ, Alagona P Jr, and all. The worldwide environment of cardiovascular disease: prevalence,
diagnosis, therapy, and policy issues: a report from the American College of Cardiology, Journal of American College of Cardiology, 2012;60(25 Suppl):S1.
2. Lee, I.M., et al., Effect of physical inactivity on major non-communicable diseases worldwide: an analysis of burden of disease and life expectancy. Lancet, 2012. 380(9838): p. 219-29.
3. Bowles DK, Laughlin MH. Mechanism of beneficial effects of physical activity on atherosclerosis and coronary heart disease. J Appl Physiol. 2011;111(1):308-310.
4. Green, L.H., S.I. Cohen, and G. Kurland, Fatal myocardial infarction in marathon racing. Ann Intern Med, 1976. 84(6): p. 704-6.
5. Black MA, Cable NT, Thijssen DH, Green DJ. Impact of age, sex, and exercise on brachial artery flow-mediated dilatation. Am J Physiol Heart Circ Physiol.2009;297(3):H1109-1116.
6. Dalli E, Segarra L, Ruvira J, Esteban E, Cabrera A, Lliso R, Lopez E, Llopis E, Sotillo JF. [Brachial artery flow-mediated dilation in healthy men, men with risk factors, and men with acute myocardial infarction. Importance of occlusion-cuff position]. Rev Esp Cardiol. 2002;55(9):928-935.
76
1st ASSESSMENT PROTOCOL RESEARCH INTERNSHIP IN MEDICINE • For host supervisor and to be completed by final Radboudumc supervisor • Students hands in this form to StIP with two binded copies of the internship report
Student’s name and initials Student number
Title of the internship project
Internship starting date Internship finishing date Report handed in on (date)
What was the nature of the research internship (please tick as appropriate)
Mainly literature study Mainly study of patient file data Study of available patient-related data Study based on student’s performance of measurements in patients Mainly laboratory study performed by student Other: …………………………………………..
Supervisor’s institution, department, and name Name and phone no. of UMC supervisor in charge
Supervisor’s signature
ASSESSMENT BY SUPERVISOR A. Assessment of the student’s PERFORMANCE in the research internship
1. Inquisitive and interested in topic and research
2. Critical mindset in study design,
implementation, and report
Fail question-able
pass
fair
good
Very good
fail
question-
able
pass
fair
good
very good
3. Organization and efficiency in designing and
implementing the study
4. Time planning
Fail question-
able
pass
fair
good
very good
fail
question-
able
pass
fair
good
very good
5. Autonomy in performance
6. Commitment and participation
Fail question-
able
pass
fair
good
very good
fail
question-
able
pass
fair
good
very good
7. Openness to feedback/criticism and willingness to
use it
8. Adequately seeking assistance (well in time
and not indolently)
Fail Question-
able
pass
fair
good
very good
fail
question-
able
pass
fair
good
very good
9. Interaction with patients (if applicable)
10. Interaction with colleagues during internship
Fail question-
able
pass
fair
good
very-good
Not
appl.
fail
question-
able
pass
fair
good
very good
A. GRADE for the student’s PERFORMANCE in his/her internship on the basis of the above (half marks are allowed)
(1 = unworthy of marking, 2 = very poor, 3 = poor, 4 = strongly insufficient, 5 = insufficient, 6 = sufficient, 7 = good, 8 = very good, 9 = excellent, 10 = outstanding).
Please continue with your assessment of the GENERAL LEARNING OBJECTIVES
77
B. Assessment of the GENERAL LEARNING OBJECTIVES of the research internship To what degree has the student shown that he/she possesses the following skills? Your explanation of these items is greatly appreciated (see next page under E). If a learning objective has not been dealt with, we kindly ask you to indicate why, also under E.
1. Can indicate motive for the study, the interest it
serves, and the medical scientific context into
which it fits
2a Can retrieve, critically read, and
systematically review relevant literature with a
view to a definitive study design
fail question-
able
pass
fair
good
very good
fail
question-
able
pass
fair
good
very good
2b. Can fine-tune/elaborate the research question
with the aid of the literature with a view to a
definitive study design
2c. Can frame a study design that fits the
definitive research question
fail
question-
able
pass
fair
good
very good
fail
question-
able
pass
fair
good
very good
3. Can organize and implement the proposed study
4. Can systematically analyse the collected data
fail question-
able
pass
fair
good
very good
fail
question-
able
pass
fair
good
very good
5. Can describe the results clearly and neatly
arrange them in tables and figures
6. Can point out errors of measurement and other
limitations in the collected data
fail question-
able
pass
fair
good
very good
fail
question-
able
pass
fair
good
very good
7. Can critically reflect on the study’s design and
results
8a. Can produce a written report in line with
content requirements made upon such reports.
The quality of the report’s content is:
Fail question-
able
pass
fair
good
very good
fail
question-
able
pass
fair
good
very good
8b. Can produce a written report in line with formal
requirements made upon such reports (neat
editing, layout, use of language, correct references,
etc.). The formal quality of the report is:
9. Can orally present the study’s outcomes to the
department where the study was performed and
can engage in a fruitful debate on the subject
Fail
question-
able
pass
fair
good
very-
good
fail
question-
able
pass
fair
good
very good
B. GRADE for ACHIEVING THE GENERAL LEARNING OBJECTIVES of the internship on the basis of the above (half marks are allowed)
(1 = no performance, 2 = extremely poor, 3 = poor, 4 = strongly insufficient, 5 = insufficient, 6 = satisfactory, 7 = good,
8 = very good, 9 = excellent, 10 = outstanding).
Please note: The final grade for this research internship will be determined by the Research Internship Committee, after the report has also been assessed by a 2
nd assessor appointed by the Committee.
Please continue with the SPECIFIC LEARNING OBJECTIVES
78
C. Evaluation of SPECIFIC LEARNING OBJECTIVES of the research internship Please mention specific learning objectives and explain in your own words whether and why these have been achieved or not.
Learning objective 1: Learning objective 2: Learning objective 3:
D. Declaration of the Radboudumc FINAL SUPERVISOR
I Agree / Disagree with the assessments and grades given under A and B
Agree with A and B (in case 9 or 10 please fill out “ad 1”)
Disagree with both A and B (fill out “ad 2”)
Partial agreement (fill out both ad 1 if applicable) and ad 2)
Ad 1. Agree with grading 9 or 10: The research Training Committee strongly urges giving grades 9 or 10 as an exception, and only in case of outstanding quality. Please note: abroad the grading system often deviates from the Dutch one (higher grades abroad). If you agree with a 9 or 10, please state your reasons below.
Motivation:
Ad 2. Disagree with the grades of part A and/or B: Please state your reasons for disagreeing with the grading of part A or B and bear in mind that abroad the grading system often deviates from the Dutch one (higher grades abroad). The research Training Committee strongly urges giving grades 9 or 10 as an exception, and only in case of outstanding quality. Please give your grades.
A B Your grades: Half marks are allowed Motivation:
NB: 1 = no performance, 2 = extremely poor, 3 = poor, 4 = strongly insufficient, 5 = insufficient, 6 = satisfactory, 7 = good, 8 = very good, 9 = excellent, 10 = outstanding
Please continue with Explanation / Other remarks and your signature
79
E. EXPLANATION of assessment of general learning objectives / Other remarks
Please indicate concisely and explicitly which comments, advice, or areas for further development you wish to point out to the student with a view to his/her prospective career. Please use the reverse side of this form if needed.
Signature of Radboudumc final supervisor Date Student’s signature
80
2nd ASSESSMENT PROTOCOL RESEARCH INTERNSHIP IN MEDICINE
• To be completed by the 2nd assessor (appointed by the Research Internship Committee) Student’s name and initials Student number
Title of the research report
Name of 2
nd assessor Department Phone number
ASSESSMENT OF THE REPORT If one of the elements has not been dealt with in the report, we kindly ask you to indicate this in the Remarks section.
1. general report layout (title page, contents,
numbering, notes, literature)
2. quality of figures, tables, and graphics
Fail
question-able
pass
fair
good
very good
fail
question-
able
pass
fair
good
very good
3. quality of the summary
4. quality of the introduction
Fail question-
able
pass
fair
good
very good
fail
question-
able
pass
fair
good
very good
5. justification and formulation of the research
question
6. justification of the study design
Fail
question-
able
pass
fair
good
very good
fail
question-
able
pass
fair
good
very good
7. clear description of materials and methods
8. clear description of analysis
Fail question-
able
pass
fair
good
very good
fail
question-
able
pass
fair
good
very good
9. clear description of results and conclusions
10. quality of discussion of literature
Fail question-
able
pass
fair
good
very good
fail
question-
able
pass
fair
good
very good
11. critical reflection on study and findings 12. quality of written use of language
Fail
question-
able
pass
fair
good
very good
fail question-
able
pass
fair
good
very good
Remarks (e.g., regarding work load, innovativeness, risks, etc.)
Grade for the research internship REPORT With a view to the above, grade given by 2nd assessor (half marks are allowed).
Give marks 9 or 10 as an exception, and only in case of outstanding quality. If do so, please motivate under “Remarks”. 1 = no performance, 2 = extremely poor, 3 = poor, 4 = strongly insufficient, 5 = insufficient, 6 = satisfactory, 7 = good, 8 = very good, 9 = excellent, 10 = outstanding
Signature of 2nd assessor Date
81