improving the quality of plain english summaries for … of current practice and consultation with...

1

Improving the quality of plain English summaries for

NIHR funded research:

Review of current practice and consultation with stakeholders

TwoCan Associates

December 2012

2

Chapter 1: Introduction

1.1 In August 2012 INVOLVE was asked by the Department of Health to work with the National Institute for Health Research (NIHR) Programmes and other key stakeholders to:

review and develop the question and guidance for plain English summaries in NIHR funded research as part of the Standard Application Form

develop criteria and propose methods for assessing the quality of plain English summaries

It was anticipated that this work would be used to assist progress in developing plain English summaries for the UK Clinical Trials Gateway (UKCTG).

1.2 TwoCan Associates (www.twocanassociates.co.uk) was commissioned by

INVOLVE to help with the development of this project. An Advisory Group was also established. Membership is listed in Appendix 1.

1.3 The project was undertaken in two stages:

A review of current practice in producing research summaries in plain English. This review, described in chapter 2, informed the development of draft guidance for researchers and research programmes, which was then considered and amended by the Advisory Group.

A consultation with a range of stakeholders. This consultation, summarised in chapter 3, informed a re-drafting of the guidance, which was then further amended by the Advisory Group.

1.4 The project began in October 2012 and was completed in December 2012.

http://www.twocanassociates.co.uk/

3

Chapter 2: Review of current practice

1. Introduction 1.1 The first phase of this project was a brief review of the current practice of a

range of organisations, including NIHR Programmes, medical research charities which took part in the Association of Medical Research Charities’ ‘Natural Ground’ programme to promote the development of PPI, the National Research Ethics Service and a range of websites that include information about trials. The information was obtained via an internet search, via INVOLVE or Advisory Group members or email requests to research managers. The review was undertaken in 3.5 days, and was therefore not intended to be exhaustive.

1.2 In this chapter we summarise the findings from this review, covering:

what is required with regard to plain English summaries for different research organisations (Section 2)

the guidance provided to researchers to enable them to submit a plain English summary (Section 3)

the information we have been able to find about how these plain English summaries are assessed, who assesses them and any published criteria that are used for this assessment (Section 4)

the issues that have emerged from this review (Section 5).

2. What do research organisations require with regard to plain English summaries?

NIHR organisations

2.1 The NIHR Programmes involve patients and/ or the public in the process of funding review. In this context, the main purpose of the plain English summary is to enable lay reviewers to understand the research proposal and the likely impact of the research for patients. Non-specialist reviewers (clinicians and scientists who do not have specialist knowledge of a topic) also find the plain English summary useful for their review. Other uses of the summary include assisting in answering information requests and for Coordinating Centre staff to check the remit of the applications.

2.2 In terms of their requirements of the plain English summary, the NIHR

Programmes fall into one of two categories. Those in the first category simply state that the aim of the plain English summary is to support lay review but do not specify what details are required (see Table 1). Other NIHR Programmes do specify a list of topics that need to be covered in the plain English summary (see Table 1). All the Programmes that do this use practically the same headings as listed below:

how the application addresses the research proposed

4

the likely benefits to patients and the NHS

how and where the research will be carried out

what outcomes will be used to assess the success of the research

what (if any) are the ethical issues involved in this study and the arrangements for handling these

why this team is well placed to carry out the research

provide a justification for the costs requested (including any NHS costs).

It is not clear how these headings were agreed or who identified them. 2.3 All of the NIHR Programmes (See Table 1) publish the plain English abstracts

from the research proposals they fund. They are not explicit about the purpose of this, but some advise researchers to ensure that the information in their summary is not commercially sensitive.

2.4 The NIHR Trainees Coordinating Centre (NIHRTCC) asks for a brief summary of the proposed research (100 words), including objectives, design, expected outcomes and anticipated benefits to the ongoing improvement of health or social care (see Table 1). Abstracts of successful awards are published on their website.

Medical research charities

2.5 All of the research charities included in this review had worked in partnership with a group of relevant patients/ carers to develop the structure of their plain English summaries. In all cases this had been part of a larger piece of work to develop processes of patient/ carer involvement in grant application review.

2.6 There is considerable variation across the charities in terms of the required

structure and length of the plain English summary. Some organisations require researchers to complete a lay application form 2-4 pages long. Others request a shorter summary (see Table 2). There are many contextual factors that may explain these differences including:

How much weight is given to the lay review - for example the review process at The Alzheimer’s Society is patient/ carer led and they require researchers to complete a full lay-application form in the initial stages.

Whether lay reviewers carry out their reviews independently or work with a group of patients/ clinicians/ researchers in the review process i.e. how much they need to rely on the plain English summary to inform their judgement, and whether they can easily obtain help with explanations of other technical parts of the form, if required.

The needs of the group involved. Some service users experience communication difficulties e.g. people affected by stroke, and may need to have all the information provided in an accessible format in order to carry out their review.

5

2.7 All of the research charities included in this review specify a list of topics to be covered in the plain English summary (See Table 2). Some of these topic headings are similar, although they are expressed in different ways. For example, Parkinson’s UK asks ‘How will the results of the research be communicated?’ while the Alzheimer’s Society asks ‘How will you disseminate your findings?’ However there is a lot of variation and no obvious standard format.

2.8 Parkinson’s UK also asks researchers to provide a plain English summary in

addition to a plain English abstract. The plain English summary is only 250 words in length and is used for general dissemination in the Charity’s publications, quarterly magazine and website. The Meningitis Research Foundation uses a similar structure for its plain English summaries (250 words) which are only used for publicity purposes (they do not involve lay reviewers).

IRAS (Integrated Research Application System)

2.9 The IRAS form asks researchers to provide a research summary for a wide

audience. They state that the summary ‘should be suitable for the public and patients wanting more information about their condition, researchers reviewing current literature and doctors planning treatment’. The

summary is only 300 words in length and therefore ‘cannot be comprehensive and will need compromise to meet all audiences’. It is intended to be a

signpost, to enable any reader to assess whether they want more information about the project and then to find more details.

2.10 The suggested outline for the summary is shown below:

Why? What research question is being addressed? How is it of relevance and importance to patients and public?

What? Broadly what area (disease, therapy or service) is being studied? For therapeutic studies what is the drug, device or procedure being tested.

Who? Who would be eligible?

Where? The type of sites where the study will be conducted.

How? How long will the study last and what will the participants undergo?

2.11 The majority of the Research Networks use the IRAS summary or a shortened

version of it in their portfolios1. The exception is the Mental Health Research Network which has involved service users in writing plain English summaries.

2.12 NRES plans to publish summaries of all research ethics committee

applications, together with a summary of the ethical opinion on their website. They state this in the interests of open research and ethical research. They

6

will also include information from other parts of the IRAS form to enable readers to find out more information if they wish.

Websites with lists of clinical trials 2.13 Five different websites with lists of clinical trials were reviewed for this report:

The UK Clinical Trials Gateway, The MRC Clinical Trial Unit, CancerHelp UK, ClinicalTrials.gov and the International Standard Randomised Controlled Trial Number Register (ISRCTN). All of these websites include a plain English summary. These are similar in terms of the topics covered, although they vary in length. There is some overlap with the topics covered in summaries for grant application forms in that they ask for the background to the study and its aims. However, all the other topics are different to the funding application summaries. They include questions about:

who is eligible to take part and who is not eligible

what will happen to the participants - interventions received, extra hospital visits etc

potential risks and benefits of taking part

when recruitment is taking place

where the study is taking place including all the individual trial centres

how long the study will run for

who is funding the study

who is the main contact.

2.14 When the aim of the plain English summary is to help patients and their

clinicians to identify relevant trials that they might participate in, the information required is very different to the information required to help a reviewer assess a funding application.

3. Guidance provided to researchers to enable them to submit a plain English

summary 3.1 The guidance provided to researchers is very similar across all the research

organisations included in this review. We have therefore considered them altogether in this section. Further detailed information of what each organisation does can be found in Tables 1 and 2.

3.2 Some organisations provide one or two paragraphs of guidance that simply

state that the plain English summary should be suitable for people without a specialist scientific or medical background and therefore researchers should either avoid scientific/ technical terms or acronyms, or explain them fully.

7

3.3 Some organisations provide a link to the Plain English Campaign guidance available on its website. There are several different types of guidance available including a publication on writing medical information2. This includes examples of letters to hospital patients that have been rewritten in plain English. These examples are not directly relevant to the research context. However it also contains an A-Z list of medical terms and their plain English equivalents that may be useful for some researchers.

3.4 Some organisations have produced their own guidance on writing in plain

English. This varies slightly from organisation to organisation, particularly in the use of condition-specific terms. For example Cancer Research UK uses examples of common terms used in cancer research, and the Stroke Association uses terms relevant to stroke research. Otherwise the guidance is virtually identical.

3.5 We found a few simple rules for writing in plain English. These are consistent

in all the guidance reviewed and (See Appendix 2) include:

Advice on layout e.g. lots of white space, bullet points for lists.

Advice on style e.g. use short sentences, use simpler words and phrases, active not passive phrases.

Advice on evaluating a summary e.g. reading it out loud, asking a patient or relative if they understand it.

A list of common scientific terms that can be simplified – e.g. efficacy of X = how well X works.

4. How plain English summaries are assessed 4.1 None of the research organisations included in this review produces guidance

that specifically relates to assessing the quality of a plain English summary. Nor have we been able to find any published standards for plain English summaries.

4.2 We tested the use of the MS-Word tools for assessing reading levels by

applying the tool to a research summary provided by Current Controlled Trials before and after it had been rewritten in plain English. After rewriting the score had changed by 10 points, but this had no significant impact on the reading level. Both copies fell into the ‘difficult’ bracket scoring between 30 and 49. It is not clear what score would be appropriate for a plain English summary that was suitable for a lay reviewer. Further research would be required to assess whether the tool could be usefully applied in this context.

4.3 Many organisations provide guidance to their lay reviewers on assessing the

quality of a research funding application as a whole. The plain English summary is then one of the pieces of information used to support the funding review.

8

4.4 Two of the NIHR Programmes (PGfAR and i4i) suggest that lay reviewers consider the following issues when reading the plain English summary:

Was the language used appropriate and clear?

Were scientific terms, abbreviations and jargon explained?

However, lay reviewers are not asked to provide feedback on the plain English summary in their written review. Guidance for public contributors reviewing NETSCC research applications asks them as part of the patient and public involvement questions to reflect on whether they feel the plain English abstract gives a clear account of the research project. Whilst they are not specifically asked to provide written feedback on the plain English summary they are asked to feedback on any weaknesses of the proposal and any other thoughts or reflections they wish to share with the applicants.

4.5 Two of the medical research charities, Parkinson’s UK and the MS Society,

include questions about the plain English summary in the feedback form completed by lay reviewers. Parkinson’s UK simply asks the question ‘Did you find the lay summary useful?’ Lay reviewers simply answer, ‘yes’, ‘no’ or ‘don’t know’. The MS Society asks lay reviewers to comment on whether the plain English summary was easy to understand and well-written. If reviewers find any parts of the summary difficult, they are asked to identify these sections and to explain why they are difficult to understand. This feedback is given to the researchers with the aim of improving any future applications.

4.6 NRES advises researchers that they will receive feedback on their plain

English summaries from the REC. Their guidance states:

The REC may make suggest changes to the summary to make it more informative to patients but this is separate to ethical opinion and may be regarded as non-binding advice from the committee. The content of the summary does not determine the committee’s opinion.

4.7 A small number of organisations advise researchers that producing a high

quality plain English summary is a requirement for funding. For example, Parkinson’s UK states:

Unsuitable plain English summaries will need to be rewritten or the application may be rejected.

4.8 Some work has been done to look at ways of improving the quality of plain

English summaries that researchers produce i.e. to improve the skills of the researchers3,4. For example the Muscular Dystrophy Campaign involved fifteen early-stage researchers in writing plain English summaries of clinical trials, and provided instructions, a template, examples of well-written summaries, a glossary of terms – but still found the summaries produced to be of variable quality and requiring review by their internal editorial team4. The organisations that are renowned for producing high quality plain English summaries of research, e.g. CancerHelp UK, employ a dedicated team of

9

specialist writers, with defined skills and training in writing for a lay audience.

CancerHelp UK concludes that ‘the skill of plain English writing is universally undervalued and the difficulty in getting it right is underestimated.’4

5. Discussion 5.1 In this section we consider the implications for the NIHR Programmes based

on this review, and describe the decisions taken by the Advisory Group following discussion of these issues.

A template for a plain English summary to support the lay review of NIHR funded research

5.2 Based on the work of a number of medical research charities, there are benefits to clearly defining the structure of the plain English summary for the purpose of funding review. This ensures that researchers include the information that patient/ carer reviewers are looking for and helps researchers know what to cover. The average plain English summary for this purpose seems to be about 1000-1500 words. However, this is in contrast to the NIHR Research Programme summaries, where the maximum length is 450 words.

5.3 The key question is then what topic headings should be included in the

suggested structure. Based on the evidence from this review, the Advisory Group agreed that the topics that were most relevant to the task of a lay reviewer in the context of NIHR Programmes included:

what the aims are and what research question is being asked

whether the research question is relevant and important to patients/ carers/ members of the public

whether the research will genuinely benefit patients/ carers/ members of the public

whether the research will bring new knowledge and how it builds on previous work

whether the research project has been designed with the needs of participants in mind

how much the project will cost and why.

5.4 Some organisations include information in the plain English summary about

the relevant experience of the applicant/ research team. We are aware from previous work with medical charities that for lay reviewers this isn’t about checking the researchers’ professional background and relevant skills, but more about checking whether the researchers have experience of working with the specified patient group and is aware of, and responsive to, their interests and needs. The Stroke Association asks for this information in a written format. Other organisations e.g. The Alzheimer’s Society make this

10

assessment during interviews with researchers. The Advisory Group agreed that given the variety of topics covered by NIHR research, this question should not be included in the guidance for researchers.

5.5 We note that there is blurring of roles around the possible overlapping

contributions of PPI in early project development and the input of lay reviewers. In the context of the medical research charities, this is often one and the same thing. The lay reviewers effectively become the PPI in project design. They are able to carry out an assessment of the relevance of the research question and assess the practical design of the research, because they all have direct experience of the condition being investigated. Published evidence suggests that direct experience of the condition is essential for this role. For example, detailed knowledge of a condition is essential to be able to correct researchers’ assumptions or to judge the acceptability of the proposed approach5,6,7 (See also Box 1).

Box 1: Two examples of the contribution that patients with direct experience of a health condition can make to the process of grant review

Case 1: Arthritis Research UK has a USER committee that reviews funding applications. The USER committee is made up of clinicians and people affected by musculoskeletal disease. It considers the importance of the research and questions the assumptions of researchers. When the committee was considering an application, a service user reviewer was the only person to spot that the researcher had assumed that people with arthritis would only have one artificial joint – when in fact many people have more than one. This assumption meant the suggested methods would have given unclear results. Case 2: The MS Society involves people affected by MS in grant review. One application proposed to evaluate NHS exercise classes for people at the early stages of MS. The lay reviewers commented that most people with early MS are still able to go to a local gym and therefore such classes would be of little relevance. They noted that exercise became much more difficult for people at the later stages, when their disability may limit their access to exercise facilities and they are uncertain of what kind of exercise would be most beneficial. The patients therefore concluded that the proposed project was of little importance to them, and it was turned down for funding.

5.6 In the context of applications to the NIHR Programmes, the lay reviewers / lay

board members may not always have direct experience of the condition being investigated (in the case of applications for studies of specific conditions). This means they may not always be in a position to make judgements such as whether the research is of relevance and importance to patients/ carers. Instead they may want to know what other patients/ carers think and to be reassured that the researchers have carried out good quality PPI to address these questions themselves. The Advisory Group therefore agreed that the template for the plain English summary for NIHR Programmes should ask questions about whether researchers have consulted patients/ carers with direct experience of the condition being studied, about the importance/

11

relevance of the research and the practical and conceptual elements of the research design.

A template for a published summary of the research suitable for a general audience 5.7 The information needs of NIHR lay reviewers (who often have a personal

interest in research if not some kind of research background, or who receive training and support and learn from experience in their role) are likely to be very different from a general audience. Based on the evidence from this review it seems that plain English summaries produced for the general public (as part of the aim of increasing transparency and public accountability) tend to be shorter (250-300 words) and may need to be in even simpler language. The aims seems to be to enable the reader to quickly grasp what the project is about, to know if they then want to find out more information, which may then be available elsewhere. We therefore concluded that the plain English summaries produced for lay reviewers are not ideal for publication for a general audience. They may need to be shortened and edited for this purpose.

5.8 Whilst recognising this, the Advisory Group also noted the resource

implications for producing separate plain English summaries aimed at the general public. They therefore agreed that the summaries included in NIHR funding application forms should continue to be used for this purpose.

Guidance for researchers on producing a plain English summary 5.9 The guidance on writing in plain English is standard. The Advisory Group

therefore agreed that brief guidance would be included within the NIHR guidance for researchers, and that researchers should be signposted to the INVOLVE website which will include a new page on writing plain English summaries. It was agreed that the INVOLVE page would include the guidance, as well as examples of plain English summaries and links to other relevant sites.

5.10 The Advisory Group noted that it would also be important for the NIHR

guidance to highlight the importance of the plain English summary. Members therefore agreed that it would be valuable to include a line or two that explains that the plain English summary is often used by a wide range of non-expert reviewers and therefore is vital to ensuring reviewers understand the project and can undertake a fair review. A poor summary may lead to a poor review.

5.11 The Advisory Group also agreed that the guidance should also suggest that

the researchers seek feedback on a draft plain English summary from patients/ carers/ members of the public to check that it is clearly written. It was suggested that this guidance should be explicit about how to go about this, as researchers often comment that this practical advice is missing. It is not a straightforward process because patients/ carers/ members of the public will have very different opinions about what is accessible, what terms are acceptable to use etc. Also not everyone is skilled in writing in plain English. A

12

recent report on the experiences of DenDRON highlighted this exact problem when a researcher consulted twelve different members of a research panel on a patient information sheet and got twelve conflicting opinions back8. We suggest that researchers are advised to seek feedback from 3-4 patients/ carers/ members of the public and are prepared to respond accordingly. They will need to be prepared for differences of opinion and be ready to make a judgement call as to which editorial changes to make.

Making the production of a high quality plain English summary a condition of funding 5.12 The evidence from recent work on plain English summaries has shown that

this task is very difficult for many researchers9. The guidance on writing in plain English is almost the exact opposite to the style of writing for an academic audience. Most researchers do not receive training in this area during any stage of their career, nor is training made available through the research councils etc. More importantly, writing in plain English is a skill like any other – some people will be better at it than others. So even with training, support and guidance there will still be some researchers who will not be able to produce a high quality plain English summary. Their strengths may lie elsewhere.

5.13 The Advisory Group agreed that it would be important to give feedback to

applicants about the quality of their plain English summary, and to ask stakeholders for their views about the possibility of withholding funding as part of the consultation (see Chapter 2).

Judging the quality of a plain English summary 5.14 What constitutes a ‘good quality’ plain English summary? We suggest that

ultimately this involves assessing whether it is fit for purpose and meets the needs of the intended audience. In the context of the NIHR programmes a good quality plain English summary is one that provides the information needed to support a funding review and is written in a language and style that is accessible to lay/ non-specialist reviewers. The lay and non-specialist reviewers are therefore best placed to assess whether the summary adequately meets their needs.

5.15 We suggested that lay and/ or non-specialist reviewers are therefore asked to

comment on the plain English summary as part of their review process. If they are unable to complete their review because the plain English summary is of such poor quality then the summary should be returned to the researcher with a detailed explanation of where the problems lie in addition to advice on where to get further help and support with rewriting it. If the summary is good enough to support a review, but could do with improvement, lay reviewers and/ or non-specialist reviewers could provide constructive feedback as part of their review process. This could be given to researchers (both funded and non-funded applications) in order to help them with future applications.

13

CHAPTER 3: Consultation with stakeholders

1. Introduction

1.1 Based on the findings from our review of current practice, we developed a draft template for a plain English summary, and guidance for researchers. These drafts were discussed and amended by the Advisory Group. We then undertook a consultation on these revised draft documents, which are attached here as Appendix 2. This chapter summarises the findings from the consultation.

1.2 The primary aim of the consultation was to check whether the draft template

and guidance were workable, useful and cost-effective. The Advisory Group agreed not to seek people’s views on the guidance for reviewers. Given the uniformity of the guidance on writing in plain English, it was also agreed that we would not consult people about this.

1.3 The consultation comprised:

A workshop at the INVOLVE conference.

One-to-one interviews with lay reviewers, panel members, Programme managers, researchers working with RDSs, NIHR trainees (see Appendix 1).

A group interview with NETSCC Programme managers (Appendix 1). 1.4 The remainder of this chapter is structured as follows: Section 2: Views on plain English summaries

Section 3: How plain English summaries are used Section 4: Improving the quality of plain English summaries Section 5: Comments on the draft guidance Section 6: Additional information to include in the plain English summary Section 7: Length of the plain English summary Section 8: Using the Standard Application Form (SAF) summary to

publicise funded research on Programme websites Section 9: Involving patients/carers/members of the public in developing

plain English summaries Section 10: Making changes to the SAF Section 11: Assessing the plain English summary Section 12: Changes made in response to the consultation

2. Views on plain English summaries

2.1 All the interviewees described the plain English summaries currently written for NIHR application forms as 'very mixed'. Some summaries are excellent and some are very poor.

14

2.2 The poor plain English summaries were described as:

too short

overly-simplistic to the point of being condescending

unintelligible

repetitive - copying other parts of the form

a PR exercise – over-selling the importance of the research. 2.3 The interviewees thought that a good quality plain English summary:

is written clearly in plain English

has a logical structure and flows

is pitched at the right level – written for a well-educated, interested member of the public

provides a detailed case for support

addresses the issues of interest to patients/ the public.

2.4 Some interviewees commented that the plain English summary is not meant to stand alone as it is read with the rest of the application. If the technical parts of the form are well-written, the plain English summary is superfluous. A small number of interviewees questioned whether it was appropriate for a project to focus solely on the plain English summary.

2.5 The i4i Programme reported that the plain English summaries submitted by industry tend to be of better quality, possibly because industry is more used to producing information for the public. However even these summaries could still benefit from improvement.

2.6 A number of interviewees commented that a good quality plain English

summary often correlates with a strong application.

3. How plain English summaries are used

3.1 One of the researchers commented that many scientific reviewers depend on the plain English summary and that this should be stated more strongly in the guidance.

3.2 Some reviewers (lay and scientific) read the plain English summary first, as a

way into an application. It often provides an indication of how seriously researchers have thought about patients' interests and how the research is going to work. The quality of the summary is as much about clarity of thought as about standards of communication. One lay reviewer said he reads the plain English summary last in order to check that his understanding of the technicalities is correct.

3.3 RfPB sends out the plain English summaries to lay reviewers to find out which reviewers would be interested in reviewing the applications.

15

4. Improving the quality of plain English summaries

4.1 None of the interviewees had received any training in writing in plain English. One of the trainees said they would welcome training in this area.

4.2 Programme Grants for Applied Research offers researchers a workshop

between Stage 1 and Stage 2 applications to help with completing their application. As part of the training, a lay panel member gives a presentation about PPI in general and includes information on what makes a good plain English summary. The Panel members have seen an improvement in the quality of the applications and the plain English summary since this training was introduced.

4.3 Some RDSs provide training to researchers on developing an application for

funding from NIHR. This includes advice on producing information for a non-technical audience.

5. Comments on the draft guidance

General comments

5.1 Many of the interviewees and workshop participants welcomed the guidance and thought it would be helpful to funding applicants.

5.2 A number of the interviewees thought the guidance was not really addressing

the main problem – researchers’ poor communication skills. They suggested that the reason there are so many poor plain English summaries is because researchers either don’t know how to write one or do not recognise its value. These interviewees thought that the content of plain English summaries is not so much of a problem and that the headings/ structure would be obvious to most researchers. Therefore they were not convinced that the guidance would make much difference.

5.3 Many of the interviewees thought that researchers find it difficult to write in

plain English without becoming patronising. They suggested that the guidance provided explicit advice on what level to pitch the summary at. The guidance from other funding organisations includes advice to ‘aim to write for a reasonably well-educated audience for example if you were writing for The Economist or a broadsheet newspaper’. We therefore suggested adding this advice to the draft guidance, and recommended that the appropriate level could be illustrated through examples of summaries on the INVOLVE website.

5.4 Some of the Programme managers were concerned that the detailed

guidance would not be relevant to applicants to their Programme. The short headings for the Standard Application Form (SAF) are not a problem, but it was recognised that each Programme will have to tailor the contextual guidance to their own needs.

5.5 There were many different views amongst the interviewees as to how to

structure the plain English summary. Most were slight variations on the draft guidance. Some interviewees contradicted each other. The suggestions

16

seemed to depend on how the interviewee perceived the plain English summary. Some thought it should provide a faithful and accurate summary of the research and should therefore simply contain information about the aims, methods, background and the difference the research will make. This view was held mostly by Chairs and researchers. One researcher described this as a ‘plain English abstract’.

5.6 Others thought that the main aim of the plain English summary is to support a lay review and that it should therefore contain all the information that a lay reviewer requires for their role. This view was mostly held by lay reviewers, lay panel members and Programme managers.

5.7 The current wording on the SAF describes the plain English summary as an ‘Abstract in plain English’. Some of the Programme managers thought that this wording should be changed because researchers interpret the term ‘abstract’ as permission to use scientific language.

5.8 A number of the interviewees were concerned that applicants often ‘cut and paste’ from other sections of the form into the plain English summary. The lay reviewers/ panel members thought that the plain English summary should be distinct in focusing on the issues that are of interest to patients and the public, rather than researchers. They thought that the questions in the draft guidance should be reworded to make this distinction clearer.

Specific comments on sections in the draft guidance

5.9 Short headings: Not all NIHR research includes participants, so suggested that this needs to be clearly made optional. A small number of interviewees suggested changing the short headings to 'who, what, why, where, when'.

5.10 Terms used: Some interviewees commented that the terms ‘project’ or ‘study’

would not work for Programme grants, and that the term 'research' should be used instead.

5.11 Aims and Background: With the question ‘Why is it important?’, some

interviewees suggested adding ‘to patients/ the public’, otherwise researchers tend to answer with a lot of information about previous research which explains why the research is important to researchers – not about how the research might make a difference to people’s lives. However this question is virtually the same as ‘how will it benefit patients?’ and may force researchers into saying the same thing in slightly different ways. We therefore suggested that one of these questions could be removed.

5.12 Background: It was suggested that this should include information about how

many people are affected and the cost to the NHS, to say something about the severity of the issue. It should ask the question ‘why does this research need to be done now?’

17

5.13 Design & Methods: Interviewees suggested advising researchers not to get too technical and concentrate on what patients/ the public want to know about the methods. The main issue for lay reviewers is the rationale for choosing the methods.

5.14 Participants: Some interviewees thought that this section should not stray

into the area of ethical review (as this is not the role of the lay reviewers and duplicates the work of Ethics Committees) and should therefore not ask what will happen to participants. Some of the Programme managers thought a direct question should be asked about ethical issues. Some lay reviewers thought this section should ask ‘How do the researchers know whether the research will be acceptable to participants and how confident are they that they can get enough participants?’

5.15 Cost and duration: Many interviewees thought this section could be removed

as this information is clearly explained elsewhere in the application. 5.16 The importance of a plain English summary: Programme managers were

concerned that the sentence 'A good quality plain English summary is more likely to lead to a favourable review’ could be misinterpreted, because many factors influence the outcome of the review, not just the plain English summary. They suggested editing to say '…will contribute to a more favourable review'. One of the lay reviewers suggested changing this to a ‘fair’ review, rather than a ‘favourable’ one.

5.17 A number of interviewees suggested making the section stronger and more

directive. 6. Additional information to include in the plain English summary

6.1 There were two additional topics that some interviewees (lay reviewers/ panel members and Programme managers) thought should be included – PPI and dissemination.

6.2 Patient and Public Involvement (PPI): Interviewees thought this should be

included because:

PPI should be integral to every stage of the research – so it should have already influenced the research proposal and cannot be separated out from the work programme.

This would reinforce the importance and value given to PPI.

If the summary is going to put on a public website, then the fact that the research has PPI (even if not all the detail is provided) would be of interest to patients and the public.

6.3 Dissemination: Interviewees and participants at the INVOLVE conference

workshop suggested two extra questions should be added, the first about how the findings will be communicated to patients and the second about whether the researchers have thought about how the findings will be implemented.

18

6.4 One of the lay reviewers also requested that more information about clinical trials be included in the plain English summary, in particular to be reassured that the trial design is not biased, that the analysis will be meaningful and significant, that the study is sufficiently powered to generate significant results. However, these questions would not be relevant in the context of other kinds of research.

7. Length of the plain English summary

7.1 Many interviewees thought that the current length of their plain English summary was appropriate and did not think it should be much longer. However, there is a lot of variation across the different Programmes, so different lengths were thought to be appropriate.

7.2 A couple of interviewees commented that sometimes more words are needed

to write a phrase in plain English instead of using a technical term. So the word count may need to be longer.

7.3 Some interviewees (two researchers and the people working with RfPB)

thought that the length should be 300 words. One researcher commented that encouraging researchers to explain their research concisely encourages them to be much clearer about what they want to do and why.

7.4 The majority of interviewees thought that the most appropriate length for the

summary was between 600-700 words, or 3500 characters (average word length is 5 characters – so this is equivalent to 700 words), or ½ to 2/3 of a side of A4 (= approx. 4-500 words).

7.5 In conclusion there is no consensus on word limit. Nor is there any evidence

to support any particular word count. We therefore suggested that this is trialled prior to making a decision. Once the final structure is agreed, it would be worth exploring what would need to be covered in the plain English summaries from different Programmes, to see how many words are required to cover all the new topic headings.

8. Using the Standard Application Form (SAF) summary to publicise

funded research on Programme websites

8.1 Many of the interviewees and workshop participants thought that if the aim is to provide the general public/ interested patients with an overview of funded research, then the plain English summary would need to be different from that in the SAF. The information needs would be different from that of a lay reviewer. They thought that the SAF summary would therefore need to be edited for this purpose and would need to be shorter. This also has implications for assessing the quality of the plain English summary (see below).

19

8.2 If the summaries of funded applications are to be published then the interviewees thought that:

Applicants should be informed of this fact in the guidance, and advised not to include confidential information or information that is commercially sensitive.

The Programmes need to be very clear about the purpose of publishing the summary on the NIHR website, in particular whether the aim is to raise public awareness or assist potential participants in finding relevant research. The summaries would need to be different for these different purposes.

Poor quality summaries should be improved prior to publication.

8.3 All of the NIHR Programmes currently publish plain English summaries of

funded research on their websites. Some interviewees thought this should continue. However, many of those interviewed were not aware that these summaries are already available.

9. Involving patients/carers/members of the public in developing plain

English summaries

9.1 Many interviewees thought this was valuable to do. They suggested that the aim of this involvement is to check that the ideas have been expressed clearly and that the information is accessible.

9.2 None thought that the patients/ members of the public should be asked to

write the plain English summary, as they may not have the skills to do this. 9.3 It was thought that the practical guidance to researchers should suggest that

a small number of patients/ members of the public are asked to comment on a draft summary. The main practical issue would then be finding people to do this. Ideally, researchers would have been working with a relevant group of patients/ members of the public during the early development of their research proposal. These people could then be asked to check the plain English summary. Otherwise the guidance could suggest:

Contacting a user panel at a local university or hospital trust, whose members are used to reviewing patient information.

Working with the local RDS who may provide funding to support early involvement or may have a panel of lay members to consult.

10. Making changes to the SAF form

10.1 There were different views about the ease of making changes to the current

plain English abstract question on SAF, however the Programmes are able to change the contextual guidance, which means they would be able to make their Programme-specific changes quite easily.

20

11. Assessing the plain English summary

11.1 The interviewees had different ideas about the aims of assessing the quality of a plain English summary. They identified these aims as:

assessing the impact of implementing new guidance

improving the quality of plain English summaries prior to funding review

providing feedback to researchers to help them improve their writing skills

obtaining a high-quality plain English summary for publication on a website.

11.2 They therefore had different ideas about who should assess the summary and

when they should do this, as well as what guidance any reviewer might need. These different options will be discussed in turn.

Aim 1: To assess the impact of implementing the new guidance

11.3 One interviewee thought the aim would be to assess whether the new guidance had made any difference to the quality of the plain English summaries. She suggested asking the Chairs of the various panels to seek feedback from their committees as to whether they had noticed any improvements subsequent to implementing the guidance.

Aim 2: To improve the quality of plain English summaries prior to funding

review

11.4 The interviewees thought this could only be done by introducing an extra step in the review process, either asking staff or a panel of lay reviewers to check the plain English summaries prior to further review. However, most thought this would be resource-intensive given the large number of applications received. A few questioned why the focus would be on the plain English summary, when many other parts of the application form may need improving.

11.5 Some Programmes have a scrutiny stage where applications are assessed for

their suitability to the Programme. Some lay reviewers/ panel members suggested providing feedback on the plain English summary at this stage. However, the Programme managers commented that applicants are not able to make changes at this stage.

11.6 Some Programmes have two stages to their application process and could

introduce feedback on the plain English summary in between the outline and full-proposal stage. This could help to increase the quality of the plain English summary prior to final review.

11.7 It was suggested that experienced lay reviewers could provide feedback at

this stage as they would be able to spot if there were gaps in the information. Other non-specialist reviewers could also be asked to comment as they may raise different issues.

21

Aim 3: To provide feedback to researchers to help improve their writing skills

11.8 Many interviewees suggested that lay reviewers and/ or lay panel members be asked to comment on the plain English summary as part of the review process and that these comments could be included in the feedback to successful and unsuccessful applicants. This would help researchers develop their writing skills and may improve the quality of plain English summaries over time, but it would be too late in the process to support the lay review.

11.9 It was suggested that an additional section be included in the reviewer

feedback form asking them to comment on whether the plain English summary was clear, whether parts needed improving, whether any information was missing and whether it met the needs of reviewers. Some Programmes already suggest that reviewers consider these questions.

11.10 There was some concern that this might overburden reviewers and this would

need to be discussed with them. 11.11 The researchers said they would find this helpful and would welcome this

feedback. Aim 4: To obtain a high-quality plain English summary for publication on a

website etc

11.12 The interviewees thought that a plain English summary for publication would need to be:

short and clearly written

technically accurate

up-to-date i.e. including any changes requested as part of the funding decision or following ethical approval.

11.13 It was suggested that researchers who are funded should be asked to

produce such a summary as a condition of funding. The revised summary would need to be assessed again to check that the standard had improved or changes incorporated, as appropriate. Some suggested asking the lay review panel member(s) who lead on the original review to comment on a revised version. Others thought a member of the public should review these summaries as they may not have as much background knowledge as patients. The researcher would also have to agree a final version to ensure technical accuracy, and to ensure that the information was current and that there were no confidentiality issues. Some thought that NIHR staff would need to be overseeing these negotiations and have final editorial say.

11.14 Many interviewees were concerned that the process of producing a plain

English summary for publication should not hold up any research, and that good proposals should not be rejected solely because of a poor plain English summary. It was suggested that the development and final approval of a plain English summary could become part of the contractual negotiations for funded projects – as long as no further delay occurred. Some commented that this

22

would be an effective incentive for researchers to produce a good quality summary. However, not all interviewees agreed that the quality of the plain English summary was of sufficient importance to withhold funding.

11.15 Some Programme managers were concerned about what would happen if a

researcher (because of lack of skills or motivation) did not produce an acceptable summary. They did not think this should stop the research going ahead and were not clear who else could then rewrite them. Lay reviewers/ panel members or staff may not have the skills themselves and/ or require training. One Programme manager thought the ideal scenario would be to have a bank of freelancers who could take on this role. However, she did not believe this would be possible to resource.

11.16 The Advisory Group agreed that they key aims for reviewing plain English

summaries were aims 3 and 4 above, i.e.

to provide feedback to researchers to help them improve their writing skills

to obtain a high-quality plain English summary for publication on a website.

12. Changes in response to the consultation

12.1 We produced revised draft guidance for researchers, based on the findings from the consultation. The Advisory Group considered these revisions and made further amendments.

12.2 Guidance for reviewers was then developed to complement the guidance for

researchers. TwoCan Associates December 2012 The final draft guidance agreed by INVOLVE and the Advisory Group and submitted to the Department of Health is available in the following document: INVOLVE (2013) Improving the quality of plain English summaries for National Institute for Health Research (NIHR) funded research.

23

Appendix 1: Thanks

Thanks to:

Members of the Advisory Group

Sarah Buckland, INVOLVE (Chair)

Mike Clark, NIHR School for Social Care Research (SSCR)

Jean Cooper-Moran, NIHR Central Commissioning Facility (CCF)

Alison Ford, NIHR Evaluation, Trials and Studies Coordinating Centre (NETSCC)

Gill Green, Research Design Services (RDS)

Helen Hayes, INVOLVE

Mark Petticrew, INVOLVE

Lesley Roberts, INVOLVE

Peter Thompson, NIHR Trainee Coordinating Centre (TCC)

People who responded to an email request for information about plain English summaries:

Emily Hughes, Parkinson's UK

Sarv Kaur, MS Society

Martin Lodemore, Diabetes Research Network

Matt Murray, Alzheimer's Society

Katherine Reekie, Arthritis Research UK

Leanne Reynolds, Asthma UK

People who took part in an interview with TwoCan Associates:

Garry Barton, Deputy Director of the RDS for East of England

Philip Bell, lay member, i4i Board

Isabel Boyer, lay member, Health Technology Assessment (HTA) Clinical Evaluation

and Trials Board

David Britt, lay reviewer, CCF

Charlotte England, Senior Programme Manager, i4i

Ray Fitzpatrick, Director and Chair of the Health Services and Delivery Research

(HS&DR) Commissioning Board

Steve Goodacre, Deputy Chair of the HTA Clinical Evaluation and Trials board

Adrian Grant, Director, Programme Grants for Applied Research (PGfAR)

Paul Jarvis, Lay reviewer, NETSCC

24

Louise Marsland, East of England RDS

Carolyn Morris, lay member Research for Patient Benefit (RfPB) Board

Ksenia Orekhov, Senior Programme Manager, PGfAR

Jonathan Scales, East of England RDS

David Symes, lay reviewer, NETSCC

Ciorsdan Taylor, Senior Programme Manager, RfPB

People who took part in an interview with Helen Hayes

Amanda Daley, NIHR Trainee

Thomas Jaki, NIHR Trainee

People who took part in a discussion led by Alison Ford at a NETSCC PPI programme managers meeting:

Vicky Bowness, Programme Manager (PPI), NETSCC

Tansy Evans, PPI lead for HS&DR

Claire Kidgell, PPI lead for Public Health Research programme

Jules Newman, PPI lead for EME programme

Sue Pargeter, HS&DR representative

Ann Saunders, HS&DR representative

Sabrina Touzel, HTA representative

Gaynor Young, PPI lead for HTA programme

Thanks also to people who took part in the workshop at the INVOLVE conference, and to Thomas Kabir and Kate Fuller from the Mental Health Research Network for presenting their work on plain English summaries at this workshop.

25

Appendix 2: Sources of guidance on writing in plain English included in the review of current practice

Asthma UK Cancer Research UK International Standard Randomised Controlled Trial Number Register JISC Patients participate! The Plain English Campaign Royal National Institute for the Blind (RNIB) Skills for Care West Midlands - An Introductory Briefing on the Production of Accessible Information. Stroke Association TwoCan Associates

26

Appendix 3: Draft Plain English summary section for completion within the NIHR standard application form for research funding. (Draft developed for consultation – this was subsequently revised following the consultation)

We suggest that the short guidance about completing the ‘plain English summary’ section of the application should say: This should cover:

aim of the research

background to the research

design and methods used

participants

project cost and duration. More detailed guidance should say: The importance of a plain English summary A plain English summary is a clear explanation of your research. It is used by non-specialist reviewers to inform their review of your funding application. Non-specialist reviewers include patients, carers, members of the public and clinicians and researchers who do not have specialist knowledge of your field. A good quality plain English summary is more likely to lead to a favourable review. It is good practice to involve patients/ carers/ members of the public in developing the summary. What to include in your plain English summary When completing the different sections of the plain English summary it may be helpful to consider the following questions: Aim of the research

What are you aiming to find out?

What is your research question?

Why is this important?

How will patients/ carers/ members of the public and the NHS benefit from your research - either directly or in the longer term?

Background to the research

What is known already?

What will this study add? Design and methods used

What design and methods have you chosen and who are the potential participants (if relevant)?

27

Participants

Outline what taking part in the project will involve for the participants.

How will the research team take into account the interests and needs of participants?

Project cost and duration

What is the total budget for your project?

How long will the project last? The guidance will also include a short paragraph on how to write in plain English and a link to further information on a new INVOLVE webpage which will provide further guidance.

28

Appendix 4: Interview schedules

Interview schedule: lay reviewers/committee members

Remind re aims of project (as email)

Reminder we’ll take notes but won’t use their name in any reports we write.

I think this interview will take about 30 minutes (to review after first interview).

Is that still OK timewise for you?

Is there anything you’d like to ask me before we start?

Introduction and warm up 1. Can you tell me what you think of the plain English summaries that you see in the

application forms you read?

2. How do you use the plain English summary when you’re reviewing an

application?

3. What do you think makes a good plain English summary?

Draft guidance 4. Did you have a chance to look at the draft guidance I sent you? The NIHR

programmes cover a really wide range of research – from small projects to very

large programmes of work and this guidance would need to work for all of them.

What did you think about the suggested headings in the draft guidance?

5. Is there anything else you think we should be asking researchers to cover in the

plain English summary?

6. Do you think this summary could be used on its own – for example, posted on the

funders’ website – to raise public awareness of the research taking place?

7. Do you think researchers should include a summary of any PPI that has

happened or is planned within the summary?

8. Do you think that researchers should involve patients/carers/members of the

public when writing plain English summaries?

9. If so, what practical advice should they be given about how to do this?

10. What do you think of the current length of the lay summary? (CCF 350-450,

NETSCC 500-800, Trainees 100 words) Do you think it needs to be any longer to

cover the suggested content?

Assessing the summary 11. Who do you think is best placed to assess the quality of a plain English

summary?

12. Why do you say that?

13. What advice would you give to someone who was asked to review the plain

English summary?

14. What should be done about plain English summaries that aren’t clearly written or

do not include the relevant information? (e.g. should funding be withheld until

they are improved?)

29

Close 15. Do you have anything else you’d like to say about the draft guidance, or about

how plain English summaries should be assessed?

Interview schedule: programme managers

Introduction and warm up 1. Can you tell me what you think of the plain English summaries that you see in

the application forms you read?


Draft guidance 3. Did you have a chance to look at the draft guidance I sent you? What did you

think about the suggested headings in the draft guidance – would they be useful

for the programme you work in?



5. Do you think this summary could be used on its own – for example, posted on

the funders’ website – to raise public awareness of the research taking place?




7. How easy is it to make changes to the standard application form and what are

the practical implications for example cost, time and technical considerations?


summary?




they are improved?)

11. Do you have any practical suggestions as to how the summaries could by

assessed by the programmes or can you identify any practical challenges to

assessing the summaries?

Using the summary 12. If you don’t already display the summaries on your website are there practical

issues for making the summaries more widely available on the website?

13. Would it need to be edited or checked? If so, who should do this?



30

Interview schedule: chairs/vice chairs

Introduction and warm up 1. Can you tell me what you think of the plain English summaries that you see in

your committee?



think about the suggested headings in the draft guidance?

4. Would summaries containing this information be useful in your committee?



6. Do you think this summary could be used on its own – for example, posted on

the funders’ website – to raise public awareness of the research taking place?

7. Do you think researchers should include a description of any PPI that has

happened or is planned within the summary?





summary?




they are improved?)

12. Do you have any practical suggestions as to how the summaries could by

assessed by the programmes or can you identify any practical challenges to

assessing the summaries?

Using the summary 13. Could the plain English summaries included in the application form be used on

your programme’s website to give information about the research you fund?

14. Would it need to be edited or checked? If so, who should do this?



31

Interview schedule: researchers/NIHR trainees

Introduction and warm up 1. What has been your experience of writing plain English summaries as part of the

NIHR standard application form? Is it an easy task?

2. Have you had any training or support in this area?

3. If yes, what was it and was it helpful? If no, would this be helpful? What kind of

training or support would you find helpful?


think about the suggested headings in the draft guidance? Would they help you

write a plain English summary?



6. Do you think this summary could be used on its own – for example, posted on the

funders’ website – to raise public awareness of the research taking place?




8. Would you find it helpful to be able to access practical advice on how to involve

patients/ carers/ members of the public in producing summaries?

Assessing the summary 9. Would you find it helpful to receive comments about the plain English summary

as part of the feedback you get from NIHR funding committees?




they are improved?)



32

References 1. Current Controlled Trials. Research Summaries. July 2012. 2. Plain English Campaign. How to write medical information in plain English. 2001

www.plainenglish.co.uk/files/medicalguide.pdf 3. Current Controlled Trials. Lay summaries for clinical trials: CCT’s experience

March 2011 to October 2011. October 2011. 4. JISC Patients Participate. Case Studies. 2011

blogs.ukoln.ac.uk/patientsparticipate/files/2011/10/Case-study-report-Final.pdf 5. TwoCan Route map. (www.twocanassociates.co.uk/routemap). 6. Boote J, Baird W, Beer D. Public involvement at the design stage of primary

health research: A narrative review of case examples. Health Policy 2009; 95:10-23.

7. Staley K. Exploring Impact: Public Involvement in NHS, Public Health and Social

Care Research. Eastleigh: INVOLVE 2009. 8. Iliffe, S, McGrath T, Mitchell D. The impact of patient and public involvement in

the work of the Dementias & Neurodegenerative Diseases Research Network (DeNDRoN): case studies. Health Expectations 2011 (Epub ahead of print).

9. Duke, M. How to write a Lay Summary’. DCC How to Guides. Edinburgh: Digital

Curation Centre. 10. Staley K, Kabir T, Szmukler G. Service users as collaborators in mental health

research: less stick, more carrot. Psychological Medicine, FirstView Article, August 2012, pp 1 - 5.

http://www.plainenglish.co.uk/files/medicalguide.pdf

http://blogs.ukoln.ac.uk/patientsparticipate/files/2011/10/Case-study-report-Final.pdf

http://www.ncbi.nlm.nih.gov/pubmed?term=Iliffe%20S%5BAuthor%5D&cauthor=true&cauthor_uid=21902772

http://www.ncbi.nlm.nih.gov/pubmed?term=McGrath%20T%5BAuthor%5D&cauthor=true&cauthor_uid=21902772

http://www.ncbi.nlm.nih.gov/pubmed?term=Mitchell%20D%5BAuthor%5D&cauthor=true&cauthor_uid=21902772

33

Table 1: NIHR Programmes’ requirements of the plain English summary

Central Commissioning Facility (CCF) – Managed Programmes

NIHR Evaluations Trials and Studies Coordinating Centre (NETSCC – Managed

Programmes

NIHR Trainees

Programme grant for Applied Research – Stage 1

Programme grant for Applied Research – Stage 2

Research for Patient Benefit

Invention for Innovation

Health Technology Assessment

Public Health Research*

Health Services and Delivery*

Efficacy and Mechanism Evaluation

States purpose is to support lay review

√ √ √ √ √ √ √ √

Publishes plain English abstract

√ √ √ √ √ √ √ √

Does not define detailed content of summary

√ √ √ √ √

Uses list of topics to define content of summary

√ √ √ √

Suggested length

450 words 450 words 300 words

300 words

6500 characters

6500 characters

6500 characters

3500 characters

100 words

One or two paragraphs of guidance to researchers

√ √ √ √ √ √ √ √ √

Links to Plain English Campaign guide

√ √ √ √

*The guidance is the same for Researcher –led and commissioned research calls.

34

Table 2: Medical research charities’ requirements of the plain English summary

Park

inso

ns

’

UK

Alz

heim

er’

s

So

cie

ty

Str

oke

Asso

cia

tio

n

MS

So

cie

ty

Art

hri

tis

Rese

arc

h U

K

Asth

ma U

K

Lay application form √ √

Lay summary √ √ √ √

Suggested length 1000 words

2-4 pages

1000 words

1500 words1

? 1200 words

Topics covered Plain English title √ √ Aim of research √ √ √ √ √ √ Importance of research √ √ √

How fits with Charity aims/ priorities

√

How builds on previous research

√ √ √

What is already known about the topic

√ √

How current knowledge may be advanced

√ √

Timescale to impact √ How is this project different to others

√

Methods used √ √ √

Have experts been consulted e.g. statistician and what advice given

√

What taking part involves

√ √

What information collected and how used

√ √

What happens to this info at the end

√ √

Travel expenses reimbursed?

√ √

How will you ensure no-one is out of pocket

√

How participants kept informed

√ √

1 The word length was not specified in the documents provided, but an example plain English

summary contained 1500 words.

35

Park

inso

ns

’

UK

Alz

heim

er’

s

So

cie

ty

Str

oke

Asso

cia

tio

n

MS

So

cie

ty

Art

hri

tis

Rese

arc

h U

K

Asth

ma U

K

How will results be communicated

√ √ √ √

How research will lead to advances in understanding

√

How will it benefit patients/ carers

√ √ √ √ √ √

How will contribute to research in other areas

√

Outline budget and projected spend

√ √

Applicants’ experience √

How long is the project √

Details of PPI in the project development

√

Guidance to researchers

No guidance given

One paragraph √ √

Link to Plain English Campaign Guidance

√ √

Provides written guidance on writing in plain English

√

improving the quality of plain english summaries for … of current practice and consultation with...

Documents