sop 28: standard operating procedure for statistics hap-dreialon iechyd gorllewin cymru – yn...

Canolfan Hap-dreialon Iechyd Gorllewin Cymru – yn Abertawe

West Wales Organisation for Rigorous Trials in Health (WWORTH) – the Clinical Trials Unit in Swansea Prifysgol Abertawe Coleg Meddygaeth Swansea University College of Medicine Athrofa Gwyddor Bywyd 2, Parc Singleton Institute of Life Science 2, Singleton Park Abertawe SA2 8PP Swansea SA2 8PP Ffon (01792) 606545 Ebost [email protected] Phone (01792) 606545 Email [email protected]

____________________________________________________________________________

WWORTH-SOP28StatisticsV2.1-140507 of 13 Not guaranteed if printed

SOP 28: Standard Operating Procedure for Statistics

Authorship Team: Sinead Brophy, Alan Watkins for Joint SOP Group on Trial Techniques (viz Helen Snooks, Ceri J. Phillips, Ian Russell, Frances Rapport, Thompson, Steve Allen, Mel Storey, Kerina Jones, David Ford, Alison Porter)

Approved by WWORTH JMG (Ian Russell in chair)

Signature Date 07 May 2014

0 Version Record

Version Number Effective Date

Reason for Change

0 31 Jan 2009 NWORTH SOP

0.1 26 May 2009 First draft for JSOPG group

0.2 16 Jun 2009 Amendments following review meeting

0.3 02 Jul 2009 Amendments following reviewers comments

1.0 31 Jul 2009 Approved in principle

1.1 25 May 2010 Minor formatting amendments

1.2 04 Jul 2011 Egs in trials in practice SAFER1, COGNATE, AW trial of choice. Prepare SOP for JSOPG by MS.

1.3 16 Mar 2012 Amendments following the JSOPG 15.7.2011

1.4 20 Mar 2012 Logo inserted

1.5 21 May 2012 Minor amendments following JSOPG

2.0 12 Jun 2012 Approved for use by JMG

2.1 07 May 2014 Remove training log post JSOPG discussion - MS & CS

mailto:[email protected]


Canolfan Hap-dreialon Iechyd Gorllewin Cymru – yn Abertawe West Wales Organisation for Rigorous Trials in Health (WWORTH) – the Clinical Trials Unit in Swansea

Prifysgol Abertawe Coleg Meddygaeth Swansea University College of Medicine Athrofa Gwyddor Bywyd 2, Parc Singleton Institute of Life Science 2, Singleton Park Abertawe SA2 8PP Swansea SA2 8PP Ffon (01792) 606545 Ebost [email protected] Phone (01792) 606545 Email [email protected]

____________________________________________________________________________


1 Table of Contents

0 Version Record ............................................................................................................ 1

1 Table of Contents ........................................................................................................ 2

2 Glossary ....................................................................................................................... 3

3 Introduction.................................................................................................................. 3

4 Purpose ........................................................................................................................ 3

5 Roles and Responsibilities ......................................................................................... 3

6 Procedure ..................................................................................................................... 4

6.1 General principles ...................................................................................................... 4 6.1.1 Statistical Analysis Plan ..................................................................................................... 5 6.1.2 Analysis by treatment allocated (formerly known as Intention to treat) ............................. 5 6.1.3 Primary and secondary outcomes ..................................................................................... 5 6.1.4 Power and sample size calculations .................................................................................. 5 6.1.5 Randomisation ................................................................................................................... 6 6.1.6 Blinded analysis ................................................................................................................. 6 6.1.7 Participant withdrawal ........................................................................................................ 6 6.1.8 Missing data ....................................................................................................................... 7 6.1.9 Manipulation of data after export ....................................................................................... 7 6.1.10 Other analyses ..................................................................................................................... 7 6.1.11 CONSORT ........................................................................................................................... 8

6.2 Procedure Flow Chart ................................................................................................ 8 6.3 Design and planning .................................................................................................. 8 6.4 Analysis plan, including interim analysis .................................................................... 9 6.5 Analysis ....................................................................................................................10 6.6 Outcome data reports and statistical reporting ..........................................................10 6.7 Statistical Quality Assurance .....................................................................................11

6.7.1 Statistical Analysis Plan ................................................................................................... 11 6.7.2 External checking of the analysis .................................................................................... 11

6.8 Responsibility for reporting and interpretation ...........................................................11

7 Training Plan ...............................................................................................................11

8 References ..................................................................................................................12

9 Related SOPs ..............................................................................................................13





____________________________________________________________________________


2 Glossary

The full Glossary is in Swansea University H drive/Documents/526-WWORTH/Development Group/Glossary.

3 Introduction

Standard Operating Procedures (SOPs) are succinct formal documents designed to achieve consistency in specified trial functions by specifying standard practice in performing those functions (GCP 1.55 & 5.1.1 – EMeA, 2002) [1]. While SOPs should cite relevant legislation & regulations, and key references & evidence, they need not expound theory.

WWORTH SOPs should accord with all relevant regulations, including the European Union Clinical Trial Directive1, ICH Good Clinical Practice (GCP)2

and the current NHS Research Governance Framework3. They will seek to distinguish between regulations for Clinical Trials of a Medicinal Products (CTIMPs) and for other research.

4 Purpose

The purpose of this SOP is to describe the processes involved in, and the people responsible for, all statistical aspects of trials or other rigorous quantitative studies run, adopted, approved or advised by WWORTH.

5 Roles and Responsibilities

The following people are responsible for using and implementing this SOP:

Chief Investigator (CI) (required) Responsible for the trial but may delegate responsibility for conduct of statistical aspects of the trial (including power and sample size calculations, design, and analysis) to the trial statistician.

WWORTH Senior Statistician (optional) The senior statistician is responsible for supplying statistical support information and advice on best practice to all members of the statistical team and the CI, the mathematical integrity of the algorithms and quality assurance of systems written for randomisation. The Senior Statistician takes general responsibility and has an overview for all WWORTH adopted trials.

Trial Statistician (required) The trial statistician has responsibility for all statistical aspects of the trial. This includes working with and reporting to the CI, TDG, TMG, DMEC and TSC throughout the trial. The trial statistician may, with the approval of the TDG or





____________________________________________________________________________


TMG, delegate some or all of these tasks to an assistant statistician, but retains full responsibility. The trial statistician has responsibility to seek advice from the senior statistician or other appropriate sources to identify best practice. The trial statistician is responsible for statistical aspects of the design, planning, monitoring, analysis and reporting of the trial, including the development of power calculations, statistical analysis plan and statistical reports. The trial statistician is also responsible for directing the work of the trial data manager (TDM) in the preparation of datasets. Whenever feasible the trial statistician should be independent of TDM. In relation to DMEC meetings and members, the trial statistician has particular responsibility for revision and ultimate approval of the statistical analysis plan, monitoring and auditing the conduct of the trial, and, with the TDM, for un-blinding the DMEC statistician.

Assistant Statistician (Optional) Responsible for the conduct of all delegated tasks, as specified by the trial statistician.

Trial data manager (Required) Responsible for collection of data in the agreed manner, for the reliability and completeness of such data and for preparing, under the direction of the trial statistician, suitable data sets for analysis. This may include data cleaning and validation, blinding datasets to allocation, maintaining the codes that link blinded data sets to the main trial databases, exporting data to statistical software and variable labelling. Whenever feasible the TDM should be independent of the Trial Statistician.

The TDM may, with the approval of the TDG or TMG, delegate part or all of these tasks to an assistant, but retains full responsibility.

WWORTH statistical team This team comprises all statisticians employed within WWORTH, and provides mutual support and advice on any project as appropriate.

6 Procedure

6.1 General principles

General principles of good practice as outlined by the Royal Statistical Society (RSS) Code of Conduct4 apply to all qualified Statisticians in WWORTH. WWORTH requires minimum standards to be set. Particularly relevant is the following paragraph, taken from the RSS Code of Conduct.

Fellows should not allow any misleading summary of data to be issued in their name. In particular, a statistical analysis may need to be





____________________________________________________________________________


amplified by a description of the way the data were selected, and the way any apparently erroneous data were corrected or rejected. Explicit statements will generally be needed about the assumptions made when selecting a method of analysis. Views or opinions based on general knowledge or belief should be clearly distinguished from views or opinions derived from the statistical analyses being reported.

Unless approved trial-specific protocols state otherwise, WWORTH projects will be guided by the following statistical principles:

6.1.1 Statistical Analysis Plan

The statistical analysis plan is a comprehensive and detailed description of the methods of data analysis proposed for a trial, in order to avoid post hoc decisions that may affect the interpretation of the statistical analysis. The statistical analysis plan will be written and approved before data analysis commences, and before any un-blinding of the data (WWORTH SOP13 Protocol Development and WWORTH SOP25 Outcome Measures).

6.1.2 Analysis by treatment allocated (formerly known as Intention to treat)

The primary analysis will be by treatment allocated (‘intention to treat)’. Analyses ‘per protocol’ (i.e. limited to participants who comply with the protocol) or by ‘treatment received’ are inherently biased, but may be included among secondary or sensitivity analyses if appropriate research questions have been defined in the protocol.

As far as possible, data should be collected on all participants recruited to the trial as defined in the protocol (WWORTH SOP13 Protocol Development). This should include participants who have been treated in a way inconsistent with the trial protocol, for example by receiving an unallocated treatment.

6.1.3 Primary and secondary outcomes

Primary and secondary outcomes consistent with the trial objectives should be agreed with the TDG and clearly stated in the protocol and the analysis plan (WWORTH SOP13 Protocol Development and WWORTH SOP25 Outcome Measures.

6.1.4 Power and sample size calculations

Trials should be powered on the analysis of primary outcomes as defined in the protocol or the analysis plan. Expected drop out rates should be explicitly incorporated. Two sided tests and confidence intervals will be used; any exceptions should be fully justified in the protocol. Trials are not usually powered for subgroup analysis. Therefore these should be avoided





____________________________________________________________________________


unless approved in trial protocol or analysis plan, eg to generate hypotheses for next study. Instead group membership should be incorporated as a factor in statistical models.

6.1.5 Randomisation

Randomisation procedures are outlined in WWORTH SOP24 Randomisation.

6.1.6 Blinded analysis

Each trial will maintain the highest level of blinding compatible with the study design. The trial statistician and others conducting the analysis should remain blind to the identity of allocation groups for as long as feasible, ideally until the first formal presentation of the analysis. Data preparation and analysis should be conducted so as to permit this.

Unblinding procedures, including those for early unblinding if requested by the DMEC, should be summarised in the research protocol and given in detail in the analysis plan. The TDM can unblind results for the DMEC where necessary.

6.1.7 Participant withdrawal

Participant withdrawals may be of several types:

Withdrawal from treatment but not from assessment or data collection

Withdrawal from some aspects of data collection but willingness to be contacted for others

Passive withdrawal: the participant must not be contacted directly, but data may be collected from other sources eg GP notes or clinic records

Active withdrawal: no further data on the participant may be collected from any source

Participants who withdraw from some or all aspects of the trial will have data collected in accordance with the protocol up to the point of withdrawal. If a participant partially withdraws, for example from self-completion questionnaires, data from other sources should continue to be collected as per protocol. Data from patients who withdraw will contribute to the analysis by treatment allocated and may also contribute to secondary and sensitivity analyses, so it is important to collect as full a data set as is practicable and ethical. (MRC guidelines5 model for





____________________________________________________________________________


withdrawal; ICH-GCP2; Data Protection Act6exemption for research)

The analysis plan should include a description of how missing data should be treated, and of planned sensitivity analyses to test the internal and external validity of data.

6.1.8 Missing data

Best efforts to minimise missing data will be made at all stages of data collection and cleaning. This may include reviewing source data such as medical records where appropriate.

Data missing within an existing (validated) measure will be treated in accordance with the instructions for that measure.

Further imputation or multiple imputation will be used where appropriate and stated in the analysis plan, to reduce the impact of missing data. The choice of suitable imputation methods and variables should be clearly justified. Any data excluded from analysis, for example removal of extreme outliers, will be reported.

6.1.9 Manipulation of data after export

Data manipulation may include cleaning the data (checking for inconsistencies and correcting where possible using raw dataset), deriving new variables (such as the calculation of disease duration from age at onset, date of birth and date at data collection), and coding or grouping variables. The raw and master dataset (before manipulation) should be kept and all manipulation should be conducted on a copy of this dataset. In principle raw data sets will be archived before cleaning, and new versions archived after cleaning and again after exporting to statistical software. Further cleaning, missing data imputation and scale calculations will where possible be conducted centrally to provide master data sets from which subsets of the data will be drawn as appropriate for analysis. Once the master data sets have been frozen they should not be altered. All subsequent analyses and further manipulation of subsets of data should be recorded (using syntax where possible) and an audit trail back to the master data set should be kept (WWORTH SOP22a Data Management, and WWORTH SOP21 IT Databases). If subsequent analysis reveals substantial errors in the master, the TMG may agree to replace it by a new version. However both the original and revised master dataset must be retained in archives for the full time specified (WWORTH SOP08 Archiving).

6.1.10 Other analyses

The trial statistician or TDM will provide data as requested to the trial economist (WWORTH SOP26 Economic Evaluation), outcomes specialist





____________________________________________________________________________


and other analysts as approved by the TMG.

6.1.11 CONSORT

WWORTH trials will be designed and reported according to best practice as defined in the relevant CONSORT7 statements (http://www.consort-statement.org/).

6.2 Procedure Flow Chart

Trial design Research questions, treatments, outcome measures, design etc

Methodology including power and sample size calculations

Protocol

Analysis plan Protocol revisions

Collection of data and preparation of datasets

Pre-specified interim analyses (if any)

Data cleaning, including validity checks

Preliminary analysis

Reports to TMG, DMEC, funders as required

Sensitivity analyses, post-hoc hypothesis generation,

concurrent analyses (eg economic)

DMEC

TSC

Full analysis as specified in analysis plan

Main report to TMG, TSC DMEC; then to funders; incorporation in main trial

papers Other reports, publications



http://www.consort-statement.org/




____________________________________________________________________________


6.3 Design and planning

A WWORTH statistician will work within the TDG on methodological aspects of the trial, including choice of method, research questions, outcome measures, and data collection and handling (WWORTH SOP13 Protocol Development).

The trial statistician will draft an appropriate summary of power and sample size calculations, and an appropriate analysis plan, for the outline and definitive grant applications, and ensure that these are consistent with the aims and methods of the trial. The final versions incorporated in the draft protocol should be approved by both the statistician and the TDG.

After the protocol has been agreed by funders the trial statistician will produce the final analysis plan for agreement at the inaugural meetings of the TSC and TMG.

The CI, trial statistician and TDM will jointly develop the format and contents of datasets for approval by the TMG. The trial statistician should approve the databases developed for this purpose by the IT specialist.

6.4 Analysis plan, including interim analysis

The trial statistician is responsible for producing a draft analysis plan during the planning stage, and a definitive analysis plan to be approved by the DMEC, TMG and by external monitoring committees or funders if required. The analysis plan should cover all analyses to be performed including qualitative, genetics, biochemical and Health Economics.

The analysis plan should include:

a statement of trial aims and objectives

a definition of treatments or allocation groups to be compared

a list of primary and secondary outcomes

And descriptions of:

how missing data will be handled

methods for handling outliers, withdrawals and protocol deviations.

all primary and secondary analyses

any tertiary and sensitivity analyses required.

methods for avoiding bias

rules for calculating derived variables

use of baseline values

methods for handling multi-centre data

analysis of subgroups (if adequate power)





____________________________________________________________________________


All new versions of the analysis plan should describe and explain differences from previous versions.

Any procedures relating to a planned examination of the data before the defined end of the trial (as specified in the trial protocol) should be specified, either in a separate interim analysis plan or as part of the final analysis plan.

6.5 Analysis

Preliminary analysis is a check after data cleaning that the full analysis plan remains feasible; it includes summaries of demographics and distributions which can feedback into the data cleaning process. This analysis may also include calculating derived scores such as quality of life measures.

Analysis will be carried out in accordance with the analysis plan, by or under the supervision of the trial statistician. All persons carrying out analyses will have appropriate qualifications and training.

Any interim analyses, except those required by the DMEC, should be pre-specified and incorporated into the relevant power calculations.

Any divergence from the analysis plan should be identified, justified and explained when reporting the results. In particular, new analyses suggested by other results should normally be regarded as hypothesis-generating rather than definitive.

Sensitivity analyses should normally be specified in the analysis plan, but may be developed during the analysis in response to unexpected features of the data sets or results. The role of a sensitivity analysis is to determine whether the assumptions made or decisions taken do in fact have a major effect on the results of the analysis.

6.6 Outcome data reports and statistical reporting

The trial statistician and TDM will prepare internal reports as requested for the TMG, DMEC and funding body. All publications and external reports will be written in accordance with the CONSORT guidelines and sound statistical principles.

The trial statistician will usually draft the results section and tables in the principal report and published papers; and will provide input to sections on methodology, discussion and conclusions (WWORTH SOP29 Trial Reporting). Where this is not the case, (for example an economics paper) the statistician should be given the opportunity to comment on the relevant





____________________________________________________________________________


paper, and approve the final version. Results of analyses should be reported using estimates and confidence intervals which may be supplemented by significance levels.

6.7 Statistical Quality Assurance

6.7.1 Statistical Analysis Plan

The Statistical Analysis Plan will include checks on distributions and model assumptions (see section 6.4 above). Multivariate analysis will be validated by univariate comparisons. Sensitivity analysis may include the analysis of complete data to check imputation (where used).

The CI and health professionals should check the choice of covariates to make sure the input to the analysis makes sense clinically. The CI should also check that the final analysis plan answers all research questions appropriately.

An independent statistician, often the DMEC statistican, should check the final analysis plan for statistical soundness.

6.7.2 External checking of the analysis

The analysis should follow the predetermined analysis plan (see section 6.5 above).

There should be an independent check of the results for consistency and face validity. This can be done by the TDM or another quantitative researcher.

It is recommended that key analyses are conducted by two independent statisticians and results compared for validation. This should always include the primary outcome measure, and usually a sample of other analyses. Analyses with unexpected results should also be repeated independently.

6.8 Responsibility for reporting and interpretation

The trial statistician is responsible for the validation and quality of the statistical analysis conducted. Interpretation of the findings is in collaboration with all authors and investigators of the study.

7 Training Plan

All WWORTH staff involved with trials must undertake the appropriate generic and trial-specific training to ensure that they meet with the specific employers’





____________________________________________________________________________


mandatory training requirements and the specific requirements of the trial. For example, for SU staff, all new employees must attend induction, fire and safety training (as well as role-specific training courses, e.g. laboratory safety). For new staff, additional training requirements should be identified alongside the specific role requirements and the WWORTH Unit Manager should make provision for the new staff member to attend the necessary courses as soon after appointment as is practicably possible.

It is the responsibility of the WWORTH Unit Manager (alongside the CI or TM) to identify all the SOPs that are relevant to a specific trial and in which the new member of staff should be trained. The WWORTH UM or the SOP author will provide group training for trial staff and/or one-to-one training, as required for new staff in relation to the specific SOPs identified. Training records should be filed both by the main employer and the staff member, in accordance with the specific employer requirements. Trial specific training should be filed in TMF or TSF as appropriate and every individual involved in a trial should have an individual training record (see WWORTH SOP02 Training).

Where the tasks specified in the individual SOPs are delegated to WWORTH staff, CIs/PIs or TMs, these delegated staff must ensure that they have attended a training course on GCP and keep up-to-date through attending refresher courses.

It is the responsibility of the CI/PI to ensure that all staff allocated duties on the study delegation log template of responsibilities are suitably trained in the activities linked to those duties (see WWORTH SOP16 Site Setup, Appendix 9 and Appendix 10).

Each trial should maintain a central training log and ensure that WWORTH has access to that log, not least to integrate the logs of staff who work on more than one trial. Similarly trials should ensure that each site maintains a local training log, not least to integrate the logs of staff who work for more than one sponsor.

Initial training on the operation of this SOP will be given to the statistical team by the Senior Statistician; thereafter to all persons acting as trial statistician, assistant statistician or data manager on trials. WWORTH core staff should be involved in the training as appropriate.

8 References

1. European Clinical Trials Directive. See http://www.eortc.be/services/doc/clinical-eu-directive-04-april-01.pdf



http://www.eortc.be/services/doc/clinical-eu-directive-04-april-01.pdf

http://www.eortc.be/services/doc/clinical-eu-directive-04-april-01.pdf



____________________________________________________________________________


2. Good Clinical Practice. See http://www.emea.europa.eu/pdfs/human/ich/013595en.pdf

3. Department of Health. Research Governance Framework for Health and Social Care, 2005.

4. The Royal Statistical Society Code of Conduct http://www.rss.org.uk/

5. MRC http//www.mrc.ac.uk

6. Data Protection Act http//www.ico.gov.uk

7. CONSORT guidelines http://www.consort-statement.org/

9 Related SOPs

WWORTH SOP1a on SOPs WWORTH SOP02 Training WWORTH SOP24 Randomisation WWORTH SOP13 Protocol Development WWORTH SOP17 Monitoring WWORTH SOP03 Master Site File WWORTH SOP21 IT Database WWORTH SOP07 Site Closure WWORTH SOP08 Archiving WWORTH SOP26 Economic Evaluation WWORTH SOP25 Outcome Measures WWORTH SOP31 Sponsorship Adoption WWORTH SOP32 Detecting and Managing Misconduct, Serious Breaches and Deviations from GCP/Protocol



http://www.emea.europa.eu/pdfs/human/ich/013595en.pdf

http://www.rss.org.uk/
