sas validation

7/28/2019 SAS Validation

http://slidepdf.com/reader/full/sas-validation 1/31

CROS NT srl

Contract Research Organisation

Clinical Data Management

Statistics

Dr. Paolo Morelli, CEO

Dr. Luca Girardello, SAS programmer

Best Practice in SAS

programs validation. A CaseStudy



AGENDA AGENDA

Introduction

Program Verification: a Business Approach

Program Verification: some case studies



FACTSFACTS aboutabout CROS NTCROS NT• Headquarters in Verona (Italy)

• Founded in 1993

• Offices in Milan and Munich

• 40 employees

• Data Management, Statistical, PhV and

hosting services• Services to Pharma, Biotech and CROs

• Cooperation with Universities of Padua, Bologna, Milan



IntroductionIntroduction

• Topic of the presentation: how to maximize the quality of

programming while minimizing the time to verify program.

• In the first part of the presentation we will discuss about the

business part:

What is program verification?

Why program verification is necessary?

When is program verification done?

Who performs program verification?

How does the verification process work?

• In the second part of the presentation we will discuss about a

case study



What is program verificationWhat is program verification

• Making certain that the program does what it is

supposed to do, producing a documented evidence

of this



Why program verification is necessaryWhy program verification is necessary

• The aim of SAS validation in pharmaceutical research area

is that end-users will produce high quality programs that fitthe purpose for which they are designed and provide

accurate results with a style that they promote:

•Reliabity

•Efficiency

•Portability

•Flexibility

•Ease of use



When is program verification doneWhen is program verification done

• Program verification should performed as soon after the

development of the SAS code, before putting the “product” in

production

• Development and production environment should be clearly

defined;

•Audit trail of program changes should be present as soon

the program is released to production



Who performs program verificationWho performs program verification

The SAS programmer who create the code should perform basic

testing and follow coding rules, like:

• Error log search

• Warning evaluation

• Comments on critical steps

• Comments on Macro usage

• Details of the SAS program (datetime of creation, SAS

programmer name, dataset used, datetime of verification, Name

of second SAS programmer, etc)

•It should be emphasized to perform then a program

verification by a second SAS programmer



How does the verification process workHow does the verification process work

Biostatistician creates specs then

Submits request

SAS developer produces TLGs Then submits verification request

Quality Control programmer verifies results

Interactive Process

Interactive Process



Different Verification ProceduresDifferent Verification Procedures•SOP should define different verification procedures.

üIndependent programming

üReviewing resultsüRandom review of results

üVisually verify code

•Some of them should mandatory, other optional.

•The Document Containing the programming specs (for example the

SAP) should define which approach to follow, illustrating program

verification techniques (for example using alternative SAS

programming procedures)

•The determination of the level of validation should follow a risk-based

model. The key is to determine the effect on the process if the program

does not produce the desired result.



Error TypesError Types

• Business strategy should identify common ‘error types’ found in:

ü Statistical tables

ü Listings

ü Graphs

ü Data analysis files

ü Header section of SAS programs

ü Bad programming specifications

• Metric report related to error type should be analyzed in order to

perform preventive action correction



Specific CDISC SDTM Validation specsSpecific CDISC SDTM Validation specs – –

Metadata LevelMetadata Level•Verifies that all required variables are present in the dataset

•Reports as an error any variables in the dataset that are not

defined in the domain

•Reports a warning for any expected domain variables whichare not in the dataset



Specific CDISC SDTM Validation specsSpecific CDISC SDTM Validation specs --

Metadata LevelMetadata Level

•Notes any permitted domain variables which are not in the

dataset

•Verifies that all domain variables are of the expected data type

and proper length

•Detects any domain variables which are assigned a controlled

terminology specification by the domain and do not have a

format assigned to them



SAS Programming Rules whenSAS Programming Rules when

validatingvalidating

Ø Emphasizing well commented programs.

Ø Macro in order to use programs repeatedly to verify different

programs (re-usability)

Ø Using alternative SAS programming procedures when

validating.

Ø Define a workflow if error are identified



How to optimize the processHow to optimize the process

Good specs & Good standards & Good training

=

Good programming results



A Case A Case StudyStudy



ExampleExample of of DerivedDerived DatasetsDatasets

ValidationValidation (1/4)(1/4)

PROC COMPARE

Compare

original derived datasets

versusvalidation derived datasets

“Second Programmer” programsall derived datasets

“First Programmer” programsall derived datasets





The COMPARE Procedur e

Compar i son of WORK. LI STI NG wi t h WORK. VALI DATI ON

( Met hod=EXACT)

Obser vat i on Summar y

Obser vat i on Base Compare I D

Fi r st Obs 1 1 pt=121

Fi rst Unequal 79 79 pt=201

Last Unequal 79 79 pt=201

Las t Obs 89 89 pt =212

Number of Obser vat i ons i n Common: 89.

Tot al Number of Obser vat i ons Read f r om WORK. LI STI NG: 89.

Tot al Number of Obser vat i ons Read f r om WORK. VALI DATI ON: 89.

Number of Observati ons wi th Some Compared Vari abl es Unequal : 1.

Number of Observat i ons wi t h Al l Compared Var i abl es Equal : 88.

proc compare base=listing compare=validation

listbase listcomp;id pt;run;



Val ues Compar i son Summar y

Number of Var i abl es Compared wi t h Al l Observat i ons Equal : 3.

Number of Vari abl es Compared wi th Some Observat i ons Unequal : 1.

Tot al Number of Val ues whi ch Compare Unequal : 1.

Maxi mumDi f f erence: 1.

Vari abl es wi t h Unequal Val ues

Vari abl e Type Len Label Ndi f MaxDi f

age NUM 8 AGE ( years) 1 1. 000

Val ue Compar i son Resul t s f or Vari abl es

___ ___ __ __ __ __ __ __ ____ __ ___ __ __ __ __ __ __ ____ __ ___ ___ _ __ __ _

| | AGE ( years)

| | Base Compare

pt | | age age Di f f . % Di f f

__ ___ __ | | __ __ ____ _ __ __ __ __ _ _ __ ____ __ ____ _ __ __

| |

201 | | 41 40 - 1. 0000 - 2. 4390 _ __ ___ __ __ __ __ __ __ ____ __ ___ __ __ __ __ __ __ ____ __ ___ ___ _ __ __ _





The COMPARE Pr ocedur e

Compar i son of WORK. LI STI NG wi t h WORK. VALI DATI ON

( Met hod=EXACT)

Obser vat i on Summar y

Obser vat i on Base Compare I D

Fi r st Obs 1 1 pt =121

Last Obs 89 89 pt =212

Number of Obser vat i ons i n Common: 89.

Tot al Number of Obser vat i ons Read f r om WORK. LI STI NG: 89.

Tot al Number of Obser vat i ons Read f r om WORK. VALI DATI ON: 89.

Number of Obser vat i ons wi t h Some Compared Vari abl es Unequal : 0.

Number of Observat i ons wi t h Al l Compar ed Var i abl es Equal : 89.

NOTE: No unequal val ues were f ound. Al l val ues compared are exactl y equal .





ExampleExample of of TablesTables ValidationValidation (1/3)(1/3)

“First Programmer” programs

all tables applying the set of

layout specifications and

saves outputs in Word

“Second Programmer” programs

all tables avoiding to add

additional SAS code to control

output

Compare of outputs



ExampleExample of of TablesTables ValidationValidation (2/3)(2/3)________________________________________________________________

Tmt A Tmt B

________________________________________________________________

Age (years)

n 41 48Mean (SD) 51.44 (10.39) 52.10 (11.00)

Median 55.00 55.00

Min - Max 30.00- 66.00 27.00- 71.00

Gender

Female 14 (34.15%) 21 (43.75%)Male 27 (65.85%) 27 (56.25%)

________________________________________________________________

First Programmer -

Output in Word

Second programmer -

Output SAS

proc means data=demog n mean stddev

median min max;var age;by tmt;

run;



________________________________________________________________

Tmt A Tmt B________________________________________________________________

Age (years)n 41 48Mean (SD) 51.44 (10.39) 52.10 (11.00)

Median 55.00 55.00Min - Max 30.00- 66.00 27.00- 71.00

GenderFemale 14 (34.15%) 21 (43.75%)

Male 27 (65.85%) 27 (56.25%)________________________________________________________________

First Programmer -

Output in Word

Second programmer -

Output SAS

proc freq data=demog;

tables gender*tmt;

run;

ExampleExample of of TablesTables ValidationValidation (3/3)(3/3)



ExampleExample of of ListingsListings ValidationValidation (1/2)(1/2)

“Second Programmer” prints

derived datasets in SAS

“First Programmer” programs

all listings applying the set of layout specifications and

saves outputs in Word

Compare

listing output in Word

versus

output in SAS of derived dataset



ExampleExample of of ListingsListings ValidationValidation (2/2)(2/2)

Li st i ng 1 Demographi c Characteri st i cs

Subj ect I D Gender Age Race

__ __ __ __ __ __ __ _ __ ____ _ __ __ __ __ _

121 M 50 3

122 M 34 3

123 F 58 3

124 M 64 3

125 M 57 3

126 F 64 3

127 M 39 3

128 M 55 2

129 M 41 3

130 M 44 3

131 M 32 3

132 M 37 3

133 M 61 3

134 F 56 3

135 M 34 3

136 M 34 3

Listing Output in Word Print of Derived Dataset



ExampleExample of of RegistrationRegistration ErrorsErrors



Programming

41%

Specification

14%

Layout

45%

MetricsMetrics onon ProgrammingProgramming ErrorsErrors

Selection of

Variables

14%

Calculation of

variables

20%

SAS

Programming

66%

Specification

not detailed

40%

Wrong

interpretation

of

specification

60%

Output

Writing

56%

Output

Structure

30%

Display

Variables

14%



ExamplesExamples of of ErrorsErrors

• Layout

Writing of a note in table

Incorrect: “Percentages are calculated number of patients”

Correct: “Percentages are calculated on number of patients”




data age;

set demog;if age<20 then age_c=1;else if 20<age<40 then age_c=2;

else if age>=40 then age_c=3;

run;

• Programming

data age;

set demog;if age<20 then age_c=1;else if 20<=age<40 then age_c=2;

else if age>=40 then age_c=3;

run;




• Wrong interpretation of specification

Note of a table (in SAP):

“Note 1: Only patients with all value for primary analysis are

included in the table.”

In SAS Program:

In the table, all patients are included



ThankThank youyou forfor youryour attentionattention

QuestionsQuestions??

sas validation

Documents