LDS QA/QC

Quality Control/Quality Assurance

Jill S. Heaton, UNR Melissa Brenneman, Topoworks Clarence Everly and staff, MDEP

2

What is QA/QC? z A set of documented procedures that

– outline responsibilities and responsible parties – documents the methods of data collection – ensure that the chosen methods actually support the

research objectives – ensure the data are accurate – produce usable data – provide for archived, easily accessible data

3

Topics z Evolution of the Data Collection process z Evolution of the QA/QC process z QA/QC Products z QA/QC Status

4

UTM FERP w EX ""MCL~~ ,.."EAS""~T~'NG"""_H~NO,,,,,RT);:,:,:'::N~G;j f~'~'~S~1]~11'~'C~'~'~="1:r'~l;'lo~O~.~l ~m"11 (mm)04Q,1.Q" !,II" \")<;'1 , ,-, ;:0, ." . 14 1= ~ 50 ~1"'-"".:f.

~'-"§T~C~0~Y~Egs~Ga~N050~'-"'!!:!!K9j~LE!;:"~C~N~S~0~Y'CE~s:;Il!lfi=NG5o;;d!u=NK=jdIICOMMENTSCAR c=m PlAS~

~ARCASS

EASTINGI ",qcllLo

UTMEASTlNG

~IM!, Pffi' OlSI,! :.1'MeL ,csmON'1I 11MEo,PST! SIZE CLASS SI NCE1~S\1 1 ""'\\O) II iO"'l.O II.:"lt, , s:a.. \J \"1..9-, 11\1- OEATH ~ 1.2Y I$

lITM ",M!, :N'jl, Tl MEOIIPSTl 51 NCe 1l!S.<1'1!

.3B"lIP-.l.\l? IUD2.'O :z. .'b f" _')(,..,-== OEATHOl-2yr1o

0 2-4 '1111

D~ y"

COMMENTS

lITMEASTlNG

..

EASTING

COMMENTS

E§M ~IME SEX \IMCL tmmlOR Il 0'.,-NORTHING II (1001 < ,~DEATHD l-2 yrs O >4yr.

E§M ~'ME SEX POS"'ON'~ I; TIMED· 02-. ....,.O€Ant \-2 ylll >4 'fI1I

lIT" E§~!MEI I P.ERP C1~T II§EXEASTING NORTHING (to Meetmm'Co ,osmON'~11CLASS

D€ATH '-2.,.111.... .,. ..

0 "'011 '1 1'

Data Collection

z 2001

– Paper data sheets

T TOlse 3

UTM TIME TEMP 01$1 (gl POSITI BEHAVI OR

I SIGNS n

1

UTM UTM II II 1Io Olml ii II 'mm, OR IINORTHING II II II <1 yr

COMMENTS

ARCASS 2

\I "'EO'OIST'1 SEX IMCLlmm'"O' 1NORTHING!I IIllo0 1 mlii SI ZE ClASS II II

ARCASS 3

(Psn II PERPOISTI! II SIZE CLASS It POSITION · II TIMED m \ II II SINCE 1yI

COMMENTS

UTM II 'ERPOIS" I10 GPS Coordinates x 11 numbers

NORTH ING PST II

CARCASS

I'MCl1mm,CO'!,a O.1ml ll IlS1ZECtASS II= 110 data entry key strokes SlNCE o< 'yr

- ARCASS 5

PST II 0 1 ml \1

II II I nMEo< 1 'If 0 2SIZE SINCE 0

COMMENTS

0

5

Data Collection z In 2001 there were as few as 5 GPS points per

transect. In 2005 there were as many as 25 per transect.

z In 2001 there were 10 columns of data. In 2005 there were 58 columns of data.

– Threats-exotics, ravens, roads, canids, etc. – Disease-extensive health forms, blood collection – Genetics – Behavioral – etc.

6

Data Collection

z Composed of a large number of records Characteristic Yearly variation Total records 12,000 to 24,000

Observers 50 to 100

Transects 700 to 2,200 Total km walked 3,000 to 9,500 Waypoints 9,000 to 22,000

Observations 1,500 to 2,100

7

Data Collection

z 2002-2005

– Paper data sheets – PDA

8

Need for QA/QC

z The need for a formal Data Management Plan and QA/QC process were identified after the 2003 sample season.

z A Data Management Plan and QA/QC Process were implemented simultaneously/retroactively with the 2004 sampling year.

9

P"ndnogonfu<m

database

--

Prooeoss

o D"~

Workflow

Data Collection

[J

o Data

10

waypo ints w~h

ct

s~e

(mon~oring

w~ hin

Spatial Data Needs Assessment Featu re classes

waypo ints, transects

waypo ints, transects

Feature types

po int , line

po int , line

Need/Quest ion should be co incident

transect lin es . Should not be more than 'x ' waypo ints on one transe Check data to determ ine the value of x.

Mechanism to add ress Use geodatabase topology ru le points must be covered by line

Specify cardinality in relationship class . Check to see if exceptions can ex ist

QC process Valida te topology

Valida te.

Store buffered sites and use

waypo ints , transects , site waypo ints observations

observations

po int , line, poly po int po int

po int

Transects and waypoints should be within a buffer (50m ) of the

strata). Add elevation from NED Add elevation from NED Observations should be a butffer (50m or 100m) of the transe ct.

geodatabase topology ru le for waypoints: must be properly inside polygons Create a custom relationship class extension or a geoprocessing script to check that the points are within the correct site . If poss ible, check for 25 m, but all ow up to 50. Create geoprocessing script. Create geoprocessing script. Create a custom relationship class exte nsion or a geoprocessing script.

If rei class extension: validate

If script validate + run script

Run script. Run script. Validate or run script

11

Transe<:ts Groops Teams

t.anslK:tg' OUp

address obsente~year

obsenterl Observersobsentea

""~COOlaet persootumIj~t

coordinate. lastcomments

Way;>oints

transactwaypolntlead obsanterlollowobsenter

Dalasl1eets

transact lransact~evatkln hat>ilat

(yes/no) descr iption

n umber

obsente rcommoots

GsubO_observatioos GsubO_tortoiS<ls

Gsubllobsentation •••

,",,',",,'behavioreasting

ete.ation(yes/no)

euorcomments

transact observationobsentatlon burlnglast local buring

azimuthobsenterobsenter commentspoerptortolsa

(yes/no) Tortoises.oe

'"obsentation

easting

e~vation

(yes/no)error

beal<lforelimt>sbehavior

dyske<alosis

Done sculeD100d sam~etype

number lubesrommoots

enWes attriootes addresses objecls!romttle use< pe<specti.e, does r8P<eSenl final databaselaDies Attriootes(object) they related,enWes attrib<J\eS user-Ir;endly names, specifoc

wWI be used database. doesspecift<:a lly eOd'ess irllemal othe< allrlootesmay be necessary to estab~sh relat""'shlps. Those beaddressed the des'!!n

should be utraCled s611arate (willarid represented ttlis

Tortoises enmy the atlrioote'BkJodSamp~Taken' '~ood

flnerTortoises enmy the atlrioote

"'l'resented by

table lable beused trad< lortoises

rep'esents requ ired

Data Dictionary

2004 LOS: Master Data List of Entities and Attributes

Information describing transects

Sites

site number abbreviation

name name

date city state zip

start_time end_time name

name

nu mber numbe.

time Photos e" sting northing number number

PDF Ii '" name gps grab gps_ermr file name comments

GsubO Data

GsubO

GsubO site date

GsubO number tortoise number GsubO number t ime tran smitter number visib le pos ition

northing

gps grab gps

Observations of live tortoises and carcasses

Observations Observalioo_measurements

number number

waypolnt time

Intended number

radial distanc e position

distance state

MCl gt 180mm

number posit ion behaviornorthing mass exudategps grab eyes abnormalgps

dirtcomments normal

cutaneous shell slate

luster

cap

Notes: This master lisl 01 and

needs It not and attributes, are listed with the main entity

with wh ich are most directly The names 01 and are not

names that in a It also not numbers or that

will during database phase

lSTS data to a db include red# white#) and are not in diagram On lable, named

(yes , no) is represented by sample Iype' (none, paper, capillary tUbes) On lable, named 'SamplesTaken' (f,lter paper, cap 1, cap 2, cap 3) is 'number cap tubes' (0, 1,2,3) The GSlJbOTortoises entity is a new that coold

to information about the GsubO Bold anributes

12

data should be e"m.cted to a separate induOeredlt and wh~llIII)

tile Tr8llsects !abies transecUl be unique lor an years<>rd. kM!' datI conskl&nl aro:110 aYOld the need lor

C115QOdlng edllS bet_ Transects, Wilypoonts orObservations, da18shoold ootv be stored ooce on theTrllnsecls table, Would require IIlIorl;emen\ openotingprocedure 11\81 trllfll8Cl. is s1Opll8d mid-way. obsllrveB!i.IootJId rIO! finish the Iransect """,thar

Tortoises table, the lItlribute nlWTle<l '8loodSlMTlpleTlIken'(yes, was repLaOld bloocCs.ample_!yp8 Mil"paf*, eapilary lubH)

Tort<M1S table, the e1lrtlut" nameoj 'SarnplesT"k",, ' (fiK.,..paper, cap cap l:ap was replacednumber_ClIp_tubes 2,Important atlribulAlS tile Tortoises table could beInco<porated tile Observations table. Ad<antaglS ar.:tQrtc>ise_'late ~ 00 the I8 me lab'll liS altribut" thale..,dependent 00 the value Of tortolS8_state, 1Or1OiI8_state couklbe modeled as lubtypll (LN. atld CaIQUs), witha~\fdomiolns lor the YllrioulaUributes.May want to separate blood sample attributes their ownteble redllC8 null valuator lhosa attribul~ (whlIraIlImp1es weren·tlXlllactld).

Notes:

...lotio..

northingelevallon

gps_error Observallon_maasuremants~_.

obserya!loof

'" Inlanded_bearir>g

2 lol:al_baa ringazimuth

<>0 ObHrvallons radlal_dlslanl:8~.

gburvallAA'

tranlll:tt

0" llOS t_wlypolnl TortQjsel

.m.m observe" pOIlI lon

oburverJQSitionbahavlor

parp_dlsl8nc:e m_tortolsa_sllIlI

exooaternl:I..JII_1811mm

eyes_abnolmalmd

" 'beak_lore~mbs_dirt

ealtlngbeloavlor_noonall:\IIaneoutI_dysk813tosls

alevallon s helLs1atebone_lCUte_lultlfblood_sample_type

gps_arrornumber_ClIp_robes

~.~.

0"m

P,,",M-'"' lranlec.habitllt

<><

H-------,

I i- -I

'" '"

da lalbn"

' ren.ectalPDF_fill_ name

'"

-0< ""'""~....-

-H Ilr. l_namelas l_name

0" teamlll

Trensects

<.>< " ~trlnlect

'"' 11101

,'M,..0" obIlrv.,1m obIerver2

comments

~

GsuOO_obserJatlons

gtlservltlonl

V'GsubO_118IUf;

0" GsuM_Iortoll8/ll,,-vislbll

"""""_...lulingnorthl~

.0< elevarlonllPS.-.IIrab_boolgplu"rror~_.

H---

H- -

FlO grOlJpt'

Teaml-"1-,---1

oK IUml

H--------

Sitel

" ""'-abbrlvlatlon

QGSubO_lllIrt

....0" obslrvl"

~-~

""""'" ........

P'-"""-""_.contae1Jlllfson

GcubO_~

lorlo!,,'

" 'lranl miller/ll".,

Database Design 5.3.1 Conceptual Geodatabase Design

LSTS db (will

In mUSI

PK

In 10

of , if e on day

delc rlptlon On file_nameH-­ no) with (nonl ,

On name 1, 2, 3) with

(0. 1, 3) from

InIO a)PK

b)tranSIte" a

I tarl_llme e nd_lime FK'

intodalo

I tate'. H

H

r--, -------------------­,,

10 bloodtime a"tlnll

_ _ L __

, " llps_ llrab_boot , , "

= PK.FK1

,PK

H---PK

PK GlybO l !artf dlte PK,FK1 gblaryltlanlPK GluM

date oK'

PK GlubO

0'"

north ing

llPs_lIrab_boot

13

QA/QC 101

z General QA/QC process z Establish a set of rules to flag potential errors z Identify violations (records that broke the rules) z Review and resolve violations (1000’s per year)

z Three levels of QA/QC Identify and correct common, easily corrected errors

Combine contractor databases Verify contractor QA/QC Identify/correct complex errors

Contractor QA/QC

2nd Level QA/QC

Final QA/QC Verify other levels of QA/QC Identify/correct complex errors Ensure final consistency throughout entire database Create final usable products

14

Coo,,,,,,,databaSoll

ft",..-.

....l'i5"FeoPendragon foIms,--oonstllli'lts

Generate

"""""'",...." .._~ Training

"""'"

""""'" CoIoa Raw field...-m." ~ ...~!Irt AI;K

i~e with submittal ofY MDEP

2"'.!eveI QA1QC2"~QNQC I- .....

GTW!88: MatPgms: 15 Mat

-le.... ONdalabal;.

l'Idude al"id with01 & (j(T()( reporto m""

-

W.dne""

'~I

""',,"'"f_

Pfelirnnary 8ffOf

,,""

I-~

r;-rdude with

/~btrilIlll fA (4)10

- _..FNI database l- i-'...- IMIC8ssary) --I ONOC

Com """'"

"""""""''''S&rip(s {topologyrules. ~tilll

""""

Disll'butlon

"'"""'"dalabll58S,e«;, varlous ~-=~

(if necessary)

2005 Line Distance Sampling Process

2005 Workflow START

I I /

'2)-1 / '"

(1), (2), & (3) (1)(2) 10.I

MDEP

Final ­,.,,., (if

2.. T QC

"I

(1) (21 bmittal l3)

-to

1

15

QA/QC Products z GIS files (Geodatabase, Shapefiles, FGDC

compliant metadata) – transects

– observations

– threats – health status – any supporting data (monitoring strata, random start

points, available sample area, etc.) – G0

– etc. z Scanned copies of any paper datasheets z Microsoft Access Database z Microsoft Excel files

16

QA/QC Status z 2001-2004 Beta release z 2005 Beta scheduled for mid April z Beta versions do not include

– data sheets – FGDC metadata – G0