qa of hydroacoustic instruments · why do we care about qa? many procedures exist for mechanical...

43
QA of Hydroacoustic Instruments What's My Responsibility?

Upload: others

Post on 06-Jul-2020

5 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

QA of Hydroacoustic Instruments

What's My Responsibility?

Page 2: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

Overview of Presentation

Hydroacoustic Instruments – Discharge measurements and Index Velocity

Instrument Quality Assurance

Acceptance Testing [HIF + WSC]

Post Factory-Repair Testing [HIF and(or) WSC]

Routine QA [WSC]

QA for Instruments for which formal testing by OSW is incomplete [WSC]

ADVM QA

Question and Answer

Page 3: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

Why do we Care about QA?

Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange, etc.)

Need have equivalent practices for acoustic instruments

1

33

47

57

69 73

0

10

20

30

40

50

60

70

80

90

100

1995 2006 2007 2008 2009 2010

Perc

en

t

Percent of Discharge Measurements Made with Hydroacoustic Instruments

Approximately 6% of streamflow stations

are index velocity stations

Page 4: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

What’s My (Your) Responsibility?

Objective of this presentation:

Create awareness of QA activities and their importance

Provide overview of QA activities led by OSW and others

Focus on QA activities you can influence or control for hydroacoustic instruments used to make discharge measurements

Brief overview of ADVM QA

QA is EVERYONE’s responsibility! (i.e. not just OSW)

Page 5: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

Hydroacoustics Instruments

Discharge Measurement Instruments

Tested by OSW with formal policy/guidance

Incomplete testing with no formal policy/guidance

Page 6: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

Hydroacoustics Instruments

Acoustic Doppler Velocity Meters

Index velocity instruments

Instrument and measurement QA are closely linked

Details require a separate webinar

Page 7: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

Three Kinds of Testing

Acceptance Testing [HIF + WSC]

Testing conducted to determine if basic requirements instrument operation are met

Example: Staff sections are tested to verify that the distance indicated on the staff is accurate to within some spec

Post Factory-Repair Testing [HIF + WSC]

Testing conducted after a repair. May include all or part of acceptance test procedures

Routine QA [WSC]

Routine tests done to assure user that instrument is functionally normally

Page 8: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

Advantage of HIF Purchase

Acceptance testing: 100% of Flowtrackers / ADCPs purchased through the HIF are tested

HIF coordinates recalls/updates, etc.

HIF maintains database with info on every Flowtracker purchased/repaired/tested

Lower overhead means lower cost for WSCs

Page 9: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

HIF Acceptance Testing - Flowtrackers

Power-up check

Thermistor test

Beam Check

Automated tow tank test

Two cart speeds 18 & 33.5 cm/s

Page 10: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

WSC Acceptance Testing - Flowtrackers

Check Flowtracker operation

Refresh batteries if necessary

Power up and check menus

Perform a bucket test

Best Practice: Perform a check discharge measurement to verify that the Flowtracker is working. Compare to expected discharge (rating or shifted discharge) provided it is relatively accurate. Doesn’t have to be a comparison to AA or ADCP

Page 11: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

HIF Acceptance Testing - ADCPs

Tests consist of:

Power-up check: comms, heading, pitch/roll, ping

Thermistor validation

Distance test in flume – StreamPro and M9/S5s only

Page 12: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

HIF Acceptance Testing - ADCPs

New ADCP testing tank being constructed

Dimensions: 80 x 4 x 4 ft.

Page 13: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

WSC Acceptance Testing - ADCPs

System checkout

Apply power

Check comms

Run diagnostic tests

Note transformation matrix – TRDI ADCPs

Best Practice: Perform at least 1 check measurement to verify that the ADCP is working. Compare to expected discharge (rating or shifted discharge). Comparison doesn’t have to be with AA, Flowtracker, or ADCP. >PS3

1.4789 -1.4747 -0.0082 0.0035

-0.0222 0.0269 -1.4724 1.4705

0.2673 0.2643 0.2702 0.2612

1.0391 1.0453 -1.0403 -1.0451

Page 14: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

Post Factory Repair Testing - Flowtrackers

Must be returned to HIF after factory repairs – regardless of who (HIF or WSC) initiated the repair

Best Practice: Arrange for repairs via HIF

When received by HIF they will:

Verify to the extent possible that repair was successful

Perform routine acceptance tests

When received at WSC, hydrographer:

Should conduct power-up testing and bucket test

Best Practice: Make a check QM

Page 15: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

Post Factory Repair Testing - ADCPs

Not required that ADCP be returned to HIF – though HIF can assist with this

WSC staff should perform system checkout

Apply power

Check communications (BBTalk)

Run diagnostic tests

Check values of transformation matrix and compare to values prior to repair (should be the same)

Best Practice: Make a check QM to verify that ADCP is functioning normally. Not required.

Page 16: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

Routine ADCP QA – Beam Alignment

Testing all aspects of an ADCP is almost impossible under any circumstances. Beam alignment errors are potentially the largest source of BIAS

30

40

V*Sin (40) V*0.64

V*sin (30) V*0.5

V

Page 17: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

Routine ADCP QA – Beam Alignment

OSW Technical Memo 2009.05 states that beam alignment test should be done for

All new instruments not tested by HIF

Any ADCP returned from factory repair or evaluation

Every instrument once every 3 years

After firmware upgrades (?)

Checking that transformation matrix is unchanged is probably sufficient after a firmware upgrade. Make sure to store a log of matrix.

>PS3

1.4789 -1.4747 -0.0082 0.0035

-0.0222 0.0269 -1.4724 1.4705

0.2673 0.2643 0.2702 0.2612

1.0391 1.0453 -1.0403 -1.0451

Page 18: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

Routine ADCP QA – Beam Alignment Requires:

Manned boat with ADCP/GPS mounts

GPS with differential capability (WAAS) (GGA & VTG)

Lake or water body shallow enough to reliably bottom track with no rapid changes in bed elevation

Ability to navigate at constant heading & speed for long distance

Considerations:

Test does not evaluate water tracking, though beam alignment affects both BT and WT

Need to maintain bottom tracking

Heading is important – NOT course!

Page 19: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

Mount ADCP (note beam orientation)

Conduct internal ADCP diagnostic tests

Begin pinging – but not recording

Configure software to view Distance Made Good (DMG) and ratio of BT and GPS DMG (known as BC/GC)

Record GPS (GGA/VTG) and ADCP data

When boat at desired speed, begin recording

Maintain constant compass heading /speed

Beam Alignment Procedure

Page 20: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

Beam Alignment Procedure Traverse a long (1,200–2,500

ft) course. Longer is better

At the end of the course but before reducing speed or changing heading, stop recording

Record BC/GC value

Do a reciprocal traverse (a transect of the same length at a heading ~180 degrees from the previous course)

Bad

Good

Page 21: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

Beam Alignment Procedure

Record BC/GC value

Average BC/GC values for reciprocal courses

Rotate ADCP 45 degrees and repeat 4 times

For most ADCPs: 0.995 < BC/GC < 1.003

If BC/GC is outside this range, arrange for return to factory for factory beam alignment test

Alternative: Pay for factory to perform test and report results

Page 22: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

Beam Alignment Procedure

Page 23: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

Beam Alignment Procedure

Document and store results in accordance with QA plan / Electronic data archive. Preferably use an Instrument History Log

Page 24: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

Routine ADCP QA – Beam Alignment

Instrument-specific Considerations

M9/S5s: Similar performance to Rio Grande? Can ask HIF to perform distance test for a fee

StreamPros: Cannot be tested without compass & GPS. HIF/Mfgr can conduct beam alignment test.

RiverRay ???

Policy/experience with new instruments will evolve. Makes comparison QMs are all the more important.

Page 25: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

Routine QA - Flowtrackers

Review auto-beam checks for every QM

Document any changes/anomalies

Run bucket test if QC Check fails

Page 26: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

Routine QA - Flowtrackers

Bucket tests

After questionable results on auto-beam check

After possible damage to instrument (a drop, etc.)

Log and compare with previous log tests – should be consistent over time

Page 27: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

Routine QA - Flowtrackers

Flowtracker Calibration Program

Every ADV recalled every 3 years (OSW Memo 2010.12)

Tests performed include: thermistor check, bucket test, and tow tank test at 2 speeds

Flowtracker re-calibrated if necessary

A WSC may request that a Flowtracker be tested at any time

Page 28: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

Routine QA - Flowtrackers

OSW Flowtracker Calibration Program - Hydrographer Responsbilities:

Register all WSC Flowtrackers in HIF database

(http://1stop.usgs.gov/flowtracker/)

Respond in a timely manner to recall or let HIF know

Review results (ask questions as necessary)

Page 29: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

Routine QA - Flowtrackers

Sample Flowtracker QA Test Results

Page 30: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

FlowTracker QA Test Results

23 meters (~9%) failed at least 1 of the QA tests

Temperature probe failures: 4 (~2%)

Velocity failures: 11 (~4%)

Remainder: beam check

Page 31: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

Periodic Instrument Check

Best Practice: Conduct periodic instrument checks. Not required. Conduct at regular interval, e.g. annually?

Individual ADCP comparison QMs

Compare discharge measured with your ADCP with discharge measured with a current meter, some other accepted instrument, or stable S-Q rating

Don’t always use the same measurement location

What happens when the comparison is outside an acceptable range (for example 5%)? Repeat measurements – also at different sites

Record results in Instrument History Log

Page 32: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

Periodic Instrument Check

ADCP Regattas

Compare discharges measured by many/all WSC ADCPs

Select site where multiple sections can be used

Ideally have one instrument measuring continuously

Steady – or near steady-flow conditions

Synchronize times for ADCPs/Laptops before starting

Maintain 5 channel depth separation for similar acoustic frequencies

Good opportunity for review of important policies, new info regarding ADCPs, etc.

Page 33: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

Example Regatta Summary http://hydroacoustics.usgs.gov/publications/CA-ADCP-Check-2007.pdf

2000

2200

2400

2600

2800

3000

8:00 AM 9:00 AM 10:00 AM 11:00 AM 12:00 PM 1:00 PM 2:00 PM 3:00 PM 4:00 PM

ADCP Comparison Data June 22, 2010

(each point represents a 4 transect mean)

Baseline (OSW 1406 / 600 kHz) IL 9521 (1200 kHz)IL 8630 (1200 kHz) IL 8619 (600 kHz)IL 8589 (1200 kHz) IL 8570 (600 kHz)IL 6223 (1200 kHz) IL 6222 (1200 kHz)

Time

Dis

charg

e (c

fs)

IL WSC 2010 Regatta

Page 34: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

New Profilers

M9

S5

SonTek/YSI RiverSurveyor TRDI RiverRay

Page 35: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

New Instruments: RiverSurveyor and RiverRay

OSW doesn’t have sufficient data to find all potential issues

Make comparison QMs for the range of conditions Water velocities

Streambed type

Flow depths

Turbulence

Sediment concentration

Use the instrument if there are no significant differences from the comparison measurements. Consistent biases are of particular concern

Document and submit comparisons to OSW Sharepoint site https://xcollaboration.usgs.gov/wg/OSWHA/Testing

Page 36: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

Sharepoint Testing Pages

https://xcollaboration.usgs.gov/wg/OSWHA/Testing

Page 37: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

Sharepoint Testing Pages

https://xcollaboration.usgs.gov/wg/OSWHA/Testing

Page 38: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

Ott ADC Make comparison QMs for the range of conditions

Water velocities

Streambed type

Flow depths

Turbulence

Sediment concentration

Use the instrument if there are no significant differences from the comparison measurements. Consistent biases are of particular concern

Ott ADC

Page 39: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

ADVM QA

Independent water temperature checks every site visit (at depth of & near to ADVM transducers)

Monitor time series data for changes (SNR, temperature, velocities, pitch/roll, standard error, cell end, water depth, etc.)

Water Temperature

Page 40: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

ADVM QA

Cell End

Heading, Pitch, and Roll

Page 41: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

ADVM QA

Range-averaged Velocity

Multi-cell Velocities

Page 42: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

ADVM QA

SNR

Pressure & Water Depth

Page 43: QA of Hydroacoustic Instruments · Why do we Care about QA? Many procedures exist for mechanical current meters (spin tests, checking pivots/bucket wheels/yokes, current meter exchange,

Questions?