assessment method for routine dam safety monitoring

Hosted by

Black & Veatch Corporation

GEI Consultants, Inc.

Kleinfelder, Inc.

MWH Americas, Inc.

Parsons Water and Infrastructure Inc.

URS Corporation

21st Century Dam Design

Advances and Adaptations

31st Annual USSD Conference

San Diego, California, April 11-15, 2011

On the CoverArtist's rendition of San Vicente Dam after completion of the dam raise project to increase local storage and provide

a more flexible conveyance system for use during emergencies such as earthquakes that could curtail the regions

imported water supplies. The existing 220-foot-high dam, owned by the City of San Diego, will be raised by 117

feet to increase reservoir storage capacity by 152,000 acre-feet. The project will be the tallest dam raise in the

United States and tallest roller compacted concrete dam raise in the world.

The information contained in this publication regarding commercial projects or firms may not be used for

advertising or promotional purposes and may not be construed as an endorsement of any product or

from by the United States Society on Dams. USSD accepts no responsibility for the statements made

or the opinions expressed in this publication.

Copyright 2011 U.S. Society on Dams

Printed in the United States of America

Library of Congress Control Number: 2011924673

ISBN 978-1-884575-52-5

U.S. Society on Dams

1616 Seventeenth Street, #483

Denver, CO 80202

Telephone: 303-628-5430

Fax: 303-628-5431

E-mail: [email protected]

Internet: www.ussdams.org

U.S. Society on Dams

Vision

To be the nation's leading organization of professionals dedicated to advancing the role of dams

for the benefit of society.

Mission USSD is dedicated to:

Advancing the knowledge of dam engineering, construction, planning, operation,

performance, rehabilitation, decommissioning, maintenance, security and safety;

Fostering dam technology for socially, environmentally and financially sustainable water

resources systems;

Providing public awareness of the role of dams in the management of the nation's water

resources;

Enhancing practices to meet current and future challenges on dams; and

Representing the United States as an active member of the International Commission on

Large Dams (ICOLD).

Routine Dam Safety Monitoring Programs 1541

ASSESSMENT METHOD FOR ROUTINE DAM SAFETY MONITORING PROGRAMS

Jay N. Stateler1

ABSTRACT

The U.S. Department of Interior (DOI) recently tasked a team, involving representatives of all the DOI agencies that have dams, to look at quality control/quality assurance issues with respect to routine dam safety monitoring. The team developed a Monitoring Program Assessment Form (MPAF) that breaks the routine dam safety monitoring process down into eight elements, with all but two of these elements further broken down into sub-elements. Breaking the process down in this way allows a close and careful evaluation of each of the building blocks of an effective routine dam safety monitoring program. One area of the MPAF is used to document the approach currently being used by the organization with respect to each element/sub-element so as to ensure that it is being appropriately carried out/satisfied. Another area assesses whether the current approach is apparently in need of significant improvement, and another area is used to generate possible solutions to the apparent problems that are identified. To aid assessments performed within DOI, the Team developed DOI Standards with respect to all the elements/sub-elements. Also, weighting factors were developed for each of the elements/sub-elements so that a quantitative rating between 0 and 100 could be developed using the MPAF, if desired. While rather simple in concept, initial use of the MPAF in the DOI agencies has proven it to be a powerful tool for: (1) pinpointing problem areas in current routine dam safety monitoring work and approaches, and (2) identifying possible solutions that are focused on the identified problem areas.

INTRODUCTION The U.S. Department of Interior (DOI) recently tasked a team to address the following matter with respect to dam safety work: Develop processes to ensure monitoring is being performed on schedule and is being properly evaluated/reported. The team consisted of representatives of each of the DOI agencies that have responsibility for a significant number of dams: Bureau of Reclamation (BOR), Bureau of Indian Affairs (BIA), National Park Service (NPS), Bureau of Land Management (BLM), Fish and Wildlife Service (FWS), and Office of Surface Mining (OSM). Recommendations developed by the Team needed to be applicable to small dams, like those owned and operated by the majority of the DOI agencies, as well as to large dams, such as Hoover and Grand Coulee Dams, that are operated by BOR. In other words, the results of the Teams efforts needed to be scalable to all sizes of dams. As the Team pursued its work, organizations outside DOI were contacted, and their knowledge and experience regarding dam safety monitoring was utilized. These organizations included: U.S. Army Corps of Engineers, Tennessee Valley Authority, BC

1 Civil Engineer, Bureau of Reclamation, Denver, CO, [email protected]

21st Century Dam Design Advances and Adaptations 1542

Hydro, Pacific Gas and Electric, Arizona Public Service Company, and the Metropolitan Water District of Southern California.

MONTIORING PROGRAM ASSESSMENT FORM (MPAF) The fundamental product/tool developed by the Team is the Monitoring Program Assessment Form (MPAF), which is presented below:

Monitoring Program Assessment Form (MPAF) Agency/Organization/Entity: ____________________________________________ Assessment by: Date: Elements and Sub-Elements of an Effective Monitoring Program (Maximum Points for this Element or Sub-Element)

Approach Currently Being Used

In Need of Significant Improvement?

Additional Tools, Processes, Etc. Needed

1. Monitoring program is appropriately defined and communicated to operating personnel

a. Monitoring program, including monitoring frequencies and ranges of expected performance, is developed using potential failure mode analysis and risk analysis (7)

b. Monitoring program requirements are concisely defined/presented (for ease of use and understanding) (4)

c. Monitoring program requirements, including ranges of expected performance, are provided to operating personnel (3)

2. Monitoring work is performed on schedule (13)

3. Monitoring work is performed by appropriately trained and qualified personnel

a. Personnel have a good general understanding of the basic principles of dams and dam safety (3)

b. Personnel have a good general understanding of the design issues and dam safety concerns regarding their dam(s) (4)






c. Personnel have a good understanding of how to properly obtain instrument readings and perform routine visual inspections (4)

4. Monitoring work is performed properly

a. Instrument readings are properly obtained and routine visual inspections are properly performed (4)

b. Personnel have access to appropriate information and tools to allow effective initial evaluations of the data to be performed at the time instrument readings are obtained (4)

c. Instrument readings are checked as they are obtained for anomalies, and re-checked as appropriate (5)

5. Monitoring work results are being properly reported

a. Monitoring data are promptly transmitted (5)

b. Anomalous visual observations and/or instrument readings are immediately reported, when appropriate (7)

6. Monitoring data are evaluated in a timely fashion (9)

7. Monitoring data are evaluated by appropriately trained and qualified personnel

a. Personnel have a good understanding of the principles of dam design and analysis (5)

b. Personnel have a good understanding of the design issues and dam safety concerns regarding their dam(s) (6)

8. Monitoring data are properly evaluated.






a. Personnel have access to appropriate computer tools and information to allow effective evaluations to be performed (5)

b. Personnel effectively assess the data for anomalies, including undesirable gradual data trends (6)

c. Personnel appropriately alert other personnel when anomalous data are noted (6)

Total Points (100 is maximum) Other Comments:

In the first column of the above Monitoring Program Assessment Form (MPAF), the routine dam safety monitoring process is broken down into eight elements, with all but two of these elements further broken down into sub-elements. Breaking the process down in this way allows a close and careful evaluation of each of the building blocks of an effective routine dam safety monitoring program. The second column of the above form is used to document the approach currently being used by the organization with respect to each element and sub-element so as to ensure that it is being appropriately carried out/satisfied. The third column assesses whether the current approach being used is apparently in need of significant improvement, and the fourth column is used to generate possible solutions to the apparent problems that are identified. In some situations, it may be desirable to have the ability to develop quantitative ratings via the MPAF. These might include tracking progress and improvements made over time in a particular program, and allowing rough comparisons to be made regarding the programs of different entities (within an organization or between organizations). To accommodate this, weighting factors were developed for each of the sub-elements and for the elements themselves that have no sub-elements. These appear as numbers in parenthesis in the boxes in the first column of the MPAF. If everything about the routine dam safety monitoring program is perfect, then the full amount of the weighting factor is the score for each element and sub-element, and a total score of 100 is achieved. Where problem areas are identified, appropriate reductions are made in the various weighting factors (down potentially to a score of zero for that element or sub-element), and then each of the element or sub-element scores are totaled to achieve a Routine Monitoring Assessment Score (RMAS) that falls somewhere between 0 and 100. It is very important to note that the MPAF can be very effectively used to identify problems and develop lists


of possible solutions without employing this quantitative rating approach and developing a RMAS. However, to reiterate, the ability to develop quantitative ratings may be a useful capability in some circumstances (e.g. benchmarking).

ELEMENTS AND SUB-ELEMENTS OF THE MPAF The figure below depicts the eight elements of the MPAF:

Element 1.- Monitoring program is appropriately defined and communicated to operating personnel (14 points).- If the monitoring program is not appropriately developed, considering the current potential failure modes for the dam and their risks, then all the time and energy spent on the other aspects of the routine dam safety monitoring program are compromised. Good execution of a poor plan is not a recipe for success. Similarly, a well-developed plan that is not successfully communicated to the operating personnel means that the benefits of the plan will not be realized. For facilities regulated by the Federal Energy Regulatory Commission (FERC), a Surveillance and Monitoring Plan (SMP) based on the results of a Potential Failure Mode Analysis (PFMA) clearly would represent a monitoring program that has been developed using an appropriately rigorous process. Element 2.- Monitoring work is performed on schedule (13 points).- Obviously if the specified monitoring is not carried out, then there is no routine dam safety monitoring program.


Element 3.- Monitoring work is performed by appropriately trained and qualified personnel (11 points).- For effective instrumented monitoring, data collection and instrument maintenance need to be properly performed so that valid, representative data are collected. No data is better than invalid data, because at least one is not being misled. In the area of visual inspection work, training and experience are important with respect to knowing what to look for (which may require effort and searching to see/find), and knowing how to interpret what is seen. Unlike instrumentation data, numerically defining unexpected versus expected monitoring information is not possible regarding visual inspection information, so having knowledge about normal/abnormal dam performance is vital. Element 4.- Monitoring work is performed properly (13 points).- It is one thing to have received the appropriate training. It is another to actually execute properly. Obviously proper execution is fundamental to an effective routine dam safety monitoring program, and improper execution makes qualifications and training received of little relevance. Element 5.- Monitoring work results are being properly reported (12 points).- For instrument readings, typically the raw data collected at the damsite needs to be processed to produce the desired information (in engineering units). Until this is done, benefits from the monitoring efforts regarding these instruments are not obtained. If there is anomalous data, obviously it would be highly desirable to recognize the anomalous instrumentation data quickly, so that appropriate follow-on activities can be instituted promptly. For visual monitoring, it is strongly preferred to have immediate communication regarding anomalous observations (rather than processing the data through the normal pipeline), so that the appropriate follow-on activities can be instituted promptly. In some instances, the time between the first indication of anomalous dam performance and ultimate dam failure may not be very long. Element 6.- Monitoring data are evaluated in a timely fashion (9 points).- In a similar vein to the discussion in Element 5, prompt data review may be crucial. Particularly regarding instrumentation data, until the data is processed and reviewed, the benefits of the monitoring work probably have not been obtained. Element 7.- Monitoring data are evaluated by appropriately trained and qualified personnel (11 points).- Relative to instrumentation data, being able to hone in on that which is crucially important is vital, and requires knowledge of dams in general, and the specific circumstances of the dam in question. With respect to visual monitoring information, the dam safety training of the evaluation personnel typically exceeds that of the data-gathering personnel, and can therefore help in the assessment and response to unusual observations. Element 8.- Monitoring data are properly evaluated (17 points).- In the same vein as Element 4, it is one thing to have received the appropriate training, and it is another to actually execute properly. Obviously proper execution of the data evaluation role is fundamental to an effective routine dam safety monitoring program, and improper execution makes qualifications and training received of little relevance.


It is apparent from the above discussion that assigning weighting points to the above elements is difficult, to put it mildly, since failure regarding just one of the elements can in some instances wreck the entire monitoring effort and nullify all the benefits that could potentially be obtained. In a sense, the elements/sub-elements might be more appropriately viewed as a chain, where the ultimate strength and value of the chain may be determined by its weakest link.

EXAMPLE USE OF THE MPAF To illustrate the use of the MPAF, the initial assessments made of the Bureau of Reclamations routine dam safety monitoring program by a team of Reclamation personnel in 2010 is presented below. For conciseness, the entire MPAF is not presented, but only the results for several of the more interesting elements/sub-elements.

Monitoring Program Assessment Form (MPAF) Agency: Bureau of Reclamation Date: February 17, 2010 Assessment Prepared by Jay Stateler, based on input received during four 1-hour conference calls from a team of Reclamation personnel. Elements and Sub-Elements (Maximum Points)




1. Monitoring program is appropriately defined and communicated to operating personnel: c. Monitoring program requirements, including ranges of expected performance, are provided to operating personnel (3)

The Schedule of Periodic Monitoring (L-23) and the Ongoing Visual Inspection Checklist (OVIC) are incorporated in the Standing Operating Procedures (SOP) document for the dam.

Yes. Currently SOPs often have out-of-date information.

(0)

An expedited procedure for getting L-23s and OVICs incorporated in SOPs (within 90 days) could be required in the Reclamation Manual. A component could be added to the Facility Reliability Rating (FRR) associated with the SOP having the current L-23 and OVIC. (The FRR is a quantitative rating method for scoring the current status of all efforts for a dam - operational, maintenance, dam safety, etc.)






2. Monitoring work is performed on schedule (13)

Oversight by BOR Area Office (AO) personnel (and occasional checks by 86-68360, the Instrumentation and Inspections Group in Denver. Report on Infrequently Read Instruments (IRI) (less frequent than annual) due to be read in the coming year prepared each January. Quarterly reports on delinquent data provided to all involved parties, including the Dam Safety Office (DSO). Quarterly informal feedback reports include this topic. Preparation and discussion in Annual Dam Safety Report (for each BOR region). FRR has a component regarding this. Dam safety training touches on this topic.

Yes. However, for the most part, monitoring breakdowns tend to be associated with some special or unique circumstance, rather than a general lack of program execution. (10)

Increased weighting could be provided for the component of FRR associated with performing monitoring on schedule. The Data Tracker program could be re-vamped and expanded (including 86-68360 process changes) to allow real-time feedback on monitoring program status. Quarterly informal feedback effort could be expanded to highlight IRI monitoring, and this effort could be more closely tracked to ensure its full and satisfactory execution. More emphasis/discussion could be provided on the quarterly delinquent data report results for the past year at the time of the Annual Dam Safety Meetings (for each BOR region). More emphasis/discussion on this topic could be provided during dam safety training activities.






3. Monitoring work is performed by appropriately trained and qualified personnel: b. Personnel have a good general understanding of the design issues and dam safety concerns regarding their dam(s) (4)

Training requirements for all dam operators include on-site dam tender training at least every 6 years.

Yes. The principal issue is when a dam operator leaves and a new dam operator comes onboard. It may take some time before on-site dam tender training can be accomplished for the new dam operator.

(3.5)

On-site training is currently required in the Reclamation Manual to occur as soon as practical (within a few months per FAC 02-01), though this may not always happen. Within 90 days might be a more appropriate requirement, with the understanding that the normal in-person on-site training would be preferred, but training using videoconferencing, a telephone conference call, etc. for portions of the training may be employed when this represents the only realistic way to promptly provide the needed training. The FRR currently does include points for dam operator training, though the weighting in the FRR for this could be increased, due to the importance of this training.

While some of the information in the above table is a bit cryptic, and fully understandable only to Reclamation personnel, the table illustrates the use of the MPAF by one organization. As indicated in the above table, none of the three elements/sub-elements shown achieved the maximum points available, with sub-element 1.c. scoring zero out of a possible 3 points. Clearly this indicates an area where some action should be taken.


DOI STANDARDS AND EXAMPLES To aid assessments performed within DOI, the Team developed Department of the Interior Standards with respect to each of the elements and sub-elements in the MPAF. Necessarily, many of the standards are not concrete in nature; however, they attempt to set the bar appropriately. Additionally, along with the standards, examples illustrating possible ways the standards could be met also were developed. The examples are intended to aid efforts to reach the standards presented. Standards and examples for two elements/sub-elements are presented below, for illustration purposes:

Monitoring Program Assessment Form Standards and Examples Relative to the Elements and Sub-Elements Elements and Sub-Elements

Standards and Examples Relative the Elements and Sub-Elements

2. Monitoring work is performed on schedule.

Standard: Monitoring work is to be performed per the stated requirements, and sufficient oversight of the work is to occur to ensure its proper execution. Examples of methods that could be used to achieve the Standard: 1. Manual tracking of monitoring work performance on forms, charts, etc. 2. Computerized tracking of monitoring work performance. E-mails could be automatically generated regarding overdue work. (Note: Reclamations Data Tracker program could be re-vamped and expanded (including process changes in Reclamations Instrumentation and Inspections Group) to allow real-time feedback on monitoring program status. It could record the following dates: (a) when readings/monitoring was performed, (b) when the data was received by Reclamations Instrumentation and Inspections Group, (c) when the data was entered into the instrumentation computer system, (d) when the Engineering Technician completed work regarding the data, and (e) when the Engineer completed reviews of the data. The program could monitor instrument reading schedules and time intervals between completing milestones, and send e-mail alerts, post messages, etc. with respect to overdue work.) 3. Discussions regarding performance of monitoring work are performed as necessary and appropriate. These could occur at the time of Annual Inspections, Intermediate Exams, on-site dam tender training, etc. 4. Quarterly or semi-annual reports that allow effective oversight of the monitoring work.


Monitoring Program Assessment Form Standards and Examples Relative to the Elements and Sub-Elements Elements and Sub-Elements

Standards and Examples Relative the Elements and Sub-Elements

4. Monitoring work is performed properly: b. Personnel have access to appropriate information and tools to allow effective initial evaluations of the data to be performed at the time instrument readings are obtained

Standard: Operating personnel are to have access to ranges of expected performance for all instruments at the time instrument readings are obtained. Examples: 1. Charts, tables, etc. 2. Readings and other pertinent information (date, reservoir elevation, etc.) are entered into a hand-held data collection unit at the time instrument readings are obtained, and the unit provides an immediate check relative to expected performance limits. 3. A spreadsheet is available that can compute the expected performance limits for all instruments at the damsite in the units the instruments will be read in (e.g. expected depths to water for standpipe piezometers). When data is to be collected, personnel enter the current reservoir level and other pertinent information into the spreadsheet so the spreadsheet can compute the expected performance limits for all the instruments for the current situation at the damsite, and then the actual readings taken can be compared to this information for an instant check for anomalous readings.

INITIAL MPAF USE IN DOI

Each of the DOI Team members used the MPAF to perform an initial evaluation/assessment of their respective agencys routine dam safety monitoring program, in some instances incorporating input from a group or team of agency personnel. Looking at the results of these MPAF evaluation efforts, and focusing on the area of possible solutions to identified issues/problem areas (column 4 on the MPAF forms), some common ground was evident where future collaboration and/or pooling of resources among DOI agencies may be appropriate and beneficial. Also, the Team recommended that formal MPAF assessments occur at least every 5 years for all DOI agencies, and preferably every 3 years. For illustration purposes in this paper, the issues/problem areas noted for Reclamation will be discussed. The most significant problem area identified is that, while Reclamation has good processes in place to develop and document an appropriate routine dam safety monitoring program, it sometimes can take an excessive amount of time for the associated forms (Schedule for Periodic Monitoring (L-23) and Ongoing Visual Inspection Checklist (OVIC)) to actually be updated in the Standing Operating Procedures (SOP) that dam operating personnel use as the operating manual for the dam. It is not uncommon during onsite dam operator training activities to open the SOP and


see L-23 and OVIC forms that have been superseded, perhaps one or more years ago. This creates a problem for operating personnel since they are required to follow the SOP, yet they know the monitoring documents in the SOP are not current. Other significant BOR problem areas that were noted involved achieving timely training of new dam operating personnel (within a short time after they start) and performing visual inspections and obtaining instrument readings as close to the L-23 requirements as realistically possible. Regarding this latter item, performance generally is quite good, but room for improvement definitely exists. The RMAS score from the initial evaluation of Reclamation was 90.0. It is important to note that use of the MPAF can identify issues/problem areas, and possible actions to take with respect to them, but actually implementing actions typically is an issue for management personnel, since expenditures are typically involved, new processes may be needed, choices among alternative possible approaches may need to be made, etc.

CONCLUSIONS Dam safety programs rely on routine dam safety monitoring efforts to provide timely detection and recognition of anomalous dam performance, in a timeframe that allows a response such that dam failure can be avoided. The Monitoring Program Assessment Form (MPAF) was created as a collective effort by DOI agencies to provide a convenient vehicle for evaluating the routine dam safety performance monitoring program of an organization, looking for the weak links in the chain of the program. While rather simple in concept, initial use of the MPAF in the DOI agencies has proven it to be a powerful tool for: (1) pinpointing problem areas in current routine dam safety monitoring work and approaches, and (2) identifying possible solutions that are focused on the identified problem areas. The MPAF is applicable for any size dam, as the basic elements of an effective routine dam safety monitoring are the same, regardless of dam size.

ACKNOWLEDGEMENTS The members of the U.S. Department of Interior (DOI) Team that worked to develop the Monitoring Program Assessment Form (MPAF) and associated DOI standards, besides the author of this paper, are as follows:

Brad Iarossi, Fish and Wildlife Service (FWS) Pearl Chamberlain, Bureau of Indian Affairs (BIA) Yulan Jin, BIA Mike Montgomery, Bureau of Land Management (BLM) Dennis Clark, Office of Surface Mining (OSM) Nathan Tatum, National Park Service (NPS)

Bureau of Reclamation (BOR) personnel that contributed significantly to the development of the MPAF through initial use of the form relative to BORs routine dam safety monitoring program are as follows: Jim Dean, Scott Foster, Jeff Nettleton, Matt Gappa, Clyde Thomas, Rick Scott, and Bill Dutton.

assessment method for routine dam safety monitoring

Documents