maintaining mission critical systems in a 24/7 …
TRANSCRIPT
MAINTAINING MISSION CRITICAL SYSTEMS IN A 24/7 ENVIRONMENT
Peter M. Curtis
EEE PRESS SERIES I 0N POWER ENGINEERING
IEEE IEEE PRESS
B I C E N T E N N I A L
1 8 O 7
©WILEY 2 O O 7
B I C E N T E N N I A L
WILEY-INTERSCIENCE A JOHN WILEY & SONS, INC., PUBLICATION
Contents
Foreword xv
Preface xvii
Acknowledgments xix
1. An Overview of Reliability and Resilience in Today's Mission
Critical Facilities 1
1.1 Introduction 1 1.2 Risk Assessment 4 1.3 Capital Costs Versus Operation Costs 6 1.4 Change Management 7 1.5 Testing and Commissioning 7 1.6 Documentation and Human Factor 8 1.7 Education and Training 11 1.8 Operation and Maintenance 11 1.9 Employee Certification 12 1.10 Standard and Benchmarking 13
2. Policies and Regulations 15
2.1 Executive Summary 15 2.2 Introduction 16 2.3 Industry Regulations and Policies 16
2.3.1 U.S. Patriot Act 16 2.3.2 The National Strategy for the Physical Protection
of Critical Infrastructures and Key Assets 19 2.3.3 U.S. Security and Exchange Commission (SEC) 20 2.3.4 Sound Practices to Strengthen the
Resilience of the U.S. Financial System 20 2.3.5 Federal Real Property Council (FRPC) 21 2.3.6 Base! II Accord 21
vi Contents
2.3.7 Sarbanes-Oxley (SOX) 22 2.3.8 NFPA 1600 23
3. Mission Critical Facilities Engineering 25
3.1 Introduction 25 3.2 Companies' Expectations: Risk Tolerance and Reliability 27 3.3 Identifying the Appropriate Redundancy
in a Mission Critical Facility 29 3.4 Improving Reliability, Maintainability, and
Proactive Preventative Maintenance 29 3.5 The Mission Critical Facilities Manager
and the Importance of the Boardroom 31 3.6 Quantifying Reliability and Availability 31
3.6.1 Review of Reliability Versus Availability 32
3.7 Design Considerations for the Mission Critical Data Center 32 3.8 Mission Critical Facility Start-Up 33 3.9 The Evolution of Mission Critical Facility Design 34
4. Mission Critical Electrical Systems Maintenance 35
4.1 Introduction 35 4.2 The History of the Maintenance Supervisor and the
Evolution of the Mission Critical Facilities Engineer 37 4.3 Internal Building Deficiencies and Analysis 39 4.4 Evaluating Your System 40 4.5 Choosing a Maintenance Approach 41 4.6 Standards and Regulations Affecting How
Safe Electrical Maintenance Is Performed 42 4.7 Maintenance of Typical Electrical Distribution
Equipment 44
4.7.1 Infrared Scanning 44 4.7.2 15-Kilovolt Class Equipment 46 4.7.3 480-Volt Switchgear 46 4.7.4 Motor Control Centers and Panel Boards 47 4.7.5 Automatic Transfer Switches 48 4.7.6 Automatic Static Transfer Switches (ASTS) 48 4.7.7 Power Distribution Units 49 4.7.8 277/480-Volt Transformers 49 4.7.9 Uninterruptible Power Systems 49 4.7.10 A Final Point on Servicing Equipment 50
4.8 Being Proactive in Evaluating the Test Reports 51 4.9 Data Center Reliability 51
Contents VÜ
5. Standby Generators: Technology, Applications, and Maintenance 53
5.1 Introduction 53 5.2 The Necessity for Standby Power 54 5.3 Emergency, Legally Required, and Optional Systems 55 5.4 Standby Systems that Are Legally Required 57 5.5 Optional Standby Systems 57 5.6 Understanding Your Power Requirements 58 5.7 Management Commitment and Training 58
5.7.1 Lockout/Tagout 59 5.7.2 Training 59
5.8 Standby Generator Systems Maintenance Procedures 60
5.8.1 Maintenance Record Keeping and Data Trending 60
5.8.2 Load Bank Testing 62
5.9 Documentation Plan 63
5.9.1 Proper Documentation and Forms 63 5.9.2 Record Keeping 64
5.10 Emergency Procedures 64 5.11 Cold Start and Load Acceptance 65 5.12 Nonlinear Load Problems 66 5.13 Conclusions 68
6. Fuel Systems and Design and Maintenance for Fuel Oil 69 Howard L. Chesneau, Edward English III, and Ron Ritorto
6.1 Fuel Systems and Fuel Oil 69
6.1.1 Fuel Supply Maintenance Items 70 6.1.2 Fuel Supply Typical Design Criteria 70
6.2 Bulk Storage Tank Selection 71 6.3 Codes and Standards 72 6.4 Recommended Practices for all Tanks 73 6.5 Fuel Distribution System Configuration 76
6.5.1 Day Tank Control System 79
7. Automatic Transfer Switch Technology, Application, and Maintenance 89
7.1 Introduction 89 7.2 Overview 90
viii Contents
7.3 Transfer Switch Technology and Applications 91
7.3.1 Types of Transfer Switches 92 7.3.2 Bypass-Isolation Transfer Switches 94 7.3.3 Breaker Pair ATSs 96
7.4 Control Devices 96
7.4.1 Time Delays 96 7.4.2 In-Phase Monitor 97 7.4.3 Programmed (Delayed) Transition 98 7.4.4 Closed Transition Transfer (Parallel Transfer) 99 7.4.5 Test Switches 100 7.4.6 Exercise Clock 102 7.4.7 Voltage and Frequency Sensing Controls 102
7.5 Optional Accessories and Features 102 7.6 ATS Required Capabilities 103
7.6.1 Close Against High In-Rush Currents 103 7.6.2 Withstand and Closing Rating (WCR) 103 7.6.3 Carry Füll Rated Current Continuously 104 7.6.4 Interrupt Current 104
7.7 Additional Characteristics and Ratings of ATSs 104
7.7.1 NEMA Classification 104 7.7.2 System Voltage Ratings 105 7.7.3 ATS Sizing 105 7.7.4 Seismic Requirement 105
7.8 Installation, Maintenance, and Safety 105
7.8.1 Installation Procedures 105 7.8.2 Maintenance Safety 106 7.8.3 Maintenance 107 7.8.4 Drawings and Manuals 107 7.8.5 Testing and training 107
7.9 General Recommendations 110
8. The Static Transfer Switch 113
8.1 Introduction 113 8.2 Overview 114
8.2.1 Major Components 114
8.3 Typical Static Switch One Line 115
8.3.1 Normal Operation 116 8.3.2 STS and STS/Transformer Contigurations 117
8.4 STS Technology and Application 117
8.4.1 General Parameters 117 8.4.2 STS Location and Type 118
Contents ix
8.4.3 Advantages and Disadvantages of the Primary and Secondary STS/Transformer Systems 118
8.4.4 Monitoring and Data Logging and Data Management 118
8.4.5 STS Remote Communication 119 8.4.6 Security i 19 8.4.7 Human Engineering and Eliminating Human Errors 120 8.4.8 Reliability and Availability 121 8.4.9 Reparability and Maintainability 122 8.4.10 Fault Tolerance and Abnormal Operation 123
8.5 Testing 123 8.6 Conclusion 124
9. The Fundamentals of Power Quality and their Associated Problems 127
9.1 Introduction 127
9.2 Electricity Basics 128
9.2.1 Basic Circuit 129
9.3 Transmission of Power 130
9.3.1 Life Cycle of Electricity 130 9.3.2 Single- and Three-Phase Power Basics 132 9.3.3 Unreliable Power Versus Reliable Power 134
9.4 Understanding Power Problems 134
9.4.1 Power Quality Transients 135 9.4.2 RMS Variations 137 9.4.3 Causes of Power Line Disturbances 142 9.4.4 Power Line Disturbance Levels 147
9.5 Tolerances of Computer Equipment 148
9.5.1 CBEMA Curve 149 9.5.2 ITIC Curve 149 9.5.3 Purpose of Curves 150
9.6 Power Monitoring 150
9.6.1 Example Power Monitoring Equipment 153
9.7 The Deregulation Wildcard 153 9.8 Troubleshooting Power Quality 154
10. An Overview of UPS Systems: Technology, Application, and Maintenance 159
10.1 Introduction 159 10.2 Purpose of UPS Systems 160
X Contents
10.3 General Description of UPS Systems 163
10.3.1 What Is a UPS System? 163 10.3.2 How Does a UPS System Work? 163
10.4 Static UPS Systems 164
10.4.1 Online 165 10.4.2 Double Conversion 165 10.4.3 UPS Power Path 165
10.5 Components of a Static UPS System 167
10.5.1 Power Control Devices 167
10.5.2 Line Interactive UPS Systems 172
10.6 Rotary Systems 173
10.6.1 Rotary UPS Systems 173 10.6.2 UPSs Using Diesel 174
10.7 Redundancy and Configurations 175
10.7.1 Redundancy 176 10.7.2 Isolated Redundant 177 10.7.3 Tie Systems 177
10.8 Batteries and Energy Storage Systems 178
10.8.1 Battery 178
10.8.2 Flywheel Energy 181
10.9 UPS Maintenance and Testing 182
10.9.1 Steady-State Load Test 185 10.9.2 Harmonie Analysis 185 10.9.3 Filter Integrity 186 10.9.4 Transient Response Load Test 187 10.9.5 Module Fault Test 188 10.9.6 Battery Run Down Test 188
10.10 Static UPS and Maintenance 188
10.10.1 Semi-Annual Checks and Services 189
10.11 UPS Management 190 10.12 Additional Topics 190
10.12.1 Offline (Standby) 190
11. Data Center Cooling: Systems and Components 193 Don Beaty
11.1 Introduction 193 11.2 Building Cooling Overview 193
Contents xi
11.3 Cooling Within Datacom Rooms 194 11.4 Cooling Systems 195
11.4.1 Airside 195 11.4.2 Waterside 196 11.4.3 Air-and Liquid-Cooling Distribution Systems 198
11.5 Components Outside the Datacom Room 202
11.5.1 Refrigeration Equipment—Chillers 202 11.5.2 Heat Rejection Equipment 206 11.5.3 Energy Recovery Equipment 215
11.6 Components Inside Datacom Room 224
11.6.1 CRAC Units 224
12. Raised Access Floors 229 Dan Catalfu
12.1 Introduction 229
12.1.1 What Is an Access Floor? 229 12.1.2 What Are the Typical Applications for Access
Floors? 230 12.1.3 Why Use an Access Floor? 231
12.2 Design Considerations 231
12.2.1 Determine the Structural Performance Required 231 12.2.2 Determine the Required Finished Floor Height 234 12.2.3 Determine the Understructure Support Design Type
Required 235 12.2.4 Determine the Appropriate Floor Finish 236 12.2.5 Airflow Requirements 237
12.3 Safety Concerns 239
12.3.1 Removal and Reinstallation of Panels 239 12.3.2 Removing Panels 240 12.3.3 Reinstalling Panels 241 12.3.4 Stringer Systems 241 12.3.5 Protecting the Floor from Heavy Loads 242 12.3.6 Grounding the Access Floor 248 12.3.7 Fire Protection 249 12.3.8 Zinc Whiskers 249
12.4 Panel Cutting 250
12.4.1 Safety Requirements for Cutting Panels 250 12.4.2 Guidelines for Cutting Panels 250 12.4.3 Cutout Locations in Panels;
Supplemental Support for Cut Panels 251 12.4.4 Saws and Blades for Panel Cutting 251
xii Contents
12.4.5 Interior Cutout Procedure 251 12.4.6 Round Cutout Procedure 252 12.4.7 Installing Protective Trim Around Cut Edges 252
12.5 Access Floor Maintenance 253
12.5.1 Standard High-Pressure Laminate Floor Tile (HPL) 253 12.5.2 Vinyl Conductive and Static Dissipative Tile 254 12.5.3 Cleaning the Floor Cavity 255 12.5.4 Removing Liquid from the Floor Cavity 255
12.6 Troubleshooting 256
12.6.1 Making Pedestal Height Adjustments 256 12.6.2 Rocking Panel Condition 256 12.6.3 Panel Lipping Condition (Panel Sitting High) 257 12.6.4 Out-of-Square Stringer Grid (Twisted Grid) 258 12.6.5 Tipping at Perimeter Panels 259 12.6.6 Tight Floor or Loose Floor: Floor
Systems Laminated with HPL Tile 259
13. Fire Protection in Mission Critical Infrastructures 261 Brian K. Fabel
13.1 Introduction 261 13.2 Philosophy 262
13.2.1 Alarm and Notification 263 13.2.2 Early Detection 265 13.2.3 Fire Suppression 265
13.3 Systems Design 268
13.3.1 System Types 268 13.3.2 Fire and Building Codes 268
13.4 Fire Detection 270 13.5 Fire Suppression Systems 277
13.5.1 Watermist Systems 282 13.5.2 Carbon Dioxide Systems 285 13.5.3 Clean Agent Systems 286 13.5.4 Inert Gas Agents 287 13.5.5 IG-541 287 13.5.6 IG-55 288 13.5.7 Chemical Clean Agents 289 13.5.8 Fire Extinguishers 293
Contents xiii
Bibliography 295
Appendix A: Critical Power 297
Appendix B: BITS Guide to Business-Critical Power 393
Appendix C: Syska Criticality Levels 459
Glossary 473
Index 481