Rx For Emergency Power Reliability

Just because EP systems are out of sight doesn’t mean they should be out of mind. Many facilities don’t know theirs are unhealthy until they really need them. As some unlucky operations have learned, that’s too late.

What is emergency power (EP) reliability?  It’s having EP available to power WHAT it needs to, WHEN it needs to, FOR AS LONG AS it needs to. That’s quite a tall order, but one that can be satisfied with a comprehensive approach.

How do we make emergency power systems more reliable? It’s a matter of holistic management, including:

EP system master planning for future flexibility and load growth.

Designs that facilitate shutdowns, maintenance, reliability and regular testing.

Commissioning and comprehensive installation acceptance testing.

A management program that includes determining the maximum demand loading, fuel management and accurate, useful documentation.

A rigorous inspection, testing and maintenance program.

Emergency preparedness for both internal and external power failures.

Continuous quality improvement with both maintainer and user education.

A process to investigate and resolve systemic issues.

Master planning
EP system master planning can address generation and distribution deficiencies, backup feeders and redundancy, optimal locations for generator(s), fuel tanks, distribution or support equipment and both horizontal and vertical utility corridors. Other master-planning examples include allowing for planned load growth, installing empty conduits at opportune times—such as when ceilings are down and when slabs are poured—avoiding choke points and planning for a future electrical-power management system when new generation and distribution components are installed.

Designing for reliability
Designing for EP system reliability includes consideration of operational flexibility, load growth, susceptibility to damage from both internal and external causes and facilitating testing, shutdowns and maintenance. It also includes having a power system with selective (protective) coordination that works effectively.

Equipment and feeder locations, of course, are a major issue. It’s important to avoid designs that are susceptible to common mode failures. Consider areas where equipment (or even EP feeders) might be subject to flooding from internal causes—i.e., broken water mains, risers and sprinkler piping—as well as external ones.

Designs should facilitate maintenance. Examples of this include having enough room for maintenance to occur, and using draw-out power circuit breakers, where practical, for ease of maintenance and primary injection current testing.

One design feature that can improve operational reliability during testing is using uninterruptible power supplies (UPSs) for building-management system panels and air-temperature control panels. Some operations are also installing UPSs on the output sides of selected transfer switches to ease regular testing and maintenance.

Using more transfer switches—and smaller ones at that—is usually a more reliable overall approach than using fewer (and larger) ones. Although increasing the quantity of devices increases system complexity, the “many eggs in one basket” issue and longer load-side distribution feeders associated with fewer transfer switches can have a negative effect on overall system reliability. Locating transfer switches closer to their loads increases reliability because transfer switch load side feeder failures, such as from core drilling, are very problematic.

Designs also should facilitate shutdowns for maintenance. Many EP systems operate 24/7/365. Since they’re often difficult to shut down due to facility operational concerns, they may not be adequately maintained in some plants. If not shut off on a pre-planned basis and maintained, however, these systems will shut themselves off unexpectedly when they fail.

System specifications should take the following points into consideration:

Designing in system redundancy and using backups can make shutdowns easier to conduct.

Using vision ports for major distribution equipment facilitates infrared thermographic evaluations of equipment interiors.

Using door-in-door panel trims leads to easier access for maintenance during shutdowns.

Permanent load banks sized at 30 – 50% of generator ratings are very helpful for troubleshooting, mainte-nance and testing.

Multiple distribution pathways, such as main-tie-main switchboards or overlapping receptacle coverage patterns, can help provide operational flexibility.

Switchboard rooms, electric closets and mechanical equipment rooms need extra EP lighting and EP outlets for maintenance and troubleshooting during normal power shutdowns.

Management processes
Updating infrastructure documentation benefits reliability because inaccurate documentation can lead to incorrect decisions. Project as-built drawings can lose their usefulness because of overlapping renovations. A main one-line diagram, updated regularly with renovation-related changes, is an extremely useful disaster-management tool. Usefulness is enhanced when this diagram includes updated demand loading at generators, transfer switches and switchboards.

It is necessary to understand your facility’s peak EP demand load at each generator and transfer switch. Because of the time of day that monthly testing occurs, the test loading on the generator(s) is unlikely to be the real peak demand load during normal power outages. Familiarize yourself with National Electrical Code® paragraph 220.87, entitled “Determining Existing Loads,” for further guidance on this topic.

EP-system commissioning can improve reliability by ensuring that the design intent is realized during the procurement and construction process. The NFPA 110 Installation Acceptance Test in new unoccupied facilities forces a new facility to operate only on EP—thereby verifying that equipment needing to be on EP is in fact connected to EP.

Consider the lessons learned from planned outages and unplanned outages. These lessons can help identify opportunities to improve the infrastructure, the management systems and the processes, which, in turn, enhances EP system reliability.

Proactive use of utility failure incident reports, including assessment of random electrical “glitches” or nuisance events, also can enhance reliability. These seemingly random events can be precursors of incipient catastrophic events. Proactive facility managers understand that when it comes to electrical systems, odd things typically happen for a reason. They investigate such events to determine possible generic relevance.

Several actions can be taken to prepare for extended generator runs: Obtain and follow all manufacturer recommendations. Design features can include dual fuel filters and transfer valves, as well as inlet and outlet fuel pressure gauges to monitor fuel clogging. Checks should be made of secondary filter elements, fuel oil system, lube oil level gauge, air filters and control systems. (The list of references on page 18 includes an article on extended generator runs by John Diamond that contains numerous worthwhile suggestions.)

EP system maintenance
EP systems and equipment should be maintained in order to continue performing reliably. Maintenance should generally be in accordance with the equipment manufacturer’s recommendations. This is an area where some sites fall down: They consider shutting down emergency power systems to be a facility operations problem.

Considering the difficulty of scheduling EP shutdowns, why should they be maintained? The answer is simple: Maintenance allows electrical equipment to continue to perform as it was designed. Working “hot” is a dangerous activity and can even compromise the infrastructure—as many unfortunate individuals and facility owners have discovered when unplanned outages are accompanied by destructive short circuits. Sites that do not shut down their power systems for maintenance are actually practicing a high-risk “run-to-failure” or “breakdown” style of maintenance that has no place in the electrical-system arena.

The purpose of rigorous electrical inspection, testing and maintenance is to improve operational reliability by finding and fixing incipient failures before they occur. Facilities that do not maintain their electrical systems have higher equipment failure rates. In fact, lack of maintenance has been cited as the most common cause of EP system failures.

Fuel-oil system maintenance is another area where EP system reliability can be improved. NFPA 110 Annex A.7.9.1.2 includes a comprehensive discussion on fuel-oil contamination issues and maintenance recommendations. This area will become more important as fuel-oil storage capacity increases.

Remember that electrical-distribution system failures occur for many reasons. Chief among them are:

• Aging systems;
• Human error;
• Construction/renovation activity;
• Deferred maintenance (or plain lack of maintenance);
• An unwillingness to accept planned shutdowns; and
• The combination of conductive concrete dust and high humidity.

What proactive steps can you take to help deflect or eliminate potential problems? Take advantage of construction/renovation (C/R) project shutdowns of electrical risers, panels, switchboards, etc., and schedule maintenance on equipment while it is de-energized. This should be discussed with design engineers at the schematic design stage to ensure that shutdown planning allocates sufficient time for maintenance to be conducted. The lessons learned (i.e., temporary wiring) from C/R shutdowns should be documented for future emergency planning.

Testing and training
NFPA 110 requires weekly inspections, monthly load testing, annual load testing for some generators and extended-run load tests every 36 months. Continuous quality improvement in the EP-system management process can include the cross-training of testing personnel and providing ongoing competency training for maintainers and users. Effective training would incorporate findings and lessons learned from the testing program.

Competency training for EP-system maintainers can include responses to various internal failures, responses to simultaneous multiple utility failures, operation of different equipment during tests instead of just the same equipment every month and understanding EP-test-related interactions with other systems and equipment.

NFPA 110 also requires that every transfer switch be exercised monthly. Despite this mandate, some operations still do not regularly exercise all of their transfer switches due to user resistance. This approach is a mistake because it masks potential latent defects that can reduce operational reliability during utility outages. Rigorous EP-system testing catches failures before normal power outages occur—which is when we want to catch incipient failures. Rather than being problems, failures during testing are valuable opportunities to correct deficiencies that would have occurred anyway during the next unanticipated normal power outage. These failures are not problems; they allow the facility to avoid much worse future problems.

Written test procedures increase EP operational reliability because they can reduce the potential for incorrect actions by testing personnel and provide documentation for later reference and analysis. Analysis of monthly testing considers the second-order consequences of that testing and follows up on lessons learned to discover incipient failures and take corrective action.

Use databases or spreadsheets as management tools to analyze your EP-system testing. Doing so will improve EP reliability by highlighting surprises and hidden trends in the interactions between the EP system and its loads, not just the engine mechanical parameters. This, in turn, can help identify training and systemic issues requiring further investigation or resolution.

Keep in mind that maintenance can help design for EP system testing. Tell your design engineers how your tests are conducted, what problems sometimes occur, what lessons have been learned from the testing and what design features would make them easier.

Emergency management
There have been hundreds of lessons learned from the natural disasters and other events of the last two decades—and they’re clearly too numerous to list here. The referenced 2009 ASHE monograph [Ref. 1] lists many of the lessons that affect EP-system reliability. By applying them to the management of your own EP system(s), you can boost system reliability.

Proactive risk assessment of internal electrical failures is best accomplished before they occur. Such planning should consider different failure points—and not just at the normal power mains or generators. This is an essential part of disaster planning because, in many cases, the responses will be different for each type of failure, and it will be too late to formulate a response after the failure has occurred. Thus, your planning should include not only the generators, but also the transfer switches, paralleling switchgear, risers and branch panels or motor control centers.

Performing an EP-system vulnerability analysis improves reliability by identifying vulnerable areas to address. The issues these analyses take into account include (but aren’t limited to):  system and equipment condition, placement, capacity, failure history and redundancy and susceptibility to potential external and internal disasters; staff preparedness and training; accuracy of documentation; and potential for common mode failure.

Conclusion
Boosting EP system reliability calls for a holistic approach. Think of the recommendations in this article as a prescription for healthier emergency power. As a maintenance and reliability professional/facility manager, you have more influence on the well-being of these critical systems than you might have thought. Be proactive. Make your concerns regarding system maintenance heard during the specification and design process. Moreover, never let emergency power become an “out-of-sight/out-of-mind” matter at your site. MT

A senior consultant at Smith Seckman Reid, in Nashville, TN, David Stymiest specializes in facilities engineering and regulatory compliance. He also has a background in health facilities management. E-mail: DStymiest@SSR-inc.com.

References
Need more information on emergency power system reliability? The author used the following resources in the preparation of this article.
1. “Managing Hospital Emergency Power Systems – Testing, Operation, Maintenance and Power Failure Planning,” David Stymiest, ASHE Management Monograph, American Society for Healthcare Engineering, Chicago, 2009, www.ashe.org. [Note: this document contains references to several dozen additional documents pertaining to this topic.]
2. NFPA 70-2008, National Electrical Code®, (NEC®) National Fire Protection Association, Quincy, MA: NFPA, 2008, www.nfpa.org.
3. NFPA 110-2010, Standard for Emergency and Standby Power Systems, National Fire Protection Association, Quincy, MA: NFPA, 2010, www.nfpa.org.
4. “How to Prepare for Extended Generator Runs, John Diamond,” Electrical Construction and Maintenance, July 1, 2000, © 2007 Penton Media, Inc., http://ecmweb.com/mag/electric_prepare_extended_generator/.
5. “Full power – Guarding against electrical malfunctions with preventive maintenance,” David Stymiest, Health Facilities Management, Chicago: Health Forum, Inc., November 2004, http://www.hfmmagazine.com/hfmmagazine_app/jsp/articledisplay.jsp?pf=true&domain=HFMMAGAZINE&dcrpath=HFMMAGAZINE/PubsNewsArticleGen/data/Backup/0411HFM_FEA_Electrical
6. “A Practical Guide to Electrical Reliability, Wally Vahlstrom,” Electrical Construction and Maintenance, New York: October 1, 2004, ©2007 Penton Media, Inc., http://ecmweb.com/mag/electric_practical_guide_electrical/.
7. “Ideal Reliability,” Douglas E. Stover, Consulting-Specifying Engineer, New York: © 2007, Reed Business Information, a division of Reed Elsevier Inc. All Rights Reserved, July 2005, http://www.csemag.com/index.php?id=1398&cHash=081010&tx_ttnews[tt_news]=24000.

NFPA Disclaimer
Although the author is Chairman of the NFPA Technical Committee on Emergency Power Supplies, which is responsible for NFPA 110 and 111, the views and opinions expressed in this article are purely those of the author and shall not be considered the official position of NFPA or any of its Technical Committees and shall not be considered to be, nor be relied upon as, a Formal Interpretation. Readers are encouraged to refer to the entire texts of all referenced documents. NFPA members can obtain staff interpretations from www.nfpa.org.