How To Reduce Automation Obsolescence Risks (Without Losing Your Mind)

Don’t wait until it’s too late to think about managing the life cycles of your automation investments. Your operation’s bottom line is at stake. 

By Lonnie Morris, Rockwell Automation

From a constant stream of mobile-device releases to the explosion of cloud applications for all areas of business, technology advances are occurring at such an exhausting pace these days, it can be easy to become paralyzed by the possibilities. It’s intimidating to think how an organization—especially one in a production environment dealing with capital budget constraints, global competition, rising maintenance costs, less access to skilled labor, information security and safety concerns—can plan quickly enough to leverage such changes for competitive advantage.

Before a production facility can even consider trying to justify emerging plant-floor technology, it’s generally necessary to maximize the life of its existing automation investments. Managing aging equipment is an often-overlooked component of this: In a 2010 study by ARC Advisory Group (ARC), over 90% of process manufacturers acknowledged the use of automation beyond the OEM’s obsolescence date. 

In the same ARC study, 58% of users acknowledged having no formal plan for managing the life cycle of their equipment. Production facilities that continue to run aging equipment without a plan for addressing and managing end-of-life technology face a variety of risks that threaten to drastically increase downtime and decrease profitability should legacy systems fail:

• Supply chain fragility. A number of control systems that were designed and launched in the 1970s and 1980s are still in use today.  Shifts in technology, diminishing demand and economic impacts have made some vital components and subassemblies used in these systems hard to come by—some are in short supply and others are completely extinct.
• Support challenges.Difficulties locating legacy products can be further compounded by maintenance challenges—not only is maintenance expensive, attrition among subject- matter experts brings an added layer of complexity.
• Regulatory restrictions. Safety regulations and rules regarding hazardous substances (e.g. the Restriction of Hazardous Substances directive) have expanded significantly in recent years, leaving many older products at risk for compliance.

Mitigate or eliminate?
Operating legacy equipment beyond its obsolescence date will always carry a degree of risk. But by identifying and quantifying this risk, production facilities can determine whether to mitigate the risk until scheduling and/or capital funding becomes available, or to eliminate risk through product migration.  Here are some critical questions to ask in determining the best path forward:

• How long do you intend to operate equipment?
• Is support readily available?
• Is capital funding available?
• Is applicable new technology available?
• What are the migration implications related to spare parts, training, support, etc.?

Considerations for mitigation
In many cases, mitigating obsolescence risk is the best option given budget and resource constraints. Though specific plans for effectively mitigating obsolescence risk may vary from one organization to another, best-practice plans contain four basic pillars:

1. Conducting consistent preventive maintenance…
According to ARC, there is currently more than $65 billion worth of legacy automation assets reaching the end of their useful life. And though some would like to think these systems can run and be serviced indefinitely, it is simply not possible.

After operating a PLC-5® for over 20 years, for example, maintenance might feel the unit is bulletproof, and since it has run so long without a problem that it will continue to do so. But as it does in our own lives, age wins in the end, and serviceability may be limited or even non-existent.

As equipment reaches the end of its useful life, age and wear take their toll. Failure rates drastically increase, as do maintenance costs (Fig. 1). Plants are perpetually one major part break or machine failure away from a shut down, and legacy equipment is more susceptible to these sorts of hiccups. As a result, preventive maintenance becomes absolutely essential.

To drive consistent preventive maintenance of discontinued products, maintenance engineers need to routinely ask questions like these:

• Are the filters replaced regularly?
• Is the current operating environment within OEM specifications?
• Are cooling fans operational and clear of obstruction?
• Does the equipment possess the latest firmware update?
• Is there an updated logbook documenting inspections of obsolete equipment?
• When was grounding last checked?

After getting the answers, the maintenance team needs to remedy problems and correct any irregularities. This, unfortunately, is often easier said than done…

Most organizations realize the importance of preventive maintenance, but many simply don’t have the tools or the organizational bandwidth to do it. Finding a way to get it done—either by outsourcing or reprioritizing—is paramount to extending the useful life of equipment. Systems that aren’t properly maintained are much more likely to fall victim to extended downtime and lengthy shutdowns.  Developing an enhanced preventive maintenance program is vital to maximizing the life of legacy automation.

2. Training to support legacy equipment…
Preventive maintenance activities will only be productive if performed by personnel with the know-how to handle the machines they are maintaining. It’s not uncommon for a production facility to be running equipment that’s more than 20 years old, and most of the people that designed and installed it have moved on from the department or organization. Often, these experts have been replaced by younger engineers that can’t be expected to hold the same level of knowledge on legacy equipment.

Developing a training program to address these gaps is critical, but according to ARC, 58% of companies have faced difficulties in training young engineers and technicians to operate and maintain older control systems. Effectively training staff to maintain legacy products can be even more challenging because it requires a great breadth of knowledge. Employees need to know how to maintain all legacy products—this includes installation, configuration, programming, maintenance, diagnosis, troubleshooting and repair. 

An undertrained workforce can have significant business impact on performance and metrics like downtime, on-time deliveries, vendor support expenses, overtime, scrap rate, programming costs and maintenance. In fact, the United States Department of Labor estimates that raising the educational level of employees by one year results in eight to 13% higher labor productivity. If the in-house staff doesn’t have the time or expertise required to teach new talent, it’s valuable to invest in an outside resource that does.

3. Planning for spare replacement and legacy repair support…
Just as important as having the right people is having the right parts—an ineffective spares-management process can extend downtime by 15 to 35%, and legacy products can make this statistic even worse.

MRO (maintenance, repair, overhaul) operations are constantly seeking opportunities to cut carrying costs by reducing inventory. To do so, they examine the turnover rate of each part. During this process, internal financial entities often pressure the department to get rid of parts that aren’t turning over quickly. While this approach makes intuitive sense, it is the opposite of what should be done, especially with legacy automation equipment.  

As availability of legacy parts begins to dry up on suppliers’ shelves, firms that rely on them need to carry extra inventory to counter potential shortages. When they don’t, the result is often less than ideal—some production managers have been known to consider purchasing unverified spare parts on eBay. Allowing a machine’s health and, ultimately, the company’s profitability to depend on someone’s last eBay purchase is not a sound business model.

A spare-part-replacement strategy should involve more than just stockpiling parts. It’s a process that involves:

• Calculating the optimal amount of spare parts
• Determining the condition and supply of the spares
• Identifying spares of legacy equipment
• Identifying a trustworthy and timely supplier of legacy spares

Because of the complexity that can be involved in effectively managing spares for legacy equipment, companies sometimes choose to leverage a third party to manage the process. Parts-management programs can help reduce inventory and carrying costs, and can help provide more immediate spare parts availability. This was the case with the City of College Station, TX, when it decided to leverage a vendor-managed parts-management agreement after a recent process-control system upgrade. Since a vendor owns and manages the inventory, College Station is able to budget a monthly fixed cost. Previously, the City’s control system stocked spare parts inventory was about $250,000, but it was able to reduce it to less than $20,000 while also significantly lowering the square-footage required for onsite inventory storage. A resource that understands the intricacies of the process can help ensure that the right parts are in the right place at the right time.

4. Managing obsolescence status…
Further complicating risk mitigation is the fact that it’s in a constant state of flux. Within a year or two of conducting a comprehensive audit of the installed base, production facilities often start to discover products that were once active have become discontinued. If risk isn’t being continually assessed, it’s much more likely status changes and threats associated with them will be missed.

To prevent the above situation, companies need to establish a process for monitoring life-cycle stages of equipment. This should include developing a database and assigning subject-matter experts within the organization to collect and maintain all life-cycle information. Vendors also can help provide information around life-cycle statuses, parts and service availability and migration recommendations that align with business goals. 

The bottom line
While most facilities agree that better management of automation life cycles is becoming critical, few programs are in existence and most are yet to get off the ground. One reason is that most facilities don’t have the personnel for it. In most cases, there isn’t a role specifically responsible for creating, monitoring and updating a database. Consequently, these tasks are often overlooked.

Now is the time to develop a life-cycle strategy for your automation assets (before it’s too late). Lack of resources to invest in dedicated personnel is no excuse for neglecting this process. Enlisting the help of third-party experts can double the benefits of risk mitigation—saving operations the time and money required to train in-house personnel and keeping these resources from being squandered on unnecessary downtime. MT

Lonnie Morris is a Senior Manager with Rockwell Automation.

Creating An Obsolescence Plan

The overarching goal of obsolescence planning is to quantify risk in order to determine whether it makes more business sense to mitigate through maintenance and support or eliminate it by migrating systems. A phased approach can help ensure risk is addressed effectively and efficiently: 

Identify. Define the goals and scope of the obsolescence plan, and develop a strategy for safely collecting data on all legacy equipment. Dedicating qualified resources to creating the plan generally nets the best results.

Research. Collect product life-cycle status information by reviewing manufacturer Websites and notifications, publications, and distributor and reseller information. Identify inventory gaps and orphans by comparing MRO spares versus the installed base and by comparing repair activity with replace activity.   

Prioritize. Aggregate data to highlight areas of greatest obsolescence concern. Using that information, design a plan to accept, mitigate and eliminate obsolescence risk. Next, develop a process for gathering ongoing life-cycle status changes for all installed products.