AR Accelerates Efficiency

Connecting humans and machines through augmented reality is changing the face of industrial manufacturing.

When digital content is overlaid on a piece of machinery, the equipment comes to life. Detailed instructions pop into focus to guide the operator, helping him/her to identify and fix any problems. Animated 3D renderings, in exact proportion, explode into view and visually show the operator how to repair and maintain the machinery through step-by-step visual aides. Like something from “The Terminator” or “Minority Report” movies, the operator can instantly see the digital content directly in front of him.

This is one example of how augmented reality (AR) is revolutionizing the efficiency of industrial operations. Thanks to this technology, paper manuals will soon be as ancient and obsolete as a rotary phone mounted on a wall. No longer will operators need to search and research to find answers to technical questions. With AR technology, solutions to their problems literally appear before their eyes.

Through a digital device [goggles, a head-mounted display (HMD), or a smartphone or tablet], augmented reality allows humans to “merge” with the machines. The technology recognizes the equipment and understands how it works. It then springs into action with real-time solutions.

“We use augmented reality to combine the digital world with the physical world,” said J.P. Provencher, vice president of manufacturing strategy and solutions for PTC (ptc.com). The Boston-based global software company delivers a technology platform and solutions to help companies design, manufacture, operate, and service strategies for a smart, connected world.

The Evolution of AR

Augmented reality is different from virtual reality, although they are often confused as being the same thing, Provencher said. “In virtual reality you are totally immersed in a virtual world where you can only see the virtual data. Augmented reality is about overlaying digital information on top of the physical world. Whether it’s manufacturing key performance indicators or instructions on how to maintain and operate the product at the right scale and the right orientation, the information is directly overlaid on top of the product.”

Jennifer Richey is the chief strategy officer and one of the founders of Gravity Jack Inc., Spokane, WA (gravityjack.com), a software company begun in 2009 on the vision of then-infant AR technology. “If you really look at the simple definition of AR, which is layering digital information on top of physical things, you could say it’s been around 20 years,” Richey said. “The scrimmage line and first-down line on a television football field is a great example of augmented reality. It’s all about placing digital content onto a live viewing scene.”

Manufacturers are now beginning to pay attention to the return on investment that digital solutions can provide. “One of the hardest things in the earlier years was approaching a company, and usually they didn’t already have an AR line item on their budget,” Richey said. “Now, companies are at least talking about allocating funds to the digital-experience realm.”

Jennifer Richey, GravityJack.

If you’re not using it now, augmented reality is likely in your future. As you consider how it can help you, it’s inevitable that you’ll ask, How do we get started? In this podcast, Jennifer Richey, chief strategy officer at augmented-reality specialist Gravity Jack, Spokane, WA (gravityjack.com), helps you identify what’s needed in terms of hardware and, most important, how to identify and develop the applications you’ll need to be successful. It’s a discussion we’re confident will give you enough information to ask the necessary questions and realize success with your first augmented-reality project.

For the past few years, assisted reality—a less-immersive flavor of AR—has been most prevalent in manufacturing, according to Aaron Tate, vice president of solution strategy for Upskill, Vienna, VA (upskill.io), an enterprise software company focused on AR devices and experiences in industrial settings.

“Assisted reality enables the most proven, repeatable use cases, such as complex assembly, remote assistance, and workforce training, and is often used in settings that require several consecutive hours where the worker is wearing the device on his or her head,” Tate explained. “New immersive experiences are emerging (such as mixed reality), as are new devices, giving manufacturers a breadth of options for bringing AR to their workforces.”

For example, Upskill’s core offering is its flagship Skylight AR platform, which allows hands-on workers to seamlessly interact with a simple, intuitive software interface available directly in their line of sight, using smart glasses, smartphones, or head-mounted displays. Users can receive step-by-step workflow guidance, capture and share knowledge, visualize reference documentation (pdfs, videos, images), and call in a live, remote expert to assist with mission-critical tasks.

Augmented reality is most effective in the industrial manufacturing space when it is used to connect operators/maintenance technicians and assets and visually indicate the condition of various components.

AR in Manufacturing

One of the most effective ways AR is being used to improve efficiency in the industrial manufacturing space is in the connection between the operators/maintenance technicians and assets. “The operator gets an alert that a machine has a problem,” Provencher explained. “AR allows him to point a device that has a built-in camera in front of the machine. He sees the machine through the eyes of the camera and can see digital information overlaid on a real-time video feed. A visual indicator sensor physically shows him what is causing the problem on the machine. Even if he walks around the asset, the sensor is always positioned at the physical location of the problem on the machine. The technology then delivers a digital, guided, step-by-step instruction of how to disassemble the machine to get to the sensor and also the steps to actually solve the problem.”

Provencher said that some panels turn into a different color. Then the panel can be removed using a 3D digital assembly directly on top of the machine at the right scale and at the correct orientation.

Teams now can create digital AR instructions rather than paper-based instructions. “We have developed tools in the past 30 years that allow customers to take a complex 3D assembly model of a machine and create exploded views and animations and combine those with the augmented-reality solution,” Provencher explained.

Hand-held devices and wearables are used to provide the human user with access to the digital content overlays, Richey said. “In the industrial space, workers are generally used to wearing a hard hat, so it’s not so uncomfortable. It’s so simple and user friendly that my grandma could put on a HMD and walk through any kind of training, repair, or procedure because the AR opens up this entire new world.”

Depending on the complexity, AR implementation generally takes a couple months. Customers begin with a proof of concept—an area where they want to validate the speed at which they can create AR experiences. “We make the platform available as a pretrial for customers so their subject-matter expert can work with it,” Provencher said. “If you are a machine builder and have the 3D CAD data of your machine, it can take only a couple days to create an initial AR experience to be tested.”

AR can also be an effective tool for training new technicians and designing plant upgrades. “The guys on the floor no longer need to memorize thousands of processes and how-tos,” Richey said. “If you want to add to the plant and add new equipment, you can use AR to place it right into the scene to determine if it will fit, how it will function with the flow of the other equipment, and whether it provides an efficient workspace.” 

Machine builders are also incorporating AR manuals. “If you are an operator of equipment, you typically must create all kinds of standard operating processes for the workers who operate the machine,” Provencher said. “Instead of building in a Word document or Power Point, you can build it in AR, which can significantly increase the productivity of your operators. We have customers who are increasing productivity by as much as 40% by delivering AR instructions rather than a paper-based manual.”

Instead of generating Word or PowerPoint files, machine builders increasingly incorporating augmented-reality manuals with their equipment. AR instructions have been found to increase productivity by as much as 40%. Photos: Gravity Jack

Case Examples

Some companies are using augmented-reality devices such as Microsoft’s HoloLens technology. This year Chevron doubled its inventory and now uses more than 110 HoloLens sets to reduce operational downtime and improve productivity by “beaming in” technical experts from remote locations around the world. The HoloLens goggles serve as holographic computers for the workers, allowing them to view a video-call screen projected within their field of vision.

PTC developed a similar AR platform called Remote Expert Guidance—an entry-level version of AR in which one can enable a Skype session with a remote expert. “If you are a junior technician in front of complex equipment, a subject-matter expert in an office or even a different factory can conduct a Skype session in which the technician shares the live video feed,” Provencher said. “The expert can add annotations directly on the video feed. Because it’s AR, the platform recognizes the physical object and, as the technician moves around the equipment, the annotations made by the expert stick to the physical areas.”

The Boeing Co. uses Upskill’s Skylight on Glass Enterprise Edition, and has dramatically improved its complex wiring-harness-assembly process. “With thousands of miles of wiring in a single jetliner and no room for errors, Boeing sought a more effective way to help workers configure these wires,” Tate explained. “Previously, technicians relied on large, paper manuals and instructions on laptops. But these were cumbersome and created the same basic problem, constant look-away interruptions as workers received directions and cross-checked diagrams and schematics. Now, Skylight delivers instructions right in the workers’ field of view and allows them to keep their hands on their work at all times.” As a result, Boeing has increased productivity by 25% and reduced errors effectively to zero.

Another industrial use example involves The Coca-Cola Co., which is using Skylight for equipment inspections, service calls, and routine audits in plants around the world. “Technicians wearing smart glasses can stream, in real time, what they see and hear to a subject-matter expert watching on a computer or mobile screen thousands of miles away,” Tate said. “Skylight saves nearly $10,000 every service call by eliminating the need to fly an expert onsite to resolve maintenance issues. This also reduces plant machine downtime for greater throughput and higher-quality outcomes.”

Why Incorporate AR?

AR helps facilities become more efficient by empowering hands-on workers. “By connecting these workers with the instructions, reference materials, and real-time information needed to do their jobs, technicians with no prior experience completing a complex task can quickly match the performance of veterans and even work faster with fewer errors,” Tate said. “This not only helps close the skills gap, but it also increases the value each worker adds to the business without disrupting their routine or requiring burdensome training.”

Manufacturers are trying to ensure that their products and their factories are connected, Provencher stated. “A lot of vendors are talking about self-healing and self-controlled robots. But we believe that the Internet of Things is not only about the
machine, it’s also about the humans. As equipment and machines are becoming more and more complex, we are asking workers to keep track of more and more data. We believe AR is the way to help workers keep up with the complexity of manufacturing operations.”

Simply put, augmented reality increases efficiency, Richey added. “Manufacturers need to be implementing their digital strategies proactively and not retroactively. Some manufacturers are using AR to display a product or show a procedure, which is great. They are dipping their toes in and pursuing the technology, but are they really investing the amount of money and time to get the maximum return? They could invest $400,000 in technology that could save them $1.5 million in time and effort. They still are seeing a return on investment, but perhaps not the full potential of the ROI capability.”

Certain AR use examples require a larger viewing experience, as connected workers need to browse, access, and interact with multiple screens and large-form content (such as pdfs), at once and in a real-world context. This is why companies such as Upskill incorporate mixed-reality into the AR experience.

“Essentially, mixed-reality experiences allow a worker to flip through and cross-reference these materials on a larger screen, using available real-world space to place, move, and ‘tuck away’ windows as needed,” Tate explained.

Manufacturers are preparing for the advent of 5G, which will enable faster computing and richer content for AR devices. Although manufacturing jobs are growing domestically, the question remains: How will organizations get all of these new workers up to speed and producing efficiently and in a cost-effective manner?

“AR is increasingly recognized as a viable tool to more quickly train and upskill workforces through immersive learning,” Tate said. “This is a trend we will continue to see as manufacturers look to close the skills gap, and replace retiring Baby Boomers, armed with valuable tribal knowledge, without missing a beat.”

Using augmented reality to build wiring harnesses for Boeing’s aircraft has increased productivity by 25% and effectively reduced errors to zero. Photo: Upskill

The Future of Augmented Reality

With 5G on the way and the hardware ecosystem evolving, the Upskill company is preparing for even more immersive experiences with AR. “While the technology may be here, the challenge will be helping customers create the content needed to support those experiences at scale and upon request,” Tate said. “With the right hardware, software, and content, we’ll see more enterprises not only deploying AR, but also scaling it company-wide as part of the ‘last mile’ of their digital transformations.”

Augmented reality, holistically, will be the future way that we compute, Gravity Jack’s Richey said. “We eventually will not have mobile phones. Screens are going to go away. What is that future going to look like? Now, it’s big, huge wearables. Then it will move to goggley things. Then it will be glasses the size of Raybans. Then it will be more like a contact lens. Then there will be devices to put the input into the eye without wearing a lens. For the consumer, they need to start looking for it. The more consumers speak the technology, the more manufacturing companies will provide it, and the more digitally enhanced our over-arching world will be,” she explained.

Provencher agrees that AR will be a huge factor for the future of manufacturing. “AR is ready for prime time,” he said. “One way to look at AR is to not necessarily think about the wearables and the Google glasses. A lot of our customers are more practical and pragmatic and they start with a phone and an iPad, which still deliver tremendous value. It doesn’t have to be fancy to get results.”

Once manufacturing gets on board, Richey said, the sky is the limit. “If you can think ‘Iron Man’ and his display on his screen, this is AR,” he said. “We will have that someday, but for now, the money should be invested in industrial manufacturing because you have direct ROI. For example, it may take 42 minutes to perform a procedure with current manuals. With HMDs and AR, a trained worker can perform the same procedure in 31 minutes. The direct money savings is real, and it’s accountable.” EP

5 High-Value AR Trends

Global software company PTC, Boston (ptc.com), performed a survey among its customer base to identify five key augmented-reality (AR) trends in three categories:

Service/maintenance

Step-by-step procedural guidance:Manufacturers have more and more complex equipment, but less and less specialized maintenance and service teams, Provencher said. “This means technicians need to perform maintenance on a larger number of assets and without detailed training. AR allows the technician to understand exactly what needs to be performed on equipment without requiring a lot of training.”

Remote expert guidance: This is an entry-level version of AR in which a Skype session can be enabled with a remote expert. “The secret sauce here is that AR recognizes the physical object through a live video feed. It understands the orientation of the device so we can overlay digital information at the right scale and the right orientation.”

Manufacturing

Machine set-up and changeover: The work of the operator on a complex assembly line, such as in food-and-beverage manufacturing, requires a changeover for anew batch. If the clean-in-place (CIP) procedure is not performed properly, the next batch can be wasted. “Today operators are using all sorts of to-do lists and checklists to ensure proper CIP,” Provencher said.

Step-by-step instruction guidance: Rather than looking to the side at a manual, you look in front of you through a wearable (glasses with camera pointing at station or iPad) to show you how to perform the assembly.

Marketing/sales

Customers use an application rather than an instruction manual. “For example, Mercedes uses this,” Provencher said. “You point your phone at the dash in the car. Numbers will appear directly on top of the buttons. The manual is delivered in context directly to your phone. It recognizes what you are looking at and delivers the proper instructions.”

Michelle Segrest is president of Navigate Content Inc. She specializes in creating content for the processing industries. If you have an interesting efficiency, maintenance, and/or reliability story to tell, please contact her at michelle@navigatecontent.com.