Virtual and augmented reality

Movies such as “Tron,” “The Matrix” and “Strange Days” introduced the world to a new and sinister virtual reality. Fortunately, the latest virtual and augmented reality tools being introduced into the workplace have a more benign purpose.

These tools are training new hires much faster in complex engineering actions and shop-floor assembly tasks while helping engineers and designers devise new solutions. And they are being harnessed in the field to simplify inspection, add precision to routine inspections and to assist troubleshooters in isolating root causes.

Baker Hughes GE, Elliott, Siemens, Howden, MAN Energy, MTU Maintenance, GE Aviation and others are already using augmented reality tools in their offices, factories and energy plants. Daqri, Microsoft, Ubimax, PTC, Upskill, RealWare, HTC, Glass, Atlas Copco, Worklink are among the experts producing them.

Virtual definitions

Virtual reality (VR) immerses the user in a digital environment on a computer screen or other digital device that is a version of real life. The latest versions of VR allow the user to roam through a virtual world and create realistic-seeming motions, scenes and experiences.

Augmented reality (AR) deals with overlaying virtual objects on top of a real-world environment. PDFs, diagrams, lists of instructions and other aids are superimposed on a device, such as eyeglasses, so the user is assisted in viewing, understanding or carrying out a physical action.

Mixed reality (MR) is an extension of AR that goes beyond mere overlays of data or documents to anchor virtual objects to the real world. In other words, virtual data is tied to the physical object in such a way that as the user moves, the virtual object adjusts to continue in alignment with the physical object. AR and MR are often used interchangeably.

Worldwide spending on AR/VR was around $18 billion in 2018, almost double the previous year, according to International Data Corp. (IDC). Commercial deployments represented more than 60% of the last year’s total and will account for more than 85% of the market by 2021.

On-site assembly, safety, process manufacturing training and industrial maintenance are among the most widely used tools in the manufacturing sector. Field service, too, is a popular application of AR.

By next year, IDC predicts that 25% of field service personnel will use the technology, with the oil and gas sector leading the way in the number of deployments. About 14.4 million U.S. workers will wear smart glasses by 2025, up from around half a million today according to analyst firm Forrester Research.

AR tools provide those in the field or on the shop floor the ability to access step-by-step instructions on repairs while performing work. They reduce the number of human errors, execution times and downtime and lessen the number of breakdowns, while improving worker productivity, fix rates and profitability. In addition, maintenance and assembly processes can be more easily documented and reviewed by supervisors.

AR can help new employees move through complex inspection, assembly and maintenance tasks, helping them attain competence faster than wading through manuals or tagging along with a veteran.

“As equipment becomes more complicated, problem isolation and diagnosis increase in difficulty,” said Ralph Rio, an analyst at the ARC Advisory Group. “Obtaining a high, first-time fix rate requires good tools to determine the true source of equipment failures.”

Rio explained that sensors inside equipment can provide technicians with process data, such as the temperature and pressure of materials in a pipe, or equipment data, such as vibration. By displaying these parameters, it is easy to see what is going on at a glance.

In the case of employee inexperience, the AR device can be hooked onto the screen of a remote expert who guides the user through the necessary actions. Having this take place via 3D headsets, such as Daqri Smart Glasses and Microsoft HoloLens, is less cumbersome than trying to set up a Skype video or conference call. The remote expert can annotate the display for the user and have that annotation remain aligned to the equipment even when technicians move around.

In all likelihood, OEMs will begin offering premium support services that enable access to remote experts via AR. This could speed field service and avoid delays while service providers deploy personnel to distant locations for inspections and repairs.

Meanwhile, OEMs in the turbomachinery sector are using AR or VR in a variety of ways.

Elliott

Dan Butler, Application Engineer at Elliott, said his company has focused on VR via an HTC Vive headset along with software from Unreal Engine and an Alienware laptop.

“By using VR for internal design reviews, we can more easily and quickly detect potential manufacturing issues before the equipment hits the shop floor,” said Butler. “We also have provided presentations to customers using VR, which gives them a much clearer understanding of the equipment layout and functionality.”

Viewing things in VR gives a better representation than looking at a 2D drawing or even a 3D model on a computer screen, he said. The viewer is given a clearer sense of the scale of equipment.

As a result, designers can ensure such things as valve handles are easily accessed. Within the VR system, the designer reaches out a hand to determine accessibility. You can also judge if gauges are at the proper level by virtually standing at the right location.

So far, Elliott has not used any of this equipment in the field. But early experience indicates that it helps to identify potential issues faster. No training is required, only a quick explanation of the controls, said Butler.

MTU

MTU has an AR pilot project known as “Inspection 4.0” running at its facility in Berlin-Brandenburg in collaboration with the Brandenburg University of Technology Cottbus-Senftenberg. It has two main goals:

• To centralize all data regarding a specific part into one system and improve knowledge management in the shop. This has the benefit of reducing the amount of time mechanics spend switching between documents and enabling them to focus on inspecting the part in question

• To converge the real and virtual world of maintenance.

“Data will be transferred to tablets or smart glasses so that the mechanic has a complete overview at all times,” said Dr. Friedhelm Kappei, Head of Industrial Engineering, MTU Maintenance.

Siemens

Siemens SGT-8000H combustor inspection virtual maintenance trainer (VMT)visualizes the hardware, tools, environmental health and safety requirements, and the sequence of steps to disassemble and assemble a burners on an SGT-8000H.

Trainees learn via tutorial mode and can repeat the training as often as necessary before starting and completing an assessment. A successful assessment is documented and is required before trainees can participate in practical training with the actual turbine. This has reduced training time on site and at Siemens Berlin training center.

“Trainees are better prepared, practical training is faster and the time saved is used for more exercises and additional scope,” said Michael Stavenhagen, Siemens Gas & Power, Head of Digital Training Center. “VR/AR can also save in travel costs as collaboration can take place in a virtual environment.”

As technicians often have to deal with situations onsite where additional support is required, live audio, video and AR software collaboration are available from remote experts. Siemens Omnivise Remote Service supplies field workers with the knowhow to monitor, maintain and repair assets using cloud-based communication tools.

Benefits include:

• Increased productivity by helping to improve uptime to immediate expert support

• Enabling capabilities to support customers where travel restrictions apply

• Extending accessibility of experts by avoiding travel time

• Improved training effectiveness through availability of virtual information sharing

• Improved documentation through enhanced information sharing

“VR solutions focus on training, sales and digital twins,” said Julian Zingel, Head of Marketing & Sales for Siemens Omnivise Remote Service. “AR solutions focus on enhancing field service and maintenance at site.”

BHGE

Baker Hughes, a GE company, (BHGE) uses VR and 3D laser scanning and modelling in different phases of rotating equipment design, ergonomics validation, training and outage excellence optimization. VR and 3D support features, such as rotation of components to focus on details, zoom in, zoom out, and viewing the model from different directions, as well as highlighting interference between different simulated bodies. This offers more accuracy and clarity during the evaluation and design review phases.

By combining CAD and laser-scanning tools for machine design with a virtual plant scenario, BHGE has improved the effectiveness of the digital engineering process, risk has been reduced and productivity improved.

Unexpected installation hurdles have been avoided at an early stage, leading to safer and more effective project implementation. VR tools have also been used to design for maintainability, to train people on maintenance procedures and for outage excellence optimization.

“Virtual reality offers great opportunities to maximize production and enhance the safety of operations when maintaining or serving existing plant and installed turbomachinery equipment, as well as in outage optimization,” said Alessandro Bresciani, Vice President, Global Services Turbomachinery & Process Solutions, BHGE. “In preparation for planned maintenance, the entire operation can be simulated virtually, identifying and addressing potential bottlenecks in advance, as well as training personnel. Effectively and accurately laying out an outage can help ensure that the work is undertaken efficiently and safely.”

In preparation for a shutdown, technical surveys and outage simulations help to minimize issues in the execution of an outage. Layout is a crucial part of maintenance planning as sites involve the coordination and movement of large quantities of materials as well as lifting and logistic devices and people.

VR enables all items to be positioned on the virtual deck and a virtual simulation can be performed to ensure environmental health and safety requirements are met and the spaces are properly optimized for the job to be performed. 3D drawings and videos can then be passed to a manpower contractor for proper layout area preparation a few days prior to the outage.

In addition, high-definition 3D laser scanning technology is used to capture the existing plant configuration in the form of a high-accuracy point cloud. Combined with camera imaging, a point cloud can be rendered to provide a photo-realistic virtual view of the facility, bringing the plant to the project engineer’s desktop as a 3D model. This is a step towards replacing traditional paper drawings.

BHGE has found that projects based on digital execution are more efficient than conventional design methods as they eliminate misalignments among drawings, optimize OEM and EPC contractor work processes and avoid redundancies during engineering development.

Once the 3D environment is ready to be explored, a feasibility study of the maintenance scope can be developed through analyses and simulations. VR adds the ability to check pros and cons of a project or tool, modeling it according to the needs and ensuring suitability for the actual configuration of the site.

VR transfers information in more realistic situations than traditional classroom and workshop training methods, engaging the trainee in a higher level of critical thinking, and providing the ability to see immediate results. It accurately simulates operational scenarios, so personnel are ready to face critical issues with the right preventive and corrective actions.

MAN Energy

MAN Energy Solutions offers PrimeServ EyeTech, a remote support solution based on smart glasses. A head-mounted device enables live and mobile video conferences between experts in remote operating centres and customers on site working on equipment. EyeTech is one part of the Man PrimeServ Assist service which offers 24/7 expertise support.

“The service engineer can see whatever the technician onsite sees and guides him or her what to do next,” said Caroline Horn, Digital Lab Leader Turbomachinery, MAN Energy Solutions Switzerland. “The customer has both hands free and gets all the information needed displayed on a head-mounted screen.”

She added that issues can be resolved faster, travel costs saved, and the availability of machinery improved. For example, a specialist in Europe can solve an issue for a customer in the U.S. without spending time or money for travelling.

MAN uses Microsoft HoloLens glasses in VR training, and Realware HMT-1 glasses for onsite applications, as they are more robust, lighter and more comfortable to wear.

Supplier perspective

Ubimax, a provider of AR solutions, offers the Ubimax Frontline platform to act as software that operates with glasses such as Microsoft HoloLens. As well as existing as in-house software, it can also make use of the Microsoft Azure cloud to obtain data from additional sources.

The HoloLens is well suited to showing PDF files, blueprints and manuals. Multiple windows with different information can be on display right in front of the user’s eyes, enhancing the amount of information accessible when needed.

With its eye tracking functionality, it helps the user navigate through texts. The glasses adapt to reading speed and automatically scroll down as the end of the page is reached.

Ubimax software adds more 3D features to make the experience more immersive. As a result, 3D models of component parts can be touched, grasped and moved in ways that feel natural.

“Ubimax Frontline includes support in maintenance, repair and overhaul of turbomachinery equipment,” said Percy Stocker, President Americas, Ubimax. “With a combination of augmented reality and wearable computing solutions such as smart glasses, the worker gains relevant instructions including safety warnings into the field of view exactly when needed and can use both hands for the task.”

Additionally, workers can document problems via picture or video. Safety warnings and security procedures can be included. Workers can also start video calls with remote experts to get help or receive decisions about finishing a task.

Airbus Helicopters, for example, implemented Ubimax Frontline to improve the inspection processes of helicopter gearboxes. Previously, the worker had to document every step, take pictures and upload them onto a computer.

Further, they had to leaf though manuals. The company achieved a paperless working environment in inspection processes with Ubimax Frontline. The speed of the inspection process was increased by 40% as data is automatically transmitted to enterprise systems.

“Errors through manual input have been reduced to zero because of data digitization,” said Stocker.

She cited further applications of Ubimax software:

• BHGE’s smart helmet is used for machinery inspection at its LNG plant in Qatar. By collaborating directly on-site with headquarters via video calls the number of trips for an inspection process is reduced

• In a BHGE facility in Florence, Italy, AR is used on the factory floor. For quality insurance matters, workers have to make around 100 measurements for gas turbine nozzles and enter them into a spreadsheet. With AR, the next measurement place is highlighted, the measurement automatically recognized and transmitted into the database. Workers are notified when problems occur. This leads to a reduction of errors and a higher process speed

• Siemens Industrial Turbomachinery uses AR for collaboration purposes in inspection, saving travel expenditures and reducing machine downtime.

As well as smart glasses, wrist computers can also be employed to make the job of the inspection worker more convenient. They can execute the tasks with both hands and full concentration, making the job safer, more productive and efficient. Smart glasses can also be combined with safety glasses and hard hats, depending on the working environment.

Stocker said it takes an hour or two to get used to wearable and AR technology. “After one to two weeks of on-the-job training, they are fully used to the technology and see an increase in productivity.”

Daqri’s smart helmet lets engineers see 3D images above assets and prompt them with instructions. Siemens partnered with Daqri on its gas burner assembly training application.

A proof-of-concept study helped to better understand how AR-based training affects assembly efficiency at the Siemens Power Service Training Center in Berlin. Four participants completed the assembly of a part using the AR application.

Two new employees were able to successful complete the tasks despite no experience. This approach can help OEMs and service organizations to avoid variability in assembly sequence, minimize errors, streamline data collection and make reporting easier.

Daqri Worksense software for smart glasses has a series of versions for different use cases. These include hands-free remote expert guidance and annotation, being able to attach notes or multimedia assets to assets, ways to map out the environment and equipment to create a 3D digital model, multimedia work instructions, and immersive walkthroughs of equipment and plants.

UpSkill Skylight’s AR technology is said to cut production time by 25% for wiring harness assembly operations at Boeing. It is also used by oilfield works. Scope AR’s WorkLink platform works in conjunction with Microsoft HoloLens.

PTC, too, is an AR innovator via its Vuforia platform, which is used in field service, maintenance manufacturing, collaborative design reviews, and virtual work instructions for assembly and training. Its 3D sales aids demonstrate how complex systems and equipment work in virtual reality. This enables potential customers to visualize the product in their own setting before it is built.

AR challenges

Observers agree that it may take a few years before the technology reaches full maturity. Current AR devices can be bulky. Some technicians complain that they are uncomfortable to wear for hours on end.

There are also safety issues. Just as pedestrians engrossed in their smart phones or wearing earphones are liable to walk in front of a vehicle while crossing the street, some worry that operators equipped with AR glasses may lose track of their physical surroundings due to a lack of situational awareness.

Cost is another concern. The current user base is dominated by large OEMs. Ubimax smart glasses for industrial use start at about $1,500. Additionally, a software license of $1,000 to $1,500 per user is often part of the price. And integration into existing systems and 24/7 support can add further to cost.

“Creating an AR application is relatively expensive, but since a typical OEM has many similar machines to service, it has economies of scale to make this a sound business investment,” said Rio of ARC Advisory Group. However, he expects that as the volume of users picks up, significant price declines are inevitable.

Butler of Elliott said VR is already becoming more affordable. He said a full setup including a high-end laptop and headset can be had for under $3,000. He kept costs down by using free software available from the open-source software community.

But one turbomachinery engineering company interviewed for this article said they did not use the technology due to various reasons:

• Some sites have strict ‘no cell phone’ policies near equipment. Refineries and chemical plants, for example, are reluctant to let outsiders use any of their electronics on their premises

• General safety concerns: operating (or wearing) a device may negatively affect worker’s situational awareness to fall, slip and trip hazards

• Remotely located plants may not have good cell phone coverage, and reception quality inside buildings is further reduced due to walls and steel framing

• One user felt it was easier to take pictures on a smart phone than to use AR

Pros and cons

AR and VR, then, may be right for some and not for others. At the moment, initial costs may prohibit many except for OEMs and large aftermarket service providers. But that is likely to change as prices come down and word of successful applications spreads.

Technology maturity is another factor. Over time, form factors will be reduced, systems will simplify further, and network connectivity issues will be resolved. Just as video took years to overcome bandwidth limitations and move into the mainstream, so it might well be with AR.

Additionally, VR and AR seem more suited to complex equipment and systems. For those engaged in relatively simple assembly or maintenance operations, they may not be needed. But the likelihood is that these technologies will earn a place in the toolbox in the near future.