Both virtual reality (VR) and its big brother augmented reality (AR) have been enjoying the status of the next big thing for some years now. In the gaming sector, of course, but gradually also on the shop floor. While both technologies are slowly but surely conquering their place in the most diverse industrial sectors and in healthcare, it seems only a matter of time before the broad consumer market will also take the virtual path.
VR & AR in figures
- In a recent report, consultancy giant PwC estimates that AR and VR could add $1.5 trillion in economic value worldwide by 2030.
- In the US and the UK alone, it is expected that in the next ten years
824,000 new jobs thanks to AR and VR.
- In Europe, there were a total of 748 new start-ups active in AR and VR in the period 2015-2019. They raised a total of EUR 2 billion in investment capital.
The problem is not unfamiliar to avid Ikea travellers: in the shop, that sofa or bookcase not only looks very modern and trendy, at that price it would also be a shame not to take it. After which, back at home, the sobering feeling quickly sets in: the combination of the sofa and the classic dining room does not turn out to be a great success. And the bookcase turns out to be just a bit too big for daughter’s room. To avoid making radically wrong style and interior design choices, Ikea has been relying on cutting-edge technology for a while now.
Thanks to the Place app, you can use your smartphone to test out for yourself how that new table or sofa would look in your own home. You scan your living room or bedroom, then choose the new piece of furniture from the virtual catalogue and finally place it in the room virtually. Place is a very accessible example of what AR or augmented reality – literally adding something to reality – is capable of. The app even allows you to adjust the lighting and shadows so that the image looks as realistic as possible. AR is a live – direct or indirect – image of reality to which a software application adds all kinds of elements. In virtual reality (VR), the user is completely disconnected from reality and, as a viewer, is presented with a new digital reality via a VR headset.
Five years ago, Johan Smeyers was one of the founders of Arkite, a pioneer in augmented reality that found a home at C-mine. His company now has some twenty employees and is pioneering an AR application that is primarily aimed at manufacturing companies. Arkite argues that, thanks to their Human Interface Mate (HIM), workers in the most diverse production environments can perform complex actions with a certainty and accuracy approaching that of a robot. To this end, a kind of intelligent projection box crammed with sensors is suspended above the workstation. This then projects each step in the production process onto a work table and the product itself, with a screen providing additional support to the operator, ranging from “Now take key 17” to “Now put that screwdriver back”. Whenever a certain step in the process is performed correctly by the operator, it is also registered and approved by the system. Putting it bluntly, you could say that the HIM is a sophisticated version of a paper manual or the tablet with instructions that is becoming increasingly popular in manufacturing companies. “These classic systems work perfectly if you, as an operator, constantly have to deliver the same product, or at most a few variants of it, with a relatively limited and little varying number of actions,” indicates Smeyers. “However, it becomes a completely different story if you have to deliver dozens of variants, and therefore constantly have to look at your paper manual or scroll on one of those tablets. This leads to loss of efficiency: reading something or scrolling adds no value to the product itself.
If, on the other hand, AR can help an operator to do the right thing at the right time thanks to the right information and also avoid making mistakes, then you improve both the quality of that work and the operator’s working comfort. So that’s where we saw a gap in the market for augmented reality.”
“If AR enables an operator to do the right thing at the right time, thanks to the right information, you will improve the quality of the work and the comfort of the workers.
Human factor
Johan Smeyers has gained experience in very diverse production environments over the past 20 years. During that period, he learned that ever-increasing automation has systematically boosted the quality and efficiency of many production processes. But at the same time, he also saw that the interaction between the employees on the one hand and the advancing automation on the other hand is not always so easy to manage. “Because we are human beings, and therefore regularly skip a step in a process or perform a certain action incorrectly or not at all. As a result, the quality or accuracy of the end product does not always meet expectations. And no matter how you look at it, despite the strong focus on automation, digitalisation and increasingly intelligent, autonomous systems, the human factor continues to play a key role in the production process.
Most production systems are still decisive.”
A second trend that is developing strongly and also has a major impact on production companies from all possible sectors is that of personalisation. More and more so-called mass products are becoming increasingly differentiated and even personalised. A good example of this is
For example, the wide-ranging automobile industry: for every model of a modern car, there are nowadays dozens of slightly different variants. From a marketing point of view, this offers companies almost unprecedented opportunities, but for the production department, such extensive customisation is often a real nightmare. The task of the operators becomes a lot more complex, and the risk of mistakes or errors increases accordingly. “Last but not least, production companies are now increasingly confronted with the question of how to optimally integrate their employees into their production system. There are two major trends here. On the one hand, some companies choose to automate all their processes as much as possible, for example through robotics. They thus exclude the human factor as much as possible, hoping thereby to reduce the risk of errors.
On the other hand, you have those companies that prefer to focus on autonomation: give your employees the autonomy and the means to work more efficiently and with more quality. Augmented reality can provide a lot of added value here,” he says. “Our technology helps companies to work faster, more efficiently and more ergonomically. And it is a perfect illustration of what augmented reality is capable of: it enriches reality with information that can provide added value at that moment, tailored to a specific use case.”
Arkite’s HIM projects all relevant information onto a work table at a workstation, but there are also many other AR solutions that can boost productivity and efficiency. The best known of these are perhaps holosenses and so-called smart glasses. These glasses or headsets ensure that the wearer is presented with extra information depending on what he or she is seeing at that moment. For example, an arrow can be drawn for an order picker to the object that he has to take from the racks at that moment and at that exact place. When the operator subsequently turns his head, the arrow will also move with him and continue to indicate the object in question. “It’s not just about the right information at the right time, it’s also about the right amount of information. Quite a few companies try to solve a problem in production by over-engineering,” explains Smeyers. “But an operator doesn’t care about that; he needs to get just the information he needs in that specific step of a production process. If something goes wrong, and the employee takes the wrong part or uses the wrong tool, he will also be alerted immediately. This digitally pre-programmed interaction is also an essential part of AR.”
“Quite a few companies try to solve a problem by over-engineering. But an operator doesn’t care about that.”
BIM
“If you had suggested a few years ago that VR glasses should be introduced as a tool in a certain branch of industry, you would undoubtedly have been called crazy. In recent years, however, the popularity of all kinds of VR applications has grown so strongly, particularly abroad, that we are also seeing increasing interest in this in Belgium,” says Ellen Vandenbruwaene. She works as a project manager at DAE Research, the research group within the Digital Arts and Entertainment programme at Howest University. Among other things, she researches the extent to which the technologies that are already widely used in the games industry today can also be used for numerous other applications.
“The food and manufacturing industries, for example, are very curious about the possibilities of this new technology. And we are also seeing growing interest from the construction industry, especially from designers and architects.”
For a sector such as the food industry, the potential of VR applications is mainly in the areas of safety and training. “On the one hand, this sector works with increasingly high-tech machines, while on the other hand there is a relatively high staff turnover and a considerable amount of flexible work. It is, of course, extremely uneconomical to have to shut down the production apparatus at short notice in order to train people, and VR training offers an excellent alternative. At the same time, such a first virtual contact with the shop floor and with certain machines is also ideal to separate the wheat from the chaff: is this effectively the right man or woman on the right place?”
In the construction sector, VR technology is increasingly emerging as an ideal tool for optimising cooperation between the various parties involved – from architects and contractors to site managers. The BIM model (Building Information Model) is particularly popular in this respect. Whereas with a traditional plan you never know exactly what a construction project will look like, BIM provides a perfect virtual image of a building in advance. To this end, the model is based on 3D drawings that provide you with a good visual insight, but it goes a whole lot further than that. After all, it is also full of metadata and various information layers that can be easily consulted by all parties in the construction process, alongside and on site. For example, BIM can provide information about the exact location of a cable or technical pipeline, allow you to play with different materials or give you a better understanding of the energy consumption or the price tag of certain techniques. “AR glasses or hololens are then a possible tool to visualise such BIM models.”
Towbars
Companies that want to optimise their production process with an AR application such as Arkite’s HIM will pay around €20,000 per workstation for this.
Not an incredibly high amount, but not a trifle, of course, and it is therefore obvious that these kinds of AR applications will mainly be adopted by producers of rather complex applications or by products and companies that market a particularly large number of variants of the same product. “If, on the other hand, an operator has to produce something for which he only has to go through twenty different steps in a handful of different variants, then I don’t immediately see a business case for this,” indicates Smeyers. “In such a case, you can just as well get to work with, say, a tablet.” However, there are also examples of relatively simple products or very repetitive production processes where AR is indeed used. Not because the product itself is so complicated, but because a small mistake in the production process can have far-reaching consequences, so that the use of AR can provide a lot of added value. “For example, one of our customers is a Dutch manufacturer of towbars for passenger cars. Now you have to know that almost for every new model that comes onto the market, a new kit with a towbar has to be developed as well. Many of the parts are the same, of course, but the end product is always slightly different from that of another brand or model. A different way of mounting, some other screws, you name it. If mistakes are made, a lot of time is spent on phone calls, new shipments, you name it. In this particular case, it’s not so much about avoiding human error during production itself, but more about the added value you create afterwards if you can avoid those errors with the help of AR.”
If AR succeeds in creating additional economic value for operators in the manufacturing industry, this will obviously have a positive impact on the cost efficiency of a company. And then the question immediately arises whether it is not more profitable in the long run to invest in AR applications that can significantly increase the output of individual employees, rather than to invest in automation and robotisation? Based on the observation that far-reaching automation quickly comes with a price tag of hundreds of thousands or millions of euros. Or from the knowledge that some products become outdated so quickly that it is simply impossible for the producer to install a fully automated production chain, because it would simply need to be modernised again after a few years. Or because the global trend towards more customised production in the most diverse sectors often makes automation a lot harder.
“A big advantage of our product is that it is completely contactless: I don’t have to put on special gloves or bracelets, nor do I have to put on glasses. In terms of ergonomics in a production environment, that’s a big plus.”
Yet neither AR nor VR are really widespread in business in the year 2021. Many companies are already experimenting with so-called smart wearables. “Not least because the most suitable AR technology can also vary greatly from one application to another,” Smeyers underlines. “A major advantage of our product, for example, is that it is completely contactless: I don’t have to put on special gloves or bracelets, nor do I have to put on glasses. In view of the ergonomics of an operator in a production environment, who has to work eight hours a day with this AR application, this is therefore a great advantage. Of course, it is a completely different story if you start using AR to support, say, maintenance technicians who have to carry out repairs on site at customers. They then identify a certain problem, call in to their company and subsequently receive remote support via, for example, their smart glasses. In such a case, those glasses or a hololens can provide the perfect AR solution.”
Hololens
To make things even more complicated, AR and VR have recently been joined by MR: mixed reality. “VR in itself is a fairly well-defined technology,” explains Ellen Vandenbruwaene. All you need is a computer, a VR headset and a few square metres.
Thanks to VR, the user can be placed in any environment and situation without getting input from the real world. In theory, you can then project virtually anything around you, no matter how futuristic. After all, there is no need for reality. AR, on the other hand, adds an extra layer to that reality and, for that reason, lends itself more to very specific, context-specific applications in business. But admittedly, the distinction is not always that simple, and many companies that want to jump on this bandwagon struggle with it. We ourselves more or less make the following distinction: VR can be perfectly used to teach employees something new or to provide training, and is also very suitable for visualising unsafe situations, for example. The Ministry of Defence, for example, uses VR a great deal because they need to train their people for situations that will occur in real life.
Of course, reality can never be fully simulated. AR, on the other hand, lends itself better to certain forms of automation or the facilitation of certain routines. The choice between the two applications is often a difficult one. The combination of both, i.e. MR, is not really obvious yet, but in time, I see them gradually growing together. The most concrete example of this today is actually the hololens, with which you effectively project something into your real surroundings. This is different from a regular AR application that you always view with a ‘tool’ such as a smartphone or tablet. A hololens allows for much smoother interaction: you walk around, you interact with what you really see in your surroundings through those glasses. In the long run, I see enormous possibilities for this in a business environment. After all, this technology allows you, for example, to register all kinds of actions in a very natural and accessible manner – through high-tech glasses – and to scan products. The added value may still be relatively small for the operators who have to perform all those actions, but it is much greater for the production line behind them. After all, much more control is possible, so that the error margin will be significantly smaller. On average, you still pay around 4000 euros for high-quality MR glasses today. “This is considerably more than what you pay for VR glasses: a so-called stand alone VR set, for which you no longer need a computer, is available from 350 euros. But in the long run, I personally see much more potential for AR,” he continues.
Apple chief Tim Cook once outlined the huge potential of AR as follows: “AR will be as important as eating three meals a day.” Cook is of course a techie, but he is also a very shrewd marketer. So anyone who has ever lost themselves in the virtual world of Pokémon Go can certainly imagine the marketing potential of augmented reality. The Swedish furniture giant mentioned earlier provides striking proof of this, but in theory, the possibilities are more or less endless: for just about every product imaginable, you could enhance the experience by adding a virtual layer of extra information at certain places or certain moments. Car manufacturers are already doing this by projecting your speed or other relevant information onto your windscreen via a so-called ‘heads up display’. In this case, it is still a purely functional application that will mainly benefit safety, but in the long run your car dealer might just as well set up a virtual showroom next to a photo of your favourite model, where you can admire the car from all angles. Now that almost everyone is constantly walking around with a smartphone in their pocket, using it has become extremely easy. This is also the biggest advantage of AR compared to VR, for which you still need a rather flashy pair of VR glasses.
“AR has a lot to offer the consumer market, especially visually, now that almost everyone has a smartphone. We are on the threshold of a major breakthrough in this area.
For the time being, of course, things are not going that well, but according to Vandenbruwaene that has everything to do with the lead time from the industry. “First, the various industries need to become fully aware of the possibilities offered by AR and VR. Only then can you expect to see a move towards broader consumer applications. The automotive sector is undoubtedly a forerunner in this respect, because major car manufacturers are already using AR and VR very intensively in their production chain. Moreover, they also have the budgets to further experiment with this technology. Especially visually, AR has a lot to offer the consumer market. I suspect that we are now on the threshold of a major breakthrough in this area.
Rehabilitation therapy
Because VR, unlike AR, is one hundred percent programmable and therefore virtual, this technology is already gaining importance in environments where control and safety are priorities. The healthcare sector is a perfect example of this, and the number of new VR applications is gradually mushrooming. Just imagine: you are struggling with a neck or back injury and your doctor prescribes a number of sessions with VR glasses as a rehabilitation programme. A video is played in which you have to perform a number of games with a kind of stick in a totally virtual setting. For example: knock down as many balloons that are swirling down within a certain time margin. Subsequently, not only is your score calculated and compared, the programme also measures the exact angle at which you can move your arm, for example, or how fast you can perform certain movements. The Israeli start-up XRHealth, among others, has been pioneering for a few years now with some promising customised applications for doctors and hospitals. The founder of that company has a solid past in the Israeli army, where he served as a fighter pilot for years. At one point, he had to give up flying because he was struggling with persistent pain in a cervical vertebra. He was then prescribed a rehabilitation programme in the army, but he found it extremely frustrating that he himself had absolutely no insight into the progress he was making. Like everyone else in such a case, he had to complete a compulsory exercise programme with the physiotherapist every day, but he was given very little concrete information about how his health was evolving. This gave him the idea of looking for technology that could help him both to complete his exercise programme more dutifully and to make his physical progress more measurable. From his background and knowledge as a fighter pilot quickly led him to the potential of virtual reality.
More than four years on, the Israeli start-up has developed a much broader platform, which targets a number of very different VR technologies.
applications in the wider healthcare sector. “Rehabilitation therapy is one of them, but VR can also be used, for example, to measure the cognitive abilities – or decline – of patients,” explains Tal Arbel, head of data at XRHealth. “To do this, we bring them into a VR environment and have them carry out all kinds of tasks in which we measure and compare the response time. Think about people with early onset dementia. In both domains, VR games allow us to collect very precise and structured data, something that is much more difficult if you have people complete an exercise set at home or just monitor them in a hospital environment. VR is a lot more accessible to a much larger group of patients. You can do the exercises at home, in your familiar surroundings, but also people in wheelchairs can access certain rehabilitation programmes. And for patients who have been in hospital for weeks at a time, it’s a pleasant change when they are catapulted into a beautiful, sunny VR environment for an hour before their daily work-out.”
Another area of healthcare in which VR is expected to be of great use is pain management. “If I immerse you here and now in a mesmerising VR environment – full of beautiful colours and images and pleasant sounds – I will also draw your attention away from the pain you may be experiencing at the moment,” Arbel explains. “In Boston, for example, clinical trials have been conducted with patients who have to undergo hand surgery. Today, they are given anaesthetic so that, in theory, they no longer feel pain from the elbow down. However, they are usually still very anxious during the treatment, so that they often have to be fully anaesthetised. That costs a lot of time and money. Via VR, they can be put into a kind of sedative state in advance, in combination with the local anaesthetic. The results of this are very promising. If you look at the extent of addiction to painkillers in the US in 2019 and the social costs involved, then this type of application has huge potential.
That he is not alone in this opinion is also evident from a Goldman Sachs report, which estimates the global market for VR healthcare applications at USD 5.1 billion by 2025. Healthcare is already proving to be a huge growth market: in 2017, the sector – after fintech but ahead of foodtech – raised the most investment money worldwide. “In time, both AR and VR are likely to become mainstream,” concludes Ellen Vandenbruwaene. “We live in an era where the visual has gained tremendous importance, and let AR and VR now lend themselves perfectly to that as well.”