What is augmented reality?

What is augmented reality (AR)? Put simply, it’s a technology that overlays computer-generated imagery onto the world around us. Think of it as a tool that literally augments or adds to reality with digital graphics and often sound. As with any technology though, there’s a lot more to it than that. That’s why we’ve put together this helpful explainer, in which we’ll cover the basics of AR, what it’s used for, how it really works, how it differs from virtual reality (VR), and what it means for gaming and other experiences.

If you’re reading this, it’s likely you know that AR is a hot topic in the technology space right now. Accessible devices such as tablets, smartphones, and game consoles now come AR-ready, using a combination of the camera and display to see 2D or 3D graphics overlaid on your surroundings for your education, entertainment, fitness, and more. Whether it's a global character hunt or a place-your-furniture app AR experiences have already started to filter out into our everyday lives.

Now let’s find out more about how AR works and how it’s changing our world.

What is an augmented reality game?

An AR game blends game visuals and audio with your real-world surroundings. AR gained a wellspring of attention in gaming around the release of Pokémon Go, which became a cultural phenomenon in 2016. The mobile game challenged players to find and catch Pokémon in everyday surroundings, encouraging them to get outside and search for sought-after creatures.

Since then, the accessibility of the technology to players and developers has led to many AR mobile games. But while AR has made it onto handheld game consoles like the Nintendo 3DS, it’s still largely confined to smartphone apps.

What is augmented reality used for?

At this point, a better question might be “What isn’t AR used for?,’ such is the breadth of its usefulness across numerous industries. Need to create a way for people to try on clothes virtually? AR could be the solution. Want to see more information on landmarks or museum exhibits? AR devices can be used to present floating text boxes around the object. How about a way to visualize a construction project in the actual environment? That’s AR too.

Epic Games | Microsoft | HoloLens 2

Here are some industries we haven’t already mentioned using AR in ingenious ways:

Healthcare - From training professionals with virtual scenarios to educating students on anatomy, AR has wide-ranging and increasingly accessible applications for healthcare.
Retail - AR fitting rooms, in-store displays, and product visualizers, are just the tip of the iceberg when it comes to the technology’s potential for retail.
Manufacturing - Safety and efficiency are key to manufacturing, and AR is providing a welcome boost to both by helping to train workers in heavy machinery.
Entertainment - Interactivity and immersion in entertainment have already reached new heights thanks to AR, from singalongs to mobile games, and even social media filters.

Courtesy of Sky Sports

Military - Whether it's data displayed on a vehicle’s windshield, or training simulations, AR is changing things for our military too.
Theater - AR can be used to visualize a stage design or layout ahead of production.

The exact way that AR works can vary. Training a pilot, for example, requires an altogether more robust solution than using your phone to catch creatures in the backyard. AR requires a compatible device, which can range from a smartphone or a tablet to smart glasses, laptops, or Head Mounted Displays (HMDs). What these devices all have in common are the necessary hardware components to make AR possible.

Processors, sensors, tracking, input, and display are all needed for AR to work. Your smartphone has all this, with the touch screen, camera, GPS, Inertial Measurement Unit (IMU), solid-state compass, and CPU all working together to power AR experiences, keep track of the user’s location, and determine device orientation and position.

More powerful AR experiences, like those used to train professionals in a range of fields, might also require components that recognize particular objects and gestures in the real world. This kind of AR can devour computing power, in which case data can be streamed from another device. Additional technology like haptic feedback gloves can be combined with AR to create even more realistic or immersive experiences.

It also needs a robust and powerful tool for creation, for instance, a production-proven game engine like Unreal Engine. Unreal Engine is free to download and free to use for creating linear content like films, and for custom and internal projects. It’s also free in many cases for game development—a 5% royalty only kicks in if and when your title earns over $1 million USD.

What is an example of augmented reality?

Use cases for AR are as wide-ranging as the industries it’s used in. Here are a few fascinating examples powered by Unreal Engine:

Live events - Coachella and Flume went all in on live concert AR for a YouTube live stream earlier this year.
Hologram performances - BTS joined Coldplay for a live hologram performance of 'My Universe' with the help of AR technology
Sporting events - Mixed reality put the purrr-fect twist on Carolina Panthers games last year
Aviation training - In-flight aviation training with Red 6 took on a whole new dimension thanks to AR

What is the difference between augmented reality and virtual reality?

While AR and VR appear to do similar things, their differences come down to the experiences they offer. VR refers to a virtual environment presented as if it were your actual surroundings. These virtual environments are experienced through HMDs that utilize a visual display and sound to immerse you in the VR content. There are no real-world elements visible, your senses are engaged in a fully rendered, virtual setting.

Courtesy of Inlusion Inc

AR, on the other hand, places rendered, virtual elements onto your visible, real-world surroundings. VR simulates a new reality, while AR augments your existing reality with digital elements. An easy way to tell if someone is using AR or VR on their project is the hardware. VR requires a HMD to deliver immersive visuals and audio. AR requires devices such as smartphones, tablets, or AR HMDs to augment your surroundings.