Mac Week: The Apple of your eyes: how Apple's AR and VR plans might take shape

Apple's AR and VR plans

Virtual reality is grabbing the headlines, with Oculus Rift, HTC Vive and Samsung Gear VR headsets already on sale and Sony's PlayStation VR imminent. But, while Apple thinks virtual reality (VR) has some "interesting applications," Tim Cook seems to believe that augmented reality (AR) will be even bigger.

We know that Apple has likely hundreds of people working on VR and AR technologies, but we don't know when or if an Apple AR/VR product will ship – or what it'll be like. But, there are plenty of clues that can help us make some informed predictions.

AR vs VR: what's the difference?

Virtual reality is when you see an entirely virtual world: everything before your eyes has been computer generated. AR doesn't do that. Instead, it augments the real world – think Pokémon Go, or a heads-up display on a car's windshield.

As Tim Cook says, AR enables you to be "very present" when using the technology. You're not bouncing around your living room with the best part of a motorcycle helmet on your head.

We expect Apple to be working on both AR and VR projects. But, we think AR is the one Tim Cook is more excited about.

Who's involved?

Apple's AR/VR team includes some big names, including computer science professor and immersive 3D interface expert Doug Bowman, former Magic Leap computer vision engineer Zeyu Li and former Oculus research scientist Yury Petrov.

Apple has also acquired multiple AR/VR companies, including real world object-tagging company Flyby Media, facial capture firm Faceshift, augmented reality developers Metaio, 3D sensor firm – and Kinect creator – PrimeSense, computer vision startup Perceptio and facial analysis developer Emotient.

In addition, Apple is believed to have hired people from Microsoft's Hololens AR project and from innovative camera firm Lytro.

How long has Apple been working on this?

Years. Apple filed multiple patents in 2008 for head-mounted displays, some of which resembled Google Glass and others which took the same "phone on your face" approach as Samsung's Gear VR.

In 2011, Apple filed a patent for augmented reality map displays, and it has also filed multiple patents involving 3D interfaces and sensors.

When can we expect to see some products?

Don't hold your breath for exciting new hardware. While Piper Jaffray analyst Gene Munster says that Apple is working on several "mixed reality" projects, he projects that initially they will be software-driven and on iOS devices.

According to Seeking Alpha, Munster predicts that Apple may offer a VR/AR API to hardware developers, much like the Made For iPhone program by 2018 – but a glasses-style "mixed reality" headset is at least five years off.

Munster's predictive track record is a bit patchy – he's been banging on about an Apple television set for years, to the point where it's become a running gag among Apple watchers. But, the prospect of an Apple headset taking its time to appear seems reasonable.

We've seen the challenges of getting enough power and battery into the Apple Watch; to miniaturize that even further to fit into something like normal glasses or sunglasses won't be easy.

What's Apple up against?

There are lots of smart-glass models on the market, but they're limited, bulky and expensive – and mainly for niche markets such as professional cyclists. The closest to an all-purpose headset is probably Vuzix's M100 Smart Glasses, which cost $799 (about £615) and work with both Android and iOS; or their imminent successor, the M300, which began shipping to VIPs this week, as of this writing.

But, while both systems are undoubtedly impressive and well suited to their target market of business customers, they're also rather bulky. They're not Oculus-style monsters, but they aren't something you're going to want to wear as you walk down the street, either.

What we're likely to see first is Apple AR in software, and the iPhone 7 Plus is a pretty big clue that that's not too far in the future. Its dual cameras aren't just useful for getting creamy bokeh into portraits.

More importantly, they're also useful for 3D mapping, whether that's of a space or your face. That opens up a whole world of potential applications from interior design and online retail to funny FaceTime filters.

And don't forget, AR isn't necessarily a visual technology. The new Apple AirPods put Siri in your ears, just like in the film Her, and that has potential for adding descriptive information to the world around you.

Smartphone screens are our primary way of receiving information because that's how technology has evolved, but the future is much more modular. The phone will still be the engine, but increasingly we'll get our information in multiple ways: a haptic tap here, an auditory signal there, an icon or an alert in our field of view.

What would an Apple headset look like?

Back in 2007, Apple patented a head-mounted display that would "resemble ski or motorcycle goggles" – but, that was before the iPhone debuted, and mobile tech has improved considerably since. A more recent patent filed in 2015 and granted in 2016 shows a lighter, less goggle-y headset designed to work with an iPhone.

The Financial Times reports that Apple prototypes have resembled both Oculus Rift headsets and Microsoft Hololens, but that doesn't mean Apple will ship either. It was building VR prototypes at the turn of the century, and those projects were ultimately scrapped most likely.

When it comes to a new product line, Apple tends to do three things: it learns from others' mistakes; it waits until it thinks there's a big enough market for the product; and it obsesses over the details that others missed.

Remember, Apple didn't invent the MP3 player, the smartphone, the tablet computer or the smartwatch, but it's followed the same pattern with all of those categories. Nothing about that strategy should change with its headset.

What will an Apple AR or VR headset look like? The answer's obvious. First, look at what everybody else is doing. Now, think different.

• So, what about that MacBook Pro 2016 revision?

This article is part of TechRadar's Mac Week. This year marks not only the 10th anniversary of Apple's unibody MacBook, but the triumphant return of macOS. So, TechRadar looks to celebrate with a week's worth of original features delving back into the Mac's past, predicting the Mac's future and exploring the Mac as it is today.

September 30, 2016 at 09:00PM
Gary Marshall