Why AR, not VR, will be the heart of the metaverse

The 2nd Annual GamesBeat & Facebook Gaming Summit will be held in January of 2022. You can learn more about the event.

Louis Rosenberg is the CEO and chief scientist at Unanimous AI.

My first experience in a virtual world was in 1991 when I was a PhD student at NASA. I was using early virtual reality systems to model the distance between your eyes. I found the virtual reality experience to be rather miserable despite being a true believer. I knew that would steadily improve, but it felt cramped and claustrophobic to have a scuba mask strapped to my face for long periods of time.

The sense of confinement didn't go away even when I used early 3D glasses. I had to keep my gaze forward, as if I were in the real world. I wanted to take the blinders off and allow the power of virtual reality to be used in my real life.

The Virtual Fixtures system was developed for the U.S. Air Force to allow users to interact with virtual objects that were accurately integrated into their perception of a real environment. This was before phrases like "augmented reality" or "mixed reality" were used. I was convinced that the future of computing would be a seamless merger of real and virtual content when I watched users enthusiastically experience the prototype system.

GamesBeat: Into the Metaverse 2 is a sequel to the 2nd Annual GamesBeat and Facebook Gaming Summit.

Learn more.

The phrase "metaverse" has become a rage after 30 years. The hardware for virtual reality is cheaper, lighter, and has more fidelity. Three decades ago, I experienced the same problems. Most people don't like wearing a scuba mask because it makes them feel cut off from their surroundings.

The metaverse will be an augmented reality environment accessed using see-through glasses. Even though full virtual reality hardware will offer higher fidelity, this will hold true. Broad adoption is not dependent on visual fidelity. Adoption will be driven by which technology gives the most natural experience to our perceptual system. Integrating digital content directly into our physical surroundings is the most natural way to present it to the human perceptual system.

Perceptual consistency is more important than a minimum level of fidelity. All sensory signals feed a single mental model of the world within your brain. If virtual elements are registered to your surroundings in a convincing way, augmented reality can be achieved with relatively low visual fidelity. It is not hard for this to be convincing because of our coarse sense of distance.

It is difficult to provide a unified sensory model of the world in virtual reality. It is easier for virtual reality hardware to provide high-fidelity visuals without lag or distortion. Unless you use impractical hardware, your body will be sitting or standing still while most virtual experiences involve motion. The virtual world presented in your headset makes it difficult for your brain to build and maintain two separate models of your world, one for your real surroundings and one for the virtual world that is presented in your headset.

When I tell people this, they often push back, forgetting that regardless of what is happening in their headset, their brain still maintains a model of their body sitting on their chair, facing a particular direction in a particular room. Your brain is forced to maintain two mental models because of this perceptual inconsistency. There are ways to reduce the effect, but only when you combine real and virtual worlds into a single experience that is consistent.

This is the reason augmented reality will inherit the earth. It will replace phones and desktops as our primary interface to digital content and will overshadow virtual reality as our primary gateway to the metaverse. It's not the most natural way to experience content to the human perceptual system to walk down the street with your neck bent and staring at a phone in your hand. I firmly believe that within 10 years, augmented reality hardware and software will be the main focus of our lives.

This will allow artists, designers, entertainers, and educators to embellish our world in ways that defy constraint, as they are suddenly able to do. Each of us will be able to alter our world with the flick of a finger or the blink of an eye thanks to augmented reality. As long as designers focus on consistent signals feeding our brains and worry less about absolute fidelity, it will feel real. Perceptual design was the name I gave to this principle as I worked on augmented and virtual reality back in the early '90s.

The vision portrayed by large platform providers of a metaverse filled with cartoonish avatars is misleading. Virtual worlds for socializing will become more popular, but they won't be the means through which society is changed. The central platform of our lives will be an augmented world. It will be everywhere by the year 2030.

Louis is the Chief Scientist at the company.

The VentureBeat community is welcoming you!

Technical people doing data work can share their insights with experts at DataDecisionMakers.

At DataDecisionMakers, you can read about cutting-edge ideas, up-to-date information, and the future of data and data tech.

You could even contribute an article of your own.

Data decision makers have more to say.