Mark Zuckerberg’s augmented reality

Mark Zuckerberg hopes that one day you will be able to see the same thing through a pair of augmented reality glasses.

A device that will redefine our relationship with technology, similar to the introduction of smartphones, is what Facebook's founder calls augmented reality goggles. During the special effect-laden video announcing Facebook's corporate rebrand to Meta last October, they acted as the core of his metaverse pitch, letting people play games and work with virtual humans Star Trek-style. He wore them while fencing with a hologram.

The near-term reality of smart glasses is far less lofty than what Facebook's founder has in mind. The virtual chess demonstration that was part of the Meta presentation was not based on any functioning hardware or software. Meta doesn't yet have a working prototype of its planned augmented reality glasses, but rather a stationary demonstration that sits on a table.

When his high-tech glasses will be a reality, he has ambitious goals. Employees are racing to deliver the first generation by 2024 and are already working on a lighter, more advanced design for 2026, followed by a third version in 2028. A person for Meta wouldn't comment on the story.

Image: Meta

People familiar with his thinking say that the push for augmented reality glasses and Facebook's new name is a way for the company to be innovative again. A series of privacy and content moderation scandals have stained the social network's reputation. Regulators are trying to break the company up. It has become known as a ruthless copycat among its Silicon Valley peers.

The company could be cast in a new light if the glasses catch on.

The CEO of Meta believes that the augmented reality glasses called Project Nazare are a way to get out from under the thumb of Apple and Google, which together dictate the terms that apps like Facebook have to abide by on mobile phones. The first version of Nazare is designed to work on its own, without the help of a mobile phone, and with the help of a phone shaped device. A marquee feature will be the ability to communicate and interact with holograms of other people through the glasses, which Zuckerberg believes will, over time, provide people with a more compelling experience than the video calling that exists today.

The first version of Meta's augmented reality glasses will be aimed at early adopters and developers, despite the fact that the company has already spent billions on developing them. The device will be more expensive than the company's $299 Quest VR headset, given that the bill of materials is thousands of dollars. The price of the hardware will be a test of whether or not Facebook will subsidize it to encourage adoption.

A pair of cheaper smart glasses called Hypernova are also planned for 2024. Nazare is designed to operate independently of a phone, but Hypernova will pair with a nearby phone to show incoming messages and other notifications through a smaller, heads-up display similar to the North smart glasses.

Meta hopes to sell tens of millions of smart glasses by the end of the decade, according to its VP of augmented reality, Alex Himel.

Image: Meta

It is not clear if people will use augmented reality glasses in the next few years. Similar products from Microsoft are not mainstream. The stakes couldn't be higher for Meta. The company's division making metaverse hardware and related software cost it $10 billion last year. The price of talent in the industry has gone up due to Meta's efforts to build the glasses and future hardware.

While his business is under pressure from all sides, Facebook founder and CEO Mark Zuckerberg has said he plans to increase spending on building augmented and virtual reality hardware in the years to come. Meta's stock has been hammered due to its slowing social media business and younger users flocking to competitors like TikTok. Big acquisitions like the purchase of the photo-sharing website,Instagram, have been ruled out by antitrust scrutiny. After kneecapping Meta's core ads business with recent tracking changes in iOS, Apple is readying an assault on Zuckerberg's hardware strategy, too, starting with a high-end, mixed reality headset as soon as this year and eventually its own.

One former team member says that since Nazare was brought out of research, the project has been shown a special interest by the leader of the company. In a recent interview, Zuckerberg admitted that employees sometimes use the phrase to explain his involvement in a project.

The first version of Nazare must offer a full augmented reality experience with 3D graphics, a large field of view, and a socially acceptable design. The team wanted it to have a 70 degree field of view, but it won't be achieved. The current design of the glasses is similar to Superman's black frames. They are four times heavier than a normal pair of glasses.

There is no guarantee that Nazare will meet the ship goal. The ship year has slipped multiple times. The product experience is still being worked on, especially on the software side. At the end of last year, an effort to build a custom microkernel operating system for the device off a version of the open-sourced Fuchsia OS was scrapped because it wasn't going to be ready in time. Meta is working on a version of the first version of the augmented reality glasses that is built on top of the operating system of their choice.

At first, Nazare won't be a mainstream device. The battery life is only four hours, and the glasses are intended to be used indoors. It will take a while before the glasses sell in high volume, but they have spared no expense. The displays are powered by expensive equipment. The first version will have eye tracking and a front-facing camera. Employees are working with Asia's Semiconductor Fabs to build custom chips for the planned roadmap through the latter half of the decade.

Image: Meta

Meta plans to bundle with the glasses a wrist device that will control them with the wearer's mind, something that will likely be the company. The wristband uses a technique called differential electromyography, or EMG, to measure electrical impulses in the arm's neurons, which can be used to create a phantom limb. The result is that someone can think to type or control a virtual interface, which Meta believes will aid in interacting with smart glasses that don't have a keyboard. The tech is based on the company's acquisition of a startup in 2019.

Everyone I have talked to who has tried a prototype of the band Meta is working on says it is one of the most impressive tech demos they have ever experienced. The company thinks it could have the next mouse and keyboard. One former senior Meta employee said that if the other stuff works, none of the other stuff needs to matter.

Meta is planning to debut its first-ever wristwatch as soon as this year. The plan is for the third generation to include it and tie up with the debut of Nazare and Hypernova in 2024. The second version of the company's smart glasses with Ray-Ban is in the works. Meta sold about 120,000 pairs of the glasses with Ray-Ban during the period when it went on sale last September through December, missing its initial goal of 300,000. A higher-end headset with pass-through video capable of Blending the real and virtual worlds is being prepared for later this year, ahead of a similar device Apple is planning to release.

The team has seen a lot of turnover with the recent heads of product, design, and software leaving. The company leaders have been installed in key positions. Alex Himel has been at the company for 13 years and is the VP of glasses. Sue Young has worked at Facebook for a decade and is the direct leader for Nazare. Two other senior leaders on the team are hardware engineering head Caitlin Kalinowski, who recently moved over from running hardware for Oculus, and former Microsoft executive Don Box, who now runs software engineering for the glasses.

It is going to be a long time before augmented reality glasses get mainstream appeal.