Report Reveals Wave of New Features for Apple's Mixed-Reality Headset, Including Digital Crown for Switching to Real-World View

Apple's mixed-reality headset will feature a physical dial for switching to a view of the real-world, a waist-mounted battery pack, small motors to automatically adjust its lenses, and much more, according to The Information.

Concept render based on purported leaked information by Ian Zelbo

The paywalled report purports to reveal a broad range of previously unheard-of features for the headset, including:

• A waist-mounted battery, connected via a magnetic, MagSafe-like power cable to the headset's headband. One battery charge lasts no longer than two hours, but users can swap the battery out for longer sessions.


• A design that uses aluminum, glass, and carbon fiber to reduce its size and weight. Cameras are largely concealed for aesthetic reasons.


• A small, Digital Crown-like dial on its right side that enables users to quickly transition between the virtual and physical world. Unlike the Apple Watch, it will not offer haptic feedback.


• Apple has developed different headbands, including one for consumers made of a similar material to Apple Watch sport bands with built-in speakers, and one targeted at developers.


• Magnetically attachable custom prescription lenses for glasses-wearers.


• Small motors to automatically adjust its lenses to match the wearer's interpupillary distance, providing the largest field of view possible for each individual.


• A 120-degree field of view – more than the Meta Quest Pro and Valve Index's 106-degree field of view.


• The H2 chip for ultra-low latency connection with the second-generation AirPods Pro.


• Two chips, including a main SoC, including a CPU, GPU, and memory, and a dedicated image signal processor. Both are fabricated with a 5nm process, meaning that the headset misses out on advancements expected to come to other Apple chips later this year. The chips communicate via a custom streaming codec that Apple was forced to develop due to unacceptable latency.


• The dedicated ISP translates the distorted images captured by the external cameras into a faithful video representation of the user's surroundings with low latency. The chip contains custom high-bandwidth memory made by SK Hynix.


• The ability to run existing iOS apps in 2D.

Third-party Bluetooth headphones apparently do not work well with the headset and suffer from too much lag, and there is no 3.5mm headphone jack on the device. Apple has focused on making recent and future AirPods models work with low-latency when paired with the headset. Both the second-generation ‌AirPods Pro‌ and the headset contain the H2 chip, which is said to offer an "ultra-low-latency" mode when paired together.

Apple has apparently debated whether AirPods should be mandatory for communication activities on the headset to better protect privacy.

Early prototypes of the headset used large external batteries designed for power tools strapped to the wearer's waist, before Apple integrated the battery directly into the headband like the Meta Quest Pro. As of 2022, Apple reportedly shifted the power source back to a battery pack intended to be worn on the user's waist, connected via a cable to the headband – a design advocated by Apple's former design chief, Sir Jony Ive.

While it was once rumored that the headband would be interchangeable like Apple Watch bands, that is no longer the case, since engineers faced technical challenges with the headband acting as a conduit for power to the headset.

Apple is said to favor hand-tracking and voice recognition to control the headset, but it has tested a "wand" and a "finger thimble" as alternative control input methods.

The headset is not expected to be accompanied by a dedicated gaming controller, and Apple has apparently not focused on gaming for the device. That being said, the company does have a plan to allow Unity Technologies to be the first partner to offer full VR experiences in the headset through its game engine.

Apple is focused on videoconferencing on the device, with digital avatars that have a high-level of accuracy when mimicing a user's facial expressions and body movements. It has created software to allow users to perform activities like dragging the Maps app off a Mac screen and use it to display a 3D model of a city on a table top, and has devoted resources to educational resources on the device.

The Information also independently confirmed the presence of features previously reported by sources like Bloomberg, Nikkei Asia, and DigiTimes:

• The headset contains a large outward-facing display on the front of the device. It can show the facial expressions of the headset's wearer to people around them, seeking to reduce a sense of isolation when using the device.


• This outward-facing display features an ultra-low refresh rate and reduced power consumption, much like the always-on display of the Apple Watch and iPhone 14 Pro.


• Interior Micro OLED displays for each eye with a 4K resolution, made by Sony, forming an 8K image overall.


• Each eye is tracked by at least one camera, allowing the headset to accurately show the user's gaze on an avatar. Eye-tracking also enables the headset to perform foveated rendering to conserve power by only rendering imagery in full resolution directly where the user is looking.


• More than a dozen cameras and sensors capture both facial expressions and body movements, including the user's legs. Apple removed cameras to capture eyebrow and jaw movements when it developed machine learning to accomplish this.


• Both short- and long-range LiDAR scanners to map surfaces and distances in three dimensions.

The report claims that Apple supplier Pegatron assembled thousands of prototype units of the headset last year at a facility near Shanghai, having passed multiple prototyping stages and entering engineering validation testing. It added that the device was originally supposed to launch in 2022.

In addition, Apple has allegedly discussed pricing the headset at around $3,000 or more depending on its configuration, according to four people with knowledge of the matter speaking to The Information.

Related Roundup: Apple Glasses

Tag: theinformation.com

Related Forum: Apple Glasses, AR and VR

This article, "Report Reveals Wave of New Features for Apple's Mixed-Reality Headset, Including Digital Crown for Switching to Real-World View" first appeared on MacRumors.com

Discuss this article in our forums



source https://www.macrumors.com/2023/01/03/report-reveals-wave-of-features-for-apple-headset/