Apple’s mixed-reality headset to offer iPad apps, virtual workouts

Apple Inc. is racing to build a trove of software and services for its upcoming mixed-reality headset, seeking to win over potentially wary consumers with apps that use the device’s novel 3D interface.

The offerings will include gaming, fitness and collaboration tools, new versions of Apple’s existing iPad features, and services for watching sports, according to people with knowledge of the plans. The roughly $3,000 headset is set to debut at an event in June, with the product going on sale months later.

Apple’s hardware products have long depended on having a robust assortment of software — epitomized by the Steve Jobs-era slogan, “There’s an app for that.” But there may be even more pressure when it comes to the mixed-reality headset, a technology that will be unfamiliar to many consumers and very expensive at the outset. Apple will have to sell the world on why anyone needs such a device and how to use it.

A big part of the effort is adapting iPad apps for the new headset, which blends virtual and augmented reality. Users will be able to access millions of existing apps from third-party developers via the new 3D interface, according to the people, who asked not to be identified because the plans are still under wraps.

The push will include optimized versions of the Safari web browser and Apple’s services for calendars, contacts, files, home control, mail, maps, messaging, notes, photos and reminders, as well as its music, news, stocks and weather apps. There also will be a version of the FaceTime conferencing service and Apple’s TV app. The features will look similar to their iPad counterparts.

A spokesperson for Cupertino, California-based Apple declined to comment.

Besides the core apps, the company is working on a version of Apple Books for the headset that will allow users to read in virtual reality. The company is also testing a camera app that can take pictures from the headset. On the wellness front, an app will help wearers meditate with a series of calming graphics, sounds and voice-overs.

In what is likely to be another highlight of the device, Apple is also working on a version of its Fitness+ service for the headset, which will let users exercise while watching an instructor in VR. Meta Platforms Inc., the market leader in VR headsets, has also tried to make workouts a part of its offerings.

And Apple is developing a version of its Freeform collaboration app for the headset, an effort that it sees as a major selling point for the product. The service, launched on the company’s other devices last year, will let users work on virtual whiteboards together while in mixed reality.

FaceTime, meanwhile, will generate 3D versions of users in virtual meeting rooms, Bloomberg News has reported. The idea is to make participants feel like they’re talking together in the same place — a goal of virtual reality goggles for some time. Meta has pursued a similar objective with its Quest headsets, but so far it hasn’t proven to be a must-have feature for most consumers.

The challenge for Apple’s headset is to become more than a niche product. Existing mixed-reality models haven’t generated much momentum, and they’ve lacked the kind of killer apps that fueled the iPhone’s popularity.

Apple will make the case that the headset is a compelling new way to both produce and consume content. The hope is to create a platform that can eventually replace the iPhone, though that’s likely a long way out.

Getting app creators on board is key to the mission, and that’s underscored by the place Apple chose for the headset’s unveiling: its Worldwide Developers Conference. Some company engineers have been working 80-hour weeks in the run-up to the launch, scheduled for June 5.

One selling point for the headset will be viewing sports in an immersive way. The company already offers games from Major League Soccer and Major League Baseball on Apple TV+, but it’s looking to make that a richer experience. In 2020, Apple acquired a Southern California company called NextVR to bolster this effort.

Immersive video will also extend to the TV app, where the company is planning to let users watch video in different virtual environments, such as a desert or the sky.

Apple aims to position the headset as a device for getting work done. The platform will support its Pages word processing, Numbers spreadsheet and Keynote slide deck apps, as well as iMovie and GarageBand for video and music production.

Gaming will be a central piece of the device’s appeal, too. That’s a reversal from Apple’s stance earlier in the product’s development, when it wasn’t planning to focus on that category as much.

The company has been working with a small number of developers for months to help them upgrade their existing software for mixed reality. And the gap between the device’s announcement and its release will give other developers time to optimize their apps for the new platform.

A key part of the product will be a dial — like the digital crown on the Apple Watch — that lets the wearer move between virtual and augmented reality. When in VR, users will be fully immersed in their apps. With AR enabled, they’ll see the real world around them in a similar fashion to ARKit experiences on iPhones and iPads.

The device will be able to run multiple apps at the same time, floating within the mixed-reality interface. Like other headsets, the product will remember where you were physically when using apps. If, say, a user runs the web browser from the kitchen, goes to another room and then comes back, the browser will reappear.

Many iPad apps from the App Store will be able to run on the headset, either automatically or with slight modifications. The device’s operating system, xrOS, is based on the iOS platform, laying the groundwork for that compatibility. It’s all part of a broader push by Apple to have apps run more seamlessly across its hardware, including Macs, iPhones and iPads.

Wearers will be able to operate the headset using eye and hand gestures. It can determine where people are looking and then users can pinch their fingers to select items and move through menus. The device will feature an in-air virtual keyboard but can also connect to an actual keyboard for improved productivity.

The headset’s home screen will look similar to the iPad’s, and it will offer a familiar control center for adjusting Wi-Fi, Bluetooth and volume levels. Like the iPad, it will also have a biometric system for unlocking the device. But it will rely on a scan of the user’s eyes, rather than a face or fingerprint.

It also will have another familiar touch: Siri voice control.