First there was Google Tango. Then it was Mark Zuckerberg promising Facebook AR Studio, and now the next biggest company to throw its hat in the Augmented Reality ring is Apple. The Cupertino-based company announced at its Worldwide Developers Conference (WWDC) that ARKit, an augmented reality platform, is going to be a key feature of its latest OS.
An Apple AR Wasn’t a Surprise
It has been nearly a year since Tim Cook first hinted at the fact that Apple may be developing an AR platform. During 2016’s Apple earnings day, Cook fielded questions about the Pokémon Go phenomenon that had driven the world to obsession as they tried to catch them all. His answer was as follows. To get the full effect read it in Tim Cook’s soft southern accent and pronounce Pokemons like Pokay-mans:
“We have been and continue to invest a lot in this. We are high on AR for the long run, we think there’s great things for customers and a great commercial opportunity. The number one thing is to make sure our products work well with other developers’ kind of products like Pokemon, that’s why you see so many iPhones in the wild chasing Pokemons.”
His response suggests Apple may be telling the truth when it calls ARKit “the largest platform in the world” because it has the potential to be an AR tool built for developers. In the past, AR developers have had to tweak existing camera software in order to support an AR environment, in which it has to track its own position in relation to objects in reality. It’s an added step that software developers have to engineer before they can release an app that supports AR.
With ARKit built right into iOS 11, Apple has taken over the hardest job an AR developer has to do, allowing these creators to focus entirely on the gaming experience itself.
Apple Relies on World Tracking
World tracking is a consumer friendly term that’s perfectly intuitive in meaning. Though it describes a highly complex process, ultimately ARKit is engineered to chart the world it “sees” and keep track of the objects that inhabit it.
Its visions relies on sensory input that most other augmented reality use: VIO, which is short for Visual Inertial Odometry. VIO compares the raw data collected from the camera to the info gathered by the phone’s built-in sensors responsible for creating its inertial reference frame. Instruments like the accelerometer and gyroscope contribute towards creating this frame, and they’re integral to the software’s ability to measure the position and orientation of an object.
The ARKit platform will synthesize the data pulled from these sensors in order to pinpoint or anchor with improved accuracy digital objects in a 3D space. In order to project the AR world realistically onto the 3D space, the software will take into consideration scale and depth in addition to location. It will even calibrate its Augmented Reality according to the amount of light available in the real world. Its expected ARKit’s motion tracking will be faster, more stable, and better at scaling than previous AR platform attempts regardless of how you use it.
It’s too early to tell if ARKit will impede other apps or vice versa, but Apple claims it will require less CPU and battery to run. As for hardware, it’s safe to say it will work well with accessories applied to the iPhone. Take, for example, the skin wrapped around your handset. It won’t hinder ARKit’s abilities, as long as it has an accurate cut around the camera that doesn’t obstruct the lens. In fact, it will make using your phone as your navigate AR a little easier. The special iPhone 7 skins designers at dbrand have exclusive rights to authentic 3M vinyl. Not only does this mean they offer colors and finishes you’ll find nowhere else on the Internet, but it also means iPhone 7 Plus skins from dbrand are texturized to provide better grip as you explore the space where the digital and real worlds collide.
ARKit’s Earliest Applications
To show off what the platform can do, Apple shared a video of how it will affect apps like Pokémon Go at WWDC. Previously, the iPhone performed notoriously poor with the app’s AR portion of the game, as caught Pokémon would float within the field of view without any relation to the 3D environment. Most users chose to opt out of it once the novelty of seeing a Pokémon in the park wore off.
In the keynote’s Pokémon Go video, Pikachu was fixed in place, regardless of how the user moved his or her phone, suggesting an improved experience awaits gamers in the fall. Though subtle, the differences hold promise for the future.
Peter Jackson and his studio Wingnut also showed off a demo of their game developed with Unreal Engine 4, which is supported by ARKit. The demo involved an AR landscape on top of a table. A small town, which eventually became the scene of a war, could be seen through the presenter’s iPad. The table itself remained clear.
Other iOS developers have spent the last few weeks experimenting with Apple’s ARKit, the result of which you can find here. These early renderings are a little silly compared to the sleek demos at the WWDC, but it demonstrates its simplest applications well.
iOS 11 beta will be available to testers later on this month, but consumers worldwide will only have to wait until Fall 2017 to get its final version. Soon, what’s only available to a handful of software developers will be accessible to billions around the world, and it’s going to change the way we use our iOS handsets as mobile gaming devices forever.