Apple has unveiled multiple open-source artificial intelligence (AI) models throughout the year, primarily focusing on small language models tailored for specific tasks. Continuing this trend, the tech giant from Cupertino has introduced a new model called Depth Pro. This vision-based model is capable of generating monocular depth maps from any given image, a capability that proves advantageous for creating 3D textures, augmented reality (AR) applications, and more. Developers associated with the project assert that the depth maps generated by Depth Pro surpass those produced by traditional multi-camera setups.
Apple Debuts Depth Pro AI Model
Depth estimation plays a crucial role in 3D modeling as well as in various applications including AR, autonomous vehicles, and robotics. The human eye functions as a complex lens system, adept at accurately perceiving object depth even from a single vantage point. In contrast, conventional cameras often struggle with this task, resulting in images that lack a convincing sense of depth.
To address this issue, traditional methods typically employ multiple cameras, which can be both labor-intensive and resource-heavy. Apple discusses these challenges in a research paper entitled “Depth Pro: Sharp Monocular Metric Depth in Less Than a Second,” showcasing how a vision-based AI model can generate effective zero-shot depth maps from single-camera images.
Visual representation of Depth Pro AI model’s depth mapping capabilities
Photo Credit: Apple
In creating the Depth Pro AI model, researchers utilized a Vision Transformer-based (ViT) architecture. While the output resolution was set at 384 x 384 pixels, the input and processing resolution elevated to 1536 x 1536 pixels, granting the model an enhanced capacity to capture intricate details.
The pre-print version of the paper, which is available in the online journal arXiv, claims that the AI model demonstrates impressive accuracy in generating depth maps for visually intricate objects, such as cages and the bodies and whiskers of furry animals. Notably, the depth map generation process is completed in just one second. The open-source AI model’s weights are currently accessible on a GitHub repository, enabling interested users to deploy the model using a single GPU for inference.
