Multimodal Autoregressive Pre-training of Large Vision Encoders | #ai #computervision #apple #2024

Paper: https://arxiv.org/pdf/2411.14402 Github Link: https://github.com/apple/ml-aim This research introduces AIMV2, a family of large-scale vision encoders pre-trained using a novel multimodal autoregressive method. Unlike previous contrastive methods, AIMV2 simultaneously predicts image patches and text tokens, offering scalability and simplicity. The resulting models demonstrate strong performance across various downstream tasks, including image recognition, object detection, and multimodal understanding, often outperforming state-of-the-art alternatives. Extensive experiments explore AIMV2's scaling properties and the impact of design choices, showing its robustness and versatility. The work concludes that AIMV2's unified objective function enables efficient training and superior performance. ai , computer vision , cv , apple , artificial intelligence , arxiv , research , paper , publication