Apple’s research team has made a significant leap in artificial intelligence with new methods for training Large Language Models (LLMs).
A multimodal LLM is essentially an advanced type of artificial intelligence that can understand and process different types of data, specifically text and images, at the same time, while then giving an answer based on what it has learned from. Examples include OpenAI’s ChatGPT and Google’s Gemini.
This advancement, detailed in a paper titled “MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training,” introduces techniques for combining text and images in AI training, enhancing the capabilities and flexibility of future AI systems, reports VentureBeat.
The study, which was recently published, showcases how blending various types of training data and model architectures can significantly improve AI performance across several benchmarks. “Our large-scale multimodal pre-training approach, utilizing a mix of image-caption, interleaved image-text, and text-only data, is essential for state-of-the-art few-shot results,” said Apple researchers.
Apple found that having clear, high-resolution images and a good image encoder—a tool that helps the AI to read and understand images—is really important for the AI to perform well.
The way the AI connects and makes sense of both images and language together is not as important. This means for the AI to be really effective, especially in understanding and working with images, it needs to have high-quality images and the right tools to process them.
Apple’s research found that their most advanced AI model, which has 30 billion parameters (a measure of its complexity and potential for learning), is really good at understanding and solving complex problems with minimal guidance This model can look at several pictures and, just like a human piecing together a story or solving a puzzle, it can think through multiple steps to come up with an answer or explanation.
According to a previous Bloomberg report, Apple plans to invest about $1 billion annually in AI development. Apple is said to be working on a large model framework codenamed “Ajax,” while a chatbot named “Apple GPT” is underway. These investments in AI are said to change how Siri, Apple Music and other company services will work.
“AI and machine learning are core to our products. While I won’t divulge specifics, our significant investments will become evident in our products, with AI at their core,” said CEO Tim Cook on a recent earnings call. Please make Siri great again, Tim.
All eyes are on Apple at its upcoming WWDC, where it will likely reveal more about its work on AI and how it will be integrated into its mobile and desktop operating systems.
