Google Details New Android Features Powered by Gemini AI

Google has announced new AI features coming right within the Android operating system, set to change how users interact with their devices. According to Sameer Samat, President of the Android Ecosystem, these updates mark a transformative moment in mobile technology, speaking at Google I/O.

Circle to Search, now built into the Android user experience, allows users to search anything on their screens with a simple gesture. Initially launched at Samsung Unpacked, this feature now includes full-screen translation and expanded availability to more Pixel and Samsung devices.

Starting today, Circle to Search can assist students with homework, providing step-by-step solutions for physics and math problems directly from their devices. Future updates will enable the feature to handle more complex problems, thanks to enhancements from Google’s LearnLM initiative.

The demo shows you can just circle text and you’ll get answers. This is unreal and students have it so easy nowadays with AI!

Gemini is also seeing improvements. The assistant will soon be able to overlay on top of any app, allowing users to drag and drop images into messages or get specific information from YouTube videos. Gemini Advanced users will have the added capability to ask questions about PDFs without scrolling through pages. These updates will be rolled out in the coming months.

Another key update is the introduction of Gemini Nano, an on-device foundation model that brings multimodal capabilities to Android.

Starting with Pixel devices, Gemini Nano will process text, sights, sounds, and spoken language, enhancing features like TalkBack to provide richer image descriptions for visually impaired users. This model also enables real-time scam detection during phone calls by recognizing fraudulent conversation patterns. Google says it will share more about this opt-in feature later. This scam feature is much-needed given how scammers are getting craftier nowadays.

Google says it will detail Android 15 and more on Wednesday, so stay tuned for that.