Google aims to “reimagine” Android with Gemini AI, touting it as a “once-in-a-generation event that reimagines what you can do on your phone.”
At Google I/O 2024, the search giant said it will build AI into Android in three ways: Build AI search into Android, make Gemini its new AI assistant, and leverage on-device AI.
Translated into everyday conversation, this means AI search tools like Circle to Search will be at the heart of Android. The AI-powered tool, which can identify physically circled objects and text in photos and screens, will be enhanced later this year to handle more complex problems like graphs and formulas.
Gemini AI, currently in the Google Pixel 8a, is the AI ​​foundation for Android, bringing multimodal AI (technology that processes, analyzes, and learns from information and input from multiple sources and sensors) to mobile interactions. system.
In practice, this means Gemini works with all kinds of apps and tools to provide context-aware suggestions, answers, and prompts. One example of this is using his AI in the Android Messages app to generate AI-made images to share in chats. Another is the ability to answer questions about the YouTube videos users watch, or pull data from sources like PDFs to answer very specific queries, such as specific rules for a sport.
Moreover, Gemini can learn from all this and use that information to predict what a person wants. For example, by knowing that a user is interested in tennis and chatting about the sport, you can provide them with the option (just kidding) to find tennis courts nearby.
The third aspect of Android's AI-enablement is ensuring that many smart things can be done on the phone without requiring an internet connection. Gemini Nano therefore provides a low-latency base model for onboard AI processing with multimodal capabilities. This allows the AI ​​to effectively understand what is being asked of and what is happening.
A real-world example is how Gemini detects fraudulent calls trying to trick you into your bank account information and alerts you to fraud before it happens. And since this process is done over the phone, you don't have to worry about remote AI eavesdropping on your private conversations.
Similarly, AI technology can use its contextual understanding to help visually impaired people accurately describe what they see, both in real life and online.
In other words, Google intends to make AI-centric Android more useful and powerful when it comes to finding and doing things. With Gemini Nano arriving later this year with multimodal capabilities on Pixel devices, we can certainly expect the Google Pixel 9 series to be the first smartphones to run a reimagined Android.