Google’s Gemini screen automation could soon allow placing orders or booking rides effortlessly

Google Gemini

• Google tests Gemini to automate tasks like booking rides or placing orders
• Gemini's new screen automation may work with select apps on Android 16 QPR3
• Users are accountable for Gemini's actions; privacy warnings will be issued.

Did our AI summary help?

Google is set to improve Gemini’s agentic capabilities, allowing the AI tool to perform everyday tasks automatically on the user’s behalf. According to new details spotted in the latest beta version of the Google app, an upcoming screen automation feature is expected to let Gemini interact and automate basic actions on behalf of a user, such as placing orders or booking rides, to save time. Here’s how the feature could work.

Gemini could soon perform tasks in various apps on the user’s behalf

During an APK teardown of the Google app version 17.4 beta, 9to5Google has discovered a new Labs feature called ‘Get tasks done with Gemini,’ which is internally codenamed ‘bonobo,’ which uses Gemini screen automation to interact with certain apps on a user’s smartphone. Current string codes suggest that this feature will work on Android 16 QPR3 and would only be available on a handful of apps.

In practical terms, this could mean asking Gemini to order dinner, place orders, or book a cab without ever opening a certain app. However, the company will reportedly warn that users remain responsible for actions carried out by Gemini on their behalf, and they can stop the agent at any time to perform that specific task manually.

Google could also alert users about potential privacy implications, as screenshots can be reviewed by trained reviewers to improve Gemini’s capabilities when the optional Keep Activity setting is enabled. The company also urges people not to enter payment details or other sensitive information into Gemini chats and to perform tasks manually during emergency situations. Google has not announced a release timeline or confirmed the full list of compatible apps that will work with this feature of Gemini, as the current screen automation tool is still in the experimental stage.