Google’s TalkBack can now answer questions about on-screen images: Here’s how it works

Google TalkBack

Earlier this week, on the occasion of Global Accessibility Awareness Day, Google announced a suite of AI and accessibility tools that aim to offer richer and clearer image descriptions. TalkBack, which was first launched in 2009, reads aloud what's on your screen and lets you navigate your device with a hands-free operation is now getting more useful, as users can ask follow-up questions about the contents of the images or the entire screen.

TalkBack now becomes more useful

Up until now, TalkBack, which gained Gemini capabilities last year, has been able to provide assistance to blind and low-vision Android users by offering more precise and detailed descriptions of images found online without accompanying alt text. However, the updated feature now allows users not only to hear descriptions but also to ask questions about the images or content on their screen.

For example, if a friend texts you a picture of a football, Gemini-powered Talkback will give you a description of the image and also answer follow-up questions about things like the make and the description of the on-screen football. In other scenarios, where this new TalkBack feature could help is if you are shopping from a particular app, you can ask Gemini about the material of an item you’re interested in, or if there is any discount available.

This update is rolling out in English in the US, UK, Canada, and Australia for devices running Android 15 and above, with a broader rollout to follow soon. Other than this, Google has also introduced a wider release of its Expressive Captions feature, easy access to PDF files on Chrome, and the Page Zoom functionality in Chrome for mobile users.