Google Assistant is getting generative AI capabilities with Bard

Google Assistant is expected to roll out as an opt-in experience to all users on Android and iOS mobile devices in the coming months.

Google is bringing generative artificial intelligence (AI) capabilities to its virtual assistant, amid a broader push to integrate the technology across all its products.

On October 4, Google introduced Assistant with Bard, a virtual assistant that combines the generative and reasoning capabilities of its AI chatbot Bard with Assistant's personalised help.

During the company's Made by Google event today, Sissie Hsiao, vice president, Google Assistant and Bard said that the assistant has been good at handling quick tasks such as setting timers, giving weather updates, and making quick calls.

However, she said there is "so much more that we've always envisioned a deeply capable personal assistant should be able to do. But the technology to deliver it didn't exist, until now."

"Bard's language understanding, complex reasoning and generative capabilities have fueled our excitement in realizing our vision for Google Assistant. It's a step towards our vision to deliver the world's most helpful personal assistant," she said.

The new virtual assistant is rolling out shortly to "trusted testers" and is later expected to roll out as an opt-in experience to all users on Android and iOS mobile devices in the coming months. One can interact with it through text, voice or images and also use it to take a few actions for them.

Read: Generative AI helping us reimagine Search, other products: Alphabet's Sundar Pichai

The new assistant builds on Bard's latest update that features deeper integration with several of Google's consumer applications and services including Gmail, Docs, YouTube, and Maps.

For instance, one can open Assistant with Bard and ask questions such as "Catch me up on any important emails I missed this week" to get a rundown on the key emails they need to know about, from their inbox.

Assistant with Bard can also pull together all the information one needs from different apps and services, to help users get personal tasks done faster. This could be activities such as trip planning, making a grocery list, asking for place recommendations and building a weekend itinerary among others.

On Pixel devices, one can also pop it open on the top of a photo to get contextual assistance with personal tasks, like asking to write a social caption among others.

Read: YouTube unveils new generative AI tools to help creators make better videos

First device to run Google's foundation AI models on-device

Google's senior vice president, devices and services, Rick Osterloh said that Google Pixel 8 Pro, which was unveiled at the event, will be the first phone to run the company's foundation AI models directly on the device. It will also feature an on-device custom generative AI image model.

"We've worked closely with research teams across Google to take advantage of their most advanced foundation models and to distill them into a version efficient enough to run on our flagship Pixel," he said.

This would lead to improved AI experiences on Pixel 8 Pro, since the device can run generative AI with up to 150 times more computations than the largest machine learning (ML) model on Pixel 7, he said.

For instance, an improved magic eraser feature can help users remove large objects and people without smudging their photo. In the coming months, the recorder app can provide on-device summaries while Pixel 8 Pro's on-device large learning model (LLM) will power smart replies on the company's keyboard app Gboard.

At the event, Google also unveiled new versions of its smartwatch Pixel Watch, wireless earbuds Pixel Buds Pro, Fitbit fitness tracker and its custom silicon chip Tensor G3.