At Google I/O 2024, Google DeepMind unveiled Project Astra, which the company says is the future of AI assistants.
Project Astra can interact with the world around it by taking in information, remembering what it sees, processing that information and understanding contextual details. It also speaks much more naturally than the current form of Google Assistant and has no lag or delay.
In a demo, the user asks Project Astra to tell her when it sees something that makes sound. The assistant then replies, saying that it can see a speaker that makes sound. Continuing with the conversation, the user draws an arrow pointing to the tweeter and asks the assistant what that part of the speaker is called. Without delay, Project Astra could explain to her what that part of the speaker was called and provide details about it.
Project Astra is a prototype from @GoogleDeepMind exploring how a universal AI agent can be truly helpful in everyday life. Watch our prototype in action in two parts, each captured in a single take, in real time ↓ #GoogleIO pic.twitter.com/uMEjIJpsjO
— Google (@Google) May 14, 2024
Next, she moves over to crayons and, without saying, “Hey, Google,” she simply asks for an alliteration about “these.” And again, without delay, the assistant provides an alliteration about the crayons.
She shows Project Astra someone’s monitor displaying code and asks, “What does that part of the code do?” And the assistant, without hesitation, explains what that part of the code does.
Impressively, she even asks Project Astra if it remembers where it saw her glasses, to which it replies, telling her it’s by a red apple.
With its advanced reasoning capabilities, our prototype agent Project Astra was able to identify a number of famous faces in science – from just a few drawings. ✏️ pic.twitter.com/dK4O8vpqQx
— Google DeepMind (@GoogleDeepMind) May 15, 2024
It’s unclear what glasses she’s wearing, but it also features Project Astra and can interact with what she sees.
With this demo, it’s clear that Project Astra can process information faster. The tech is doing this by continuously encoding video frames and combining video and speech into a timeline of events, it then caches information for recall, such as when she asks the assistant if it remembers the location of her glasses.
Google says some of these features will be added to the Gemini app and other products later this year.
The demo was all done in one go, without stopping and without needing to recall the assistant; it’s pretty impressive and excites me for Gemini’s future. However, when it launches officially, the cynic in me thinks it’ll probably come with some bugs and won’t be as perfect as the demos. Plus, it’s worth keeping in mind that Google has a history of editing demos to make its AI look faster and more capable than it really is, so maybe take Astra with a grain of salt.
Find all of our Google I/O 2024 coverage here.
Image credit: Google
MobileSyrup may earn a commission from purchases made via our links, which helps fund the journalism we provide free on our website. These links do not influence our editorial content. Support us here.
