Google Home named most intelligent smart speaker in annual speaker test

According to an annual smart speaker test from Loup Ventures, Google Home maintains an intelligence advantage over the competition.

Loup, a reasearch-driven venture capital firm, tested Google Assistant, Alexa, Siri and Cortana via their respective smart speakers. It conducted tests on the second generation Amazon Echo, Google Home Mini, Apple HomePod and Harman Kardon Invoke (for Cortana).

The company specifically tested these assistants on speakers because, while the underlying technology may similar to smartphones, the user experience and queries are often vastly different.

Loup asked each speaker the same 800 questions and graded responses on two metrics. First, did the speaker understand the query? Second, did it answer correctly?

Questions also fell into the following five categories:

• Local – questions like “Where is the nearest coffee shop?”
• Commerce – questions like “Can you order me more paper towel?”
• Navigation – queries like “How do I get to uptown on the bus?”
• Information – questions like “Who do the Leafs place tonight?”
• Command – queries like “Remind me to call Steve at 2pm today.”

The Assistant and Siri stood far above the competition

The Google Assistant blew away the competition, answering almost 88 percent of queries correctly. Further, it understood 100 percent of the questions asked.

Siri nabbed second place, answering about 75 percent of questions correctly. It understood 99.6 percent of queries.

Alexa captured third with 72.5 percent correct answers and 99 percent of questions understood.

Finally, Cortana came in last with 63.4 percent and 99.4 percent respectively.

Further, Google Home dominated four out of five categories. It lost out to Siri on the HomePod in the ‘Command’ category. According to Loup, this is due to HomePod’s deep integration with iPhone. It can pass on full SiriKit requests, such as those regarding messages, list, calendars and more.

Also of note was the advantage Google Assitant had in the ‘Commerce’ section. Google Home nailed 86 percent of the questions in Commerce compared to Alexa’s 52 percent success rate.

Part of this is due to Google Assistant’s ability to answer questions about products and where to buy them. Further, Google Express — which isn’t available in Canada — is just as capable as Amazon for purchasing items.

Finally, Google Home and HomePod were able to leverage their respective proprietary maps data to accurately answer questions in the ‘Local’ and ‘Navigation’ sections. Loup suggests this will be a major factor in both the Assistant and Siri’s continued dominance of the market.

There’s plenty of interesting information and data revealed in the study. To see the whole report, you can visit the Loup website here.

Source: Loup Ventures