Google Assistant has emerged victorious in the latest “Digital Assistant IQ Test.”
Loup Ventures’ annual digital assistant test results were revealed today, providing a glimpse as to how the top four compare to one another. According to this year’s data, Google Assistant came out on top by a significant margin over Apple’s Siri, Amazon’s Alexa, and Microsoft’s Cortana.
The test involved asking all four assistants the same 800 questions each, whose answers were then graded on two metrics. The first metric was determining whether the assistant understood what it was being asked, while the second was whether the assistant gave a correct response to said query. According to Loup Ventures, the question set was broken into 5 categories:
All four assistants were great at understanding queries — Google Assistant had a 100% success rate, Apple’s Siri understood 99% of queries, and Microsoft’s Cortana and Amazon’s Alexa were only slightly behind with a 98% success rate.
Answering questions correctly, however, was a different story completely. Google Assistant managed to give the correct answer 85.5% of the time, while Siri answered correctly in 78.5% of cases, up much higher than its April 2017 result of 66.1%. Alexa was able to answer 61.4% of queries correctly, while Microsoft’s Cortana managed to answer only 52.4%, up from last year’s 48.8%.
Siri’s biggest strengths were in answering music-related queries, and its versatility in controlling the phone, smart home accessories, and other features, which Loup said included greater flexibility in determining a user’s intent. Notably, unlike the limited versions of Siri found on the HomePod and Apple TV, Siri on the iPhone is fully featured, enabling it to reach higher scores than in the earlier smart speaker test.
“We found Siri to be slightly more helpful and versatile (responding to more flexible language) in controlling your phone, smart home, music, etc,” the study’s authors said. “Our question set also includes a fair amount of music-related queries (the most common action for smart speakers). Apple, true to its roots, has ensured that Siri is capable with music on both mobile devices and smart speakers.”
While Apple has definitely managed to improve Siri over the last one year, Google’s Assistant continues to have a major lead in many categories thanks to Google’s core strength: search.
