Share:

Random Test Reveals Google Text Searches Are More Accurate Than Siri

Share:

Gene Munster from Piper Jaffray has published the results of an unscientific test to put Siri against Google text searches. iPhone 4S and new iPad users know Siri is able to return Google search results via voice–but how accurate are they compared to actual text searches?

Here are the results from asking Siri 1600 questions (half indoors and half out in the streets):

  • Google understands 100% of the questions (not surprisingly, since they are keyed in)
  • Google replies accurately 86% of the time
  • Siri comprehends 83% of queries in noisy conditions, 89% in a quiet room
  • Siri answers accurately 62% of the time on the street and 68% in a quiet room.

“In order to become a viable mobile search alternative,” Munster writes, “Siri must match or surpass Google’s accuracy of B+ and move from a grade D to a B or higher.”

Part of Munster’s conclusions about the abysmal performance of Siri were based on the erroneous responses to questions such as:

Where is Elvis buried? Responded I can’t answer that for you. It thought the persons name was Elvis Buried.

What spices are in Lasagna? Responded with a Yelp search with lasagna on the menu.

Siri pulls results from a variety of sources, with the majority coming from Google, then Yelp, WolframAlpha, Yahoo and Wikipedia. The tests were completed using Siri on iOS 5.

Currently, Siri has been improved in iOS 6, so it would be interesting to see the same tests performed using the latest version as dictation has been widely improved. More importantly, I’d like to see how Siri performs against Google’s own voice search assistant, as comparing it against a text search isn’t really fair.

What do you think? Siri is still in ‘beta’ but improvements are coming. Should it still, be better?

Update 1: Siri on iOS 6 vs Android 4.1 Jelly Bean Google Search. The new Google search is quite impressive, as shown by Techno Buffalo:

[via Tech Fortune]

Share:

  • Garth

    I don’t think the question is whether it’s “fair” to pit a text input against a voice input, but whether siri is useful at the end of the day. As the results show, siri is still more of a novelty than anything.

  • definingsound

    I would like to see what Siri has to say to the query “Where was Elvis buried?”; the burying happened in the past and this is a properly formed sentence.

    It is silly to treat Siri like an english speaking human. Siri cannot be expected to understand a sentence fragment such as “Where is Elvis buried”. Particularly because the sentence mixes a present tense “is” with a past tense verb “buried”. An english native can understand the query, but a computer would hear something like “Where is Elvis jumped”. That Siri assumed the past tense “Buried” must be a name is quite advanced – Siri knew that “buried” could not be paired with “is”.

    How about “Where was Elvis Presley buried”, or “Where is Elvis Presley’s place of burial”. As long as we are speaking our searches, there is almost no penalty to using an extra word or two, in order to give the search agent a valid sentence.

  • Agreed bias shit from Fanboys. I love Apple too but am not so ignorant or oblivious that I ignore facts posted that are unbiased. It’s true that goggle search is better as of now. Sorry it disappoints me as well. But it’s time for apple to step up game on the ‘beta’ feature they used to sell the phone.

  • DaMan05

Deals