Siri’s search accuracy graded as a D, Google text search gets a B+

Curious to figure out which is more accurate for searching, Apple’s Siri or Google’s traditional search engine, Piper Jaffray’s Gene Munster pitted Siri’s voice-activated search against Google’s old-fashioned text-based searching. He and his team set out to ask their iPhones a total of 1,600 questions, 800 on the busy streets of Minneapolis, 800 in a quiet room. His findings will no doubt come as a surprise to even the most vocal Siri proponents out there…

Philip Elmer-DeWitt over at the Fortune blog summarized Munster’s findings:

 • Google understands 100% of the questions (not surprisingly, since they are keyed in)
• Google replies accurately 86% of the time
• Siri comprehends 83% of queries in noisy conditions, 89% in a quiet room
• Siri answers accurately 62% of the time on the street and 68% in a quiet room.

I think Munster’s non-scientific test paints a fair picture of a lot of catching up that Apple needs to do if it’s to match Google’s search experience, which is still the best bar none.

Musnter concludes in his note to clients:

In order to become a viable mobile search alternative. Siri must match or surpass Google’s accuracy of B+ and move from a grade D to a B or higher.

True that.

The analyst also opined that Apple is taking Siri criticism seriously, just as CEO Tim Cook recently suggested.

Munster estimates that Siri is more than two years behind Google in its learning curve, but he’s optimistic: “With the iOS 6 release in the fall, we expect Siri to improve meaningfully while reducing its reliance on Google from 60% to 48%.”

After all, Siri is is just a casual search engine.

However, with last October’s Siri introduction, Apple began containing Google on the iOS platform by driving some of the search queries that went to Google – especially local search ones – to its AI-driven personal assistant.

Apple also taught Siri a few new tricks in iOS 6, which largely stem from Siri’s integration with Apple Maps and partnerships with the social network for user reviews Yelp, movie reviewer Rotten Tomatoes and restaurant reservation service OpenTable.

It’s been estimated that Siri currently gets 60 percent of its answers from Google, 20 percent from Yelp, 14 percent from WolframAlpha, four percent from Yahoo and two percent from Wikipedia.


But competition isn’t standing still.

With this week’s introduction of Android 4.1 Jelly Bean, Google is looking to challenge Siri with Google Now, an improved search for Android devices which combines Voice Actions with everything Google has on you – your social graph, searches, calendar information, browsing habits, +1s and what not – to deliver more accurate and personalized results (Google Now vs. Siri comparison here).

Here’s Google’s promo clip highlighting Google Now features.


What do you think of Munster’s Siri vs. Google accuracy test?

Disclaimer: we’re still an iOS-focused blog, but this week is Google I/O and we think it’s worth keeping tabs on what the competition is doing.