Siri’s search accuracy graded as a D, Google text search gets a B+

By , Jun 29, 2012

Curious to figure out which is more accurate for searching, Apple’s Siri or Google’s traditional search engine, Piper Jaffray’s Gene Munster pitted Siri’s voice-activated search against Google’s old-fashioned text-based searching. He and his team set out to ask their iPhones a total of 1,600 questions, 800 on the busy streets of Minneapolis, 800 in a quiet room. His findings will no doubt come as a surprise to even the most vocal Siri proponents out there…

Philip Elmer-DeWitt over at the Fortune blog summarized Munster’s findings:

 • Google understands 100% of the questions (not surprisingly, since they are keyed in)
• Google replies accurately 86% of the time
• Siri comprehends 83% of queries in noisy conditions, 89% in a quiet room
• Siri answers accurately 62% of the time on the street and 68% in a quiet room.

I think Munster’s non-scientific test paints a fair picture of a lot of catching up that Apple needs to do if it’s to match Google’s search experience, which is still the best bar none.

Musnter concludes in his note to clients:

In order to become a viable mobile search alternative. Siri must match or surpass Google’s accuracy of B+ and move from a grade D to a B or higher.

True that.

The analyst also opined that Apple is taking Siri criticism seriously, just as CEO Tim Cook recently suggested.

Munster estimates that Siri is more than two years behind Google in its learning curve, but he’s optimistic: “With the iOS 6 release in the fall, we expect Siri to improve meaningfully while reducing its reliance on Google from 60% to 48%.”

After all, Siri is is just a casual search engine.

However, with last October’s Siri introduction, Apple began containing Google on the iOS platform by driving some of the search queries that went to Google – especially local search ones – to its AI-driven personal assistant.

Apple also taught Siri a few new tricks in iOS 6, which largely stem from Siri’s integration with Apple Maps and partnerships with the social network for user reviews Yelp, movie reviewer Rotten Tomatoes and restaurant reservation service OpenTable.

It’s been estimated that Siri currently gets 60 percent of its answers from Google, 20 percent from Yelp, 14 percent from WolframAlpha, four percent from Yahoo and two percent from Wikipedia.

But competition isn’t standing still.

With this week’s introduction of Android 4.1 Jelly Bean, Google is looking to challenge Siri with Google Now, an improved search for Android devices which combines Voice Actions with everything Google has on you – your social graph, searches, calendar information, browsing habits, +1s and what not – to deliver more accurate and personalized results (Google Now vs. Siri comparison here).

Here’s Google’s promo clip highlighting Google Now features.

What do you think of Munster’s Siri vs. Google accuracy test?

Disclaimer: we’re still an iOS-focused blog, but this week is Google I/O and we think it’s worth keeping tabs on what the competition is doing.

  • Share:
  • Follow:
  • Charles Valdez

    Google Now is nothing like Siri. It’s just an app that tells you information. Siri is like the update Google search.

  • http://twitter.com/michaelhernandy michael hernandy :)

    i still find it creepy that it looks through your browsing habits and stuff, its like its always watching what you do.

    • Falk M.

      It’s just processing what Google already knows about you.
      ACTUALLY, they only added a good feature, BASED ON already existing creepy technology.

  • CollegiateLad

    Wait??? They’re comparing voice input to text input?

    • http://twitter.com/Nashafa Nathan

      NO! it’s voice vs. voice. Google and Siri both support voice.

      • http://www.facebook.com/people/Garza-Dave/1712273629 Garza Dave

        No it’s comparing voice input to text imput. “Google understands 100% of the questions (not surprisingly, since they are keyed in)”

  • Aric Bolf

    I think its great they did this before Siri 6.x comes out. I’d like to see comparisons between Siri 5.x and 6 on content Siri is capable of on both versions to see if it got better or worse.

    It’s speculation, but considering half of Siri if server side, I suspect they have been improving Siri on that end since it’s introduction. If they have, then they have learned some things about making the improvements on the server end easier to do and those changes will be included in Siri 6.x

    To think Siri doesn’t change for a whole year is, IMO, ridiculous.

    • http://www.facebook.com/nihasnebas Nihas Nebas

      siri is way better in ios 6…………………actuall yis damn fast in catching the voice and all

    • http://twitter.com/Nashafa Nathan

      I saw comparison with io6 (beta) and google is still WAAAAY better

  • http://twitter.com/MCaudebec Maxim∑

    Googles assistant just does a google search for you, its just not the same as siri

  • http://www.GoldenGateDomains.com/ Golden Gate Domains

    I was surprised at how inaccurate Siri was in a quite room compared to a noisy street!

    I would have fully expected to see a significant difference in accuracy without the confusion of a noisy street.

  • http://www.facebook.com/profile.php?id=1562666980 Milad Khahil

    All of those android fan here , remember who start all of this when the iphone came on 2007 there was no android no touch screen phone and Apple were the first so technically yes all android phone and windows phones are copy cats …..so shut up and smell the roses…..

    • http://www.facebook.com/people/John-Reden/100000410627122 John Reden

      Actually there were plenty of touch screen devices out long before the iPhone.
      Honestly Apple didn’t invent touchscreen technology. Everyone should know that. Apple gets all the credit because people refuse to acknowledge the things that inspired the iPhone. I remember watching Star Trek The Next Generation and seeing the characters use the PADD. Which is basically an iPad.

      Now before you say everyone else copies off of Apple again, think about who the real copycat is.

  • http://twitter.com/jc_ru ilya kazakov

    never had issues with siri, on contrary had with google voice recognition

  • Nicolas Loots

    Google Now is a bit creepy. Some robot voice looking through all of your personal stuff :S

  • Falk M.

    It’s pretty solid built for a “D”…
    Lately I played some loud music, iPhone resting in the dock next to the speaker placed behind (!).
    Spoke into it from a little closer, picked up everything, currectly responded.

    I don’t care about artificial tests, it’s working well on my end, that’s all that counts in my personal usage.
    Typical YMMW situation.
    It learns your voice actually, I’m sure Siri will get much better by time. :)