The tech world has seen its fair share of Greatest Battles of All Time. But there’s one currently brewing that has captured the attention of industry after industry, with the rewards and consequences having effects on countless aspects of everyday life.
It’s Google versus ChatGPT — and the hype is legit!
ChatGPT, a fledgling chatbot too young to remember when Elon Musk bought Twitter, is already in a battle royale with Google for what Google holds most dear. The search behemoth, clearly taken aback with ChatGPT’s initial impact, quickly announced its counterattack in the form of Bard. (And that heavyweight match doesn’t even factor in Bing, sprinting toward the ring with its own chatbot that could instantly give it Search cred.)
Right now, as ChatGPT takes aim at Google’s Search, a team of communication and search engine experts with the e-learning platform Preply worked to see which was smarter, assessing their performances in response to 40 questions.
“The questions cover a wide range of topics — from civics to sex — and they require answers both long and short, subjective and objective,” wrote Preply’s Matt Zajechowski. “Some are practical, and others are existential. A few are based on false premises. Others are morally sensitive, and at least one is unanswerable. We challenged each AI to get the answer right, to deliver it swiftly and clearly, to add appropriate context, and to be generally helpful to the reader.”
Source: Preply
In response to what Preply categorized as “basic questions,” Google won 7 (against ChatGPT’s 4 wins, with one tie), excelling at some with “more current” answers as well as answers that featured available Google functions such as a calculator or video. Both had examples of wins based on being “more clear.”
Source: Preply
With “intermediate questions,” ChatGPT won 15 out of 21 times, often with “more thoughtful” answers, or providing “more context” or “more detail.” As Preply explains regarding ChatGPT’s win for the question, “Does Facebook listen to me,” “ChatGPT swiftly explains the confusion in a helpful way. Google serves articles with similar explanations, but there’s more to wade through than necessary. Also, the featured snippet on Google is from Facebook, which could make some users suspicious.”
In half of Google’s “intermediate” wins, a deciding factor was an answer that was “more actionable” (something ChatGPT did not achieve at this stage). In response to “best shoes for running,” for instance, Google provided helpful options through Google Shopping as well as an up-to-date relevant article in its top links.
Source: Preply
The category of “advanced questions” was much closer, with ChatGPT being deemed the winner 4 out of 7 times. One telling difference was to the question “Is God real,” to which Preply praises ChatGPT’s answer as “tactful, impartial, inclusive, and complete” while calling out the elder search engine for displaying four ads from churches. “In responding to one of the most fundamental and defining questions known to humankind — one that is wholly subjective and without consensus — Google sells its answer to the highest bidder.”
Not only did ChatGPT beat Google in wins accumulated, but it won in the subcategories Preply designated as static- and fluid-questions, objective- and subjective-questions, and low- and medium-stakes. (Google won on questions considered “high stakes.”)
In the end, Preply acknowledged the strengths and weaknesses of both. “Google is a quick and dynamic information delivery system,” the article said, “but one that appears to fail almost as much as it succeeds by putting users in front of the content we don’t want, in front of content that’s reductive, or simply leaving us to do a lot of the work ourselves. Of course, it’s hard to judge Google too harshly for any of this. After all, its search engine has always been a go-between, a tool that presents the best of what we have to offer right back to ourselves.”
In addition to deeming ChatGPT the winner, Preply tries to pinpoint what makes its answers special.
“Meanwhile, ChatGPT performs like something else — yes, an entity trained entirely on human thought that preceded it, but also an author unto itself.”
To read more about Preply’s methodology — as well as ChatGPT’s response to their analysis — visit preply.com.
SEE FOR YOURSELF
The Magazine Manager is a web-based CRM solution designed to help digital and print publishers manage sales, production, and marketing in a centralized platform.
