We’re all familiar with the amazing capabilities of generative AI. Enthusiasts talk about reshaping the economy, saving white-collar workers from repetitive tasks, and breaking new creative grounds. With our faithful robot companions by our sides, we will stride forward into a better future, and so on.
I decided to check whether we humans were, in fact, using our newfound capabilities to push back the frontiers of human knowledge and capability. Google search is often the quickest path to the zeitgeist, so I wrote some code to ask Google some simple questions about the most popular AI tools currently available (based on ClickUp’s “Always Up-to-date List of the 50 Best AI Tools”, with a few extras added in) and then to scrape the section on the results page entitled “People also ask,” which lists popular questions often asked on the same topic. I then search Google again with those questions to get a few more questions, for about 8–10 per tool.
Reading through this list of questions pretty quickly gives you a sense of what each tool isactually being used for, based on what people want to know about it. To put some objective rigor around my conclusions, I identified the less common words among the frequently asked questions, defining “less common” as any word outside the top 1000 words in the Google Web Trillion Word Corpus (I excluded a few that were not meaningful in the context, such as ‘app’, as well as some common typos).
I then classified the distinctive words that remained into different categories, based on how they were used in the individual questions.
- NSFW: words generally used in an adult context. Google screens most of these out, but there are others that make it through that have similar implications, based on how they’re used in these questions.
- Cheating: words associated with questions about using generative AI when it’s not allowed, such as school. Most of these were words related to detection, such as “turnitin,” a common anti-plagiarism tool, or to the context of cheating (“teachers” or “university”).
- Utilitarian: everything else.
I then used the ratio of these three types of words in the questions asked about each AI tool to give those tools a score along each dimension.
The chart below shows the scores for a sample of AI tools scaled and plotted on a ternary plot, with a Venn diagram overlaid on top to make the classification more apparent. I’ve attached a representative question to each, in order to give a sense of what people are asking.
The most obvious point is that there are plenty of people out there looking for something questionable from generative AI, and plenty of tools designed more or less explicitly to give it to them. NSFW is mostly its own little island (and to be fair, neither of the tools above allow adult content), while cheating is more of a spectrum–ChatGPT has a reasonable proportion of people who want to know if they’ll get caught using it for their homework, while a number of people ask about legitimate use cases on Wordtune.
OpenAI’s dominance is also easy to see. “ChatGPT” is all but a synonym for generative AI for most people, and so many of the questions asked about it are much more basic questions that are really about AI overall. Other foundation models (including OpenAI’s own GPT-4) are relegated primarily to questions comparing them to ChatGPT and to each other.
The final observation from these commonly asked questions is that aside from a few household names and the two use cases of cheating and adult content, most gen AI tools are struggling to break through. I had to modify question wording for every tool named after an ordinary word (like Spinach or Paradox) because otherwise I only got questions about the common use of that word. And to my surprise, almost none of the questions people are asking show any real understanding of what a given tool (outside the categories above) is for. There were a few questions about coding-assist tools that demonstrated at least some level of familiarity, but most of the other AI tools’ questions were dominated by basic queries like “What is it?”, “How does it work?”, and “Is it free?”
Obviously we are still at the beginning of the generative AI revolution, and it probably shouldn’t be a surprise that cheating and talking dirty are two of the earliest use cases to get traction. Over time, we should see people building their understanding of what tools are available to them more broadly. I’m optimistic that next year’s version of this analysis won’t fit inside a three-circle Venn diagram.
