Visibility in the AI Future: Which Brands are Recommended By OpenAI?

Johannes Beus

(Author)

Published: 15.05.2023

The progress of generative artificial intelligence in recent months has been breathtaking. Through its close ties with OpenAI (ChatGPT), Bing already has the feature integrated into the search results. Google will have to follow, and is has already announced the first steps.

If you’re short on time, we’ve compiled a summary of the key findings from our data study on brand recommendations from OpenAI here:

• We asked OpenAI for brand recommendations for 10,000 different product categories.
• For almost all queries, OpenAI was able to recommend 10 brands. There were only 674 topics for which there was no response – these were often topics from the fields of adult entertainment, health or politics.
• In total, OpenAI recommended 23,856 different brands. This means each brand was recommended on average around 3.9 times.
• However, as is so often the case in search, successes are not evenly distributed: 15,795 brands (i.e. 66%) were recommended only once, only 570 brands (2.4%) were mentioned at least 20 times by OpenAI.
• The top 10 most recommended brands by OpenAI (incl. the number of recommendations): Samsung (1871), Sony (1427), LG (1042), Apple (936), HP (809), Nike (740), Dell (719), ASUS (699), Philips (637), Adidas (637).
• Some brands are more relevant to OpenAI than others. Apple, Shimano, Wacom and Fluke are particularly high up in the recommendation lists. On the other hand, Mikrotik, BMW and Dual Electronics are at the bottom of the lists more often than not.

Interested in the details and background now? Let’s jump straight into the longer version of our analysis of brand recommendations in the AI future:

Approach: What exactly did we measure and how?

To date, OpenAI is the leading technology for practical text content creation. Although there are signs that other providers or open source solutions may become more important in the future, that remains uncertain. For our analysis, we use the ChatGPT API from OpenAI (for the nerds: chat.completion with gpt-3.5-turbo-0301).

First, we created a list of 10,000 generic keywords from the product environment. For these keywords, large e-commerce domains such as amazon.de or otto.de rank in the first positions in the Google search results, but there are no sitelinks (to exclude brand terms).

To give you can idea of these keywords, here is an excerpt from the keyword list: calculator, washing machine, greenhouse, pendant light, motion detector, night-vision device, domestic waterworks, tomato sauce & side awning.

For the list of 10,000 keywords, we asked OpenAI for 10 recommendations of relevant brands. For only 674 keywords OpenAI did not respond, for all other keywords 10 recommendations were provided.

Whenever the AI did not want to (or could not) respond, it was usually for keywords that contradict American moral or legal practices: Adult entertainment, politics and health seem to be so dangerous for OpenAI that it prefers not to comment.

23,856 different brands but only 2.4% with more than 20 mentions

Since OpenAI also reliably recommended 10 brands for all 9,326 keywords, there were 93,260 results in total.

These results are divided among 23,856 different brands – so on average, each brand is mentioned around 3.9 times in OpenAI’s recommendation lists.

However, the average can sometimes show a distorted picture of reality. This is also the case here. Although most brands are recommended only once, there are some brands that are recommended more than 20 times:

OpenAI recommends more than half of all brands (66%) only once. Another 24% of brands are mentioned no more than five times for different keywords. Only a very small percentage of brands, namely 2.4%, are recommended more than 20 times in the queries.

We see that the visibility is mainly concentrated on a few, very well-known brands. This is comparable to Google search results, where only a few websites have a Visibility Index of more than one point.

Top Brands: 109 brands had over 100 recommendations from the AI

Next, let’s take a closer look at the particularly successful brands. In the following table that you can scroll through at the end by clicking “Next” you will find the brands that have been recommended at least 100 times by the AI for various keywords:

OpenAI’s top recommendations are largely in line with our expectations. Many well-known retail brands are included, although it seems that some US brands are slightly more represented. Comparing these with the top brands on Amazon, we see many matches. This means that the AI appears to make ‘correct’ recommendations.

Above Average: Which brands are recommended first?

As described above, we asked the AI to give us ten suggestions for each product category. The most important and relevant suggestions are at the top of this list, while the less relevant ones are at the bottom – similar to Google search results.

If a brand is moderately relevant and has some recommendations, it should generally be in the middle of the list. This is confirmed by our data: the average ranking of all brands that were mentioned at least 25 times is 5.1, right in the middle.

Now it is interesting to find out which brands deviate from this norm. Are there certain brands that are often at the top of the recommendation lists and vice versa? Are there also brands that usually end up at the bottom of these lists?

On the left-hand side, you will find brands that are favoured by the AI model and are often placed at the top of the recommendation lists. A good example of this is Huion. When the AI recommends this brand, it is almost always at number one.

In contrast, the right-hand side lists brands that the AI tends to place at the bottom of the recommendation lists. They seem to be used more as a kind of “filler material”. The AI does not seem to attach much relevance to these brands.

One brand seldom comes alone: Which competitors does the AI identify?

Although many brands do not want to admit it, they usually compete with others for the same customers and offer similar products. The data we collected clearly shows these correlations:

Many technology brands produce some of the most popular products. When these brands appear together in the lists it’s possible to see competition and relative popularity of these brands. Isolating the TV brand relationships gives us the following picture:

Conclusion

There is no doubt that internet search will change more in the coming months and years than in previous years. But how the changes will come about in concrete terms remains to be seen.

Our analysis of brands based on OpenAI responses shows us that measurability of mentions and thus visibility of brands and products in those responses will also play an important role.

Change is a part of the internet. Competition stimulates business, and for Google, a breath of fresh air is helpful after a long period of market dominance. That Google will win the battle for AI search is by no means certain today.