AI is like Sugar

The number of AI models is growing fast leading to AI being thought of as a commodity. But it doesn't mean what you think it means.

In February 2024, in part 1 of our thesis about the need for verifiability, we made a statement about the proliferation of AI models:

The end-state of this pace of development and proliferation of models is that there will be as many models as there are websites on the internet. An alternative thought is to imagine there being one or more models for each person and organization in the world (including posthumous entities).

We’ve been tracking this closely to see how close this prediction meets reality. Six months later, it is showing signs of being true. We’ve been counting the number of models that are derivatives of popular foundation models on Huggingface. As of today (June 19, 2024), here is a sample of what what we’ve seen:

• llama has spawned 37,447 derivative models

  • llama-3 has spawned 12,450 derivative models

  • llama-2 has spawned 20,779 derivative models

  • llama-1 has spawned 8,417 derivative models

• mistral has spawned 16,516 derivative models

• bert has spawned 27,969 derivative models

• phi has spawned 6,243 derivative models

• dbrx has spawned 53 derivative models

Total # of Models Spawned: 88,228

This number is not precise, it is just based on a simple search on Huggingface to see how many results come up when searching for one of a few base models.

The number of models counted here is surely an underestimate of the number of AI models being developed. First, not all models are posted to Huggingface, there are many models that are closed source and developed internally. Second, we’re only counting derivatives of popular large language foundation models, it does not count the many other model architectures that developers are using for a variety of use cases.

Let’s assume our count is off by 10x (perhaps even 100x).

AI and The Early WWW

The point is that directionally, our prediction is coming true. If we compare this trajectory to the early days of the web, we are at about the 1995-1998 era of the internet when there were 25k-2M websites.

This is when Amazon (only with books), Ebay (AuctionWeb) and Google launched.

The web 1995-1998

The web wasn’t particularly dynamic or interactive like we know it to be today. There were no mobile phones or apps, not very many laptops, and much of the news was distributed on TV and Radio.

Many AI developers (and their employers) are still tinkering with these tools to figure out how to build rich and dynamic experiences that we all love and use every day. The current interfaces are simple and straightforward, think ChatGPT and other “chat” interfaces described as “co-pilots” within specialized applications like code editors.

As application developers and designers get a better feel for this technology and as users get comfortable with probabilistic outputs (what you get won’t always be the same or correct based on what you ask), we’ll see more AI-powered applications. More AI-powered applications will spur demand for more AI models to do ever more specialized things.

The AI era & the PC era

The 1995-1998 era of the internet also relates to the growth in PCs / computers. The internet was a reason why people bought more computers. And more computers led to more websites, more investment in broadband, and generally much more software.

Marc Andreesen had a good breakdown for how the history with PCs and chips over latter half of the 20th century could relate to the AI era we are now in.

As the PC related hardware costs came down (thanks Moore’s law!) more applications were built, the more applications built, the more demand was created for hardware which further reduced costs. That cycle continues today whereby we see computers integrated with everything.

These same patterns are starting to emerge within AI development. We currently bottlenecked by similar “physical costs” which include:

1. GPUs for training and inference

2. Interconnect tools to wire up thousands of GPUs into clusters

3. Datacenters to house many clusters of GPUs

4. Energy sources to power those datacenters

Granted there are more physical costs with AI than with PCs - PC manufacturers didn’t need to worry about securing energy sources like AI model builders do.

However, these costs will come down:

We get better at training models with less data → Which leads to needing less GPUs → Which lead to smaller GPU clusters → Which reduces the load on data centers → Which reduces energy requirements.

And as models become more capable, it will lead to more interest and demand in AI-powered tools → which will lead to more AI models.

Is AI a Commodity?

This might lead us to think of AI will be a commodity, like sugar or oil. We would argue that this is an overly simplistic characterization of AI-powered systems. Indeed, AI will be as pervasive as a commodity but the landscape of AI will be highly variant. There will be many different forms of AI that work differently in different situations. Different form factors like mobile phones, sensors to giant clusters. Different interfaces like chatbots vs autonomous agents. Different access points like internal applications to consumer applications to kiosks at the local strip mall.

So sure AI will be like sugar. But go to your local grocery story and count all the different kinds of sugar you can buy: cane, brown, turbinado, powdered, stevia, truvia, etc. And then count all the different products that contain sugar that you eat everyday. That’s where we are headed.