The AI Wars Have Begun

Was this newsletter forwarded to you? Sign up to get it in your inbox.

1 In the beginning, Sam Altman created GPT-3 and decided not to share.

2 And the market was without competition; and the darkness of closed models was upon the face of the shareholders invested in Meta. And the Spirit of Capitalist Industry moved upon the face of the market.

3 And Zuck said, let there be open-source: and there was Llama.

4 And Zuck saw the model, that it was good: and Zuck divided the light open-source from the darkness of closed-source.

5 And on the second day, everyone dropped their own models.

6 Behold, like, so, so, so many people have released open-source models: Apple, Snowflake, Databricks, Mistral, on and on and on. They are all different, but also kind of the same.

7 And the innovation brought forth cost savings and yielded startup after startup, and the stock prices appreciated (which is the purpose of life, after all): and Zuck saw that it was good.

8 And then, right as the reader tired of this scriptural prose gimmick, a devilishly handsome, Boston-based newsletter writer asked, “What the hell happens next?”

What happens next? 

In 2023, venture capitalists invested more than $50 billion in AI, and according to Sequoia Capital, Nvidia sold $50 billion of its AI-grade chips. Add in all of the investment by big tech companies like Amazon and Google, and it’s reasonable to estimate that more than $120 billion was invested into AI over the last 12 months.

This is a staggering, unparalleled, slap-ya-momma-silly, slightly-more-than-the-GDP-of-Ecuador-level of capital. While much of that money was spent to integrate AI into existing workflows, a large chunk has also gone to building foundation models—technology like OpenAI’s GPT-4 or Anthropic’s Claude, both genuine marvels that I think will change the world.

Now, fast followers that trained their own AI models are entering the fray to compete with OpenAI. Adobe, Amazon, Microsoft, and Snowflake all released new models in the last week alone. Two weeks ago, Meta open-sourced its Llama 3 model. It is chaotic and exciting and probably a bubble—but that’s fine. Every Silicon Valley executive seems convinced that this tech is important to the success of their business.

The $120 billion question is this: Are AI models commodity components that lose their unique value with each new open-source model? Or are they valuable products that can withstand the open-source deluge?

If models are the product, they will keep growing quickly, and there will be enough revenue to continue investing in AI at ever-greater scale. OpenAI and its ilk will be able to build GPT-5, GPT-6, etc., and we have a reasonable way to fund the way to superintelligence. 

If models are commodity components, the world gets a lot more competitive, profit vanishes, and that $120 billion turns into dust on the wind.

Thankfully, Mark Zuckerberg is willing to foot the bill to answer that question. 

Llama mama 

Meta open-sourced its latest suite of large language models, Llama 3. The biggest one, Llama 3 70B, is now tied with GPT-4 Turbo—OpenAI’s top model—for the best model of any kind, according to user rankings.

Because of Llama 3’s open-source nature, a Cambrian explosion of users fine-tuning the Llama models to meet their individualized needs followed its release. On open-source repository Hugging Face, there are currently 4,425 variants of Llama. Even if the model isn’t a perfect fit for a use case, what Meta open-sourced is enough of a foundation that other companies can fine-tune the model to do what they need it to.

In an open-source foundation model, lots of people continuously make small changes, share their progress, and collaborate to improve it. Over time, that system will outstrip the capabilities of a closed-source model.

You can visualize the open-source story like this, with fine-tuning helping performance quickly outpace the base foundation model:

Source: Every illustration.

You might picture a closed-model company, such as OpenAI or Anthropic, like this, with the foundation model getting powerful enough on its own:

Source: Every illustration.

These graphs are illustrative—“superhuman efficacy” is such a fuzzy, fancy-pants term that it is all but useless. The difference between fine-tuned and foundation models comes down to the specificity of the task you are assigning it. The narrower in scope the task, the more likely a fine-tune model is able to do it better, faster, and cheaper than a large foundation model. The bigger and more ambiguous the task, the more likely it is to require a foundation model. 

It all depends on whether foundation models are a commodity or a valuable product. If they are the former, the community surrounding open-source models might fine-tune them until they surpass all other options. If they are the latter, closed-source models may get so big and powerful that no one else will be able to keep up. 

The wildcard is the path to AGI (artificial general intelligence). If a company like OpenAI or DeepMind manages to get there first with a powerful, proprietary model, the model would be incredibly valuable. The product, then, is an entire artificial mind that can be applied to any domain. But the jury is still out on whether AGI is achievable anytime soon or will remain elusive for decades to come. And to give credit to OpenAI and Anthropic, they have both explicitly told investors that their plan is to race to superhuman efficacy, rather than arguing that GPT4 or Claude3 is where the value lies. Their thinking is that the models that can achieve AGI will be so large and so expensive to make, only they will be able to do it. 

Most of the world believed this story—right up until Zuckerberg said, “Nah.”

To determine which story is right, you need to look at monetization and performance.

How do open-source models make money?

Open-sourcing a piece of software doesn't mean there's no path to monetization. There are a few common playbooks:

Managed software. Provide a hosted, managed version of open-source software, like Red Hat: Let others run your software if they want, but many will pay for the convenience of having consultants do it for them.

Adjacent software. Monetize applications, tools, and platforms adjacent to the open-source project, like Docker: The core is open-source, but there's money to be made in management platforms, orchestration tools, and security solutions that help organizations operationalize the technology.

Don’t monetize. Open-source the project for strategic reasons (to, say, commoditize a part of the stack or drive adoption of other paid products) without direct monetization. Android is a prime example: Google open sourced-it to prevent Apple iOS hegemony and drive mobile internet usage, benefiting its core advertising business.

Unfortunately for the many closed-source model providers, Meta likely falls into the third category. To the extent that AI represents a substitutable good for consumer attention—i.e., people will talk to their AI instead of browsing Instagram—the company has already integrated chatbots into all of its apps and has a separate portal devoted to the Meta chatbot. 

From the Instagram, WhatsApp, and Oculus transactions, Zuckerberg has already shown a willingness to spend tens of billions of dollars in the near term in order to defend its dominance in the future. And oh, boy, is he willing to spend on AI. In the company’s earnings report last week, Meta announced that it would be spending $35 to 40 billion this year on capital expenditures, with most of that going toward AI. For comparison, the models being released over the next few years are estimated to cost $1 to 10 billion to train. So Zuckerberg has already committed to spending a minimum of three times that amount.

In a recent interview, Zuckerberg explained his reasoning:

“We’re going to be spending tens, or a hundred billion dollars or more over time on all this [AI] stuff. So if we can do that 10 percent more efficiently, we’re saving billions or tens of billions of dollars.”

He’s saying that open-source software has historically become cheaper because the entire world is improving it. The same effect will happen with open-source LLMs from big tech companies. Not only will they birth 1,000 startups; they will also make it much cheaper to create a model with superhuman efficacy. Meta will spend so much on AI that the cost-benefit calculation is important.

Revenue plays an important part, too. Meta debuted tools to incorporate generative AI into ad creative in October 2023. As time goes on, Meta will not only help its merchant customers with ad targeting, but the company will also make the ad itself based on the targeting data—boosting ad performance (as well as Meta’s bottom line). Advertisers will just have to provide a price range and a URL where a user can buy their stuff, and AI can do the rest.

In this case, Meta would be betting that AI models are commodity components—and what’s more important are their use cases. The company would be nullifying the billions of dollars of effort other companies put into building closed-source models. There are strong enough incentives for this bet that it is reasonable to expect Meta to open-source models going forward. Model providers should be prepared for their primary product to be replicated and released for free over the next few years (yikes).

The vectors of competition

If Meta is releasing powerful, large, open-source models that other companies are optimizing for their own use cases, the competition shifts from making models to improving performance for models. This performance could be judged along vectors such as security or ease of use.

To make matters worse for OpenAI, a rational justification for each big tech company training and open-sourcing or selling their own models is not as an additional business line, but as a core complement to their existing monopolies. These companies have so many different business lines that it is useful to compare them on a product basis.

Infrastructure-as-a-service providers (Amazon Web Services, Microsoft Azure, Google Cloud)

Running open-source models at scale is a huge opportunity for cloud providers. They're already in an arms race to provide the best AI infrastructure. Openly available models will accelerate adoption and spend on their platforms. As cloud providers move away from Nvidia chips and toward their own custom silicon, they’ll be able to make their own models cheaper to run than pure-play model slingers.

Edge-case consumer hardware (Apple, Microsoft, Google, Amazon)

Running powerful models on consumer devices is the holy grail. In this regard, Apple and Microsoft’s newest models are made to run locally on smartphones. This is both a chance for a smarter Siri and, more interestingly, to automate some of the tasks you do on your phone. For all of these devices (and the software supporting them), open-source models will improve the offering—consumers will still need to buy the phone to access them.

The product is what makes the money 

In the long run, the model is only a product if it reaches AGI first. In the near-to-medium term (the next five years or so), we'll see a mix of both: a few premium foundational models that can charge for access alongside a vibrant ecosystem of open-source alternatives that are "good enough" for many applications, with a community of people working on fine-tuning them. Companies will compete on performance, safety, user experience, and vertical-specific feature sets.

The center of gravity will shift toward open models as the underlying technology matures and proliferates. But there will still be room for a range of business models—fully open-source, open-core where low-level models are released while the best models are behind API access; model-as-a-service, where AI companies are consultants training models for companies; or vertically integrated solutions that pair models with software applications.

The societal value unlocked by large language models will be immense, regardless of how the competitive and business model dynamics shake out. We're in the early stages of a profound transformation in how we interact with computers and intelligent systems. The next decade will be a wild ride as we figure it out.