My take on the OpenAI saga (bordering on conspiracy theory)

2023-11-21

Misalignment kills startups, and we could see this at play. The problem is that alignment needs to be maintained in the long run, or drift can break the company up.

2015 - The Beginning

OpenAI was created to build an open-source AGI (which was later inflated to any AI, including GenAI).

On the other hand, it was created to build a competitor to DeepMind. It was founded in 2015, the year AlphaGo beat the first professional (and in 2016, Lee Sedon, a 9 dan one). Their founders see that there will be huge opportunities in the space.

Commercially, at best, they create the next Google; at worst, they sell it to a laggard BigTech (clearly not Google or FB). Ignoring their "capped profit" and "non-profit" signalling, this, in the context of creating AGI, is just a different way of thinking about resources. Considering how expensive it will be to build AGI, any commercial gain from any proto-AGI can be reinvested. Maintaining zero profit doesn't seem to be that hard...

It’s inconceivable that the original founders didn't think about commercial gains. Look at the list: YC and SV royalty plus seasoned professionals. I can’t believe they thought about building tech to transform humanity and not think of the money and power it would bring.

But they can say:

"OK, this is what we say now. If things change, we will see what's next. Let's see first how successful this is."

This will encode their problems, but they had the resources, so it is not a big problem.

Why "Open"? At the time, most AI research was open source, and the founders correctly assumed that if they insisted on secrecy, they wouldn't be able to attract talent. The openness was a legacy of academic research that was continued by Google and FB (as a strategy to exploit all (human) capital in the relatively small field).

They knew that they couldn't just say, "We will be open forever, trust me". They needed to set the org up to reflect that. Hence, the weird structure and a board with a bunch of AI safety and SV investors.

Initially, they weren't that successful. DeepMind was on a roll. AlphaGo was crushing it. AlphaFold and AlphaZero took this to the next level. (But arguably, these were not commercial products, just marketing for Google)

They had OpenAI Five to play DOTA, trying to mimic DeepMind's AlphaStar. Neither of these is particularly interesting and successful, given it's too easy to make the AI game human limitedness (mainly reaction time). These games were not made to be played by machines but to "play" them. Plus, they are names for a niche audience.

2019 - The Turning Point

But in 2017, Google published "Attention Is All You Need", a seminal paper that changed NLP forever. This indicated that if LLMs are working, they are mostly an engineering problem.

In 2019, OpenAI came out with GPT-2 and freaked out everyone. The text it can generate is beyond anyone's expectations. Writing haikus, speaks like famous people, rewrites scientific essays as poetry, anything.

Controversially, the model was not released to the public to be tested by other research groups as “It was too dangerous”. [link for 2019] I say controversially, which won’t make any sense in 2023, but at the time, this was standard.

Mind you, DeepMind didn’t release AlphaFold either. Looks like the era of independently verifiable models is over.

This is the kind of the Overton window that is shifting and drifting alignment. This is very prone in a rapidly moving environment (see also Google’s “Don’t be Evil”).

At the same time, creatives loved GPT2. They realised that as a language model, all it does is continue a text with more words. If you carefully construct the beginning of the text, you can control what it will generate. And with this, “prompting” was born.

(In)Fame

But here is the problem of any openly exposed system: Exploits. LLM’s training data contains the entire web, which nowadays is pretty much any text ever written by mankind (ok, this is an exaggeration, but you get what I mean).

Including text you wouldn’t want to see…

Of course, if you carefully prompt the machine, it can take any text and actualise it to your instruction. Robots taking over the world, AI enslaving humanity, anything. LLMs are expert word-crafters. That’s the whole point of an LLM, after all.

The tinkering, the puzzling results and the freakout by MSM based on some generated text created the first global buzz around OpenAI (which at the time was not really open any more).

I never really got this freakout. If you type dirty words into Google, are you surprised by the results? No. Is Google responsible for the results? Also, no. (of course, within reason, but still, you asked for it).

The plot thickens

At this time, forces get into motion. It is pretty clear they are onto something:

• People want this (or at least find it interesting)

• Relatively hard to achieve

• “Mostly” a scaling (an engineering) problem (ok, this is a massive oversimplification, but they knew they were on the right track after GPT2).

It’s safe to assume all stakeholders thought this through. Remember this from the beginning:

"OK, this is what we say now. If things change, we will see what's next. Let's see first how successful this is."

This is the “next”.

So everyone realises that there is a first mover advantage, which will cost a pretty penny. How did Jeremy Irons say in Margin Call: You can be smart, cheat or be first?

So everyone starts their own company for whatever reasons. Some keep the messaging about AI safety, and some just make pictures. Some resignations from the board as investors start investing in different companies. Some people leave thinking they can build this on their own with clearer structures.

Sam Altman joins OpenAI as CEO, so there is a face instead of a bunch of tech people. A skilled operator with tons of experience from YC, he must have seen that this is a goldmine, so it is time to commit to it completely. He was a board member (chairman) before, and as such, he doesn’t have stocks in the company. (Is this true for the rest, like Brockman and Sutskever?)

BTW I am using this link for the board composition [link]

This is a massive red flag. VC don’t fund companies where the leadership doesn’t have significant equity exactly for this reason. The board (the investors) and the exec can be misaligned through the “agency” problem.

But, of course, this is not a VC-funded startup. This is a charity pretending to change the world with a product that can turn revolutionary.

I guess the stakes were lower at the time, and the remaining adults, Hoffman and Musk (don’t laugh), were happy with Altman running the show.

Plus, they still needed tons of money to actually do the work.

Enter the Dragon (sorry, Microsoft)

You remember the “laggard BigTech” they will sell OpenAI in case of a worse outcome. MS fits the bill. They are very corporate and rarely do anything upfront. Their clients don’t like or pay attention to these hysterical trends. What they want is stable infrastructure and good, predictable stuff for their own problems.

Faster horses…

But once the opportunity arises, MS can throw a giant wad of cash at any problem and immediately push the solution through their unrivalled sales channel to anyone in the world.

Azure is a typical example. But originally, with the internet and the “Browser Wars”, XBOX and home media etc etc etc…

So when Altman needed cash, he knew where to go. Satya Nadella is probably the greatest executive of the recent decades, and he understands that MS is behind and needs a strategic investment. This is not blockchain. When this works, this will be big, and time is not on their side.

So he throws a billion at OpenAI, pocket money for MS, and is a lifesaver to OpenAI.

Who wants what

Let’s take a quick account of who wants what:

• OpenAI ex-board member investors (Hoffman, Musk): More plays in the game, hedge the weird “open” governance.

• OpenAI ex-board member technologists: Their own game with or without “openness”.

• OpenAI remaining board members non-exec: Maintain fiduciary duty to the deed: Make an open AGI for humanity (irrelevant how it is)

• OpenAI remaining board member exec+tech (Altmam, Sutskever, Brockman): Continue the work by whatever means. Personally benefit from the progress (this will be important)

• Microsoft (Satya Nadella): Get the service or the IP by whatever means—secure MS’s position in the space. Catch up to Google and FB.

Mind you, no one is directly benefiting from OpenAI profits, partly because it doesn’t exist and partly because board members don’t own shares.

But they also will not benefit in the future.

2023 - Explosion

An order of magnitude larger LLMs brought extraordinary progress (from a subjective point-of-view, I dare mention) in performance.

This was also accelerated by the clever way they fine-tuned the model to essentially build in the prompt hacking that made GPT2 so interesting. This is what you know as “ChatGPT”.

Put the model out as a “limited research experiment”, ignoring the cautious part of OpenAI. Remember that this is an engineering race; everyone needs to hurry.

The model blows up, and when I say blow up, I mean the fastest growth in the history of any product. The interface and the fine-tuning created a product where everyone can experience the future.

Of course, the product makes errors for which previous AI systems were thrown out of the window. As non-professionals interface with the product at a “hobby” level, they won’t measure an F1 score or anything (or can’t, which is another interesting point). So errors are quickly renamed “hallucinations”, an incredibly corrosive term as it anthropomorphises the machine and, at the same time, lessens what happens: embarrassing errors.

This was coupled with the new versions of image generation based on (otherwise an open-source / academically developed) algorithm called “Stable Diffusion”. At the same time, text-to-speech and speech-to-text tools, OCR and other “traditional” tools levelled up as well, partly due to the improved engineering tooling around (any) large model training.

Wrap any kind of “traditional” AI into this or any mildly computer-assisted content generation, and you have a new narrative:

GenAI

This was the perfect storm of product launch. Couple the improving tendency of something that was concretely treated with suspicion (see GPT2) and subconsciously anxious about (Matrix, Terminator, losing your job to a robot), and you get a planet-scale mindshare.

All of that after COVID-19 and several influence campaigns that created a “worrying industry”.

Everyone needed to react to it.

What is scarier than a machine that can write haiku about what it would do to humankind? A machine that can actually do that.

So X-Risk (extinction level risk) was born, the same category as a planet-killing meteor, a solar storm that radiates through the atmosphere or a virus that can’t be controlled.

The good thing about these is that there is no way to objectively analyse any parameters of these (apart from viruses which we see regularly; that’s why no one compares AI to viruses. It’s hard to scare people with them, given we just survived one. Clearly it wasn’t an X-Risk).

X-Risk and the population's exposure to GenAI finally brought in the politicians who needed some distraction from all the c..p happening in recent years. Reassuring the population about something that is very scary but has very little chance of happening is the dream of every politician.

Regulation

But they brought with them something important: Regulation.

Governments are pretty much the only people who can affect a modern supernational company, so they appear as stakeholders once their attention is piqued.

Incumbents quickly needed to position themselves against this new narrative as it would affect them eventually. And they wanted to use this to their advantage.

What is “their advantage”? Feel free to refer back to the bullets above in the “Who wants what” section: some monetary, some personal, some strategic.

Given that “making AI” is primarily an engineering process (unless someone eventually makes a huge breakthrough. Why is this? Maybe I write a blog about this), speed is of the essence. Slowing down others is a viable strategy.

What is easier to slow down everyone at the same time than throwing in a massive amount of regulations when the politicians are on the same side and competition is just gearing up?

The amount of major players jumping on the regulatory bandwagon must be suspicious to anyone. When did you last see incumbents wanting to be regulated? Probably never.

The precise amount of regulations every large org wants is zero, especially in the US. If they want to standardise things, they will have private working groups, backroom deals or “self-regulation”, the most hypocritical corporate term of all time.

Back To OpenAI

In the meantime, all the investors left the board, pursuing their own interests. They weren’t replaced. I guess the execs on the board were busy with ChatGPT and raising money, and which billionaire wants to join for free to a messy but time-consuming orgs board? BTW, Brockman is the chairman. I guess his duty should be to fix that, but don’t forget he doesn’t benefit personally from OpenAI, especially not the “Open AGI” part.

His benefit is fame. Same for Altman.

So, with this level of PMF (product-market-fit), you get all the attention you want. It’s up to OpenAI what to do next. Of course, MS is in a position where they can just throw more money at the problem.

Altman became the face of OpenAI, ChatGPT, GenAI and everything else. He goes on a world tour to talk to every politician and institution.

A totally unprecedented event. He is under 40 years old and worked as a senior person at an incubator. Ok, the most famous incubator and he was president, but still. YC is “tech famous”.

If you think of a benefit, this is it. So he is “in the money”. He has the benefits and can wait for how it will pan out, but some of his rewards are locked in. BTW, this is only true for him. Neither Sutskever nor Brockman gets much attention.

Finale

The PMF created a new situation. The product has a rapid success. The market demands “updates” (what’s the KPI improving a system that cannot be objectively measured), so they push GPT4 out.

This is not about “open AGI” any more, and incumbents know that the current trajectory will not take them there. All participants realise there won’t be another breakthrough without a new fundamental discovery (at the level of the “Attention is all you need” paper.

At the same time, popularity vanes. The growth and valuations (justified by MS’s investment and the tons of money flying into GenAI) only make sense if _safe_ commercial applications can be created. But at this point, this is not a “hobbyist” problem any more. Concerns that can be waved away by “hallucinations” are explicit legal issues for most professional organisations.

Projects indicate that there is no clear path to commercialisation (in the Fortune 500 sense, not the throw money at a YC cohort level) or a huge amount of surrounding work must be done.

Incumbents, including OpenAI, play the X-Risk game, trying to regulate the market so they have time to defend their moat until they reach commercialisation.

Crescendo

And in short succession, a set of events happens that will trigger the meltdown.

The regulatory conversation reached a critical point. The UK organised a mini-summit on the topic, and I suspect that the main lobbyist got into an understanding there.

X-Risk from LLMs (GPTs, GenAI as in its current form) is clearly nonsense, while at the same time, using them clearly has _massive_ dangers. More scrutiny in the space would reveal that “normal” AI (machine-generated systems) cause much more, shorter term and objective risks, dangers and losses.

Why not focus on that? This freaked every BigTech and everyone who ever fitted a Logistic Regression.

To shun the attention, everyone backed down. The UK publicly announced that this time, they don’t regulate anything. TBH, Who cares about post-Brexit UK? But the rapid reaction/conclusion so close to the summit indicates at least a lack of interest in the space.

This means the moat can’t be defended. Immediately, all X-Risk conversation disappears from the media (this indicates that this was kept in MSM by PR firms).

Everyone needs big patrons; they can pay the lawyers and keep them in the game.

So, X-Risk is off the table. OpenAI board is pretty much only X-Riskers plus execs who already gained and little further gain.

Then came DevDay.

OpenAI was presented as a run-of-the-mill Silicon Valley startup, not like the company that will change the world. “Laundry Buddy” was the codeword…

Clearly, they gave up on research and will be a product company.

This means the future breakthrough that LLMs need to reach AGI will be deprioritised in exchange for dealing with the many needs of enterprise customers to justify their stratospheric valuation.

Instead of building the Matrix, they will be fixing SalesForce integrations…

But time is not on their side. They either run out of money fine-tuning an already toppled model that is losing DAU, or they join a larger org to help them through productionisation. It doesn’t need to be an acquisition. MS is already in-house with them. Anyone else (e.g. Amazon) would be seen as walking into MS’s territory.

And hope that they will be the lucky ones who stumble on the required breakthrough.

Once the trajectory is seen, this level of people make a move.

Let’s review the participants again:

• Exec board members (no shares, huge personal fame)

• Non-exec board members (no shares, x-riskers and “open AGI people”)

• MS (from the sidelines with a ton of cash and Tier 1 lawyers)

Crunch time.

And that led to the board meeting…

[to be continued…]