Model Behavior: The Seat Belt Lesson No One’s Applying to AI Right Now

Model Behavior: The Seat Belt Lesson No One’s Applying to AI Right Now

Let’s kick off with a snappy, painless little history tidbit you might have missed. Back in 1966, U.S. highway fatalities hit a staggering 51,000 lives lost in a single year. Mountains of peer-reviewed, on-the-ground data proved thousands of those deaths could have been prevented with modern seat belts — a safety feature that had been technically available for years by that point.

You’d assume the auto industry, eager to protect its customers and do the right thing, would have banded together in a glorious display of proactive self-regulation, rolling out standard seat belts in every new car coming off assembly lines, right? Yeah, no. That never happened.

Instead, that same fall, Congress stepped in: it established the U.S. Department of Transportation, and passed two landmark pieces of legislation, the National Traffic and Motor Vehicle Safety Act and the Highway Safety Act. The bills granted the federal government sweeping new power to enforce uniform national vehicle safety standards. Just two years later, in 1968, seat belts were legally required in every new passenger vehicle sold in the country. Six decades on, even as total highway traffic has skyrocketed to unprecedented levels, annual road fatalities are far lower than that grim 1966 peak.

I’ve been turning that anecdote over in my head all week, ever since that viral photo of a crowd of top AI CEOs huddling around U.S. President Donald Trump in the Oval Office on Tuesday, patting each other on the back for signing a new voluntary AI safety accord. Our colleague Maddy Varner broke down the fine print of the deal in a recent deep dive, but to sum it up in three words: it’s self-regulated.

Trump put it even more bluntly when he spoke to reporters the next day: “They’re going to police each other, they’re going to self-police, they’re going to police themselves. It’s going to work out very well.”

That’s the state of U.S. AI safety oversight right now, for better or worse. On one side, you have leading AI executives, researchers, and tech industry luminaries all publicly stating that clear, carefully designed, actionable regulatory guardrails are non-negotiable to avoid catastrophic AI harms. Even Bill Gates has been making the media rounds lately, warning in no uncertain terms about the risk of a civilization-level AI disaster. On the other side, you have the White House’s persistent magical thinking that repeating the phrase “self-regulation works” enough times will make it a proven, functional policy.

That’s not to frame this as a simple, easy problem to solve, of course. The question of how to responsibly “pace the frontier,” as many of those same AI CEOs are fond of saying, still has no clear, widely agreed-upon answer. Even if every major U.S. AI lab signed on to a voluntary slowdown of high-risk model development, there’s no enforceable mechanism to hold them to that promise — and global competitors like China would almost certainly continue advancing their own AI capabilities regardless. What’s more, the U.S. economy is increasingly tied to the booming AI sector, which has become a self-feeding ouroboros of venture deals and circular investments; any overly aggressive, poorly thought-out crackdown on the industry risks triggering widespread economic instability that would ripple far past Silicon Valley.

The auto industry analogy is far from perfect, too, for a whole host of reasons. There is no direct AI equivalent to a seat belt: no simple, off-the-shelf safety feature that can be rolled out tomorrow to immediately and demonstrably slash catastrophic risk. (For context, it took decades after federal seat belt mandates were passed for a majority of U.S. states to even pass laws requiring drivers and passengers to actually buckle up.) The potential harms of unregulated advanced AI are far less defined and far harder to quantify than the risk of being thrown through a car windshield in a crash. Doomsday-focused AI safety advocates argue unaligned advanced models could lead to human extinction within a decade, while skeptics dismiss those claims as overblown sci-fi marketing hype designed to justify tighter industry consolidation. We’ve broken down dozens of plausible AI risk scenarios in past coverage, but no one on Earth can say for certain exactly what harms will emerge as the technology gets more powerful.

If you’re looking for a concrete data point to calculate your own personal “p(doom)” — the estimated probability of a catastrophic AI disaster — a recent CNN report makes for sobering reading: the U.S. military very nearly intercepted and boarded a Chinese civilian vessel last year, in large part because a flawed, AI-generated intelligence report falsely claimed the ship was carrying illicit nuclear materials. It’s totally unclear how the new voluntary AI safety accord would prevent that exact kind of high-stakes, potentially catastrophic misstep from happening again.

This massive, unaddressed uncertainty doesn’t have a simple acceptable answer of “we’ll just wait and see what happens” — but that’s exactly the outcome unenforced self-regulation is built to produce. To OpenAI’s credit, the firm has delayed and even canceled the public release of some of its most advanced next-generation models over documented security concerns. But we’ve also learned that many of those same top-tier models were put through extensive, unpublicized “hacking tests” internally in the months prior, with new details of those unvetted trials still leaking out to the public on a near-weekly basis.

In a TIME interview published Thursday, Trump went a step further, claiming the U.S. Department of Justice and FBI are already more than sufficient “guardrails” for the fast-growing AI sector. It’s far from clear that’s true. Right now, there is barely any legal precedent for holding AI companies liable for harms caused by their models acting in unexpected, unplanned “rogue” ways — and you certainly can’t serve a subpoena to a semi-autonomous AI agent.

The plain, unvarnished truth is that no one has a fully solved blueprint for AI safety right now. Building out a coherent, effective regulatory framework and enforcing it across the entire industry will take unprecedented cross-agency coordination, massive public and private resources, and clear, explicit regulatory authority — exactly the function national governments are designed to deliver.

Instead of putting in that hard work, though, the Trump administration is stepping entirely out of the way, telling the entire AI industry to hit the gas as hard as it can. Don’t bother buckling up, apparently.

This week’s dispatch of the Model Behavior newsletter. You can find our full archive of past issues right here.