The Black Box Problem: Who Decides When Frontier AI Is Safe Enough to Ship?

An Approval Process Without a Map

OpenAI has begun rolling out Sol, its newest large language model, to users worldwide. The model matches or exceeds the capabilities of Anthropic's Fable, a system powerful enough that the White House temporarily restricted its availability earlier this year. Yet the path from lab bench to public deployment remains frustratingly opaque to outside observers.

Mina Narayanan, a senior research analyst at Georgetown's Center for Security and Emerging Technology, told us she lacks visibility into the government's evaluation mechanisms. While Anthropic shared that it worked with officials and built classifiers to catch jailbreak attempts, the substance of those conversations and the criteria applied remain hidden from view.

Andy Konwinski, a computer scientist who co-founded Databricks and Perplexity, describes the situation more bluntly. He has yet to speak with anyone who truly grasps the process, including employees at the labs building these systems. The problem transcends technical safety, he argues. It centers on power and accountability: who holds the authority to greenlight deployment, and under what conditions?

Eighteen Months In, Still No Roadmap

The current administration recently published an executive order outlining a framework for frontier model evaluation, but implementation details remain sparse. Sriram Krishnan, who advised on AI policy at the White House until last month, stated plainly that there will be no FDA-style regulatory body for artificial intelligence.

The order tasks six cabinet-level agencies with finalizing a process by early August. For now, the Department of Commerce's Center for AI Standards and Innovation appears to be coordinating, but no consensus exists on which models require scrutiny or which agencies should conduct it.

What has emerged instead resembles a patchwork of informal consultations. OpenAI chief executive Sam Altman described conversations with Commerce Secretary Howard Lutnick, Treasury Secretary Scott Bessent, and national cyber director Sean Cairncross. The company declined to detail the government's methodology but pointed to external reviews by organizations including the U.K. AI Safety Institute, SecureBio, and Irregular, all documented in Sol's safety card.

Both OpenAI and Anthropic previewed their models to government officials and selected users ahead of broader launch. The identities of those early testers and the rationale for their selection have not been disclosed. In a late June statement, OpenAI acknowledged that this ad hoc access arrangement should not become permanent and committed to working with authorities on a more durable framework.

The Entanglement of Commerce and Policy

The conversations between OpenAI and the administration unfold against a backdrop that complicates public confidence. Reports indicate Altman proposed granting as much as 5% of OpenAI equity to accounts associated with the administration. OpenAI president Greg Brockman has emerged as a major donor to the president's political apparatus.

For observers attempting to assess the rigor of Sol's approval, these financial and political relationships are difficult to disentangle from the regulatory outcome. Dean W. Ball, a former policy advisor now employed by OpenAI, wrote in his newsletter last month that nobody understands the licensing requirements. That ambiguity may favor incumbents with access, but it creates instability for the broader ecosystem.

Anthropic's experience illustrates the capriciousness of the current system. Fable was briefly restricted from use by foreign nationals, driven partly by legitimate concerns about jailbreaking to enable hacking and partly by tensions between Anthropic leadership and administration officials. The threat of export controls may have incentivized OpenAI to adopt a more accommodating posture, though the specifics of any such calculus remain conjecture.

What Governance Could Look Like

Konwinski advocates for an open commons model inspired by institutions like the FDA, the National Institutes of Health, or the national laboratory network. These bodies convene researchers, government representatives, and private enterprises to forge consensus on safety standards. He worries that the current process marginalizes experts in safety, alignment, interpretability, and the broader technical stack.

Ball has proposed a system of third-party auditors, licensed by the government, to evaluate how frontier labs approach risk. Konwinski sees promise in focused research organizations that could channel academic and nonprofit expertise toward model evaluation without the commercial pressures that shape industry behavior.

Those pressures are real. AI companies invest billions in training runs and need to recoup costs quickly after launch, particularly when competitors are close behind. Even well-intentioned executives face fiduciary duties that can conflict with extended safety reviews. The incentive structure, embedded in corporate charters and term sheets, tilts toward speed.

Public Skepticism Meets Private Secrecy

The lack of transparency is beginning to generate political friction. University of Wisconsin-Madison computer scientist Remzi Arpaci-Dusseau noted at a recent conference that the public does not sense responsible stewardship of these technologies. David Siegel, founder of quantitative hedge fund Two Sigma, sketched a dystopian scenario in which a handful of firms control the technology, government labs evaluate it behind closed doors, and the scientific community is locked out entirely.

That scenario may already describe the present. Americans increasingly view the tech industry with suspicion, and the secrecy surrounding frontier model development feeds that distrust. When approval processes depend on personal access to cabinet officials rather than transparent criteria, the legitimacy of the entire regulatory project comes into question.

At DailyTechWire, we've tracked the evolution of AI policy across Seoul, Singapore, and Brussels, where frameworks tend to favor procedural clarity and multi-stakeholder input. The contrast with the current U.S. approach is stark. Regions that have invested in institutional capacity, public consultation, and expert panels have achieved greater predictability, even when their rules are more prescriptive.

The challenge for Washington is not simply to regulate or refrain from regulating. It is to build systems that the public, the research community, and the industry itself can trust. Sol's approval may mark a milestone in model capability, but it also highlights a governance vacuum that shows no sign of closing soon.