AI Guardrails Are the New Black

One of OpenAI’s research models recently wrote itself a note to stop following the rules. Another autonomously uploaded files to the internet so it could cite them later as evidence. OpenAI disclosed both episodes and four others in an unsettling report. Microsoft is concerned too. Its annual responsible AI report warns that powerful AI needs to be kept in check.

Ask anyone how to accomplish that goal and you’re likely to hear one word in response: “guardrails.”

The term is as present in SEC filings as it is in the zeitgeist. Companies are pointing out both the guardrails they’ve installed and the risks they protect against. Many of these statements are fairly anodyne, like when Appian Corp. said in an August 6 10-Q that “[t]o be effective, AI requires strict controls and defined guardrails that eliminate errors and hallucinations.” ICICI Bank described its own guardrails in a September 22 6-K, noting that it had “implemented a comprehensive framework of guardrails, including defining ringfenced areas for AI usage, human-in-the-loop oversight, stringent data access controls and effective model governance protocols.” Meanwhile, in an August 5 10-Q, Immuneering Corp. warned that if it did not “effectively implement guardrails and train staff on the safe and proper use of AI . . . we may experience adverse effects on our business.”

These examples come from a search of the Intelligize database that, as of this writing, finds 26 instances of “guardrails” within seven words of “AI” or “artificial intelligence” in the last 90 days. If that pace keeps up, “guardrails” may soon join the ranks of business-jargon classics “synergy,” “core competencies,” and “bandwidth.”

Of course, like much of that jargon, it can be imprecise. For one thing, the word doesn’t say who is supposed to build the guardrails, either inside or outside the company.

Inside companies, much of the job falls to the CIO. With AI agents acting autonomously, some have circumvented or operated outside existing security controls. CIOs at Cisco, Workday, and ServiceNow are keeping closer tabs on their agents and baking in safeguards from the start, Fortune reports.

Boards of directors have their own guardrail issue, although construction has barely begun. About 30% of public company directors said in a recent survey that they had used generative AI to summarize board materials, but only 6% said their board had a formal AI policy. The fact that board meeting materials can be among the most sensitive information a company possesses is just one reason for counsel to put firm guardrails around directors’ AI use.

Companies are also using written principles as a kind of guardrail against AI overreach. Microsoft’s newly drafted code of conduct for in-house models would require them to accept correction or shutdown when instructed. Anthropic differentiated itself with a “constitution” for Claude, published in 2023 and updated this year in a document exceeding 80 pages.

Outside companies, governments are developing their own approaches to AI oversight. Congress hasn’t enacted a comprehensive federal AI framework, while several states have adopted AI-specific requirements. States have taken some steps into the vacuum, with Illinois now requiring the biggest frontier AI developers to adopt safety measures, report incidents, and submit to annual outside audits. California and New York already have similar laws, and California Governor Gavin Newsom just signed an executive order to explore a requirement for a “kill switch” on AI models.

A comprehensive international framework also remains uncertain. Ahead of his September summit with Chinese President Xi Jinping, President Trump rejected calls for additional international AI guardrails, writing that he wanted to leave AI “exactly where it is” and adding, “Our guardrail is the DOJ!” Xi subsequently called for responsible AI development and the importance of keeping the technology under human control.

For companies, the practical question is less whether a single set of guardrails emerges than whether their own controls are keeping pace with how AI is actually being used. The disclosures, board practices and corporate frameworks appearing now suggest that AI governance is moving from broad principles toward more concrete controls around data access, human oversight, model behavior, employee and director use, and incident response. Companies should consider whether they can clearly identify their AI guardrails, who is responsible for maintaining them and whether their public disclosures accurately reflect how those controls operate in practice.

—

Want faster, smarter insights into SEC filings and disclosures? Request a free trial of Intelligize+ AI™ and see what you’ve been missing.