Governing the Governors: Is the AI Governance Problem Really a Human Governance Problem?
The dominant way people talk about artificial intelligence, especially lately after the explosion of generative models and the uneasy scramble by governments in 2024 and 2025 to regulate them, goes something like this: AI is becoming immensely powerful. Maybe dangerously so. Maybe liberating, too, depending on who’s speaking and whether they own stock in a benefiting tech company. The challenge, supposedly, is figuring out whether human beings can govern these systems well enough to capture the upside without accidentally blowing up civilization (Albert Einstein: “It has become appallingly obvious that our technology has exceeded our humanity”).
At first glance, that framing feels sensible. Clean. Almost comforting, weirdly enough. But it leaps over a prior problem, and honestly, maybe the more disturbing one.
AI governance is never really about machines. It is basically humans governing other humans through institutions, incentives, laws, threats, norms, bureaucracies, and occasionally panic (which hopefully we can avoid in this case). To “govern AI” means coordinating researchers, executives, regulators, military actors, investors, and ordinary citizens who barely understand the systems but will still live under their consequences. If those mechanisms of coordination are healthy, AI governance is difficult but manageable. If they are fractured, corrupted, sluggish, or just worn out and tired, then the AI problem looks less like a technological dilemma and more like a mirror held up to human civilization itself. Which sounds dramatic, sure, though history rarely moves quietly.
This article explores both interpretations. The first sees AI as a genuinely novel challenge requiring unprecedented tools: alignment research, frontier model evaluations, compute regulation, international agreements, maybe even institutions we haven’t invented yet. The second interpretation is older and less glamorous. It argues that AI merely amplifies a preexisting failure, namely humanity’s long-standing inability to govern itself across borders, ideologies, and competitive pressures. Both views matter. Fixing AI governance without fixing human governance is a bit like installing an expensive smoke detector in a house where someone keeps lighting fires for fun.
I. The Case That AI Governance Is the Problem
There is a serious argument that artificial intelligence introduces governance problems unlike anything before it. Nick Bostrom, in Superintelligence, argued that sufficiently advanced AI systems could acquire decisive strategic advantages not because they are evil in some cartoonish way, but because optimization processes pursuing badly specified goals can outrun the humans who designed them in the first place. That idea still unsettles people a decade later. Honestly it worries me too, even after rereading Bostrom’s argument several times late at night with too much coffee and the glow of a laptop screen making everything feel more apocalyptic than it probably was.
Stuart Russell makes a related point in Human Compatible, a book focused on AI safety. The issue is not necessarily malicious actors deploying AI recklessly, though that certainly exists. The deeper issue is that specifying objectives for highly capable systems may itself be impossible to do perfectly. A machine that optimizes better than its creators can expose flaws hidden inside the goals it was given. Like a GPS that, told to find the shortest route, calmly steers you off a cliff.
Under this framing, AI governance failures are not reducible to ordinary politics or institutional incompetence. They stem from epistemic limitations. Humans may simply lack the cognitive capacity to fully predict emergent behavior in systems that are more sophisticated than themselves. The main issue with human oversight of AI is that we built AI to overcome human cognitive limitations, and we somehow expect humans to provide oversight. We cannot reliably inspect internal objectives. We struggle to anticipate misuse vectors before they materialize. And the pace mismatch is staggering. Frontier capabilities evolve in months while legislation drags through committees for years. Sometimes decades. Reviewing congressional hearings on AI recently felt less like an attempt at governance and more like relatives trying to configure a smart TV after Thanksgiving dinner.
Even a society with unusually competent institutions could fail here because the technical challenge itself may exceed human bandwidth. AI also differs from earlier technologies because it can participate in governance processes directly. It can draft legal language, shape public opinion, negotiate contracts, automate persuasion campaigns, and flood information ecosystems with synthetic content at industrial scale. The distinction between regulator and regulated begins to blur. That is not merely a policy inconvenience. It changes the geometry of governance itself, if that phrase means anything anymore.
If these arguments hold, then “fix human institutions first” may miss the point. The danger is downstream of capability growth that cannot easily be paused and upstream of institutions too slow to adapt. A category error, perhaps. Or maybe a civilization running old software on hardware that suddenly became alien.
II. The Case That Human Governance of Humans Is the Deeper Problem
The competing perspective predates computers by centuries. Without diving deep into centuries-old philosophical wisdom, Thomas Hobbes (circa 1651) argued in Leviathan that without enforceable agreements among rational actors, life collapses into conflict, “the war of all against all.” Governance, for Hobbes, was about restraining human impulses through artificial structures of authority (the social contract). Rousseau pushed back, arguing that institutions themselves often become instruments of inequality and domination rather than expressions of the general will. Hannah Arendt argued that the real danger isn’t technology, but the slow disappearance of places where citizens can question those in charge.
From all three perspectives, AI changes the scale of governance problems, but not their essence.
Think about what governing AI requires in practice. Nation states must cooperate despite strategic incentives to defect, the same problem faced in nuclear arms control and climate negotiations. Corporations must absorb costs the market does not naturally reward, which regulators have struggled with in finance, labor, pharmaceuticals, and pollution for generations. Citizens must reason collectively about technical risks they did not choose and often do not understand. We saw how difficult that became during the pandemic. Some societies coordinated impressively, others dissolved into conspiracy and exhaustion and endless online shouting.
None of these failures originated with AI.
If humanity already possessed reliable mechanisms for long term coordination, institutional trust, collective reasoning under uncertainty, and the containment of concentrated power, then AI governance would still be difficult but probably achievable. The uncomfortable truth is that we do not possess those mechanisms. Not consistently anyway. We never really have.
Fire transformed warfare and agriculture. Gunpowder destabilized empires. Fossil fuels reshaped economies while quietly cooking the atmosphere. Nuclear fission introduced extinction level risk into geopolitics. Social media altered cognition itself in ways people still barely understand. Each time, governance lagged capability. Sometimes disastrously. Humanity adapted through improvisation, luck, partial learning, and catastrophe avoidance that in hindsight feels almost accidental. AI may simply be the latest version of an old human habit: failing to learn much from the last crisis before rushing headlong into the next one.
And there’s another thing people rarely admit openly. Many institutions are already brittle before AI enters the picture. Trust in governments has eroded across democracies. Regulatory capture remains pervasive. Information systems reward outrage over coherence because outrage monetizes better. There are days when scrolling through online discourse feels like being trapped in a packed New York subway car at rush hour, where everyone is shouting contradictory advice, nobody makes eye contact, and something smells vaguely electrical. The constraint, ultimately, may not reside in technology at all. It may lie in the widening gap between humanity’s collective power and its collective wisdom.
III. Conclusion
Both framings capture something important, and pretending otherwise would flatten the problem into ideology. The first reminds us that AI introduces genuinely novel technical challenges: speed, opacity, recursive improvement, strategic unpredictability. These strain even competent institutions. The second reminds us that institutions themselves are products of human moral and political capacity. An AI governance regime cannot exceed the civilization operating it.
The two failures compound each other.
Nations incapable of coordinating climate policy and global trade are unlikely to coordinate effectively on AI governance. A regulator captured by incumbents will not suddenly become neutral because the subject is artificial intelligence instead of banking or energy. A public unable to reason coherently about probabilistic risk during a pandemic will struggle with model risk too, perhaps even more so because the systems are invisible and abstract.
So the question this article began with, whether the core issue is human governance of AI or human governance of humans, points to a directional answer. The deeper and more determinative problem is the latter. AI governance outcomes will probably track, and lag, humanity’s broader ability to govern itself.
That conclusion about humanity sounds pessimistic; but typically the path it usually takes.
Human beings have repeatedly built coordination mechanisms they previously lacked once pressure became unavoidable. International institutions emerged after catastrophic wars. Public health systems expanded after epidemics. Environmental regulation strengthened, imperfectly but still materially, after visible ecological collapse. Crisis has always been civilization’s harshest teacher. Exhausting, yes. Effective sometimes too.
But any strategy focused narrowly on alignment techniques, evaluations, licensing frameworks, or model restrictions while ignoring the institutional conditions underneath them risks mistaking symptoms for disease. As a friend in Systems Thinking once told me, “You can’t solve a problem without understanding the underlying system that created it.” The defining question of the AI century may not be whether our oversight mechanisms have become sophisticated enough. It may be whether human beings finally learn to govern themselves with enough maturity to deserve the tools they keep inventing.
The argument of this paper points to an uncomfortable conclusion: a globally coordinated regulatory response to AI is unlikely to arrive in time, if it arrives at all. If the cavalry isn’t coming, the rational move is to start reducing risks ourselves, one category at a time. Privacy is the lowest-hanging fruit. Every document uploaded to an AI system is a quiet handover of personal information to an infrastructure that no government fully governs, and no user fully sees. PII Anomalyzer, by Southwest Management Technology, strips the identifying details out of documents before they leave the user’s control, letting people use AI without unwittingly contributing the private data of their clients, patients, or employees. It doesn’t solve the governance problem. It just refuses to wait for it.
Want to use AI without handing over your clients’ data? Download PII Anomalyzer and run a 7-day free trial. It runs entirely on your desktop, detects a wide range of PII entity types across PDF, Word, Excel, and scanned images, and never sends your documents to the cloud.
References
- Bostrom, Nick. Superintelligence: Paths, Dangers, Strategies. Oxford: Oxford University Press, 2014.
- Russell, Stuart. Human Compatible: Artificial Intelligence and the Problem of Control. New York: Viking, 2019.
- Hobbes, Thomas. Leviathan. 1651. Edited by Edwin Curley. Indianapolis: Hackett Publishing, 1994.
- Rousseau, Jean Jacques. The Social Contract. 1762. Translated by G.D.H. Cole. New York: Dover Publications, 2003.
- Rousseau, Jean Jacques. Discourse on the Origin of Inequality. 1755. Translated by Donald A. Cress. Indianapolis: Hackett Publishing, 1992.
- Arendt, Hannah. The Origins of Totalitarianism. New York: Harcourt, 1973.
- Arendt, Hannah. The Human Condition. Chicago: University of Chicago Press, 1958.
Robert Bergman is CEO of Southwest Management Technology and Next Level Mediation.