Sixteen
years on, the dilemma is familiar. The stakes are not.
In April 2010 I opened a post on my blog with a story about horse
manure. In 1898, delegates to the world’s first international urban planning
conference in New York were consumed not by housing or land use but by the dung
produced by the horses that moved the great cities of the world. The story, as
it is usually retold, comes with a forecast from London that by 1950 every
street would be buried nine feet deep. The forecast never came true. The
internal combustion engine arrived, solved the problem, and a century later
became a problem of its own.
My
argument then was simple. Every great innovation travels the same road, from
saviour to Frankenstein, and in financial services that journey had become
alarmingly short. Derivatives built to hedge risk became tools to push risk
onto unsuspecting investors. Securitisation deepened debt markets and then
ended in a situation where nobody really owned the risk of the underlying
asset. I concluded that the two principal challenges of the day were the regulation
of innovation and innovative regulation.
Sixteen
years later I find myself returning to that post, because artificial
intelligence is forcing the same question on us. The issues are different, and
I will come to how different they are. But the shape of the dilemma is
uncannily familiar.
What has not changed
The first thing that has
not changed is human nature. In May 2010, writing about the two bullies in my
high school who got away with their transgressions because they won trophies
for the school, I quoted John Mack of Morgan Stanley on the cause of the 2008
meltdown: greed, leverage and lax standards. Replace leverage with scale and
you have a fair description of the AI race. Firms compete to ship new
capabilities every few months. Rob Reich, Nate Persily and Anka Reuel have
described the frontier laboratories as “tragic accelerators”: they acknowledge
the risks, call for safeguards, and keep pushing capabilities forward anyway,
under pressure from rivals at home and from China. When Europe debated delaying
its AI rules this year, critics warned that the delay could set off a race to
market before the rules bite. As I said at the AI-DPI conference in June,
market incentives do not naturally reward careful and transparent development.
The second is opacity. Hank
Paulson observed that markets became opaque as structured products grew too
complex even for sophisticated investors to understand. The difference, and it
is an important one, is that a CDO was complex by design; someone could, in
principle, take it apart. A neural network with hundreds of billions of
parameters is opaque even to the people who built it.
The third is the argument
against innovation itself. In 2010 many voices said the answer to the financial
crisis was to curtail innovation. I disagreed then, and I disagree now with
those who would freeze AI until it is perfectly understood. Claiming we were
insulated from the turmoil because our markets were unsophisticated was like
saying I never fell because I never rode. The same logic applies to a country
of 1.4 billion people that needs AI to deliver health, credit, education and
justice at a scale its institutions cannot otherwise reach. Hibernation is not
a strategy. But I must also qualify what I said in 2010. My argument that
mistakes trigger better controls assumed the mistakes were survivable. A
mis-sold financial product can be compensated. Some AI failures cannot be
undone, and for those, learning by falling is not an option.
The fourth is that
self-regulation is still asking too much. Nothing in the last three years of AI
suggests the technology industry has discovered a virtue that finance lacked.
Voluntary commitments are useful. They are not a governance system.
What is genuinely different
Now the differences,
because they are real, and a regulator who treats AI as just another derivative
will fail.
Speed. The printing press,
electricity and the internet each took decades to reshape society. AI
capabilities shift in months. A rule written around today’s architecture risks
being obsolete by the time it is notified.
The object of regulation is
unclear. In
finance we knew what we were regulating: a product, an institution, a
transaction. With AI it is not even settled what the regulated object is. As I
argued in my April essay on the DPDP Act, we cannot yet say whether the law
should attach to the training dataset, the trained model weights or the
outputs. A right to erasure that makes perfect sense for a customer database
becomes close to meaningless once the data has been dissolved into billions of
parameters.
General purpose. A credit default swap does
one thing. A foundation model can draft a contract, read a scan, write
malicious code and tutor a child in the same afternoon. Paulson’s complaint in
2010 was that a regulatory structure organised around traditional business lines
had not kept up with the markets, leaving duplication in some areas and gaping
holes in others. AI makes that problem far worse, because the same model sits
underneath every sector at once. And the risks run through the whole stack,
from chips, energy and data to the models and the applications built on them.
The knowledge gap has
flipped.
In 2010 I argued that regulators must attract people with the experience,
domain knowledge and attitude to match or better industry strengths. In
frontier AI, the expertise, the compute and the data sit overwhelmingly inside
a handful of private laboratories. The regulator does not merely lag the
industry. In many cases it cannot see what the industry is doing.
Jurisdiction. Technology moves globally;
regulation remains stubbornly national. I made this point recently about
digital assets and BRICS. It applies with greater force to AI, where a model
trained in one country, fine-tuned in a second and distributed through an app
store in a third affects citizens in a fourth.
Some of the harm cannot be
undone. This
is the difference that worries me most. Reich, Persily and Reuel sort AI harms
into four families: malicious use and weaponisation; malfunction and loss of
control; societal, economic and environmental damage; and harm to individuals
and their rights. Finance gave us plenty of the last two. It never gave us the
first two. A model that gives someone real help towards a chemical or
biological weapon, or an autonomous agent that takes actions nobody intended
across thousands of systems at machine speed, is not a mis-sold product that a
fine and a refund can fix. The harm may be irreversible, and it may come from a
system that was never deployed for a dangerous purpose at all.
Three answers, and the pendulum
Look at how the major
jurisdictions have responded and you see the pendulum I wrote about in July
2010, when I described how auditing swung from judgement to rigid rules and was
beginning to swing back towards principles.
Europe went first with a
comprehensive, rules-heavy law, and is now retreating. Under the Digital
Omnibus agreed this year, obligations for stand-alone high-risk systems have
moved to 2 December 2027, and those for AI embedded in regulated products to 2 August
2028. The stated reason is that the standards and guidance needed to comply
were not ready. The rulebook arrived before the toolkit.
The United States has gone
the other way. A December 2025 executive order set up a Justice Department task
force to challenge state AI laws, in pursuit of what it calls a minimally
burdensome national framework. An earlier attempt to impose a ten-year moratorium
on state AI laws was stripped out by the Senate by 99 votes to 1. This is
regulation by litigation.
India has chosen a third
path. The AI Governance Guidelines released by MeitY in November 2025 are
non-binding, rest on seven principles including “innovation over restraint”,
and leave binding obligations to existing laws and sector regulators. The RBI’s
FREE-AI framework for the financial sector goes a step further and calls for a
tolerant supervisory stance towards first-time AI errors by institutions that
have put proper safeguards in place.
I have a good deal of
sympathy for India’s approach. It also fits a view gaining ground among
scholars, that in many respects AI is a normal technology whose impact depends
more on how fast it spreads through institutions than on what the frontier
model can do. And tolerance for honest first mistakes is what I argued for in
2010. But that tolerance needs a boundary. It is right for a credit model that
misprices a loan. It is wrong for a system that could help someone towards a
weapon, or that can act on its own across critical infrastructure. On the first
kind of risk India’s framework is thoughtful. On the second it says much less,
and leans largely on voluntary commitments such as those announced by frontier
AI companies at the New Delhi summit in February.
But here is the
uncomfortable part. Light touch works only when the hand is strong. A
light-touch regime run by a regulator without capacity, data or technical depth
is not light touch. It is no touch. The guidelines themselves lean on
institutions such as an AI Governance Group and the AI Safety Institute.
Whether those bodies get the talent, the budget, the compute and, above all,
the courage to act will decide whether the Indian model proves wise or merely
permissive. In 2010 I praised SEBI for calling out ULIPs as mutual funds
masquerading as insurance. The AI equivalent will be the first time an Indian
regulator tells a large and popular AI deployment that it must stop. We have
not yet seen that moment.
What innovative regulation looks like
for AI
If the challenge in 2010
was innovative regulation, what does that mean for AI? Let me offer seven
propositions, drawn from what I saw work while building NSDL and ONDC, and from
what I see on the boards of financial institutions grappling with AI today.
First, regulate the
function, not the technology. Ask the old questions. Who bears the loss if it fails?
Can the decision be challenged? Who is accountable? A loan decision is a loan
decision whether a credit officer or a model makes it, and the borrower’s
rights should not depend on which.
Second, tier by risk, but
admit how hard that is. The obvious approach is to tier by use, with testing,
incident reporting and audit for systems that decide on credit, employment,
medical treatment, policing or critical infrastructure, and to add a second
tier for dangerous capabilities such as weapons uplift or autonomous action at
scale. The trouble is that these are not two kinds of system. A bank’s
customer-service assistant built on a frontier model carries that model’s full
capabilities, whatever the bank intends. The safeguards meant to keep them apart
are brittle. Researchers have stripped the safety training from open-weight
models in minutes, for less than a dollar of computing time, and a study by CDT
and MIT found that even good-faith fine-tuning to improve medical knowledge
eroded guardrails in ways that were hard to predict. The line also blurs on
purpose: the International AI Safety Report notes that restricting harmful
cyber uses can slow the defensive tools that find vulnerabilities first.
Then there is the value
chain. High-risk obligations fall on the deployer, the Indian bank or hospital,
while knowledge of what the model can do sits with the provider, often abroad,
whose terms of use offer few assurances. A deployer cannot certify what it
cannot see.
I do not think anyone yet
has a clean answer to this, and I would rather say so than pretend. The best
available approach is defence in depth: capability testing before release,
controls and monitoring at the point of use, incident reporting afterwards, and
a duty on model providers to share their test results with those who build on
them, with liability shared accordingly. Where those lines should sit ought to
be the first assignment for India’s AI Safety Institute, working with RBI, SEBI
and the health regulators, rather than something each deployer guesses at
alone.
Third, combine rules with
principles.
My “True and Fair” post argued for a sensible balance of rules and principles,
with the weight shifting towards principles. Rules alone will be gamed or will
age badly. Principles alone give no certainty. What works is a small number of
hard rules on the things that must never happen, and principle-based
supervision, backed by board accountability, for everything else.
Fourth, build regulation
into the infrastructure. When we built the depository, we did not govern
dematerialised shares only by writing a rulebook and sending inspectors; we
built the rules into the rails. UPI did the same for payments, the account
aggregator framework for consented data sharing, and ONDC for open commerce. AI
governance can follow the same techno-legal path: consent artefacts that travel
with data, verifiable provenance for content, and logging and audit hooks in
public AI platforms. Compliance designed into the architecture is cheaper and
far more effective than compliance inspected after the fact.
Fifth, put a review date on
every rule.
Every AI rule is a bet on how the technology, human behaviour, incentives and
institutions will evolve, and some of those bets will be wrong. No AI rule
should therefore be permanent by default. We need fixed review cycles,
sandboxes with real exit paths, and the willingness to repeal rules that no
longer serve their purpose, or to tighten them quickly when a new capability
appears. The rules of T20 are not the rules of Test cricket, though both are
cricket. The regulator’s job is to watch how the game is changing and modify
the rules accordingly.
Sixth, make those who screw
up pay up.
Tolerance for honest first errors must be matched by real consequences for
negligence and repeat failure. Liability is the most efficient regulator ever
invented, because it aligns the incentives of the people who understand the
system best. Without it, every other measure is advice.
Seventh, do not treat a
human click as a safeguard. Many frameworks, ours included, rely on keeping a human
in the loop. But requiring a person to approve what an AI agent does assumes
that the person can understand the action, catch the error and resist simply
approving it. Approval fatigue and the pressure to stay productive turn
oversight into rubber-stamping, and a signature that satisfies the rule
protects no one. Human oversight has to be designed, staffed and tested, and
the human must have both the authority and the time to say no.
The second challenge has changed shape
The other half of my 2010
title, the regulation of innovation, was about something else. I argued that
governments must enable and protect disruptive innovations, because incumbents
invested in the old system would work overtime to stop them. I gave the example
of the oil industry, and recalled the stiff resistance we faced from custodians
and registrars when we set up the depository for the Indian capital market.
With AI there is a twist I
did not anticipate. The incumbents are now the innovators. The firms at the
frontier of AI are among the largest companies the world has seen, and some of
them have been among the loudest voices calling for regulation. In 2023 the
head of the best-known AI company told the US Senate that a licensing agency
might be needed. Licensing can be a genuine safety measure. It can also be the
most effective moat ever built, because the cost of compliance falls lightly on
a trillion-dollar firm and heavily on a startup in Bengaluru or Jakarta.
So the
regulation-of-innovation challenge today cuts both ways. We must protect
citizens from the incumbents’ innovations, and we must protect the next
generation of innovators from the incumbents’ regulation. I wrote in March that
the deeper risk of AI is the concentration of intelligence infrastructure in
very few hands. Regulation designed carelessly will accelerate that
concentration. Regulation designed well, combined with public digital
infrastructure for compute, data and models, can diffuse AI to the edges.
The regulator we need
In 2010 I described the
regulator we needed: intellectually capable, courageous enough to take
unpopular decisions, morally strong enough not to be swayed by money or
influence, and wise enough to get out of the way of genuine game changers. I
said such leadership is often available only in limited edition.
The AI age demands one more
quality: humility. The regulator must accept that it will often be wrong, and
design systems that detect and correct those errors quickly. And it must bring
into the room those who bear the costs: the data annotator, the back-office
worker, the borrower declined by a model she cannot question. Governance shaped
by a few people in a few rooms will serve those few.
The horses did not leave
our streets because a regulator banned them. They left because a better
technology arrived. Then that technology needed seat belts, traffic lights,
emission norms and driving licences, each introduced after the damage became
visible, and each late. For everyday uses of AI we can afford some of that
lateness, provided we learn quickly and make those who cause harm pay. For the
most dangerous capabilities we cannot, because some accidents cannot be counted
afterwards. There the seat belts have to be built while the car is still being
designed. We will get it far more wrong if we pretend the choice is between
innovation and regulation. It never was. The real choice is between clumsy
regulation and innovative regulation.
I closed the 2010 post with
a line from Arthur C. Clarke. It reads differently now that we have built an
intelligence that is not our own.
“It has yet to be proven that intelligence has
any survival value.” Arthur
C. Clarke
References
From my own writing
2.
T Koshy, “To be or not to
be IV: Challenges of Regulation”, Rollingstone Revelations, 25 May 2010
3.
T Koshy, “Of being True and
Fair”, Rollingstone Revelations, 20 July 2010
5.
T Koshy, “Training
Artificial Intelligence Under India’s Data Protection Regime: Navigating the
DPDP Act’s Silent Fault Lines”, Rollingstone Revelations, 17 April 2026 (also at tkoshy.me)
6.
T Koshy, “My Speech at the
AI-DPI ’26 Conference”, tkoshy.me, June 2026
Other sources
9.
CMS, “The EU’s Digital
Omnibus on AI: What the Provisional Agreement Really Changes”, May 2026
10. Tech Policy Press, “EU’s AI
Act Delays Let High-Risk Systems Dodge Oversight”, April 2026
12. Mintz, note on the December 2025 AI
executive order and the Senate’s 99-1 vote
14. MeitY, India AI Governance
Guidelines, 5 November 2025
15. Reserve Bank of India,
Framework for Responsible and Ethical Enablement of AI (FREE-AI), August 2025
17. Center for Democracy &
Technology, “New Report Reveals Unexpected Safety Risks from AI Fine-Tuning”
19. Future of Life Institute,
“Can we rely on information sharing?”, October 2023


