Winter of AI Discontent
Senator Stone leaned back in his leather chair, feeling his back fit into the long-worn cushions conforming to him. His fingers rested lightly in front of his face as his chief of staff, Jenny Patel, sat down across from him.
“Jenny, walk it through me again in layman’s terms. How exactly did this rogue algorithm cause the market meltdown yesterday?”
“It was a domino effect. One slight miscalculation in the system. These things aren’t exact. They have a slight chance to make mistakes.”
“But how could something so complex have such a trivial flaw? I thought these systems were protected by fail-safes and redundancies.”
“They are supposed to be. But even if they are 99.99% accurate, there is still the chance for a black swan event.”
“But why did it happen?”
“We don’t really know. These models are full of billions of parameters. Any one could’ve led to the miscalculation that caused the meltdown. It’d take forensic engineers years to find it.”
“We don’t have years,” Stone sighed, feeling a weight of responsibility. “Millions of Americans are stuck having to pick up the pieces of this disaster. Their retirement accounts are hanging in the balance. We cannot let this happen again. We need to act swiftly.”
“We are Senator,” Jenny said with enthusiasm. “We are already drafting a comprehensive legislative package to address the causes of the flash crash. This will include stricter regulations of AI development, requirements around transparency and accountability, and a series of robust safeguards.”
Stone nodded slowly and approvingly. “Good. We must move swiftly and decisively. Start drafting a press release. The future of our financial system, and perhaps the whole nation, will depend on our work here today.”
The “Accountability for Internet-connected systems Act”, Senator Stone’s proposed legislation, quickly created opinions on both sides of the issue. This bill, when the draft hit the newswire, was a sprawling behemoth of legal jargon and agency directives. The goal was to design a strict framework for AI systems to prevent them from having too much power.
First it created a new agency, the Algorithmic Oversight Bureau. This AOB monitored and evaluated all AI systems that would touch industries like finance, energy, healthcare, and another dozen. A cadre of AI experts would be hired, in addition to ethicists, legal scholars, and political staffers who would study each code change and each model upgrade to ensure they met standards before they were licensed to operate.
Additionally, this act imposed stringent requirements around transparency. Regulators would have complete access to its internals to allow for independent scrutiny. They were also empowered to make their own changes to the models before they were available to the public.
Senator Stone, with his controversial and broad legislation, found himself at the center of a storm. He had allies and detractors on both sides of the aisle.
As he was in his office one afternoon when he was visited by Ronald Klein, the charismatic CEO of the world’s biggest AI company. He was famous for his broad pronouncements and his slightly-too-optimistic predictions. More importantly, he was a vocal critic of this proposed bill.
“Mr. Klein, please sit down. I suppose I don’t need to guess what you’ve come here to speak to me about,” the senator said in a cordial Colorado accent.
“It is an honor to meet you in person Senator,” Klein said in a voice dripping with charm. “I understand you have concerns about the flash crash. I share those concerns too. But I have taken a close look at this bill. It won’t do anything to prevent future crashes. Instead, all it will do is stifle innovation, cripple our industry, and ultimately hurt the common people you say you protect.”
Stone leaned back in his chair, feeling that comforting cushioning.
“Mr. Klein, I appreciate your perspective. I also cannot ignore the risks of unchecked AI. You’ve had it good for too long. This crash was a wake-up call.”
“With all due respect Senator, the market has already bounced back most of the way.”
“What if it doesn’t happen a second time? What if something worse were to happen to another critical piece of infrastructure? We are in a brave new world now, and we must tread carefully.”
“There are real concerns and I’m willing to offer my help to address them. However, this legislation is like using a sledgehammer to crack a nut. Regulations should foster innovation, not stifle it. AI is the engine of our economic growth, and if passed this law would grind the engine to a halt.”
“I understand your concerns, but I need to think of more of my constituents than the CEOs. We cannot allow your pursuit of profit go on without considering the potential dangers of AI. We need to strike a balance between innovation and caution.”
And the senator remained unmoved. The debate continued until finally it was brought to a vote. In the end, a compromise was reached. Several amendments had softened the impact on the industry. The AOB was established with less authority than expected.
Still, the flash crash had a profound impact on the industry, marking an end to the Wild West era.
Klein stepped out of the Senate hearing room, now a year after the bill’s passage. The act, even in its watered-down form, acted as a dagger to his throat and to his company. He had fought tooth and nail against it, but he was no longer the hero in the spotlight. The media had turned against him, portraying him as a threat to democracy.
His worst fears came true in the following months. The AOB became a bottleneck to innovation. Every new AI product had to undergo months of approvals. Even small code changes needed their sign-off. The costs of compliance caused the costs of everything to skyrocket. He saw small startups squeezed out of the market. While it bolstered his own market share, he knew that this outcome wouldn’t be for the best.
The effect on company moral was large. The labs, once full of creativity and cutting-edge research, turned into empty hallways and eerie silence. Employees left in droves, either moving to other industries or turned into mid-level bureaucrats. As the company’s stock price fell, investors left in droves.
The impact was felt everywhere. Startups folded. Universities lost funding. Innovation stagnated. The few companies which remained, like his own, were focused only on providing incremental improvements to existing products.
Klein’s own following shrunk over time. He grew into a bitter critic of the government’s heavy-handed approach. He argued that the United States was ceding its leadership to nations like China and India, but fewer people listened to him.
The flash crash was brought up every time he tried to find a third way. They were critical of him specifically. Yet the stock market continued to grow. Nobody was actually harmed. But what made him the most upset was that the market continued to grow without him.
Klein looked up at the ceiling from his hospital bed. A decade of stress and melancholy had turned him into a bitter and lonely man. A decade had passed since the crash. A decade of meager but steady economic growth.
The country was in a prosperous place. He saw the sprawling metropolis out his window. Yet that same urban utopia had a number of challenges that were unsolved. The air was thick with pollutants. Traffic snarled the streets, keeping people bottled up in small metal boxes. News feeds streamed headlines of political unrest and social strife.
He closed his eyes as his thoughts drifted to the AI projects that never saw the light of day. AI models could’ve optimized energy grids and helped push forward nuclear fusion. Medical robotics could’ve revolutionized healthcare, diagnosing diseases and personalizing treatments. Intelligent transportation could’ve eased congestion and prevented the many traffic fatalities that happened every day.
What ever happened to his former employees? He lost touch with so many of them, and so many of his friends. They were all brilliant minds who he fostered to their full potential. Now? He didn’t know.
On his bedside stand was lukewarm coffee, which he had been sipping on all morning. It was tasteless. Hadn’t he done his best to warn them? He told them the dangers of over-regulation. He said the world would suffer as a result.
They may not have considered any better futures, but he did. His prognosis was short. His company had been working on technology to address this very kind of outcome. Perhaps, if it had been allowed to proceed, his own life wouldn’t now be at its end.
He had put his life into his company. Years were spent struggling to make it a success. Perhaps it was poetic justice then, that his life was tied so closely to his company’s demise.
He felt a profound sense of loss, for himself and for all those who failed to live in the better future he had dreamed about for so long.
This story was meant to be an analogue to the Three Mile Island incident, which ushered in an era of nuclear fears and regulatory red tape. Nuclear power, if it continued to develop, it would’ve reduced tons of carbon emissions in the following forty-five years. France has long been a leader in low emissions. Many of the climate issues we had today may not have happened if we panicked less.