Kill or cure? The challenges regulators are facing with AI

The scrapheap of history is piled high with failed technologies. In just the past few years, we’ve witnessed cryptocurrencies and the “Metaverse” go from the “next big thing” to “an embarrassing mistake” quicker than you can utter the initials “NFT”.

But this time, the tech seems to live up to the hype.

The public release of ChatGPT at the end of last year may one day be looked back on as a technological moment akin to Netscape releasing the first (commercial) web browser in 1994, or Steve Jobs unveiling the iPhone in 2007. Because in an instant, it changed everything.

You’ve probably tried it for yourself by now. You type in a query and ask it for, well, anything – a story, a poem, an essay, a speech – and within seconds it will spit out a not-too-bad rendering of what you’re looking for, in a way that feels almost magical. Just don’t ask it for a joke.

Like how the iPhone wasn’t the first smartphone, and Netscape wasn’t the first browser, it wasn’t so much that ChatGPT proved the technology possible – the underlying GPT3 AI model had already been around for a few years – but parent company OpenAI turned what had been a neat idea into something mainstream users can play with, and that developers can build on.

Since then, we’ve seen jaw-dropping new AI capabilities appear on an almost daily basis as investment has poured into the space, covering not just text but the generation of images, music, and even video. OpenAI itself quickly found itself signing a $10bn partnership agreement with Microsoft.

As a result, today, tasks that would have required hours of tedious manual work not so long ago, like automatically summarising the key points from a lengthy report, subtly adjusting the language or tone of a speech, or writing carefully personalised messages that match a specific person’s tone of voice, can be performed in seconds.

And it is a leap forward that Bill Gates has described as “every bit as important” as the invention of the personal computer itself.

What’s perhaps most incredible is that it is still only very early days for the technology. It’s like we’ve just split the atom, to prove it is possible. So the real transformation is yet to come.
And much like splitting the atom – this could turn out to be a mixed blessing. Which is why, perhaps sensing both the huge opportunities and risks, the United Kingdom government has tried to get ahead of the industry, and the rest of the world.

It’s like we’ve just split the atom, to prove it is possible… And much like splitting the atom – this could turn out to be a mixed blessing

At the start of November, Rishi Sunak will be convening the world’s first major intergovernmental AI conference at Bletchley Park, the historic home of Britain’s Second World War codebreakers and some of the earliest computers.

Because just as splitting the atom gave us both nuclear power and nuclear bombs, the fear is that – though AI may prove incredibly useful – in the wrong hands it could literally drive humans to extinction.

Large language models

To understand why AI is seen as such a risk, you first need to understand what makes it special.

AI – or machine learning – technology has been around for about a decade now. In fact, there has even been a dedicated “neural engine” AI chip inside every iPhone produced since the iPhone 8 in 2017.

It’s the technology that, for example, will automatically pick out the faces of your friends and family in your photos app. And the idea behind AI is actually pretty simple; it’s about pattern recognition and making predictions.

So instead of trying to write “rules” in software code about how to do something, you “train” an algorithm to “learn” how to perform a task by providing examples. For example, if you show an AI model 10,000 photos of dogs, it will be able to pretty reliably guess correctly whether there is a dog in any other photo that you show it, having identified the mathematical similarities in all of the images. And if you feed it even more images – say, 100,000 photos of dogs – it will be able to guess with even greater accuracy.

And the more recent breakthrough is essentially the result of following this same principle, but doing it on an almost unbelievably supersized scale.

What unites GPT, the AI model behind ChatGPT, and the likes of Google’s Bard, Meta’s Llama and the image generation tool Midjourney, is that each are powered by “Large Language Models” – or LLMs.

These are trained in a similar way. An AI model is built by analysing the connections between different words in an enormous dataset, and crunching the results down into a statistical model. In the case of GPT4, the most sophisticated LLM at the time of writing, this accounts for trillions of different connections – and the training data was virtually every publicly available scrap of data on the internet.

And what makes these LLMs special is that because they have been trained on such a vast scale, they’re essentially very, very good at guessing what text to show you next. When you give ChatGPT a prompt, it is essentially making a statistical prediction about which words would follow the previous words in the sentence, given all of the prior context. And the results are both slightly eerie and feel almost magical.

The alignment problem

So AI then is this amazing new, powerful technology, which unlocks some incredible capabilities. But what’s particularly exciting is that we’re only at approximately the Wright Brothers stage of development. At the moment, ChatGPT, Bard, and all of the other models are little more than toys to experiment with.

It’s only over the next few years that we’re going to start seeing AI-powered tools become properly integrated behind the scenes in apps, software and services that we use everyday. For example, early next year, Microsoft will be integrating AI capabilities directly into Microsoft Office, and it will be able to look through your emails and personal documents, and do things like provide summaries of the key points raised in meetings.

This all sounds pretty useful, so you can be forgiven for wondering why so many people are worried about the technology. Why is a clever app that can write a passable covering letter for a job application possibly going to cause the end of the world?

The answer is to do with what AI theorists call “value alignment”. The problem is that AIs approach problems very differently to humans, and that when an AI system approaches a problem, it may lack the common sense that we humans take for granted.

The problem was famously described by the philosopher Nick Bostrom using paperclips; if you were to ask a human to make as many paperclips as they can, however important the task felt to them, their quest would be limited by morals, ethics and values. It’s unlikely that even the most zealous paperclip enthusiasts would scheme to obtain all of the world’s metals, or even harm others to achieve their goals.

But if you were to task a sufficiently intelligent “frontier” AI model with the same goal of producing paperclips, the fear is that with a singular focus on this goal, it would do whatever it could to produce more. And because the imagined system is smarter than any individual human, it can instrumentalise us towards this goal – perhaps even by manipulating us.

This is a hard problem to get around because it is tricky to issue specific instructions to an AI system that account for everything we take for granted. And that means they may not figure out how to solve a given problem in the “right” way.

Any hope of reigning in some of the negative consequences of AI, like the production of deep-fake images and audio, is already lost

One famous example of this is from way back in 2013, when a coder taught an AI system to play the classic video game Tetris. Instead of the system learning to ever more masterfully place the falling blocks into the right places, it figured out that it could avoid defeat in the game simply by hitting the game’s pause button, and leaving the game in that paused state indefinitely.

So with the same sort of logic it’s possible to imagine dystopian science-fiction scenarios, such as an AI that destroys all humans because they are getting in the way of the production of paperclips. And yes, it might sound silly, but looking at a future where AI is even more sophisticated, and has many more conduits to affect the real world (such as control of more real-world machines like autonomous cars and factory equipment), many existential risk theorists do take these concerns seriously.

But even if you’re still not entirely convinced about the paperclip problem, there are still many more “real-world” problems that can emerge from AI misalignment.

For example, as AI systems are integrated into more aspects of our lives, they could lead to unfair outcomes. Imagine an AI system designed to make law enforcement or job hiring decisions that has been trained on historic data about the sorts of people who do specific jobs or commit specific crimes. Such systems might result in racist or sexist decisions being made about who to hire and which people to declare suspects in investigations, because of the biases in the underlying training data.

Or imagine an AI system tasked with trading stocks to maximise value (which already happens in some cases). Similar to the Tetris problem above, it’s conceivable that an AI system may design a trading strategy that inadvertently leads to market crashes. Or if you have a recommendation AI that is tasked with increasing the watch-time on videos above all else, it may find out that extremist content is what is most effective at achieving that goal, so may promote undesired material.

However, this is just scratching the surface. The fundamental problem is that unlike traditional computing, where decisions are made using clear, conditional logic, it is much harder to unpick the “black box” decision making of AI models. We can observe what goes in, and what comes out – but the risk is that we may not be entirely aware of what happens inside.

Into the unknown

This is ultimately why AI could prove to be a mixed blessing. It presents a huge opportunity to grow economies, increase productivity and improve human wellbeing. But the same “frontier” AI models could also prove dangerous.

For politicians and policymakers, the core challenge is that the technology is still evolving at speed.  Given the wide-ranging impact AI could have across every aspect of society, the challenge of setting the rules now is like asking politicians after the Wright Brothers’ first flight in 1903 to write the rules for air traffic control and aircraft safety, but expecting them to imagine there will be tens of thousands of gigantic commercial airliners in the sky at the same time.

But we are starting to see the parties tentatively grapple with the AI question. For example, back in June, Keir Starmer promised that Labour would introduce an “overarching regulatory framework”, and that it would be “stronger” than the Tories’. Lucy Powell, who then held the party’s digital brief, told The Guardian that AI models would be regulated in a similar way to the pharmaceutical, medical and nuclear industries.

However, the current government appears to be taking a slightly more hands-off approach, earlier this year announcing that it would be leaving it up to individual regulators, like the Competition and Markets Authority and Health & Safety Executive, to set their own rules.

This, though, is essentially just regulation of the downstream consequences of AI – setting the rules about when and where AI models will be allowed to be used.

When it comes to the much bigger question of regulating the enormous frontier models themselves, like GPT, it will likely require international agreement. Rules and norms will be determined by the actions of the United States, the European Union and China, as the home and biggest markets for the most prominent AI players.

The good news for any prospective regulators is that they do have one advantage; marshalling the computational power to train a model as sophisticated as GPT4, or a hypothesised “frontier” model, is incredibly expensive, and requires vast server farms of computers crunching data. That means that only a handful of the largest companies will be able to build them.

The bad news for regulators, however, is that once a model has been computed, the results can typically be crunched down significantly – and any modern computer can produce AI-generated outputs. In fact, Llama, the model pioneered by Meta, is designed to run on an ordinary desktop computer. That essentially means any hope of reigning in some of the negative consequences of AI, like the production of deep-fake images and audio, is already lost. So, the AI genie is already out of the bottle. 

PoliticsHome Newsletters

Get the inside track on what MPs and Peers are talking about. Sign up to The House's morning email for the latest insight and reaction from Parliamentarians, policy-makers and organisations.