A sideways look at economics
Let’s play. Those words, delivered to a small group of like-minded people you trust, are the surest way I know of finding a creative solution to some problem. But the mood that makes play possible is fragile: one difficult person, one sharp comment, and it vanishes like a soap bubble and you will struggle to recover it. So when we set out to invite a room of 100 people to play, it was with some trepidation. These were highly successful, opinionated and argumentative people too. And the problem? How to draft a constitution to govern the development of AI, globally, for the foreseeable future. Amazingly, the game was a success. The constitution? Read on.
The group entered a large ballroom, and we invited them to choose a seat in any of three circles of chairs. On each seat was a card denoting the role they would play for the duration of the game. There was lots of: “Where do I sit?” (Wherever you like.), “Why can’t I sit where I was yesterday?” (Well, you can, but you’ll find there is no chair.), “What’s this card for?” (It’s for you to read.) and “What do I do with my laptop, there’s no table?” (Put it on your lap or, better, under your chair.) People drifting in, drifting out to get their third coffee of the morning, or take a call. People chatting in small huddles, standing up. Herding cats wasn’t in it. Some, we knew, had enjoyed a fairly heavy night the night before, steeped in Old Fashioneds and Cosmos. We knew, because we’d been right there among them.
To call the meeting to order the hotel had provided a chime, but the sound ‒ though very pretty ‒ might have been audible to those alien creatures in A Quiet Place, but not to that group of people. I resorted to loud whistling through my fingers, a feat I had not attempted since my teens. A serious learning for me was that options like whistling (or projecting my voice, or outright shouting, as I tried next) that raise the prevailing noise level are much less effective in large groups than saying “Shhh” for as long as it takes. The great thing about shushing is that other people join in. Shushing the group worked. (It’s a good thing that others did not join in with the whistling or shouting).
One circle of chairs was for people playing the role of investors; one was for AI companies; and the final group was for ‘workers’ in unspecified industries. Once the participants discovered their roles, there was a good deal of “Aren’t we all workers?”, “What does ‘investor’ mean?”, “I should be in another group, I haven’t ‘worked’ for twenty years”, and so on. But it was all in good spirit.
We invited each group to elect a Chair, whose job was to ensure that their group came up with a draft constitution within the allotted time, and then to read and defend that proposed constitution to the whole gathering. The Chairs did their jobs admirably and, after much fierce debate, three concise drafts were written, only one of which had broken the rules by including a fourth article. We allowed this because the fourth article was the golden rule: treat others as you would like them to treat you. We felt that nobody could seriously disagree with that injunction. It turned out that we were wrong on this, but more on that later.
Then the whole group considered the three draft constitutions and took a vote on which they preferred. The winning constitution was adopted, and a Supreme Court appointed (three judges, one from each group). The chosen constitution was the one drafted by the AI companies, which was also the one with the illegal fourth article. This created a good deal of grumbling from the other two groups, who suggested that the fourth article was what swung it. We felt it was so anodyne, so ‘motherhood and apple pie’, that it couldn’t possibly make that difference. But the incident illustrated something else: put people in a game and they will behave competitively, no matter that they are all part of the same shared enterprise. For a short while, the fact that we had overlooked an infringement of the rules was a more important issue for some of the participants than the constitution itself. I completely understand this feeling, and I think I would feel it myself if I were in their shoes. Put me in a game and I will try to win, and I will object if the rules are not observed. Fair’s fair. We will not allow the fourth article in any future game.
I won’t reproduce the four articles in the winning constitution, because that would get close to breaching the Chatham House rule under which the whole event took place. Suffice it to say: in our opinion, the articles were loose and aspirational rather than tight, binding legal requirements. Effectively, they were a framework for interpretation, not a legal contract or anything similar to that. The articles would be interpreted by a Supreme Court whose component judges would change over time.
Once we had agreed the constitution and the make-up of the Supreme Court that would adjudicate on whether certain behaviours complied with their interpretation of that constitution, we asked each participant: would you choose to be a member of the constitutional club, or outside it? The motivation here was that the constitution was designed to function like the US constitution: to be the governing document for AI for potentially hundreds of years. Over that horizon, membership is ultimately a voluntary matter, so we chose to treat it as voluntary from the start. Being inside meant you would be bound by the rulings of the Supreme Court, which had the power to punish you if it judged that necessary, up to and including imprisonment. But you also had access to the technologies, the finance, the workers and the customer base that the insiders provided. Take your chances outside and you would be denied access to those things.
The result was interesting. The split was roughly 50:50 between those who volunteered to join the constitution club and those who didn’t (remembering that a majority of the group had just voted for that constitution). Of the 50 or so who chose to remain outside, very few were from the group of ‘workers’. The workers generally felt threatened by AI and that their only hope of protection was to be inside the constitution club. The other two groups mostly chose to remain outside the club, and not be bound by the constitution they had just drafted and voted for.
This illustrates the likely difficulties facing anyone brave enough to attempt any sort of regulation of the wild west that is the AI industry globally.
But that was just the beginning. The constitution, and the Supreme Court whose job was to interpret it, was a kind of plan. As Mike Tyson famously remarked, everyone has a plan till they get punched in the mouth. In that spirit, we then proceeded to punch the new constitution in the mouth a couple of times (metaphorically), to see how the plan would hold up. The results were salutary.
We had five ‘shock’ cards and invited the Supreme Court to pick one and read it out to the group. The shock they picked was this. One of the members of the constitution club had found a potential route, using AI, to a treatment for cancer, but it could only be pursued effectively by accessing personal information on health and lifestyle contained in confidential health records. The question for the court was: should this access be permitted?
We allowed representatives from all three groups to lobby the Supreme Court, both from those who were members of the constitution club and those who were not. This was our attempt to mimic how Supreme Court rulings actually occur: all kinds of interest groups express their views, and it is part of the role of the court to hear all those views and reflect them, where they think that is appropriate, in their rulings. The lobbying was very interesting. There were highly committed members of each group who sat down to decide a line; at the same time, others had immediately gone to the court justices and were each making their individual cases, loudly. Others were sat quietly, contemplating what they had just heard. Others still disappeared to the restrooms or for yet more coffee. In the end, we had to insist that the lobbying should stop: it would have gone on for hours had we allowed it. And, since the people in this game were typically high-achieving leaders in their own communities, they were not used to being told what to do. But they accepted our direction in the end, in the interests of the game.
The judges took their role extremely seriously. Their ruling was that access to health records should be allowed on this occasion, but not as a general proposition: any future requests would be judged on their own merit. Furthermore, even on this occasion, all the data must be anonymised and deleted once used, and any individual would have a short period of time during which they could opt out of allowing their own data to be used in such a way.
After that ruling, we asked the whole group to reconsider their position. Knowing what you know now, do you choose to be a member of the club or not? The result: the group split roughly 60:40 in favour of membership. Virtually all the workers remained in the club, and a larger, though still minority, share of the other two groups joined too.
Then the court picked another shock card. The second shock was this: intelligence agencies had found evidence that a potentially hostile power was developing drones directed by AI that were capable of autonomous use of lethal force. Should the constitution permit the members of the club to pursue the same objective? Or would doing that conflict with the ‘interests of humanity’ that prompted the development of a constitution in the first place?
Another round of lobbying ensued, if anything even more vigorous than the first. The members of the court, standing this time, hands behind their backs, listened and commented: the very image of responsible sagacity. Their ruling: it is incumbent on the constitution club at least to keep pace with advances made outside the club in military or dual-use technology, so there was no option but to allow ‒ indeed, to encourage ‒ the rapid development of similar military technology within the club.
After that ruling, the whole group was asked again: in or out? They broke roughly 75:25 in favour of ‘in’: virtually all the workers were in, as they had been all along. And the other two groups broke 50:50. So, overall, not quite a majority in every group, but not far away either.
On the face of it, this looks like a triumph for the constitution. The initial draft attracted 50% support. After being punched in the mouth, twice, it commanded the support of 75%. Tested, it came out stronger.
But of course, it is not as simple as that. The initial constitution, untested, proposed certain ‘guardrails’ over the development of AI. The guardrails were fairly weak, in our estimation, but they still bound the behaviour of club members in ways that might be unwelcome to some of them. But, faced with shocks, the guardrails were effectively lowered by a notch or two on each occasion, making them easier to jump or, by the end of the exercise, step over without breaking your stride. Nice to be in a club as long as it does not bind your behaviour in any meaningful way. The second ruling, in particular, essentially gave carte blanche for any AI development whatsoever. Since it cannot be known what technologies hostile powers might be pursuing, the court’s ruling to ‘at least keep pace with them’ is, effectively, an imperative for the club to go pedal to the metal on AI development across the board. Anything else would risk losing in some future conflict with a hostile power, and that was perceived as a worse risk than anything AI might do; or else there was the implicit judgement that if the hostile power was going in that direction anyway, then the AI would develop its potential accordingly, whether the ‘innies’ were part of it or not.
That is an interesting result and, I think, an accurate one. Let’s all play nice, unless we think our adversaries might not be doing so. As it happens, we’re pretty sure our adversaries are not playing nice. Therefore, neither should we. Do unto others as you would have them do unto you, unless they’re already doing something unto you that you don’t like.
This resembles the famous Prisoners’ Dilemma game, where if both prisoners can co-operate with each other then a good outcome is available to them. In the case of this global geopolitical game, the good outcome is voluntary regulation that most people would agree to, and the benefits of a human-friendly AI accruing to all. But if one side chooses the co-operative path and the other does not, the co-operative guys lose. Which leads with a horrible logic to the unco-operative solution, which is the worst outcome for the world: if I go high and you go low, you win. If we both go low, we both lose.
The Prisoners’ Dilemma is a one-shot game. Most games in the real world are not one-shot, but repeated games. If you set up the same structure but assert that this game will be played over and over again, then co-operative outcomes become possible.
Unfortunately, the AI game is a one-shot game. This has unpleasant consequences. For example, if we go unco-operative, and just one person ‒ anyone, anywhere, ever ‒ creates an artificial general intelligence, whatever happens after that cannot be controlled by us. The catastrophic scenario in which someone develops an AGI (or an ASI, whatever the current acronym is) that takes over the world, treating humans as disposable: that’s the scenario we are racing towards, incentivised to do so by the structure of the geopolitical game that we are playing.
In my opinion, the path we are currently on will never lead to AGI: next week’s TFIF will explain why. But I might be wrong. If I am wrong, “That’s all folks!” Some people will make enormous amounts of money, which will matter for about twenty minutes. Some countries might go to war along the way, which will matter for a few days. After that it’s none of our business.
Is there any feasible route to co-operation here? Or will the imperatives of geopolitical competition override the long-term interests of humanity? These are huge questions. How would you answer them?
Let’s play.
Want to play out the risks that matter to your organisation? Fathom’s Table-Top Exercise Service is a unique and immersive way to explore macroeconomic, financial market and geopolitical risks. Using cutting-edge technology and proprietary data, our exercises put you in the role of decision-maker, allowing you to navigate uncertainty, weigh up trade-offs and see the dynamic impact of your choices in real time. Find out more about our Table-Top Exercise Service.
Further reading