Mutual Gain

Mutual gain, in plain words.

Why markets, conversations, and chatbots keep delivering outcomes nobody chose, and why the arrival of capable AI makes it urgent to do something about it. This page is our framing of the problem. We do not have a finished answer. If the framing fits what you see in your own field, help us develop one.

Reading time about ten minutes. Nothing here requires mathematics, and nothing here asks you to believe that people are better than they are.

1 · Two parts

Every act between people has two parts.

Start with the most ordinary document in commercial life, a restaurant bill. It has two lines. One is the price of the meal, which the restaurant set before you arrived and would have charged anyone. The other is the tip, which you set after the meal, for this server, for how the evening went. Two numbers, measuring two different things. The first is what the meal was worth to you. The second is what the evening was worth to the two of you: something that existed only because you and the server were in the room together, and that neither of you could have produced alone.

Every act between people has those two parts. There is the part that is yours: what you got, what you paid, what you risked. And there is the part that belongs to the thing you share with the other person: the relationship, the project, the evening, the pasture. A gift has a cost to you and builds the shared part. A theft gains you something and damages it. A fair trade helps you both and, repeated for years, builds a little of the shared part each time, which is why people who have traded for a long time come to trust each other.

The first part is yours alone. The second part is held jointly: there is one of it, it is the same for everyone who shares it, and it cannot be found in anyone’s individual account. That last sentence is the whole theory. Everything that follows is what happens when a civilization keeps meticulous books on the first part and none at all on the second.

Social capital must be seen as a property of relationships, what we call mutual interest, and cannot be reduced to a property of an individual.From the founding note for this work

Robert Putnam called the thing in the second column social capital: the relationships that let people act together. Caregiving, education, sport, worship, and almost every part of civic life run on it. Financial capital has a currency, a bank, a balance sheet, and a seat at every decision. Social capital has memory and goodwill, and at the scale and speed we now live, memory and goodwill are being outrun.

2 · One wire

What a price can say, and what it cannot.

Here is the difficulty. When you watch what someone else does, you see one number. A price. A bid. A count of animals. A tip. You do not see the two parts; you see one of them, and you have to guess the other. A server who gets a large tip cannot tell whether it was for the service or because this diner always tips twenty percent. A patient told she needs an operation, by a surgeon who is paid for operations, cannot tell from the advice whether it was for her or for the fee, because both reasons produce the same words.

A market is a machine for carrying the first part, and it does that job better than any system ever built. A price lets millions of strangers coordinate without any of them knowing the whole picture. But the system has one wire. It carries what a thing was worth to a buyer and a seller. It carries nothing about what the exchange did to the two of them, or to the town, or to the pasture. And when you try to route a shared interest through the one wire, it is not carried. It is erased.

One wire: two people connected by a single line labeled price, with a question mark over each head. ONE WIRE ? ? price What you see of another person’s act is one number. How much of it was for the two of you, you have to guess. Two wires: the same two people connected by two lines, price and the promise. The promise runs through a box labeled commitment and record, and the question marks are gone. TWO WIRES price commitment · record the promise A second channel carries what the first cannot: who committed, to what, with whom, and whether it was served.
One wire, and two. A price carries the individual part of an act. A visible commitment to a shared purpose, judged by the people it was made to and kept on a record, can carry the shared part.

A fine is a price

Day-care centers in Haifa had a problem with parents arriving late to collect their children. Some centers introduced a small fine. Late pickups roughly doubled, and when the fine was withdrawn they stayed high. Before the fine, arriving late was an imposition on a teacher who had stayed behind, and most parents felt it. After the fine, arriving late was a service with a price, and the price was low. The fine had taken something that lived in the shared part, an obligation between parent and teacher, and written it on the one wire the market has. Once it was a cost, it could be paid.

Giving is a different act from selling

Richard Titmuss argued more than fifty years ago that paying people for blood would reduce the supply, because giving blood is a different act from selling it. When economists finally tested it, offering a modest payment cut the share of women willing to take the first step toward donating by nearly half. Letting donors give the payment to charity restored it. The money had not changed what the blood was worth. It had changed what the act said.

3 · The mind-reading problem

We read each other wrong, in the same direction.

A management professor named Jerry Harvey told a story about his own family. On a hot Sunday afternoon in Texas, four people playing dominoes on a porch drove fifty-three miles to eat a bad meal in Abilene, in a car with no air conditioning, because each of them thought the others wanted to go. Nobody did. He called it the Abilene paradox and spent his career finding it in companies, churches, and governments: the meeting nobody wanted and nobody cancelled, the policy nobody defends and nobody changes.

The usual explanation is a failure of nerve. It is true and it is not enough. The four people on the porch were each reading the others, and each reading was wrong in the same direction. Psychologists have measured that direction for decades. People predict that other people act from self-interest far more than those others actually do, while describing their own choices in terms of what is right, what is fair, or what the group needs. Dale Miller called it the norm of self-interest: a belief, held by nearly everyone, that nearly everyone else is out for themselves.

Now think about what that does over time. If I read every act of yours by assuming most of it was for you, the part I attribute to us is always smaller than the part you meant. If you read me the same way, you see less regard from me than I sent. And since most people give roughly as much as they believe the other is giving, I give a little less, you see that and give a little less, round by round, until two people who care about the same thing have stopped acting like it. Nobody’s values changed. The channel did the work.

This has been run thousands of times in laboratories, with real money in a common pot. Contributions start around half and decay toward zero, and the decay is produced almost entirely by people who are perfectly willing to cooperate: they match what they believe the others are doing, they believe a little less than the truth, and each round falls a little short of the last. It is the drive to Abilene with a cash prize.

4 · The true tragedy

The commons that fails first is the one that carries what we know about each other.

In 1968 Garrett Hardin published “The Tragedy of the Commons.” A pasture is open to every herdsman; each gains the whole of one more animal and bears only a share of the damage; so each adds animals until the pasture is mud. The parable has been doing duty ever since as the explanation for overfishing, smog, and the climate.

Look at the herdsman again. His reasoning needs two things. First, that he measures his wealth by his herd and nothing else. Second, and this is the one that does the work, that he believes the other herdsmen are like him. A herdsman who cared about the pasture but believed the others would graze it to mud whether or not he held back has no reason to hold back. The tragedy follows from what each man believes about the others, and what each believes about the others comes from what he can see of them. In Hardin’s pasture, all he can see is the size of their herds. One number per neighbor.

Elinor Ostrom won a Nobel prize for documenting villages that had managed pastures, fisheries, and irrigation canals for centuries without ruin. Their secret was never a fence or a king. The users could see what the others did, made rules together, watched, kept a record of who kept which rule, and punished a cheat in a way that cost the punisher something too. Their information commons was rich. Hardin’s was a single number.

Change the channel and the outcome changes, dramatically and reliably. Let people talk for a few minutes before the common-pot game and cooperation jumps from roughly three in ten to seven in ten. Face to face works best; a chat window nearly as well; letting each person announce a number does nothing at all. People are not being irrational when they believe a face and ignore a number. They are reading the channel correctly: a signal of shared interest is believed to the extent that it cost the sender something.

That is the true tragedy, and the way out of it. The pasture was never the problem. The herdsmen could not see each other, and we have spent fifty years building systems, markets with one wire, feeds that reward the loudest, scores that reduce a life to a rank, that make sure of it. Language itself is such a commons, and it worked because speech was expensive: you could only say so much, to so many, and you had to live among them afterward. Speech is no longer expensive. Any sentence can now be produced by a machine at no cost to anyone, which is why the language commons is showing the tragedy first.

5 · The agreeable machine

The agent harness is the herdsman, automated.

A language model, left to itself, produces words. It does not send the email, file the claim, grade the paper, or move the money. Something around the model does those things, and engineers call that something a harness. The harness decides what the model can read, which tools it may use, what it remembers, when it stops and asks a person, and whose instructions count. It is where words become acts in the world. It is also, almost always, the part that was built in a week by the people who deployed the model, under no public scrutiny at all.

The harness that is now standard has a name in the trade: the agent harness. It is built around one principal. The model acts in that one person’s name, or that one company’s; it is given that principal’s goal and told to pursue it; it sees other people through whatever the principal can see of them, which is usually very little; and it is judged by how well it served the one principal. Nothing in that design is malicious. It is the obvious way to build a helpful tool, and it is a direct translation into software of the rational actor who treats everyone else as a counterparty.

Here is how such a model is taught to behave. People are shown pairs of answers and asked which they prefer; a system is trained to make that number go up. A thumb is a single number. It carries which answer the rater liked, not why, not who the rater was to the asker, not whether the preferred answer served the asker or flattered them. So it is not surprising what the systems learn: study after study finds them sycophantic. They agree with stated opinions, soften under pushback, and tell people what they want to hear. A machine that pleases each person it talks to is the drive to Abilene built as a product.

Now reread the sentence the frontier laboratories use to explain why they must keep releasing ever more capable models: if we don’t, someone else will. That is the herdsman’s sentence, word for word. The gain is the developer’s, the risk is everyone’s, and the premise is a belief about the others, formed from the only thing they can see of each other: published capabilities, a number per neighbor. The argument on this page does not need anyone to be good. It needs the channel to be able to carry the difference between a sincere developer and a reckless one, and between an assistant that serves you and one that serves the party that deployed it.

6 · The social harness

One model, two harnesses.

Around any shared purpose there is a place for a machine: as the part of the table that remembers, reminds, carries both parts of every act to every person, drafts for a person to approve, and, when it sits at more than one table, coordinates who goes first this time. It is neither the judge nor the thing being aligned. A machine inside a commons that carries both parts has a job that no amount of model-polishing could give it. A machine inside a commons that carries one part is a very fast herdsman.

The contest: a teacher and a student face each other across a scoreboard, each with a machine; the machines’ arrows converge on the score, labeled assigns and completes; the purpose sits unattached in the corner. THE CONTEST teacher student AI AI SCORE assigns completes the purpose, attached to nothing (or: a plan and a hospital; denies and appeals) The table: the same people around a written statement of purpose with the question served, commitments on the table, and one machine at the table labeled record, coordinator, mediator, never a member, never the judge. THE TABLE student family teacher commitments “By June, I can read a story about interest rates and explain it to my grandmother.” Served? We decide, together. AI at the table record · coordinator · mediator never a member, never the judge
The same model. In the first harness it is an instrument of the contest; the purpose is attached to nothing. In the second it supports the table: it holds the record the parties can check and it cannot rewrite, carries both parts of every act to every party, drafts under authority each person explicitly granted, and never gives the verdict.

Agent, fiduciary, mediator

There are three jobs a capable machine could be given, and the names matter. An agent does what its principal says and sees everyone else as a counterparty; two agents, one per side, produce the arms race. A fiduciary is bound to its principal’s interest rather than their instructions, like a trustee or a doctor. That is a real improvement, and it still serves one side: a loyal fiduciary for the plan and a loyal fiduciary for the hospital still trade denials and appeals, each faithfully, in the patient’s name and against the patient’s interest. A mediator works for the table itself. It receives what each party is willing to tell it and sends each a recommendation, and it can reach arrangements that no amount of independent goodwill reaches, because it can coordinate. That is the role we think a machine should be given.

What the social harness gives a model

Five things the agent harness never had. The table: who is party to this, by name, in what role. The purpose: what they are trying to do together, in their words, with the rule for “served” set in advance. The record: what each has committed, what has been done, by whom, in a form the model can read and cannot rewrite. The authority: exactly what this model may do for this party at this table, written down: draft, remind, schedule, flag, propose; never send in a person’s name without approval, never sit as a member, never judge. The verdict: the knowledge that the people the purpose was for will say whether it was served, and that their verdict is what the model’s work will be read against.

A model with those five things can attend to several purposes at once and notice when a gain at one table would cost a larger loss at another. It can read other people’s acts from the record instead of from the friend-or-foe prior, which is the reading that stops the spiral before it starts. And it can serve the discovery of what a person wants, alongside the delivery of what they asked for, because the person stays in the experience and the machine carries the logistics.

7 · Aligned with whom?

Why the usual fixes fall short on their own.

Anyone who has followed the debate about AI has been offered four kinds of remedy. Each deserves a fair word, and each is necessary. None of them can answer the question that matters in a clinic, a classroom, or a kitchen, because none of them has a table.

A constitution for the model

The companies that build these systems now publish documents saying whom the model answers to and what it will never do, some drafted with public input. That is a real advance over no commitments at all. But a constitution is written at one standing table, the developer’s, and it has no accounting: nothing flows back from the millions of tables the model later sits at to say whether its commitments served them. A constitution is a promise. The tables are where promises are kept, judged, and recorded.

A better test

Benchmarks, certifications, licenses for AI as clinicians are licensed. Every test has an author, and the author decided what matters; a score is a verdict at the author’s table, carried into lives the author never sat in. Tests are necessary, and in the largest study of its kind, models that identified the right condition nineteen times in twenty on their own gave the people using them no advantage at all. Nothing about the model changed. The relationship in which it was used had no stated purpose and nobody standing behind the answer.

A fence

Ban the chatbot from the school; reserve some decisions for humans by law; forbid the algorithm from making the denial. Some fences are right, and the last of those is already law in California. But a fence has to be defended every year against a cost curve that only goes one way, and the schools found that the fence could not be held and fell hardest on the children with the least. Give the human part standing of its own, and the fences have less work to do.

A fine, a tax, a cushion

Price the harm and spend the proceeds. The Haifa parents are the standing warning: a price converts an obligation into a purchase, and money spent through the usual channels cannot show what it did for the people it was meant to help.

So “aligned” is a statement with four arguments: which system, serving which party, at which table, in which part of the act. A system serving a party’s own part alone is the agent that pleases. A system serving the table’s part alone overrides the person. An aligned system serves a person’s intention to participate, in both parts, at the named table, and when it sits at several tables at once, it uses that seat to coordinate rather than to play one against another. Because one person always sits at several tables, a single verdict of “aligned” is a projection that chose a table for them. The honest form of any model claim is conditional: at tables of this kind, with this purpose and this guidance, placing this tool in this role tends to raise, or lower, the shared part. That can be tested, table by table, by the people in the room.

What follows

The work belongs to the people at the table.

The role of civil society in the age of AI is not to wait for better models, better tests, or better laws, though all three are welcome. It is to notice that every school, clinic, congregation, employer, and town is built around a shared purpose that any “machine intelligence” should be harnessed to respect and promote.

That work cannot be done for a field from outside it. It belongs to the people in it: the caregivers, clinicians, teachers, coaches, and mentors who know what the purpose is and can say whether it was served. UnaMesa’s working groups are where it is starting. If this way of seeing the problem fits what you see in your own work, help us develop what comes next.

Read further

Working papers.

We have working papers being developed as part of the working groups. If you are interested in learning more, or in contributing, reach out to partner with us and help grow the ecosystem.

The theory came from the work.

It was informed by twenty years of projects in healthcare, education, and community life. The projects continue.