Originally an economic observation, Goodhart’s Law observes something we see more in today’s tech-driven, track-everything world: that measuring something inherently changes user behavior. Let’s take a look at Goodhart’s Law and what it means in your organization.
Data: how and why
When a company defines the data it collects, it defines what information is important to the company.
Companies use data in several ways. Companies might collect data to improve services or measure user behavior. Some data is required for a service to function. For example, a company needs to know a user’s name if they are to address the user by name.
Other data the company chooses to read into might not be necessary for the company to function. But they continue to collect the because it is deemed “useful to provide better services to its users”. Some questions a company can read into, given existing data, might be:
- What hours is the user asleep?
- How popular is the user among their friends?
- Which way does the user lean politically?
How a company reads into their user data is subject to scrutiny and ethics questions. This is where Goodhart’s Law comes in.
What is Goodhart’s Law?
The Goodhart Law recognizes a paradoxical line around known and unknown information. In 1975, economist Charles Goodhart wrote:
“Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.”
Goodhart observed this in the domain of economics, which he detailed in the book Problem of Monetary Management: The U.K. Experience. Within a couple years, anthropologist Marilyn Strathern recognized that this law could apply in domains beyond economics. She summed up Goodhart’s Law:
“When a measure becomes a target, it ceases to be a good measure.”
It is an admission: When data is collected, people will refine their behaviors to optimize for those data points. And, suddenly, when people are adapting their behaviors to suit those items, those data points are not as valuable a measure as they were before.
Where authenticity becomes ingenuine
Measurement can turn authentic human behavior into ingenuine human behavior. A teacher may look at his students to see which students are the better students. He might assess that based on:
- How attentive they are in class
- Whether they are asking questions
- Their grades
- Their socializing with other kids in the class
When these items become the basis for the class rubric, these behaviors become ingenuine.
No one likes that kid who knows these things about the teacher, the kid who plays into them accordingly out of self-interest. Some students game the system, trying really, really hard to be liked. They might reside at the top of the social structure in the eyes of the higher power.
Can they be blamed though? They operate within the rules given to them. And they operate within the rules available to the entire class. In one way, they have successfully listened and performed exceptionally well. Any other member of the class also had the same knowledge of the rules (we assume). But in another way, those students are somehow also perceived as ingenuine.
In person-to-person interactions, people tend to be good at identifying who is good and who is bad. All the other students know this person’s behaviors are ingenuine. The teacher likely does, too.
On social media feeds, however, identifying this behavior is much more difficult. When using technology, people no longer listen holistically with their eyes and ears. Instead, people need to create benchmarks and generalizations to grade their user’s behaviors—they are often called Key Performance Indicators (KPIs) or user metadata. And no matter how much information is taken, or paid attention to, through metadata, a person is still only making judgements about a person through distilled information.
The benefit of meeting someone in person, not online, is what you can learn or assume about that person, both with and without explicit information. When you and another person are physically present, you can pick up on tiny cues, mannerisms, and other things that online metadata likely doesn’t know. In person, you can make a holistic assessment based on how that person speaks, how they appear, how they engage with you, how they treat the café barista, and a whole host of tangibles and intangibles.
Technologically, far fewer data points are available. Interactions are impersonal. Bad actors have an easier time of gaming the system.
Whether it is from a classroom setting, a small company, a one-on-one meeting, or measuring the GDP of a country, Goodhart Law’s seems to find value in social settings: “When a measure becomes a target, it ceases to be a good measure.”
Google Search Engine
Google’s search engine can make or break a company’s profits. Being in the top 1-5 search results versus the 6-10 search results can turn a company’s profits sour by thousands of dollars or more.
When a company has that much to lose—or gain—they will want to understand how to break into the top five results and hold their position. For them, it’s worth it to hire marketing gurus and SEO savants and pay the thousands per week just to hold their position in the top spot. (This means entire industries are created just to game algorithms.)
But, at its core, it is Google’s job to provide the best search results it can. Google is to give people the information for which they search. What we see, then, is a continuous struggle between people:
- Those with authority trying to provide the best services they can.
- Those who will exploit the system to get themselves on top.
The problem with this becomes a game for Game Theorists to analyze: one where private information exists—and the existence of private information doesn’t always sit well with people.
TikTok
Tiktok was recently pressured into exposing its content algorithm. People wondered, “Why is it so addicting?” “How do they make it work?”
Content creators (a fancy way of saying humans with a camera and a fun idea) get lucky with a viral video, and suddenly their social worth, their clout, jumps significantly. Many people want their Warholian 15 minutes of fame and will game the system to earn it.
Now, the TikTok algorithm undergoes incredible scrutiny by the creators because they want to figure out how to get their names at the top of people’s news feeds. They ask questions like:
- Should I wear a certain color?
- What song should be in the background?
- Should I buy followers?
- What time should I make the post?
- How do I get people to share this post? Do I pay them?
TikTok’s initial aim would be to let people naturally like and share a video on its marketplace of videos. Whatever video gets the most likes and shares should propagate around the internet. If that’s what people like, then that’s what people like.
But we know that people are gaming their likes just so their video can be seen more. Now, TikTok’s initial aim is gone. Instead, they must ask the question, “What is this person’s intent with pushing their content into the world?”
When people game the algorithm, it forces platforms like Google, TikTok, Pinterest, Facebook, YouTube, Twitter, et al., to:
- Judge the creator’s intent.
- Moderate what information is good and what information is bad.
Companies need to keep algorithms private
To combat people gaming their algorithms, companies need to keep their algorithms private.
The masses prefer authentic behavior. But it isn’t the recent phenomenon of social media that has changed this. For decades, Hollywood has bred more and more crazies who try to game the show business system. They try to have the right hair, the right personality, the right jawline to be accepted by Hollywood and to get their role in a play, show, or movie.
Social media has opened this game to the world, creating more types of Hollywood stars—influencers—pushing the same identity and personality problems from Hollywood to a global audience. In any environment of gamblers and actors, those seeking to game a system, there will be winners and losers. The two groups will need to learn how to play well with one another.
But, when the company keeps their algorithm private, people grow skeptical about how the company makes its decisions. They doubt the company’s integrity. They question the company’s agenda. When the users do not know the parameters of the game, they are lost and less trusting of its service.
Goodhart’s Law requires balance
Likely, the line drawn by Goodhart’s Law is one of those paradoxes not to be solved, but to be balanced. It is the delicate fabric which shifts and balances through the tensions of two forces; the ones measuring, and the ones behaving.
Additional resources
For related reading, explore these articles:
