Wednesday, January 29, 2014

Peter Norvig's Economic Simulator

Let's build an economy!

I've wanted to do this blog for a while, but I had no idea where to begin.  So I want to thank Peter Norvig for giving me the entry point I was looking for.

Peter Norvig is an artificial intelligence super guru, and a director of research at Google.  He's a very bright guy. Last week he posted an entry on his blog about how to program an economic simulator in Python. This is something I've wanted to do for quite some time, so I was very interested to see how he put it together.  I would recommend you at least skim through his blog post before reading on, if you're interested in understanding how the simulation works.  In his words:
This is a simulation of an economic marketplace in which there is a population of actors, each of which has a level of wealth (a single number) that changes over time. On each time step two agents (chosen by an interaction rule) interact with each other and exchange wealth (according to a transaction rule). The idea is to understand the evolution of the population's wealth over time.
The simulator has four components.  There's a population of people with randomly distributed wealth. There's an interaction model that matches people in the population to trade with each other.  There's a transaction model that determines who gets what when they trade.  And there's a simulation model that ties it all together by creating a population, randomly choosing which people interact, executing their transactions, and reporting on the simulation results.  (In this case, the results are measurements of how the wealth is distributed among the members of the population.)

Can we do better?

Now, this is by no means a criticism of Peter Norvig, but I felt that the transactions model he used in his simulation could use some improvements. In this case, every time two people meet, the transaction function randomly choose some amount of wealth for one person to give the other, with the giver receiving nothing in return.  Nothing is produced and nothing is consumed, so the total amount of wealth stays constant. And in the end, after running several simulations with different population, interaction, and transaction models, the results are very consistent: the rich get richer and the poor get poor.

What bothers me about this simulation is that there is really no motivation for people to trade with each other in this economy. If one person is just taking from another and giving nothing in return, it is a model of theft, not trade. It's basically a simulation of a bunch of pickpockets running around and stealing from each other.

In the real world, if you and I are going to trade with each other, then:
  • You need to have something that I want.
  • I need to have something that you want.
  • Each of us must want what the other has more than we want what we need to give up to get it.

How to define economic agents

So how do we build this into Norvig's simulation? Well, first there needs to be more than one thing for people to trade. We also need some way of measuring how much people want things (economists call this measure utility). Finally, we need to figure out how people are going to negotiate with each other and strike a deal.

To keep things simple, let's look at a simple economy with only two goods. There's no production or consumption, so the total quantities stay constant. And there's no money; it's a barter economy. In this model, we can't just describe members of a population by how much wealth they have, so we're going to define an agent class as follows:

(I'm only sharing snippets here; you can find my full code for the project on GitHub.)

class Agent(object):
    def __init__(self, endowment1, endowment2, preference1, preference2):
        Agents are characterized by their allocations and preferences
        of goods 1 and 2. Economists usually call the starting value of
        a good an "endowment".
        The preference variables should be between 0 and 1.
        self.good1 = max(0, endowment1)
        self.good2 = max(0, endowment2)
        self.pref1 = preference1
        self.pref2 = preference2
    def utility(self):
        We'll use the Cobb-Douglas utility function for our model.
        return pow(self.good1, self.pref1) * pow(self.good2, self.pref2)

    # The allocation property is just syntactic sugar that lets us assign
    # allocations using tuples a little more cleanly.
    def allocation(self):
        return (self.good1, self.good2)
    def allocation(self, values):
        self.good1, self.good2 = values

    # We need to define comparison operators in order to sort the
    # agents based on utility. I always prefer to define all of them
    # if I need to define any.
    def __gt__(self, other): return self.utility > other
    def __lt__(self, other): return self.utility < other
    def __eq__(self, other): return self.utility == other
    def __ge__(self, other): return self.utility >= other
    def __le__(self, other): return self.utility <= other
    def __ne__(self, other): return self.utility != other

    # We need to define multiplication and addition operators
    # in order to use Norvig's normalize function.
    def __mul__(self, other):
        self.good1 *= other
        self.good2 *= other
        return self
    def __radd__(self, other):
        return other + self.good1 + self.good2

How to create agents in the simulation

Now, to get this to work with the rest of Norvig's code, we'll need a function that can be used to generate a random agent, a list of which can be fed into the normalize function, which in turn is called by the sample function. So here's how we create a random agent:

def random_agent(mu_e1=mu, mu_e2=mu, sigma_e1=mu/3, sigma_e2=mu/3, 
                 mu_p1=0.5, mu_p2=0.5, width_p1=0.5, width_p2=0.5):
    # e for endowment, p for preference
    e1 = max(0, gauss(mu_e1, sigma_e1))
    e2 = max(0, gauss(mu_e2, sigma_e2))
    p1 = uniform(mu_p1, width_p1)
    p2 = uniform(mu_p2, width_p2)
    return Agent(e1, e2, p1, p2)

How to make the agents trade

Here's the tricky part. Now that we have a model for economic agents, how do we figure out the terms of their trades? For this, I'm going to use an Edgeworth box as my trade model.  I'll explain what these are in my next post, but for those who can't wait that long, the Wikipedia article is a great starting point. The main idea is that if you have two people, each with some quantity of two different goods, then as long as they value good 1 differently in terms of how much of good 2 they would be willing to trade for it, they will be able to find a deal in which they trade some quantity of good 1 for some quantity of good 2, and both will be better off as a result of the trade. Now, using the Cobb-Douglas utility functions that define our agent class, that quantity of good 1 to be traded for good 2, and the resulting allocations of each good to each agent, are given in the function below:

def edgeworth_trade(X, Y):
    # Useful constants for the following calculations
    alphaX = X.pref1 / (X.pref1 + X.pref2)
    betaX  = X.pref2 / (X.pref1 + X.pref2)
    alphaY = Y.pref1 / (Y.pref1 + Y.pref2)
    betaY  = Y.pref2 / (Y.pref1 + Y.pref2)
    total_1 = X.good1 + Y.good1
    total_2 = X.good2 + Y.good2
    # Equilibrium price of good 1 relative to good 2
    price = (X.good2 * alphaX + Y.good2 * alphaY) / (X.good1 * betaX + Y.good1 * betaY)

    # Allocation of good 1, following from demand function and market clearing conditions
    allocation_x1 = alphaX * (price * X.good1 + X.good2) / price 
    allocation_x1 = min(total_1 - 0.0001, max(0.0001, allocation_x1)) 
    allocation_y1 = total_1 - allocation_x1
    # Allocation of good 2, following from budget constraint and market clearing conditions
    allocation_x2 = price * allocation_x1 * X.pref2 / X.pref1 
    allocation_x2 = min(total_2 - 0.0001, max(0.0001, allocation_x2)) 
    allocation_y2 = total_2 - allocation_x2
    return (allocation_x1, allocation_x2), (allocation_y1, allocation_y2)

I'll explain how I derived that function in my next post, along with more details of how we can use Edgeworth boxes and Cobb-Douglas functions to build a model for how people will trade with each other.

And that's it. There are a few more implementation details that need to be tweaked to get our new functions to fit in Norvig's simulation code, and anyone who is interested in playing around with the code can find those in my GitHub repository.

So what's the outcome?

Left: Utility, over time, of select agents in the economy
Right: Utility, over time, of select agents in the economy, relative to initial value
Well, for this model of trade, it turns out that everyone ends up better off than they started. Now, in absolute terms, the richest people still gain the most. However, in relative terms (that is, as a percentage of what people started with), the poorest members of the population gain the most from trade.

In fact, the very bottom percentile have a remarkably greater increase in utility than any of the other groups, who have always been lumped fairly close together in the simulations I have run. I haven't yet looked into what is causing this pronounced difference, but my suspicion is that the agents who start with the lowest utility are the outlier cases whose random endowment of one good or the other is very close to 0; this would give them a very low starting utility regardless of how much of the other good they start with. And since they start with such a low value, any increase in their allocations will have a very large impact on their utility.

Where do we go from here?

There are lots of things we can do to improve this model further. We can change the trade rules so that instead of two parties bartering, one agent trades in such a way to get the maximum possible utility while keeping the other agent's utility constant. (If that doesn't make sense now, hopefully my next post will clear it up a bit.)

We can spice up the interaction model by placing all of the agents on a 2D grid, and only letting them trade with their neighbors. Peter Norvig did this, with all agents located on a line. We could take that same idea and allow agents to move a certain number of spaces around the grid per iteration. They could wander or we could give them an algorithm that makes them move according to some deterministic behavior.

We can add production and consumption of the goods into the model as well. Suppose each agent needs to eat some amount of good 1 every round in order to survive, and they can use good 2 to produce more of good 1.

We still haven't even introduced money into this economy. And it's still only based on two goods. There are lots of places we can go with this. And that's a good thing, because even though this model captures some very fundamental principles, it's still just a simulation of a very simple, boring little world.


  1. If I recall my microeconomics correctly, there are more than one Nash equilibrium to the bargaining problem. Essentially, the question is how to divide the economic surplus - the utility available from trade. If I'm reading your code correctly, you divide the economic surplus evenly.

    I'd love to see different ways of dividing the economic surplus (e.g. random_split, winner_take_most, winner_take_all, redistribute).

    A priori, dividing the economic surplus evenly seems very similar to Norvig's "redistribute" function, and your results seem compatible with his results - a reduction in inequity. But of course data trumps a priori argument.

    1. You're absolutely right. There are lots of ways to slice the pie, and this is only one of them. I'm planning to make a post on the economics behind the model in the next couple of days, but there is a continuum of Nash equilibria that can solve this problem.

      Also, you don't need to just choose one trade function; you could provide a set of such functions, and then provide a negotiations model that determines which of these methods the agents will choose when they do trade.

    2. Actually, I take that back. One of the nuances of these results is that the simulation actually *increases* inequity over time, while at the same time giving everyone an allocation of goods that they prefer to what they originally had. This is because the wealthiest agents see the biggest increases in absolute terms, while the poorest agents see the biggest relative gains. And all of this happens in spite of no new goods being produced.

      Lots of people find these kinds of results to be counterintuitive when they first study economics, so I'Il be sure to address this issue in my next post, which will take a closer look at the economic model I used in this simulation.

  2. So, what you're saying, Cliff, is that the poor people were dying in the gutter at the beginning but now they have a shack to live in. While the rich people started out with only one vacation place in the Hamptons, and ended up with only 3 vacation homes in Maui. So the poor people are MUCH better in a relative sense than the rich people?