Rational Exuberance

Professor Robert Shiller subscribes to a theory about the psychology of asset bubbles called "irrational exuberance". The theory makes three basic claims about asset bubbles.

1. They get triggered by something external.
2. The prices rise because of a contagious fever of excitement about getting rich based on stories they hear from their friends and in the media.
3. The bubble bursts at some point because it can not rise forever, but we have little clue of the timing.

After thinking about the psychology of bubbles from an AI standpoint, I have come to the conclusion that he may be wrong on all three counts.

I'd like to propose my theory of asset bubbles which I will call "rational exuberance". Now the first step in describing my theory is to consider our thought processes in determining what we are willing to pay for an asset. I suggest that it is made up from two components that are considered separately and then combined in to a final conclusion. The components are:

A) The estimated reasonable price of the asset excluding any thoughts about how the assets price may change in the future. This is a kind of true price which reflects how much we desire the asset for its own sake, rather than as a speculative investment.

B) Our estimate of what we expect the price to be at the time when we expect to sell the asset.

Now in a stable, zero inflation, environment, estimates A and B would be one and the same value. However we would generally expect B to be different to A because of a combination of factors including general inflation, immigration rates, the birth rate, unemployment rates, social trends, the rate of housebuilding and all sorts of complicated things that people find hard to predict. I would suggest that people generally feel unable to estimate the sum total of these effects and so instead will use what I will call the "insect tracking" method:

Imagine someone observing a small spider that has just been placed on a stick. They have no idea about its motivations or what its going to do, but then they see it start to crawl in a particular direction along the stick, at first they are not sure whether its going to keep going in the same direction or perhaps it will suddenly stop or maybe head in the opposite direction.... but then imagine that it appeared to move fairly constantly in a particular direction (lets say from left to right) for several seconds. The longer it kept going, the more confident the observer would become that in the next instant it would be further right than it is now. This kind of prediction based on previous events without the slightest understanding of the underlying driving forces is both completely natural and makes good logical sense. It is certainly true that with almost any system you care to observe in life, its future behavior tends to be the same as its past behavior at least in the short term. In the longer terms things can change. Let us say that our spider suddenly (and of course unexpectedly) stopped. What will happen next? Well the fact that its behavior suddenly changed (from moving to stationary) makes us suddenly less confident about where it will be in the next instant. I could go in to a lot of detail about the relationship between its behavior up to a particular point in time and our prediction of what will happen in the next instant afterwards, but for now I will summarize it with the statement "The longer a system is observed displaying behavior X, the more confident we become in the assumption that in the next instant it will continue to display behavior X". Now I am going to add an additional complication to the insect watching analogy. Let us say that we have arranged to have a heat source placed at the right hand end of the stick. The temperature at the end of the stick is known to be uncomfortably hot for the spider. Now lets run the experiment again: the spider, as before, heads off from left to right - and again the longer it continues to travel in this direction the more confident we become that it will continue to do so, but this time we need to add a proviso - we know that the spider will not carry on walking right up the the heat source - we know that's too hot. We know it will have to stop at some point. So now our spider watching prediction could be summarized as follows. "The longer a system is observed displaying behavior X, the more confident we become in the assumption that in the next instant it will continue to display behavior X, but if we know that behavior X becomes increasingly improbable over time, then our confidence in the predictive powers of past behavior will progressively diminish".

Now lets apply the insect watching ideas to the housing market. Shiller states that something external is needed to start a housing bubble - well, I'd re-phrase that as something external can start a housing bubble, but it can also start all by itself just by random fluctuations. Consider a small town in which every year a certain number of people move out and a certain number of people move in. Let us imagine that on average these numbers are approximately equal, but simply according to the law of random numbers, in one particular year the numbers may be quite different. Imagine that as a one-in-100-year event, there was a significant predominance of new people moving in, and not only that they were on average wealthier than the average of the people living there already. This entirely random event would cause house prices to rise significantly in this town over the year.... now the spider has started to move from left to right... the people watching house prices in the town have no idea that this was a special one off rare event that will not be repeated. Instead they start to predict that house prices in the area will continue to move upwards. Estate agents will start writing in their brochures that this is an up and coming town and purchases here would be a great investment. These "truthful" statements will make the purchase of properties in this town more attractive... the ball has started rolling and you can imagine what happens next. The spider is making a steady march from left to right. The higher prices in the town will cause the prices in the suburbs rise, then neighboring towns and so on and so on.

I'd like to nit pick with Shiller again at this point. Shiller states that the "hype" and "newspaper" stories are what is driving the rise. Where as I would state that its to a large degree, the rise that is driving the rise! The newspapers are just helping the process along. Like a second spider observer telling you "Hey, did you see that! The spider is going from left to right!". You could see that already yourself - but of course the confirmation from the second observer will support or intensify your belief (that you would of had anyway) that the spider will continue in that direction.

Now lets consider the bubble bursting. Shiller appears to think the turning point is beyond analysis. I however, feel it is an entirely logical process. Remember my earlier statement: "...but if we know that behavior X becomes increasingly improbable over time, then our confidence in the predictive powers of past behavior will progressively diminish". This could be re-phrased for the housing market as "In a rising market, the more improbably stupid house prices become, the less likely we are to believe that the current rising pattern will continue." The bubble will burst when the price rise non-believers start to outweigh the believers. I don't know if surveys have been done but I would hazard a guess that in any asset price bubbles one could a detect an increase in the proportion of "non believers" before the bursting point.

It should be noted that the point at which non-believers start to outnumber the believers may be helped along by some external event. Something which makes rising prices less probable. And the further the price rises had drifted in to the realms of the improbable, the smaller the external event that would be required to trigger the reversal in the direction of prices. Now there will always be people around who will point to this trigger and claim that this was the true reason prices reversed - but it is very likely that this was simply the straw that broke the camels back and if that event hadn't happened then something else, or even nothing at all other then the inevitable rise in the fraction of non-believers, would have shortly afterwards.

P.S. In response to a comment about this blog entry, I will add a little extra detail:

When I mention looking at the proportion of "non-believers", I don't mean the proportion of politicians/journalists/economists who say "This bubble has got to burst sometime". And I don't mean that we should listen to how vociferously they are proclaiming this. We all know people who were saying "this can't go on forever" (some were saying it many years in advance). Instead I mean simply monitor the fraction of ordinary potential house buyers who say "I'm NOT going to buy a house right now because I think that its value will not rise or may even fall by the time I want to sell it". They are the people who will stop the bubble. And the greater the preponderance of them, the sooner the bubble will burst.