The Misuse and Abuse of "Market Failure"

You keep using that word ...

A recurring theme in applied economic analysis is the concept of “market failure”. In casual use the phrase often refers to market outcomes that the speaker doesn’t like. But in technical terms its use in economics refers to specific situations, such as external costs or benefits, or natural monopoly cost structures (as I discussed previously in February).

Economic Foundations: Natural Monopoly Theory II

Both of those situations come up a lot in electricity economics, energy economics, environmental economics, and even the economics of innovation and research. I’m planning to write a few posts about the concept and language of “market failure” and how its use in policy analysis is sometimes misguided and often incorrect.

To set this stage I’m channeling the wisdom and insight of my dear, departed friend and excellent economist Steve Horwitz. Steve’s been gone for two years and I miss him every day. He wrote a great blog post in 2011 that has been lost to the bowels of the internet, but I excerpted it and riffed on it at the time. Below is an edited and revised version.

Steve Horwitz’s column is a great explication of why the phrase “market failure” is so questionable, and so often misused and abused in public policy analysis when employed to criticize market outcomes. He explains the origins of the phrase in the standard textbook case of “perfect competition”: in equilibrium in that simple benchmark model, resources are allocated to their highest-valued use, all Pareto-improving trades have occurred, and while firms have earned inframarginal profit, the marginal profit at the equilibrium level of output is zero. More simply, all gains from trade have been exploited and no one has left any value on the table. Thus, the argument goes, when applying that simplified benchmark model to reality, if we see outcomes that deviate from that and do have some misallocation or unrealized gains from trade, the logical conclusion is that the market has failed to enable agents to achieve that optimal outcome.

Steve highlights two reasons why this conclusion is incorrect. The first reason is a misunderstanding of the nature of competition and the market process as it operates in real conditions of the knowledge problem, imperfect foresight, differentiated products, small numbers of agents, etc. People using “market failure” and making the above critique of markets expect a threshold of unrealistic perfection, and consequently make an unfair comparison of a simplified benchmark model with the complications and nuances of a real-world application.

In his Journal of Law & Economics paper, Information and Efficiency: Another Viewpoint (1969), Harold Demsetz christened this flawed thinking the “Nirvana fallacy”. It’s a fallacy to compare an “ideal norm with an existing ‘imperfect’ institutional arrangement” (1969, p. 1). The ideal is just that, a theoretical benchmark against which to evaluate different market institutions and the outcomes they yield, but the comparison should be across the different institutional arrangements, not all of which should be rejected because they are “imperfect” from this narrow efficiency perspective.

I encounter this argument all the time in electricity regulation, which is predicated precisely on this type of false, over-simplified argument, and has a century’s worth of regulatory institutions built upon the false presumption that achieving such a static outcome in reality is possible.

One thing I particularly like about Steve’s argument is how he points out that these cognitive-epistemological characteristics of the real world are features, not bugs, with respect to how market processes create value and gains from trade:

… these sorts of imperfections (a better term than “failure”) are not only part and parcel of real markets; they also are what drive entrepreneurship and competition to find ways to improve outcomes.  In other words, what markets do best is enable people to spot imperfections and attempt to improve on them, even as those attempts at improvement (whether successful or not) lead to new imperfections.  Once we realize that people aren’t fully informed, that we don’t know what the ideal product should look like, and that we don’t know what the optimal firm size is, we understand that these deviations from the ideal are not failures but opportunities.  The effort to improve market outcomes is the entrepreneurship that lies at the heart of the competitive market.

Thus the value of markets is not that they will achieve perfection, but that they have endogenous processes of discovery that enable people to correct the market’s imperfections.  Just as it’s the very friction of the soles of our shoes on the floor that enable us to walk, it is the imperfections of the market that encourage us to find the new and better ways to do things.

He then counters a second aspect of the “market failure” argument: this argument is typically coupled with a recommendation for some form of government intervention or regulation to “correct” the perceived failure. But if market processes in realistic contexts have imperfections, don’t government intervention and regulation have imperfections too? The relevant comparison is between the results of market institutions and government institutions in realistic contexts, not in simplified blackboard theory. This is precisely Demsetz’s point, as relevant today as when he first made it in 1969.

I would add a third point to this analysis. Often when I encounter the “market failure” argument I reply that “markets don’t fail, they fail to exist”, which is the Coase/transactions cost response. Transactions costs interfere with the ability of parties to find mutually beneficial exchange, thus impeding optimal resource allocation and the creation of maximum gains from trade. Transactions costs lead to missing markets, as in the case of environmental pollution and other common-pool resource situations. This driver of so-called “market failure” complements Steve’s process-oriented argument and reinforces his points … and it implies that one high-priority objective of public policy should be to reduce transactions costs, not to impose regulations that are intended to “correct” market failures but have little realistic hope of doing so effectively.

If we take these critiques of the “market failure” concept seriously, what are the implications for regulation and markets in this time of such vibrant technological change and complicated dominant policy objectives? What should we be doing differently?