Failure Part I - Market Failure

In economics, there are several competing descriptions of "efficiency". In Econ 101, you will usually be shown how market transactions can produce one or more of these efficient outcomes. However, that demonstration is usually conducted under ideal circumstances of atomistic competition and other constraints. Later on in your academic career, you relax those constraints and find out how the real world interferes with the nice, neat perfect market.

Things which interfere with market transactions are known as "market failures". A market failure is not the failure of the market to produce the thing you desire, like a $500 Ferrari. Rather, it is the failure of the market to produce the efficient outcome. For example, if you would be willing to purchase something for no more than $500, and someone would sell it to you for no less than $250, any transaction between $250 and $500 would be efficient. If something interferes, the market is said to fail. Market failures include (in broad categories):

• Structure (increasing economy of scale or scope, path dependency, oligopoly, monopoly) - problems in the structure of the market in which one or a few firms may come to dominate. When that occurs and oligopoly or monopoly obtain, they may attempt to restrict output artificially and thereby raise prices.
  
• Opportunism/Knowledge (asymmetric knowledge, adverse selection, moral hazard, principal-agent, specialized assets, measurement) -- problems in the distribution of knowledge where one side of a trade knows more about a key aspect than the other. Used car owners know whether or not their vehicle is a good car or a lemon, but potential buyers don't and therefore assume average quality. Owners of superior used cars will refuse to sell at average prices, so the market becomes dominated by lemons (canonical example of asymmetric information). Similarly, only unhealthy elderly consumers will be willing to buy health insurance (adverse selection). If an insurer allows an asset owner to insure the asset for far more than it is worth, he runs the risk that the insured will fail to exercise proper prevention strategies and therefore be in the position of collecting on the insurance (moral hazard). A principal who hires an agent is dependent on the agent's judgment of what constitutes due diligence. A company that invests in specialized equipment (e.g. dies) to make products for a particular buyer runs the risk that the buyer will attempt to expropriate the entire value of the investment with the threat of leaving him in the lurch. When a manufacturer sells through complex distribution channels, he is frequently unable to monitor whether customers are given adequate support at the retail level.
  
• Public good (free rider, externality, holdout, commons tragedy, merit good) -- If I broadcast a radio signal to one customer, it is impossible to prevent others from receiving that same signal. Therefore, no single customer would pay for it knowing that others will free ride. On the other hand, if I agree to broadcast only if everyone agrees to pay equally, a single holdout can scuttle the entire deal. Similarly, if I agree to provide electricity to a single customer using a coal-fired plant, I cannot help but create pollution that will affect everyone (public bad). If we all decide to use a common resource, none of us can capture the rewards of conservation, but all of us can capture the rewards of increasing our use until it collapses. Some goods are thought to have externalities so valuable that they should be provided even if nobody would buy it themselves (e.g. education, vaccinations against communicable disease).
  

The existence of such failures is often considered to be the entry point for government action. This "Theory of Second Best" has been around for a while but is usually associated with both paleoprogressives (roughly 1890-1920) and neoprogressives (roughly 1990-present). The "second best" solutions open to the government are the Pigovian tax (including negative tax for merit goods) and simple public ordering (regulation). However, this is exactly the kind of problem I suggested a couple of essays ago when I stated that our choices are not between imperfect markets and perfect government policies. There are two problems here: the first is that the policies themselves may give rise to new problems, and second that the market itself may produce solutions.

A number of private solutions are available for each of the problems identified above.

• Structural problems may be countered with substitution and/or innovation. Gort and Argawal ("First-Mover Advantage and the Speed of Competitive Entry, 1887-1986", JLE vol XIV, April 2001) found that "first to the market" status conferred a positive advantage, but that the length of that advantage had declined from 30 years in 1887-1906 to about 4.5 years in 1986. In other words, the rate of change is accelerating and the advantage of the "first to market" is declining. The danger of monopoly was never as great as it was made out to be, and becomes less important every year.
• Another solution to monopoly problems is to remove public policies that create the condition in the first place. These include patents, subsidies, and legal monopolies. Many if not most historical monopoly problems were the result of regulatory capture by so-called "natural" monopolies that were anything but natural. AT&T comes to mind. Southwest Airlines is still hampered by the Wright Amendment. The USPS still has exclusive rights to carry First Class Mail.
  
• Opportunism can be countered with contractual remedies. For example, if you see that investing money in a very specialized asset for a single customer is going to open you to a situation where they can take advantage of you by threatening to abandon you after you have made the investment, you can demand a hostage or deposit.
• You can counter knowledge asymmetry with information gathering, as Akerlof even suggested in "The market for lemons" (though his suggested private remedies are usually overlooked). These include using branding and/or franchising to create or leverage existing reputation, using guarantees, or using third-party reputation sources such as licensing and certification providers. Underwriters Labs and Orthodox Union are two of the better known 3rd party certifiers, though FairTrade and others serve the same purpose. Note that these remedies are not necessary when repeated transactions within a local market are common. I used Consumers Union and Carfax when purchasing my most recent used vehicle. Hmmm - apparently, information gathering costs money, and if someone can gather it more efficiently than you, you can both profit by it.
  
• Contractual remedies are also applicable to asymmetric knowledge problems. If you think you might have a difficult time monitoring retailers of your product, you can try bundling, setting price floors, using standard contracts (common with car rental agencies), and franchising rules that prevent quality "shading" by franchisees.
• Finally, even public good problems are not entirely insurmountable. Radio is operated by bundling the public good (music, news, etc.) with public bads (commercials). Free rider and holdouts can be captured in mutual benefit contracts (covenants, for example). Commons can be divided up and parceled out with the equivalent of barbed wire and property rights assignments (such as auctioning off the RF spectrum).

The last example may benefit from a hybrid solution of public and private remedies. By assigning property rights in an application of the Coase theorem via legislative fiat, the commons problem is removed. Pigovian taxes are strictly ruled out by this (see David Friedman, Law's Order, on problems with joining Coase and Pigou), but some minor regulation may still be beneficial. For example, cap & trade markets in sulfur & carbon are generally accepted as the proper way to address the commons problem that results in pollution.

In this respect, I am not averse to public solutions so long as it can be shown that they are better than any possible private solution. That being a nearly impossible task (because of the "any possible" requirement on something as dynamic as our sophisticated economy), I would rather see state action that attempts to align incentives and allow the market to work without dictating the outcome; well-designed cap & trade markets are preferred to Pigovian taxes which are preferred to command & control regulations, so give me the sulfur market over a carbon tax over a new CAFE rule. I would also like to see such state interference written in such a way that it is subject to review and renewal frequently. Regulations are frequently written in a toxic mix of public hysteria, rent-seeking by corporate lobbyists, and power-seeking by legislators and bureaucrats, then set in stone and milked for 3-4 generations the way railroad, trucking, airline, and natural gas regulations were.

Heck, we just recently paid off the Spanish-American War.

UPDATE: This has a follow-up, Failure Part II, Government Failure.