Angrist on Levitt, Instrumental Variables, Identification, and Defending Cuteness

Josh Wright —  25 April 2007

It is apparently vogue to ask whether Steve Levitt is ruining economics? The serious question behind all of this is whether today’s economists have become too enamored with cute and clever questions rather than the day’s “big” questions.  I’ve already gone on the record on this one with a comment to a post over at Co-Op by Frank Pasquale:

Many people conducting “cute” research are quite aware of the details of legal and other institutions and in fact often rely on these details to construct instruments for their econometric analysis.  I am sympathetic to the view that the economics profession as a whole might be better off with greater attention paid to larger problems. But for what it is worth, do not agree with the claim that Freakonomics is in the least bit responsible for this trend, and disagree strongly with the proposition that Levitt or Freakonomics has made economics worse off.

Greg Mankiw offers a bit of a luke-warm answer to the question, noting that it is “disconcerting, to a degree” that “young economists today are doing Levitt-style economics and fewer are studying the classic questions of economic policy,” but concludes that he is not too worried about it because economics will correct itself because of diminishing returns to call research programs and strong incentives to address the “big ones.”

I prefer Joshua Angrist’s less apologetic response to Scheiber’s attack defending instrumental variables, natural experiments, and even cuteness, in the TNR (HT: Marginal Revolution):

I’ll plead guilty, however, to being especially pleased when students manage to come up with clean identification–that is, they have a convincing strategy for uncovering causal effects.  Clean identification is not a fetish; without it, little of value is learned.  On this score, our students typically do better than the empiricists of H.G. Lewis’s generation … .
Second, in his rush to tar some up-and-comers with the “cute-o-nomics” brush, Scheiber misses a central feature of the clean-identification research agenda, best explained by example.  One of the enduring scientific and policy questions in Labor economics is the sensitivity of hours worked to changes in pay (this matters for tax policy, for example).  The best evidence labor economists have on the relation between wages and hours worked comes from a small experiment (by Ernst Fehr and Lorenz Goette) involving the wages of bicycle messengers in Switzerland.  The second best comes from a study of stadium vendors by Gerald Oettinger.  Who cares about the riders of Veloblitz or snack sellers at Camden Yards?  We care because economics is predicated on the notion that a few simple principles explain behavior in many settings.  These studies produce results that are convincing and may well be general, though, as always in science, it will take replication to know for sure.

I like that answer alot.  While I’ve written here before about the potential harms of fetishizing formal mathematical theory rather, especially with respect to law and economics, I see that as a separate problem altogether.  Economists need not apologize for uncovering clean causal relationships  in less complex settings.   There is no shame in adding to our theoretical and empirical understanding of small subjects with the hopes of generalizing to larger phenomena.

For that matter, though I’m a little late to the party, academic economists dont need to be defensive about playing parlor games from time to time either (If the bar for academic work is immediate policy relevance, how much law review scholarship do you think would qualify?).

6 responses to Angrist on Levitt, Instrumental Variables, Identification, and Defending Cuteness

  1. 

    Regarding Kate Litvak’s comment, why do we need to assume we can find, or must have, patterns? Why is it so important to generalize, particularly if it may be true that efforts to generalize are not generally possible, perhaps because human nature is not irreducible? Maybe it is in the ungeneralizable particulars that the truth is to be found? Is Levitt’s study of abortion rates and crime rates simply cute, or is it also elegant and clever? Admittedly, this is the comment of someone who finds more truth in stories than in data (even though my day job exposes me to an unending river of data).

  2. 

    The best evidence labor economists have on the relation between wages and hours worked comes from a small experiment (by Ernst Fehr and Lorenz Goette) involving the wages of bicycle messengers in Switzerland. The second best comes from a study of stadium vendors by Gerald Oettinger. Who cares about the riders of Veloblitz or snack sellers at Camden Yards? We care because economics is predicated on the notion that a few simple principles explain behavior in many settings.

    And why do we think that the data on bicycle messengers in Switzerland tell us anything useful about lawyers in New York or drill sergeants in Iraq or call center employees in Bangalore? Because of the irreducible human nature? Because we assume so? Granted, if we had lots and lots of papers in different representative settings, with similar results, we may have informally concluded that there is a pattern. But the problem is that these studies are rare (per issue) and not representative. And they cannot be made systematic or representative because the settings are selected for the sole reason of involving a cute instrument (the availability of which may or may not be correlated with the factors that we study). The generalizability of those findings is highly questionable.

  3. 

    I would echo Marc’s comment. “Cute-o-nomics” is the real and sophisticated (just because the ignorant call it “cute” doesn’t mean it isn’t rigorous and complex) application of a mode of inquiry, of a system of analysis, to real life problems. These are problems the solution to which, one hopes, will illuminate the human condition.Here’s why I think a lot of detractors actually detract: The underlying principle of this work is that “people respond to incentives” and they do so in fundamentally similar ways. It is the application of basic, classical intuitions to the sort of problems many like to point to as transformative exceptions to neoclassical analysis. Frank Pasquale loves to construct unreal hypotheticals to demonstrate a set of facts where traditional economic intuition yields the “wrong” result. But the more these sorts of quirky situations are shown to be “solved” by classical economic intuition creatively applied, the less force there is in the arguments that the entire edifice should be scrapped.Not entirely unrelated is the fact that this mode of analysis is real economics in the purest sense. The folks like Levitt who do this work, who can perceive problems and think through creative solutions to them, are applying real economic intutition. They may use a lot of math, to be sure, but the underlying logic is generally quite simple (not as in “easy” but as in “not-complicated”) and complex mathematics is not necessary to explain or to understand what’s going on. It is economics in the style of Tullock and Alchian and Coase. These are real lessons learned through application of a powerful system of analysis. In contrast, too much of economics today seems to be little more than mathematic gymnastics. It is largely devoid of real fundamental understanding of human behavior and of the analytic power of a few simple rules to explain it. The debates don’t turn on seemingly fundamental questions like, “how well does this explain observed behavior?” Rather the debates are about the elegance of complex models and the proper use of this variable or that equation. Important endeavors, to be sure, but hardly deserving of more accolades than the work of incisive “natural” economists.

  4. 

    I agree that the critique of clean ID is misplaced. I would go a step further and say it’s way off base. The idea that economics would be better off sticking to “big problems” of inflation, unemployment, poverty, economic growth, regulation, political economy, etc. is actually a reflection of the hubris of academics who really want to be “big-issue” policy-makers.

    Here’s a check their egos: Everyone can’t come up with the Theory of Relativity. Someone needs to actually measure the warping of a specific wavelength of light in the gravitational field of a specific star.

    IMHO, one of the reasons economics is a “dismal science” is that the big questions have been so controversial, and their big answers have been so unconvincing. They’re unconvincing because they are trying to attack entrenched opinion with messy data and inherently weak controls. How many times have we heard that, “Sure, minimum wages may, in theory, reduce overall employment, but this study in New Jersey showed it didn’t?”

    Until we develop a hundred “cute” studies showing that that mainstream economics works in ‘this’ and ‘this’ and ‘this’ and ‘this’ and ‘this’ situation, even if it doesn’t work as well in ‘that’ one, we will be hearing about ‘that’ in New York Times editorials arguing for more enlightened intervention.

    I think Levitt and his clones are doing God’s work.

Trackbacks and Pingbacks:

  1. TRUTH ON THE MARKET » How Steve Levitt Might Save Economics - April 26, 2007

    […] Geoffrey Manne on Angrist on Levitt, Instrumental Variables, Identification, and Defending Cuteness.M. Hodak on Angrist on Levitt, Instrumental Variables, Identification, and Defending Cuteness.Puzzles and Problems Redux « Organizations and Markets on Angrist on Levitt, Instrumental Variables, Identification, and Defending Cuteness.Michael Guttentag on Congratulations to Kate Litvak!.The Harvard Law School Corporate Governance Blog » An Inconvenient Location for the Annual Shareholders’ Meeting on Halliburton Annual Shareholders’ Meeting Location.M. Hodak on North Dakota Publicly Traded Corporation Act.Jerry Kohl on A New Defense of the Per Se Prohibition Against RPM?.D. Daniel Sokol on A New Defense of the Per Se Prohibition Against RPM?.TRUTH ON THE MARKET » A New Defense of the Per Se Prohibition Against RPM? on A Response to Commissioner Harbour’s “Open Letter” on Leegin.Heidi Feinstein on Rent a CFO?. […]

  2. Puzzles and Problems Redux « Organizations and Markets - April 26, 2007

    […] from Levitt (very unhappy), Joshua Angrist (also unhappy, though not for personal reasons), Josh Wright (mildly peeved), and Greg Mankiw (neutral). Sadly, no one has picked up my (very clever) reference […]