Do Expert Agencies Outperform Generalist Judges? Some Preliminary Evidence from the Federal Trade Commission

Josh Wright —  6 February 2012

I’ve posted a new project in progress (co-authored with Angela Diveley) to SSRN.  In “Do Expert Agencies Outperform Generalist Judges?”, we attempt to examine the relative performance FTC Commissioners and generalist Article III federal court judges in antitrust cases and find some evidence undermining the oft-invoked assumption that Commission expertise leads to superior performance in adjudicatory decision-making.  Here is the abstract:

In the context of U.S. antitrust law, many commentators have recently called for an expansion of the Federal Trade Commission’s adjudicatory decision-making authority pursuant to Section 5 of the FTC Act, increased rulemaking, and carving out exceptions for the agency from increased burdens of production facing private plaintiffs. These claims are often expressly grounded in the assertion that expert agencies generate higher quality decisions than federal district court judges. We call this assertion the expertise hypothesis and attempt to test it. The relevant question is whether the expert inputs available to generalist federal district court judges translate to higher quality outputs and better performance than the Commission produces in its role as an adjudicatory decision-maker. While many appear to assume agencies have courts beat on this margin, to our knowledge, this oft-cited reason to increase the discretion of agencies and the deference afforded them by reviewing courts is void of empirical support. Contrary to the expertise hypothesis, we find evidence suggesting the Commission does not perform as well as generalist judges in its adjudicatory antitrust decision-making role. Furthermore, while the available evidence is more limited, there is no clear evidence the Commission adds significant incremental value to the ALJ decisions it reviews. In light of these findings, we conclude there is little empirical basis for the various proposals to expand agency authority and deference to agency decisions. More generally, our results highlight the need for research on the relationship between institutional design and agency expertise in the antitrust context.

We are in the progress of expanding the analysis and, as always, comments welcome here or at my email address on the sidebar.

5 responses to Do Expert Agencies Outperform Generalist Judges? Some Preliminary Evidence from the Federal Trade Commission

  1. 

    Hi Josh,

    I’m interested in this topic too. I used to believe the hypothesis until I heard a comment by Judge Diane Wood at an ABA Antitrust Section panel. She pointed out that if the antitrust laws are supposed to be useful guides for business practices, their rules should be comprehensible to ordinary business persons. One way to promote that goal would be to force litigants to make antitrust cases comprehensible to generalist judges. It may be especially useful to have this as a counterweight to the tendency to make cases relatively incomprehensible, since doing so drives up demand for specialist lawyers and economic experts to help with the rent-seeking,

    I do have one quibble, which is using appeals as a measure. If an appeal is Trial 2.0, I’d agree with you. However, at least ideally, appeals are supposed to be about errors of law, not fact, implying that one could easily observe correct factual analyses overturned and factually incorrect analyses not appealed. Of course, an advantage of appeals is that the alternative, whether the court or the FTC is more likely to get it “right,” may get invariably tied up with the controversy over what the right answer is.

    I’m curious what you think of the Chevron doctrine regarding judicial deference to regulatory expertise outside antitrust.

    Thanks,

    Tim

    • 

      Hi Tim,

      Thanks for the comment. Rather than go through great lengths to discuss appeal as a measure of quality or likelihood of “economic error” in antitrust cases here, let me refer you to the Baye & Wright (2011) piece around pages 10-12. Short version: when the issue of “law” is whether a merger substantially lessens competition under Clayton Act Section 7, and that is interpreted as being directly related to the likely competitive effects of the merger, the distinction disappears.

  2. 

    This is an interesting topic. I am not sure who your intended audience is, but it seems to me that the thesis of your paper accepts the “expert hypothesis” and just shows that it has little empirical support in the Antitrust context. A more interesting idea, it seems to me, would be to build on the literature that has critically challenged the “expert hypothesis” generally. See, for example, Tetlock, “Second Thoughts About Expert Political Judgment” 22 Critical Review 4, 486 (2010). Check this:

    “Expert judgment often falls short of [the] basic epistemic ideals of empirical accuracy and logical rigor, and … the resulting errors can be extremely expensive.”

    There was a symposium on this topic in the Critical Review issue I cited above. Very interesting stuff. Imagine, you are a Professor with a Phd in Economics and a JD in law. You are employed by the FTC and paid big bucks to examine anticompetitve trade practices. People listen to you and rely on your judgment. Then it comes to light that your predictions have ZERO emprical accuracy. What happens then? What should happen?

    Read the article to find out!

    • 

      MGM: Thanks much for the comments.

      The paper is designed to test the expert hypothesis — one that is taken very seriously in and around the antitrust world — and evaluate it empirically. In that sense, of course, we “accept” it for the purposes of testing. I’ve written a great deal about the perils of prediction in antitrust markets; while I think the tie-in to the expert political judgment literature is interesting, and I thank you for bringing it to my attention, I think there is a distinction here worth pointing out. One of the things that makes the FTC interesting in this context is that in addition to the Commissioners, there is no real dispute that the FTC has a top notch band of industrial organization economists at its disposal in the Bureau of Economics. This, I think, adds a nice layer of complexity to the problem. Do the agencies use the economic expertise? How so relative to federal judges who don’t have access to the same economic expertise inputs (see Baye & Wright, 2011)? In other words, I think the tie-in to the literature you cite is that it is an interesting question *whether* the agency decision-makers are merely expert political judges in this context.

      Don’t get me wrong, I understand there is a separate and interesting question about whether the predictions of the expert economists are better. But given current antitrust law, the role of economic analysis therein, and the economic methods used by the FTC economists, I think a more interesting framing of the problem is whether and how the Commission uses these inputs. I am also working on a project that would expand this across agencies in different settings.

      Lastly, as to your hypothetical, I think the distinction above gets directly at it. The predictions being tested here are those of the politically appointed decision-makers and not of economists and evidence that the Commission does not outperform generalist judges is not appropriate to use to test the hypothesis that the economic experts’ predictions do not have predictive value. There is a long literature on the latter; but no need to reprise here. But suffice it to say they are different enough problems to warrant different empirical tests.

  3. 

    The FTC is expert in what? I think the correct wording of your question shold be: “Do Recently Appointed Political Hacks Outperform Senile Political Hacks?