One of the highlights of my recent time as Scholar in Residence at the Federal Trade Commission was the opportunity to work with some of the brightest minds around on antitrust issues on investigations and policy projects as well some academic projects. The subject of this post is one of those academic projects. Motivated by the conventional wisdom that the technical demands placed on federal courts in antitrust cases in terms of evaluating expert economic and econometric evidence has increased substantially over the last twenty years or so, Former Bureau of Economics Director (now returned the Kelley School of Business at Indiana University) Mike Baye and I decided to try to take a swing at measuring the empirical effects of economic complexity of judicial decision-making in antitrust litigation. Both “economic complexity”(as opposed to say, legal complexity) and “judicial performance” (in terms of quality of evaluation of economic evidence) are nebulous and difficult to measure concepts. But understanding the impact of economic complexity and economic training on the quality and accuracy of judicial decision-making in antitrust is incredibly important as both a legal and policy matter. Antitrust also seems uniquely suited for such an inquiry since the nearly wholesale integration of economic analysis into antitrust legal standards (e.g. did the merger substantially lessen competition has come to mean something like “are prices going up or down”?) such that most if not all decisions on substantive antitrust issues turn on economic analysis.
While there is now little doubt that complex economic and econometric analyses are standard fare in modern antitrust litigation, but there is a dearth of empirical evidence addressing what impact, if any, this complexity has had on judicial decision-making. An ABA Task Force survey of 42 antitrust economists revealed that only 24 percent believe that judges “usually” understand the economic issues in a case. The ABA Task Force Report and other commentators have suggested a number of possible solutions to the “problem” of economic complexity and expert evidence ranging from increasing use of court appointed experts pursuant to Federal Rule of Civil Procedure 706 (a), expanded use of Daubert to deter unsupported economic testimony, introduction of concurrent evidence procedures, creating specialized courts, and supplying basic economic training to judges.
We undertook to empirically examine these issues directly by evaluating the relationship between economic complexity and appeals in district court decisions reaching substantive antitrust matters from 1996-2006. We also have some unique data that we’ve collected on which federal judges attended the George Mason University Law and Economics Center economics training and examine how that training impacts the appeal rate in economically simple and complex cases. Check out the paper for details on our measure of complexity, a defense and discussion of the appeal rate as a measure of “judicial economic error” holding all else constant, and various other methodological issues. Now seems like a good place and time to thank my team of research assistants that worked tirelessly at putting this database of decisions and coding various aspects of those decisions and information about the judges together over the past few years as well as very helpful comments we received presenting the paper at workshops at UCLA, Stanford, and Texas. I envision the paper as addressing two related research questions: (1) what is the impact of economic complexity on the quality of judicial-decision making in antitrust? and (2) does basic economic training (at the LEC) improve judicial decision-making in antitrust cases?
I also note that readers who have been following the occasional controversy over the George Mason Law and Economics Center, the Feingold-Kyl Amendment, and allegations that “economics training” is some sort of proxy for “right wing conspiracy” to teach “conservative economics” to federal judges might have an interest in the paper as well. Our results suggest that economics training has some real but limited benefits for judicial-decision making and do not support the critics’ accusations. I’ll discuss some results that pertain to this issue below.
Here’s the abstract:
Modern antitrust litigation sometimes involves complex expert economic and econometric analysis. While this boom in the demand for economic analysis and expert testimony has clearly improved the welfare of economists-and schools offering basic economic training to judges-little is known about the empirical effects of economic complexity or judges’ economic training on decision-making in antitrust litigation. We use a unique data set on antitrust litigation in district courts during 1996-2006 to examine whether economic complexity impacts decisions in antitrust cases, and thereby provide a novel test of the frequently asserted hypothesis that antitrust analysis has become too complex for generalist judges. We also examine the impact of one institutional response to economic complexity: basic economic training by judges. We find that decisions involving the evaluation of complex economic evidence are significantly more likely to be appealed, and decisions of judges trained in basic economics are significantly less likely to be appealed than are decisions by their untrained counterparts. Our results are robust to a variety of controls, including the type of case, circuit, and the political party of the judge. Our tentative conclusion, based on a revealed preference argument that views a party’s appeal decision as an indication that the district court got the economics wrong, is that there is support for the hypothesis that some antitrust cases are too complicated for generalist judges.
I discuss some of our results below the fold, but you’ll have to check out the full paper for details.
The evidence here suggests that economic complexity and training influence the appeal rate in opposite directions. Economic complexity significantly increases the probability of appeal, while judicial training reduces it. The effects are of similar magnitude. More specifically, our first finding is that there is a statistically and practically significant positive relationship between economic complexity and appeals. Decisions involving some evaluation of economic or econometric evidence are appealed approximately 9 percent more frequently than cases demanding less economic skill. That a decision was appealed suggests that at least one party is willing make a costly investment for the opportunity to persuade an appellate court that the district court judge erred. This should be more likely in cases involving complex economic evidence because there are likely to be reasonable fact-finding dispute and thus, more room to persuade an appellate court that a reversible error has been committed by the lower court. This result provides some empirical support for the claim that economic complexity is having an important impact on antitrust litigation. Our finding also provides some support for the oft-raised claim that the ever increasing levels of economic sophistication and complexity in modern antitrust litigation are now generating negative marginal returns. While one may reasonably dispute whether this relationship between complexity and appeals is strong evidence of a divergence between the technical demands of contemporary antitrust analysis and the technical economic skills of the federal bench, it is clear that economic complexity is an important part of the modern antitrust litigation landscape.
Our second finding is that the decisions of judges who attended LEC programs to learn basic economic skills are appealed at the same rate as their untrained counterparts in complex cases, but 9 percent less often in cases that do not involve the evaluation of sophisticated economic or econometric evidence. This effect of economic training has interesting implications for understanding the impact of economic complexity in antitrust litigation. The “glass half full” interpretation of these findings is that while adding basic economic skills does not enhance the accuracy of judicial fact-finding in technically complex cases, these skills do allow trained judges to identify and “correctly” decide simple cases which are about 2/3rd or our sample. This interpretation is consistent with the “entry level” content of LEC programs. Exposure to basic economics is not likely to prepare a district court judge to evaluate the type of complex econometric testimony seen in many modern antitrust cases. However, the glass half empty view is that economic training is not likely to produce skills that enable a district court judge to accurately evaluate economic evidence in antitrust matters requiring sophisticated economic analysis.
Third, with respect to the LEC debate, our results do not support the frequently heard allegations that this basic economic training amounts to junkets designed to influence federal judges with a distinctively free-market brand of economics. First, at the most basic level, we are able to control for the political party of the district court judge (measured by the party of the appointing president) and find that ideology does not impact our measure of quality. Second, and more interestingly for the LEC issue, we also examined the possibility that LEC training impacts Republican and Democrat judges differently based on these underlying and unobserved differences by estimating variants of the model that allow differential effects of LEC training on Republican and Democratic judges. The results are similar to those in the basic specifications in both magnitude and direction, i.e. -10.6 percent for Republican judges and -10.1 percent for Democrats. Thus, it does not appear that the reduction in appeals associated with LEC training is an artifact of the ideology of those opting to take such training in the first place. Third, recall that we find a significant impact of economic training in simple cases but not complex cases. This seems completely consistent with an “economics” effect. That is, it is conceivable that basic economic training would help federal judges more efficiently evaluate economic facts in simple cases but not highly complex cases, say, involving econometric evidence. However, it doesn’t make much sense for the selection effect story described by critics. More specifically, the “free market indoctrination” story would seem to apply to both simple and more complex cases and not just the former. Further, we have no evidence that LEC training impacts defendant win rates separately from its impact on the appeal rate.
Finally, we’re working on a few extensions of this project. Antitrust Contests (with Baye and Paul Pautler) is an attempt to structurally estimate a Tullock contest model of antitrust litigation using some unique data on economic labor effort by both sides in FTC litigation while controlling for other important determinants of litigation outcomes. In future research, we plan to tackle the issue of different forms of economic expertise, experience as a substitute for expertise, and the possibility differential performance between federal judges and administrative law judges and/ or Commissioners. Disentangling the experience versus training issue has some important policy implications with respect to the relative merits of specialized courts versus reforms designed to arm the judge with greater economic knowledge through training. Further, comparing the performance of administrative law judges, Commissioners, and federal court judges has important implications for understanding the relevant costs and benefits of deference to agency decisions.
You can download the current paper here.
