Has the empirical question of whether affirmative action harms blacks been answered with a resounding no? Paul Butler thinks so. In a post about UCLA Professor Richard Sander’s next installment of research, Butler writes:
UCLA law professor Richard Sanders has done it again. He wrote an article a few years ago making the claim that affirmative action was responsible for lowering the number of black lawyers. This thesis was persuasively debunked by Blackprof’s Devon Carbado among others. Not to be deterred, Sanders has now advanced the same theory with regard to black partners in law firms.
Perhaps Butler is alluding to Carbado’s excellent post, which attempts to sharpen the empirical debate by asking what data would be sufficient to establish that affirmative action harms blacks and suggesting that such data must include information regarding “where blacks are in society as a result of affirmative [action].” Perhaps not. Carbado’s post doesn’t appear to even attempt such a “debunking.” So where does Sander’s empirical result stand? Carbado’s post warns: “Of course, there is no way to give proper treatment to this issue in this forum; it’s hard to have an empirical discussion on a blog.” But a discussion of the empirical issues here seems worthwhile, and I am not convinced this medium renders such a discussion impossible. I am willing to bet the ELS Bloggers agree with me that the blog medium does not prevent such a discussion of the empirical evidence.
Like much of the legal academy, I have read Sander’s work, the criticisms (57:6 Stan. L. Rev.), Sander’s replies (e.g., here and here), and even a sur-reply. I also found this useful Legal Affairs Debate Club exchange between Sander and ELS Blog’s Bill Henderson. While I have read most of the literature at some point or another, I admit that I have only casually followed the empirical exchange between Sanders and his various critics. That caveat aside, Butler’s claim that Sander’s results have been “persuasively debunked” struck me as either: (1) premature, (2) wrong, or (3) a sign that I have really not been following this debate.
I’m not making any normative statements here. For that matter, I’m not taking a position on the empirics without doing some more homework. This is, however, one of the most followed empirical debates the legal academy has seen recently (perhaps only second to this) and so I thought it worthwhile to dig a little bit deeper here.
My relatively uninformed impression of the debate was that Sander’s fundamental result, that affirmative action reduced the number of black lawyers, had not been “debunked.” I had assumed that either no consensus had been reached and the question was still “live,” or the literature had moved on to more precisely identifying the causal mechanisms at work (is it mismatch or something else?) but had at least on some scale accepted the result. Such a state of play, of course, would not be abnormal for a relatively young empirical debate. As such, my hunch was that Butler’s description was probably inaccurate on the merits. Does anybody following the empirical exchange more closely than I have a sense of whether a consensus has in fact been reached in either direction? If so, what is it? Is the perception that Sander’s results have been debunked? If so, what is it that he has identified? If there is no consensus, what are the open questions?
I am really looking for empirical answers from folks that have read the literature and are familiar with the evidence rather than more general views on affirmative action. ELS guys? Bill?
NOTE: For those interested in a shorter version of at least Sander’s views and his responses to critics, his posts at the Volokh Conspiracy are a nice place to start.
RSS - Posts
Thanks for this thoughtful response, Rick. As I note in my post, I have not been keeping up with the literature in this area and I am grateful that you chimed in here with your take on the state of play here. Like you, my hope is that increased attention to the empirics will generate a fruitful and relevant policy debate centered around the actual impact of affirmative action. The post is an attempt to further that purpose — one I’m sure you will agree is a worthy one.
As for this:
“If you don’t feel competent to judge this exchange yourself, just give key parts to your favorite econometrician friend and ask for his or her views.”
Perhaps I am completely misreading this sentence to imply something about my own personal competence. If not, I assure you that lack of competence was not a motivating factor for asking the question about the state of play of the literature. I feel fairly confident in my abilities to both read and do econometrics — though not too confident to ask others econometricians and statistically oriented scholars for their “take”. If not, well… no harm no foul.
My own competence aside, I think folks should take very seriously your suggestion that scholars read the relevant literature before passing judgment or relying on subsequent results communicated to broad audiences.
The point here was to ask the legal community following the debate if there was, in fact, the type of consensus view that you describe. As you can see from the comment thread that includes just you and I … well, the experiment didn’t go too far (though it did provoke Bill’s excellent post on ELS Blog and your comment there as well — which are both worth reading).
I have been an active participant in the debate with Sander and an active debunker and so hardly qualify as a neutral observer. But I am left with the question of what does it take to debunk someone who confidently asserts as “fact” conclusions that are poorly supported or unsupported by data and analyses. Specifically Sander’s analyses in his original Stanford piece have been reexamined using Sander’s data by numbers of scholars at various law schools (e.g.,Ian Ayres & Richard Brooks, Daniel Ho, Joshua Bernstein & Albert Yoon, Kathy Barnes plus our group (David Chambers, Bill Kidder and Tim Clydesfale) in published and soon to be published work. Others like Michelle Landis and David Wilkins have criticized the work without reanalyzing the data. Not one of these authors or author teams finds the data properly analyzed or agrees with Sander’s conclusions. Serious and obvious errors have been pointed out in Sander’s analyses (e.g. misstatements of what statistical signifiance tells us, misapplication of Linda Wightman’s grid model,) which Sander in his various replies has not responded to. Similarly a likely coding error in a crucial table which Jim Lindgren called to Sander’s attention before his first article was published is only noted in a footnote, and the degree to which correcting for it undercuts his model is not noted. To my knowledge there is not one piece published or unpublished that has reexamined Sander’s data and supports his conclusion that affirmative action so hurts black students that it actually diminishes the production of black attorneys and all the analyses, including Sander’s own though he never acknowledges it, indicate that blacks who get into the nation’s elite schools through affirmative action are not hurt and/or likely benefited.
The very fact that serious legal scholars can ask in good faith whether the work has been refuted suggests there are deep flaws in how social science evidence is published in law reviews and later results communicated to readers.
But as I noted I am identified now as a partisan in this debate. So let me suggest that those still unpersuaded on the issue read the articles, replies and critiques, including the reply I and my coauthors wrote to Sander’s Stanford reply (as weak as the original analysis despite seemingly persuasive rhetoric, and, if properly understood, a repudiation of much of the original analysis)which can be found at: http://www.equaljusticesociety.org/research.html. If you don’t feel competent to judge this exchange yourself, just give key parts to your favorite econometrician friend and ask for his or her views.
Rick Lempert