An interesting post on the University of Pennsylvania Reg Blog from Michael Abramowicz, Ian Ayres, and Yair Listokin (AAY) on “Randomizing Regulation,” based upon their piece in the U Penn L. Rev.
If legislators disagree about the efficacy of a proposed policy, why not resolve the disagreement with a bet? One approach would be to impose one policy approach randomly on some members of the population, but not on others, to determine whether the policy meets its goals. This solution would overcome the measurement problems of conventional regression analysis and would provide a useful way to compare regulations and promote bipartisan agreement. Legislators might agree that once such a test is complete, the winning approach would apply to everyone.
For example, regulators could test the Sarbanes-Oxley Act’s most controversial provisions, such as those requiring public companies to institute internal controls and then to have their CEOs and CFOs certify their financial statements, by randomly repealing one or more of those provisions for some corporations for some period of time. Randomization would enable analysts to determine which regulatory regime is optimal by assessing which test-group of corporations has the highest level of success, whether measured by stock price, investor confidence in financial reporting, lack of fraud, or other yardsticks.
Conventional statistical and econometric analytical techniques are often used to measure the efficacy of statutes and regulations, but they face problems that randomized trials would not. Researchers may purposefully or mistakenly omit variables from their regression analyses, leading to incorrect results. Publishers are more likely to feature work that provides statistically significant results, even if those results are not correct, a phenomenon known as publication bias.
No doubt many economists and empiricists are nodding their heads in agreement and drooling at the opportunity to more accurately identify and measure the effects of regulation. Randomization would allow application of techniques far superior to what is typically used. AAY discuss some of the common critiques of randomization in the blog post, and at greater length in the paper. The longer version is worth reading, but here is the short version from the blog post:
Ethical concerns are important, but may not present a significant barrier to using randomized tests. While legal randomized tests would lack the informed consent provided in medical experiments, the government regularly imposes regulations on the public – within constitutional and other legal bounds. Also, randomization sometimes makes the imposition more equal than regulation imposed using predetermined criteria. We tend to think it is worse to impose rules on people because the selected people are unpopular rather than simply because they were selected randomly.How should randomized trials work? The experiments should be large enough to produce meaningful results. The test groups, meanwhile, should be the smallest possible without changing the results outside those test groups. For example, driving speed limits cannot be randomized at the individual level because such a test group size would significantly increase the risk of accidents. However, the test group could be at the county level.Experiments should also be of sufficiently long durations to prevent test subjects from changing their behavior temporarily for the duration of the experiment. For example, if different income tax levels are imposed on different people to see if imposing a higher income tax reduces work output, an experiment of short duration would be more likely to be biased. Workers could wait out a temporary increase in income tax level by temporarily working less, and plan to work more once their income tax level decreases.There is no problem, under current standards of judicial review, with administrative agencies testing out different regulations on their own. Agencies could put their proposed experimental regulations through the regular notice and comment process. After running the experiment, the agencies could provide a randomization impact statement explaining why the agency decided to test regulations through that process, describing the experiment, and providing its results. Because randomization provides for more objective analysis of policy results, courts should be more deferential in conducting hard look review to agencies that have selected policies through this approach.
RSS - Posts
Didn’t the US already have a means to determine if regulations were useful in the form of State governments? And didn’t they already ignore the results of this process when they adopted Obamacare, the major elements of which had all failed at State level?
It amazes me that academics and statists excitedly propose further manipulation of the populace and of productive people by imposing random “experiments”.
Instead of bullying participants at random, why not embrace a free market, where companies could adopt Sarbanes-Oxley ideas as they wish. The market price of their shares would reveal if stock investors cared about meeting those provisions. Really, how would the researchers do any better?
No one mentions any more the restrictions long in effect on “hostile takeovers”. Let groups of wise people buy out those companies which are mismanaged. If they made big profits, it would be because there was prior mismanagement. Lack of hostile takeovers has entrenched managements and stifled transparency. This lack is a direct result and manipulation of government, at the request of current managements.
Government has thwarted the operation of a market in corporate control. Then, these academics propose coercive experiments to allow central measurement and planning of business, to correct the resulting problems. The aim is to run society in an efficient manner without the disturbing outbreak of profits. Where has this ever been successful?
Andrew: thanks for the comment. Note that the “experiments” could run in all directions. E.g., the experiment could be to do away with some regulation in randomized areas so as to learn the effects of deregulation where one would not get it otherwise. If the relevant counterfactual is no regulation, I can see lots of reasons to object to randomized regulation — as you point out. In some cases, its pretty clear that is not the relevant counterfactual, i.e. without randomization the next most likely alternative is blanket federal regulation or some such. Small experiments to learn those effects could be useful in preventing larger scale regulation — or in learning how to best deregulate, etc. In any event, the idea of firms opting in is discussed in the paper as well.