Opinio Juris

First, I would like to thank David Zaring again for his comments on my essay: Empiricism and International Law: Insights for Investment Treaty Dispute Resolution. I was deeply humbled to read David’s thoughts about both the utility of the essay and the direction of my current research. David’s comments, however, raise a variety of issues worthy of a bit deeper exploration.

I wholeheartedly agree that empirical work in international law is decidedly not déclassé. As a proud alumna of the University of Minnesota Law School, I had the privilege of taking classes with Bob Hudec. I have profound respect for Hudec’s empirical exploration of international trade dispute settlement system and have been grateful to see others continue the empirical exploration of international economic law in the trade context. Perhaps more selfishly, as my research has developed, I have found myself wishing that Hudec was still with us so that I could benefit from his methodological insights related to the analysis of international economic dispute resolution. I would nevertheless hope that Hudec would have been pleased the development of empirical methodologies to areas of international law outside of the trade context. The work of junior scholars such as Oona Hathaway, Elena Baylis, Bill Burke-White, and Melissa Waters spring immediately to mind although there are certainly others engaging in research. In the meantime, I will look forward to hearing the remarks of David Trubek and my co-panelist Juscelino Colares at the forthcoming Society of International Economic Law as they both have critical observations about Hudec’s legacy to empirical assessment of international economic law phenomena.

David also makes an interesting point about how research in international investment law might evolve in the future. I am sympathetic to the reference to Wright & Miller’s recent exploration of content analyses and the value of using rigorous social science methodology to study the content of judicial decisions. No doubt this literature, and its underlying methodological rigor, will aid the evolution of methodological approaches involving the analysis of investment treaty arbitration awards. As the essay also suggests, there are other critical methodological approaches that could be likewise as this area evolves. For example, provided that scholars are sufficiently transparent in the description and analysis of their quantitative research, meta-analysis may be possible. As discussed eloquently in Jeremy Blumenthal’s article Meta-Analysis: A Primer for Legal Scholars, meta-analysis can synthesize empirical analysis across studies in order to summarize the research and identify variables influencing the findings of particular research. These additional methodological approaches only scratch the surface of potential ways to develop international law empiricism. One might even imagine – much like social science counterparts – the development of research methodologists, who are dedicated to the exploration and improvement of empirical methodologies, as a type of international law sub-specialty. But my suspicion is these last two evolutionary advances may be years in the offing.

Finally, in providing the cautionary observation about the need for training, David observes that inter-disciplinary collaboration can help bridge the methodological divide by providing much needed skills in this regard. I decidedly agree with David about the value that collaboration brings on the methodological side and offer two counter-points.

First, there are also practical benefits to be garnered from collaboration beyond methodological insights. Chief among these benefits is: sleep. Empirical research, while rewarding, can take time. The development of datasets can be labor intensive – even with the help of able research assistants (and I have been profoundly blessed in that department with the assistance of Melanie Neely and Jenna Perkins). Working in collaboration with others means that some of the most laborious aspects of empirical research – namely data collection – can be shared. In other words, it means that you are less likely to be sleep deprived and able to work more effectively. (And yes, there is empirical literature in related contexts to back up this claim.) Sharing of such tasks may also mean, provided proper research protocols related to inter-coder reliability are followed, that the reliability of data collection may be enhanced. It also means that research methodology choices can be considered with a view to considering multiple perspectives; and as none of us is perfect, the use of group-think to develop research and analysis can be invaluable. Collaboration also creates research efficiencies. For example, those with an expertise in or aptitude for the creation of graphs, tables and charts can develop them readily. I may, however, say this given my own graduate coursework at the University of Nebraska Law and Psychology JD/PhD program with Cal Garbin on multivariate research design and data analysis (see here and here) and the learning curve I have experienced in the creation of graphs during my work for Cal this summer.

The second counterpoint suggests that there may be a gap to fill within legal education. More particularly, while there are benefits to collaborating in an inter-disciplinary manner, one wonders whether such collaboration may be even more fruitful if legal scholars had access – for example in law school – to methodological classes to provide basic training. There are certainly useful programs for professors such as the Northwestern/WashU “bootcamp” or programs for quantitative methods at the University of Michigan, University of Essex or the European Consortium for Political Research’s program at the University of Ljubljana. Nevertheless, the essay explores the unique benefit of systematically providing the next generation of research assistants, lawyers and law professors with training in the law school context. Some law schools, such as Berkeley, Cornell, Harvard, Illinois, Leiden University, Northwestern, Penn, Stanford, University of Chicago, Vanderbilt, Washington University, and Yale have classes related to the empirical methods and the law (and apologies for the lack of a complete list in this regard for other law schools with separate courses focused on empirical methods). A casebook with an accessible teacher’s manual, such as the one being developed by faculty at the University of Illinois, goes a long way to filling this particular gap in U.S. legal training. I understand Empirical Methods in Law (Aspen, forthcoming), written by Bob Lawless, Jennifer Robbennolt and Tom Ulen, should be available for Fall 2009; and based upon the draft chapters I have seen, I am looking forward to getting my copy.

Ultimately, if we are willing to take on the challenge, we are at the first step of a journey of empirical assessment of investment treaty dispute resolution. There are inevitably places where we can grow and develop in collaboration or consultation with others. And that, at least in my view, is certainly a worthy undertaking.

[David Zaring is Assistant Professor of Legal Studies and Business Ethics, Wharton School, University of Pennsylvania]

Susan Franck’s essay makes the case for more empirical research in international economic law; a project that she has pursued – I might venture to say that she owns an important part of the field, given the unique and comprehensive data on investment arbitrations that she has collected and continues to analyze – in her other work. It is worth noting both how precise and how important Susan’s recommendations are. First, when I think about empirical research in international economic relations more generally (and here I should clear my throat apologetically – I’m not an empirical scholar except in a rather mild sense, but primarily a reader of some of that work), I think of the big picture claims made by some applied economists – Andrew Rose’s claim that membership in the WTO doesn’t lead to growth, for example, and the endless debates over whether NAFTA or bilateral investment treaties have been good or bad for their signatories. Susan cites a number of examples of these in her essay.

What is underdeveloped is empirical research on the law part of international economic law, specifically the litigation part. There is so much we don’t know about what happens when investments disputes are subject to arbitrations, but that hasn’t stopped people like Bolivia’s president Evo Morales making consequential decisions based on their priors about what the process will mean.

So that’s why I say Susan’s empirical claims are precise – she thinks we need a new perspective on what happens in actual litigation, which is arguably both the signal offer of an investment or trade treaty and its most lawyerly manifestation. And it makes sense to recommend that assumptions like those of Morales be tested.

I will also commend her essay for giving us a bit of an intellectual history of the place of empiricism in international law ... which I think she would say comes largely from the increasing empiricism of international relations scholarship and the increasing popularity of ELS. She also offers outline of one way to pursue empirics in international legal research, which is all to the good.

This is already a long post, but I’d like to do three more things in it: 1. Note that there increasingly is empiricism in international legal scholarship, 2. Urge a bit of caution on the part of would-be empiricists, and 3. Speculate about some empirical research that might be particularly worth reading in the future.

1. The sort of work that Susan urges is, I agree, unnecessarily rare, but it is not déclassé. It has a worthy past and increasingly vibrant future. Robert Hudec, after all, kept track of WTO decisions, and Simon Lester and Kara Leitner continue to build on his work. John Yoo and Eric Posner have made empirical claims about effective international tribunals that occasioned a useful response by Anne-Marie Slaughter and Larry Helfer. And so on.

2. Cutting edge empirical work is technical – increasingly so – and takes training. Disciplinary barriers are often higher than you think. Pick up a copy of the American Political Science Review and Quarterly Journal of Economics and give them a casual read if you think you might disagree. I am, as I have said, no empiricist, but my sense is that legal scholars are often best-served by co-authoring if they feel the urge to do empirical work and by training themselves to read empirical papers (no easy task!) as a prelude to pursuing empirical work.

3. A couple of predictions about the future. Careful event studies might be useful evidence in determining whether and which international legal institutions matter. Ron Wright and Marc Miller have noted the growing prevalence of content analyses in legal research, and, done well, it seems to me that they are a natural complement to doctrinal work (it’s often hard getting enough observations to make that useful, though).

[Professor Susan Franck is Associate Professor of Law, Washington & Lee University School of Law. Please find her SSRN author page here.]

First, I want to take the opportunity to thank the editors of the Virginia Journal of International Law and Opinio Juris for an opportunity to discuss my recent essay, Empiricism and International Law: Insights for Investment Treaty Dispute Resolution. I would also like to thank David Zaring in advance for commenting on the essay. I look forward to a thoughtful and provocative discussion.

This essay develops ideas implicit in the work of certain international law, international relations and empirical legal scholars - namely that there are areas of international law deserving of and ripe for empirical analysis. More particularly, the use of empirical methodologies can create new facts, offer different perspectives and promote a more nuanced analysis of international law phenomena.

The potential benefits of such an epistemological approach are easily understood in the context of international investment law and dispute resolution.

The President of Bolivia, Evo Morales, has suggested that the World Bank's International Centre for Settlement of Investment Disputes (ICSID) is an international organization where no country, except perhaps the US, will ever win. The International Herald Tribune quotes Morales as saying “Governments in Latin America and I think all over the world never win the cases. The transnationals always win”. Although this assertion was used to justify Bolivia's withdrawal from ICSID, quantitative data flatly contradicts Morales’ allegation.

Meanwhile, a March 2008 story in the Financial Times, referred to investment treaty arbitration awards involving: (1) "hundreds of millions of dollars in compensation," (2) a claim by Mobil for "billions of dollars," and (3) Bolivia's "loss" in a case against Bechtel. Empirical information aids the contextualization of these claims. It is useful to consider that: (1) the average value of awards in one study was in the order of US$10 million, (2) the difference between amounts claimed and awarded was in the order of US$333 million, and (3) Bolivia's "loss" at the jurisdictional phase actually resulted in a settlement where the investors dropped their claims and were paid nothing. Meanwhile, issues related to international economic law – particularly trade and investment – are becoming a focal point for politicians and the public.

Against this backdrop, this essay does three things to consider the potential integration of empirical methodologies into the analysis of international investment law and dispute resolution.

First, it explores the historical relationship with international law and empirical methods. Acknowledging the re-integration of international relations and international law as well as the increased appreciation for empirical legal studies in domestic law contexts, the essay suggests that the mutual interest in empirical methodologies - broadly defined - could be developed further. For example, this might involve international lawyers drawing upon the methodological debates and insights from the empirical legal scholars and adapting them to analyze international law phenomenon; likewise, empirical legal scholars can gain new areas of research that may provide a useful counterpoint to study of domestic (whether in the U.S. or broad) legal phenomenon.

Second, the essay adopts Professor Korobkin's definition of empiricism and argues for a broad understanding of empirical methodologies in international investment law. The essay posits that quantitative, qualitative and mixed methods may all be usefully applied to the analysis of investment treaty dispute resolution and encourages debate on how different methodologies might be suitable for different research questions.

Third, the essay argues that the benefits of using empirical methodologies outweigh the costs and suggests five steps for developing an empirically infused research agenda in investment treaty dispute resolution. These steps may include, but not be limited to: (1) building research capacity, (2) obtaining data, (3) designing research methodology, (4) conducting research, analyzing the results, and contextualizing the findings properly, and (5) disseminating the results to stakeholders for consideration.

As befits an essay, it concludes on a note designed to spark conversation as much as it is to articulate a particular position. In particular, the essay argues that while empirical methodologies may not work for every research area or question, the use empirical methodologies can infuse international investment law with information to inform normative choices. Particularly for investment dispute resolution, systematically gathered and properly analyzed empirical data can correct misperceptions about existing dispute resolution processes, permit considered analysis of legal issues affecting the public, and could – for example – facilitate informed decisions about the negotiation and revision of investment treaties.

Particularly in the United States, Latin American countries such as Ecuador and even in Norway, there is a hot political dialogue about the proper terms of investment treaties. The issue is sufficiently compelling that the American Society of International Law’s International Economic Law Interest Group is going to be hosting a conference in November 2008 to discuss the intersection between the political dimensions of the debates about trade and investment. (A previous Opinio Juris post is here, and the call for papers is here.) Given this context, there is particular utility in examining what opportunities there might be to infuse scholarship with empirical methodologies to generate information that can form part of a dialogue to promote a more informed discourse on international investment law.

It is too early to know whether the approach I recommend will provide utility in the long run. There are inevitable challenges and practicalities that will be difficult to ignore that we might discuss here in greater detail. Nevertheless, I do believe that there is tangible, potentially useful research that can be done with relative inconvenience in the short term. These small, foundational projects can form the basis for replicating, developing and converging research in the hopes of ultimately creating a more dynamic and informed tapestry of international investment law. And after all, if we do not at least try, we will never know what might have been.