The Wheatley Institution

Field Experiments in International Relations

Robert O. Keohane
September 20, 2012

Related Videos

Abdul Aziz Said

Islamic-Western Peacemaking: Partnerships for a Better World
Abdul Aziz Said

February 8, 2011

Amaney Jamal

Successes & Failures Since the Arab Spring
Amaney Jamal

February 25, 2015

Cas Mudde

Why Should We Study Populism?
Cas Mudde

January 29, 2016

Click here to view the transcript
Click here to hide the transcript

I am very pleased to be at BYU. I recall that I spoke here exactly twenty-five years ago, almost to the day, because the Bork hearings were going on for the Supreme Court, and it was not confirmed. It was a big sensation. I remember listening to them on the way down to the hotel here. That was ancient history, of course, from the point of view of most people here. More than ninety years ago, at the end of World War I, Max Weber gave two now famous lectures, published in English as "Science as a Vocation" and "Politics as a Vocation." They well repay reading and rereading. They are short, so if you haven't read them, I recommend you do so. Thinking of those lectures, it seemed important to me to reflect on political science as a vocation. As the title of my lecture indicates, I am directing my comments both to my fellow political scientists in the audience and to students in attendance here who may be beginning careers in our field.

Now, I gave a version of this lecture (this is not news to our sponsors; I told them when I agreed to do it) at the Sheffield Graduate School of politics in 2008, which was directed principally towards graduate students. There are still some remnants of that focus. I realize that this audience is colleagues on the one hand and undergraduates on the other. I am going to reflect on our vocation from the vantage point of someone who has been a practicing political scientist—teaching, reflecting, and writing about politics—for forty-seven years. It is hard for me say that and to believe that, but it is true.

I begin by pointing out that, viewed historically, you are in distinguished company. Aristotle was probably the first systematic Western political scientist, theorizing the relationship of politics to other spheres of life and creating a typology of regimes. He created what we would now call comparative politics. This was in the fourth century B.C. Machiavelli not only advised the prince but sought to analyze the nature of leadership, the characteristic hypocrisy of political speech (which is not a new phenomenon and is certainly still a current one) and the sources of republican greatness. Hobbes provided what is still one of the most compelling discussions of the causes of political violence and the sources of and justification for the state. Montesquieu and Madison developed a durable theory of constitutionalism, and Tocqueville put forth insights into the nature of democracy that remain vibrant today (for example, in the work of Robert Putnam). I have already mentioned Max Weber.

In the generation of political scientists born in the first three decades of this century, I will list (somewhat arbitrarily) Gabriel Almond, Robert Dahl, Elinor Ostrum, Judith Shklar, and Kenneth Waltz, all of whom profoundly affected our knowledge of politics. There are so many fine colleagues doing insightful work now that to mention a few would be to risk slighting others whose work is equally important. The point is that, to the undergraduates, if you decide to go on to join in the political science profession and become a graduate student and then a faculty member in it, you are joining a vibrant profession with a rich history. If you are studying it, you are studying in a profession with a rich history. I hope your instructors don't start the syllabus in 1975. If I were conversant with classical Indian and Chinese sources, I could probably add to this list and extend this history even further into the past.

Following Virginia Woolf, many of you will probably notice that except for Ostrum and Shklar, this is a procession of men. Fortunately, however, this lamentable situation, which was characteristic of this profession as well as others, has changed. If I had listed contemporary political scientists of note, I would have had to include Theda Skocpol, Margaret Levi, Carole Pateman and Susanne Rudolph, all of whom have been president of the Association, as well as many younger women who are now leaders in the field. Although exclusion on the basis of gender and race was long a reality, our profession is increasingly opened to talented people from a wide variety of backgrounds.

What, then, is political science? What do we mean by it? I have an economist colleague who likes to say that any discipline with science in its name is not really a science and that it protests too much. Were one to adopt a narrow view of science as requiring mathematical formulations of its propositions, precise quantitative testing, or even experimental validation, political science would, indeed, for the most part, be an oxymoron. Today I will defend our nomenclature by taking a broader view. I would define politics as involving attempts to organize human groups, to determine internal rules, and, externally, to compete and cooperate with other organized groups, and reactions to such attempts. This definition is meant to encompass a range of activity, from a governance of a democracy such as Great Britain to warfare, from corporate takeovers to decisions made from the U.N. Security Council. It includes acts of leadership and resistance to leadership, behavior resulting from deference and defiance. I will define science as a publicly-known set of procedures designed to make and evaluate descriptive and causal inferences on the basis of the self-conscious application of methods that are themselves subject to public evaluation. As transparency, public evaluation, and criticism are inherent in science, all science is carried out with the understanding that all conclusions are uncertain and subject to revision or refutation. So political science is simply the study of politics through the procedures of science. Most of my lecture will be devoted to an explication of how, in my view, political science should be carried out. That is, the process of thinking and research that yields insights into politics.

I want to begin by talking about teaching. Teaching is sometimes disparaged. Colleagues bargain to reduce their "teaching loads." The language is revealing, since we speak of research opportunities but teaching loads. National and global reputations are built principally on written work, not on teaching. But when we look around, we see that virtually all top ranked political scientists in the world today are active teachers. Some are not good teachers, but they are all active teachers.  Few of them have spent their careers at a research institutes or think tanks. In my view, there is a reason for this. Teaching undergraduates compels one to put arguments into ordinary language accessible to undergraduates, and therefore to smart people who have not absorbed the arcane language of social science, which can be evasive as well as it can be illuminating. Teaching graduate students exposes one to new ideas from younger and more supple minds, as long as the students are sufficiently critical of the professor's views. I want to emphasize this view about criticism. In my experience, most students, rarely the best, are too deferential. In fact, one of the disappointments in my career is that students were less deferential in the 60s and 70s. They are more deferential now, and it is a bad thing.

In 1927, so the story goes (I think it is a true story that comes out of my father's notebooks; he was an honest man), the Chief Justice of the United States, former president William Howard Taft, came to Yale Law School, where his host was a dean of Yale Law School, Robert Maynard Hutchins. Taft was former president, chief justice, a man of ample girth (350 pounds), somewhat pompous as well as very intelligent and distinguished, and he was a conservative. Hutchins was dean at 28 years old; he was a radical. Yale was seen as a radical place. So the Chief Justice said to Dean Hutchins, "Well, I understand that you at Yale teach your students that all the judges are fools," to which Robert Maynard replied, quick as a wink, "No, Mr. Chief Justice. At Yale we teach our students to find that out for themselves." Like the Yale law students, you need to discover for yourself which senior political scientists are wise and which are fools. It doesn't come from their CVs, their nominal reputations, or their titles. You have to do so by using your own critical faculties.

Teaching is rewarding other ways. I learned a lot from my colleagues and directly from my students, who come to me with insights or works to read that were suggested to them by other faculty members. In the long run, one might see former undergraduate students become politicians or even rise to other high positions. A former student of mine is now a U.S. Senator, and another one just retired as president of the World Bank. (Both undergraduates, of course. The graduate students are all professors.) With former PhD students, ties are much stronger, since they remain in the profession. Two of my most valued colleagues and best friends at Princeton are former students, including Helen Milner, who is co-running this conference, and the eminent student of the European Union, Andrew Moravcsik. Former students of mine are scattered at colleges and universities around the United States, with some in Europe or Japan. In my office, I keep a shelf of books that began as Ph.D. dissertations under my supervision.

Never disparage teaching. It is an intrinsic part of political science as a vocation. Furthermore, it provides much more immediate gratification than research. When I am working on a major project, I never know where the results will be worthwhile. I have left unpublished a good deal of work when I realize the premises or methods I used were flawed. Sometimes, even once-published work will be ignored. Like politics for Weber, research can seem like a long, hard "boring of hard boards," in his phrase. Eventual awards may be substantial or may be meager; you never know until quite a bit later. If you give a good lecture or teach a lively, thoughtful seminar, the gratification is immediate. You know you accomplished something that day. During periods of self-doubt, teaching can keep you going.

Now let me turn to the science in political science. Turning to research. What is the process political scientists go through in their search for knowledge? I am going to focus on four activities: puzzling, conceptualizing, describing, and making causal inferences.

First, puzzling. Interesting work begins, in my view, not just with a problem—for example, how does democracy work in the United States?—but with a puzzle. Puzzles are anomalies: what we have observed does not fit with our preconceptions based on established theory. Hobbes sought to make sense of civil war and regicide. Tocqueville wanted to understand how a decentralized, individualized society, such as the United States in the 1830s, could exhibit such overall cohesion and even suffer from oppressive public opinion, in his view. Barrington Moore and a line of successors have sought to explain why some societies develop stable democracies while others do not. Scott and others seek to account for great revolutions and their absence.

Great leaps forward in political science often take place when someone sees puzzles where others are only seeing facts. The great philosopher of science Imre Lakatos say that science proceeds on a "sea of anomalies," which certainly applies to political science. That is, science only really progresses when you find anomalies, puzzles that don't fit either conventional wisdom or established theory, not when you just plug away, crossing the "t" or dotting the "i."

Now, I am not very satisfied with much of political science today. It seems to me that it often just does call attention to relationships that conventional thinking expects to obtain. I think there is a herd instinct in political science. It has been quite marked in the last decade or two, where people follow the same lines of work, just adding small amounts to it. So showing with more precision that democracies are relatively peaceful toward one another is an example. It is an important piece of knowledge that they are. The 150th article estimating in a little different way how they are, it seems to me, is not. That civil wars thrive in poorly-governed countries with places to hide, or that economic failure is bad for incumbents, resolves no major puzzles. It is important, then, I think, to begin with an interesting puzzle for which the answer is not obvious.

One implication of my point here is to find something puzzling to study. Don't just stop when you find something that is a topic in political science, that is a subject matter. There is another implication of the importance of puzzles: never dismiss what appears to be a naïve question. This is true for teaching and also true in research. The naïve questions are often the most fundamental ones. I will give you a true story which taught me this.

I was at Stanford in the 1970s teaching political science with no training in economics and trying to understand economics better. I was trying to pick it up on the fly by attending economics seminars. I would walk across the street and attended those. I remember one such seminar by an eminent student in a multinational corporation whose work I knew. He spoke very well in a highly organized way. After about three minutes, in true economics fashion, a young bearded man asked a question which the speaker answered to my satisfaction. After three or four minutes, the same hand went up again, and then again a few minutes later with a different question, each seeming rather obvious to me. After three or four questions, I was getting annoyed and impatient saying to myself, "Can't you let the speaker proceed?" After five or six questions, it dawned on me (perhaps I was the last person in the room to see it) that the questioner was the only person in the room who understood the topic and that his series of apparently naïve questions had dismantled the questions of this professor's talk. There was literally nothing left.

I forgot everything else about the seminar, except I remember how the room looked and where the guy was. I know who he is now. But the apparently naïve questions of this questioner had dismantled the talk's premises.  So the lesson is that the apparently naïve questions are often the most fundamental. If you are puzzled, ask. In our field, there are no dumb questions. It is true. Probably 90%, maybe 95% of naïve questions are just that. They are naïve. You don't understand what the speaker said, and then you can only gain by asking the question because you learn something about it. Maybe 5% of the time, you may be the only person in the room to see the anomaly and to sense, like a good detective, that there is something wrong about the story you are being told. The rewards for identifying the major puzzles, for the profession and for yourself, are very large indeed.

After you have a puzzle, the next step is conceptualization and being clear about the meaning of concepts. Giovanni Sartori pointed out long ago that concepts get stretched out of shape by political scientists trying to do too much with too little. It much often depends on definitions. How we think about the relationship between democracy and liberty, for example, depends on how we conceptualize democracy and liberty. Are they in conflict or not? Likewise, whether democracies ever fight one another, or whether international institutions degrade or enhance democracy, depends heavily on how one defines democracy. Whether civil wars are becoming more or less frequent may turn on how we conceptualize what a civil war is, rather than a lesser form of civil conflict. Whether peace requires justice or is at odds, in conflict with, justice depends on how we define both of these contested terms. There are no right or wrong definitions—I am a positivist in that way—but there are explicit and implicit definitions, those consistent with ordinary usage and those that are not. Authors can be consistent or inconsistent in their use of terms. At the conceptualization stage, it is our obligation to pull forward explicit definitions (I tried to define political science earlier in this lecture) and to seek to operationalize them consistently. The more definitions conform to ordinary usage, furthermore, the less confusion is likely to result.

Now I come to description and interpretation. The core of what most political scientists do most of the time is descriptive inference. Inference means drawing more general conclusions from established premises plus a particular set of facts. For example, from known facts, such as that each of 150 countries has a different form of government and economic characteristics, we may infer that there is a correlation between wealth and democracy. That is a descriptive inference. Properly speaking, though, such a conclusion rests on a chain of inferences. It is often necessary to probe those. For instance, we may have inferred from a sample survey of tax returns what per capita GDP is for the country, or from observations of three elections where the country is democratically governed. These inferences are subject to error. People might systematically falsify their tax returns in one country, for example, and not in another. The incumbent government might conceal decisive manipulations from election monitors. Other examples of descriptive inferences in political science are the generalization that democracies don't fight one another, the claim that international institutions typically provide information to governments but other governments compliance with rules, and the claim that the European Union was formed more about economic gains than security benefits.

So these examples indicate that descriptive inferences can be generalizations about a wide set of cases or statements about a particular set of events at a particular time and place. We can make inferences about individuals; for example, the sincerity or hypocrisy of leaders or our perceptions about other leaders' behavior. We can also make inferences about the behavior of collective actors that may have subunits pulling in different directions: what the "United Kingdom" or "China" did in a situation, or whether the "United States" (or, rather, unauthorized individuals) engaged in torture at Abu Ghraib or Guantanamo. We can make inferences about relationships: were there backchannel communications during the Cuban Missile Crisis between the Soviet Ambassador and Robert M. Kennedy, and, if so, what were they about? Are the informal conversations between Israeli and U.S. leaders today as hostile as might be inferred by their public comments? Often, political scientists try to make their descriptive inferences more precise by attaching numbers to whatever process they are seeking to understand. Precision is certainly enhanced by numbers that are both reliable and valid, and so such activities are to be encouraged.

It is important to recognize the importance of both reliability and validity. Reliability essentially means that, using the criteria publicly employed, the number could be replicated by an independent observer. If by criteria defining "wars" as organized violence involving 1,000 or more deaths, one team of observers finds fifty wars at a given period of time, another team using the same criteria should also find fifty wars. That is the reliability side. Get the same answer to the same question if you do it again.

Validity is different. It refers to whether the measurement used—in this case, 1,000 battle casualties—fairly reflects the underlying phenomenon being discussed—that is, war. Are all conflicts involving 1,000 deaths war? Even if some involve small societies so that 1,000 deaths is a large portion of the population and some involve very large societies that suffer more deaths by traffic accidents in a couple of weeks? Should all wars, from the skirmish that took 1,100 lives to World War II, be counted equally? Are they all wars? You see right away, these are constructions. These aren't factual descriptions. They are dependent on an interpretation of what we mean by a term, what we mean by a war. How are we going to categorize something? Very often, our conclusions rely on these, and all too often, people don't interrogate the interpretations that lie behind the supposed descriptions. So these are questions of validity that can't themselves be solved by clarification, but for which one has to think hard about how one's conceptualization relates to the phenomena that can be measured. Are we measuring what we are meaning to measure? So before we accept a descriptive inference, we need to have asked questions of validity as well as reliability.

This issue is highlighted by the famous philosophical distinction between a wink and a twitch. As Clifford Geertz writes, "The difference between a wink and a twitch is vast; as anyone unfortunate enough to take the first for the second knows." I want to add vice versa—if someone who is attractive to you is moving her eyelids rapidly, you need to engage in an interpretation before moving to a descriptive inference. If you interpret the eye moving as a wink, you may infer, "She loves me, too!" But what if you act on the interpretation and it was only a twitch?

Political scientists engage in interpretation all the time. When states "reject" a public offer, as Spain has rejected loan terms from the European Central Bank, are they really rejecting it entirely, or are they establishing a bargaining position? When the United States brings an auto trade case against China, as it did this week, is it seeking to maintain a level playing field or to help President Obama's reelection chances, in Ohio in particular?  When Hillary Clinton was overseas during the Democratic National Convention, was she merely following non-partisan tradition for Secretary of State, or was she signaling disinterest in a run for the presidency? Or, perhaps, sealing interest in the run for the presidency through an odd twist of logic? My point of this inference is twofold. It is not the same as simple description. We may think we are describing something, like I am describing that I am hitting this table. It involves an inference from known to unknown that can be incorrect or otherwise flawed. Both description and descriptive inference often rest on the interpretation of inherently (sometimes deliberately) ambiguous actions such as the wink-twitch.

Now I come to causal inference. Causality necessarily involves consideration of a counterfactual situation. If Charles II had not been executed in 1649, would Great Britain have a different political system now? If nuclear weapons had not been invented, would the United States and the Soviet Union have fought World War III in the 1950s? If the United States had not supported democratization in Egypt, would the Muslim Brotherhood be in charge there now? Since we inherently cannot observe at the same time what actually happened and what didn't happen, making causal inferences is always difficult. The descriptive inferences are hard enough, but causal differences are really difficult because you can't observe both the action and the non-action. You can't observe two situations. So we do not observe causality. You never observe causality. You always infer causality, and it is always a problematic or uncertain inference.

Now, if we can find and measure many highly similar instances of the same action—for example, votes for parliament or expressions of party preference in surveys—we might be able to make quite good causal inferences through the use of quantitative methods. If it is a common unit (unit homogeneity, in the jargon word), a vote is a vote is a vote, as opposed to being different kind of actions. Even then, our procedures may contaminate our findings. For example, people often make up answers to public opinion polls because they don't want to seem to not to know something. We know they recall having voted for winning candidates more than can actually have been the case. You can prove, with high significance, that more people report voting for the presidential winner than actually voted for the presidential winner, which of course is a contradiction.

More seriously, our inferences may be flawed because of omitted variables: something else changed that we failed to measure. This change, rather than the one on which we focused, may explain what we want to understand. Or we can confront endogeneity: what we stipulate as the effect is actually in wholer part a cause. There is a reciprocal relationship, for example. Furthermore, anticipation of consequences may create false impressions of causality. States may comply with international law, not because they have incentives to follow it or because they believe they are obliged to do so, but because they have carefully agreed only to rules with which they intended, in any event, to comply. If that is the case, it might appear to be causal, that the law leads to the action, but it would not be true, given my assumption. Conversely, real causality may be obscure. Political scientists seeking to determine the effect of deterrence threats in international crises did not find any significant effects. But critics point out that the states that would submit to deterrence threats should have anticipated the threats and their submission and therefore would not have stimulated a crisis in the first place.

The difficulties of causal inference seem endless. Indeed, I am here at BYU today because of an attempt led by some members of your faculty to explore another approach to the problem of causal inference: experiments. Experiments are standard in medicine, not yet in political science. Now, it is not possible to experiment on some issues. We can't replace the Egyptian military regime a year ago with, on the one hand, a military dictatorship, and, on the other hand, a second democracy and see how they would react differently to attacks on the American Embassy. For problems on which we can use experiments, the method has enormous advantages. At this conference, we are going to will talk about those.

There are also limitations of this and any other method. The one limitation of experimental methods is what is known as external validity. You may be quite sure that you got the right answer and that you have a causal inference for this particular set of people at this time, at this place, in this context. There is no guarantee that, if the context would have been a little bit different, you will obtain the same results. With respect to large scale problems, like the ones I am mostly interested in, experiments will probably always remain unavailable. Unfortunately for science (but probably fortunately for the human race), we cannot manipulate large scale political phenomena for our convenience, even if human subjects committees would allow us to do so. So on these issues, our causal inferences are always likely to be problematic. We could not construct a French Revolution for the treatment and have no French Revolution be the placebo.

Aspirations to causal inference are often linked to hopes for prediction. Our causal knowledge of gravity helps us to predict the movement of planets and other celestial objects, and our causal knowledge of biology and, increasingly, genetics help us to predict the incidence of disease. Sometimes we can make predictions of election outcomes on the base of economic conditions, but even our best predictions are imperfect. For my own subject of international politics, the situation is even worse, because it revolves around conscious strategies of reflective actors: I act as I do because I anticipate what you will do. But you, knowing this, act differently than I expect, and I, in turn, anticipating this, change my behavior. This is an infinite regress about which no prediction can be made. One would have to know how many cycles the players would go through, but this could be ascertained. Smart players would learn it and go one step further.

So why choose political science as a vocation? If causal inferences in our field and predictions are as intractable as I have argued, why should we choose our profession? My short answer is that we study politics not because it is beautiful or easy to understand, but because it is so important to all fields of human endeavor. I readily admit that I cannot prove that politics are important. Weber, in "Science as a Vocation," says that "[the] presupposition [of something as worth knowing] cannot be proved by scientific methods. It can only be interpreted with reference to its ultimate meaning."

Without governance, though, as Hobbes said, life is "poor, nasty, brutish, and short." Democracy is, in my view, however flawed it is (and I will mention that in a minute), immensely better than autocracy, much less tyranny. Making democracy work is hard, and imposing democracy seems close to impossible. Ironically, around the world, the flaws of democracy are becoming increasingly evident in the wake of the spread of democracy worldwide. At the same time, we have a celebration of democracy; the last fifteen years has probably been the period of the greatest celebration of democracy in the history of the world. And yet, you can't pick up a newspaper or a political science journal without seeing the pathologies of one democracy or another.

One of the interesting experimental papers for this conference by Dan Nielson, Helen Milner, and Mike Findley purports to be about foreign aid in Uganda. It shows that people in Uganda seem to prefer multilateral or even bilateral foreign aid from foreigners to government projects of a similar type from their own government. What the paper doesn't observe is how devastating that is for theories that say that democracy is the best form of government, because Uganda is a democracy (at least nominally), and foreign aid by the World Bank or U.S. Government is not democratic. The people of Uganda have no accountability or control over it. Yet the situation where they have no influence, they can't hold the provider accountable, they regard in a higher esteem than the situation in which they have (at least in principle or nominally) some ability to hold the providers to account.

We are going to see, I think, in the next ten or fifteen years, a lot of soul-searching about democracy. We have been through a period of celebration and, as C. Wright Mills said a long time ago of the United States, periods of celebration are usually overdone, and there is an underside. I think we are going to see more critical attention to democracy rather than the assumption that has, in fact, dominated public policy but also much of the profession, that democracy is necessarily and, out of context, the best form of government. I am kind of with Churchill that it is the worst form of government except for all others. That is a somewhat different orientation toward it.

Back to political science. Without a political science, it would be hard to have effective democratic leadership because it depends on accountability, and that is based on critiques of action which are realistic with respect to what is possible. Without a vibrant political science, leaders would be guided only by their limited personal experiences, historical analogies, and folk wisdom—all highly unreliable as a basis for inference. Publics would be deprived of a scientific basis for criticism. We should therefore judge our work not according to some idealization of science or by the standards of the physical and biological sciences, in which case we fall short and will for a long time fall short. Unlike Newtonian physics, we cannot properly aspire to knowledge of grand, covering laws that explain a myriad of disparate events. Instead, we should ask the more modest question: whether knowing the political science literature on a given topic has prepared us better to anticipate what could happen, to assign probabilities to these various scenarios, and to judge actual results.

Are the findings of political science superior as a guide to historical analogies, extrapolation from the very recent past, and common sense? The answer is not always affirmative, unfortunately. Indeed, as I already criticized the herd instinct in political science, I think we need to widen our focus to topics that we haven't focused on enough. For example, I would say one of the most important questions in politics right now is, "What is the impact of new social media on politics, comparatively and internationally?" We don't know very much. We know virtually nothing, scientifically, about that subject. I think that is fair to say. That is not surprising; it is new. What is surprising is that almost nobody I know in political science is studying that question. I don't know why every graduate student in the country is not after that question. The only answer I can think of is that their professors aren't asking that question because we are too old to understand social media. That is not a good reason. So, we need a lot more boldness and imagination in the pursuit of our field.

At the outset of this lecture, I praised criticism and counsel of one's elders. I hope that all of you, from students to middle-aged professors, are dissatisfied with the accomplishments of earlier generations, skeptical about their inferences, and skeptical about what I am telling you right now. I hope you see puzzling anomalies in some of the conventional wisdom, issues that need to be unpacked, and I hope that you have objections to express to what I have just been saying. Many of you will have noticed that my sources and examples come almost entirely from Europe and the United States and from international politics, which has been dominated for five centuries by Europe and its offshoots. There is a sort of parochialism, therefore, about the way I have interpreted and presented this subject. This parochialism is presumably due, in some measures, to my own limitations, but also reflects the discipline, which is heavily American, to some extent European, with relatively few genuinely important, independent contributions from scholars on other continents.

As the economic and political centers of gravity shift away from Europe and the United States, and as we move into the "post-American era," as Fareed Zakaria calls it, this is bound to change. Political science will become a global discipline; it will, however, only prosper if liberal democracy, for all its weaknesses, thrives. If we are doing our job, we political scientists will be irritating to political leaders since we illuminate their deliberate obscurities and deceptions, point to alternative policies that could be followed, question their motivations, and dissect the operations of organizations that support them and the governments over which they preside. They will try to buy us off or, failing that, if not prevented from doing so, to shut us up. As a result, we have a symbiotic relationship with democracy. We can only thrive when democracy flourishes, and democracy, in a smaller way, needs us, if only as a small voice of dispassionate reason.

Our symbiotic relationship to democracy means that political science cannot be value neutral, nor can we be neutral with respect to order verses chaos and war verses peace. In our particular investigations, we need to seek objectivity, a goal that is never realized but that we should strive for. Otherwise, people with other preferences or who do not know what our values are will have no reason to take our findings seriously. In the absence of a serious culture of objectivity, no cumulative increases in knowledge can take place. But the overall enterprise, despite what I have said about objectivity, can never be value neutral.  We should choose important problems because we care about improving human behavior and human politics. We should explain those choices to our students and readers, and we should not apologize for making value-laden choices even as we search unflinchingly for the truth, as unpleasant as the truth may be.

I hope you will consult your values as well as the literature in deciding what to work on. Try to work on things that are important, where the answer matters to you.  It is not simply a puzzle, like you might solve a crossword puzzle; it is something that really matters, and if people knew the right answer to this puzzle or this anomaly, we would have a better world in one respect or another. In conclusion, let me express the hope that younger political scientists (and probably everyone in this room is younger than I am, so that is an easy call) will see openings where we older political scientists see closure, and that you have ideas about how to move through those openings to the insights that surely lie behind them. There are certainly productive, new interpretations to offer and new descriptive and causal discoveries to be made. Some of you have already been formulating some of these new views. I look forward to your questions and to a stimulating discussion. Thank you for listening.

Question and Answer

Question: To me it seems that there is a wide history whenever major legislation is passed, let's say in America, there are always major, major unintended consequences, or the anticipated financial impact is way off. So how can we look with any credibility at major, complex legislation recently passed and the estimation of its impact (financially, for instance) with some degree of accuracy? How do we get any credibility in that, and why isn't there more effort, if that were true, for more complex legislation to be tried in some states in the messy field of real life before these federal laws are applied so suddenly to all states at the same time?

Dr. Keohane: I am not going to comment on the particular issue of federal-state relations, about which I know very little, but I think your general question is a good question. The first thing to distinguish about unintended consequences is motivated verses unmotivated bias, in the psychological expression. That is, sometimes people just make mistakes, right? Problems are complicated. There may be a systemic effect that is not understood by anyone at the time that some legislation is passed, and there is an indirect effect that is not known. But more relevant to us as political scientists is motivated bias. Very few tax revenue estimates in American states probably are understatements of the tax actually going to be received. I am thinking of my state, New Jersey, where they governor egregiously overestimated tax revenues in a plausible way, and it is now showing to be overestimated in order to pass a certain kind of budget.

So we have a lot of motivated bias. Very few of us would expect about any government program that is about to be implemented, whether it is buying a set of fighter planes or providing medical assistance to people, that the costs have been underestimated by the incumbents. We would assume that the costs have been overestimated. They have always been underestimated because it makes it easier to sell. So there is a certain kind of motivated bias. If we understand what my colleague/student/graduate calls the organized hypocrisy of politics, we will understand that we will look with a gimlet eye at all these estimates, and we will have a pretty good idea of where we think the error is going to be. If it is wrong, it is going to be underestimating the cost of program that somebody wants. Conversely, if somebody doesn't want it, it is overestimated. There is also unmotivated bias, which is a matter of the inherent complexity of much of the world.

Question: You mentioned other topics that you saw weren't being studied in political science, like social media and its impact on politics. What are some other fields that you see need to be studied?

Dr. Keohane: Think about the triad in international relations of trade, monetary relations, and migration (flows of goods, money and people). If you do a search of those three fields, you will find that migration is vastly understudied. It involves very important flows, because these people actually do things in the countries that affect the politics there. Yet the time we probably spend on it (I haven't done a quantitative study, but this is my guess) is one tenth as much as trade and one fifth as much as monetary relations. I think migration is a topic in which we need more work. There are a lot of issues of international environmental politics with very little work on them. Only one of the top ten, maybe none of the top ten departments, only one or two top ten universities, have figures working prominently on international environmental politics.

I think there is a systemic reason for these shortages. In the last twenty years, graduate students have been very cautious. They tend to follow in the footsteps of their professors, so if you get a lot of professors who are working on a given topic, you will get a lot of graduate students interested in that topic.  If you have almost no faculty working on a topic (that is true of both migration and environmental politics at major departments), you don't get graduate students working on it. It is perverse of course, because when you look at the job openings, it is reverse, right? There are going to be more jobs in areas which people in the undergraduate level think should be studied, where there are fewer jobs relative to people out there in the overstudied areas. We still have this perverse pattern, a herd instinct, of graduate students going in the direction where their professors go and not striking out, not being bold enough and critical enough of their seniors and elders.

Question: My question is regarding the upcoming election and the international relations that are happening right now, such as with Libya and Egypt. What kind of questions are political scientists asking right now about those, and how are they going to affect the election? What kind of questions should we be asking about it? How is that affecting our domestic politics?

Dr. Keohane: I think that the effects of most events abroad in this election, with the exception of a major war, will be inconsequential. That is, we are already late in the election cycle, early voters are beginning in the various polls, 5% of people are undecided, and we know that foreign affairs are very low on the list of people's concerns. My guess is that, unless we get something like an Israeli attack on Iran or some other major international episode, we are not going to have much impact.

Now, I do believe that we know something from political science and that if we do have a major international interest, it will help the president, whoever the president is. Kennedy's popularity rating went to a high point after the Bay of Pigs fiasco. So his worst foreign policy action led to the highest popularity. George W. Bush's went to a high point after the invasion of Iraq. So it is not correlated with doing well. If there is a major international crisis, it will help Obama. That is the one generalization that I would make. Otherwise, I think there will be no impact.

Longer term (this is very speculative—I implied before that we should always assign uncertainty; I have assigned a very high level of uncertainty to this statement. It is not a certain statement. It is my intuition.). we are going to see a reconsideration of democratization in the light of the Arab Spring actions because we had a pattern (one of my colleagues wrote a book about this recently) by which the United States systematically, in the interests of what it saw as its national security and economic interests, favored autocratic regimes in the Middle East. They did so on one the grounds that anti-Americanism was so high in the Middle East that if they were democratic, they would be anti-American, and that would be bad for us; therefore, they should not be democratic.

Condoleezza Rice announced that change in that policy in 2005. I am not sure if it was implemented, but Obama certainly did implement it in Egypt in the last two years. If we have an Egyptian regime which is hostile to the United States—a democratic Islamist regime—there is going to be a lot of soul-searching about the relationship between democracy and U.S. interests, which has been taken for granted as a positive one in the last fifteen, twenty years. When it was just the Soviets, their successes, or the Chinese communists on the other side, then democracy looks great. When it is anti-American democratic regimes or pro-American authoritarian ones, it is a more mixed picture.

Question: My question is less political, but I am interested in how and when you made the decision to make politics a vocation, how you have arrived to where you are today, and where you were when you were in our shoes.

Dr. Keohane: Well, it is a boring story actually, because I was raised deeply immersed in politics. My dad was a professor of social science, and he taught political science. My mother and he wrote a high school civics text before I was born. So that is exogenous; I couldn't have had any influence on that. They were both very active amateur democrats, so to speak, in Chicago. My mother was a member of the Socialist Party in 1932, and one of the greatest regrets in her life was that she didn't vote for Roosevelt because she was too radical and he was too conservative. She became an adamant, strong New Dealer, and a very active participant. She was actually a socialist in 1932, and in 1938 she served as a Republican poll watcher in the city of Chicago.

Why a republican? She changed her views so fast. Well, democratic machine-controlled Chicago, in their practice of running an election, was having a policeman walk into the voting booth with the voter and guide her hand. That way, you don't have to worry about them taking money and reneging on you. She didn't think this was a very good way of running politics. Since there was no Republican poll watcher (there was no Republican Party), she was the Republican poll watcher. So I was kind of raised on politics. It was the stuff of family discussions and so it is a little bit boring. It would be much more interesting if I had been in an apolitical family and suddenly seen a light on how one should study politics.

Question: I read in an interview with foreign policy that you see one of the most important issues in international relations of our time being the rise of China. So just a few interrelated questions, do you think that China wants to serve this more involved role in world politics? Does that matter? How do you see the world with China as that more key player?

Dr. Keohane: There are three questions I want to respond to. The first is a general issue of the rise of China. If you approach that as a political scientist, the first thing you will do is ask, "What is the class of events of which this is one—that is, the revival, economically, of a major continental power, a state—and its ability, therefore, to challenge whichever power has been dominant before?" If you look at that issue, there is a lot of work out there, so you can learn a lot. You learn a lot about the rise of England in the seventeenth century against the Dutch. You can learn a lot about the failed attempt by the French to overthrow English hegemony overseas. You can learn a good deal about the way the British accommodated the rise of U.S. power (the U.S. was the China of one hundred years ago, in a certain way) and didn't succeed in accommodating the rise of German power. So, World War I and arguably World War II examples.

So the first thing you do is you look at this in a comparative perspective. You are not going to find enough from one case—especially one case that is future. Now, the comparison is not going to give you a set of answers; it is going to give you a set of questions. It may raise some of the right questions, and you will see there has been variation in the outcomes. There was no Anglo-American war in 1900, although people thought there might be. There was a World War I, which was with Britain and Germany. So you have variation in the outcomes, you have a class of events, and then you can get working on that kind of problem. So you have to understand that background. Anybody who is going to look at China ought to have some kind of understanding of that generic problem. Just raising the question will not give you answers.

The second issue is you said what China wants. I warn you against that question because China doesn't want things. China is a country of 1.3 billion people. The government of China is a complex, shifting set of political coalitions. There are almost certainly different preferences among different Chinese leaders and different Chinese groups. So the second question, I would say look at the various influential groups in China, what their interests are, and where their long-term advantage lies. For example, you might want to look at which groups would benefit and profit from long-term stable trading relationships with the West and which groups might benefit and profit from territorial aggrandizement, or an extension of power in the South China Sea.

Your last question is a very good question. What is the world with China going to be like? I think the first start there goes back to Kenneth Waltz. He would say that there is a difference between a unipolar world where there is one dominant state and a bipolar world where there are two states of rough equality. We have had a unipolar world since 1990, and at some point in the future, if China keeps rising, we will have a bipolar world. We know some things about those relationships. You would want to learn about them. It is quite contested actually, still, whether a bipolar world is relatively stable, as Waltz held, or unstable, as traditional balance-of-power theory held. You would want, then, to study other situations where there has been bipolarity with the rise of one state. You have to ask, "Under what conditions, then, could that be peaceful?" I don't think you want to ask the question, "What will happen?" because these are probabilistic. The scientific question is not "is this going to happen?" but "under what conditions will it happen?" Under certain conditions, the U.S. and Britain became a peaceful pairing where there could have been a war. Under other conditions, Britain and Germany fought it out, even though, after all their monarchies in 1914, they are from the same family. So we are not talking about inherently different. Those are the kinds of questions I would ask.

Question: How do you see the Syrian conflict ending?

Dr. Keohane: I have a very good friend who says, "Never comment on a country that you haven't at least flown over."  So I am not going to comment on this specifically, but I will say that we know some things that are relevant to this. I think we are very sure that it will not end in Syrian democracy in my lifetime, because violent revolutions in places that have no history of a democracy don't lead to democracy. So we could rule out that option that, five years from now, Syria is going to have a thriving democracy. That is very low probability, virtually zero.

Who will win the struggle is another question. You would have to be an expert on ethnic politics to know what is likely to happen if the Alawite minority no longer runs Syria, and you have to ask who is going to run it. My guess is that it is going to be a long period of violence and instability in Syria. There is no easy end to this conflict. One question you could ask is, "How likely is it that it will spill over into an internationalized conflict?" Once again, that can happen. So it is mostly questions, I think. I don't think anybody can give you a very good answer. The one thing I would rule out is the dream that somehow we are going to have Syrian democracy in 2015.

Question: The prospect of using my political science degree to enter academia, to study, research, and teach these things, is very attractive to me to understand the systems better as a whole, but I am also extremely tempted to simply enter the systems, participate in them, and engage in the political systems. To what extent do you also feel that temptation to enter in and participate in the systems instead of just studying, teaching, and researching about it? What would you advise political science students moving forward as to what their engagement should be in these systems as they are learning about it as well?

Dr. Keohane: I was raised in a family that was more activist than research, as I mentioned before. Neither of my parents was an independent scholarly researcher to any extent. They were both extensive political activists at certain times. So I came from that background, and in 1968, I was an amateur democrat. I was in charge of Eugene McCarthy's campaign in a congressional district of Pennsylvania. I spent almost all that time in winter and spring campaigning for McCarthy delegates for the Democratic National Convention, which was, given the rules at the time, a difficult and frustrating task. So I have been both places.

I will say two things. One is that I think it is very hard to do both at the same time. Basically, when I left Swarthmore College for Stanford in 1973, I cut off my activist political ties. Once you are doing it, if you are in the same place, you get phone calls all the time. I was going to California, and nobody ever knew I had been an activist. So I think you have to choose. Also, I think the temperament is different. The scholarly temperament stands back, is critical of one's own preferences: I prefer democracy and I have been critical of democracy here. You can't do that in politics. The words will be thrown back at you. You are running for office and you think democracy is terrible—end of story. You really have to make a choice. I think that ultimately, it is what Weber said; it is what you have a passion for. Well, it is two things. One, it is what you think you are good at, because you may decide you are really good at one of these things and really only okay at the other. You should probably do what you are really good at. But also, you should do what you are passionate about.

So, you would interrogate yourself here. Are you not just happier, but do you feel more fulfilled, do you accomplish more, if you spend a month working on a political campaign where you care about the candidate and you feel that you made some contribution to the candidate's performance, success or not? Do you feel better about that, or do you feel better about sitting in the library mastering the theories of Robert Putnam and Elinor Ostrum. I think, deep down, it comes down to your gut feeling about which you really love to do. The people who want to sit in the library are a minority. So not knowing you ex ante, my guess is probably the activist, the 90%. But Weber said this in "Science as a Vocation"; if you don't think that the state of your soul depends on solving a particular problem, you have no vocation for science. You really have to think about the most important thing in the world that you solve this political problem. If you don't, if you just think it is a job or a task, you won't be very good at and you shouldn't do it. You have to ask yourself, do you feel this way about discovery and about science, or not? If you don't feel that way, you have no vocation for science. You should probably do something else.

Click here to hide the transcript