On April 17, 2008, David Pollock and Dalia Mogahed addressed a Policy Forum at The Washington Institute. Dr. Pollock is a visiting fellow at The Washington Institute and author of its recently released Policy Focus Slippery Polls: Uses and Abuses of Opinion Surveys from Arab States. Ms. Mogahed is executive director of the Gallup Center for Muslim Studies and coauthor of Who Speaks for Islam? What a Billion Muslims Really Think. The following is a rapporteur's summary of their remarks.
DAVID POLLOCK
Most Arab and Muslim public opinion polls are unreliable for several reasons. Often, the problem is not the pollster, but rather the conditions in the Arab world. Governments tend to ban or censor polls, and people are often are reluctant to speak frankly to strangers. Demographics also pose a problem in that most polls fail to reach rural areas or women effectively.
That said, some pollsters use badly designed surveys or conduct their research unprofessionally. For example, it is common for polls to use the results of as few as six countries to represent the opinion of the entire Arab world, and as few as four countries to reflect the entire Muslim world. The wording and sequence of questions can be misleading as well. Some questions are phrased in a manner that encourages a particular response, as in "courtesy bias," where respondents are led to agree with a statement as presented. And some interviewers pose questions or offer a series of facts that are critical of U.S. policy immediately before asking about a respondent's opinion of said policy. Such a strategy almost invariably produces negative answers. Indeed, some polls are unreliable to the point of being fraudulent -- Zogby, for example, go so far as to advertise their ability to procure biased results for political ends.
Although there are mechanisms that pollsters can employ to make their surveys more reliable, these measures are usually ineffectual in the Middle East. For example, phone polls have become more popular because, in theory, they allow the pollster to reach all sectors of the population. Most Arabs, however, are predisposed to be more forthcoming with a stranger face to face rather than on the phone. For instance, in one phone poll conducted in Saudi Arabia, King Abdullah received a 95 percent approval rating, which is not reflective of the general sentiment.
Trend analysis -- the idea that one poll is merely a snapshot, but the aggregate data of many polls over many years is accurate -- is regionally problematic as well. Such analysis cannot improve accuracy if the polls are unreliable. Likewise, it is difficult to attain reliability by crosschecking a number of different polls on the same topic if all of the polls are similarly faulty. Moreover, even the most reliable polls tell us little about public behavior in the Arab world, and even less about the behavior of Arab governments. In this respect, polls are not an ideal guide for U.S. policy.
There is, however, one poll result -- confirmed by straightforward questions asked by many different pollsters over the past four years -- that Washington should find useful: across the Arab world, support for terrorism has decreased substantially among all people (except the Palestinians), while unfavorable attitudes toward the United States have remained unchanged. This seemingly paradoxical finding leads to an important conclusion: Arab support for al-Qaeda and other terrorist organizations is not related to disapproval of the United States. Polls reveal that whenever a terrorist attack takes place in a regional country, support for terrorism plummets in that country and stays down.
All of these points have important policy implications for the United States. First, policymakers must take poll results from the Arab world with a grain of salt. The United States may be winning part of the battle for hearts and minds, but not by any positive action on its part; rather, it is increased resentment of al-Qaeda that tips the balance in Washington's favor. Policymakers should consolidate this advantage. Finally, Washington should not assume that public opinion trends can provide accurate long-term information about the behavior of Arab publics or the policies of Arab governments.
DALIA MOGAHED
Although there are serious problems with polls in the Arab and Muslim worlds, it is nevertheless worthwhile to conduct them. Public opinion is important and should matter to policymakers; ignoring public views on certain issues would be fundamentally anti-American. Surveying public opinion in undemocratic states is especially important, in part because polls have the potential to predict change; for example, when a society's perception of corruption is high, change often follows. In addition, acknowledging global anti-Americanism is important in the fight against terrorism, and polls are central to this understanding.
The decrease in support for terrorism in the Arab and Muslim worlds and the lack of change in support for the United States are related issues. Two competing forces are at work here: the sense that killing innocent people is morally wrong, and the perceived injustices committed by the United States. When a terrorist attack occurs in a regional country, the first force is usually strengthened and the second remains the same, leaving the first comparatively stronger. But in countries where the people believe they have suffered injustices at the hands of the United States, anti-Americanism is the stronger factor. It is therefore crucial to have good polls because they can help predict key trends along these lines.
A reliable poll must be independent; it cannot be funded or conducted by special interest groups. It must also be consistent -- the same questions must be asked in every responding country, whether Arab, Muslim, non-Arab, or non-Muslim. For example, in one poll on anti-American sentiment, researchers found that France and Germany have the same amount of anti-Americanism as Saudi Arabia, while Sierra Leone, a Muslim nation, has an eighty percent approval rating for the United States.
There are a number of steps pollsters can take to ensure their surveys are both representative of the population and effective. Polls should be conducted in person, not by phone. The length of the questionnaire is also important; the beginning section should be used to build trust, and then more sensitive questions should be asked at the end. Moreover, female pollsters must be made available to interview female respondents because women are more likely to be open and honest when polled by other women. And in order to escape "options loading" and other poll biases, surveys should offer a sufficient number of open-ended or free-response questions where people can express their views without being cued by the answer options. In sum, although polling will always be problematic, it is the best way to assess public opinion.
Policy #1365
