Abstract
Based on their success at predicting the outcome of elections opinion polls are used by the media, government and the political parties to measure public attitudes to a very wide range of other issues, helping to shape policy proposals and inform debate.
Despite their importance within the political process, the media, political parties and pressure groups nevertheless want feedback from opinion polls quickly and cheaply. Large scale random probability surveys may provide the best quality data but fail miserably on speed and cost.
Among practical survey methods the telephone has become the most commonly used mode of interview for general population opinion polls despite some doubts over response rates. Online polls have an increasing share of the market, despite the obvious drawbacks of relatively low internet penetration and the fact that they rely on panels of willing participants.
This paper shows that while telephone polls produce answers that are similar to those obtained from large scale random surveys there are sharp differences in results obtained online. The paper shows that these differences cannot be removed by weighting by demographics, newspaper readership or by using attitudinal variables. The research does, however, uncover evidence of significant and disturbing mode effects. In particular, a growing army of professional online panel members seem to race through online surveys, giving responses that explain a good measure of the differences between online and telephone research. These findings suggest that the online research industry needs to devise methods to ensure online respondents carefully consider the answers they give and design questions and answer codes that do not inadvertently lead online respondents to certain answers.