Friday, April 30, 2004
Polls apart
It's time to lift the veil a little on the mysteries of public opinion polls, which are of course running hot in the United States with the Presidential election looming. In New Zealand - still 16 months or so out from an election - they are having a surprising profile also. The fortunes of three parties are hinging surprisingly on them at the moment.
One of the most misunderstood things about an opinion poll is the much-bandied about phrase "margin of error". You might assume that this means "margin of error", as in how accurate the poll is. Wrong.
Here is what the market research companies don't tell you:
"Margin of error" means purely "to what degree does the sample of people we have spoken to represent the population at large," particularly in terms of age and sex. It tells you nothing about whether they represent the population in terms of views expressed (that is an assumption the polling companies make); it tells you nothing about whether the questions were biased; and so on. What the market research companies also don't tell you is that if they do not get a close match with their sample compared to the population at large, they weight the age and sex groupings to bring the sample into line. There is no way of knowing whether that distortion affects the responses or not.
This is only one of many problems attached to market research. Problems that often create "rogue" polls, which bear no resemblance to the true state of affairs. It's also why two different polls taken at the same time can produce quite different results.
It's time to lift the veil a little on the mysteries of public opinion polls, which are of course running hot in the United States with the Presidential election looming. In New Zealand - still 16 months or so out from an election - they are having a surprising profile also. The fortunes of three parties are hinging surprisingly on them at the moment.
One of the most misunderstood things about an opinion poll is the much-bandied about phrase "margin of error". You might assume that this means "margin of error", as in how accurate the poll is. Wrong.
Here is what the market research companies don't tell you:
"Margin of error" means purely "to what degree does the sample of people we have spoken to represent the population at large," particularly in terms of age and sex. It tells you nothing about whether they represent the population in terms of views expressed (that is an assumption the polling companies make); it tells you nothing about whether the questions were biased; and so on. What the market research companies also don't tell you is that if they do not get a close match with their sample compared to the population at large, they weight the age and sex groupings to bring the sample into line. There is no way of knowing whether that distortion affects the responses or not.
This is only one of many problems attached to market research. Problems that often create "rogue" polls, which bear no resemblance to the true state of affairs. It's also why two different polls taken at the same time can produce quite different results.
