Tuesday, February 28, 2006

Cooking the Results

Drudge has a link to Greg Sheffield's always-informative NewsBusters site. Greg has analyzed the latest CBS News poll, which shows President Bush's approval ratings slipping to an all-time low. His findings? CBS cooked the books, oversampling Democrats, while under-sampling Republicans. It's easy to drive down poll numbers when your sample is 37% Democrats, and only 28% Republicans.

Meanwhile, here's how CBS News explains its polling methodology and results. Of particular note are these comments on "weighting."

"We take great pains to adjust our data so that I accurately reflects the whole population. That process is called “weighting.” We make sure that our final figures match U.S. Census Bureau breakdowns on age, sex, race, education, and region of the country. We also “weight” to adjust for the fact that people who share a phone with others have less chance to be contacted than people who live alone and have their own phones, and that households with more than one telephone number have more chances to be called than households with only one phone number."

So, despite these "great pains," CBS still wound up with a sample that has a disproportinate number of Democrats, and fewer Republicans that the populace as a whole. Hmmm....I think CBS needs to take greater pains in its polling, or out-source the job to a more reliable outfit like Rasmussen. According to Rasmussen's daily poll, the president's approval rating is now at 43%, and has been as high as 49% earlier in the month.

1 comment:

Huan said...

the more statistically truthful thing to do would be not to weigh the results but prior to revealibng the rsult post a chart comparing the polled demographic vs the expected or "real" demographic and the p value as to whether the polled demographic is significantly different from the expected/real demographic.

the standards for scientific research is to post methods first, followed by statistical analysis for validity of the study population and then the results.
but then it is a political poll and has nothing to do with validity does it?