Our Survey says #1

Following the release of Australian Bureau of Statistics Unemployment Data last week Roy Morgan Research did a Poll where electors (sic) were asked:

The monthly Australian Bureau of Statistics unemployment estimate is 4.9%* while the monthly Roy Morgan estimate is 9.3%. Which do you think is closest to Australia’s true level of unemployment – 4.9%* or 9.3%?”

The Results in

This special telephone Morgan Poll…with an Australia-wide cross section of 651 aged 14 and over

were stunning. Apparently

“Only 30% of Australians believe the ABS unemployment figure”

That would be “30% of Australian’s with a margin of error of…”? According to their own data, something in the region of 4% btw.

(As Stilgherrian says in his Podcast lionising Dennis Shanahan, if you don’t mention the margin of error in your poll, then you are a…well listen to the Podcast.)

Is a sample of 527 people – 124 of the people they talked to can’t vote – large enough to get a reasonable result without too great a margin of error? Is it satisfactory to accept the results of a poll and Roy Morgan’s supposed miniscule margin of errors when such a small sample is intended to represent more than 15 million Australians? That’s one person interviewed for every 23,000 Working Age Australians.

That they are surveying people about the results of another of their own surveys exaggerates the suspicion this survey was done simply to benefit Roy Morgan and the chooks they need to feed. Even the Onion would find it hard to parody that.

Perhaps it’s time for surveys to be instead reserved for something far more valuable?

Especially if you win a twenty one inch remote control television!


3 thoughts on “Our Survey says #1

  1. There is nothing wrong, per se, about the sample being a “small percentage” of the population. That is the entire point of statistical inference – making inferences about a population based on a slice that is “representative”, because it is simply infeasible to survey the entire population. Unless there is something systematically wrong with the sample – for instance, if the sample is entirely consisting of men, or is purely drawn from a particular location that has a strong anti-government sentiment – otherwise sampling techniques are generally sound.

    Based purely on what you have there I’d be a lot more worried about the cynicism and scepticism people have shown towards the ABS data. This goes to a far more systemic issue where firm, strong evidence is discarded “just because people can”. No doubt people are entitled to their own opinions; but for people to systematically disregard scientific data “just because they can” is only detrimental to rational thinking, in favour of emotion-driven mob-mentality political discourse.

    It is far more important to report the possible limitations (eg. sample bias, convenience sampling, etc) than “margin of error” per se. Generally polling numbers around the 500 mark margin of error rarely exceeds a few percent.

  2. I think that it just proves yet again that although we think more information enlightens people and can get them to shift views that the opposite is the case – more information is always caste in a way which reinforces their views. There are the ABS believers and disbelievers each interpreting the “facts” their own way.

  3. Pingback: Our Survey Says #2: Mobile Market Share by CNET | Making Hay

What do you think about what I wrote?

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s