Wednesday, July 11, 2012

NetHui Part 3 - How Not to do Research

The World Internet Project got up again and went through some rather wishy washy "facts". Some of the politicians who came after did point out the fault that I spotted last year. If the primary form of data collection is via the phone, then the information being gathered is favoured towards those that are more likely to have the Internet, thus, not only the fact that they possibly have access to the Internet, but their usage of the Internet is within a particular context.

Put that against the options nowadays. Internet via mobile and naked broadband or, only mobile phone access or even, no access to the Internet or phone.

I took great exception to their presentation this year though.

They had snapshots of 3 different "people" (extrapolated from their mean data I'm guessing rather than inferring to particular individuals) through their 3 different surveys (2 years apart).

In the first one, an early twenties Asian male didn't consider the radio important to his entertainment at all in which case, it appeared as a "negative" (a score of 1). The second one is where things got a little hinky. The Maori male, in his 60's, in 2007, had never used the Internet, and never intended to use the Internet. Yet, the importance of the Internet for that first year appears as a score of 3 - "neutral".

There was a question around the form of the media. i.e. if a newspaper is now read online, does that appear in the:

  1. Newspaper column?
  2. Internet column?
  3. Both?

I wasn't impressed by the research last year and I find myself even less impressed this year. It's kind of a really tragic version of the Berenstain Bears. If it costs around $20,000 per question for research of this type (a figure I heard recently), but the research is fundamentally flawed thus making it useless, what could that money have been used for? How many people could it have benefited? A perfect example of how not to do research.

No comments:

Post a Comment