I don't know about you, but I take Internet surveys on a regular basis. Yougov.com allows you to trade in points for goods or gift cards. E-rewards.com allows you to add money to your Upromise account (kids' college savings--yay!). And e-miles.com allows you to earn airline miles so you don't have to keep shopping to keep your miles active (I can't say I've been flying as much as before kids, but I want to keep them active). I don't mind taking these surveys. They can provide useful information to companies and government. However, when they are poorly thought out/written, the data they gather isn't going to be that useful. I suspect the people writing these surveys are just too homogeneous a group to think outside their box. Maybe, they'll stumble upon this and learn a little more about their survey takers. Or maybe not. At any rate, yougov.com does the best job on their surveys BECAUSE they allow you to evaluate the survey at the end, including a comment section, and they frequently will allow you to leave an item blank. But anyway, onto the problems.
In my experience, there are a few areas where survey developers frequently fail. The number one thing is early in the survey: they show their sexism. Seriously. At least 90% (probably closer to 95%) of the surveys I see list "male" before "female." You would think they would try to make questions neutral, such as doing things alphabetically (which they frequently do on race questions, though a few years ago they weren't as good about it). In that case, it would be the opposite or they could use man and woman alphabetically. Another neutral way to do it would to make it randomly alternate with each survey you develop. But if they were doing it this way, about 50% of the surveys should list male before female. If you want to do it by adult population (which should be those taking the surveys), the last census data I looked at showed that there is a slightly higher percentage of females living in the U.S., where most of the surveys are based, so again, "female" should come first if you're going to base it on a neutral way. Instead, they base it on what comes to their mind first: men. It's sexist. Find a better way to do it.
The other big thing is they often don't have an option for "don't know" or "none of the above." Usually, survey developers aren't so perfect as to think of every possible answer. They should acknowledge this by giving an "out" option. Especially when they just asked a question to which your answer said that the next question clearly wouldn't apply to the first. Also, they rarely seem to acknowledge that some questions just aren't applicable to everyone. Not everyone has a favorite sports team. Some people hate all sports. Give people the option to be honest.
Which leads to the problem of inconsistency within the survey. If a person just told you they don't have _____, and your next question is about their _____, then you need to either have a "doesn't apply" option or to program your survey to skip that question. Most of these surveys do not allow you to move on until you've answered. So, unless you want people to just insert random answers, you should have a way to answer the question if it doesn't apply.
Another problem: poorly worded questions. I've had some of the vaguest questions asked of me in surveys. Often, context would change my answer. So when you ask me a vague question and don't even give me a "not sure" or similar answer, my answer will either give you inaccurate data or I will quit your survey, depending on how strongly I feel or on how aggravating your survey has been.
No comments:
Post a Comment