While on Facebook earlier today (I was connecting with some colleagues on a work-related issue, honest!), I came across a survey for a local non-profit initiative. As someone who both identifies as a researcher and generally likes filling out surveys, I eagerly clicked the link … and found myself looking at ten open-ended, fill-in-the-blank questions.
Now, I don’t have anything against this style of question: indeed, as I noted in an earlier post, it’s good to provide space for respondents to share their own perspectives and stories without being boxed into a particular set of responses. In my opinion, though, inviting only written responses is a move too far in the other direction. Some respondents may not have the time to write down their thoughts, while others may feel pressured to provide insightful, well-crafted responses to each question and decide to take a pass on the survey as a result. I remember a conversation with a community group where one member personally disliked open-ended questions: this person’s view was perhaps a bit extreme, but it brings up the good point that individuals may simply have preferences for one question type over another. Accessibility is also a potential concern: will people who have low literacy skills or other challenges around writing feel comfortable participating? A final consideration is analyzing this type of data, which takes more time and effort compared to compiling statistics from multiple choice or rating questions.
Again, I have nothing against open-ended questions: depending on the intended audience and purpose of the survey, it may even be completely appropriate to only use that type of response. For most general surveys, though, a little bit of variety is probably a good thing.
4 replies on “What’s in a Question (type)?”
You make excellent points, especially with regard to respondent concerns. I recently saw a needs assessment instrument that contained all open-ended questions – maybe a dozen or so. I always wonder when I see open-ended questions whether the qualitative data collected is indeed systematically analyzed (not just scanned or read through). I’d love to see a survey on that! I strongly believe that question type choices should be based on information needs and feasibility of analysis. Hmmm…having written about survey questions myself, I’m starting to feel another blog post coming on! Thanks for the inspiration!
Thanks Sheila! You raise a great point about analyzing this data – if we only give a cursory glance to open-ended question responses, we run the risk of all sorts of biases in making sense of the data. Hey, that could be another post too!
Actually, that’s exactly where I was going with that thought. When we just read through narrative responses without a process for systematic analysis (i.e. coding), cognitive bias takes over and we tend to remember that with which we agree, or that which matches our internalized frameworks, understanding, etc.; also, we tend to remember or focus on responses in which strong feelings are expressed – the extremes, whether they are positive or negative.
Great minds 🙂 Looking forward to reading your post!