Dave's Adventures in Business Intelligence

I recently attended the 2009 TechEd conference. Here is a copy of the top portion of the session evaluation form. Notice anything that might cause a problem?

I know that form was reduced to fit the size of the page so it may not be immediately obvious what I think the issue is. In order to make it more clear I have posted a larger view of the specific survey choices.

Look at it for a bit and when you’re ready …

I spent several days filling out this evaluation form before I decided that I needed to read it closely. Unfortunately, here’s how I filled them out:

Now do you see the problem? 😉 And how it’s not the report writers fault? Once the data is collected, there’s no way to go back and redo it.

Still don’t see what my concern is?

The survey form is not consistent. In four of the five questions the best possible response is all the way to the right. However, for one question (the fourth one) that is not the case. The question in this case was related to how technical the content was. The best answer would be 3 or “Just right” instead of 5 for this one question. By the time I realized that, it was too late. I would love to see the overall speaker survey results, because I suspect I am not the only one making this mistake. Once I saw the pattern from the first few questions on the form, I made the (incorrect) assumption that the rest of the questions followed the same pattern. As a result of my error, the data from my survey submissions are wrong.

Because the form is (in my opinion) poorly designed, the data is likely to be wrong. If the data is wrong, the report will be wrong. Because the report is wrong, the speaker will be given the wrong indication as to the quality of their presentation. And it’s not the report writers fault.