Usability as:The issues to be considered while designing a good questionnaire:
Extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use. (from ISO 9241-11)
- Avoid positively-phrased statements (acquiescence bias).
- Reliability (信度): consistency and repeatability.
- Validity (效度): accuracy.
Other than questionnaires, we can adopt the alternative of the Microsoft Desirability Toolkit to control the tendency for the participants to rate their experience better than it actually was.
- Prepare a simple paper checklist of adjectives. For a particular study, replace some of the words with others that may be more relevant. Here is the list developed by Userfocus.
- Ask people to read through the words and select as many as they like that they think apply to the interface.
- Ask them to circle just 5 adjectives.
- The selected adjectives then became the basis of a post-test guided interview.
- 根據研究目的，寫出一份內容適合的形容詞列表（參考英文版）。亦可加入品牌/產品形象用字。
- 請使用者瀏覽並選擇他們認為符合該產品/介面的字詞。
- 請使用者從他們選出來的字詞中，再圈選出五個。
- 這五個字詞是測後訪談的討論重點。先前加入的品牌/產品形象用字，此時可一一和使用者討論他們選/不選、認同/反對的原因。
"I see that one of the cards you selected was 'Sophisticated'. Tell me what was behind your choice of that word."
This approach is ideal as a qualitative approach to guide an interview. You can also derive metrics from these data via word cloud (try Wordle) and verbal protocol analysis (to count the percentage of positive and negative comments.)
Also apply randomisation or counterbalancing approach to generate the word list to prevent order effects.
(Source: Measuring satisfaction: Beyond the usability questionnaire by David Travis.)
