Strengthening Design Research Moderating Skills
-
Upload
susan-mercer -
Category
Design
-
view
113 -
download
1
description
Transcript of Strengthening Design Research Moderating Skills
![Page 1: Strengthening Design Research Moderating Skills](https://reader034.fdocument.pub/reader034/viewer/2022051819/54c7236c4a7959cf018b45ce/html5/thumbnails/1.jpg)
APRIL 9, 2013
Susan Mercer Senior Experience Researcher [email protected] @susanamercer
Boston CHI Monthly Mee>ng
STRENGTHENING DESIGN RESEARCH MODERATION SKILLS
![Page 2: Strengthening Design Research Moderating Skills](https://reader034.fdocument.pub/reader034/viewer/2022051819/54c7236c4a7959cf018b45ce/html5/thumbnails/2.jpg)
Strengthening Moderating and Interviewing Skills
§ Once I learned the basics
§ And ran a couple of studies,
§ Now what?
§ How do I strengthen my skills?
2
![Page 3: Strengthening Design Research Moderating Skills](https://reader034.fdocument.pub/reader034/viewer/2022051819/54c7236c4a7959cf018b45ce/html5/thumbnails/3.jpg)
Story Time
Don’t play the notes on the page…
…play the music
This applies to moderating and
interviewing too.
Don’t just read the questions from the page…
…have a conversation
3
![Page 4: Strengthening Design Research Moderating Skills](https://reader034.fdocument.pub/reader034/viewer/2022051819/54c7236c4a7959cf018b45ce/html5/thumbnails/4.jpg)
The Art of Moderating and Interviewing
§ Not just reading questions
§ It’s also…
§ Building rapport with the participant
§ Creating a smooth conversation
§ Encouraging the participant to talk
§ Remaining objective
§ Listening effectively
§ Being human
4
![Page 5: Strengthening Design Research Moderating Skills](https://reader034.fdocument.pub/reader034/viewer/2022051819/54c7236c4a7959cf018b45ce/html5/thumbnails/5.jpg)
What Is Our Goal?
§ To elicit honest thoughts from participants in an unbiased manner.
§ “Would you like to try one of my chocolate chip oyster and herring brownies? They’re really good!”
§ Sometimes it’s hard to be fully honest
5
![Page 6: Strengthening Design Research Moderating Skills](https://reader034.fdocument.pub/reader034/viewer/2022051819/54c7236c4a7959cf018b45ce/html5/thumbnails/6.jpg)
The Rules of Politeness
Linguistic anthropologists have studied this phenomenon
The Rules of Politeness 1
1. Don’t Impose
2. Give Options
3. Be Friendly
6 Sources: 1 Lakoff, 1976; 2 Tannen, 2005
“Human beings are always balancing the paradoxical fact that they are simultaneously individuals and social creatures.”
-‐ Deborah Tannen
![Page 7: Strengthening Design Research Moderating Skills](https://reader034.fdocument.pub/reader034/viewer/2022051819/54c7236c4a7959cf018b45ce/html5/thumbnails/7.jpg)
How Do We Get Beyond Politeness?
§ In other words, we’re often polite rather than honest to strangers.
§ As human beings we have a need to be socially accepted
Social Acceptance > Honesty
§ We guess what is expected and answer to avoid offending
§ It’s second nature – we’re often not aware we’re doing it
7
![Page 8: Strengthening Design Research Moderating Skills](https://reader034.fdocument.pub/reader034/viewer/2022051819/54c7236c4a7959cf018b45ce/html5/thumbnails/8.jpg)
How Do We Get Beyond Politeness?
Build their trust
8
Politeness Trust
Strangers Good Friend
Familiarity
![Page 9: Strengthening Design Research Moderating Skills](https://reader034.fdocument.pub/reader034/viewer/2022051819/54c7236c4a7959cf018b45ce/html5/thumbnails/9.jpg)
Building Trust in an Interview
1. Be trustworthy upfront
2. Build rapport
3. Engage in comfortable conversation
4. Stay neutral and accepting
9
![Page 10: Strengthening Design Research Moderating Skills](https://reader034.fdocument.pub/reader034/viewer/2022051819/54c7236c4a7959cf018b45ce/html5/thumbnails/10.jpg)
1. Be Trustworthy Upfront
§ Confidentiality
§ How are you recording what is said?
§ Who is listening?
§ What will you do with the information you collect?
§ Neutrality
§ You’re not emotionally involved in the design/project
§ There are no right or wrong answers
§ Your job is to get honest opinions
§ Consent Form
§ Put it all in writing – using their language
![Page 11: Strengthening Design Research Moderating Skills](https://reader034.fdocument.pub/reader034/viewer/2022051819/54c7236c4a7959cf018b45ce/html5/thumbnails/11.jpg)
2. Build Rapport
§ Greeting a Participant
§ Smile
§ Use their name
§ Handshake / Warm phone greeting
§ Make them feel important - sincerely
§ Small Talk – Find Common Ground
§ Safe topics: travel to office, traffic, weather
§ Avoid asking direct questions
§ Listen and look for shared experiences
A person's name is to that person the sweetest and most important sound in any language.
-‐ Dale Carnegie
Source: Carnegie, 1936.
![Page 12: Strengthening Design Research Moderating Skills](https://reader034.fdocument.pub/reader034/viewer/2022051819/54c7236c4a7959cf018b45ce/html5/thumbnails/12.jpg)
2. Build Rapport
§ Be Empathetic
§ Apologize if they had trouble finding the office
§ Show you understand their point of view
§ “Oh, it’s raining there? It is here too. I hate rainy days.”
§ Inject Some Humor
§ Joke about yourself
§ Joke about the situation
§ Don’t joke about them
§ Short version – Show them that you are human too
![Page 13: Strengthening Design Research Moderating Skills](https://reader034.fdocument.pub/reader034/viewer/2022051819/54c7236c4a7959cf018b45ce/html5/thumbnails/13.jpg)
3. Engage in Comfortable Conversation
Two schools of usability testing moderating techniques
§ Moderator keeps silent and says “Keep talking” 1
§ Awkward feeling for participant
§ Reminds them that they are in a “study”
§ Moderator creates conversation where participant is primary speaker 2
§ Still accomplishes goals of gathering information
§ Meets the participant’s expectations of a socially acceptable conversation
§ Moderator can still be neutral and minimize bias
13 Sources: 1 Ericsson and Simon, 1980, 2 Boren and Raney, 2000.
![Page 14: Strengthening Design Research Moderating Skills](https://reader034.fdocument.pub/reader034/viewer/2022051819/54c7236c4a7959cf018b45ce/html5/thumbnails/14.jpg)
3. Engage in Comfortable Conversation
What is a Comfortable Conversation?
§ Conversational cues and turn-taking are expected
§ Acknowledgement tokens – “Uh huh”, etc.
§ Encourage the continuation of the other speaker’s talk
§ Usually implies that the other speaker’s prior talk is incomplete
§ Some indicate change of speakership
§ “Yeah” – more than half the time indicates changing turns in speaking
§ “Yeah. I heard that the other day and…”
§ “Oh!” - may indicate noticing something, then transitioning to another topic
§ “Oh! That reminded me…”
![Page 15: Strengthening Design Research Moderating Skills](https://reader034.fdocument.pub/reader034/viewer/2022051819/54c7236c4a7959cf018b45ce/html5/thumbnails/15.jpg)
3. Engage in Comfortable Conversation
§ Some may introduce bias
§ “Oh!”, “Interesting” – indicating unexpected answer
§ “Yes”, “Perfect”, “Great” – indicating agreement
§ “Hmmm.”, “Really?” – indicating disagreement
§ Notice that tone is key
§ Neutral is best
§ “Mhmm”, “Uh huh”, “Continue”, “Tell me more”, “OK”
§ “Mhmm” or “Uh huh” vs. silence à interviewees saying 31% more phrases. 1
Source: 1 Matarazzo et. al., 1964
![Page 16: Strengthening Design Research Moderating Skills](https://reader034.fdocument.pub/reader034/viewer/2022051819/54c7236c4a7959cf018b45ce/html5/thumbnails/16.jpg)
4. Stay Neutral and Accepting
§ Ask Open-ended Questions
§ Start with Who, What, When, Where, Why, How?
§ Really Listen
§ Pay attention – stay in the moment
§ Look at the participant
§ Take notes if you can
§ Be quiet - give them time to say what they need to
![Page 17: Strengthening Design Research Moderating Skills](https://reader034.fdocument.pub/reader034/viewer/2022051819/54c7236c4a7959cf018b45ce/html5/thumbnails/17.jpg)
4. Stay Neutral and Accepting
§ Watch Your Reactions
§ Don’t show surprise
§ May make them think that they are giving a wrong answer
§ Don’t overly agree
§ May make them think that they are giving the right answer
§ Don’t be negative
§ Watch your tone – stay neutral and accepting
§ Try not to laugh
![Page 18: Strengthening Design Research Moderating Skills](https://reader034.fdocument.pub/reader034/viewer/2022051819/54c7236c4a7959cf018b45ce/html5/thumbnails/18.jpg)
4. Stay Neutral and Accepting
§ Be yourself
§ No one is perfectly neutral
§ Recover gracefully and move on
§ “Perfect” – “That’s the level of detailed feedback we’re looking for.”
§ “Interesting!” – “I haven’t heard that perspective yet, tell me more.”
§ (something surprising) – “I can understand that.”
§ Interject some Rapport-building comments when needed
§ Quiet or uncomfortable participants
§ “I hate it when that happens.”, “I can imagine that was challenging”, etc.
§ Again, showing that you are human like them
![Page 19: Strengthening Design Research Moderating Skills](https://reader034.fdocument.pub/reader034/viewer/2022051819/54c7236c4a7959cf018b45ce/html5/thumbnails/19.jpg)
5. Encourage them to talk
§ Be Quiet!
§ Most agreements happen immediately. Most people delay before disagreeing.1
§ If you don’t respond to their answer, it encourages them to talk more
§ People often delay speaking before disagreeing – give them time
§ Some people are uncomfortable with silence, so they will keep talking
§ The best way to stay neutral J
Source: 1 Goodwin and Heritage, 1990.
![Page 20: Strengthening Design Research Moderating Skills](https://reader034.fdocument.pub/reader034/viewer/2022051819/54c7236c4a7959cf018b45ce/html5/thumbnails/20.jpg)
Building Trust in an Interview
1. Be trustworthy upfront
2. Build rapport
3. Engage in comfortable conversation
4. Stay neutral and accepting
5. Be Quiet
20
![Page 21: Strengthening Design Research Moderating Skills](https://reader034.fdocument.pub/reader034/viewer/2022051819/54c7236c4a7959cf018b45ce/html5/thumbnails/21.jpg)
KEEPING YOUR SKILLS SHARP
21
![Page 22: Strengthening Design Research Moderating Skills](https://reader034.fdocument.pub/reader034/viewer/2022051819/54c7236c4a7959cf018b45ce/html5/thumbnails/22.jpg)
1. Identify your improvement areas
§ Watch your videos
§ Have colleagues give you feedback
22
![Page 23: Strengthening Design Research Moderating Skills](https://reader034.fdocument.pub/reader034/viewer/2022051819/54c7236c4a7959cf018b45ce/html5/thumbnails/23.jpg)
2. Learn From Others
§ Watch others moderate
§ Be a participant
§ Listen to talk radio interviews
23
![Page 24: Strengthening Design Research Moderating Skills](https://reader034.fdocument.pub/reader034/viewer/2022051819/54c7236c4a7959cf018b45ce/html5/thumbnails/24.jpg)
3. Practice These Skills in Everyday Life
§ Be quiet and listen in everyday conversations
§ Ask open-ended questions more frequently
24
![Page 25: Strengthening Design Research Moderating Skills](https://reader034.fdocument.pub/reader034/viewer/2022051819/54c7236c4a7959cf018b45ce/html5/thumbnails/25.jpg)
The most important thing is…
25
PRACTICE!
![Page 26: Strengthening Design Research Moderating Skills](https://reader034.fdocument.pub/reader034/viewer/2022051819/54c7236c4a7959cf018b45ce/html5/thumbnails/26.jpg)
References
Boren, T. and Ramey, J. (2000) Thinking aloud: reconciling theory and practice. IEEE Transactions on Professional Communication, 43 (3), 261-278.
Carnegie, D. (1936) How to win friends and influence people. Simon & Shuster.
Dumas, J. and Loring, B. (2008) Moderating usability tests, Morgan Kaufman.
Dumas, J. and Redish, J. (1999) A practical guide to usability testing, Intellect Ltd.
Ericsson, K. and Simon, H. (1980) Verbal reports as data. Psychological review. 87 (3), 215-251.
Goodwin, C., & Heritage, J. (1990). “Conversation analysis.” Annual review of anthropology 19 (1990): 283-307.
Matarazzo, J.D.., Wiens, A. N., Saslow, G., Allen, B. V., & Weitman, M. (1064). Interviewer Mm-Hmm and interviewee speech durations. Psychotherapy: Theory, Research & Practice, 1(3), 109.
Tannen, D. (2005). Conversational Style: Analyzing Talk among Friends. Oxford University Press, USA.
26
![Page 27: Strengthening Design Research Moderating Skills](https://reader034.fdocument.pub/reader034/viewer/2022051819/54c7236c4a7959cf018b45ce/html5/thumbnails/27.jpg)
Coming Soon!
New book on Moderating
• Local Authors
• Coming this Fall
• Follow @ModSurvivalUX
27
![Page 28: Strengthening Design Research Moderating Skills](https://reader034.fdocument.pub/reader034/viewer/2022051819/54c7236c4a7959cf018b45ce/html5/thumbnails/28.jpg)
28
Questions?