Testing Techniques & Interview Guides

Why we interview

Right questions for the right answers

Creating interview guides

1

2

3

Why we interviewAt it’s best, interviews will look for and explore all

the stuff numbers can’t tell you

There aren’t numbers for everything and that’s the dead simple reason why

qualitative research is important.

Data can tell us what’s happening, but qualitative research explains why

‘Why’ is a hard question to ask quantitatively

Data showed that only 4% of people interacted with this area.

But Why?

“this looks like the weather"

“I’m not sure what

this is”

“I didn’t know you could tap

on this”

Research helps us understand peoples needs and usability issues,

ultimately showing us what problems we should solve

Empathy helps us understand

the problems of others.

What design research interviews are not

Statistically significant

A way to determine purchasing intent

Asking people what they want

Market research

You can better understand the problem space by not only explaining ‘what’ is

happening with Data but explaining ‘why’ with research insights

The right questions, the right answersInterview to learn about current

behaviors not attitudes or opinions

“If I had asked people what they

wanted, they would have said faster

horses”

-Henry Ford

“If I had asked people what they

wanted, they would have said faster

horses”

-Henry Ford didn’t actually say this

but you should know this quote

It’s the users job to tell us about their problems,

it’s our job to solve problems

We don’t act a the user’s request, we are acting on their behalf

Poor questions tell you what you want to hear

Rather than getting real insight into user needs

“would you use this if…..”

asking hypothetical questions about the future

“so what do you want?”

directly pitching your idea

Here are two types of questions

to avoid but seem intuitive

…If you ask customers if they think stores

are too cluttered, the answer is a

predictable yes.

“Walmart can’t escape clutter, can you?” New York Times

What is striking is the wide gap

between what users say they do, and

what they actually do.

“Walmart can’t escape clutter, can you?” New York Times

What are people’s current behaviors

and why?

Is our product or site usable?

What are people’s current behaviors

and why?

1. What are people trying to do?

2. How do they currently do this?

3. What could be a better way to do this?

Interview to learn about current behaviors not attitudes or opinions

Goals & Motivations:

What are they trying to accomplish?

Do their goals ever change?

What tasks do they need to achieve their goals?

Behaviors:

How do they currently accomplish their goals?

What are their workarounds?

Habits:

What are their natural habits?

Pain Points:

What makes their experience bad or unsuccessful? How often?

What other services handle their pain points better?

FRAMEWORK FOR INTERVIEWS

TO LEARN ABOUT BEHAVIOR &

PROBLEMS WITHOUT DIRECTLY ASKING FOR SOLUTIONS

2. Can you tell me the last time that happened?

3. Why was that hard?

4. How did you solve that problem?

5. Why was your solution not awesome?

1. What was the hardest part about…?

Meet Dave, a tech

startup founder looking

for an HR solution

“Knowing the bare minimum of what I had to do”

“When I hired more people and wanted my company to be legit”

“Everything was way too technical, I didn’t understand most of the terms”

“Felt like lawyers were reaming you every second”

“I paid lawyers to help me set it up”

2. Can you tell me the last time that happened?

3. Why was that hard?

4. How did you solve that problem?

5. Why was your solution not awesome?

1. What was the hardest part about…?

What are people’s current behaviors

and why?

Is our product or site usable?

Is our product or site usable?

1 2 3

??

!

+

+

!!

Usability is how someone interacts with a site or product

alert

alert

alert alert

alert

Observing is just as important as listening during usability testing

INTERVIEW FRAMEWORK

FOR LEARNING ABOUT MENTAL

MODELS

“where do you go next?” 3. BEFORE EACH STEP ASK..

“what do you expect will happen?”4.…AND THEN ASK

1. SHOW YOUR PRODUCT OR PROTOTYPE

2.GIVE A SAMPLE TASK

5. KEEP AN EYE OUT FOR ANY CONFUSION

“In 25 years of doing this, in virtually every usability test I’ve

seen, we knew what the top 3 problems were once the final user had left the

session.”

- the 7 deadly sins of user research, David Travis

Interview GuidesOver 50% of a success of a research interview

happens before you ask the first question.

How many people should we speak with?

5 - 20 carefully selected people

You’ll start seeing clear patterns after speaking with a few carefully selected people.

But it depends on the project and quality of the test

1:3

One interviewer should not talk to more than three users

Always run sessions in pairs

One Interviewer

One Observer

Always run sessions in pairs

One question asker

One note taker

Remember your hypothesis

Choose your hypothesizes carefully

Choose your participants carefully

and what to do next will almost always reveal itself quickly

Interviews are structured conversations

Interview Guides, not

scripts

Lets talk about how to structure your time for 1 hour interviews

Introductions: 5 minutes

Introduce yourself

Introduce the project

Sign consent forms

Tell them why you’re interviewing people like them

Give them a chance to ask any questions

Get to know them: 5 minutes

Ask about relevant questions to your project

How long have they paid for cable?

What are their favorite shows?

What’s a typical day like for you?

Focused questions: 45 minutes

Ask specific questions about needs ,workarounds, and behaviors

Take the user through your site or prototype asking questions along the way

Wrap Up: 5 minutes

Is there anything we should have asked you but didn’t?

Do you have any questions for us?

Don’t ask leading questions

Don’t put the answer in the question

Don’t ask people what they want

Don’t ask hypothetical questions

Don’t ask yes or no questions

👥Interview pairs in pairs

👤You’re not your user

⏰Wait 5 seconds before

asking another question

$One interviewee per three people

↻Repeat back

what they say

☻Bring gum

📷Always be documenting

(Offer a drink

🌙Leave time to linger after

the interview