Questionnaire Design

A QUESTIONNAIRE IS ONLY AS GOOD

AS THE QUESTIONS IT ASKS

A questionnaire ensures standardization and comparability of the data across interviews increases speed and accuracy of recording

Facilitates data processingAlso allows the researcher to collect the

relevant information necessary to address the management decision problem

Purposes of the Questionnaire

Logical Steps to Develop a Good Questionnaire

bull Recall the research objective the research questions and hypotheses

bull Identify the variables to be measuredbull Formulate questions (items in the questionnaire)bull Order and wording of questions and the layout of the

questionnairebull Test for omissions and ambiguitybull Correct the problems (pretest again if necessary)

What Should Be Asked

Questionnaire RelevancyA questionnaire is relevant to the extent that all information collected addresses a research question that will help the decision maker address the current business problem

Questionnaire AccuracyAccuracy means that the information is reliable and valid

Validity and reliability

Validitybull question measure what you claim it

measures problem with self-report Reliabilitybull results are reproducible or consistent with

similar groups of respondents over time and when other people administer the questionnaire

Type of information

bull 1048715Knowledge - what people knowbull 1048715Opinions attitudes beliefs values -

what people think about an issuebull 1048715Behaviour - what people dobull 1048715Attributes - what are peoplersquos

characteristicsbull 1048715Remember - based on self-report

KnowledgeWhat is the recommended intervalbetween eye checks for patients withuncomplicated diabetesbull 1048715 6 monthsbull 1048715 1 yearbull 1048715 2 yearsbull 1048715 Not sure

Attributesbull When did you graduate from university_______________________

Opinions What do you think are the major issuesaffecting general practice in Australia atthe moment________________________________

BehaviourHave you developed a care plan for any ofyour patientsbull 1048715 Yesbull 1048715 No

Types of Questions1 Open-Ended question

People look for different things in a job What would you prefer most in a job

2 Closed-Ended questionPeople look for different things in a job What would you prefer most in a jobndash Work that pays wellndash Work that gives a sense of accomplishmentndash Work where you make most decisions by yourselfndash Work that is steady with little chance of being laid

off

Open Ended Questionsbull Advantages

ndash Gain insight into the problemndash Too many options to listndash When verbatim responses are desired to give the flavor of the problemndash When behavior to be measured is sensitive or disapprovedndash Interviewer questionnaire structure influence can be minimal

bull Disadvantagesndash Inarticulate respondentsndash Interviewerrsquos skill in recording quickly and summarizing accuratelyndash Time consuming subjective judgments while tabulating adds to cost

Closed-response Questions

Two Basic Formats for Closed Ended or Structured Questions

bull Choice from a list of responses

bull Appropriate single-choice rating on a scale

Closed-response Questions

What type of fast-food restaurant do you visit most often

Burger MexicanChicken PizzaSeafood ChineseDonrsquot know Other (please specify)

Closed-response Questions

What is your overall satisfaction with McDonalds burgers

Very satisfied Quite Satisfied Somewhat satisfied Not at all satisfied

Very satisfied 7 6 5 4 3 2 1 Not at all satisfied

Closed-Ended Questionsbull Advantages

ndash Easy to understand quick responses possiblendash Micro-differences in responses can be capturedndash Easier tabulation and analysisndash Answers are directly comparable from respondent to respondent

bull Disadvantagesndash Neutral category may attract more responses than warrantedndash Information between categories may be lost (extreme case ndash

dichotomous categories)

Issues in Questionnaire Design

bull Respondent uncertainty

ndash Should respondents be provided with aDonrsquot know or No opinion option

ndash When it is important to differentiate between ambivalence and ignorance both options should be provided

Contents of the Questionnaire bull Question Wording ndash Vocabulary

- Depends on the nature of variable tapped- subjective feeling or objective facts

ndash Simple easy to understand commonly used language

ndash Avoid technical words and jargon (unless sample is technically qualified)

Filter questions Filter questions useful to ensure respondents

only answer relevant parts of questionsUnfilteredbull ndash If you use a medical software programbull which one do you use

Filteredbull Do you use a medical software programbull 1048715 No - jump to next questionbull 1048715 Yes - which one

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided------------------------------------------------------------------------------

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Issues in Questionnaire Design

bull Question Wordingbull vocabulary bull Questions need to be clear simple and

precisebull Poorly written questions lead to ambiguity

and misunderstandings and can be wasteful

bull Avoid Complexity use simple conversational language

bull Avoid leading and loaded questions

bull Avoid ambiguity be as specific as possible

bull Avoid double-barreled items

bull Avoid making assumptions

bull Avoid burdensome questions

Double-barrelled questionsTwo concepts in one question

bull Have you had a neck ache or a back achesince your last visit

bull Since your last visit have you had any of the following symptoms (tick as many that apply)

bull Neck ache 1048715 bull Back ache 1048715 bull Headache 1048715

Leading and Loaded questionsbull A leading question suggests or implies certain

answersDo accounting graduates who attended state universities such as Washington State University make better auditors

bull A loaded question suggests a socially desirable answer or is emotionally charged

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Vague questions

Avoid vague questionsbull Taken altogether how happy are you with

your stay in hospitalbull Overall how would you describe the care

you received in hospital

Avoid Ambiguity Words such as ldquooftenrdquo ldquooccasionallyrdquo ldquousuallyrdquo ldquoregularlyrdquo ldquofrequentlyrdquo ldquomanyrdquo should be used with caution If these words have to be used their meaning should be explained properly

Avoid Making Assumptionsbull Is the question applicable to all

respondents

Why do you like fast-food

Assumes respondents like fast foodsBetter strategy would be to ask a filter

question first

Burdensome QuestionsAvoid Burdensome Questions That MayTax the Respondentrsquos Memory telephone

A survey conducted during the 24-hour period following the airing of the Super Bowl might establish whether the respondent watched the Super Bowl and then ask ldquoDo you recall any commercials on that programrdquo

Question Wordingbull Question length

ndash Should be shortndash Longer questions confuse and fatigue respondents

bull Sensitive questionsndash Questions on information perceived to be

embarrassing like personal income criminal activities alcoholism smoking drugs habits social desirability issues etc

ndash Creativity rules (assurances of confidentiality anonymity slipping it in sideways open-ended questions asking in third person etc)

Sequence And Layout Decisions

bull Opening questions ndash easy and non-threateningbull Flow ndash smooth and logical ndash avoid jumpsbull Broad to specificbull Critical questions ndash placed in the middle bull Appealing and interestingbull Order bias ndash the possibility that subsequent responses

are influenced by preceding responses eg fewer people will say that their taxes are too high after being asked whether govt spending should be increased in certain areas

bull Demographic questions ndash first or last

Layout

bull 1048715 Just as important as wordingbull 1048715 Aim for a professional lookbull 1048715 Tipsbull ndash cover letterintroductory page giving study title organisation aims of the surveybull ndash enough space for open-ended questionsbull ndash font large enough to read without strainbull ndash consistent and clear instructionsbull ndash donrsquot split questions or answers across pagesbull ndash enough white space

Distinguishing question and responses

LAYOUT ADo you agree disagree or have no opinion that this company

has bull A good vacation policy - disagreenot sureagreebull Good management feedback - disagreenot sureagreebull Good medical insurance - disagreenot sureagree

LAYOUT B Does this company have bull Disagree Not Sure Agree bull A good vacation policy 1 2 3 bull Good feedback 1 2 3 bull Good medical insurance 1 2 3

Pretesting and Correcting Problems

Purpose of pretest To ensure that the questionnaire meets the expectations in terms of the information that will be obtained

bull Pretesting Specific Questions Forbull Meaningbull Task difficultybull Respondent interest and attention

bull Pretesting the overall Questionnairebull Flow of the questionnairebull Skip patternsbull Length

bull Put yourself in the respondentrsquos shoes and answer the questionnaire

• Questionnaire Design
• A QUESTIONNAIRE IS ONLY AS GOOD AS THE QUESTIONS IT ASKS
• Slide 3
• Logical Steps to Develop a Good Questionnaire
• What Should Be Asked
• Validity and reliability
• Type of information
• Knowledge
• Opinions
• Types of Questions
• Open Ended Questions
• Closed-response Questions
• Closed-response Questions (2)
• Closed-response Questions (3)
• Closed-Ended Questions
• Issues in Questionnaire Design
• Contents of the Questionnaire
• Filter questions
• Slide 19
• Issues in Questionnaire Design (2)
• Slide 21
• Double-barrelled questions
• Leading and Loaded questions
• Slide 24
• Vague questions
• Avoid Making Assumptions
• Burdensome Questions
• Question Wording
• Sequence And Layout Decisions
• Layout
• Distinguishing question and responses
• Pretesting and Correcting Problems

A QUESTIONNAIRE IS ONLY AS GOOD

AS THE QUESTIONS IT ASKS

A questionnaire ensures standardization and comparability of the data across interviews increases speed and accuracy of recording

Facilitates data processingAlso allows the researcher to collect the

relevant information necessary to address the management decision problem

Purposes of the Questionnaire

Logical Steps to Develop a Good Questionnaire

bull Recall the research objective the research questions and hypotheses

bull Identify the variables to be measuredbull Formulate questions (items in the questionnaire)bull Order and wording of questions and the layout of the

questionnairebull Test for omissions and ambiguitybull Correct the problems (pretest again if necessary)

What Should Be Asked

Questionnaire RelevancyA questionnaire is relevant to the extent that all information collected addresses a research question that will help the decision maker address the current business problem

Questionnaire AccuracyAccuracy means that the information is reliable and valid

Validity and reliability

Validitybull question measure what you claim it

measures problem with self-report Reliabilitybull results are reproducible or consistent with

similar groups of respondents over time and when other people administer the questionnaire

Type of information

bull 1048715Knowledge - what people knowbull 1048715Opinions attitudes beliefs values -

what people think about an issuebull 1048715Behaviour - what people dobull 1048715Attributes - what are peoplersquos

characteristicsbull 1048715Remember - based on self-report

KnowledgeWhat is the recommended intervalbetween eye checks for patients withuncomplicated diabetesbull 1048715 6 monthsbull 1048715 1 yearbull 1048715 2 yearsbull 1048715 Not sure

Attributesbull When did you graduate from university_______________________

Opinions What do you think are the major issuesaffecting general practice in Australia atthe moment________________________________

BehaviourHave you developed a care plan for any ofyour patientsbull 1048715 Yesbull 1048715 No

Types of Questions1 Open-Ended question

People look for different things in a job What would you prefer most in a job

2 Closed-Ended questionPeople look for different things in a job What would you prefer most in a jobndash Work that pays wellndash Work that gives a sense of accomplishmentndash Work where you make most decisions by yourselfndash Work that is steady with little chance of being laid

off

Open Ended Questionsbull Advantages

ndash Gain insight into the problemndash Too many options to listndash When verbatim responses are desired to give the flavor of the problemndash When behavior to be measured is sensitive or disapprovedndash Interviewer questionnaire structure influence can be minimal

bull Disadvantagesndash Inarticulate respondentsndash Interviewerrsquos skill in recording quickly and summarizing accuratelyndash Time consuming subjective judgments while tabulating adds to cost

Closed-response Questions

Two Basic Formats for Closed Ended or Structured Questions

bull Choice from a list of responses

bull Appropriate single-choice rating on a scale

Closed-response Questions

What type of fast-food restaurant do you visit most often

Burger MexicanChicken PizzaSeafood ChineseDonrsquot know Other (please specify)

Closed-response Questions

What is your overall satisfaction with McDonalds burgers

Very satisfied Quite Satisfied Somewhat satisfied Not at all satisfied

Very satisfied 7 6 5 4 3 2 1 Not at all satisfied

Closed-Ended Questionsbull Advantages

ndash Easy to understand quick responses possiblendash Micro-differences in responses can be capturedndash Easier tabulation and analysisndash Answers are directly comparable from respondent to respondent

bull Disadvantagesndash Neutral category may attract more responses than warrantedndash Information between categories may be lost (extreme case ndash

dichotomous categories)

Issues in Questionnaire Design

bull Respondent uncertainty

ndash Should respondents be provided with aDonrsquot know or No opinion option

ndash When it is important to differentiate between ambivalence and ignorance both options should be provided

Contents of the Questionnaire bull Question Wording ndash Vocabulary

- Depends on the nature of variable tapped- subjective feeling or objective facts

ndash Simple easy to understand commonly used language

ndash Avoid technical words and jargon (unless sample is technically qualified)

Filter questions Filter questions useful to ensure respondents

only answer relevant parts of questionsUnfilteredbull ndash If you use a medical software programbull which one do you use

Filteredbull Do you use a medical software programbull 1048715 No - jump to next questionbull 1048715 Yes - which one

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided------------------------------------------------------------------------------

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Issues in Questionnaire Design

bull Question Wordingbull vocabulary bull Questions need to be clear simple and

precisebull Poorly written questions lead to ambiguity

and misunderstandings and can be wasteful

bull Avoid Complexity use simple conversational language

bull Avoid leading and loaded questions

bull Avoid ambiguity be as specific as possible

bull Avoid double-barreled items

bull Avoid making assumptions

bull Avoid burdensome questions

Double-barrelled questionsTwo concepts in one question

bull Have you had a neck ache or a back achesince your last visit

bull Since your last visit have you had any of the following symptoms (tick as many that apply)

bull Neck ache 1048715 bull Back ache 1048715 bull Headache 1048715

Leading and Loaded questionsbull A leading question suggests or implies certain

answersDo accounting graduates who attended state universities such as Washington State University make better auditors

bull A loaded question suggests a socially desirable answer or is emotionally charged

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Vague questions

Avoid vague questionsbull Taken altogether how happy are you with

your stay in hospitalbull Overall how would you describe the care

you received in hospital

Avoid Ambiguity Words such as ldquooftenrdquo ldquooccasionallyrdquo ldquousuallyrdquo ldquoregularlyrdquo ldquofrequentlyrdquo ldquomanyrdquo should be used with caution If these words have to be used their meaning should be explained properly

Avoid Making Assumptionsbull Is the question applicable to all

respondents

Why do you like fast-food

Assumes respondents like fast foodsBetter strategy would be to ask a filter

question first

Burdensome QuestionsAvoid Burdensome Questions That MayTax the Respondentrsquos Memory telephone

A survey conducted during the 24-hour period following the airing of the Super Bowl might establish whether the respondent watched the Super Bowl and then ask ldquoDo you recall any commercials on that programrdquo

Question Wordingbull Question length

ndash Should be shortndash Longer questions confuse and fatigue respondents

bull Sensitive questionsndash Questions on information perceived to be

embarrassing like personal income criminal activities alcoholism smoking drugs habits social desirability issues etc

ndash Creativity rules (assurances of confidentiality anonymity slipping it in sideways open-ended questions asking in third person etc)

Sequence And Layout Decisions

bull Opening questions ndash easy and non-threateningbull Flow ndash smooth and logical ndash avoid jumpsbull Broad to specificbull Critical questions ndash placed in the middle bull Appealing and interestingbull Order bias ndash the possibility that subsequent responses

are influenced by preceding responses eg fewer people will say that their taxes are too high after being asked whether govt spending should be increased in certain areas

bull Demographic questions ndash first or last

Layout

bull 1048715 Just as important as wordingbull 1048715 Aim for a professional lookbull 1048715 Tipsbull ndash cover letterintroductory page giving study title organisation aims of the surveybull ndash enough space for open-ended questionsbull ndash font large enough to read without strainbull ndash consistent and clear instructionsbull ndash donrsquot split questions or answers across pagesbull ndash enough white space

Distinguishing question and responses

LAYOUT ADo you agree disagree or have no opinion that this company

has bull A good vacation policy - disagreenot sureagreebull Good management feedback - disagreenot sureagreebull Good medical insurance - disagreenot sureagree

LAYOUT B Does this company have bull Disagree Not Sure Agree bull A good vacation policy 1 2 3 bull Good feedback 1 2 3 bull Good medical insurance 1 2 3

Pretesting and Correcting Problems

Purpose of pretest To ensure that the questionnaire meets the expectations in terms of the information that will be obtained

bull Pretesting Specific Questions Forbull Meaningbull Task difficultybull Respondent interest and attention

bull Pretesting the overall Questionnairebull Flow of the questionnairebull Skip patternsbull Length

bull Put yourself in the respondentrsquos shoes and answer the questionnaire

• Questionnaire Design
• A QUESTIONNAIRE IS ONLY AS GOOD AS THE QUESTIONS IT ASKS
• Slide 3
• Logical Steps to Develop a Good Questionnaire
• What Should Be Asked
• Validity and reliability
• Type of information
• Knowledge
• Opinions
• Types of Questions
• Open Ended Questions
• Closed-response Questions
• Closed-response Questions (2)
• Closed-response Questions (3)
• Closed-Ended Questions
• Issues in Questionnaire Design
• Contents of the Questionnaire
• Filter questions
• Slide 19
• Issues in Questionnaire Design (2)
• Slide 21
• Double-barrelled questions
• Leading and Loaded questions
• Slide 24
• Vague questions
• Avoid Making Assumptions
• Burdensome Questions
• Question Wording
• Sequence And Layout Decisions
• Layout
• Distinguishing question and responses
• Pretesting and Correcting Problems

A questionnaire ensures standardization and comparability of the data across interviews increases speed and accuracy of recording

Facilitates data processingAlso allows the researcher to collect the

relevant information necessary to address the management decision problem

Purposes of the Questionnaire

Logical Steps to Develop a Good Questionnaire

bull Recall the research objective the research questions and hypotheses

bull Identify the variables to be measuredbull Formulate questions (items in the questionnaire)bull Order and wording of questions and the layout of the

questionnairebull Test for omissions and ambiguitybull Correct the problems (pretest again if necessary)

What Should Be Asked

Questionnaire RelevancyA questionnaire is relevant to the extent that all information collected addresses a research question that will help the decision maker address the current business problem

Questionnaire AccuracyAccuracy means that the information is reliable and valid

Validity and reliability

Validitybull question measure what you claim it

measures problem with self-report Reliabilitybull results are reproducible or consistent with

similar groups of respondents over time and when other people administer the questionnaire

Type of information

bull 1048715Knowledge - what people knowbull 1048715Opinions attitudes beliefs values -

what people think about an issuebull 1048715Behaviour - what people dobull 1048715Attributes - what are peoplersquos

characteristicsbull 1048715Remember - based on self-report

KnowledgeWhat is the recommended intervalbetween eye checks for patients withuncomplicated diabetesbull 1048715 6 monthsbull 1048715 1 yearbull 1048715 2 yearsbull 1048715 Not sure

Attributesbull When did you graduate from university_______________________

Opinions What do you think are the major issuesaffecting general practice in Australia atthe moment________________________________

BehaviourHave you developed a care plan for any ofyour patientsbull 1048715 Yesbull 1048715 No

Types of Questions1 Open-Ended question

People look for different things in a job What would you prefer most in a job

2 Closed-Ended questionPeople look for different things in a job What would you prefer most in a jobndash Work that pays wellndash Work that gives a sense of accomplishmentndash Work where you make most decisions by yourselfndash Work that is steady with little chance of being laid

off

Open Ended Questionsbull Advantages

ndash Gain insight into the problemndash Too many options to listndash When verbatim responses are desired to give the flavor of the problemndash When behavior to be measured is sensitive or disapprovedndash Interviewer questionnaire structure influence can be minimal

bull Disadvantagesndash Inarticulate respondentsndash Interviewerrsquos skill in recording quickly and summarizing accuratelyndash Time consuming subjective judgments while tabulating adds to cost

Closed-response Questions

Two Basic Formats for Closed Ended or Structured Questions

bull Choice from a list of responses

bull Appropriate single-choice rating on a scale

Closed-response Questions

What type of fast-food restaurant do you visit most often

Burger MexicanChicken PizzaSeafood ChineseDonrsquot know Other (please specify)

Closed-response Questions

What is your overall satisfaction with McDonalds burgers

Very satisfied Quite Satisfied Somewhat satisfied Not at all satisfied

Very satisfied 7 6 5 4 3 2 1 Not at all satisfied

Closed-Ended Questionsbull Advantages

ndash Easy to understand quick responses possiblendash Micro-differences in responses can be capturedndash Easier tabulation and analysisndash Answers are directly comparable from respondent to respondent

bull Disadvantagesndash Neutral category may attract more responses than warrantedndash Information between categories may be lost (extreme case ndash

dichotomous categories)

Issues in Questionnaire Design

bull Respondent uncertainty

ndash Should respondents be provided with aDonrsquot know or No opinion option

ndash When it is important to differentiate between ambivalence and ignorance both options should be provided

Contents of the Questionnaire bull Question Wording ndash Vocabulary

- Depends on the nature of variable tapped- subjective feeling or objective facts

ndash Simple easy to understand commonly used language

ndash Avoid technical words and jargon (unless sample is technically qualified)

Filter questions Filter questions useful to ensure respondents

only answer relevant parts of questionsUnfilteredbull ndash If you use a medical software programbull which one do you use

Filteredbull Do you use a medical software programbull 1048715 No - jump to next questionbull 1048715 Yes - which one

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided------------------------------------------------------------------------------

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Issues in Questionnaire Design

bull Question Wordingbull vocabulary bull Questions need to be clear simple and

precisebull Poorly written questions lead to ambiguity

and misunderstandings and can be wasteful

bull Avoid Complexity use simple conversational language

bull Avoid leading and loaded questions

bull Avoid ambiguity be as specific as possible

bull Avoid double-barreled items

bull Avoid making assumptions

bull Avoid burdensome questions

Double-barrelled questionsTwo concepts in one question

bull Have you had a neck ache or a back achesince your last visit

bull Since your last visit have you had any of the following symptoms (tick as many that apply)

bull Neck ache 1048715 bull Back ache 1048715 bull Headache 1048715

Leading and Loaded questionsbull A leading question suggests or implies certain

answersDo accounting graduates who attended state universities such as Washington State University make better auditors

bull A loaded question suggests a socially desirable answer or is emotionally charged

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Vague questions

Avoid vague questionsbull Taken altogether how happy are you with

your stay in hospitalbull Overall how would you describe the care

you received in hospital

Avoid Ambiguity Words such as ldquooftenrdquo ldquooccasionallyrdquo ldquousuallyrdquo ldquoregularlyrdquo ldquofrequentlyrdquo ldquomanyrdquo should be used with caution If these words have to be used their meaning should be explained properly

Avoid Making Assumptionsbull Is the question applicable to all

respondents

Why do you like fast-food

Assumes respondents like fast foodsBetter strategy would be to ask a filter

question first

Burdensome QuestionsAvoid Burdensome Questions That MayTax the Respondentrsquos Memory telephone

A survey conducted during the 24-hour period following the airing of the Super Bowl might establish whether the respondent watched the Super Bowl and then ask ldquoDo you recall any commercials on that programrdquo

Question Wordingbull Question length

ndash Should be shortndash Longer questions confuse and fatigue respondents

bull Sensitive questionsndash Questions on information perceived to be

embarrassing like personal income criminal activities alcoholism smoking drugs habits social desirability issues etc

ndash Creativity rules (assurances of confidentiality anonymity slipping it in sideways open-ended questions asking in third person etc)

Sequence And Layout Decisions

bull Opening questions ndash easy and non-threateningbull Flow ndash smooth and logical ndash avoid jumpsbull Broad to specificbull Critical questions ndash placed in the middle bull Appealing and interestingbull Order bias ndash the possibility that subsequent responses

are influenced by preceding responses eg fewer people will say that their taxes are too high after being asked whether govt spending should be increased in certain areas

bull Demographic questions ndash first or last

Layout

bull 1048715 Just as important as wordingbull 1048715 Aim for a professional lookbull 1048715 Tipsbull ndash cover letterintroductory page giving study title organisation aims of the surveybull ndash enough space for open-ended questionsbull ndash font large enough to read without strainbull ndash consistent and clear instructionsbull ndash donrsquot split questions or answers across pagesbull ndash enough white space

Distinguishing question and responses

LAYOUT ADo you agree disagree or have no opinion that this company

has bull A good vacation policy - disagreenot sureagreebull Good management feedback - disagreenot sureagreebull Good medical insurance - disagreenot sureagree

LAYOUT B Does this company have bull Disagree Not Sure Agree bull A good vacation policy 1 2 3 bull Good feedback 1 2 3 bull Good medical insurance 1 2 3

Pretesting and Correcting Problems

Purpose of pretest To ensure that the questionnaire meets the expectations in terms of the information that will be obtained

bull Pretesting Specific Questions Forbull Meaningbull Task difficultybull Respondent interest and attention

bull Pretesting the overall Questionnairebull Flow of the questionnairebull Skip patternsbull Length

bull Put yourself in the respondentrsquos shoes and answer the questionnaire

• Questionnaire Design
• A QUESTIONNAIRE IS ONLY AS GOOD AS THE QUESTIONS IT ASKS
• Slide 3
• Logical Steps to Develop a Good Questionnaire
• What Should Be Asked
• Validity and reliability
• Type of information
• Knowledge
• Opinions
• Types of Questions
• Open Ended Questions
• Closed-response Questions
• Closed-response Questions (2)
• Closed-response Questions (3)
• Closed-Ended Questions
• Issues in Questionnaire Design
• Contents of the Questionnaire
• Filter questions
• Slide 19
• Issues in Questionnaire Design (2)
• Slide 21
• Double-barrelled questions
• Leading and Loaded questions
• Slide 24
• Vague questions
• Avoid Making Assumptions
• Burdensome Questions
• Question Wording
• Sequence And Layout Decisions
• Layout
• Distinguishing question and responses
• Pretesting and Correcting Problems

Logical Steps to Develop a Good Questionnaire

bull Recall the research objective the research questions and hypotheses

bull Identify the variables to be measuredbull Formulate questions (items in the questionnaire)bull Order and wording of questions and the layout of the

questionnairebull Test for omissions and ambiguitybull Correct the problems (pretest again if necessary)

What Should Be Asked

Questionnaire RelevancyA questionnaire is relevant to the extent that all information collected addresses a research question that will help the decision maker address the current business problem

Questionnaire AccuracyAccuracy means that the information is reliable and valid

Validity and reliability

Validitybull question measure what you claim it

measures problem with self-report Reliabilitybull results are reproducible or consistent with

similar groups of respondents over time and when other people administer the questionnaire

Type of information

bull 1048715Knowledge - what people knowbull 1048715Opinions attitudes beliefs values -

what people think about an issuebull 1048715Behaviour - what people dobull 1048715Attributes - what are peoplersquos

characteristicsbull 1048715Remember - based on self-report

KnowledgeWhat is the recommended intervalbetween eye checks for patients withuncomplicated diabetesbull 1048715 6 monthsbull 1048715 1 yearbull 1048715 2 yearsbull 1048715 Not sure

Attributesbull When did you graduate from university_______________________

Opinions What do you think are the major issuesaffecting general practice in Australia atthe moment________________________________

BehaviourHave you developed a care plan for any ofyour patientsbull 1048715 Yesbull 1048715 No

Types of Questions1 Open-Ended question

People look for different things in a job What would you prefer most in a job

2 Closed-Ended questionPeople look for different things in a job What would you prefer most in a jobndash Work that pays wellndash Work that gives a sense of accomplishmentndash Work where you make most decisions by yourselfndash Work that is steady with little chance of being laid

off

Open Ended Questionsbull Advantages

ndash Gain insight into the problemndash Too many options to listndash When verbatim responses are desired to give the flavor of the problemndash When behavior to be measured is sensitive or disapprovedndash Interviewer questionnaire structure influence can be minimal

bull Disadvantagesndash Inarticulate respondentsndash Interviewerrsquos skill in recording quickly and summarizing accuratelyndash Time consuming subjective judgments while tabulating adds to cost

Closed-response Questions

Two Basic Formats for Closed Ended or Structured Questions

bull Choice from a list of responses

bull Appropriate single-choice rating on a scale

Closed-response Questions

What type of fast-food restaurant do you visit most often

Burger MexicanChicken PizzaSeafood ChineseDonrsquot know Other (please specify)

Closed-response Questions

What is your overall satisfaction with McDonalds burgers

Very satisfied Quite Satisfied Somewhat satisfied Not at all satisfied

Very satisfied 7 6 5 4 3 2 1 Not at all satisfied

Closed-Ended Questionsbull Advantages

ndash Easy to understand quick responses possiblendash Micro-differences in responses can be capturedndash Easier tabulation and analysisndash Answers are directly comparable from respondent to respondent

bull Disadvantagesndash Neutral category may attract more responses than warrantedndash Information between categories may be lost (extreme case ndash

dichotomous categories)

Issues in Questionnaire Design

bull Respondent uncertainty

ndash Should respondents be provided with aDonrsquot know or No opinion option

ndash When it is important to differentiate between ambivalence and ignorance both options should be provided

Contents of the Questionnaire bull Question Wording ndash Vocabulary

- Depends on the nature of variable tapped- subjective feeling or objective facts

ndash Simple easy to understand commonly used language

ndash Avoid technical words and jargon (unless sample is technically qualified)

Filter questions Filter questions useful to ensure respondents

only answer relevant parts of questionsUnfilteredbull ndash If you use a medical software programbull which one do you use

Filteredbull Do you use a medical software programbull 1048715 No - jump to next questionbull 1048715 Yes - which one

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided------------------------------------------------------------------------------

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Issues in Questionnaire Design

bull Question Wordingbull vocabulary bull Questions need to be clear simple and

precisebull Poorly written questions lead to ambiguity

and misunderstandings and can be wasteful

bull Avoid Complexity use simple conversational language

bull Avoid leading and loaded questions

bull Avoid ambiguity be as specific as possible

bull Avoid double-barreled items

bull Avoid making assumptions

bull Avoid burdensome questions

Double-barrelled questionsTwo concepts in one question

bull Have you had a neck ache or a back achesince your last visit

bull Since your last visit have you had any of the following symptoms (tick as many that apply)

bull Neck ache 1048715 bull Back ache 1048715 bull Headache 1048715

Leading and Loaded questionsbull A leading question suggests or implies certain

answersDo accounting graduates who attended state universities such as Washington State University make better auditors

bull A loaded question suggests a socially desirable answer or is emotionally charged

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Vague questions

Avoid vague questionsbull Taken altogether how happy are you with

your stay in hospitalbull Overall how would you describe the care

you received in hospital

Avoid Ambiguity Words such as ldquooftenrdquo ldquooccasionallyrdquo ldquousuallyrdquo ldquoregularlyrdquo ldquofrequentlyrdquo ldquomanyrdquo should be used with caution If these words have to be used their meaning should be explained properly

Avoid Making Assumptionsbull Is the question applicable to all

respondents

Why do you like fast-food

Assumes respondents like fast foodsBetter strategy would be to ask a filter

question first

Burdensome QuestionsAvoid Burdensome Questions That MayTax the Respondentrsquos Memory telephone

A survey conducted during the 24-hour period following the airing of the Super Bowl might establish whether the respondent watched the Super Bowl and then ask ldquoDo you recall any commercials on that programrdquo

Question Wordingbull Question length

ndash Should be shortndash Longer questions confuse and fatigue respondents

bull Sensitive questionsndash Questions on information perceived to be

embarrassing like personal income criminal activities alcoholism smoking drugs habits social desirability issues etc

ndash Creativity rules (assurances of confidentiality anonymity slipping it in sideways open-ended questions asking in third person etc)

Sequence And Layout Decisions

bull Opening questions ndash easy and non-threateningbull Flow ndash smooth and logical ndash avoid jumpsbull Broad to specificbull Critical questions ndash placed in the middle bull Appealing and interestingbull Order bias ndash the possibility that subsequent responses

are influenced by preceding responses eg fewer people will say that their taxes are too high after being asked whether govt spending should be increased in certain areas

bull Demographic questions ndash first or last

Layout

bull 1048715 Just as important as wordingbull 1048715 Aim for a professional lookbull 1048715 Tipsbull ndash cover letterintroductory page giving study title organisation aims of the surveybull ndash enough space for open-ended questionsbull ndash font large enough to read without strainbull ndash consistent and clear instructionsbull ndash donrsquot split questions or answers across pagesbull ndash enough white space

Distinguishing question and responses

LAYOUT ADo you agree disagree or have no opinion that this company

has bull A good vacation policy - disagreenot sureagreebull Good management feedback - disagreenot sureagreebull Good medical insurance - disagreenot sureagree

LAYOUT B Does this company have bull Disagree Not Sure Agree bull A good vacation policy 1 2 3 bull Good feedback 1 2 3 bull Good medical insurance 1 2 3

Pretesting and Correcting Problems

Purpose of pretest To ensure that the questionnaire meets the expectations in terms of the information that will be obtained

bull Pretesting Specific Questions Forbull Meaningbull Task difficultybull Respondent interest and attention

bull Pretesting the overall Questionnairebull Flow of the questionnairebull Skip patternsbull Length

bull Put yourself in the respondentrsquos shoes and answer the questionnaire

• Questionnaire Design
• A QUESTIONNAIRE IS ONLY AS GOOD AS THE QUESTIONS IT ASKS
• Slide 3
• Logical Steps to Develop a Good Questionnaire
• What Should Be Asked
• Validity and reliability
• Type of information
• Knowledge
• Opinions
• Types of Questions
• Open Ended Questions
• Closed-response Questions
• Closed-response Questions (2)
• Closed-response Questions (3)
• Closed-Ended Questions
• Issues in Questionnaire Design
• Contents of the Questionnaire
• Filter questions
• Slide 19
• Issues in Questionnaire Design (2)
• Slide 21
• Double-barrelled questions
• Leading and Loaded questions
• Slide 24
• Vague questions
• Avoid Making Assumptions
• Burdensome Questions
• Question Wording
• Sequence And Layout Decisions
• Layout
• Distinguishing question and responses
• Pretesting and Correcting Problems

What Should Be Asked

Questionnaire RelevancyA questionnaire is relevant to the extent that all information collected addresses a research question that will help the decision maker address the current business problem

Questionnaire AccuracyAccuracy means that the information is reliable and valid

Validity and reliability

Validitybull question measure what you claim it

measures problem with self-report Reliabilitybull results are reproducible or consistent with

similar groups of respondents over time and when other people administer the questionnaire

Type of information

bull 1048715Knowledge - what people knowbull 1048715Opinions attitudes beliefs values -

what people think about an issuebull 1048715Behaviour - what people dobull 1048715Attributes - what are peoplersquos

characteristicsbull 1048715Remember - based on self-report

KnowledgeWhat is the recommended intervalbetween eye checks for patients withuncomplicated diabetesbull 1048715 6 monthsbull 1048715 1 yearbull 1048715 2 yearsbull 1048715 Not sure

Attributesbull When did you graduate from university_______________________

Opinions What do you think are the major issuesaffecting general practice in Australia atthe moment________________________________

BehaviourHave you developed a care plan for any ofyour patientsbull 1048715 Yesbull 1048715 No

Types of Questions1 Open-Ended question

People look for different things in a job What would you prefer most in a job

2 Closed-Ended questionPeople look for different things in a job What would you prefer most in a jobndash Work that pays wellndash Work that gives a sense of accomplishmentndash Work where you make most decisions by yourselfndash Work that is steady with little chance of being laid

off

Open Ended Questionsbull Advantages

ndash Gain insight into the problemndash Too many options to listndash When verbatim responses are desired to give the flavor of the problemndash When behavior to be measured is sensitive or disapprovedndash Interviewer questionnaire structure influence can be minimal

bull Disadvantagesndash Inarticulate respondentsndash Interviewerrsquos skill in recording quickly and summarizing accuratelyndash Time consuming subjective judgments while tabulating adds to cost

Closed-response Questions

Two Basic Formats for Closed Ended or Structured Questions

bull Choice from a list of responses

bull Appropriate single-choice rating on a scale

Closed-response Questions

What type of fast-food restaurant do you visit most often

Burger MexicanChicken PizzaSeafood ChineseDonrsquot know Other (please specify)

Closed-response Questions

What is your overall satisfaction with McDonalds burgers

Very satisfied Quite Satisfied Somewhat satisfied Not at all satisfied

Very satisfied 7 6 5 4 3 2 1 Not at all satisfied

Closed-Ended Questionsbull Advantages

ndash Easy to understand quick responses possiblendash Micro-differences in responses can be capturedndash Easier tabulation and analysisndash Answers are directly comparable from respondent to respondent

bull Disadvantagesndash Neutral category may attract more responses than warrantedndash Information between categories may be lost (extreme case ndash

dichotomous categories)

Issues in Questionnaire Design

bull Respondent uncertainty

ndash Should respondents be provided with aDonrsquot know or No opinion option

ndash When it is important to differentiate between ambivalence and ignorance both options should be provided

Contents of the Questionnaire bull Question Wording ndash Vocabulary

- Depends on the nature of variable tapped- subjective feeling or objective facts

ndash Simple easy to understand commonly used language

ndash Avoid technical words and jargon (unless sample is technically qualified)

Filter questions Filter questions useful to ensure respondents

only answer relevant parts of questionsUnfilteredbull ndash If you use a medical software programbull which one do you use

Filteredbull Do you use a medical software programbull 1048715 No - jump to next questionbull 1048715 Yes - which one

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided------------------------------------------------------------------------------

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Issues in Questionnaire Design

bull Question Wordingbull vocabulary bull Questions need to be clear simple and

precisebull Poorly written questions lead to ambiguity

and misunderstandings and can be wasteful

bull Avoid Complexity use simple conversational language

bull Avoid leading and loaded questions

bull Avoid ambiguity be as specific as possible

bull Avoid double-barreled items

bull Avoid making assumptions

bull Avoid burdensome questions

Double-barrelled questionsTwo concepts in one question

bull Have you had a neck ache or a back achesince your last visit

bull Since your last visit have you had any of the following symptoms (tick as many that apply)

bull Neck ache 1048715 bull Back ache 1048715 bull Headache 1048715

Leading and Loaded questionsbull A leading question suggests or implies certain

answersDo accounting graduates who attended state universities such as Washington State University make better auditors

bull A loaded question suggests a socially desirable answer or is emotionally charged

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Vague questions

Avoid vague questionsbull Taken altogether how happy are you with

your stay in hospitalbull Overall how would you describe the care

you received in hospital

Avoid Ambiguity Words such as ldquooftenrdquo ldquooccasionallyrdquo ldquousuallyrdquo ldquoregularlyrdquo ldquofrequentlyrdquo ldquomanyrdquo should be used with caution If these words have to be used their meaning should be explained properly

Avoid Making Assumptionsbull Is the question applicable to all

respondents

Why do you like fast-food

Assumes respondents like fast foodsBetter strategy would be to ask a filter

question first

Burdensome QuestionsAvoid Burdensome Questions That MayTax the Respondentrsquos Memory telephone

A survey conducted during the 24-hour period following the airing of the Super Bowl might establish whether the respondent watched the Super Bowl and then ask ldquoDo you recall any commercials on that programrdquo

Question Wordingbull Question length

ndash Should be shortndash Longer questions confuse and fatigue respondents

bull Sensitive questionsndash Questions on information perceived to be

embarrassing like personal income criminal activities alcoholism smoking drugs habits social desirability issues etc

ndash Creativity rules (assurances of confidentiality anonymity slipping it in sideways open-ended questions asking in third person etc)

Sequence And Layout Decisions

bull Opening questions ndash easy and non-threateningbull Flow ndash smooth and logical ndash avoid jumpsbull Broad to specificbull Critical questions ndash placed in the middle bull Appealing and interestingbull Order bias ndash the possibility that subsequent responses

are influenced by preceding responses eg fewer people will say that their taxes are too high after being asked whether govt spending should be increased in certain areas

bull Demographic questions ndash first or last

Layout

bull 1048715 Just as important as wordingbull 1048715 Aim for a professional lookbull 1048715 Tipsbull ndash cover letterintroductory page giving study title organisation aims of the surveybull ndash enough space for open-ended questionsbull ndash font large enough to read without strainbull ndash consistent and clear instructionsbull ndash donrsquot split questions or answers across pagesbull ndash enough white space

Distinguishing question and responses

LAYOUT ADo you agree disagree or have no opinion that this company

has bull A good vacation policy - disagreenot sureagreebull Good management feedback - disagreenot sureagreebull Good medical insurance - disagreenot sureagree

LAYOUT B Does this company have bull Disagree Not Sure Agree bull A good vacation policy 1 2 3 bull Good feedback 1 2 3 bull Good medical insurance 1 2 3

Pretesting and Correcting Problems

Purpose of pretest To ensure that the questionnaire meets the expectations in terms of the information that will be obtained

bull Pretesting Specific Questions Forbull Meaningbull Task difficultybull Respondent interest and attention

bull Pretesting the overall Questionnairebull Flow of the questionnairebull Skip patternsbull Length

bull Put yourself in the respondentrsquos shoes and answer the questionnaire

• Questionnaire Design
• A QUESTIONNAIRE IS ONLY AS GOOD AS THE QUESTIONS IT ASKS
• Slide 3
• Logical Steps to Develop a Good Questionnaire
• What Should Be Asked
• Validity and reliability
• Type of information
• Knowledge
• Opinions
• Types of Questions
• Open Ended Questions
• Closed-response Questions
• Closed-response Questions (2)
• Closed-response Questions (3)
• Closed-Ended Questions
• Issues in Questionnaire Design
• Contents of the Questionnaire
• Filter questions
• Slide 19
• Issues in Questionnaire Design (2)
• Slide 21
• Double-barrelled questions
• Leading and Loaded questions
• Slide 24
• Vague questions
• Avoid Making Assumptions
• Burdensome Questions
• Question Wording
• Sequence And Layout Decisions
• Layout
• Distinguishing question and responses
• Pretesting and Correcting Problems

Validity and reliability

Validitybull question measure what you claim it

measures problem with self-report Reliabilitybull results are reproducible or consistent with

similar groups of respondents over time and when other people administer the questionnaire

Type of information

bull 1048715Knowledge - what people knowbull 1048715Opinions attitudes beliefs values -

what people think about an issuebull 1048715Behaviour - what people dobull 1048715Attributes - what are peoplersquos

characteristicsbull 1048715Remember - based on self-report

KnowledgeWhat is the recommended intervalbetween eye checks for patients withuncomplicated diabetesbull 1048715 6 monthsbull 1048715 1 yearbull 1048715 2 yearsbull 1048715 Not sure

Attributesbull When did you graduate from university_______________________

Opinions What do you think are the major issuesaffecting general practice in Australia atthe moment________________________________

BehaviourHave you developed a care plan for any ofyour patientsbull 1048715 Yesbull 1048715 No

Types of Questions1 Open-Ended question

People look for different things in a job What would you prefer most in a job

2 Closed-Ended questionPeople look for different things in a job What would you prefer most in a jobndash Work that pays wellndash Work that gives a sense of accomplishmentndash Work where you make most decisions by yourselfndash Work that is steady with little chance of being laid

off

Open Ended Questionsbull Advantages

ndash Gain insight into the problemndash Too many options to listndash When verbatim responses are desired to give the flavor of the problemndash When behavior to be measured is sensitive or disapprovedndash Interviewer questionnaire structure influence can be minimal

bull Disadvantagesndash Inarticulate respondentsndash Interviewerrsquos skill in recording quickly and summarizing accuratelyndash Time consuming subjective judgments while tabulating adds to cost

Closed-response Questions

Two Basic Formats for Closed Ended or Structured Questions

bull Choice from a list of responses

bull Appropriate single-choice rating on a scale

Closed-response Questions

What type of fast-food restaurant do you visit most often

Burger MexicanChicken PizzaSeafood ChineseDonrsquot know Other (please specify)

Closed-response Questions

What is your overall satisfaction with McDonalds burgers

Very satisfied Quite Satisfied Somewhat satisfied Not at all satisfied

Very satisfied 7 6 5 4 3 2 1 Not at all satisfied

Closed-Ended Questionsbull Advantages

ndash Easy to understand quick responses possiblendash Micro-differences in responses can be capturedndash Easier tabulation and analysisndash Answers are directly comparable from respondent to respondent

bull Disadvantagesndash Neutral category may attract more responses than warrantedndash Information between categories may be lost (extreme case ndash

dichotomous categories)

Issues in Questionnaire Design

bull Respondent uncertainty

ndash Should respondents be provided with aDonrsquot know or No opinion option

ndash When it is important to differentiate between ambivalence and ignorance both options should be provided

Contents of the Questionnaire bull Question Wording ndash Vocabulary

- Depends on the nature of variable tapped- subjective feeling or objective facts

ndash Simple easy to understand commonly used language

ndash Avoid technical words and jargon (unless sample is technically qualified)

Filter questions Filter questions useful to ensure respondents

only answer relevant parts of questionsUnfilteredbull ndash If you use a medical software programbull which one do you use

Filteredbull Do you use a medical software programbull 1048715 No - jump to next questionbull 1048715 Yes - which one

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided------------------------------------------------------------------------------

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Issues in Questionnaire Design

bull Question Wordingbull vocabulary bull Questions need to be clear simple and

precisebull Poorly written questions lead to ambiguity

and misunderstandings and can be wasteful

bull Avoid Complexity use simple conversational language

bull Avoid leading and loaded questions

bull Avoid ambiguity be as specific as possible

bull Avoid double-barreled items

bull Avoid making assumptions

bull Avoid burdensome questions

Double-barrelled questionsTwo concepts in one question

bull Have you had a neck ache or a back achesince your last visit

bull Since your last visit have you had any of the following symptoms (tick as many that apply)

bull Neck ache 1048715 bull Back ache 1048715 bull Headache 1048715

Leading and Loaded questionsbull A leading question suggests or implies certain

answersDo accounting graduates who attended state universities such as Washington State University make better auditors

bull A loaded question suggests a socially desirable answer or is emotionally charged

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Vague questions

Avoid vague questionsbull Taken altogether how happy are you with

your stay in hospitalbull Overall how would you describe the care

you received in hospital

Avoid Ambiguity Words such as ldquooftenrdquo ldquooccasionallyrdquo ldquousuallyrdquo ldquoregularlyrdquo ldquofrequentlyrdquo ldquomanyrdquo should be used with caution If these words have to be used their meaning should be explained properly

Avoid Making Assumptionsbull Is the question applicable to all

respondents

Why do you like fast-food

Assumes respondents like fast foodsBetter strategy would be to ask a filter

question first

Burdensome QuestionsAvoid Burdensome Questions That MayTax the Respondentrsquos Memory telephone

A survey conducted during the 24-hour period following the airing of the Super Bowl might establish whether the respondent watched the Super Bowl and then ask ldquoDo you recall any commercials on that programrdquo

Question Wordingbull Question length

ndash Should be shortndash Longer questions confuse and fatigue respondents

bull Sensitive questionsndash Questions on information perceived to be

embarrassing like personal income criminal activities alcoholism smoking drugs habits social desirability issues etc

ndash Creativity rules (assurances of confidentiality anonymity slipping it in sideways open-ended questions asking in third person etc)

Sequence And Layout Decisions

bull Opening questions ndash easy and non-threateningbull Flow ndash smooth and logical ndash avoid jumpsbull Broad to specificbull Critical questions ndash placed in the middle bull Appealing and interestingbull Order bias ndash the possibility that subsequent responses

are influenced by preceding responses eg fewer people will say that their taxes are too high after being asked whether govt spending should be increased in certain areas

bull Demographic questions ndash first or last

Layout

bull 1048715 Just as important as wordingbull 1048715 Aim for a professional lookbull 1048715 Tipsbull ndash cover letterintroductory page giving study title organisation aims of the surveybull ndash enough space for open-ended questionsbull ndash font large enough to read without strainbull ndash consistent and clear instructionsbull ndash donrsquot split questions or answers across pagesbull ndash enough white space

Distinguishing question and responses

LAYOUT ADo you agree disagree or have no opinion that this company

has bull A good vacation policy - disagreenot sureagreebull Good management feedback - disagreenot sureagreebull Good medical insurance - disagreenot sureagree

LAYOUT B Does this company have bull Disagree Not Sure Agree bull A good vacation policy 1 2 3 bull Good feedback 1 2 3 bull Good medical insurance 1 2 3

Pretesting and Correcting Problems

Purpose of pretest To ensure that the questionnaire meets the expectations in terms of the information that will be obtained

bull Pretesting Specific Questions Forbull Meaningbull Task difficultybull Respondent interest and attention

bull Pretesting the overall Questionnairebull Flow of the questionnairebull Skip patternsbull Length

bull Put yourself in the respondentrsquos shoes and answer the questionnaire

• Questionnaire Design
• A QUESTIONNAIRE IS ONLY AS GOOD AS THE QUESTIONS IT ASKS
• Slide 3
• Logical Steps to Develop a Good Questionnaire
• What Should Be Asked
• Validity and reliability
• Type of information
• Knowledge
• Opinions
• Types of Questions
• Open Ended Questions
• Closed-response Questions
• Closed-response Questions (2)
• Closed-response Questions (3)
• Closed-Ended Questions
• Issues in Questionnaire Design
• Contents of the Questionnaire
• Filter questions
• Slide 19
• Issues in Questionnaire Design (2)
• Slide 21
• Double-barrelled questions
• Leading and Loaded questions
• Slide 24
• Vague questions
• Avoid Making Assumptions
• Burdensome Questions
• Question Wording
• Sequence And Layout Decisions
• Layout
• Distinguishing question and responses
• Pretesting and Correcting Problems

Type of information

bull 1048715Knowledge - what people knowbull 1048715Opinions attitudes beliefs values -

what people think about an issuebull 1048715Behaviour - what people dobull 1048715Attributes - what are peoplersquos

characteristicsbull 1048715Remember - based on self-report

KnowledgeWhat is the recommended intervalbetween eye checks for patients withuncomplicated diabetesbull 1048715 6 monthsbull 1048715 1 yearbull 1048715 2 yearsbull 1048715 Not sure

Attributesbull When did you graduate from university_______________________

Opinions What do you think are the major issuesaffecting general practice in Australia atthe moment________________________________

BehaviourHave you developed a care plan for any ofyour patientsbull 1048715 Yesbull 1048715 No

Types of Questions1 Open-Ended question

People look for different things in a job What would you prefer most in a job

2 Closed-Ended questionPeople look for different things in a job What would you prefer most in a jobndash Work that pays wellndash Work that gives a sense of accomplishmentndash Work where you make most decisions by yourselfndash Work that is steady with little chance of being laid

off

Open Ended Questionsbull Advantages

ndash Gain insight into the problemndash Too many options to listndash When verbatim responses are desired to give the flavor of the problemndash When behavior to be measured is sensitive or disapprovedndash Interviewer questionnaire structure influence can be minimal

bull Disadvantagesndash Inarticulate respondentsndash Interviewerrsquos skill in recording quickly and summarizing accuratelyndash Time consuming subjective judgments while tabulating adds to cost

Closed-response Questions

Two Basic Formats for Closed Ended or Structured Questions

bull Choice from a list of responses

bull Appropriate single-choice rating on a scale

Closed-response Questions

What type of fast-food restaurant do you visit most often

Burger MexicanChicken PizzaSeafood ChineseDonrsquot know Other (please specify)

Closed-response Questions

What is your overall satisfaction with McDonalds burgers

Very satisfied Quite Satisfied Somewhat satisfied Not at all satisfied

Very satisfied 7 6 5 4 3 2 1 Not at all satisfied

Closed-Ended Questionsbull Advantages

ndash Easy to understand quick responses possiblendash Micro-differences in responses can be capturedndash Easier tabulation and analysisndash Answers are directly comparable from respondent to respondent

bull Disadvantagesndash Neutral category may attract more responses than warrantedndash Information between categories may be lost (extreme case ndash

dichotomous categories)

Issues in Questionnaire Design

bull Respondent uncertainty

ndash Should respondents be provided with aDonrsquot know or No opinion option

ndash When it is important to differentiate between ambivalence and ignorance both options should be provided

Contents of the Questionnaire bull Question Wording ndash Vocabulary

- Depends on the nature of variable tapped- subjective feeling or objective facts

ndash Simple easy to understand commonly used language

ndash Avoid technical words and jargon (unless sample is technically qualified)

Filter questions Filter questions useful to ensure respondents

only answer relevant parts of questionsUnfilteredbull ndash If you use a medical software programbull which one do you use

Filteredbull Do you use a medical software programbull 1048715 No - jump to next questionbull 1048715 Yes - which one

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided------------------------------------------------------------------------------

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Issues in Questionnaire Design

bull Question Wordingbull vocabulary bull Questions need to be clear simple and

precisebull Poorly written questions lead to ambiguity

and misunderstandings and can be wasteful

bull Avoid Complexity use simple conversational language

bull Avoid leading and loaded questions

bull Avoid ambiguity be as specific as possible

bull Avoid double-barreled items

bull Avoid making assumptions

bull Avoid burdensome questions

Double-barrelled questionsTwo concepts in one question

bull Have you had a neck ache or a back achesince your last visit

bull Since your last visit have you had any of the following symptoms (tick as many that apply)

bull Neck ache 1048715 bull Back ache 1048715 bull Headache 1048715

Leading and Loaded questionsbull A leading question suggests or implies certain

answersDo accounting graduates who attended state universities such as Washington State University make better auditors

bull A loaded question suggests a socially desirable answer or is emotionally charged

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Vague questions

Avoid vague questionsbull Taken altogether how happy are you with

your stay in hospitalbull Overall how would you describe the care

you received in hospital

Avoid Ambiguity Words such as ldquooftenrdquo ldquooccasionallyrdquo ldquousuallyrdquo ldquoregularlyrdquo ldquofrequentlyrdquo ldquomanyrdquo should be used with caution If these words have to be used their meaning should be explained properly

Avoid Making Assumptionsbull Is the question applicable to all

respondents

Why do you like fast-food

Assumes respondents like fast foodsBetter strategy would be to ask a filter

question first

Burdensome QuestionsAvoid Burdensome Questions That MayTax the Respondentrsquos Memory telephone

A survey conducted during the 24-hour period following the airing of the Super Bowl might establish whether the respondent watched the Super Bowl and then ask ldquoDo you recall any commercials on that programrdquo

Question Wordingbull Question length

ndash Should be shortndash Longer questions confuse and fatigue respondents

bull Sensitive questionsndash Questions on information perceived to be

embarrassing like personal income criminal activities alcoholism smoking drugs habits social desirability issues etc

ndash Creativity rules (assurances of confidentiality anonymity slipping it in sideways open-ended questions asking in third person etc)

Sequence And Layout Decisions

bull Opening questions ndash easy and non-threateningbull Flow ndash smooth and logical ndash avoid jumpsbull Broad to specificbull Critical questions ndash placed in the middle bull Appealing and interestingbull Order bias ndash the possibility that subsequent responses

are influenced by preceding responses eg fewer people will say that their taxes are too high after being asked whether govt spending should be increased in certain areas

bull Demographic questions ndash first or last

Layout

bull 1048715 Just as important as wordingbull 1048715 Aim for a professional lookbull 1048715 Tipsbull ndash cover letterintroductory page giving study title organisation aims of the surveybull ndash enough space for open-ended questionsbull ndash font large enough to read without strainbull ndash consistent and clear instructionsbull ndash donrsquot split questions or answers across pagesbull ndash enough white space

Distinguishing question and responses

LAYOUT ADo you agree disagree or have no opinion that this company

has bull A good vacation policy - disagreenot sureagreebull Good management feedback - disagreenot sureagreebull Good medical insurance - disagreenot sureagree

LAYOUT B Does this company have bull Disagree Not Sure Agree bull A good vacation policy 1 2 3 bull Good feedback 1 2 3 bull Good medical insurance 1 2 3

Pretesting and Correcting Problems

Purpose of pretest To ensure that the questionnaire meets the expectations in terms of the information that will be obtained

bull Pretesting Specific Questions Forbull Meaningbull Task difficultybull Respondent interest and attention

bull Pretesting the overall Questionnairebull Flow of the questionnairebull Skip patternsbull Length

bull Put yourself in the respondentrsquos shoes and answer the questionnaire

• Questionnaire Design
• A QUESTIONNAIRE IS ONLY AS GOOD AS THE QUESTIONS IT ASKS
• Slide 3
• Logical Steps to Develop a Good Questionnaire
• What Should Be Asked
• Validity and reliability
• Type of information
• Knowledge
• Opinions
• Types of Questions
• Open Ended Questions
• Closed-response Questions
• Closed-response Questions (2)
• Closed-response Questions (3)
• Closed-Ended Questions
• Issues in Questionnaire Design
• Contents of the Questionnaire
• Filter questions
• Slide 19
• Issues in Questionnaire Design (2)
• Slide 21
• Double-barrelled questions
• Leading and Loaded questions
• Slide 24
• Vague questions
• Avoid Making Assumptions
• Burdensome Questions
• Question Wording
• Sequence And Layout Decisions
• Layout
• Distinguishing question and responses
• Pretesting and Correcting Problems

KnowledgeWhat is the recommended intervalbetween eye checks for patients withuncomplicated diabetesbull 1048715 6 monthsbull 1048715 1 yearbull 1048715 2 yearsbull 1048715 Not sure

Attributesbull When did you graduate from university_______________________

Opinions What do you think are the major issuesaffecting general practice in Australia atthe moment________________________________

BehaviourHave you developed a care plan for any ofyour patientsbull 1048715 Yesbull 1048715 No

Types of Questions1 Open-Ended question

People look for different things in a job What would you prefer most in a job

2 Closed-Ended questionPeople look for different things in a job What would you prefer most in a jobndash Work that pays wellndash Work that gives a sense of accomplishmentndash Work where you make most decisions by yourselfndash Work that is steady with little chance of being laid

off

Open Ended Questionsbull Advantages

ndash Gain insight into the problemndash Too many options to listndash When verbatim responses are desired to give the flavor of the problemndash When behavior to be measured is sensitive or disapprovedndash Interviewer questionnaire structure influence can be minimal

bull Disadvantagesndash Inarticulate respondentsndash Interviewerrsquos skill in recording quickly and summarizing accuratelyndash Time consuming subjective judgments while tabulating adds to cost

Closed-response Questions

Two Basic Formats for Closed Ended or Structured Questions

bull Choice from a list of responses

bull Appropriate single-choice rating on a scale

Closed-response Questions

What type of fast-food restaurant do you visit most often

Burger MexicanChicken PizzaSeafood ChineseDonrsquot know Other (please specify)

Closed-response Questions

What is your overall satisfaction with McDonalds burgers

Very satisfied Quite Satisfied Somewhat satisfied Not at all satisfied

Very satisfied 7 6 5 4 3 2 1 Not at all satisfied

Closed-Ended Questionsbull Advantages

ndash Easy to understand quick responses possiblendash Micro-differences in responses can be capturedndash Easier tabulation and analysisndash Answers are directly comparable from respondent to respondent

bull Disadvantagesndash Neutral category may attract more responses than warrantedndash Information between categories may be lost (extreme case ndash

dichotomous categories)

Issues in Questionnaire Design

bull Respondent uncertainty

ndash Should respondents be provided with aDonrsquot know or No opinion option

ndash When it is important to differentiate between ambivalence and ignorance both options should be provided

Contents of the Questionnaire bull Question Wording ndash Vocabulary

- Depends on the nature of variable tapped- subjective feeling or objective facts

ndash Simple easy to understand commonly used language

ndash Avoid technical words and jargon (unless sample is technically qualified)

Filter questions Filter questions useful to ensure respondents

only answer relevant parts of questionsUnfilteredbull ndash If you use a medical software programbull which one do you use

Filteredbull Do you use a medical software programbull 1048715 No - jump to next questionbull 1048715 Yes - which one

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided------------------------------------------------------------------------------

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Issues in Questionnaire Design

bull Question Wordingbull vocabulary bull Questions need to be clear simple and

precisebull Poorly written questions lead to ambiguity

and misunderstandings and can be wasteful

bull Avoid Complexity use simple conversational language

bull Avoid leading and loaded questions

bull Avoid ambiguity be as specific as possible

bull Avoid double-barreled items

bull Avoid making assumptions

bull Avoid burdensome questions

Double-barrelled questionsTwo concepts in one question

bull Have you had a neck ache or a back achesince your last visit

bull Since your last visit have you had any of the following symptoms (tick as many that apply)

bull Neck ache 1048715 bull Back ache 1048715 bull Headache 1048715

Leading and Loaded questionsbull A leading question suggests or implies certain

answersDo accounting graduates who attended state universities such as Washington State University make better auditors

bull A loaded question suggests a socially desirable answer or is emotionally charged

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Vague questions

Avoid vague questionsbull Taken altogether how happy are you with

your stay in hospitalbull Overall how would you describe the care

you received in hospital

Avoid Ambiguity Words such as ldquooftenrdquo ldquooccasionallyrdquo ldquousuallyrdquo ldquoregularlyrdquo ldquofrequentlyrdquo ldquomanyrdquo should be used with caution If these words have to be used their meaning should be explained properly

Avoid Making Assumptionsbull Is the question applicable to all

respondents

Why do you like fast-food

Assumes respondents like fast foodsBetter strategy would be to ask a filter

question first

Burdensome QuestionsAvoid Burdensome Questions That MayTax the Respondentrsquos Memory telephone

A survey conducted during the 24-hour period following the airing of the Super Bowl might establish whether the respondent watched the Super Bowl and then ask ldquoDo you recall any commercials on that programrdquo

Question Wordingbull Question length

ndash Should be shortndash Longer questions confuse and fatigue respondents

bull Sensitive questionsndash Questions on information perceived to be

embarrassing like personal income criminal activities alcoholism smoking drugs habits social desirability issues etc

ndash Creativity rules (assurances of confidentiality anonymity slipping it in sideways open-ended questions asking in third person etc)

Sequence And Layout Decisions

bull Opening questions ndash easy and non-threateningbull Flow ndash smooth and logical ndash avoid jumpsbull Broad to specificbull Critical questions ndash placed in the middle bull Appealing and interestingbull Order bias ndash the possibility that subsequent responses

are influenced by preceding responses eg fewer people will say that their taxes are too high after being asked whether govt spending should be increased in certain areas

bull Demographic questions ndash first or last

Layout

bull 1048715 Just as important as wordingbull 1048715 Aim for a professional lookbull 1048715 Tipsbull ndash cover letterintroductory page giving study title organisation aims of the surveybull ndash enough space for open-ended questionsbull ndash font large enough to read without strainbull ndash consistent and clear instructionsbull ndash donrsquot split questions or answers across pagesbull ndash enough white space

Distinguishing question and responses

LAYOUT ADo you agree disagree or have no opinion that this company

has bull A good vacation policy - disagreenot sureagreebull Good management feedback - disagreenot sureagreebull Good medical insurance - disagreenot sureagree

LAYOUT B Does this company have bull Disagree Not Sure Agree bull A good vacation policy 1 2 3 bull Good feedback 1 2 3 bull Good medical insurance 1 2 3

Pretesting and Correcting Problems

Purpose of pretest To ensure that the questionnaire meets the expectations in terms of the information that will be obtained

bull Pretesting Specific Questions Forbull Meaningbull Task difficultybull Respondent interest and attention

bull Pretesting the overall Questionnairebull Flow of the questionnairebull Skip patternsbull Length

bull Put yourself in the respondentrsquos shoes and answer the questionnaire

• Questionnaire Design
• A QUESTIONNAIRE IS ONLY AS GOOD AS THE QUESTIONS IT ASKS
• Slide 3
• Logical Steps to Develop a Good Questionnaire
• What Should Be Asked
• Validity and reliability
• Type of information
• Knowledge
• Opinions
• Types of Questions
• Open Ended Questions
• Closed-response Questions
• Closed-response Questions (2)
• Closed-response Questions (3)
• Closed-Ended Questions
• Issues in Questionnaire Design
• Contents of the Questionnaire
• Filter questions
• Slide 19
• Issues in Questionnaire Design (2)
• Slide 21
• Double-barrelled questions
• Leading and Loaded questions
• Slide 24
• Vague questions
• Avoid Making Assumptions
• Burdensome Questions
• Question Wording
• Sequence And Layout Decisions
• Layout
• Distinguishing question and responses
• Pretesting and Correcting Problems

Opinions What do you think are the major issuesaffecting general practice in Australia atthe moment________________________________

BehaviourHave you developed a care plan for any ofyour patientsbull 1048715 Yesbull 1048715 No

Types of Questions1 Open-Ended question

People look for different things in a job What would you prefer most in a job

2 Closed-Ended questionPeople look for different things in a job What would you prefer most in a jobndash Work that pays wellndash Work that gives a sense of accomplishmentndash Work where you make most decisions by yourselfndash Work that is steady with little chance of being laid

off

Open Ended Questionsbull Advantages

ndash Gain insight into the problemndash Too many options to listndash When verbatim responses are desired to give the flavor of the problemndash When behavior to be measured is sensitive or disapprovedndash Interviewer questionnaire structure influence can be minimal

bull Disadvantagesndash Inarticulate respondentsndash Interviewerrsquos skill in recording quickly and summarizing accuratelyndash Time consuming subjective judgments while tabulating adds to cost

Closed-response Questions

Two Basic Formats for Closed Ended or Structured Questions

bull Choice from a list of responses

bull Appropriate single-choice rating on a scale

Closed-response Questions

What type of fast-food restaurant do you visit most often

Burger MexicanChicken PizzaSeafood ChineseDonrsquot know Other (please specify)

Closed-response Questions

What is your overall satisfaction with McDonalds burgers

Very satisfied Quite Satisfied Somewhat satisfied Not at all satisfied

Very satisfied 7 6 5 4 3 2 1 Not at all satisfied

Closed-Ended Questionsbull Advantages

ndash Easy to understand quick responses possiblendash Micro-differences in responses can be capturedndash Easier tabulation and analysisndash Answers are directly comparable from respondent to respondent

bull Disadvantagesndash Neutral category may attract more responses than warrantedndash Information between categories may be lost (extreme case ndash

dichotomous categories)

Issues in Questionnaire Design

bull Respondent uncertainty

ndash Should respondents be provided with aDonrsquot know or No opinion option

ndash When it is important to differentiate between ambivalence and ignorance both options should be provided

Contents of the Questionnaire bull Question Wording ndash Vocabulary

- Depends on the nature of variable tapped- subjective feeling or objective facts

ndash Simple easy to understand commonly used language

ndash Avoid technical words and jargon (unless sample is technically qualified)

Filter questions Filter questions useful to ensure respondents

only answer relevant parts of questionsUnfilteredbull ndash If you use a medical software programbull which one do you use

Filteredbull Do you use a medical software programbull 1048715 No - jump to next questionbull 1048715 Yes - which one

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided------------------------------------------------------------------------------

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Issues in Questionnaire Design

bull Question Wordingbull vocabulary bull Questions need to be clear simple and

precisebull Poorly written questions lead to ambiguity

and misunderstandings and can be wasteful

bull Avoid Complexity use simple conversational language

bull Avoid leading and loaded questions

bull Avoid ambiguity be as specific as possible

bull Avoid double-barreled items

bull Avoid making assumptions

bull Avoid burdensome questions

Double-barrelled questionsTwo concepts in one question

bull Have you had a neck ache or a back achesince your last visit

bull Since your last visit have you had any of the following symptoms (tick as many that apply)

bull Neck ache 1048715 bull Back ache 1048715 bull Headache 1048715

Leading and Loaded questionsbull A leading question suggests or implies certain

answersDo accounting graduates who attended state universities such as Washington State University make better auditors

bull A loaded question suggests a socially desirable answer or is emotionally charged

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Vague questions

Avoid vague questionsbull Taken altogether how happy are you with

your stay in hospitalbull Overall how would you describe the care

you received in hospital

Avoid Ambiguity Words such as ldquooftenrdquo ldquooccasionallyrdquo ldquousuallyrdquo ldquoregularlyrdquo ldquofrequentlyrdquo ldquomanyrdquo should be used with caution If these words have to be used their meaning should be explained properly

Avoid Making Assumptionsbull Is the question applicable to all

respondents

Why do you like fast-food

Assumes respondents like fast foodsBetter strategy would be to ask a filter

question first

Burdensome QuestionsAvoid Burdensome Questions That MayTax the Respondentrsquos Memory telephone

A survey conducted during the 24-hour period following the airing of the Super Bowl might establish whether the respondent watched the Super Bowl and then ask ldquoDo you recall any commercials on that programrdquo

Question Wordingbull Question length

ndash Should be shortndash Longer questions confuse and fatigue respondents

bull Sensitive questionsndash Questions on information perceived to be

embarrassing like personal income criminal activities alcoholism smoking drugs habits social desirability issues etc

ndash Creativity rules (assurances of confidentiality anonymity slipping it in sideways open-ended questions asking in third person etc)

Sequence And Layout Decisions

bull Opening questions ndash easy and non-threateningbull Flow ndash smooth and logical ndash avoid jumpsbull Broad to specificbull Critical questions ndash placed in the middle bull Appealing and interestingbull Order bias ndash the possibility that subsequent responses

are influenced by preceding responses eg fewer people will say that their taxes are too high after being asked whether govt spending should be increased in certain areas

bull Demographic questions ndash first or last

Layout

bull 1048715 Just as important as wordingbull 1048715 Aim for a professional lookbull 1048715 Tipsbull ndash cover letterintroductory page giving study title organisation aims of the surveybull ndash enough space for open-ended questionsbull ndash font large enough to read without strainbull ndash consistent and clear instructionsbull ndash donrsquot split questions or answers across pagesbull ndash enough white space

Distinguishing question and responses

LAYOUT ADo you agree disagree or have no opinion that this company

has bull A good vacation policy - disagreenot sureagreebull Good management feedback - disagreenot sureagreebull Good medical insurance - disagreenot sureagree

LAYOUT B Does this company have bull Disagree Not Sure Agree bull A good vacation policy 1 2 3 bull Good feedback 1 2 3 bull Good medical insurance 1 2 3

Pretesting and Correcting Problems

Purpose of pretest To ensure that the questionnaire meets the expectations in terms of the information that will be obtained

bull Pretesting Specific Questions Forbull Meaningbull Task difficultybull Respondent interest and attention

bull Pretesting the overall Questionnairebull Flow of the questionnairebull Skip patternsbull Length

bull Put yourself in the respondentrsquos shoes and answer the questionnaire

• Questionnaire Design
• A QUESTIONNAIRE IS ONLY AS GOOD AS THE QUESTIONS IT ASKS
• Slide 3
• Logical Steps to Develop a Good Questionnaire
• What Should Be Asked
• Validity and reliability
• Type of information
• Knowledge
• Opinions
• Types of Questions
• Open Ended Questions
• Closed-response Questions
• Closed-response Questions (2)
• Closed-response Questions (3)
• Closed-Ended Questions
• Issues in Questionnaire Design
• Contents of the Questionnaire
• Filter questions
• Slide 19
• Issues in Questionnaire Design (2)
• Slide 21
• Double-barrelled questions
• Leading and Loaded questions
• Slide 24
• Vague questions
• Avoid Making Assumptions
• Burdensome Questions
• Question Wording
• Sequence And Layout Decisions
• Layout
• Distinguishing question and responses
• Pretesting and Correcting Problems

Types of Questions1 Open-Ended question

People look for different things in a job What would you prefer most in a job

2 Closed-Ended questionPeople look for different things in a job What would you prefer most in a jobndash Work that pays wellndash Work that gives a sense of accomplishmentndash Work where you make most decisions by yourselfndash Work that is steady with little chance of being laid

off

Open Ended Questionsbull Advantages

ndash Gain insight into the problemndash Too many options to listndash When verbatim responses are desired to give the flavor of the problemndash When behavior to be measured is sensitive or disapprovedndash Interviewer questionnaire structure influence can be minimal

bull Disadvantagesndash Inarticulate respondentsndash Interviewerrsquos skill in recording quickly and summarizing accuratelyndash Time consuming subjective judgments while tabulating adds to cost

Closed-response Questions

Two Basic Formats for Closed Ended or Structured Questions

bull Choice from a list of responses

bull Appropriate single-choice rating on a scale

Closed-response Questions

What type of fast-food restaurant do you visit most often

Burger MexicanChicken PizzaSeafood ChineseDonrsquot know Other (please specify)

Closed-response Questions

What is your overall satisfaction with McDonalds burgers

Very satisfied Quite Satisfied Somewhat satisfied Not at all satisfied

Very satisfied 7 6 5 4 3 2 1 Not at all satisfied

Closed-Ended Questionsbull Advantages

ndash Easy to understand quick responses possiblendash Micro-differences in responses can be capturedndash Easier tabulation and analysisndash Answers are directly comparable from respondent to respondent

bull Disadvantagesndash Neutral category may attract more responses than warrantedndash Information between categories may be lost (extreme case ndash

dichotomous categories)

Issues in Questionnaire Design

bull Respondent uncertainty

ndash Should respondents be provided with aDonrsquot know or No opinion option

ndash When it is important to differentiate between ambivalence and ignorance both options should be provided

Contents of the Questionnaire bull Question Wording ndash Vocabulary

- Depends on the nature of variable tapped- subjective feeling or objective facts

ndash Simple easy to understand commonly used language

ndash Avoid technical words and jargon (unless sample is technically qualified)

Filter questions Filter questions useful to ensure respondents

only answer relevant parts of questionsUnfilteredbull ndash If you use a medical software programbull which one do you use

Filteredbull Do you use a medical software programbull 1048715 No - jump to next questionbull 1048715 Yes - which one

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided------------------------------------------------------------------------------

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Issues in Questionnaire Design

bull Question Wordingbull vocabulary bull Questions need to be clear simple and

precisebull Poorly written questions lead to ambiguity

and misunderstandings and can be wasteful

bull Avoid Complexity use simple conversational language

bull Avoid leading and loaded questions

bull Avoid ambiguity be as specific as possible

bull Avoid double-barreled items

bull Avoid making assumptions

bull Avoid burdensome questions

Double-barrelled questionsTwo concepts in one question

bull Have you had a neck ache or a back achesince your last visit

bull Since your last visit have you had any of the following symptoms (tick as many that apply)

bull Neck ache 1048715 bull Back ache 1048715 bull Headache 1048715

Leading and Loaded questionsbull A leading question suggests or implies certain

answersDo accounting graduates who attended state universities such as Washington State University make better auditors

bull A loaded question suggests a socially desirable answer or is emotionally charged

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Vague questions

Avoid vague questionsbull Taken altogether how happy are you with

your stay in hospitalbull Overall how would you describe the care

you received in hospital

Avoid Ambiguity Words such as ldquooftenrdquo ldquooccasionallyrdquo ldquousuallyrdquo ldquoregularlyrdquo ldquofrequentlyrdquo ldquomanyrdquo should be used with caution If these words have to be used their meaning should be explained properly

Avoid Making Assumptionsbull Is the question applicable to all

respondents

Why do you like fast-food

Assumes respondents like fast foodsBetter strategy would be to ask a filter

question first

Burdensome QuestionsAvoid Burdensome Questions That MayTax the Respondentrsquos Memory telephone

A survey conducted during the 24-hour period following the airing of the Super Bowl might establish whether the respondent watched the Super Bowl and then ask ldquoDo you recall any commercials on that programrdquo

Question Wordingbull Question length

ndash Should be shortndash Longer questions confuse and fatigue respondents

bull Sensitive questionsndash Questions on information perceived to be

embarrassing like personal income criminal activities alcoholism smoking drugs habits social desirability issues etc

ndash Creativity rules (assurances of confidentiality anonymity slipping it in sideways open-ended questions asking in third person etc)

Sequence And Layout Decisions

bull Opening questions ndash easy and non-threateningbull Flow ndash smooth and logical ndash avoid jumpsbull Broad to specificbull Critical questions ndash placed in the middle bull Appealing and interestingbull Order bias ndash the possibility that subsequent responses

are influenced by preceding responses eg fewer people will say that their taxes are too high after being asked whether govt spending should be increased in certain areas

bull Demographic questions ndash first or last

Layout

bull 1048715 Just as important as wordingbull 1048715 Aim for a professional lookbull 1048715 Tipsbull ndash cover letterintroductory page giving study title organisation aims of the surveybull ndash enough space for open-ended questionsbull ndash font large enough to read without strainbull ndash consistent and clear instructionsbull ndash donrsquot split questions or answers across pagesbull ndash enough white space

Distinguishing question and responses

LAYOUT ADo you agree disagree or have no opinion that this company

has bull A good vacation policy - disagreenot sureagreebull Good management feedback - disagreenot sureagreebull Good medical insurance - disagreenot sureagree

LAYOUT B Does this company have bull Disagree Not Sure Agree bull A good vacation policy 1 2 3 bull Good feedback 1 2 3 bull Good medical insurance 1 2 3

Pretesting and Correcting Problems

Purpose of pretest To ensure that the questionnaire meets the expectations in terms of the information that will be obtained

bull Pretesting Specific Questions Forbull Meaningbull Task difficultybull Respondent interest and attention

bull Pretesting the overall Questionnairebull Flow of the questionnairebull Skip patternsbull Length

bull Put yourself in the respondentrsquos shoes and answer the questionnaire

• Questionnaire Design
• A QUESTIONNAIRE IS ONLY AS GOOD AS THE QUESTIONS IT ASKS
• Slide 3
• Logical Steps to Develop a Good Questionnaire
• What Should Be Asked
• Validity and reliability
• Type of information
• Knowledge
• Opinions
• Types of Questions
• Open Ended Questions
• Closed-response Questions
• Closed-response Questions (2)
• Closed-response Questions (3)
• Closed-Ended Questions
• Issues in Questionnaire Design
• Contents of the Questionnaire
• Filter questions
• Slide 19
• Issues in Questionnaire Design (2)
• Slide 21
• Double-barrelled questions
• Leading and Loaded questions
• Slide 24
• Vague questions
• Avoid Making Assumptions
• Burdensome Questions
• Question Wording
• Sequence And Layout Decisions
• Layout
• Distinguishing question and responses
• Pretesting and Correcting Problems

Open Ended Questionsbull Advantages

ndash Gain insight into the problemndash Too many options to listndash When verbatim responses are desired to give the flavor of the problemndash When behavior to be measured is sensitive or disapprovedndash Interviewer questionnaire structure influence can be minimal

bull Disadvantagesndash Inarticulate respondentsndash Interviewerrsquos skill in recording quickly and summarizing accuratelyndash Time consuming subjective judgments while tabulating adds to cost

Closed-response Questions

Two Basic Formats for Closed Ended or Structured Questions

bull Choice from a list of responses

bull Appropriate single-choice rating on a scale

Closed-response Questions

What type of fast-food restaurant do you visit most often

Burger MexicanChicken PizzaSeafood ChineseDonrsquot know Other (please specify)

Closed-response Questions

What is your overall satisfaction with McDonalds burgers

Very satisfied Quite Satisfied Somewhat satisfied Not at all satisfied

Very satisfied 7 6 5 4 3 2 1 Not at all satisfied

Closed-Ended Questionsbull Advantages

ndash Easy to understand quick responses possiblendash Micro-differences in responses can be capturedndash Easier tabulation and analysisndash Answers are directly comparable from respondent to respondent

bull Disadvantagesndash Neutral category may attract more responses than warrantedndash Information between categories may be lost (extreme case ndash

dichotomous categories)

Issues in Questionnaire Design

bull Respondent uncertainty

ndash Should respondents be provided with aDonrsquot know or No opinion option

ndash When it is important to differentiate between ambivalence and ignorance both options should be provided

Contents of the Questionnaire bull Question Wording ndash Vocabulary

- Depends on the nature of variable tapped- subjective feeling or objective facts

ndash Simple easy to understand commonly used language

ndash Avoid technical words and jargon (unless sample is technically qualified)

Filter questions Filter questions useful to ensure respondents

only answer relevant parts of questionsUnfilteredbull ndash If you use a medical software programbull which one do you use

Filteredbull Do you use a medical software programbull 1048715 No - jump to next questionbull 1048715 Yes - which one

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided------------------------------------------------------------------------------

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Issues in Questionnaire Design

bull Question Wordingbull vocabulary bull Questions need to be clear simple and

precisebull Poorly written questions lead to ambiguity

and misunderstandings and can be wasteful

bull Avoid Complexity use simple conversational language

bull Avoid leading and loaded questions

bull Avoid ambiguity be as specific as possible

bull Avoid double-barreled items

bull Avoid making assumptions

bull Avoid burdensome questions

Double-barrelled questionsTwo concepts in one question

bull Have you had a neck ache or a back achesince your last visit

bull Since your last visit have you had any of the following symptoms (tick as many that apply)

bull Neck ache 1048715 bull Back ache 1048715 bull Headache 1048715

Leading and Loaded questionsbull A leading question suggests or implies certain

answersDo accounting graduates who attended state universities such as Washington State University make better auditors

bull A loaded question suggests a socially desirable answer or is emotionally charged

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Vague questions

Avoid vague questionsbull Taken altogether how happy are you with

your stay in hospitalbull Overall how would you describe the care

you received in hospital

Avoid Ambiguity Words such as ldquooftenrdquo ldquooccasionallyrdquo ldquousuallyrdquo ldquoregularlyrdquo ldquofrequentlyrdquo ldquomanyrdquo should be used with caution If these words have to be used their meaning should be explained properly

Avoid Making Assumptionsbull Is the question applicable to all

respondents

Why do you like fast-food

Assumes respondents like fast foodsBetter strategy would be to ask a filter

question first

Burdensome QuestionsAvoid Burdensome Questions That MayTax the Respondentrsquos Memory telephone

A survey conducted during the 24-hour period following the airing of the Super Bowl might establish whether the respondent watched the Super Bowl and then ask ldquoDo you recall any commercials on that programrdquo

Question Wordingbull Question length

ndash Should be shortndash Longer questions confuse and fatigue respondents

bull Sensitive questionsndash Questions on information perceived to be

embarrassing like personal income criminal activities alcoholism smoking drugs habits social desirability issues etc

ndash Creativity rules (assurances of confidentiality anonymity slipping it in sideways open-ended questions asking in third person etc)

Sequence And Layout Decisions

bull Opening questions ndash easy and non-threateningbull Flow ndash smooth and logical ndash avoid jumpsbull Broad to specificbull Critical questions ndash placed in the middle bull Appealing and interestingbull Order bias ndash the possibility that subsequent responses

are influenced by preceding responses eg fewer people will say that their taxes are too high after being asked whether govt spending should be increased in certain areas

bull Demographic questions ndash first or last

Layout

bull 1048715 Just as important as wordingbull 1048715 Aim for a professional lookbull 1048715 Tipsbull ndash cover letterintroductory page giving study title organisation aims of the surveybull ndash enough space for open-ended questionsbull ndash font large enough to read without strainbull ndash consistent and clear instructionsbull ndash donrsquot split questions or answers across pagesbull ndash enough white space

Distinguishing question and responses

LAYOUT ADo you agree disagree or have no opinion that this company

has bull A good vacation policy - disagreenot sureagreebull Good management feedback - disagreenot sureagreebull Good medical insurance - disagreenot sureagree

LAYOUT B Does this company have bull Disagree Not Sure Agree bull A good vacation policy 1 2 3 bull Good feedback 1 2 3 bull Good medical insurance 1 2 3

Pretesting and Correcting Problems

Purpose of pretest To ensure that the questionnaire meets the expectations in terms of the information that will be obtained

bull Pretesting Specific Questions Forbull Meaningbull Task difficultybull Respondent interest and attention

bull Pretesting the overall Questionnairebull Flow of the questionnairebull Skip patternsbull Length

bull Put yourself in the respondentrsquos shoes and answer the questionnaire

• Questionnaire Design
• A QUESTIONNAIRE IS ONLY AS GOOD AS THE QUESTIONS IT ASKS
• Slide 3
• Logical Steps to Develop a Good Questionnaire
• What Should Be Asked
• Validity and reliability
• Type of information
• Knowledge
• Opinions
• Types of Questions
• Open Ended Questions
• Closed-response Questions
• Closed-response Questions (2)
• Closed-response Questions (3)
• Closed-Ended Questions
• Issues in Questionnaire Design
• Contents of the Questionnaire
• Filter questions
• Slide 19
• Issues in Questionnaire Design (2)
• Slide 21
• Double-barrelled questions
• Leading and Loaded questions
• Slide 24
• Vague questions
• Avoid Making Assumptions
• Burdensome Questions
• Question Wording
• Sequence And Layout Decisions
• Layout
• Distinguishing question and responses
• Pretesting and Correcting Problems

Closed-response Questions

Two Basic Formats for Closed Ended or Structured Questions

bull Choice from a list of responses

bull Appropriate single-choice rating on a scale

Closed-response Questions

What type of fast-food restaurant do you visit most often

Burger MexicanChicken PizzaSeafood ChineseDonrsquot know Other (please specify)

Closed-response Questions

What is your overall satisfaction with McDonalds burgers

Very satisfied Quite Satisfied Somewhat satisfied Not at all satisfied

Very satisfied 7 6 5 4 3 2 1 Not at all satisfied

Closed-Ended Questionsbull Advantages

ndash Easy to understand quick responses possiblendash Micro-differences in responses can be capturedndash Easier tabulation and analysisndash Answers are directly comparable from respondent to respondent

bull Disadvantagesndash Neutral category may attract more responses than warrantedndash Information between categories may be lost (extreme case ndash

dichotomous categories)

Issues in Questionnaire Design

bull Respondent uncertainty

ndash Should respondents be provided with aDonrsquot know or No opinion option

ndash When it is important to differentiate between ambivalence and ignorance both options should be provided

Contents of the Questionnaire bull Question Wording ndash Vocabulary

- Depends on the nature of variable tapped- subjective feeling or objective facts

ndash Simple easy to understand commonly used language

ndash Avoid technical words and jargon (unless sample is technically qualified)

Filter questions Filter questions useful to ensure respondents

only answer relevant parts of questionsUnfilteredbull ndash If you use a medical software programbull which one do you use

Filteredbull Do you use a medical software programbull 1048715 No - jump to next questionbull 1048715 Yes - which one

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided------------------------------------------------------------------------------

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Issues in Questionnaire Design

bull Question Wordingbull vocabulary bull Questions need to be clear simple and

precisebull Poorly written questions lead to ambiguity

and misunderstandings and can be wasteful

bull Avoid Complexity use simple conversational language

bull Avoid leading and loaded questions

bull Avoid ambiguity be as specific as possible

bull Avoid double-barreled items

bull Avoid making assumptions

bull Avoid burdensome questions

Double-barrelled questionsTwo concepts in one question

bull Have you had a neck ache or a back achesince your last visit

bull Since your last visit have you had any of the following symptoms (tick as many that apply)

bull Neck ache 1048715 bull Back ache 1048715 bull Headache 1048715

Leading and Loaded questionsbull A leading question suggests or implies certain

answersDo accounting graduates who attended state universities such as Washington State University make better auditors

bull A loaded question suggests a socially desirable answer or is emotionally charged

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Vague questions

Avoid vague questionsbull Taken altogether how happy are you with

your stay in hospitalbull Overall how would you describe the care

you received in hospital

Avoid Ambiguity Words such as ldquooftenrdquo ldquooccasionallyrdquo ldquousuallyrdquo ldquoregularlyrdquo ldquofrequentlyrdquo ldquomanyrdquo should be used with caution If these words have to be used their meaning should be explained properly

Avoid Making Assumptionsbull Is the question applicable to all

respondents

Why do you like fast-food

Assumes respondents like fast foodsBetter strategy would be to ask a filter

question first

Burdensome QuestionsAvoid Burdensome Questions That MayTax the Respondentrsquos Memory telephone

A survey conducted during the 24-hour period following the airing of the Super Bowl might establish whether the respondent watched the Super Bowl and then ask ldquoDo you recall any commercials on that programrdquo

Question Wordingbull Question length

ndash Should be shortndash Longer questions confuse and fatigue respondents

bull Sensitive questionsndash Questions on information perceived to be

embarrassing like personal income criminal activities alcoholism smoking drugs habits social desirability issues etc

ndash Creativity rules (assurances of confidentiality anonymity slipping it in sideways open-ended questions asking in third person etc)

Sequence And Layout Decisions

bull Opening questions ndash easy and non-threateningbull Flow ndash smooth and logical ndash avoid jumpsbull Broad to specificbull Critical questions ndash placed in the middle bull Appealing and interestingbull Order bias ndash the possibility that subsequent responses

are influenced by preceding responses eg fewer people will say that their taxes are too high after being asked whether govt spending should be increased in certain areas

bull Demographic questions ndash first or last

Layout

bull 1048715 Just as important as wordingbull 1048715 Aim for a professional lookbull 1048715 Tipsbull ndash cover letterintroductory page giving study title organisation aims of the surveybull ndash enough space for open-ended questionsbull ndash font large enough to read without strainbull ndash consistent and clear instructionsbull ndash donrsquot split questions or answers across pagesbull ndash enough white space

Distinguishing question and responses

LAYOUT ADo you agree disagree or have no opinion that this company

has bull A good vacation policy - disagreenot sureagreebull Good management feedback - disagreenot sureagreebull Good medical insurance - disagreenot sureagree

LAYOUT B Does this company have bull Disagree Not Sure Agree bull A good vacation policy 1 2 3 bull Good feedback 1 2 3 bull Good medical insurance 1 2 3

Pretesting and Correcting Problems

Purpose of pretest To ensure that the questionnaire meets the expectations in terms of the information that will be obtained

bull Pretesting Specific Questions Forbull Meaningbull Task difficultybull Respondent interest and attention

bull Pretesting the overall Questionnairebull Flow of the questionnairebull Skip patternsbull Length

bull Put yourself in the respondentrsquos shoes and answer the questionnaire

• Questionnaire Design
• A QUESTIONNAIRE IS ONLY AS GOOD AS THE QUESTIONS IT ASKS
• Slide 3
• Logical Steps to Develop a Good Questionnaire
• What Should Be Asked
• Validity and reliability
• Type of information
• Knowledge
• Opinions
• Types of Questions
• Open Ended Questions
• Closed-response Questions
• Closed-response Questions (2)
• Closed-response Questions (3)
• Closed-Ended Questions
• Issues in Questionnaire Design
• Contents of the Questionnaire
• Filter questions
• Slide 19
• Issues in Questionnaire Design (2)
• Slide 21
• Double-barrelled questions
• Leading and Loaded questions
• Slide 24
• Vague questions
• Avoid Making Assumptions
• Burdensome Questions
• Question Wording
• Sequence And Layout Decisions
• Layout
• Distinguishing question and responses
• Pretesting and Correcting Problems

Closed-response Questions

What type of fast-food restaurant do you visit most often

Burger MexicanChicken PizzaSeafood ChineseDonrsquot know Other (please specify)

Closed-response Questions

What is your overall satisfaction with McDonalds burgers

Very satisfied Quite Satisfied Somewhat satisfied Not at all satisfied

Very satisfied 7 6 5 4 3 2 1 Not at all satisfied

Closed-Ended Questionsbull Advantages

ndash Easy to understand quick responses possiblendash Micro-differences in responses can be capturedndash Easier tabulation and analysisndash Answers are directly comparable from respondent to respondent

bull Disadvantagesndash Neutral category may attract more responses than warrantedndash Information between categories may be lost (extreme case ndash

dichotomous categories)

Issues in Questionnaire Design

bull Respondent uncertainty

ndash Should respondents be provided with aDonrsquot know or No opinion option

ndash When it is important to differentiate between ambivalence and ignorance both options should be provided

Contents of the Questionnaire bull Question Wording ndash Vocabulary

- Depends on the nature of variable tapped- subjective feeling or objective facts

ndash Simple easy to understand commonly used language

ndash Avoid technical words and jargon (unless sample is technically qualified)

Filter questions Filter questions useful to ensure respondents

only answer relevant parts of questionsUnfilteredbull ndash If you use a medical software programbull which one do you use

Filteredbull Do you use a medical software programbull 1048715 No - jump to next questionbull 1048715 Yes - which one

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided------------------------------------------------------------------------------

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Issues in Questionnaire Design

bull Question Wordingbull vocabulary bull Questions need to be clear simple and

precisebull Poorly written questions lead to ambiguity

and misunderstandings and can be wasteful

bull Avoid Complexity use simple conversational language

bull Avoid leading and loaded questions

bull Avoid ambiguity be as specific as possible

bull Avoid double-barreled items

bull Avoid making assumptions

bull Avoid burdensome questions

Double-barrelled questionsTwo concepts in one question

bull Have you had a neck ache or a back achesince your last visit

bull Since your last visit have you had any of the following symptoms (tick as many that apply)

bull Neck ache 1048715 bull Back ache 1048715 bull Headache 1048715

Leading and Loaded questionsbull A leading question suggests or implies certain

answersDo accounting graduates who attended state universities such as Washington State University make better auditors

bull A loaded question suggests a socially desirable answer or is emotionally charged

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Vague questions

Avoid vague questionsbull Taken altogether how happy are you with

your stay in hospitalbull Overall how would you describe the care

you received in hospital

Avoid Ambiguity Words such as ldquooftenrdquo ldquooccasionallyrdquo ldquousuallyrdquo ldquoregularlyrdquo ldquofrequentlyrdquo ldquomanyrdquo should be used with caution If these words have to be used their meaning should be explained properly

Avoid Making Assumptionsbull Is the question applicable to all

respondents

Why do you like fast-food

Assumes respondents like fast foodsBetter strategy would be to ask a filter

question first

Burdensome QuestionsAvoid Burdensome Questions That MayTax the Respondentrsquos Memory telephone

A survey conducted during the 24-hour period following the airing of the Super Bowl might establish whether the respondent watched the Super Bowl and then ask ldquoDo you recall any commercials on that programrdquo

Question Wordingbull Question length

ndash Should be shortndash Longer questions confuse and fatigue respondents

bull Sensitive questionsndash Questions on information perceived to be

embarrassing like personal income criminal activities alcoholism smoking drugs habits social desirability issues etc

ndash Creativity rules (assurances of confidentiality anonymity slipping it in sideways open-ended questions asking in third person etc)

Sequence And Layout Decisions

bull Opening questions ndash easy and non-threateningbull Flow ndash smooth and logical ndash avoid jumpsbull Broad to specificbull Critical questions ndash placed in the middle bull Appealing and interestingbull Order bias ndash the possibility that subsequent responses

are influenced by preceding responses eg fewer people will say that their taxes are too high after being asked whether govt spending should be increased in certain areas

bull Demographic questions ndash first or last

Layout

bull 1048715 Just as important as wordingbull 1048715 Aim for a professional lookbull 1048715 Tipsbull ndash cover letterintroductory page giving study title organisation aims of the surveybull ndash enough space for open-ended questionsbull ndash font large enough to read without strainbull ndash consistent and clear instructionsbull ndash donrsquot split questions or answers across pagesbull ndash enough white space

Distinguishing question and responses

LAYOUT ADo you agree disagree or have no opinion that this company

has bull A good vacation policy - disagreenot sureagreebull Good management feedback - disagreenot sureagreebull Good medical insurance - disagreenot sureagree

LAYOUT B Does this company have bull Disagree Not Sure Agree bull A good vacation policy 1 2 3 bull Good feedback 1 2 3 bull Good medical insurance 1 2 3

Pretesting and Correcting Problems

Purpose of pretest To ensure that the questionnaire meets the expectations in terms of the information that will be obtained

bull Pretesting Specific Questions Forbull Meaningbull Task difficultybull Respondent interest and attention

bull Pretesting the overall Questionnairebull Flow of the questionnairebull Skip patternsbull Length

bull Put yourself in the respondentrsquos shoes and answer the questionnaire

• Questionnaire Design
• A QUESTIONNAIRE IS ONLY AS GOOD AS THE QUESTIONS IT ASKS
• Slide 3
• Logical Steps to Develop a Good Questionnaire
• What Should Be Asked
• Validity and reliability
• Type of information
• Knowledge
• Opinions
• Types of Questions
• Open Ended Questions
• Closed-response Questions
• Closed-response Questions (2)
• Closed-response Questions (3)
• Closed-Ended Questions
• Issues in Questionnaire Design
• Contents of the Questionnaire
• Filter questions
• Slide 19
• Issues in Questionnaire Design (2)
• Slide 21
• Double-barrelled questions
• Leading and Loaded questions
• Slide 24
• Vague questions
• Avoid Making Assumptions
• Burdensome Questions
• Question Wording
• Sequence And Layout Decisions
• Layout
• Distinguishing question and responses
• Pretesting and Correcting Problems

Closed-response Questions

What is your overall satisfaction with McDonalds burgers

Very satisfied Quite Satisfied Somewhat satisfied Not at all satisfied

Very satisfied 7 6 5 4 3 2 1 Not at all satisfied

Closed-Ended Questionsbull Advantages

ndash Easy to understand quick responses possiblendash Micro-differences in responses can be capturedndash Easier tabulation and analysisndash Answers are directly comparable from respondent to respondent

bull Disadvantagesndash Neutral category may attract more responses than warrantedndash Information between categories may be lost (extreme case ndash

dichotomous categories)

Issues in Questionnaire Design

bull Respondent uncertainty

ndash Should respondents be provided with aDonrsquot know or No opinion option

ndash When it is important to differentiate between ambivalence and ignorance both options should be provided

Contents of the Questionnaire bull Question Wording ndash Vocabulary

- Depends on the nature of variable tapped- subjective feeling or objective facts

ndash Simple easy to understand commonly used language

ndash Avoid technical words and jargon (unless sample is technically qualified)

Filter questions Filter questions useful to ensure respondents

only answer relevant parts of questionsUnfilteredbull ndash If you use a medical software programbull which one do you use

Filteredbull Do you use a medical software programbull 1048715 No - jump to next questionbull 1048715 Yes - which one

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided------------------------------------------------------------------------------

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Issues in Questionnaire Design

bull Question Wordingbull vocabulary bull Questions need to be clear simple and

precisebull Poorly written questions lead to ambiguity

and misunderstandings and can be wasteful

bull Avoid Complexity use simple conversational language

bull Avoid leading and loaded questions

bull Avoid ambiguity be as specific as possible

bull Avoid double-barreled items

bull Avoid making assumptions

bull Avoid burdensome questions

Double-barrelled questionsTwo concepts in one question

bull Have you had a neck ache or a back achesince your last visit

bull Since your last visit have you had any of the following symptoms (tick as many that apply)

bull Neck ache 1048715 bull Back ache 1048715 bull Headache 1048715

Leading and Loaded questionsbull A leading question suggests or implies certain

answersDo accounting graduates who attended state universities such as Washington State University make better auditors

bull A loaded question suggests a socially desirable answer or is emotionally charged

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Vague questions

Avoid vague questionsbull Taken altogether how happy are you with

your stay in hospitalbull Overall how would you describe the care

you received in hospital

Avoid Ambiguity Words such as ldquooftenrdquo ldquooccasionallyrdquo ldquousuallyrdquo ldquoregularlyrdquo ldquofrequentlyrdquo ldquomanyrdquo should be used with caution If these words have to be used their meaning should be explained properly

Avoid Making Assumptionsbull Is the question applicable to all

respondents

Why do you like fast-food

Assumes respondents like fast foodsBetter strategy would be to ask a filter

question first

Burdensome QuestionsAvoid Burdensome Questions That MayTax the Respondentrsquos Memory telephone

A survey conducted during the 24-hour period following the airing of the Super Bowl might establish whether the respondent watched the Super Bowl and then ask ldquoDo you recall any commercials on that programrdquo

Question Wordingbull Question length

ndash Should be shortndash Longer questions confuse and fatigue respondents

bull Sensitive questionsndash Questions on information perceived to be

embarrassing like personal income criminal activities alcoholism smoking drugs habits social desirability issues etc

ndash Creativity rules (assurances of confidentiality anonymity slipping it in sideways open-ended questions asking in third person etc)

Sequence And Layout Decisions

bull Opening questions ndash easy and non-threateningbull Flow ndash smooth and logical ndash avoid jumpsbull Broad to specificbull Critical questions ndash placed in the middle bull Appealing and interestingbull Order bias ndash the possibility that subsequent responses

are influenced by preceding responses eg fewer people will say that their taxes are too high after being asked whether govt spending should be increased in certain areas

bull Demographic questions ndash first or last

Layout

bull 1048715 Just as important as wordingbull 1048715 Aim for a professional lookbull 1048715 Tipsbull ndash cover letterintroductory page giving study title organisation aims of the surveybull ndash enough space for open-ended questionsbull ndash font large enough to read without strainbull ndash consistent and clear instructionsbull ndash donrsquot split questions or answers across pagesbull ndash enough white space

Distinguishing question and responses

LAYOUT ADo you agree disagree or have no opinion that this company

has bull A good vacation policy - disagreenot sureagreebull Good management feedback - disagreenot sureagreebull Good medical insurance - disagreenot sureagree

LAYOUT B Does this company have bull Disagree Not Sure Agree bull A good vacation policy 1 2 3 bull Good feedback 1 2 3 bull Good medical insurance 1 2 3

Pretesting and Correcting Problems

Purpose of pretest To ensure that the questionnaire meets the expectations in terms of the information that will be obtained

bull Pretesting Specific Questions Forbull Meaningbull Task difficultybull Respondent interest and attention

bull Pretesting the overall Questionnairebull Flow of the questionnairebull Skip patternsbull Length

bull Put yourself in the respondentrsquos shoes and answer the questionnaire

• Questionnaire Design
• A QUESTIONNAIRE IS ONLY AS GOOD AS THE QUESTIONS IT ASKS
• Slide 3
• Logical Steps to Develop a Good Questionnaire
• What Should Be Asked
• Validity and reliability
• Type of information
• Knowledge
• Opinions
• Types of Questions
• Open Ended Questions
• Closed-response Questions
• Closed-response Questions (2)
• Closed-response Questions (3)
• Closed-Ended Questions
• Issues in Questionnaire Design
• Contents of the Questionnaire
• Filter questions
• Slide 19
• Issues in Questionnaire Design (2)
• Slide 21
• Double-barrelled questions
• Leading and Loaded questions
• Slide 24
• Vague questions
• Avoid Making Assumptions
• Burdensome Questions
• Question Wording
• Sequence And Layout Decisions
• Layout
• Distinguishing question and responses
• Pretesting and Correcting Problems

Closed-Ended Questionsbull Advantages

ndash Easy to understand quick responses possiblendash Micro-differences in responses can be capturedndash Easier tabulation and analysisndash Answers are directly comparable from respondent to respondent

bull Disadvantagesndash Neutral category may attract more responses than warrantedndash Information between categories may be lost (extreme case ndash

dichotomous categories)

Issues in Questionnaire Design

bull Respondent uncertainty

ndash Should respondents be provided with aDonrsquot know or No opinion option

ndash When it is important to differentiate between ambivalence and ignorance both options should be provided

Contents of the Questionnaire bull Question Wording ndash Vocabulary

- Depends on the nature of variable tapped- subjective feeling or objective facts

ndash Simple easy to understand commonly used language

ndash Avoid technical words and jargon (unless sample is technically qualified)

Filter questions Filter questions useful to ensure respondents

only answer relevant parts of questionsUnfilteredbull ndash If you use a medical software programbull which one do you use

Filteredbull Do you use a medical software programbull 1048715 No - jump to next questionbull 1048715 Yes - which one

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided------------------------------------------------------------------------------

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Issues in Questionnaire Design

bull Question Wordingbull vocabulary bull Questions need to be clear simple and

precisebull Poorly written questions lead to ambiguity

and misunderstandings and can be wasteful

bull Avoid Complexity use simple conversational language

bull Avoid leading and loaded questions

bull Avoid ambiguity be as specific as possible

bull Avoid double-barreled items

bull Avoid making assumptions

bull Avoid burdensome questions

Double-barrelled questionsTwo concepts in one question

bull Have you had a neck ache or a back achesince your last visit

bull Since your last visit have you had any of the following symptoms (tick as many that apply)

bull Neck ache 1048715 bull Back ache 1048715 bull Headache 1048715

Leading and Loaded questionsbull A leading question suggests or implies certain

answersDo accounting graduates who attended state universities such as Washington State University make better auditors

bull A loaded question suggests a socially desirable answer or is emotionally charged

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Vague questions

Avoid vague questionsbull Taken altogether how happy are you with

your stay in hospitalbull Overall how would you describe the care

you received in hospital

Avoid Ambiguity Words such as ldquooftenrdquo ldquooccasionallyrdquo ldquousuallyrdquo ldquoregularlyrdquo ldquofrequentlyrdquo ldquomanyrdquo should be used with caution If these words have to be used their meaning should be explained properly

Avoid Making Assumptionsbull Is the question applicable to all

respondents

Why do you like fast-food

Assumes respondents like fast foodsBetter strategy would be to ask a filter

question first

Burdensome QuestionsAvoid Burdensome Questions That MayTax the Respondentrsquos Memory telephone

A survey conducted during the 24-hour period following the airing of the Super Bowl might establish whether the respondent watched the Super Bowl and then ask ldquoDo you recall any commercials on that programrdquo

Question Wordingbull Question length

ndash Should be shortndash Longer questions confuse and fatigue respondents

bull Sensitive questionsndash Questions on information perceived to be

embarrassing like personal income criminal activities alcoholism smoking drugs habits social desirability issues etc

ndash Creativity rules (assurances of confidentiality anonymity slipping it in sideways open-ended questions asking in third person etc)

Sequence And Layout Decisions

bull Opening questions ndash easy and non-threateningbull Flow ndash smooth and logical ndash avoid jumpsbull Broad to specificbull Critical questions ndash placed in the middle bull Appealing and interestingbull Order bias ndash the possibility that subsequent responses

are influenced by preceding responses eg fewer people will say that their taxes are too high after being asked whether govt spending should be increased in certain areas

bull Demographic questions ndash first or last

Layout

bull 1048715 Just as important as wordingbull 1048715 Aim for a professional lookbull 1048715 Tipsbull ndash cover letterintroductory page giving study title organisation aims of the surveybull ndash enough space for open-ended questionsbull ndash font large enough to read without strainbull ndash consistent and clear instructionsbull ndash donrsquot split questions or answers across pagesbull ndash enough white space

Distinguishing question and responses

LAYOUT ADo you agree disagree or have no opinion that this company

has bull A good vacation policy - disagreenot sureagreebull Good management feedback - disagreenot sureagreebull Good medical insurance - disagreenot sureagree

LAYOUT B Does this company have bull Disagree Not Sure Agree bull A good vacation policy 1 2 3 bull Good feedback 1 2 3 bull Good medical insurance 1 2 3

Pretesting and Correcting Problems

Purpose of pretest To ensure that the questionnaire meets the expectations in terms of the information that will be obtained

bull Pretesting Specific Questions Forbull Meaningbull Task difficultybull Respondent interest and attention

bull Pretesting the overall Questionnairebull Flow of the questionnairebull Skip patternsbull Length

bull Put yourself in the respondentrsquos shoes and answer the questionnaire

• Questionnaire Design
• A QUESTIONNAIRE IS ONLY AS GOOD AS THE QUESTIONS IT ASKS
• Slide 3
• Logical Steps to Develop a Good Questionnaire
• What Should Be Asked
• Validity and reliability
• Type of information
• Knowledge
• Opinions
• Types of Questions
• Open Ended Questions
• Closed-response Questions
• Closed-response Questions (2)
• Closed-response Questions (3)
• Closed-Ended Questions
• Issues in Questionnaire Design
• Contents of the Questionnaire
• Filter questions
• Slide 19
• Issues in Questionnaire Design (2)
• Slide 21
• Double-barrelled questions
• Leading and Loaded questions
• Slide 24
• Vague questions
• Avoid Making Assumptions
• Burdensome Questions
• Question Wording
• Sequence And Layout Decisions
• Layout
• Distinguishing question and responses
• Pretesting and Correcting Problems

Issues in Questionnaire Design

bull Respondent uncertainty

ndash Should respondents be provided with aDonrsquot know or No opinion option

ndash When it is important to differentiate between ambivalence and ignorance both options should be provided

Contents of the Questionnaire bull Question Wording ndash Vocabulary

- Depends on the nature of variable tapped- subjective feeling or objective facts

ndash Simple easy to understand commonly used language

ndash Avoid technical words and jargon (unless sample is technically qualified)

Filter questions Filter questions useful to ensure respondents

only answer relevant parts of questionsUnfilteredbull ndash If you use a medical software programbull which one do you use

Filteredbull Do you use a medical software programbull 1048715 No - jump to next questionbull 1048715 Yes - which one

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided------------------------------------------------------------------------------

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Issues in Questionnaire Design

bull Question Wordingbull vocabulary bull Questions need to be clear simple and

precisebull Poorly written questions lead to ambiguity

and misunderstandings and can be wasteful

bull Avoid Complexity use simple conversational language

bull Avoid leading and loaded questions

bull Avoid ambiguity be as specific as possible

bull Avoid double-barreled items

bull Avoid making assumptions

bull Avoid burdensome questions

Double-barrelled questionsTwo concepts in one question

bull Have you had a neck ache or a back achesince your last visit

bull Since your last visit have you had any of the following symptoms (tick as many that apply)

bull Neck ache 1048715 bull Back ache 1048715 bull Headache 1048715

Leading and Loaded questionsbull A leading question suggests or implies certain

answersDo accounting graduates who attended state universities such as Washington State University make better auditors

bull A loaded question suggests a socially desirable answer or is emotionally charged

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Vague questions

Avoid vague questionsbull Taken altogether how happy are you with

your stay in hospitalbull Overall how would you describe the care

you received in hospital

Avoid Ambiguity Words such as ldquooftenrdquo ldquooccasionallyrdquo ldquousuallyrdquo ldquoregularlyrdquo ldquofrequentlyrdquo ldquomanyrdquo should be used with caution If these words have to be used their meaning should be explained properly

Avoid Making Assumptionsbull Is the question applicable to all

respondents

Why do you like fast-food

Assumes respondents like fast foodsBetter strategy would be to ask a filter

question first

Burdensome QuestionsAvoid Burdensome Questions That MayTax the Respondentrsquos Memory telephone

A survey conducted during the 24-hour period following the airing of the Super Bowl might establish whether the respondent watched the Super Bowl and then ask ldquoDo you recall any commercials on that programrdquo

Question Wordingbull Question length

ndash Should be shortndash Longer questions confuse and fatigue respondents

bull Sensitive questionsndash Questions on information perceived to be

embarrassing like personal income criminal activities alcoholism smoking drugs habits social desirability issues etc

ndash Creativity rules (assurances of confidentiality anonymity slipping it in sideways open-ended questions asking in third person etc)

Sequence And Layout Decisions

bull Opening questions ndash easy and non-threateningbull Flow ndash smooth and logical ndash avoid jumpsbull Broad to specificbull Critical questions ndash placed in the middle bull Appealing and interestingbull Order bias ndash the possibility that subsequent responses

are influenced by preceding responses eg fewer people will say that their taxes are too high after being asked whether govt spending should be increased in certain areas

bull Demographic questions ndash first or last

Layout

bull 1048715 Just as important as wordingbull 1048715 Aim for a professional lookbull 1048715 Tipsbull ndash cover letterintroductory page giving study title organisation aims of the surveybull ndash enough space for open-ended questionsbull ndash font large enough to read without strainbull ndash consistent and clear instructionsbull ndash donrsquot split questions or answers across pagesbull ndash enough white space

Distinguishing question and responses

LAYOUT ADo you agree disagree or have no opinion that this company

has bull A good vacation policy - disagreenot sureagreebull Good management feedback - disagreenot sureagreebull Good medical insurance - disagreenot sureagree

LAYOUT B Does this company have bull Disagree Not Sure Agree bull A good vacation policy 1 2 3 bull Good feedback 1 2 3 bull Good medical insurance 1 2 3

Pretesting and Correcting Problems

Purpose of pretest To ensure that the questionnaire meets the expectations in terms of the information that will be obtained

bull Pretesting Specific Questions Forbull Meaningbull Task difficultybull Respondent interest and attention

bull Pretesting the overall Questionnairebull Flow of the questionnairebull Skip patternsbull Length

bull Put yourself in the respondentrsquos shoes and answer the questionnaire

• Questionnaire Design
• A QUESTIONNAIRE IS ONLY AS GOOD AS THE QUESTIONS IT ASKS
• Slide 3
• Logical Steps to Develop a Good Questionnaire
• What Should Be Asked
• Validity and reliability
• Type of information
• Knowledge
• Opinions
• Types of Questions
• Open Ended Questions
• Closed-response Questions
• Closed-response Questions (2)
• Closed-response Questions (3)
• Closed-Ended Questions
• Issues in Questionnaire Design
• Contents of the Questionnaire
• Filter questions
• Slide 19
• Issues in Questionnaire Design (2)
• Slide 21
• Double-barrelled questions
• Leading and Loaded questions
• Slide 24
• Vague questions
• Avoid Making Assumptions
• Burdensome Questions
• Question Wording
• Sequence And Layout Decisions
• Layout
• Distinguishing question and responses
• Pretesting and Correcting Problems

Contents of the Questionnaire bull Question Wording ndash Vocabulary

- Depends on the nature of variable tapped- subjective feeling or objective facts

ndash Simple easy to understand commonly used language

ndash Avoid technical words and jargon (unless sample is technically qualified)

Filter questions Filter questions useful to ensure respondents

only answer relevant parts of questionsUnfilteredbull ndash If you use a medical software programbull which one do you use

Filteredbull Do you use a medical software programbull 1048715 No - jump to next questionbull 1048715 Yes - which one

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided------------------------------------------------------------------------------

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Issues in Questionnaire Design

bull Question Wordingbull vocabulary bull Questions need to be clear simple and

precisebull Poorly written questions lead to ambiguity

and misunderstandings and can be wasteful

bull Avoid Complexity use simple conversational language

bull Avoid leading and loaded questions

bull Avoid ambiguity be as specific as possible

bull Avoid double-barreled items

bull Avoid making assumptions

bull Avoid burdensome questions

Double-barrelled questionsTwo concepts in one question

bull Have you had a neck ache or a back achesince your last visit

bull Since your last visit have you had any of the following symptoms (tick as many that apply)

bull Neck ache 1048715 bull Back ache 1048715 bull Headache 1048715

Leading and Loaded questionsbull A leading question suggests or implies certain

answersDo accounting graduates who attended state universities such as Washington State University make better auditors

bull A loaded question suggests a socially desirable answer or is emotionally charged

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Vague questions

Avoid vague questionsbull Taken altogether how happy are you with

your stay in hospitalbull Overall how would you describe the care

you received in hospital

Avoid Ambiguity Words such as ldquooftenrdquo ldquooccasionallyrdquo ldquousuallyrdquo ldquoregularlyrdquo ldquofrequentlyrdquo ldquomanyrdquo should be used with caution If these words have to be used their meaning should be explained properly

Avoid Making Assumptionsbull Is the question applicable to all

respondents

Why do you like fast-food

Assumes respondents like fast foodsBetter strategy would be to ask a filter

question first

Burdensome QuestionsAvoid Burdensome Questions That MayTax the Respondentrsquos Memory telephone

A survey conducted during the 24-hour period following the airing of the Super Bowl might establish whether the respondent watched the Super Bowl and then ask ldquoDo you recall any commercials on that programrdquo

Question Wordingbull Question length

ndash Should be shortndash Longer questions confuse and fatigue respondents

bull Sensitive questionsndash Questions on information perceived to be

embarrassing like personal income criminal activities alcoholism smoking drugs habits social desirability issues etc

ndash Creativity rules (assurances of confidentiality anonymity slipping it in sideways open-ended questions asking in third person etc)

Sequence And Layout Decisions

bull Opening questions ndash easy and non-threateningbull Flow ndash smooth and logical ndash avoid jumpsbull Broad to specificbull Critical questions ndash placed in the middle bull Appealing and interestingbull Order bias ndash the possibility that subsequent responses

are influenced by preceding responses eg fewer people will say that their taxes are too high after being asked whether govt spending should be increased in certain areas

bull Demographic questions ndash first or last

Layout

bull 1048715 Just as important as wordingbull 1048715 Aim for a professional lookbull 1048715 Tipsbull ndash cover letterintroductory page giving study title organisation aims of the surveybull ndash enough space for open-ended questionsbull ndash font large enough to read without strainbull ndash consistent and clear instructionsbull ndash donrsquot split questions or answers across pagesbull ndash enough white space

Distinguishing question and responses

LAYOUT ADo you agree disagree or have no opinion that this company

has bull A good vacation policy - disagreenot sureagreebull Good management feedback - disagreenot sureagreebull Good medical insurance - disagreenot sureagree

LAYOUT B Does this company have bull Disagree Not Sure Agree bull A good vacation policy 1 2 3 bull Good feedback 1 2 3 bull Good medical insurance 1 2 3

Pretesting and Correcting Problems

Purpose of pretest To ensure that the questionnaire meets the expectations in terms of the information that will be obtained

bull Pretesting Specific Questions Forbull Meaningbull Task difficultybull Respondent interest and attention

bull Pretesting the overall Questionnairebull Flow of the questionnairebull Skip patternsbull Length

bull Put yourself in the respondentrsquos shoes and answer the questionnaire

• Questionnaire Design
• A QUESTIONNAIRE IS ONLY AS GOOD AS THE QUESTIONS IT ASKS
• Slide 3
• Logical Steps to Develop a Good Questionnaire
• What Should Be Asked
• Validity and reliability
• Type of information
• Knowledge
• Opinions
• Types of Questions
• Open Ended Questions
• Closed-response Questions
• Closed-response Questions (2)
• Closed-response Questions (3)
• Closed-Ended Questions
• Issues in Questionnaire Design
• Contents of the Questionnaire
• Filter questions
• Slide 19
• Issues in Questionnaire Design (2)
• Slide 21
• Double-barrelled questions
• Leading and Loaded questions
• Slide 24
• Vague questions
• Avoid Making Assumptions
• Burdensome Questions
• Question Wording
• Sequence And Layout Decisions
• Layout
• Distinguishing question and responses
• Pretesting and Correcting Problems

Filter questions Filter questions useful to ensure respondents

only answer relevant parts of questionsUnfilteredbull ndash If you use a medical software programbull which one do you use

Filteredbull Do you use a medical software programbull 1048715 No - jump to next questionbull 1048715 Yes - which one

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided------------------------------------------------------------------------------

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Issues in Questionnaire Design

bull Question Wordingbull vocabulary bull Questions need to be clear simple and

precisebull Poorly written questions lead to ambiguity

and misunderstandings and can be wasteful

bull Avoid Complexity use simple conversational language

bull Avoid leading and loaded questions

bull Avoid ambiguity be as specific as possible

bull Avoid double-barreled items

bull Avoid making assumptions

bull Avoid burdensome questions

Double-barrelled questionsTwo concepts in one question

bull Have you had a neck ache or a back achesince your last visit

bull Since your last visit have you had any of the following symptoms (tick as many that apply)

bull Neck ache 1048715 bull Back ache 1048715 bull Headache 1048715

Leading and Loaded questionsbull A leading question suggests or implies certain

answersDo accounting graduates who attended state universities such as Washington State University make better auditors

bull A loaded question suggests a socially desirable answer or is emotionally charged

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Vague questions

Avoid vague questionsbull Taken altogether how happy are you with

your stay in hospitalbull Overall how would you describe the care

you received in hospital

Avoid Ambiguity Words such as ldquooftenrdquo ldquooccasionallyrdquo ldquousuallyrdquo ldquoregularlyrdquo ldquofrequentlyrdquo ldquomanyrdquo should be used with caution If these words have to be used their meaning should be explained properly

Avoid Making Assumptionsbull Is the question applicable to all

respondents

Why do you like fast-food

Assumes respondents like fast foodsBetter strategy would be to ask a filter

question first

Burdensome QuestionsAvoid Burdensome Questions That MayTax the Respondentrsquos Memory telephone

A survey conducted during the 24-hour period following the airing of the Super Bowl might establish whether the respondent watched the Super Bowl and then ask ldquoDo you recall any commercials on that programrdquo

Question Wordingbull Question length

ndash Should be shortndash Longer questions confuse and fatigue respondents

bull Sensitive questionsndash Questions on information perceived to be

embarrassing like personal income criminal activities alcoholism smoking drugs habits social desirability issues etc

ndash Creativity rules (assurances of confidentiality anonymity slipping it in sideways open-ended questions asking in third person etc)

Sequence And Layout Decisions

bull Opening questions ndash easy and non-threateningbull Flow ndash smooth and logical ndash avoid jumpsbull Broad to specificbull Critical questions ndash placed in the middle bull Appealing and interestingbull Order bias ndash the possibility that subsequent responses

are influenced by preceding responses eg fewer people will say that their taxes are too high after being asked whether govt spending should be increased in certain areas

bull Demographic questions ndash first or last

Layout

bull 1048715 Just as important as wordingbull 1048715 Aim for a professional lookbull 1048715 Tipsbull ndash cover letterintroductory page giving study title organisation aims of the surveybull ndash enough space for open-ended questionsbull ndash font large enough to read without strainbull ndash consistent and clear instructionsbull ndash donrsquot split questions or answers across pagesbull ndash enough white space

Distinguishing question and responses

LAYOUT ADo you agree disagree or have no opinion that this company

has bull A good vacation policy - disagreenot sureagreebull Good management feedback - disagreenot sureagreebull Good medical insurance - disagreenot sureagree

LAYOUT B Does this company have bull Disagree Not Sure Agree bull A good vacation policy 1 2 3 bull Good feedback 1 2 3 bull Good medical insurance 1 2 3

Pretesting and Correcting Problems

Purpose of pretest To ensure that the questionnaire meets the expectations in terms of the information that will be obtained

bull Pretesting Specific Questions Forbull Meaningbull Task difficultybull Respondent interest and attention

bull Pretesting the overall Questionnairebull Flow of the questionnairebull Skip patternsbull Length

bull Put yourself in the respondentrsquos shoes and answer the questionnaire

• Questionnaire Design
• A QUESTIONNAIRE IS ONLY AS GOOD AS THE QUESTIONS IT ASKS
• Slide 3
• Logical Steps to Develop a Good Questionnaire
• What Should Be Asked
• Validity and reliability
• Type of information
• Knowledge
• Opinions
• Types of Questions
• Open Ended Questions
• Closed-response Questions
• Closed-response Questions (2)
• Closed-response Questions (3)
• Closed-Ended Questions
• Issues in Questionnaire Design
• Contents of the Questionnaire
• Filter questions
• Slide 19
• Issues in Questionnaire Design (2)
• Slide 21
• Double-barrelled questions
• Leading and Loaded questions
• Slide 24
• Vague questions
• Avoid Making Assumptions
• Burdensome Questions
• Question Wording
• Sequence And Layout Decisions
• Layout
• Distinguishing question and responses
• Pretesting and Correcting Problems

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided------------------------------------------------------------------------------

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Issues in Questionnaire Design

bull Question Wordingbull vocabulary bull Questions need to be clear simple and

precisebull Poorly written questions lead to ambiguity

and misunderstandings and can be wasteful

bull Avoid Complexity use simple conversational language

bull Avoid leading and loaded questions

bull Avoid ambiguity be as specific as possible

bull Avoid double-barreled items

bull Avoid making assumptions

bull Avoid burdensome questions

Double-barrelled questionsTwo concepts in one question

bull Have you had a neck ache or a back achesince your last visit

bull Since your last visit have you had any of the following symptoms (tick as many that apply)

bull Neck ache 1048715 bull Back ache 1048715 bull Headache 1048715

Leading and Loaded questionsbull A leading question suggests or implies certain

answersDo accounting graduates who attended state universities such as Washington State University make better auditors

bull A loaded question suggests a socially desirable answer or is emotionally charged

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Vague questions

Avoid vague questionsbull Taken altogether how happy are you with

your stay in hospitalbull Overall how would you describe the care

you received in hospital

Avoid Ambiguity Words such as ldquooftenrdquo ldquooccasionallyrdquo ldquousuallyrdquo ldquoregularlyrdquo ldquofrequentlyrdquo ldquomanyrdquo should be used with caution If these words have to be used their meaning should be explained properly

Avoid Making Assumptionsbull Is the question applicable to all

respondents

Why do you like fast-food

Assumes respondents like fast foodsBetter strategy would be to ask a filter

question first

Burdensome QuestionsAvoid Burdensome Questions That MayTax the Respondentrsquos Memory telephone

A survey conducted during the 24-hour period following the airing of the Super Bowl might establish whether the respondent watched the Super Bowl and then ask ldquoDo you recall any commercials on that programrdquo

Question Wordingbull Question length

ndash Should be shortndash Longer questions confuse and fatigue respondents

bull Sensitive questionsndash Questions on information perceived to be

embarrassing like personal income criminal activities alcoholism smoking drugs habits social desirability issues etc

ndash Creativity rules (assurances of confidentiality anonymity slipping it in sideways open-ended questions asking in third person etc)

Sequence And Layout Decisions

bull Opening questions ndash easy and non-threateningbull Flow ndash smooth and logical ndash avoid jumpsbull Broad to specificbull Critical questions ndash placed in the middle bull Appealing and interestingbull Order bias ndash the possibility that subsequent responses

are influenced by preceding responses eg fewer people will say that their taxes are too high after being asked whether govt spending should be increased in certain areas

bull Demographic questions ndash first or last

Layout

bull 1048715 Just as important as wordingbull 1048715 Aim for a professional lookbull 1048715 Tipsbull ndash cover letterintroductory page giving study title organisation aims of the surveybull ndash enough space for open-ended questionsbull ndash font large enough to read without strainbull ndash consistent and clear instructionsbull ndash donrsquot split questions or answers across pagesbull ndash enough white space

Distinguishing question and responses

LAYOUT ADo you agree disagree or have no opinion that this company

has bull A good vacation policy - disagreenot sureagreebull Good management feedback - disagreenot sureagreebull Good medical insurance - disagreenot sureagree

LAYOUT B Does this company have bull Disagree Not Sure Agree bull A good vacation policy 1 2 3 bull Good feedback 1 2 3 bull Good medical insurance 1 2 3

Pretesting and Correcting Problems

Purpose of pretest To ensure that the questionnaire meets the expectations in terms of the information that will be obtained

bull Pretesting Specific Questions Forbull Meaningbull Task difficultybull Respondent interest and attention

bull Pretesting the overall Questionnairebull Flow of the questionnairebull Skip patternsbull Length

bull Put yourself in the respondentrsquos shoes and answer the questionnaire

• Questionnaire Design
• A QUESTIONNAIRE IS ONLY AS GOOD AS THE QUESTIONS IT ASKS
• Slide 3
• Logical Steps to Develop a Good Questionnaire
• What Should Be Asked
• Validity and reliability
• Type of information
• Knowledge
• Opinions
• Types of Questions
• Open Ended Questions
• Closed-response Questions
• Closed-response Questions (2)
• Closed-response Questions (3)
• Closed-Ended Questions
• Issues in Questionnaire Design
• Contents of the Questionnaire
• Filter questions
• Slide 19
• Issues in Questionnaire Design (2)
• Slide 21
• Double-barrelled questions
• Leading and Loaded questions
• Slide 24
• Vague questions
• Avoid Making Assumptions
• Burdensome Questions
• Question Wording
• Sequence And Layout Decisions
• Layout
• Distinguishing question and responses
• Pretesting and Correcting Problems

Issues in Questionnaire Design

bull Question Wordingbull vocabulary bull Questions need to be clear simple and

precisebull Poorly written questions lead to ambiguity

and misunderstandings and can be wasteful

bull Avoid Complexity use simple conversational language

bull Avoid leading and loaded questions

bull Avoid ambiguity be as specific as possible

bull Avoid double-barreled items

bull Avoid making assumptions

bull Avoid burdensome questions

Double-barrelled questionsTwo concepts in one question

bull Have you had a neck ache or a back achesince your last visit

bull Since your last visit have you had any of the following symptoms (tick as many that apply)

bull Neck ache 1048715 bull Back ache 1048715 bull Headache 1048715

Leading and Loaded questionsbull A leading question suggests or implies certain

answersDo accounting graduates who attended state universities such as Washington State University make better auditors

bull A loaded question suggests a socially desirable answer or is emotionally charged

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Vague questions

Avoid vague questionsbull Taken altogether how happy are you with

your stay in hospitalbull Overall how would you describe the care

you received in hospital

Avoid Ambiguity Words such as ldquooftenrdquo ldquooccasionallyrdquo ldquousuallyrdquo ldquoregularlyrdquo ldquofrequentlyrdquo ldquomanyrdquo should be used with caution If these words have to be used their meaning should be explained properly

Avoid Making Assumptionsbull Is the question applicable to all

respondents

Why do you like fast-food

Assumes respondents like fast foodsBetter strategy would be to ask a filter

question first

Burdensome QuestionsAvoid Burdensome Questions That MayTax the Respondentrsquos Memory telephone

A survey conducted during the 24-hour period following the airing of the Super Bowl might establish whether the respondent watched the Super Bowl and then ask ldquoDo you recall any commercials on that programrdquo

Question Wordingbull Question length

ndash Should be shortndash Longer questions confuse and fatigue respondents

bull Sensitive questionsndash Questions on information perceived to be

embarrassing like personal income criminal activities alcoholism smoking drugs habits social desirability issues etc

ndash Creativity rules (assurances of confidentiality anonymity slipping it in sideways open-ended questions asking in third person etc)

Sequence And Layout Decisions

bull Opening questions ndash easy and non-threateningbull Flow ndash smooth and logical ndash avoid jumpsbull Broad to specificbull Critical questions ndash placed in the middle bull Appealing and interestingbull Order bias ndash the possibility that subsequent responses

are influenced by preceding responses eg fewer people will say that their taxes are too high after being asked whether govt spending should be increased in certain areas

bull Demographic questions ndash first or last

Layout

bull 1048715 Just as important as wordingbull 1048715 Aim for a professional lookbull 1048715 Tipsbull ndash cover letterintroductory page giving study title organisation aims of the surveybull ndash enough space for open-ended questionsbull ndash font large enough to read without strainbull ndash consistent and clear instructionsbull ndash donrsquot split questions or answers across pagesbull ndash enough white space

Distinguishing question and responses

LAYOUT ADo you agree disagree or have no opinion that this company

has bull A good vacation policy - disagreenot sureagreebull Good management feedback - disagreenot sureagreebull Good medical insurance - disagreenot sureagree

LAYOUT B Does this company have bull Disagree Not Sure Agree bull A good vacation policy 1 2 3 bull Good feedback 1 2 3 bull Good medical insurance 1 2 3

Pretesting and Correcting Problems

Purpose of pretest To ensure that the questionnaire meets the expectations in terms of the information that will be obtained

bull Pretesting Specific Questions Forbull Meaningbull Task difficultybull Respondent interest and attention

bull Pretesting the overall Questionnairebull Flow of the questionnairebull Skip patternsbull Length

bull Put yourself in the respondentrsquos shoes and answer the questionnaire

• Questionnaire Design
• A QUESTIONNAIRE IS ONLY AS GOOD AS THE QUESTIONS IT ASKS
• Slide 3
• Logical Steps to Develop a Good Questionnaire
• What Should Be Asked
• Validity and reliability
• Type of information
• Knowledge
• Opinions
• Types of Questions
• Open Ended Questions
• Closed-response Questions
• Closed-response Questions (2)
• Closed-response Questions (3)
• Closed-Ended Questions
• Issues in Questionnaire Design
• Contents of the Questionnaire
• Filter questions
• Slide 19
• Issues in Questionnaire Design (2)
• Slide 21
• Double-barrelled questions
• Leading and Loaded questions
• Slide 24
• Vague questions
• Avoid Making Assumptions
• Burdensome Questions
• Question Wording
• Sequence And Layout Decisions
• Layout
• Distinguishing question and responses
• Pretesting and Correcting Problems

bull Avoid Complexity use simple conversational language

bull Avoid leading and loaded questions

bull Avoid ambiguity be as specific as possible

bull Avoid double-barreled items

bull Avoid making assumptions

bull Avoid burdensome questions

Double-barrelled questionsTwo concepts in one question

bull Have you had a neck ache or a back achesince your last visit

bull Since your last visit have you had any of the following symptoms (tick as many that apply)

bull Neck ache 1048715 bull Back ache 1048715 bull Headache 1048715

Leading and Loaded questionsbull A leading question suggests or implies certain

answersDo accounting graduates who attended state universities such as Washington State University make better auditors

bull A loaded question suggests a socially desirable answer or is emotionally charged

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Vague questions

Avoid vague questionsbull Taken altogether how happy are you with

your stay in hospitalbull Overall how would you describe the care

you received in hospital

Avoid Ambiguity Words such as ldquooftenrdquo ldquooccasionallyrdquo ldquousuallyrdquo ldquoregularlyrdquo ldquofrequentlyrdquo ldquomanyrdquo should be used with caution If these words have to be used their meaning should be explained properly

Avoid Making Assumptionsbull Is the question applicable to all

respondents

Why do you like fast-food

Assumes respondents like fast foodsBetter strategy would be to ask a filter

question first

Burdensome QuestionsAvoid Burdensome Questions That MayTax the Respondentrsquos Memory telephone

A survey conducted during the 24-hour period following the airing of the Super Bowl might establish whether the respondent watched the Super Bowl and then ask ldquoDo you recall any commercials on that programrdquo

Question Wordingbull Question length

ndash Should be shortndash Longer questions confuse and fatigue respondents

bull Sensitive questionsndash Questions on information perceived to be

embarrassing like personal income criminal activities alcoholism smoking drugs habits social desirability issues etc

ndash Creativity rules (assurances of confidentiality anonymity slipping it in sideways open-ended questions asking in third person etc)

Sequence And Layout Decisions

bull Opening questions ndash easy and non-threateningbull Flow ndash smooth and logical ndash avoid jumpsbull Broad to specificbull Critical questions ndash placed in the middle bull Appealing and interestingbull Order bias ndash the possibility that subsequent responses

are influenced by preceding responses eg fewer people will say that their taxes are too high after being asked whether govt spending should be increased in certain areas

bull Demographic questions ndash first or last

Layout

bull 1048715 Just as important as wordingbull 1048715 Aim for a professional lookbull 1048715 Tipsbull ndash cover letterintroductory page giving study title organisation aims of the surveybull ndash enough space for open-ended questionsbull ndash font large enough to read without strainbull ndash consistent and clear instructionsbull ndash donrsquot split questions or answers across pagesbull ndash enough white space

Distinguishing question and responses

LAYOUT ADo you agree disagree or have no opinion that this company

has bull A good vacation policy - disagreenot sureagreebull Good management feedback - disagreenot sureagreebull Good medical insurance - disagreenot sureagree

LAYOUT B Does this company have bull Disagree Not Sure Agree bull A good vacation policy 1 2 3 bull Good feedback 1 2 3 bull Good medical insurance 1 2 3

Pretesting and Correcting Problems

Purpose of pretest To ensure that the questionnaire meets the expectations in terms of the information that will be obtained

bull Pretesting Specific Questions Forbull Meaningbull Task difficultybull Respondent interest and attention

bull Pretesting the overall Questionnairebull Flow of the questionnairebull Skip patternsbull Length

bull Put yourself in the respondentrsquos shoes and answer the questionnaire

• Questionnaire Design
• A QUESTIONNAIRE IS ONLY AS GOOD AS THE QUESTIONS IT ASKS
• Slide 3
• Logical Steps to Develop a Good Questionnaire
• What Should Be Asked
• Validity and reliability
• Type of information
• Knowledge
• Opinions
• Types of Questions
• Open Ended Questions
• Closed-response Questions
• Closed-response Questions (2)
• Closed-response Questions (3)
• Closed-Ended Questions
• Issues in Questionnaire Design
• Contents of the Questionnaire
• Filter questions
• Slide 19
• Issues in Questionnaire Design (2)
• Slide 21
• Double-barrelled questions
• Leading and Loaded questions
• Slide 24
• Vague questions
• Avoid Making Assumptions
• Burdensome Questions
• Question Wording
• Sequence And Layout Decisions
• Layout
• Distinguishing question and responses
• Pretesting and Correcting Problems

Double-barrelled questionsTwo concepts in one question

bull Have you had a neck ache or a back achesince your last visit

bull Since your last visit have you had any of the following symptoms (tick as many that apply)

bull Neck ache 1048715 bull Back ache 1048715 bull Headache 1048715

Leading and Loaded questionsbull A leading question suggests or implies certain

answersDo accounting graduates who attended state universities such as Washington State University make better auditors

bull A loaded question suggests a socially desirable answer or is emotionally charged

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Vague questions

Avoid vague questionsbull Taken altogether how happy are you with

your stay in hospitalbull Overall how would you describe the care

you received in hospital

Avoid Ambiguity Words such as ldquooftenrdquo ldquooccasionallyrdquo ldquousuallyrdquo ldquoregularlyrdquo ldquofrequentlyrdquo ldquomanyrdquo should be used with caution If these words have to be used their meaning should be explained properly

Avoid Making Assumptionsbull Is the question applicable to all

respondents

Why do you like fast-food

Assumes respondents like fast foodsBetter strategy would be to ask a filter

question first

Burdensome QuestionsAvoid Burdensome Questions That MayTax the Respondentrsquos Memory telephone

A survey conducted during the 24-hour period following the airing of the Super Bowl might establish whether the respondent watched the Super Bowl and then ask ldquoDo you recall any commercials on that programrdquo

Question Wordingbull Question length

ndash Should be shortndash Longer questions confuse and fatigue respondents

bull Sensitive questionsndash Questions on information perceived to be

embarrassing like personal income criminal activities alcoholism smoking drugs habits social desirability issues etc

ndash Creativity rules (assurances of confidentiality anonymity slipping it in sideways open-ended questions asking in third person etc)

Sequence And Layout Decisions

bull Opening questions ndash easy and non-threateningbull Flow ndash smooth and logical ndash avoid jumpsbull Broad to specificbull Critical questions ndash placed in the middle bull Appealing and interestingbull Order bias ndash the possibility that subsequent responses

are influenced by preceding responses eg fewer people will say that their taxes are too high after being asked whether govt spending should be increased in certain areas

bull Demographic questions ndash first or last

Layout

bull 1048715 Just as important as wordingbull 1048715 Aim for a professional lookbull 1048715 Tipsbull ndash cover letterintroductory page giving study title organisation aims of the surveybull ndash enough space for open-ended questionsbull ndash font large enough to read without strainbull ndash consistent and clear instructionsbull ndash donrsquot split questions or answers across pagesbull ndash enough white space

Distinguishing question and responses

LAYOUT ADo you agree disagree or have no opinion that this company

has bull A good vacation policy - disagreenot sureagreebull Good management feedback - disagreenot sureagreebull Good medical insurance - disagreenot sureagree

LAYOUT B Does this company have bull Disagree Not Sure Agree bull A good vacation policy 1 2 3 bull Good feedback 1 2 3 bull Good medical insurance 1 2 3

Pretesting and Correcting Problems

Purpose of pretest To ensure that the questionnaire meets the expectations in terms of the information that will be obtained

bull Pretesting Specific Questions Forbull Meaningbull Task difficultybull Respondent interest and attention

bull Pretesting the overall Questionnairebull Flow of the questionnairebull Skip patternsbull Length

bull Put yourself in the respondentrsquos shoes and answer the questionnaire

• Questionnaire Design
• A QUESTIONNAIRE IS ONLY AS GOOD AS THE QUESTIONS IT ASKS
• Slide 3
• Logical Steps to Develop a Good Questionnaire
• What Should Be Asked
• Validity and reliability
• Type of information
• Knowledge
• Opinions
• Types of Questions
• Open Ended Questions
• Closed-response Questions
• Closed-response Questions (2)
• Closed-response Questions (3)
• Closed-Ended Questions
• Issues in Questionnaire Design
• Contents of the Questionnaire
• Filter questions
• Slide 19
• Issues in Questionnaire Design (2)
• Slide 21
• Double-barrelled questions
• Leading and Loaded questions
• Slide 24
• Vague questions
• Avoid Making Assumptions
• Burdensome Questions
• Question Wording
• Sequence And Layout Decisions
• Layout
• Distinguishing question and responses
• Pretesting and Correcting Problems

Leading and Loaded questionsbull A leading question suggests or implies certain

answersDo accounting graduates who attended state universities such as Washington State University make better auditors

bull A loaded question suggests a socially desirable answer or is emotionally charged

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Vague questions

Avoid vague questionsbull Taken altogether how happy are you with

your stay in hospitalbull Overall how would you describe the care

you received in hospital

Avoid Ambiguity Words such as ldquooftenrdquo ldquooccasionallyrdquo ldquousuallyrdquo ldquoregularlyrdquo ldquofrequentlyrdquo ldquomanyrdquo should be used with caution If these words have to be used their meaning should be explained properly

Avoid Making Assumptionsbull Is the question applicable to all

respondents

Why do you like fast-food

Assumes respondents like fast foodsBetter strategy would be to ask a filter

question first

Burdensome QuestionsAvoid Burdensome Questions That MayTax the Respondentrsquos Memory telephone

A survey conducted during the 24-hour period following the airing of the Super Bowl might establish whether the respondent watched the Super Bowl and then ask ldquoDo you recall any commercials on that programrdquo

Question Wordingbull Question length

ndash Should be shortndash Longer questions confuse and fatigue respondents

bull Sensitive questionsndash Questions on information perceived to be

embarrassing like personal income criminal activities alcoholism smoking drugs habits social desirability issues etc

ndash Creativity rules (assurances of confidentiality anonymity slipping it in sideways open-ended questions asking in third person etc)

Sequence And Layout Decisions

bull Opening questions ndash easy and non-threateningbull Flow ndash smooth and logical ndash avoid jumpsbull Broad to specificbull Critical questions ndash placed in the middle bull Appealing and interestingbull Order bias ndash the possibility that subsequent responses

are influenced by preceding responses eg fewer people will say that their taxes are too high after being asked whether govt spending should be increased in certain areas

bull Demographic questions ndash first or last

Layout

bull 1048715 Just as important as wordingbull 1048715 Aim for a professional lookbull 1048715 Tipsbull ndash cover letterintroductory page giving study title organisation aims of the surveybull ndash enough space for open-ended questionsbull ndash font large enough to read without strainbull ndash consistent and clear instructionsbull ndash donrsquot split questions or answers across pagesbull ndash enough white space

Distinguishing question and responses

LAYOUT ADo you agree disagree or have no opinion that this company

has bull A good vacation policy - disagreenot sureagreebull Good management feedback - disagreenot sureagreebull Good medical insurance - disagreenot sureagree

LAYOUT B Does this company have bull Disagree Not Sure Agree bull A good vacation policy 1 2 3 bull Good feedback 1 2 3 bull Good medical insurance 1 2 3

Pretesting and Correcting Problems

Purpose of pretest To ensure that the questionnaire meets the expectations in terms of the information that will be obtained

bull Pretesting Specific Questions Forbull Meaningbull Task difficultybull Respondent interest and attention

bull Pretesting the overall Questionnairebull Flow of the questionnairebull Skip patternsbull Length

bull Put yourself in the respondentrsquos shoes and answer the questionnaire

• Questionnaire Design
• A QUESTIONNAIRE IS ONLY AS GOOD AS THE QUESTIONS IT ASKS
• Slide 3
• Logical Steps to Develop a Good Questionnaire
• What Should Be Asked
• Validity and reliability
• Type of information
• Knowledge
• Opinions
• Types of Questions
• Open Ended Questions
• Closed-response Questions
• Closed-response Questions (2)
• Closed-response Questions (3)
• Closed-Ended Questions
• Issues in Questionnaire Design
• Contents of the Questionnaire
• Filter questions
• Slide 19
• Issues in Questionnaire Design (2)
• Slide 21
• Double-barrelled questions
• Leading and Loaded questions
• Slide 24
• Vague questions
• Avoid Making Assumptions
• Burdensome Questions
• Question Wording
• Sequence And Layout Decisions
• Layout
• Distinguishing question and responses
• Pretesting and Correcting Problems

Do you believe that private citizens have the right toown firearms to defend themselves their families and property from violent criminal attack

Yes No Undecided

Do you believe that a ban on the private ownershipof firearms would be significantly reduce the number ofmurders and robberies in your community

Yes No Undecided

Vague questions

Avoid vague questionsbull Taken altogether how happy are you with

your stay in hospitalbull Overall how would you describe the care

you received in hospital

Avoid Ambiguity Words such as ldquooftenrdquo ldquooccasionallyrdquo ldquousuallyrdquo ldquoregularlyrdquo ldquofrequentlyrdquo ldquomanyrdquo should be used with caution If these words have to be used their meaning should be explained properly

Avoid Making Assumptionsbull Is the question applicable to all

respondents

Why do you like fast-food

Assumes respondents like fast foodsBetter strategy would be to ask a filter

question first

Burdensome QuestionsAvoid Burdensome Questions That MayTax the Respondentrsquos Memory telephone

A survey conducted during the 24-hour period following the airing of the Super Bowl might establish whether the respondent watched the Super Bowl and then ask ldquoDo you recall any commercials on that programrdquo

Question Wordingbull Question length

ndash Should be shortndash Longer questions confuse and fatigue respondents

bull Sensitive questionsndash Questions on information perceived to be

embarrassing like personal income criminal activities alcoholism smoking drugs habits social desirability issues etc

ndash Creativity rules (assurances of confidentiality anonymity slipping it in sideways open-ended questions asking in third person etc)

Sequence And Layout Decisions

bull Opening questions ndash easy and non-threateningbull Flow ndash smooth and logical ndash avoid jumpsbull Broad to specificbull Critical questions ndash placed in the middle bull Appealing and interestingbull Order bias ndash the possibility that subsequent responses

are influenced by preceding responses eg fewer people will say that their taxes are too high after being asked whether govt spending should be increased in certain areas

bull Demographic questions ndash first or last

Layout

bull 1048715 Just as important as wordingbull 1048715 Aim for a professional lookbull 1048715 Tipsbull ndash cover letterintroductory page giving study title organisation aims of the surveybull ndash enough space for open-ended questionsbull ndash font large enough to read without strainbull ndash consistent and clear instructionsbull ndash donrsquot split questions or answers across pagesbull ndash enough white space

Distinguishing question and responses

LAYOUT ADo you agree disagree or have no opinion that this company

has bull A good vacation policy - disagreenot sureagreebull Good management feedback - disagreenot sureagreebull Good medical insurance - disagreenot sureagree

LAYOUT B Does this company have bull Disagree Not Sure Agree bull A good vacation policy 1 2 3 bull Good feedback 1 2 3 bull Good medical insurance 1 2 3

Pretesting and Correcting Problems

Purpose of pretest To ensure that the questionnaire meets the expectations in terms of the information that will be obtained

bull Pretesting Specific Questions Forbull Meaningbull Task difficultybull Respondent interest and attention

bull Pretesting the overall Questionnairebull Flow of the questionnairebull Skip patternsbull Length

bull Put yourself in the respondentrsquos shoes and answer the questionnaire

• Questionnaire Design
• A QUESTIONNAIRE IS ONLY AS GOOD AS THE QUESTIONS IT ASKS
• Slide 3
• Logical Steps to Develop a Good Questionnaire
• What Should Be Asked
• Validity and reliability
• Type of information
• Knowledge
• Opinions
• Types of Questions
• Open Ended Questions
• Closed-response Questions
• Closed-response Questions (2)
• Closed-response Questions (3)
• Closed-Ended Questions
• Issues in Questionnaire Design
• Contents of the Questionnaire
• Filter questions
• Slide 19
• Issues in Questionnaire Design (2)
• Slide 21
• Double-barrelled questions
• Leading and Loaded questions
• Slide 24
• Vague questions
• Avoid Making Assumptions
• Burdensome Questions
• Question Wording
• Sequence And Layout Decisions
• Layout
• Distinguishing question and responses
• Pretesting and Correcting Problems

Vague questions

Avoid vague questionsbull Taken altogether how happy are you with

your stay in hospitalbull Overall how would you describe the care

you received in hospital

Avoid Ambiguity Words such as ldquooftenrdquo ldquooccasionallyrdquo ldquousuallyrdquo ldquoregularlyrdquo ldquofrequentlyrdquo ldquomanyrdquo should be used with caution If these words have to be used their meaning should be explained properly

Avoid Making Assumptionsbull Is the question applicable to all

respondents

Why do you like fast-food

Assumes respondents like fast foodsBetter strategy would be to ask a filter

question first

Burdensome QuestionsAvoid Burdensome Questions That MayTax the Respondentrsquos Memory telephone

A survey conducted during the 24-hour period following the airing of the Super Bowl might establish whether the respondent watched the Super Bowl and then ask ldquoDo you recall any commercials on that programrdquo

Question Wordingbull Question length

ndash Should be shortndash Longer questions confuse and fatigue respondents

bull Sensitive questionsndash Questions on information perceived to be

embarrassing like personal income criminal activities alcoholism smoking drugs habits social desirability issues etc

ndash Creativity rules (assurances of confidentiality anonymity slipping it in sideways open-ended questions asking in third person etc)

Sequence And Layout Decisions

bull Opening questions ndash easy and non-threateningbull Flow ndash smooth and logical ndash avoid jumpsbull Broad to specificbull Critical questions ndash placed in the middle bull Appealing and interestingbull Order bias ndash the possibility that subsequent responses

are influenced by preceding responses eg fewer people will say that their taxes are too high after being asked whether govt spending should be increased in certain areas

bull Demographic questions ndash first or last

Layout

bull 1048715 Just as important as wordingbull 1048715 Aim for a professional lookbull 1048715 Tipsbull ndash cover letterintroductory page giving study title organisation aims of the surveybull ndash enough space for open-ended questionsbull ndash font large enough to read without strainbull ndash consistent and clear instructionsbull ndash donrsquot split questions or answers across pagesbull ndash enough white space

Distinguishing question and responses

LAYOUT ADo you agree disagree or have no opinion that this company

has bull A good vacation policy - disagreenot sureagreebull Good management feedback - disagreenot sureagreebull Good medical insurance - disagreenot sureagree

LAYOUT B Does this company have bull Disagree Not Sure Agree bull A good vacation policy 1 2 3 bull Good feedback 1 2 3 bull Good medical insurance 1 2 3

Pretesting and Correcting Problems

Purpose of pretest To ensure that the questionnaire meets the expectations in terms of the information that will be obtained

bull Pretesting Specific Questions Forbull Meaningbull Task difficultybull Respondent interest and attention

bull Pretesting the overall Questionnairebull Flow of the questionnairebull Skip patternsbull Length

bull Put yourself in the respondentrsquos shoes and answer the questionnaire

• Questionnaire Design
• A QUESTIONNAIRE IS ONLY AS GOOD AS THE QUESTIONS IT ASKS
• Slide 3
• Logical Steps to Develop a Good Questionnaire
• What Should Be Asked
• Validity and reliability
• Type of information
• Knowledge
• Opinions
• Types of Questions
• Open Ended Questions
• Closed-response Questions
• Closed-response Questions (2)
• Closed-response Questions (3)
• Closed-Ended Questions
• Issues in Questionnaire Design
• Contents of the Questionnaire
• Filter questions
• Slide 19
• Issues in Questionnaire Design (2)
• Slide 21
• Double-barrelled questions
• Leading and Loaded questions
• Slide 24
• Vague questions
• Avoid Making Assumptions
• Burdensome Questions
• Question Wording
• Sequence And Layout Decisions
• Layout
• Distinguishing question and responses
• Pretesting and Correcting Problems

Avoid Making Assumptionsbull Is the question applicable to all

respondents

Why do you like fast-food

Assumes respondents like fast foodsBetter strategy would be to ask a filter

question first

Burdensome QuestionsAvoid Burdensome Questions That MayTax the Respondentrsquos Memory telephone

A survey conducted during the 24-hour period following the airing of the Super Bowl might establish whether the respondent watched the Super Bowl and then ask ldquoDo you recall any commercials on that programrdquo

Question Wordingbull Question length

ndash Should be shortndash Longer questions confuse and fatigue respondents

bull Sensitive questionsndash Questions on information perceived to be

embarrassing like personal income criminal activities alcoholism smoking drugs habits social desirability issues etc

ndash Creativity rules (assurances of confidentiality anonymity slipping it in sideways open-ended questions asking in third person etc)

Sequence And Layout Decisions

bull Opening questions ndash easy and non-threateningbull Flow ndash smooth and logical ndash avoid jumpsbull Broad to specificbull Critical questions ndash placed in the middle bull Appealing and interestingbull Order bias ndash the possibility that subsequent responses

are influenced by preceding responses eg fewer people will say that their taxes are too high after being asked whether govt spending should be increased in certain areas

bull Demographic questions ndash first or last

Layout

bull 1048715 Just as important as wordingbull 1048715 Aim for a professional lookbull 1048715 Tipsbull ndash cover letterintroductory page giving study title organisation aims of the surveybull ndash enough space for open-ended questionsbull ndash font large enough to read without strainbull ndash consistent and clear instructionsbull ndash donrsquot split questions or answers across pagesbull ndash enough white space

Distinguishing question and responses

LAYOUT ADo you agree disagree or have no opinion that this company

has bull A good vacation policy - disagreenot sureagreebull Good management feedback - disagreenot sureagreebull Good medical insurance - disagreenot sureagree

LAYOUT B Does this company have bull Disagree Not Sure Agree bull A good vacation policy 1 2 3 bull Good feedback 1 2 3 bull Good medical insurance 1 2 3

Pretesting and Correcting Problems

Purpose of pretest To ensure that the questionnaire meets the expectations in terms of the information that will be obtained

bull Pretesting Specific Questions Forbull Meaningbull Task difficultybull Respondent interest and attention

bull Pretesting the overall Questionnairebull Flow of the questionnairebull Skip patternsbull Length

bull Put yourself in the respondentrsquos shoes and answer the questionnaire

• Questionnaire Design
• A QUESTIONNAIRE IS ONLY AS GOOD AS THE QUESTIONS IT ASKS
• Slide 3
• Logical Steps to Develop a Good Questionnaire
• What Should Be Asked
• Validity and reliability
• Type of information
• Knowledge
• Opinions
• Types of Questions
• Open Ended Questions
• Closed-response Questions
• Closed-response Questions (2)
• Closed-response Questions (3)
• Closed-Ended Questions
• Issues in Questionnaire Design
• Contents of the Questionnaire
• Filter questions
• Slide 19
• Issues in Questionnaire Design (2)
• Slide 21
• Double-barrelled questions
• Leading and Loaded questions
• Slide 24
• Vague questions
• Avoid Making Assumptions
• Burdensome Questions
• Question Wording
• Sequence And Layout Decisions
• Layout
• Distinguishing question and responses
• Pretesting and Correcting Problems

Burdensome QuestionsAvoid Burdensome Questions That MayTax the Respondentrsquos Memory telephone

A survey conducted during the 24-hour period following the airing of the Super Bowl might establish whether the respondent watched the Super Bowl and then ask ldquoDo you recall any commercials on that programrdquo

Question Wordingbull Question length

ndash Should be shortndash Longer questions confuse and fatigue respondents

bull Sensitive questionsndash Questions on information perceived to be

embarrassing like personal income criminal activities alcoholism smoking drugs habits social desirability issues etc

ndash Creativity rules (assurances of confidentiality anonymity slipping it in sideways open-ended questions asking in third person etc)

Sequence And Layout Decisions

bull Opening questions ndash easy and non-threateningbull Flow ndash smooth and logical ndash avoid jumpsbull Broad to specificbull Critical questions ndash placed in the middle bull Appealing and interestingbull Order bias ndash the possibility that subsequent responses

are influenced by preceding responses eg fewer people will say that their taxes are too high after being asked whether govt spending should be increased in certain areas

bull Demographic questions ndash first or last

Layout

bull 1048715 Just as important as wordingbull 1048715 Aim for a professional lookbull 1048715 Tipsbull ndash cover letterintroductory page giving study title organisation aims of the surveybull ndash enough space for open-ended questionsbull ndash font large enough to read without strainbull ndash consistent and clear instructionsbull ndash donrsquot split questions or answers across pagesbull ndash enough white space

Distinguishing question and responses

LAYOUT ADo you agree disagree or have no opinion that this company

has bull A good vacation policy - disagreenot sureagreebull Good management feedback - disagreenot sureagreebull Good medical insurance - disagreenot sureagree

LAYOUT B Does this company have bull Disagree Not Sure Agree bull A good vacation policy 1 2 3 bull Good feedback 1 2 3 bull Good medical insurance 1 2 3

Pretesting and Correcting Problems

Purpose of pretest To ensure that the questionnaire meets the expectations in terms of the information that will be obtained

bull Pretesting Specific Questions Forbull Meaningbull Task difficultybull Respondent interest and attention

bull Pretesting the overall Questionnairebull Flow of the questionnairebull Skip patternsbull Length

bull Put yourself in the respondentrsquos shoes and answer the questionnaire

• Questionnaire Design
• A QUESTIONNAIRE IS ONLY AS GOOD AS THE QUESTIONS IT ASKS
• Slide 3
• Logical Steps to Develop a Good Questionnaire
• What Should Be Asked
• Validity and reliability
• Type of information
• Knowledge
• Opinions
• Types of Questions
• Open Ended Questions
• Closed-response Questions
• Closed-response Questions (2)
• Closed-response Questions (3)
• Closed-Ended Questions
• Issues in Questionnaire Design
• Contents of the Questionnaire
• Filter questions
• Slide 19
• Issues in Questionnaire Design (2)
• Slide 21
• Double-barrelled questions
• Leading and Loaded questions
• Slide 24
• Vague questions
• Avoid Making Assumptions
• Burdensome Questions
• Question Wording
• Sequence And Layout Decisions
• Layout
• Distinguishing question and responses
• Pretesting and Correcting Problems

Question Wordingbull Question length

ndash Should be shortndash Longer questions confuse and fatigue respondents

bull Sensitive questionsndash Questions on information perceived to be

embarrassing like personal income criminal activities alcoholism smoking drugs habits social desirability issues etc

ndash Creativity rules (assurances of confidentiality anonymity slipping it in sideways open-ended questions asking in third person etc)

Sequence And Layout Decisions

bull Opening questions ndash easy and non-threateningbull Flow ndash smooth and logical ndash avoid jumpsbull Broad to specificbull Critical questions ndash placed in the middle bull Appealing and interestingbull Order bias ndash the possibility that subsequent responses

are influenced by preceding responses eg fewer people will say that their taxes are too high after being asked whether govt spending should be increased in certain areas

bull Demographic questions ndash first or last

Layout

bull 1048715 Just as important as wordingbull 1048715 Aim for a professional lookbull 1048715 Tipsbull ndash cover letterintroductory page giving study title organisation aims of the surveybull ndash enough space for open-ended questionsbull ndash font large enough to read without strainbull ndash consistent and clear instructionsbull ndash donrsquot split questions or answers across pagesbull ndash enough white space

Distinguishing question and responses

LAYOUT ADo you agree disagree or have no opinion that this company

has bull A good vacation policy - disagreenot sureagreebull Good management feedback - disagreenot sureagreebull Good medical insurance - disagreenot sureagree

LAYOUT B Does this company have bull Disagree Not Sure Agree bull A good vacation policy 1 2 3 bull Good feedback 1 2 3 bull Good medical insurance 1 2 3

Pretesting and Correcting Problems

Purpose of pretest To ensure that the questionnaire meets the expectations in terms of the information that will be obtained

bull Pretesting Specific Questions Forbull Meaningbull Task difficultybull Respondent interest and attention

bull Pretesting the overall Questionnairebull Flow of the questionnairebull Skip patternsbull Length

bull Put yourself in the respondentrsquos shoes and answer the questionnaire

• Questionnaire Design
• A QUESTIONNAIRE IS ONLY AS GOOD AS THE QUESTIONS IT ASKS
• Slide 3
• Logical Steps to Develop a Good Questionnaire
• What Should Be Asked
• Validity and reliability
• Type of information
• Knowledge
• Opinions
• Types of Questions
• Open Ended Questions
• Closed-response Questions
• Closed-response Questions (2)
• Closed-response Questions (3)
• Closed-Ended Questions
• Issues in Questionnaire Design
• Contents of the Questionnaire
• Filter questions
• Slide 19
• Issues in Questionnaire Design (2)
• Slide 21
• Double-barrelled questions
• Leading and Loaded questions
• Slide 24
• Vague questions
• Avoid Making Assumptions
• Burdensome Questions
• Question Wording
• Sequence And Layout Decisions
• Layout
• Distinguishing question and responses
• Pretesting and Correcting Problems

Sequence And Layout Decisions

bull Opening questions ndash easy and non-threateningbull Flow ndash smooth and logical ndash avoid jumpsbull Broad to specificbull Critical questions ndash placed in the middle bull Appealing and interestingbull Order bias ndash the possibility that subsequent responses

are influenced by preceding responses eg fewer people will say that their taxes are too high after being asked whether govt spending should be increased in certain areas

bull Demographic questions ndash first or last

Layout

bull 1048715 Just as important as wordingbull 1048715 Aim for a professional lookbull 1048715 Tipsbull ndash cover letterintroductory page giving study title organisation aims of the surveybull ndash enough space for open-ended questionsbull ndash font large enough to read without strainbull ndash consistent and clear instructionsbull ndash donrsquot split questions or answers across pagesbull ndash enough white space

Distinguishing question and responses

LAYOUT ADo you agree disagree or have no opinion that this company

has bull A good vacation policy - disagreenot sureagreebull Good management feedback - disagreenot sureagreebull Good medical insurance - disagreenot sureagree

LAYOUT B Does this company have bull Disagree Not Sure Agree bull A good vacation policy 1 2 3 bull Good feedback 1 2 3 bull Good medical insurance 1 2 3

Pretesting and Correcting Problems

Purpose of pretest To ensure that the questionnaire meets the expectations in terms of the information that will be obtained

bull Pretesting Specific Questions Forbull Meaningbull Task difficultybull Respondent interest and attention

bull Pretesting the overall Questionnairebull Flow of the questionnairebull Skip patternsbull Length

bull Put yourself in the respondentrsquos shoes and answer the questionnaire

• Questionnaire Design
• A QUESTIONNAIRE IS ONLY AS GOOD AS THE QUESTIONS IT ASKS
• Slide 3
• Logical Steps to Develop a Good Questionnaire
• What Should Be Asked
• Validity and reliability
• Type of information
• Knowledge
• Opinions
• Types of Questions
• Open Ended Questions
• Closed-response Questions
• Closed-response Questions (2)
• Closed-response Questions (3)
• Closed-Ended Questions
• Issues in Questionnaire Design
• Contents of the Questionnaire
• Filter questions
• Slide 19
• Issues in Questionnaire Design (2)
• Slide 21
• Double-barrelled questions
• Leading and Loaded questions
• Slide 24
• Vague questions
• Avoid Making Assumptions
• Burdensome Questions
• Question Wording
• Sequence And Layout Decisions
• Layout
• Distinguishing question and responses
• Pretesting and Correcting Problems

Layout

bull 1048715 Just as important as wordingbull 1048715 Aim for a professional lookbull 1048715 Tipsbull ndash cover letterintroductory page giving study title organisation aims of the surveybull ndash enough space for open-ended questionsbull ndash font large enough to read without strainbull ndash consistent and clear instructionsbull ndash donrsquot split questions or answers across pagesbull ndash enough white space

Distinguishing question and responses

LAYOUT ADo you agree disagree or have no opinion that this company

has bull A good vacation policy - disagreenot sureagreebull Good management feedback - disagreenot sureagreebull Good medical insurance - disagreenot sureagree

LAYOUT B Does this company have bull Disagree Not Sure Agree bull A good vacation policy 1 2 3 bull Good feedback 1 2 3 bull Good medical insurance 1 2 3

Pretesting and Correcting Problems

Purpose of pretest To ensure that the questionnaire meets the expectations in terms of the information that will be obtained

bull Pretesting Specific Questions Forbull Meaningbull Task difficultybull Respondent interest and attention

bull Pretesting the overall Questionnairebull Flow of the questionnairebull Skip patternsbull Length

bull Put yourself in the respondentrsquos shoes and answer the questionnaire

• Questionnaire Design
• A QUESTIONNAIRE IS ONLY AS GOOD AS THE QUESTIONS IT ASKS
• Slide 3
• Logical Steps to Develop a Good Questionnaire
• What Should Be Asked
• Validity and reliability
• Type of information
• Knowledge
• Opinions
• Types of Questions
• Open Ended Questions
• Closed-response Questions
• Closed-response Questions (2)
• Closed-response Questions (3)
• Closed-Ended Questions
• Issues in Questionnaire Design
• Contents of the Questionnaire
• Filter questions
• Slide 19
• Issues in Questionnaire Design (2)
• Slide 21
• Double-barrelled questions
• Leading and Loaded questions
• Slide 24
• Vague questions
• Avoid Making Assumptions
• Burdensome Questions
• Question Wording
• Sequence And Layout Decisions
• Layout
• Distinguishing question and responses
• Pretesting and Correcting Problems

Distinguishing question and responses

LAYOUT ADo you agree disagree or have no opinion that this company

has bull A good vacation policy - disagreenot sureagreebull Good management feedback - disagreenot sureagreebull Good medical insurance - disagreenot sureagree

LAYOUT B Does this company have bull Disagree Not Sure Agree bull A good vacation policy 1 2 3 bull Good feedback 1 2 3 bull Good medical insurance 1 2 3

Pretesting and Correcting Problems

Purpose of pretest To ensure that the questionnaire meets the expectations in terms of the information that will be obtained

bull Pretesting Specific Questions Forbull Meaningbull Task difficultybull Respondent interest and attention

bull Pretesting the overall Questionnairebull Flow of the questionnairebull Skip patternsbull Length

bull Put yourself in the respondentrsquos shoes and answer the questionnaire

• Questionnaire Design
• A QUESTIONNAIRE IS ONLY AS GOOD AS THE QUESTIONS IT ASKS
• Slide 3
• Logical Steps to Develop a Good Questionnaire
• What Should Be Asked
• Validity and reliability
• Type of information
• Knowledge
• Opinions
• Types of Questions
• Open Ended Questions
• Closed-response Questions
• Closed-response Questions (2)
• Closed-response Questions (3)
• Closed-Ended Questions
• Issues in Questionnaire Design
• Contents of the Questionnaire
• Filter questions
• Slide 19
• Issues in Questionnaire Design (2)
• Slide 21
• Double-barrelled questions
• Leading and Loaded questions
• Slide 24
• Vague questions
• Avoid Making Assumptions
• Burdensome Questions
• Question Wording
• Sequence And Layout Decisions
• Layout
• Distinguishing question and responses
• Pretesting and Correcting Problems

Pretesting and Correcting Problems

Purpose of pretest To ensure that the questionnaire meets the expectations in terms of the information that will be obtained

bull Pretesting Specific Questions Forbull Meaningbull Task difficultybull Respondent interest and attention

bull Pretesting the overall Questionnairebull Flow of the questionnairebull Skip patternsbull Length

bull Put yourself in the respondentrsquos shoes and answer the questionnaire

• Questionnaire Design
• A QUESTIONNAIRE IS ONLY AS GOOD AS THE QUESTIONS IT ASKS
• Slide 3
• Logical Steps to Develop a Good Questionnaire
• What Should Be Asked
• Validity and reliability
• Type of information
• Knowledge
• Opinions
• Types of Questions
• Open Ended Questions
• Closed-response Questions
• Closed-response Questions (2)
• Closed-response Questions (3)
• Closed-Ended Questions
• Issues in Questionnaire Design
• Contents of the Questionnaire
• Filter questions
• Slide 19
• Issues in Questionnaire Design (2)
• Slide 21
• Double-barrelled questions
• Leading and Loaded questions
• Slide 24
• Vague questions
• Avoid Making Assumptions
• Burdensome Questions
• Question Wording
• Sequence And Layout Decisions
• Layout
• Distinguishing question and responses
• Pretesting and Correcting Problems