Questionnaire Design

Dr. Tejinder SharmaDr. Tejinder SharmaDepartment of CommerceDepartment of CommerceKurukshetra University,Kurukshetra University,

Kurukshetra 136119Kurukshetra 136119

[email protected]@gmail.com

Measurement & Measurement & Questionnaire DesignQuestionnaire Design• AgendaAgenda

– MeasurementMeasurement– Scale developmentScale development– Questionnaire designQuestionnaire design

Defining MeasurementDefining Measurement

An integrative process of determining An integrative process of determining the intensity (or amount) of information the intensity (or amount) of information about constructs, concepts or objectsabout constructs, concepts or objects

Construct : Conceptual Construct : Conceptual FrameworkFramework

• ConstructConstruct– Something that is being measuredSomething that is being measured

• Construct developmentConstruct development– An integrative process wherein the An integrative process wherein the

researchers identify the subjective researchers identify the subjective properties for which the data should be properties for which the data should be collected for solving the defined collected for solving the defined research problem.research problem.

Construct MeasurementConstruct Measurement• Concrete (measurable) propertiesConcrete (measurable) properties

• Abstract (subjective) propertiesAbstract (subjective) properties

• Number of dimensions (concrete and abstract)Number of dimensions (concrete and abstract)

• Operationalization (measurability issues of Operationalization (measurability issues of constructs)constructs)

• Reliability & validity issuesReliability & validity issues

Types of DataTypes of Data• State-of-being (verifiable facts)State-of-being (verifiable facts)

– physical and/or demographic itemsphysical and/or demographic items

• State-of-mind (mental thoughts or emotional State-of-mind (mental thoughts or emotional feelings)feelings)– Attitudes, perceptions, beliefs, etc.Attitudes, perceptions, beliefs, etc.

• State-of-behavior (past or current behaviors)State-of-behavior (past or current behaviors)– In a typical week, how often do you..In a typical week, how often do you..

• State-of-intention (planned future behaviors)State-of-intention (planned future behaviors)– Likelihood to engage in behavior in futureLikelihood to engage in behavior in future

ScalingScaling

• Assigning number to various degrees Assigning number to various degrees of opinion, attitude and other of opinion, attitude and other conceptsconcepts

• Determining quantitative measures Determining quantitative measures of subjective/ abstract conceptsof subjective/ abstract concepts

• Assigning numbers to the properties Assigning numbers to the properties of objectsof objects

Basic Levels of ScalesBasic Levels of Scales

• NominalNominal

• OrdinalOrdinal

• IntervalInterval

• RatioRatio

Nominal ScaleNominal Scale

• One level measurementOne level measurement

• Numbers are used as labels to classify the Numbers are used as labels to classify the productsproducts

• Used for demographic variables, types of Used for demographic variables, types of products, stores, etc.products, stores, etc.

• Eg. PIN numbers 136118 for KurukshetraEg. PIN numbers 136118 for Kurukshetra

• Analysis – counting, frequency, percentage, Analysis – counting, frequency, percentage, mode, binomial test, chi square test, etc.mode, binomial test, chi square test, etc.

Ordinal ScaleOrdinal Scale

• Provides information about ordered Provides information about ordered relationship among the objectsrelationship among the objects

• Contains all information of nominal scale Contains all information of nominal scale and measure of whether an object is more and measure of whether an object is more or less characteristics of the other objector less characteristics of the other object

• For example – Ranking of preferencesFor example – Ranking of preferences• Statistical Analysis – Mathematical Statistical Analysis – Mathematical

operations, percentage, median, rank operations, percentage, median, rank correlation, non-parametric testscorrelation, non-parametric tests

Example of Ordinal ScaleExample of Ordinal ScaleRespondentRespondent AdidasAdidas NikeNike ReebokReebok

AA 22 11 33

BB 11 22 33

CC 22 33 11

DD 11 33 22

EE 11 22 33

FF 11 33 22

GG 22 33 11

HH 22 11 33

II 11 22 33

JJ 22 11 33

Interval ScaleInterval Scale

• Intervals are adjusted in terms of Intervals are adjusted in terms of some rule that has been established some rule that has been established as a basis for making unites equalas a basis for making unites equal

• Units are equal only on assumptionUnits are equal only on assumption

• Has an absolute zero, or unique originHas an absolute zero, or unique origin

• Statistics – mean, s.d., correlation, t-Statistics – mean, s.d., correlation, t-test, F-testtest, F-test

Ratio ScaleRatio Scale

• Expressed as ration of few dimensionsExpressed as ration of few dimensions

• Measured from base zeroMeasured from base zero

• Measures actual amount of variablesMeasures actual amount of variables

• Amenable to all statistical techniques, Amenable to all statistical techniques, including geometric mean, harmonic including geometric mean, harmonic mean, COV, etc.mean, COV, etc.

• Examples – price of a product (Rs./unit)Examples – price of a product (Rs./unit)

Attitude MeasurementAttitude Measurement

• Trilogy ApproachTrilogy Approach– Cognitive component (beliefs, Cognitive component (beliefs,

perceptions or knowledge)perceptions or knowledge)– Affective component (emotions or Affective component (emotions or

feelings)feelings)– Conative component (intended or actual Conative component (intended or actual

behaviors)behaviors)

Scale Development Scale Development

• Process of developing reliable instrument, Process of developing reliable instrument, which measures the desired variables which measures the desired variables correctly and accuratelycorrectly and accurately

• Systematic and scientific process of Systematic and scientific process of identifying the item variables, statements identifying the item variables, statements and the scales for measuring themand the scales for measuring them

• Ascertaining whether the instrument Ascertaining whether the instrument correctly measures the desired variables correctly measures the desired variables sufficiently and accuratelysufficiently and accurately

Classification of ScalesClassification of Scales

• Subject OrientationSubject Orientation– Designed to measure characteristics of the respondentsDesigned to measure characteristics of the respondents– Stimuli – response to develop categories (smokers/non-Stimuli – response to develop categories (smokers/non-

smokers, etc)smokers, etc)• Response FormResponse Form

– Categorical and comparative scalesCategorical and comparative scales• Degree of SubjectivityDegree of Subjectivity

– Personal preferencesPersonal preferences• Scale propertiesScale properties

– Nominal, interval, ordinal, ratioNominal, interval, ordinal, ratio• Number of DimensionsNumber of Dimensions

– Uni-dimensional, multi-dimensional Uni-dimensional, multi-dimensional

Scale Construction Scale Construction TechniquesTechniques• Arbitrary ApproachArbitrary Approach

– Scales developed on adhoc basisScales developed on adhoc basis– Developed on presumptionDeveloped on presumption

• Consensus Approach (Thurstone Differential Scale)Consensus Approach (Thurstone Differential Scale)– Panel of judges evaluate the items chosen for inclusion in the Panel of judges evaluate the items chosen for inclusion in the

instrumentinstrument• Item Analysis Approach (Likert Scale)Item Analysis Approach (Likert Scale)

– Individual items are tested by gropu of respondentsIndividual items are tested by gropu of respondents– Analysed on the basis of degree of discriminationAnalysed on the basis of degree of discrimination

• Cumulative Scale (Guttman’s Scalogram)Cumulative Scale (Guttman’s Scalogram)– Conformity to some ranking of items in ascending or Conformity to some ranking of items in ascending or

descending orderdescending order• Factor Scales (Osgood’s Semantic Differential Scale)Factor Scales (Osgood’s Semantic Differential Scale)

– On the bass of intercorrelations to identify the commin factorsOn the bass of intercorrelations to identify the commin factors

Important Scaling Important Scaling TechniquesTechniques

• Rating ScaleRating Scale

• Ranking ScaleRanking Scale

Rating ScaleRating Scale

• Qualitative description of a limited Qualitative description of a limited number of aspectsnumber of aspects

• Judge in terms of specific criteriaJudge in terms of specific criteria– Like --- DislikeLike --- Dislike– Above average, average, below averageAbove average, average, below average

• 3 to 7 point scales are used3 to 7 point scales are used

• More the rating, more the sensitivityMore the rating, more the sensitivity

Rating Scale TypesRating Scale Types

• Graphical RatingGraphical Rating– Points are put in a continuumPoints are put in a continuum– Indicate rating by tick markIndicate rating by tick mark

Like Very Much

Like Somewhat

Neutral Dislike Some What

Dislike Very Much

Rating Scale TypesRating Scale Types

• Itemised RatingItemised Rating– Presents a series of statementsPresents a series of statements– Respondent selects the testRespondent selects the test

• He is always involved in some friction with his fellow He is always involved in some friction with his fellow workerworker

• He is often at odds with one or more of his fellow He is often at odds with one or more of his fellow workersworkers

• He sometimes gets involved in frictionHe sometimes gets involved in friction

• He infrequently becomes involved in friction with othersHe infrequently becomes involved in friction with others

• He almost ever gets involved in friction with his fellow He almost ever gets involved in friction with his fellow workersworkers

Ranking ScaleRanking Scale

• Make comparative/relative Make comparative/relative judgmentsjudgments

• ApproachesApproaches– Method of paired comparisonMethod of paired comparison– Method of rank orderMethod of rank order

Method of Paired Method of Paired ComparisonComparison• Respondent expresses the attitude by Respondent expresses the attitude by

making choice between two objectsmaking choice between two objects• Number of comparisons (N) to be made Number of comparisons (N) to be made

depend upon number of objects (n)depend upon number of objects (n)N = n/2 (n-1) If n= 10, N = 45N = n/2 (n-1) If n= 10, N = 45

• Reduce possible comparisons by sample Reduce possible comparisons by sample surveysurvey

• Paired Comparison can be converted to Paired Comparison can be converted to interval data by the Thurstone’s Law of interval data by the Thurstone’s Law of Comparative Judgment & Guilford’s Comparative Judgment & Guilford’s composite standard methodcomposite standard method

Scale ConstructionScale Construction

• Thurstone ScaleThurstone Scale

• Likert ScaleLikert Scale

Differential Scale (Thurstone Differential Scale (Thurstone Scale)Scale)

• Uses consensus approachUses consensus approach

• Method used in measuring attitude Method used in measuring attitude on single dimensionon single dimension

• Used to measure the issues like war, Used to measure the issues like war, religion, etc.religion, etc.

Differential Scale (Thurstone Differential Scale (Thurstone Scale)Scale)• Researcher gathers a large number of statements Researcher gathers a large number of statements

to express a point of viewto express a point of view• Submitted to a panel of judges to arrange them in Submitted to a panel of judges to arrange them in

11 groups ranging from one extreme to another11 groups ranging from one extreme to another• Sorting by each judge yields composite position Sorting by each judge yields composite position

of each itemof each item• Items of disagreement are discardedItems of disagreement are discarded• Median position of items selected items is Median position of items selected items is

decideddecided• Attitude comparison made on the basis of these Attitude comparison made on the basis of these

median scoresmedian scores

Likert Scale (Summated Likert Scale (Summated Scale)Scale)• Evaluates each item on its ability to Evaluates each item on its ability to

discriminate between those with high score discriminate between those with high score and those with low scoreand those with low score

• Respondent indicates degree of agreement or Respondent indicates degree of agreement or disagreement with the statements in the disagreement with the statements in the instrumentinstrument

• Each response is given a numerical score, Each response is given a numerical score, indicating favourableness or indicating favourableness or unfavourableness and total score represents unfavourableness and total score represents the attitudethe attitude

Likert Scale (Summated Scale) Likert Scale (Summated Scale) ProcedureProcedure

• Collect large number of statements Collect large number of statements relevant to the attituderelevant to the attitude

• Collect diverse statements which Collect diverse statements which express favourableness or express favourableness or unfavourablenessunfavourableness

• Administer it to a group of respondentsAdminister it to a group of respondents

• Do the coding 1 for lowest and 5/7 for Do the coding 1 for lowest and 5/7 for the highestthe highest

Likert Scale (Summated Scale) Likert Scale (Summated Scale) ProcedureProcedure

• Compute total score of each respondentCompute total score of each respondent• Arrange the total scires to find out Arrange the total scires to find out

discriminating power of each statementdiscriminating power of each statement• Identify top 25% and bottom 25% Identify top 25% and bottom 25%

statements, which express the attitudesstatements, which express the attitudes• Statements correlating with total score Statements correlating with total score

are retained in the final instrument and are retained in the final instrument and rest are discardedrest are discarded

Likert Scale (Summated Likert Scale (Summated Scale)Scale)

• AdvantagesAdvantages– Easier than Thurstone ScaleEasier than Thurstone Scale– Without panel of judgesWithout panel of judges– More reliable as it considers each item More reliable as it considers each item

statement and respondentstatement and respondent

• LimitationsLimitations– Just gives the difference in attitudes and Just gives the difference in attitudes and

does not quantify the samedoes not quantify the same

Criteria for good Criteria for good measurementmeasurement

• ReliabilityReliability

• ValidityValidity

• SensitivitySensitivity

• RelevanceRelevance

• VersatilityVersatility

• Ease of responseEase of response

ReliabilityReliability

• Ability to obtain similar results by Ability to obtain similar results by measuring an object, trait or measuring an object, trait or construct with independent but construct with independent but comparable measurescomparable measures

• Example: Do both CAT and MAT Example: Do both CAT and MAT scores measure the candidates scores measure the candidates performance?performance?

Assessing ReliabilityAssessing Reliability

• Stability: Measure the same objects or Stability: Measure the same objects or individuals at two different points in individuals at two different points in time and then correlate their scores. time and then correlate their scores. Also known as test-retest reliabilityAlso known as test-retest reliability

• Example: Correlation of your score on Example: Correlation of your score on the ACT in your Junior year and your the ACT in your Junior year and your score on the ACT in your Senior yearscore on the ACT in your Senior year

Assessing ReliabilityAssessing Reliability

• Equivalence: Determined by Equivalence: Determined by calculating the internal consistency calculating the internal consistency or homogeneity of the set of items or homogeneity of the set of items forming the scale. One way to forming the scale. One way to calculate equivalence reliability is to calculate equivalence reliability is to use coefficient alpha.use coefficient alpha.

ExampleExampleFor each of the following items, circle the number that best representshow you feel about making a sales call/presentation.

Definitely Definitely do not Feel Feel

Jittery 1 2 3 4 5 6 7Active 1 2 3 4 5 6 7Intense 1 2 3 4 5 6 7Energetic 1 2 3 4 5 6 7Fearful 1 2 3 4 5 6 7Vigorous 1 2 3 4 5 6 7Lively 1 2 3 4 5 6 7Tense 1 2 3 4 5 6 7

Steps for reliability in SPSSSteps for reliability in SPSS

• AnalyseAnalyse– ScaleScale

ReliabilityReliability– AlphaAlpha– StatisticsStatistics

Descriptives for ScaleDescriptives for ScaleDescriptives for Scale if item deletedDescriptives for Scale if item deletedInter-item CorrelationInter-item CorrelationOKOK

Reliability StatisticsReliability StatisticsCronbach AlphaCronbach Alpha Cronbach Alpha of Cronbach Alpha of

Standardised ItemsStandardised ItemsN of ItemsN of Items

0.7060.706 0.7080.708 66

VariablVariablee

Scale Mean Scale Mean if Item if Item

DeletedDeleted

Scale Scale Variance Variance if Item if Item

DeletedDeleted

Corrected Corrected Item Total Item Total CorrelatioCorrelatio

nn

Squared Squared Multiple Multiple

CorrelatioCorrelationn

Cronbach Cronbach Alpha if Alpha if

Item Item DeletedDeleted

V2V2 25.8925.89 22.65522.655 0.4310.431 0.2310.231 0.6690.669

V3V3 25.7625.76 22.64122.641 0.5230.523 0.3800.380 0.6410.641

V4V4 25.6925.69 22.56122.561 0.4800.480 0.3150.315 0.6530.653

V5V5 25.9025.90 24.26824.268 0.4250.425 0.1850.185 0.6710.671

V6V6 26.1426.14 23.71523.715 0.3640.364 0.1710.171 0.6900.690

V7V7 26.2826.28 23.27323.273 0.4110.411 0.1820.182 0.6750.675

ValidityValidity

• Degree to which our measures reflect Degree to which our measures reflect true differences among individuals -- true differences among individuals -- and that we’re measuring what we and that we’re measuring what we think we’re measuringthink we’re measuring

Assessing ValidityAssessing Validity

• Pragmatic ValidityPragmatic Validity

• How well the measure actually predicts How well the measure actually predicts some other characteristic or behaviorsome other characteristic or behavior

• Predictive Validity: A measure is used Predictive Validity: A measure is used to predict something in the futureto predict something in the future

• Concurrent Validity: A measure is used Concurrent Validity: A measure is used to predict something assessed at the to predict something assessed at the same point in timesame point in time

Assessing ValidityAssessing Validity

• Content Validity: The adequacy with Content Validity: The adequacy with which the domain of the characteristic is which the domain of the characteristic is captured by the measure. Also called face captured by the measure. Also called face validity.validity.

• Construct Validity: Assessment of how Construct Validity: Assessment of how well the instrument captures the well the instrument captures the construct, concept, or trait it is supposed construct, concept, or trait it is supposed to be measuringto be measuring

• Item to Total CorrelationItem to Total Correlation

Relationship between Relationship between Reliability and ValidityReliability and Validity

• Reliability is a necessary but not Reliability is a necessary but not sufficient condition for validitysufficient condition for validity

• A measure may be reliable and not A measure may be reliable and not validvalid

SensitivitySensitivity

• It is the ability of a measurement to It is the ability of a measurement to indicate changes or differencesindicate changes or differences

• Eg : Three ad campaigns showed similar Eg : Three ad campaigns showed similar sales. Possible reasons could be:sales. Possible reasons could be:– The ads were similarThe ads were similar– The period of sales was brief and insensitive to The period of sales was brief and insensitive to

changeschanges– Sales might not be the right test to measure Sales might not be the right test to measure

the effectiveness of the adsthe effectiveness of the ads

RelevanceRelevance

• To the decision madeTo the decision made

• Construct must be identical to the Construct must be identical to the description of itemsdescription of items

VersatilityVersatility

• Robustness of measurement for Robustness of measurement for various statistical interpretations, various statistical interpretations, especially validityespecially validity

Ease of responseEase of response

• How easily a person will supply the How easily a person will supply the datadata

Comparison of three modes of data collection

Parameter Interview Telephone Mail/Self

Literacy Not require Not require Require

Respondentskills

Language &skills needed

Language &skills needed

Not needed

ResponseRate

Highest Medium Lowest

Privacy Difficult Someanonymity forgiving replies

Good; noembarrassment

Consent Easy toconvince &get consent

Convincing ispossible

Convincing isdifficult soconsent

Questionnaire Questionnaire

. . . . a prepared set

of questions (or

measures) to which

respondents or

interviewers record

answers

Steps in Questionnaire Design:

Step 1: Determine Specific Data to be sought

Step 2: Determine Interview Process

Step 3: Evaluate Question Content

Step 4: Determine Response Format

Step 5: Determine Wording

Step 6: Determine Questionnaire Structure

Step 7: Determine Physical Characteristics

Step 8: Pretest – Revise – Finalize Draft

Questionnaire Design – Questionnaire Design – Identify Information NeedsIdentify Information Needs• Clarify the nature of the research problem andClarify the nature of the research problem and

objectives.objectives.

• Develop research questions to meet researchDevelop research questions to meet research

objectives.objectives.

• Identify Variables from LiteratureIdentify Variables from Literature

• Develop Statements to measure each itemDevelop Statements to measure each item

• Select the right scaleSelect the right scale

Questionnaire Design – Questionnaire Design – Clarification of Concepts:Clarification of Concepts:

• Ensure the concepts(s) can be clearly defined.Ensure the concepts(s) can be clearly defined.

• Select the variables/indicators to represent the Select the variables/indicators to represent the

concepts.concepts.

• Determine the level of measurement.Determine the level of measurement.

Information NeedsInformation Needs

• Prepare following documentsPrepare following documents– Research PurposeResearch Purpose– Information to be measuredInformation to be measured– Draft analysis planDraft analysis plan

Research Questions:Research Questions:

• What are the most important factors What are the most important factors influencing the purchase of a laptop influencing the purchase of a laptop computer?computer?

• Do employees in this organization support Do employees in this organization support diversity in the workplace?diversity in the workplace?

• What is the customer’s consideration while What is the customer’s consideration while purchasing a mutual fund?purchasing a mutual fund?

Determine Interview Determine Interview ProcessProcess

• Interview administered surveyInterview administered survey

• Self administered personal surveySelf administered personal survey

• Informal interviewingInformal interviewing

• Telephone interviewTelephone interview

• Mail surveyMail survey

Self-Completion or Self-Completion or Interviewer Assisted Interviewer Assisted Questionnaire?Questionnaire?

Respondent capabilitiesRespondent capabilities::

• Educational background.Educational background.

• Vocabulary level.Vocabulary level.

• Prior experience in completing Prior experience in completing questionnaires.questionnaires.

• Age.Age.

• Cultural issues.Cultural issues.

Questionnaire Design – Questionnaire Design – Typology of a Typology of a Questionnaire:Questionnaire:• Determine the types of questions to include Determine the types of questions to include

and their order.and their order.

• Check the wording and coding of questions.Check the wording and coding of questions.

• Decide on the grouping of the questions and Decide on the grouping of the questions and

the overall length of the questionnaire.the overall length of the questionnaire.

• Determine the structure and layout of the Determine the structure and layout of the

questionnaire.questionnaire.

QUESTIONNAIRE DESIGNQUESTIONNAIRE DESIGN

1.1. Two Types of QuestionsTwo Types of Questions::

2.2. Open-ended.Open-ended.

3.3. Closed-ended.Closed-ended.

• Open-ended Questions = place no constraints on respondents who are free to answer in their own words.

Closed-ended Questions = respondent is given the option of choosing from a number of predetermined answers.

Open-Ended QuestionsOpen-Ended Questions

• Free responseFree response

• ProbingProbing

• Projective techniqueProjective technique

• Associative techniqueAssociative technique

• Construction techniqueConstruction technique

Open-ended QuestionsOpen-ended Questions

• Typically used in exploratory/qualitative Typically used in exploratory/qualitative studies.studies.

• Typically used in personal interview surveys Typically used in personal interview surveys involving small samples.involving small samples.

• Allows respondent freedom of response.Allows respondent freedom of response.• Respondent must be articulate Respondent must be articulate andand willing to willing to

spend spend time giving a full answer.time giving a full answer.• Data is in narrative form which can be time Data is in narrative form which can be time

consuming and difficult to code and analyze.consuming and difficult to code and analyze.• Possible researcher bias in interpretation.Possible researcher bias in interpretation.• Narrative can be analyzed using content Narrative can be analyzed using content

analysis. Software is available (e.g., NUD*IST).analysis. Software is available (e.g., NUD*IST).

Open-Ended Questions: Open-Ended Questions: examplesexamples1.1. What do you think about your health What do you think about your health

insurance plan?insurance plan?

2.2. Which mutual funds have you been Which mutual funds have you been investing in for the past year?investing in for the past year?

3.3. How are the funds you are investing in How are the funds you are investing in performing?performing?

4.4. What do you think of airport securityWhat do you think of airport security??

Closed-end Questions:Closed-end Questions:

• Single Answer.Single Answer.

• Multiple Answer.Multiple Answer.

• Rank Order.Rank Order.

• Numeric.Numeric.

• Likert-Type.Likert-Type.

• Semantic DifferentialSemantic Differential

Closed-end QuestionsClosed-end Questions

• Typically used in quantitative studies.Typically used in quantitative studies.

• Assumption is researcher has knowledge to pre-Assumption is researcher has knowledge to pre-specify response categories.specify response categories.

• Data can be pre-coded and therefore in a form Data can be pre-coded and therefore in a form amenable for use with statistical packages (e.g., amenable for use with statistical packages (e.g., SPSS, SAS) – data capture therefore easier.SPSS, SAS) – data capture therefore easier.

• More difficult to design but simplifies analysis.More difficult to design but simplifies analysis.

• Used in studies involving large samples.Used in studies involving large samples.

• Limited range of response options.Limited range of response options.

Broad ConsiderationsBroad Considerations

• Sequencing of questions.Sequencing of questions.• Identification of concepts.Identification of concepts.• How many questions are required to How many questions are required to

capture each concept?capture each concept?• Question wording.Question wording.• Overall length of questionnaire.Overall length of questionnaire.• Placing of sensitive questions.Placing of sensitive questions.• Ability of respondents.Ability of respondents.• Level of measurement.Level of measurement.• Open-ended versus closed-end questionsOpen-ended versus closed-end questions..

Questionnaire SectionsQuestionnaire Sections

• Opening QuestionsOpening Questions

• Research Topic QuestionsResearch Topic Questions

• Classification QuestionsClassification Questions

Screening or Filter Screening or Filter Questions:Questions:• . . . are used to ensure respondents . . . are used to ensure respondents

included in the study are those that included in the study are those that meet the pre-determined criteria of the meet the pre-determined criteria of the target population.target population.

• ““Tonight we are talking with individuals Tonight we are talking with individuals who are 18 years of age or older and who are 18 years of age or older and have 50 percent or more of the have 50 percent or more of the responsibility for banking decisions in responsibility for banking decisions in your household. Are you that person?” your household. Are you that person?” __ Yes __ No __ Yes __ No

Rapport Questions:Rapport Questions:

• . . . are used to establish rapport with . . . are used to establish rapport with the respondent by gaining their attention the respondent by gaining their attention and stimulating their interest in the and stimulating their interest in the topic.topic.

• “ “Have you seen any good movies in the Have you seen any good movies in the last month?” last month?” __ Yes __ No __ Yes __ No

• “ “What is your favorite seafood What is your favorite seafood restaurant?”restaurant?”

Concept = a generic idea Concept = a generic idea formed in the mind.formed in the mind.

• Example ConceptExample Concept: “Customer : “Customer Interaction” Interaction”

• This customer was easy to talk to.This customer was easy to talk to.• This customer genuinely This customer genuinely

appreciated my helping him/her.appreciated my helping him/her.• This customer was friendly.This customer was friendly.• This customer seemed interested in This customer seemed interested in

me, not only as a salesperson, but me, not only as a salesperson, but also as a person.also as a person.

ConceptsConcepts

Concept IdentificationConcept Identification..

• Conceptual definition – e.g., Service Conceptual definition – e.g., Service Quality. As perceived by customers, Quality. As perceived by customers, it is the extent of discrepancy it is the extent of discrepancy between customers’ expectations or between customers’ expectations or desires and their perceptions.desires and their perceptions.

Working Definition for ConceptWorking Definition for Concept..• Decompose definition into Decompose definition into

components.components.• Search for items that are measurable.Search for items that are measurable.

Preparing and Presenting Preparing and Presenting Good Questions:Good Questions:• Use simple words.Use simple words.• Be brief.Be brief.• Avoid ambiguity.Avoid ambiguity.• Avoid leading questions.Avoid leading questions.• Avoid double-barreled questions.Avoid double-barreled questions.• Be careful about question order and Be careful about question order and

context effects.context effects.• Check questionnaire layout.Check questionnaire layout.• Prepare clear instructions.Prepare clear instructions.

Avoid Position Bias:Avoid Position Bias:

• Position BiasPosition Bias: : • ““How important are flexible hours in evaluating jobHow important are flexible hours in evaluating job• alternatives?”alternatives?”• “ “What factors are important in evaluating jobWhat factors are important in evaluating job• alternatives?”alternatives?”• No Position BiasNo Position Bias::• “ “What factors are important in evaluating jobWhat factors are important in evaluating job• alternatives?”alternatives?”• “ “How important are flexible hours in evaluating job How important are flexible hours in evaluating job • alternatives?”alternatives?”

Double-Barreled Questions:Double-Barreled Questions:

To what extent do you agree or To what extent do you agree or disagree with the following disagree with the following statements? statements?

“ “Airtel’s employees are friendly Airtel’s employees are friendly and and helpful.”helpful.” ““Airtel’s employees are Airtel’s employees are courteous and courteous and knowledgeable.”knowledgeable.”

Branching Questions:Branching Questions:• . . . are used to direct respondents to answer the right . . . are used to direct respondents to answer the right • questions as well as questions in the proper sequencequestions as well as questions in the proper sequence..

– “ “Have you seen or heard any advertisements for wirelessHave you seen or heard any advertisements for wireless– telephone service in the past 30 days?”telephone service in the past 30 days?”– “ “If ‘No’, go to question #10.If ‘No’, go to question #10.– “ “If ‘Yes’ , were the advertisements on radio or TV or If ‘Yes’ , were the advertisements on radio or TV or

both?”both?”– “ “If the advertisements were on TV or on both radio andIf the advertisements were on TV or on both radio and– TV, then go to question #6?TV, then go to question #6?– “ “If the advertisements were on radio, then go toIf the advertisements were on radio, then go to– question #8.”question #8.”

• Following questions #6 and #8 the next question would be:Following questions #6 and #8 the next question would be:

– “ “Were any of the advertisements for ‘Mahindra Xylo’?”Were any of the advertisements for ‘Mahindra Xylo’?”

Self-Completion Self-Completion InstructionsInstructions

The following issues typically are consideredThe following issues typically are considered: :

• Introducing and explaining how to answer Introducing and explaining how to answer a series of a series of

• questions on a particular topic.questions on a particular topic.• Transition statements from one section Transition statements from one section

(topic) of the(topic) of the• questionnaire to another.questionnaire to another.• Which question to go to next (branching Which question to go to next (branching

or skipping).or skipping).

Self-Completion Self-Completion InstructionsInstructions

• How many answers are acceptable, e.g., How many answers are acceptable, e.g., “Check only one response.” Or “Check as “Check only one response.” Or “Check as many as apply.” many as apply.”

• Whether respondents are supposed to Whether respondents are supposed to answer the question by themselves, or can answer the question by themselves, or can consult another person or reference consult another person or reference materials. materials.

• What to do when the questionnaire is What to do when the questionnaire is completed, e.g., “When finished, place this in completed, e.g., “When finished, place this in the postage paid envelope and mail it.”the postage paid envelope and mail it.”

Interviewer-Assisted Interviewer-Assisted Instructions:Instructions:

The following issues typically are consideredThe following issues typically are considered::• How to increase respondent participation?How to increase respondent participation?• How to screen out respondents that are notHow to screen out respondents that are not• wanted and still keep them happy?wanted and still keep them happy?• What to say when respondents ask how to What to say when respondents ask how to

answeranswer• a particular question?a particular question?• When concepts may not be easily When concepts may not be easily

understood,understood,• how to define them?how to define them?

Interviewer-Assisted Interviewer-Assisted Instructions:Instructions:

• When answer alternatives are to When answer alternatives are to be read tobe read to

respondents (aided response) or respondents (aided response) or not to be read (unaided not to be read (unaided response)?response)?

• How to follow branching or skip How to follow branching or skip patterns?patterns?

• When and how to probe?When and how to probe?• How to end the interview?How to end the interview?

Questionnaire Design – Pre-Questionnaire Design – Pre-testing of a Questionnaire:testing of a Questionnaire:• Determine the nature of the pretest for the Determine the nature of the pretest for the

preliminary questionnaire. preliminary questionnaire.

• Analyze initial data to identify limitations Analyze initial data to identify limitations

ofof

the preliminary questionnaire.the preliminary questionnaire.

• Refine the questionnaire as needed.Refine the questionnaire as needed.

• Revisit some or all of the above steps, ifRevisit some or all of the above steps, if

necessary.necessary.

Questionnaire Design – Questionnaire Design – Administering a Administering a Questionnaire:Questionnaire:• Identify the best practice for administeringIdentify the best practice for administering

the type of questionnaire utilized.the type of questionnaire utilized.

• Train and audit field workers, if required.Train and audit field workers, if required.

• Ensure a process is in place to handleEnsure a process is in place to handle

completed questionnaires.completed questionnaires.

• Determine the deadline and follow-upDetermine the deadline and follow-up

methods.methods.

Tips for question writing Tips for question writing

• Format for questions Format for questions – Look at the following layouts and decide which Look at the following layouts and decide which

you would prefer to use: you would prefer to use: – Do you agree, disagree or have no opinion Do you agree, disagree or have no opinion

that this company has:that this company has:– A good vacation policy - agree/not A good vacation policy - agree/not

sure/disagree.sure/disagree.Good management feedback - agree/not Good management feedback - agree/not sure/disagree.sure/disagree.Good medical insurance - agree/not Good medical insurance - agree/not sure/disagree.sure/disagree.High wages - agree/not sure/disagree.High wages - agree/not sure/disagree.

Tips for question writingTips for question writing

• Use simple and clear language Use simple and clear language

– Poor: Poor: How often do you punish your toddler? How often do you punish your toddler? – Better: Better: How often do you put your toddler into How often do you put your toddler into

timeout? Check only one. timeout? Check only one. – Once a day Once a day 1 1 – Several times a daySeveral times a day 22– Once a week Once a week 3 3 – Several times a weekSeveral times a week 4 4 – Once a weekOnce a week 5 5 – Several times a week 6Several times a week 6


• 1 Do not use biased words 1 Do not use biased words e.g. you should not say you are in favour of capital e.g. you should not say you are in favour of capital

punishment punishment • 2 Do not use slang 2 Do not use slang

– Example: Example: How many kids do you have? How many kids do you have? Example: Example: Should parents know the whereabouts of their Should parents know the whereabouts of their teens 24/7? teens 24/7?

• 3 Do not use double barreled questions 3 Do not use double barreled questions (one thought per question) (one thought per question) – Example: Example: Curtailing development and protecting Curtailing development and protecting

the environment should be a top priority for “Our” the environment should be a top priority for “Our” town.town.

• 4 Do not use vague words or phrases4 Do not use vague words or phrases


• 5 Do not use abbreviations 5 Do not use abbreviations • Example: Example: Should KU allow admission to MBA Should KU allow admission to MBA

without CAT ?without CAT ?• Example: Example: Which political party is responsible for Which political party is responsible for

expanding the size of the GDP? expanding the size of the GDP? • 6 Do not use jargon or technical terms Example6 Do not use jargon or technical terms Example: :

India should formulate a stricter fertility policy.India should formulate a stricter fertility policy.• 7 Do not use double negatives 7 Do not use double negatives • Example: Example: Should the Fiji not oppose the UN court? Should the Fiji not oppose the UN court? • 8. Use caution when asking personal questions 8. Use caution when asking personal questions


PoorPoor: : How much do you earn each year? $______________How much do you earn each year? $______________Better:Better: In which category does your annual income last year best fit? In which category does your annual income last year best fit? Below Rs10,000Below Rs10,000

___ Rs10,001-Rs20,000___ Rs10,001-Rs20,000___ Rs20,001-Rs30,000___ Rs20,001-Rs30,000___ Rs30,001-Rs40,000___ Rs30,001-Rs40,000___ Rs40,001-Rs50,000___ Rs40,001-Rs50,000___ Rs50,001-Rs60,000___ Rs50,001-Rs60,000___ Rs60,001-Rs70,000___ Rs60,001-Rs70,000___ Over Rs70,001 ___ Over Rs70,001

• 10 Non-exhaustive listings10 Non-exhaustive listingsDo you have all of the options covered? If you are unsure, conduct a pretest using the Do you have all of the options covered? If you are unsure, conduct a pretest using the "Other (please specify) __________" option"Other (please specify) __________" option

– Examples Examples Marital status Marital status – What are the listings? What are the listings?

• 11 Avoid loaded or leading questions 11 Avoid loaded or leading questions Leading questions such as “Do you agree with the majority of people that the health Leading questions such as “Do you agree with the majority of people that the health

service is failing?” should be avoided for obvious reasons that any right-minded service is failing?” should be avoided for obvious reasons that any right-minded individual can see. individual can see.

DiscussionDiscussion

Questionnaire Design

Documents

Transcript of Questionnaire Design