School of Public Administration & PolicyDr. Kaifeng Yang

调查研究与问卷设计

问题调查的重要性截面型调查的优点和缺点问卷设计的几个问题

Survey Questions may include

1. Social background information

2. Reports of past behavior

3. Statements of attitudes, beliefs, values

4. Behavioral intentions

5. Sensitive information.

General Social Survey

The General Social Survey (GSS) is an omnibus personal interview survey of a national probability sample conducted since 1972 by the National Opinion Research Center.

Until 1994, the survey was conducted annually (except for 1979, 1981, and 1992) with a sample of about 1500 respondents.

Starting in 1994 the GSS shifted to biennial surveys with twice as many respondents. The objective of the survey is to provide high-quality data to the social science research community.

问题调查的重要性截面型调查的优点和缺点问卷设计的几个问题

Cross-sectional Study Descriptive value:

How many MPA students drink Red Bull? How much?

What is the age and sex distribution of the students who drink Red Bull?

Analytic value: Is there an association between regular Red Bull

consumption and test scores among MPA students?• Univariate• Multivariate (controlling for “confounders”)

Cross-sectional study: structure

time

Academic Performance

Red Bull consumption

time

- Cannot determine causality

Red Bull consumption

Academic Performance

Pros and ConsPros: Relative to experiments, surveys generally can address a wider

range of topics and collect substantially more information from much larger and more representative samples.

They are more flexible and more economical in that they can address several research questions at one time.

Cons: Surveys are less effective in testing causal relationships than

experiments They are limited to self-reports of behavior, which not only are

subject to self-censoring of responses but also cannot substitute for studies of overt behavior.

Comparison

Co-var.Timeorder

Nonspurious

Experiment

Cross sectional Usually notmaybe

External V

maybe

Satisfying and Compromising!

Internal Validity: Experiment vs. Cross-sectional In Experimental studies, try to control

experiment conditions so that two groups are under same conditions other than the stimulus

In cross-sectional study, you cannot “control” other conditions, so you try to measure them and include them as control variable in your analysis.

A ----> B.70

A ----> B.70

A ----> B.70

Female

Male

A ----> B.60

A ----> B.80

Female

Male

A ----> B-.23

A ----> B.90

Female

Male

Eliminating spurious relations

A ----> B.70

A ----> B.70

A ----> B.70

Older

Younger

A ----> B.00

A ----> B.00

Older

Younger

A: Shoe size

B: Math ability

Quiz A newspaper headline reads, Heavy

Drinker Get Lower College Grades. What would you conclude from this headline? What type of study was this based on?

Your textbook states that people remember concrete words better than abstract ones. Could this finding have come from an experiment? Would it be reasonable to infer that concreteness facilitates memory?

Are people who were abused as children more likely than others to become child abusers? What type of study would be used to research this question.

问题调查的重要性截面型调查的优点和缺点问卷设计的几个问题

Major decision points in planning a survey

The major decision points in planning a survey are:1) Formulate research objectives2) Review literature3) Select units of analysis and variables4) Develop sampling plan5) Construct survey instrument

(Steps 4 and 5 are usually concurrent activities).

From Theory to Questionnaire

Theory

Model Hypothesis

Dependent VsIndependent Vs Control Vs

v1 v2 v3 v4 v5 v6 v7 v8 v9… …

1 indicatorD1 D2

6 indicator

6indicator

6indicator

Order effect Question order can affect the survey results. Mentioning

something (an idea, an issue, a brand) in one question can make people think of it while they answer a later question, when they might not have thought of it if it had not been previously mentioned.

In some cases you may be able to reduce this problem by randomizing the order of related questions.

Separating related questions with unrelated ones can also reduce this problem, though neither technique will eliminate it.

More general and therefore more susceptible to order effects.

Socially desirable responses

The tendency to give socially desirable responses may be minimized by carefully wording and placing sensitive questions, assuring anonymity and emphasizing scientific importance, making statements sanctioning less socially desirable responses, and building interviewer-respondent rapport.

Wording, Frame, Response Format

George is a 19-year-old male who has been arrestedfor assault and battery with intent to commit murder. George is pleading guilty to the charge. It is his first offense. On the basis of this information what sentence would you recommend?

Pitfalls Biased or leading questions

4-point rating with responses of “excellent,” “good,” “satisfactory,” and “poor.”

What do you see as the benefits of a tax cut? Vs. What do you see as the disadvantages of eliminating welfare?

Don’t you think that global warming is a serious environmental problem?

Loaded questions “public servants,” “public administrators,” or

“government bureaucrats?” Social desirability

Are you proud of being a public servant?

Pitfalls

Double-Barrel To what extend do you support abortion and gun-

control? Do you agree that the SU is ahead of US in the arms

race and US should increase defense spending? Ambiguous Questions

What is your income? Questions respondents may not know Questions not necessary

Other

Single-sided vs. two-sided question Do you agree or disagree with the idea that the

government should see to it that every person has a job?

Do you think that the government should see to it that every person has a job, or should it let each person get ahead on his or her own?

Closed-ended vs. Open-ended questions Comparing: meaning; time; coding; response…

Response rate and non-response bias