ECNU Educational Research Methodology

ECNU Educational Research Methodology

Lecture 2 Research design in quantitative research

2016/4/18 1 Fang Chen ECNU 陈芳华东师大


Review of last class

2016/4/18 Fang Chen ECNU 陈芳华东师大 2


QUANTITATIVE RESEARCH

Quantitative research has its roots in positivism, which Gall, Gall and Borg (2003) have defined as: The epistemological doctrine that physical and social reality is independent of those who observe it, and that observations of this reality, if unbiased, constitute scientific knowledge (p.632).



Research Design in quantitative research

Research Design in quantitative research tends to be structured and prescriptive, much more so than in qualitative research.

The outcomes are typically expressed as

numbers and enables the researchers to make valid interpretations through comparisons and partitioning of those numbers.



Purpose of design

What is the purpose of research design? 1. Provide answers to questions

2. Control variance This, to a large extent, unique to quantitative

research.



Controlling variance

Controlling variance means creating conditions that allow the researcher to get a clear look at the variable of interest by limiting or eliminating the influence of some variables and explaining the influence of others.



Example

Research statement: A study of the effect of teaching method on the

performance of high school students enrolled in chemistry

Method 80%

Method 55%



Controlling Variance

1. Randomization

2. Building conditions into the design as independent variables

3. Holding conditions constant

4. Statistical adjustments



1. Randomization

Recall that at least some of the variability may be from the differences between my groups.

If this is the case, we want it to be only because of the teaching method, NOT because of other factors Randomly pick participants from population Randomly place in groups



The randomization essentially equalized the groups with respect to ability level, but it also equalized them with respect to other variables such as motivation, attitude, to name a few.

Although this process distributes the effect of ability level evenly, it does not allow the researcher to quantify the influence of ability level on the scores of the chemistry test.

Figure 4.1



2. Build in Factors

If there is something that could affect the outcome, we can include it in the design.

They also account for variability, therefore reducing what is considered random noise.

Figure 4.3 Build in the ability factor



Issues

At times, adding additional factors is not possible

Need more examinees

May be too expensive

May take too much time



3. Holding Factors Constant

We could constrain some factors to be constant.

Think of student ability Only use students with IQ score between 100-108.



One danger is that you will lose external validity. Generalizability of research findings

Issues



4. Statistical control

Use all the data collected, Statistically control the factors that are

relevant to the study if measure of those factors exist

Statistical control would most likely account for more variance than building in student ability as a two-category, independent variable.



Example


MethodoreChemTestSc 10 ββ +=

IQMethodoreChemTestSc 210 βββ ++=

TeacherMethodoreChemTestSc 210 βββ ++=

MethodoreChemTestSc 10 ββ +=


Adequate scores on a control variable may not be available.

Statistical assumptions must be met; if they are not tenable, the statistical procedures should not be used.

Issues



Extending the chemistry example: In stead of using only 90 students, we used 180

within a school. Students come from different elementary schools

with various science background. The three teaching method is still the independent

variable of interest Three teachers teaching 6 classes in total, with 2

classes per teaching method IQ scores from 8th grade are available and

controlled

Using procedures for control in combination



Figure 4.7


1. Randomization

2. Building conditions into the design as independent variables

3. Holding conditions constant

4. Statistical adjustments


Characteristics of Good Research

Freedom From Bias

Freedom from Confounding

Control Extraneous Variables

Statistical Precision




Freedom From Bias E.g. higher ability students assigned to one method Students volunteered into a program may have more

motivation

Freedom from Confounding If each teacher uses one method, method effectiveness

may be confounded with teacher effectiveness

Control Extraneous Variables Statistical Precision 2016/4/18 23 Fang Chen ECNU 陈芳华东师大



Freedom From Bias Freedom from Confounding Control Extraneous Variables Randomization Build the factor into design Hold the factor constant Statistical control

Statistical Precision Statistical precision is increased with larger samples

and when additional independent variables are built into the research design.



When you read any article

Think about: 1. Independent and dependent variables. 2. Controlling variance. 3. Are there other factors that they might have

wanted to account for? 4. Does the study have bias or confounding

variables?


ECNU Educational Research Methodology

Documents

Transcript of ECNU Educational Research Methodology