Instructor: Da-Fu HuangInstructor: Da-Fu HuangStudents: Students: M98C0209 M98C0209 李威德李威德 EdenEden

M98C0104 M98C0104 陳泓銚陳泓銚 WilliamWilliamM98C0101 M98C0101 汪怡菁汪怡菁 DaraDara

Quasi-Experimental Research:

Vocabulary Learning Vocabulary Learning TechniquesTechniques

All life is an All life is an experiment experiment

Ralph Waldo Emerson, 1802-1882Ralph Waldo Emerson, 1802-1882

Introducing experimental research

The most appropriate way to resolve a The most appropriate way to resolve a question about language learning or question about language learning or teaching is to conduct an experiment.teaching is to conduct an experiment.

Introducing experimental research

The experience in this chapter involves an The experience in this chapter involves an experiment comparing two ways that have experiment comparing two ways that have been proposed to help learners study and been proposed to help learners study and remember second language vocabulary.remember second language vocabulary.

Exercise 7.1

The vocabulary recall experiment

There are 2 lists of 25 words each. (List A There are 2 lists of 25 words each. (List A & List B) P.196 & List B) P.196

It takes 2 minutes and then write down all the It takes 2 minutes and then write down all the words you can remember. words you can remember.

You better have timer, watch or a partner.You better have timer, watch or a partner.

List A

1.Bath1.Bath 2.Write2.Write 3.Bed3.Bed 4.Crisp4.Crisp 5.Dream5.Dream

6.Fresh6.Fresh 7.Glass7.Glass 8.Shiny8.Shiny 9.Jar9.Jar 10.Lie10.Lie

11.Cup11.Cup 12.Listen12.Listen 13.Window13.Window 14.New14.New 15.Pail15.Pail

16.Read16.Read 17.Bright17.Bright 18.Think18.Think 19.Rest19.Rest 20.Bucket20.Bucket

21.Room21.Room 22.Sleep22.Sleep 23.Speak23.Speak 24.Relax24.Relax 25.Table25.Table

List B

1.Light1.Light 2.Lamp2.Lamp 3.Candle3.Candle 4.Bulb4.Bulb 5.Lantern5.Lantern

6.Sit6.Sit 7.Stand7.Stand 8.Kneel8.Kneel 9.Bend9.Bend 10.Jump10.Jump

11.Old11.Old 12.Ancient12.Ancient 13.Tired13.Tired 14.Weak14.Weak 15.Bent15.Bent

16.Tire16.Tire 17.Wheel17.Wheel 18.Engine18.Engine 19.Gear19.Gear 20.Door20.Door

21.Desk21.Desk 22.Student22.Student 23.Teacher23.Teacher 24.Pencil24.Pencil 25.eraser25.eraser

List B (version of tip)

Exercise 7.2This experiment deals with learning This experiment deals with learning

vocabulary pairs in two languages.vocabulary pairs in two languages.

You will have ten seconds to practice You will have ten seconds to practice saying the Spanish word and its English saying the Spanish word and its English translation to yourself as many times as translation to yourself as many times as you can. you can.

Test E no. / Spanish /Translationno. / Spanish /Translation

1. Caracol 1. Caracol 2. Barro 2. Barro 3. Toalla 3. Toalla 4. Vientre 4. Vientre 5. Carpa 5. Carpa ________________________________________ 6. Comedor 6. Comedor 7. Tallarin 7. Tallarin 8. Corazon 8. Corazon 9. Prado 9. Prado 10. Parabriasa10. Parabriasa

Score Score ______ ______

11. Sabor 11. Sabor 12. Pollo 12. Pollo 13. Sabio 13. Sabio 14. Herida 14. Herida 15. Lata 15. Lata ____________________________________ 16. Jefe 16. Jefe 17. Polvo 17. Polvo 18. Espaldas 18. Espaldas 19. Helado 19. Helado 20. Viajero 20. Viajero

Test Fno. / Spanish word / Translationno. / Spanish word / Translation

1. Tarde 1. Tarde 2. Cama 2. Cama 3. Fondo 3. Fondo 4. Ciudad 4. Ciudad 5. Payaso 5. Payaso ________________________________________________ 6. Postre 6. Postre 7. Mneca 7. Mneca 8. Huevo 8. Huevo 9. Bombero 9. Bombero 10. Piso 10. Piso

Score Score ______ ______

11. Saltamontes 11. Saltamontes 12. Ayuda 12. Ayuda 13. Pulgado 13. Pulgado 14. Bolsillo 14. Bolsillo 15. Arroz 15. Arroz ________________________________________________ 16. Arena 16. Arena 17. Sabana 17. Sabana 18. Guerra 18. Guerra 19. Mujer 19. Mujer 20. Gusano20. Gusano

Test E no. / Spanish / Translationno. / Spanish / Translation

1. Caracol Snail1. Caracol Snail 2. Barro Mud2. Barro Mud 3. Toalla Towel 3. Toalla Towel 4. Vientre Belly4. Vientre Belly 5. Carpa Tent5. Carpa Tent ________________________________________ 6. Comedor Dining room6. Comedor Dining room 7. Tallarin Noodle7. Tallarin Noodle 8. Corazon Heart8. Corazon Heart 9. Prado Meadow9. Prado Meadow 10. Parabriasa10. Parabriasa WindshieldWindshield

11. Sabor Taste11. Sabor Taste 12. Pollo Chicken12. Pollo Chicken 13. Sabio Scholar13. Sabio Scholar 14. Herida Wound14. Herida Wound 15. Lata Tin can15. Lata Tin can ____________________________________ 16. Jefe Boss16. Jefe Boss 17. Polvo Dust17. Polvo Dust 18. Espaldas Back18. Espaldas Back 19. Helado Ice cream19. Helado Ice cream 20. Viajero Traveler20. Viajero Traveler

Test Fno. / Spanish word / Translationno. / Spanish word / Translation

1. Tarde Afternoon1. Tarde Afternoon 2. Cama Bed2. Cama Bed 3. Fondo Bottom3. Fondo Bottom 4. Ciudad City4. Ciudad City 5. Payaso Clown5. Payaso Clown ________________________________________________ 6. Postre Dessert6. Postre Dessert 7. Mneca Doll 7. Mneca Doll 8. Huevo Egg8. Huevo Egg 9. Bombero Fireman9. Bombero Fireman 10. Piso Floor10. Piso Floor

11. Saltamontes Grasshopper11. Saltamontes Grasshopper 12. Ayuda Help 12. Ayuda Help 13. Pulgado Inch13. Pulgado Inch 14. Bolsillo Pocket14. Bolsillo Pocket 15. Arroz Rice15. Arroz Rice ________________________________________________ 16. Arena Sand16. Arena Sand 17. Sabana Sheet17. Sabana Sheet 18. Guerra War18. Guerra War 19. Mujer Woman19. Mujer Woman 20. Gusano Worm20. Gusano Worm

Compiling experimental data

Compile the in any way that makes sense Compile the in any way that makes sense to you as a group, in a format similar to to you as a group, in a format similar to that shown in table 7.1.that shown in table 7.1.

Analyzing experimental data

One of the women wondered if One of the women wondered if the women remember things or the women remember things or learn words need make a list than learn words need make a list than men.men.

Analyzing experimental data

Some of men objected, claiming tSome of men objected, claiming they were dedicated list maker and hey were dedicated list maker and user.user.

Table 7.3 Example recall data (www)

List AList AM M (SD)(SD)

List BList BM M (SD)(SD)

Women Women NN=17=17 18.5(2.5)18.5(2.5) 22.5(2.0)22.5(2.0)

Men Men NN=17=17 14.5(3.5)14.5(3.5) 21.5(2.5)21.5(2.5)

Computing the t-test in data analysis

The t-test is the most frequently used measure The t-test is the most frequently used measure in second language research when comparing in second language research when comparing mean scores for two groups.mean scores for two groups.

)..(. upaddorofsum

Computing the t-test in data analysis

SD = standard deviationSD = standard deviation X = valuesX = values M = mean of the values M = mean of the values

N = number valuesN = number values

M= meanM= mean X= valuesX= values N= number of valuesN= number of values

Once you have computed these data, you will Once you have computed these data, you will want to knowwant to know if the difference if the difference between the between the two meanstwo means is is significant or notsignificant or not??

Significance is a statistical way of talking Significance is a statistical way of talking about whether the difference between groups’ about whether the difference between groups’ means is biggish or smallish.means is biggish or smallish.

When comparing pairs of means the statistical When comparing pairs of means the statistical measure typically used is the measure typically used is the tt-test.-test.

T test (T 檢定 )The t-test is the most frequently used The t-test is the most frequently used

measure in second language research measure in second language research when comparing mean scores for two when comparing mean scores for two groups.groups.

It can be used with very large or very small It can be used with very large or very small groups.groups.

Degrees of Freedom

Group size is (roughly) adjusted fGroup size is (roughly) adjusted for by degree of freedom.or by degree of freedom.

df. = N1+N2-2 =(N-1)+(n-1)df. = N1+N2-2 =(N-1)+(n-1)Example: (17-1)+(17-1)=32Example: (17-1)+(17-1)=32

Degrees of Freedom

Adjustments for differences in types of Adjustments for differences in types of decision is made by considering one-tailed decision is made by considering one-tailed and two tailed decisions separately.and two tailed decisions separately.

T test (T 檢定 )

w= womenw= women m= menm= men M= meanM= mean

T test (T 檢定 )

175.3

175.2

5.145.1822mw

t

Women (N=17)Women (N=17)Mean M=18.5Mean M=18.5Standard Deviation S=2.5Standard Deviation S=2.5

Men (N=17)Men (N=17)Mean M =14.5Mean M =14.5Standard Deviation S=3.5Standard Deviation S=3.5

→→

t= t= 3.833.83 df= df= 3232 How should we do if the exact df is not sHow should we do if the exact df is not shown in the table?hown in the table? The value is greater than the critical value foThe value is greater than the critical value found in the table at .01 or .05.und in the table at .01 or .05. The result means the women’s mean is The result means the women’s mean is sigsignificantly highernificantly higher than the men’s mean score. than the men’s mean score.

Abstract Gender differences on list recall of 25 wordsGender differences on list recall of 25 words Women had a mean word recall of 18.5 (SD= 2.Women had a mean word recall of 18.5 (SD= 2.

5)5) Men had a mean word recall of 14.5 (SD= 3.5)Men had a mean word recall of 14.5 (SD= 3.5) A t-test analysis of the differences between meaA t-test analysis of the differences between mea

ns yielded a t of 3.83ns yielded a t of 3.83 This was significant at the p< .01 level (df= 32)This was significant at the p< .01 level (df= 32) Women were significantly better than men at reWomen were significantly better than men at re

member wordsmember words

If the group sizes are different:

The Same Result This formula could also be used in situations where This formula could also be used in situations where

the group sizes are the same.the group sizes are the same.

Designing Your Own Experimental Studies

a)a) Experimental approaches to language Experimental approaches to language teaching researchteaching research

b)b) The big method-comparison studiesThe big method-comparison studies

c)c) Creating research questions and the Creating research questions and the accompanying hypothesesaccompanying hypotheses

d)d) Designing experiments to test hypothesesDesigning experiments to test hypotheses

Experimental Approaches to Language Teaching

What is experiment?

“ “a situation in which one observes the rela situation in which one observes the rel

ationship between two variables by delibationship between two variables by deliberately producing a change in one and loerately producing a change in one and lo

oking to see whether this alteration prodoking to see whether this alteration prod

uces a change in the other” (Anderson 1uces a change in the other” (Anderson 1

969).969).

True Experimental Studya)a) Students are randomly selected and Students are randomly selected and

assigned to two groups.assigned to two groups.

b)b) Two experimental conditions or Two experimental conditions or treatments are provided.treatments are provided.

c)c) For two groups, a pre- and post- test are For two groups, a pre- and post- test are given.given.

Two groups of participants are assumed to Two groups of participants are assumed to be equivalent.be equivalent.

Control group- it will receive a control Control group- it will receive a control treatment.treatment.

Treatment group- it will receive the Treatment group- it will receive the experimental treatment in the form of some experimental treatment in the form of some new approach. new approach.

XO

X is some kind of educational treatmentX is some kind of educational treatmentO is an observation following the treatmentO is an observation following the treatmentCampbell and Stanley note: Research Campbell and Stanley note: Research

today conforms to a design in which a today conforms to a design in which a single group only once, subsequent to single group only once, subsequent to some agent or treatment presumed to some agent or treatment presumed to cause change.cause change.

Limitation

Second language researchers can’t obtain raSecond language researchers can’t obtain randomly selected groups of participants.ndomly selected groups of participants.

The experiment already exists intact groups.The experiment already exists intact groups.Such variation from the formal requirements fSuch variation from the formal requirements f

or a true experiment causes such a study to or a true experiment causes such a study to be called Quasi-Experimental.be called Quasi-Experimental.

The Big Method-Comparison Studies

““The age of methods” in second language The age of methods” in second language instruction.instruction.

There are many methods shown in Table 7.7.There are many methods shown in Table 7.7.Many method-comparison studies were Many method-comparison studies were

undertaken with designs hoping to undertaken with designs hoping to approximate pure experimental studies.approximate pure experimental studies.

The research methodology of the studies The research methodology of the studies was very heavily criticized.was very heavily criticized.

However, these research becomes the However, these research becomes the focusing of greater qualitative studies.focusing of greater qualitative studies.

Creating Research Questions and Hypotheses

Research Questions- Questions that you Research Questions- Questions that you will try to answer in the studywill try to answer in the study

Research Hypotheses- claims that you Research Hypotheses- claims that you believe the experimental study might believe the experimental study might support support

List A: 25 randomly organized List A: 25 randomly organized

common English words.common English words.List B: 25 common English words List B: 25 common English words

organized into sets of five words.organized into sets of five words.The researcher posed a research and The researcher posed a research and

hypothesis. hypothesis.

Research QuestionWill List B be easier to recall than Will List B be easier to recall than

List A?List A?

Hypothesis 1a Participants will better recall words in List Participants will better recall words in List

B than List A.B than List A.--------------------------------------------------------------------------------------------------If this research is hypothesis rather than If this research is hypothesis rather than

questions, experimenters must also questions, experimenters must also anticipate competing hypotheses.anticipate competing hypotheses.

Competing hypotheses = Alternative Competing hypotheses = Alternative hypotheses hypotheses

Hypothesis 1b List B will promote less successful recall List B will promote less successful recall

than allowing subjects to apply their own than allowing subjects to apply their own strategies.strategies.

--------------------------------------------------------------------------------------------------1.1. This hypothesis is alternative hypothesis This hypothesis is alternative hypothesis

to 1a.to 1a.

Hypothesis 1c List B no better than List AList B no better than List A------------------------------------------------------------------------------------------------------------1.1. Competing hypothesis = Null hypothesisCompeting hypothesis = Null hypothesis2.2. It may be that neither List A nor List B It may be that neither List A nor List B

will prove superior to the other.will prove superior to the other.3.3. They will be equally effective.They will be equally effective.

Setting up Experiments to Test Hypotheses

New Experimental Research Areas in Vocabulary Learning

Significance of Experimental Research in Teaching Vocabulary

Research and pedagogical neglect during the 1950s Research and pedagogical neglect during the 1950s and 1960s.and 1960s.

New methodologies came in the 1970s.New methodologies came in the 1970s.

a)a) Communicative Language LearningCommunicative Language Learning

b)b) Silent WaySilent Way

c)c) Suggestopedia Suggestopedia

d)d) Natural Approach, etc.Natural Approach, etc.

Significance of Experimental Research in Teaching Vocabulary

Krashen (1977)Krashen (1977): Commonsensical view: Commonsensical view SoundSound and and meaningmeaning are two main elements of lan are two main elements of lan

guage.guage.Words hold sound and meaning together. hold sound and meaning together.Words are combined by sounds.are combined by sounds. When you teach how sentences convey meaning, When you teach how sentences convey meaning,

you teach how you teach how words go together. go together.

Significance of Experimental Research in Teaching Vocabulary

Laufer and Hulstijn (2001)Laufer and Hulstijn (2001):: Stimulate theoretical thinking & Empirical researStimulate theoretical thinking & Empirical resear

chch L2 learning = learning of large numbers of wordsL2 learning = learning of large numbers of words The association of new vocabulary words with exThe association of new vocabulary words with ex

isting knowledge.isting knowledge.

Summary Experimental subject vs. Control subjectExperimental subject vs. Control subject tt-test-test: to compare mean scores for two groups: to compare mean scores for two groups

mwmw

2 mm

2 ww

mw

N1

N1

2-NNS1-NS1-N

M-M t

Summary

df df (degree of freedom(degree of freedom):): to adjust group sizeto adjust group size

(N(N11-1) + (N-1) + (N22-1) = -1) = dfdf

Table 7.5 (P. 207)Table 7.5 (P. 207)

Research questions or hypothesesResearch questions or hypotheses

Example Paper #1English department students conducted

remedial teaching to non-English department students

QuestionnairesQuestionnairesPre-test and Post-testPre-test and Post-testXOXO study (1 group / 1 treatment) study (1 group / 1 treatment)use ofuse of tt-test-test

T-Test Results of the Listening Comprehension Tests

GroupsGroups NN MeanMean SDSD tt dfdf

Pre-testPre-test8080

85.2085.20 21.5721.570.610.61 158158

Post-testPost-test 83.2183.21 19.5019.50

**p=0.05

T-Test Results of the Reading Comprehension Tests

GroupsGroups NN MeanMean SDSD tt dfdf

Pre-testPre-test8080

57.2357.23 16.3316.33-4.78-4.78 158158

Post-testPost-test 70.9970.99 19.9519.95

**p=0.05

Example Paper #2

To investigate whether or not statistically To investigate whether or not statistically significant differences exist between using significant differences exist between using the task-based approach VS the Grammar the task-based approach VS the Grammar Translation Method, both integrated with Translation Method, both integrated with computer-assisted instruction, to improve computer-assisted instruction, to improve Taiwanese EFL college students’ speaking Taiwanese EFL college students’ speaking performance.performance.

Purpose

Research Question Is there a significant difference between Is there a significant difference between

integrating the task-based approach and integrating the task-based approach and the Grammar Translation Method, both the Grammar Translation Method, both with computer-assisted instruction, in with computer-assisted instruction, in terms of improving Taiwanese EFL college terms of improving Taiwanese EFL college students’ speaking performance?students’ speaking performance?

TBATBA GTMGTM

PretestPretest N=50N=50M=54.00M=54.00SD=16.78SD=16.78

N=43N=43M=52.09M=52.09SD=15.82SD=15.82

df= 91df= 91t= t= 0.160.16p>.05p>.05

PosttestPosttest N= 50N= 50M= 58.00M= 58.00SD= 16.78SD= 16.78

N= 43N= 43M= 45.58M= 45.58SD= 21.08SD= 21.08

df= 91df= 91tt= 5.022= 5.022p<.05p<.05

The End