Diagnosis and Remedy Syst ems (DRS) for Teaching English...

47

영어교육 62권 2호 2007년 여름

Diagnosis and Remedy Systems (DRS) for Teaching English Based on the Common Metric Scale (CMS) Model

Yong-Myeong Kim (Hankuk University of Foreign Studies)

Kim, Yong-Myeong. (2007). Diagnosis and Remedy Systems (DRS) for teaching English based on the Common Metric Scale (CMS) model. English Teaching, 62(2), 47-77.

On the basis of the Parallel Developmental Sequence (PDS) Model (Kim & Kwon, 2005, 2007), Kim (2006c) proposed the Common Metric Scale (CMS) Model in terms of the IL approach. The IL approach incorporating the Testability and the Bounded-domain hypothesis converts the PDS into the CMS, thereby the scales, the scale descriptors, and the hierarchies of task or item difficulty being all in an implicational relationship. To retest the CMS, this study carried out additional research to Kim (2006c). The results reconfirmed that the CMS has its validity. Thus, the CMS can function as an acquisition-oriented scale, thereby providing “a tailored teaching for a timely remedy” for IL learners. Given this possibility, this study proposes an executive system of the CMS called the Diagnosis and Remedy Systems (DRS). The DRS consists of four interrelated subsystems (i.e., the CMS, the Testing, the Diagnosing, and the Teaching system). This study shows how the DRS is implemented into language teaching and testing, and SLA research. It also gives an answer to the question of “when and how to provide what rules for IL learners.” Finally, this study suggests that the DRS on the CMS can provide a possible framework for investigating the relationship between the development of forms and that of functions.

I. INTRODUCTION As a future direction toward language testing, Bachman (1990) proposed the

development of a common metric scale for measuring language abilities in a wide variety of contexts, at all levels, and in many different languages in order to meet various needs for language teaching and second language acquisition (SLA) research. However, as Bachman (1989, 1990) pointed out, such a scale has been relatively sparse; still less the application of SLA research to this scale has not yet been tried. Also, Bachman and Cohen

Kim, Yong-Myeong

48

(1998) suggested that there should be an interface to enhance reciprocal developments between language testing and SLA research. The advance in language testing can give both practical criterion measures and theoretical insights for developments in SLA research. Conversely, the development in SLA research can also provide insightful information for developing an “interlanguage (IL)-sensitive” test tailored for IL developments. Then, where is a parallel interface to enhance reciprocal developments between SLA research and language testing? To answer this question, Kim (2006c) proposed the Common Metric Scale (CMS) Model as an acquisition-oriented scale based on the Parallel Developmental Sequence (PDS) Model (Kim & Kwon, 2005, 2007).

The PDS Model as a modular model comprises the Parallel Developmental Stages implicationally called 1X, 2X,..., 1X2, 2X2,..., 1X3,..., kXn. Each parallel stage of the PDS incorporates three different dimensions of sequential development: the Procedural, the Syntactic, and the Morphological Sequence (Full discussion will be presented in the next section). It follows that inherent in the PDS is an implicational relation in which the higher stage subsumes the parallel mechanisms and the critical structures of the lower stages, but not vice versa. In conclusion, the PDS Model can throw a new light on such crucial notions for test designer as “developmental sequence” and “teachability”, and thus help test developers to develop “IL-sensitive” or “acquisition-oriented” tests that are sensitive to, or tailored for language developmental sequence.

The CMS Model (Kim, 2006c) on the PDS was developed in terms of the IL approach along with the Testability and the Bounded-domain hypothesis. The IL approach enables us to convert the PDS into the CMS, thereby the absolute scales, the scale calibration, and the scale descriptions of the CMS being automatically defined. In addition, the Testability hypothesis enables us to gear test tasks to each stage of the PDS along which IL learners pass through. The Bounded-domain hypothesis enables us to synchronize test domains with IL discourse domains on the continuum ranging from more concreteness to more abstractness along which IL learners assume to create and expand their own domains. Thus, the scales, the scale descriptors of the CMS, and the hierarchies of task difficulty are all stand in an implicational relationship to each other, and hence can satisfy the triple requirements of an implicational relationship for the development of a CMS. In conclusion, the CMS can function as an IL-sensitive or acquisition-oriented scale, and hence constitute a parallel interface to enhance the reciprocal developments between language teaching, language testing, and SLA research.

As Kim (2006c) mentioned, the CMS based on the PDS provides some implications for language teaching and testing, and SLA research. Among other things, as Brindley (1998) suggested, the CMS along with the Rapid Profile (Pienemann & Mackey, 1993) can be used as a useful diagnostic device for identifying IL learners’ current parallel developmental stages, pinpointing the areas of deficiency in their IL grammar, and

Diagnosis and Remedy Systems (DRS) for Teaching English

49

predicting what rules or structures will be learnable or teachable next, thereby giving them a “timely remedy and tailored teaching”. Thus, the PDS and its executive counterparts, the CMS can provide an alternative to the principal question of recent research on Form-focused Instruction: When and how can we provide what rules for IL learners? (Doughty & Williams, 1998; Robinson, 1996).

Given these possibilities, the purpose of this paper is to propose an executive system of the CMS called the Diagnosis and Remedy Systems (DRS). The second section of this paper will explain theoretical aspects of the PDS and the CMS. The third will describe experimental designs, and discuss the results of the experiments. In the forth section, based on these research findings, this study will present the DRS, and show the way it is implemented into language teaching and testing, and SLA research, finally giving an answer to the question of when and how to provide what rules for IL learners.

II. THEORETICAL BACKGROUND

1. The Parallel Developmental Sequence (PDS) Model From the reviews of such theories as the Multidimensional Model (Meisel, Clahsen, &

Pienemann, 1981; Pienemann & Johnston, 1987), the Minimal Tree Approach (Vainikka & Young-Scholten, 1998), and the Processability Theory (Pienemann, 1998), Kim and Kwon (2005, 2007) proposed the Parallel Developmental Sequence (PDS) Model. The PDS comprises three separate L2 developmental modules, that is, the Procedural Module (ProDS), the Syntactic Module (SynDS), and the Morphological Module (MorDS).

The three sub-developmental modules (i.e., the ProDS, the SynDS, and the MorDS) are able to provide valid explanations for each of their own domains, but they have “complementary limitations” of not being able to offer explanations for each of the other domains related to IL development. In other words, as Kim and Kwon (2006) pointed out, the ProDS explains processability but can’t explain grammaticality. Conversely, as Kim (2006b) hinted, the SynDS explains grammaticality but can’t explain processability. As Kim (2006a) pointed out, the MorDS explains morphological development but can’t explain structural development.

A possible alternative to such complementary limitations might be what is called the Modular Approach (Ellis, 1994; Gregg, 1996; Pienemann, 1998; White, 1989), which suggests that each module is in a complementary relation with each other. According to the Modular Approach, these three sub-developmental modules were integrated into a single modular model called the Parallel Developmental Sequence (PDS) Model as illustrated in Figure 1. The PDS constitutes the Parallel Developmental Stages sequentially

Kim, Yong-Myeong

50

(i.e., 1X, 2X, ..., 1X2, 2X2, ..., 1X3, ..., kXn). Each Parallel Stage of the PDS incorporates three sub-developmental sequences: the ProDS, the SynDS, and the MorDS. Therefore, the PDS allows the complementary limitations to have complementary explanations for each other, and thus, provides an interface between the three sub-developmental sequences (for full discussion of the PDS Model, see Kim & Kwon, 2007).

FIGURE 1

Parallel Developmental Sequence Model (Kim & Kwon, 2007, p. 256)

Parallel Developmental Stages (PDS) kXn ⇑

⋮ ⇑

1X2 ⇑ 4X ⇑ 3X ⇑ 2X ⇑

1X ⇑ [N V N]

V +/∙ [V…]

VP ⇄ +/+ ⇄ [VP…]

IP ⇄ -/+ ⇄ [IP…]

CP1 ⇄ -/- ⇄ [CP1…]

CP2 ⇄ [[±/±([±/±])] ⇄ [CP2…]

⋮ ⇄ ⋮ ⇄ ⋮

CPn ⇄ [[±/±([±/±])]…] ⇄ [CPn…]

Syntactic Stages (SynDS) ⇄ Procedural Stages (ProDS) ⇄ Morphological Stages (MorDS)

A closer examination of the figure illustrates that inherent in the PDS is an implicational

relation in which the higher stage can subsume the parallel mechanisms and the critical structures of the lower stages, but not vice versa. What this implicational relation means is that no parallel stage on the PDS can be skipped or beaten, and hence, in a sense, the PDS constitutes the predetermined shedding processes of the parallel mechanisms. Therefore, L2 learners cannot proceed from one stage to the subsequent stage until they can process, parse and unify the parallel mechanisms imposed on a given stage on the PDS, just as a caterpillar goes through the ordained shedding process step by step and finally becomes a butterfly. From this, a motivating principle of language pedagogy, what Pienemann (1985, 1998) calls the Teachability principle, can be derived. This implies that any instruction will be beneficial if it focuses on structures or rules of the next or subsequent stage of an L2 learner’s current stage on the PDS. In conclusion, the PDS Model can cast a new light on such crucial notions for test developers as “language development” and “teachability principle”, and thus help test designer to develop “IL-sensitive” or “acquisition-oriented”


51

tests that are sensitive to, or tailored for language developmental sequence.

2. The Common Metric Scale (CMS) Model on the PDS In order to constitute a parallel interface to enhance the reciprocal developments

between SLA research and language testing, Kim (2006c) developed the Common Metric Scale (CMS) as an acquisition-oriented scale based on the PDS Model.

As a basis for the development of a common metric scale, Kim (2006c) proposed the interlanguage (IL) approach, the Testability and the Bounded-domain hypothesis. The IL approach derived from the developmental approaches (Ingram, 1985) relates developmental information and insights from sequence-based SLA research (Kim & Kwon 2005, 2007; Pienemann, 1998) to the overall plans for developing and implementing a CMS. The Testability hypothesis was extrapolated from what the Pienemann calls the Teachability principle (1985, 1998). Given that any instruction will be beneficial when it focuses on structures or rules of the next or subsequent stage of the IL learners’ current stage on the developmental sequence, then, any test task will be “authentic” to the test taker if it relates to structures or rules of the next or subsequent stage of the IL learners’ current stage. The Bounded-domain hypothesis was derived from intertwining what Douglas and Selinker (1985) call the discourse domains and what Pienemann (1998) calls the Steady hypothesis. Douglas and Selinker (1985) presented data from both IL research data and speaking tests to support one of their testing principles that “test takers create individual discourse domains, which are a function of the discourse domains they have already created in their IL use” (recited from Bachman, 1990, p. 324). They also presented another principle that “the closer the context supplied by the test writer to prototypical internal IL contexts created by the test taker, the more likely it is that the test will engage the test taker’s ability to perform the task at hand in the L2” (p. 220). Pienemann (1998) undertook task variation research in which ESL learners carried out different communicative tasks. The analysis of the IL profiles of the learners demonstrated that the IL of all the informants conformed to the same grammatical rules across different communicative tasks, and that the tasks did not alter the developmental status of the IL. Based on this result, he proposed the Steady hypothesis, according to which the basic nature of the grammatical system of an IL does not change in different communicative tasks as long as they are based on the same skill type in language production. What both studies imply is that content or task variation is not a “free” variation, but a “bound” variation. Thus, if we combine Douglas and Selinker’s discourse domains with Pienemann’s Steady hypothesis, then, an interesting hypothesis will emerge, namely, that discourse domains will extend not so much “unboundedly”, unshackled with syntactic developments, as “boundedly”, interlocked with syntactic developments. Thus, the

Kim, Yong-Myeong

52

Bounded-domain hypothesis postulates that as IL learners proceed from one stage to the next higher stage on the Parallel Developmental Sequence (PDS), so will they create and extend their own IL discourse domains along a continuum ranging from more concreteness to more abstractness. Both IL discourse domains and test domains are, in general, assumed to constitute a continuum of the IL discourse domains ranging from everyday situations and contexts (e.g., greeting, dinning) to more specific ones (e.g., court, war), and from more general or concrete topics and contents (e.g., food, sports) to more abstract or academic ones (e.g., history, politics, philosophy).

Gathering up all threads, Kim (2006c) developed the CMS for measuring language proficiency based on the PDS in terms of the IL approach, along with the Testability and the Bounded-domain hypothesis as seen in Table 1.

TABLE 1

The Common Metric Scale (CMS) for Language Proficiency on the PDS (Kim, 2006c, p. 91) Parallel Developmental Sequence (PDS)

SynDS ProDS MorDS Parallel Stages

Syntactic M. Procedural M. Unificational M. Critical Structures Error

Patterns

CMS Levels

1X Lexical-inserting [ㆍIFM][+COM] No unification Canonical SVO … 0

2X Theta-marking [+IFM] [+COM] Phrasal [VP…] Wh/Do-frontings Aux-en/-ing, … 1

3X Case-marking [+IFM][-COM] Clausal [IP…] Y/N-questions Topicalizations … 2

4X Force-realizing [-IFM][-COM] Sentential [CP…] Wh-questions, Neg-Inv. … 3

1X2 [ㆍSCM([ㆍIFM][+COM])] 2-sub comp, that/for-clauses … 4

2X2 [+SCM([+IFM][+COM])] Indirect-questions Relatives (subject) … 5

3X2 [+SCM([+IFM][-COM])] Relatives (object, oblique) … 6

4X2 [+SCM([-IFM][-COM])] Indirect-questions (object) … 7

5X2

Force-matching, Markedness on Operation

[-SCM([-IFM][-COM])]

Intersentential [CP [CP…]]

Long distance-questions Cleft sentences … 8

kX3 … [±SCM([±SCM])] … Double complex sentences … 9

The IL approach enables the PDS to be converted into the CMS, and thus the absolute

scales, the scale calibration, and the scale descriptions of the CMS can be automatically defined. According to the IL approach, as seen in Table 1, the CMS consists of three componential or analytic scales, that is, the Syntactic Scale, the Procedural Scale, and the Morphological Scale, with each scale corresponding to each developmental sequence of the PDS, that is, the Syntactic, the Procedural, and the Morphological Sequence. As Bachman and Palmer (1996) mentioned, the use of analytic scales can provide a “profile” of the components of language ability by differentiating among areas of relative strength and weakness. On the IL approach, the absolute scales of the CMS can be defined in terms


53

of the lowest (initial) and the highest (final) stage of the PDS. Stage 1X of the PDS is equal to the absolute “zero” proficiency (Level 0) of the CMS, and Stage kX3 of the PDS to the absolute “perfect” proficiency (Level 9) as illustrated in Table 1. Also, to follow the IL approach, the number of intermediate levels can be automatically determined in terms of the number of stages in the PDS, thereby the CMS consisting of a total of 10 scales (scale 0-9) as seen in Table 1. In addition, now that the PDS constitutes an implicational scale, the descriptions of each level can be consistently described and in an implicational relationship. To put it simply, as illustrated in Table 1, the descriptions for Level 0 corresponds to the parallel mechanism, critical structures, and error patterns of Stage 1X on the PDS, and the descriptions for Level 1 to those of Stage 2X on the PDS, and so on.

How can we establish an implicational relationship between the scale descriptions and the hierarchies of task difficulty? This is, in fact, the thorniest of the four problems to be settled in developing any CMS.1 It requires, in theory, triple implicational scales. One requirement is on an implicational scale between the scales of a CMS that a higher scale must subsume the lower ones, but not vice versa. Another is on an implicational scale between the scale descriptions that a higher description must subsume the lower ones, but not vice versa. The third requirement is on an implicational scale between the scale descriptions and the task hierarchies that the higher tasks or items mapping the higher descriptors must subsume the lower ones mapping the lower descriptors, but not vice versa. To follow the IL approach would meet the triple implicational scales in logic. In accordance with the Testability hypothesis, tasks or items geared to each stage of the PDS on which IL learners proceed from one stage to the next higher stage are developed to establish the hierarchies of task or test difficulty. Then, according to the Bounded-domain hypothesis, the task or test domains are synchronized with the IL discourse domains on the continuum ranging from more concreteness to more abstractness along which IL learners assume to create and expand their own domains. Thus, the scales, the scale descriptors, and the task or test hierarchies will all stand in an implicational relationship to each other, and hence will, in logic, satisfy the triple requirements of an implicational relationship. In conclusion, the IL approach enables us to map the PDS onto the CMS, and hence the CMS on the PDS can function as an IL-sensitive or an acquisition-oriented scale for language teaching and language testing.

1 From the reviews of language testing studies such as Bachman (1990), Bachman and Palmer (1996),

and Brindley (1998), Kim (2006c) identified four problems to be solved in developing a CMS as follows; (1) how to define absolute levels on what basis, (2) how many scales to have between the absolute levels, (3) how to specify the scale definitions (descriptors), and (4) how to establish an implicational relationship between the scale descriptions and the hierarchies of task difficulty.

Kim, Yong-Myeong

54

III. TESTING A COMMON METRIC SCALE (CMS) ON THE PDS

1. Research Method2

1) Participants The participants of the study were 162 college students (28 males and 134 females),

whose majors were linguistics (86%: Korean, English, French, Japanese, etc.), the humanities, the social sciences (14%: history, philosophy, economics, etc.). The age of the participants ranged from 17 to 32, from freshmen to seniors.

2) Instruments and Data Collection Procedures

The stimulus sentences were constructed according to the procedures adopted by Kim

and Kwon (2005, 2007). From each parallel stage on the PDS were first chosen critical structures, on which five critical sentences were composed. And in order to minimize the effects of the length of the sentences and words on the experiment, each critical sentence was made up of “7±2” thought groups according to Miller’s magic number (1956), using basic words and familiar proper nouns. These five critical sentences consisted of stimuli called the critical sentence set of forty-five items (see Appendix A). The critical sentences were converted into two kinds of Sentence Construction Tests (SCTs): the Sentence Conversion and the Sentence Combination Test. The SCTs were played in real time onto the Flash-Window Method (Full elaboration will be presented in the Testing system).

To demonstrate the relationship of rating scales on the CMS and scores on a standardized test, Grammar (GR) and Reading Comprehension (RC) tests of the TEPS (Test of English Proficiency developed by Seoul National University) were adopted. The RC test was used in an original form, but the GR test was slightly tailored for this research. Out of the 50 items in the test, the last 10 items were excluded because the types and the length of the items and the degree of difficulty were quite different from the other part of the test. As a result, the two tests were comprised of 40 multiple-choice items each.

The experiments were divided into two sections, each of which was carried out at a week interval. In the SCTs section, participants were given some explanations of the SCTs,

2 This study is additional research to Kim’s (2006c). According to the study, the cases at the lowest

(Level 0) and the highest (Level 9) levels of the CMS were too small; there were only five at Level 0 and only one case at Level 9. Consequently, failing to apply ANOVA, the study was not able to prove a mean difference among levels of the CMS. To compensate for such limitations, this study recruited 47 participants (assumed to be) with lower or higher levels of language proficiency. The research method of this study was, in fact, not different from that of Kim’s (2006c) except for recruiting 47 participants. This part was reproduced here for convenience’s sake of readers.


55

and then presented a set of practice items through a beam-projector to familiarize them with the tests. After the practice items had been run, they were presented with the randomized sentence set. The test took about 60 minutes. After the test was completed, the participants were asked to fill out a demographic information questionnaire and an excuse blank: “I couldn’t do my best in the test because of ‘_____’ (such as having a cold, mental distraction, or reluctance)”. The excuse blank was intended to give an opportunity to make any excuse for a participant who did not bias for best, and hence to exclude his or her data from the corpus. In the GR and RC tests section, participants were told not to select an answer from the choices by “wild guessing”. The GR test lasted 15minutes and the RC test 40minutes. After the two tests were finished, the participants were asked to fill out the questionnaires.

3) Data Analysis Procedures

Of all the data elicited from the participants through the SCTs, the followings were

excluded in order to minimize the effects of faulty data on the result of the analysis: (a) irrelevant or unintelligible (illegible) cases, (b) cases unanswered to more than three items in a row, (c) cases filled out an excuse blank. The other data were analyzed by the following procedures.

For the sake of the analysis, to each response of each participant to each item, a plus (+), a minus (-) or an equal (=) was given according to the degree of matching of a participant’s response against the answer.3 A “+” was given to a response when it was matched to the answer. A “-” was given when it was either not matched to the answer or incomplete. If a response was not completely matched to the answer, and interim or transitional critical structures were reconstructed from it by the IL analysis, a “=” was given to the corresponding parallel stage which the reconstructed structures belong to on the PDS. Only trivial or insignificant lexical errors with no relations to the parallel mechanisms were overlooked.

3 Consider the following examples, which were excerpted from a sample through the administration

of the SCT. s80. *Do you think did … go to … the show? s78. Where do you think they went to have a drink after the show? s16. *Do you think where they went to have a drink after show? According to the procedure, s80’s response was given a ‘-’ since it was not matched to the

answer, but s78’ was given a ‘+’ since it was matched. In the case of s16, though his or her response was not completely matched to the answer, the interim or transitional structure can be reconstructed from the response. Specifically, the response displyed that s16 was not able to construct long-distance questions, but he or she was able to produce indirect questions. Consequently, a ‘=’ was given to the reconstructed structure which belongs to 4X2 on the PDS.

Kim, Yong-Myeong

56

Criterion for Pass of each stage on the PDS was set as follows. If more than four responses of the five items of each stage match the answers, a positive (+) was marked on the implicational scale, which means that the learner was considered passing the stage in question (called the pass stage). If less than four match the answers, a negative (-) was marked on the scale, which means that the learner was considered not passing the stage (called the non-pass stage). On the other hand, in the GR and RC tests of the TEPS, one point was assigned to each item if it was a right answer, and then the number of right answers was tallied up.

2. Results4

To see whether the theoretical parallel developmental stages reflect the actual

developmental stages of the L2 learner, implicational scaling of the performance PDS was conducted. Of the responses elicited from the total 162 participants through SCTs, 12 cases in the unanalyzable categories were excluded from the analysis (see Data Analysis Procedures above). The others (150) were analyzed by the Implicational Scaling according to the Guttman Procedures (Hatch & Lazaraton, 1991, p. 212). The results of the analysis showed that the scalability of the implicational scale was .83, far greater than the minimum requirement of .60 for an acceptable scale (see Appendix B).

In order to see if there is a relationship between the rating scales on the CMS and the scores on the GR and RC tests of the TEPS, the Pearson Correlation Test and the Kruskal-Wallis Test were conducted (see the last two columns in Appendix B). The results of Pearson Correlation are illustrated in Table 2. The ratings on the CMS were highly correlated with the scores on the GR test (r=.91, p<.01), and on the RC test (r=.71, p<.01), which implies that the theoretical CMS holds statistical significance.

A more in-depth analysis, the Kruskal-Wallis Test was conducted to examine whether there is a rank difference among levels of the CMS. As shown in Table 3, the value for Kruskal-Wallis χ2 stood at 117.61 for GR and 72.74 for RC. In addition, as shown in Table 4, the order of levels of the CMS was squarely held in order of mean ranks of the GR and the RC test, which proves a statistical difference among the levels of the CMS.

4 Despite making up for 47 participants (assumed to be) with lower levels of proficiency, the results

of this research were not far different from those of Kim’s (2006c). The cases at two (the lowest and highest) levels were still too small, and homogeneity of variance test failed. Thus, failing to apply ANOVA, this study was not able to demonstrate whether there was a mean difference among levels of the CMS. This is a limitation to the statistical testing of the CMS. The results were reported here for the sake of evolving the Diagnosis and Remedy Systems (DRS).


57

TABLE 2 Correlations between CMS and GR/RC **p< .01

CMS GR RC CMS 1 GR .91** 1 RC .71** .67** 1

TABLE 3

Kruskal-Wallis Test Statistics (a, b) GR RC

Chi-Square 117.61 72.74 Df 9 9

Asymp. Sig .000 .000 a. Kruskal-Wallis Test b. Grouping Variable: CMS

TABLE 4

Kruskal-Wallis Test: Ranks Mean Rank PDS

Stages CMS

Levels N

(145) GR RC 1X 0 8 6.00 18.38 2X 1 15 22.27 41.90 3X 2 18 37.03 49.08 4X 3 20 55.58 54.58 1X2 4 22 71.25 63.02 2X2 5 17 94.06 93.09 3X2 6 17 100.18 97.53 4X2 7 11 120.32 99.95 5X2 8 14 128.71 120.75 KX3 9 3 143.33 139.17

V. DIAGNOSIS AND REMEDY SYSTEMS (DRS)

1. The Common Metric Scale (CMS) as an Acquisition-oriented Scale As mentioned above, the result of the Implicational Scaling showed that the performance

PDS by the L2 learners significantly corresponds to the theoretical PDS. The analyses of the Pearson Correlation and the Kruskal-Wallis test demonstrated that the theoretical CMS holds statistical significance. All of these imply that the theoretical CMS based on the PDS has a potential to be used as a real CMS in language testing. In sum, the CMS Model can establish a parallel interface to enhance the reciprocal developments between SLA research and language testing.

Kim, Yong-Myeong

58

The CMS Model can provide some implications for language pedagogy and assessment, and SLA research. More than anything else, as Kim (2006c) pointed out, the CMS based on the PDS can serve as an IL-sensitive scale for identifying IL learners’ current parallel developmental stages, diagnosing the sources of their errors, and predicting what rules or structures will be learnable or teachable next, thereby giving them a “timely remedy”. Thus, as Brindley (1998) suggested, the PDS and its executive counterpart, the CMS, can provide an alternative to the principal question of recent research on Form-focused Instruction: When and how can we provide what rules for IL learners? (Doughty & Williams, 1998; Robinson, 1996). Specifically, an answer to what rules to teach will be the critical structures or rules of subsequent stage on the PDS. An answer to when to teach them will be the subsequent stage of IL learners’ current stage. And an answer to how to teach them can be obtained by utilizing, for example, IL-sensitive tasks (Kim & Kwon, 2005) and task-based learning tasks (Brown, 2004; Pienemann, 1998), thereby giving a “tailored teaching” to IL learners. In conclusion, the CMS as an acquisition-oriented scale can provide not only “a tailored teaching for a timely remedy” for IL learners but also a practical, reliable measure for language testing and SLA research.

Given these possibilities, how can we implement the CMS into language teaching and testing, and SLA research? In other words, how can we provide “a tailored teaching for a timely remedy” to IL learners? This is a problem waiting to be resolved.

2. Diagnosis and Remedy Systems (DRS) on the CMS Model

As a possible alternative to this problem, this study proposes an executive system of the

CMS called the Diagnosis and Remedy Systems (DRS). As seen in Figure 2, the DRS comprises four subsystems: the CMS, the Testing, the Diagnosing, and the Teaching system. They are organically interlinked and cyclical. Put simply, the output of the Testing system is the input of the CMS and the Diagnosing system, whose output is, in turn, the input of the Teaching system. Like the CPU of a computer, the CMS system of the DRS functions as an acquisition-based criterion for developing and implementing the other three systems. The Testing system comprises various kinds of test tasks, and has a function to present these test tasks to IL learners. The Diagnosing system serves as a central function of the DRS. It makes an assessment of IL learners’ language proficiency on the basis of the test performances in terms of the CMS system. The Teaching system consists of various learning tasks. It has a function to provide tailored learning tasks for IL learners depending on the diagnoses and the remedies made by the Diagnosing system.


59

FIGURE 2 Diagnosis/Remedy Systems (DRS)

1) The CMS System

The CMS system of the DRS is the same as the CMS on the PDS (see Table 1). As

mentioned earlier, the CMS comprises three analytic scales, the Syntactic Scale, the Procedural Scale, and the Morphological Scale, thereby providing a “profile” of the components of language ability by differentiating among areas of relative strength and weakness. The CMS consists of a total of 10 scales from level 0 to 9. However, the number of scales can be tailored depending on the characteristics of test takers (e.g., beginners or advanced learners), the types of test tasks (e.g., oral interviews or multiple choice tests), and the purposes of testing (e.g., diagnostic tests or placement tests). The descriptions of each level of the CMS consist of parallel mechanisms, critical structures, and error patterns of each parallel stage of the PDS. The CMS is a pivotal system which controls the other three subsystems of the DRS.

2) The Testing System

The Testing system is comprised of two kinds of tests: IL-sensitive/tailored tests and

task/function-based tests. IL-sensitive/tailored tests, as the name suggests, are ones that are sensitive to or tailored for the developmental stages on the PDS. These tests include Sentence Construction Tests, IL-sensitive cloze tests, etc. Task/Function-based tests are a kind of elicitation procedures to be designed to elicit or collect interactional IL data. These

CMS Syste

Diagnosing

Testing

Teaching

DiagnosingSystem

TeachingSystem

CMS System

Testing System

Kim, Yong-Myeong

60

tests contain task/function-based tasks, a set of writing tasks, and interviews, etc. According to the specificity of test items or tasks5 (Omaggio, 2001), IL-sensitive tests are more convergent (more discrete-point-like), thus yields more accurate, but less interactive data, whereas task/function-based tests are more divergent (more open-ended-like), thus produces more interactive, but less accurate data. In this respect, each test of the two kinds has its own advantage and disadvantage, but has complementary relations to each other in eliciting IL samples from test takers. Thus, in order to get out of what Cronbach (1984) called the bandwidth fidelity dilemma,6 it is desirable to select at least two tests (one from IL-sensitive tests, the other from task/function-based tests) from this test battery for a given test situation, and administer them to a particular group of test takers.

(1) Interlanguage (IL)-sensitive/tailored Tests

Interlanguage (IL)-sensitive/tailored tests are developed in terms of the IL approach, along with the Testability and the Bounded-domain hypothesis. In accordance with the Testability hypothesis, test tasks or items are geared to each stage of the PDS along which IL learners pass through. In accordance with the Bounded-domain hypothesis, these task domains are also synchronized with the IL discourse domains on the continuum along which IL learners assume to create and extend. These tests include Sentence Construction Tests, Sentence Interpretation Tests, Recall Speaking Tests, and IL-sensitive cloze tests.

① Sentence Construction Tests (SCT) on the Flash-window Method

As mentioned under Research Method, Sentence Construction Tests (SCT) are developed as follows: from each parallel stage on the PDS are first chosen critical structures, on which critical sentences best reflecting the parallel mechanisms of each stage are composed. The critical sentences are converted into two kinds of Sentence Construction Tests (SCT): the Sentence Conversion and the Sentence Combination Test.

As seen in Figure 3, two kinds of SCTs are played in real time onto the Flash-window Method, which consist of four phases, that is, the Preparation, the Performance, the Completion, and the Relaxation Phase. In the Preparation Phase, the participants are asked to read a sentence or two or more sentences on the screen within a given time (160wpm). At the Performance Phase, in the case of the Sentence Conversion Tests, they are required to convert the sentence into another pattern (e.g., wh-questions) in their mind within a given

5 Omaggio (2001) represented the degree of specificity of the test items or tasks on a continuum

ranging from those that require the most convergent, or discrete-point, answers to those that require more global comprehension and/or divergent responses (pp. 399-400).

6 The bandwidth fidelity dilemma refers to the problem that broad, authentic samples of language use yield more generalizable but less accurate inferences about components of language ability, while narrow, less authentic samples yield more accurate but less generalizable inferences (recited from Bachman & Cohen, 1998, p. 22).


61

time (120wpm). In the Sentence Combination Tests, they are required to combine two or more sentences to form one complex sentence (e.g., indirect questions, relatives) in their mind within a given time. In the Completion Phase, they are asked to write down the converted or combined sentence they have just processed in their mind (60wpm). Finally, in the Relaxation Phase, music is played for about 10 seconds to help them to erase the previous item from their memories, to relieve their tension, and to prepare for the next item.

FIGURE 3

Sentence Construction Tests (SCT) on the Flash-window Method Window 1: [Preparation] [Preparing Instruction]: Please read/listen the following sentence carefully. (A cursor will blink for 3 seconds, and then the following sentence will appear.) John kissed Mary last night. (This window will disappear in 2 seconds.) Window 2: [Performance] [Performing Instruction]: Change ‘last night’ into ‘when’ in the following sentence, and make a wh-question in your mind. (This window will disappear in 5 seconds, and then the following sentence will appear.) John kissed Mary last night? ↓ When (This window will disappear in 3 seconds.) Window 3: [Completion] [Completing Instruction]: Type/Speak the question you thought of on the blank or the mike. ║________________________________ ● Mike (This window will disappear in 15 seconds.) Window 4: [Relaxation] Along with a buzzer signaling the end of writing, music is played for about 10 seconds, and then another buzzer signals the start of the next item. Since the SCT is implemented in real time, the participants have to process such

appropriate grammatical changes in parallel as constituent movements, word-order changes, and unification of grammatical features or morphemes to convert or combine sentences in real time. In this respect, the Flash-window Method can elicit more spontaneous, naturalistic responses from L2 learners than simple recall tests since it makes them rely less on their recall and more on actual processes in real time. Thus, the real-time method can provide a more sensitive measure for research into processability. The SCT can be implemented in four types according to the channel of the input (aural, visual), and of the expected response (oral, visual).

② Sentence Interpretation Tests (SIT)

As seen in Figure 4, Sentence Interpretation Tests (SIT) are ones in which a partial conversation is presented in aural or visual to test takers; they are then required to complete the conversation by translating a Korean sentence into an English one either

Kim, Yong-Myeong

62

orally or visually by modifying the given words and adding more words if necessary. The SIT can also be played onto the real-time Method. Seemingly, the SIT is similar to one type of the Test of Oral Proficiency (TOP) in English of the TEPS, but they are, in fact, different in that the stimulus sentences of the SIT are implicationally constructed in terms of the critical structures of each stage of the PDS.

FIGURE 4

Sentence Interpretation Tests (SIT) on the Flash-window Method Window 1: [Preparation] [Preparing Instruction]: Please read/listen the following conversation carefully. (A cursor will blink for 3 seconds, and then the following conversion will appear.)

Mary: John! Did you hear the news? John: What news? Mary: Jane has broken up with Mike! John: I don’t believe it! 제인이 왜 그와 헤어졌는지 아니? (In listening, the conversation will be presented without the script. In reading, it will be presented on the screen one by one at a rate of 200wpm and will then disappear). Window 2: [Performance] [Performing Instruction]: Translate the following Korean sentence into English, modifying the given words and adding more words as necessary. (This window will disappear in 5 seconds, and then the following sentence will appear.)

Mary: John! Did you hear the news? John: What news? Mary: Jane has broken up with Mike! John: I don’t believe it! 제인이 왜 그와 헤어졌는지 아니?

↓ (═ know/why/break up/?)

(In listening, the conversation is presented again.) (In reading, it will be presented on the screen one by one at a rate of 200wpm and will then disappear).

Window 3: [Completion] [Completing Instruction]: Type/Speak the translated sentence on the blank or the mike.

║________________________________ ● Mike (This window will disappear in 15 seconds.) Window 4: [Relaxation] Along with a buzzer signaling the end of writing, music is played for about 10 seconds and then another buzzer signals the start of the next item.

③ Recall Speaking Tests (RST)

Recall Speaking Tests (RST) are ones in which a short entire passage (say, 50 to 100 words) is first presented in aural; in the second presentation, test takers are required to repeat what they have just heard during the pause before the next thought group of words is presented. The RST is apparently similar to the TOP, but they are far different in that the passage of the RST are selected or edited according to the implicational scale of difficulty. In other words, the structures of the passages are tailored for the critical


63

structures of each stage of the PDS in accordance with the Testability hypothesis, and the contents/topics of the passages are sensitive to IL discourse domains in accordance with the Bounded-domain hypothesis.

④ IL-sensitive Cloze Tests

IL-sensitive cloze tests are constructed in a similar way to the “rational deletion” formats of the cloze tests. Selecting words for deletion is geared to the critical structures of each stage of the PDS, and the contents or topic of the reading passages are also sensitive to the IL discourse domains on the continuum along which IL learners assume to create and extend. IL-sensitive cloze tests can also be played onto the real-time method.

(2) Task/Function-based Tasks

Since IL-sensitive tests are designed to be sensitive to each stage of the PDS as discussed above, they can help us to make more accurate diagnoses and more timely remedies. Also, since they are pre-programmed by a computer, they can be conducted on a large scale with test takers. In addition, since they are administrated to test takers in a real time, they are likely to yield unmonitored, if not fully authentic, data. However, despite these advantages, IL-sensitive tests also have their counteractions. They are structured so narrowly that they are less likely to capture interaction, and hence to produce authentic and naturalistic data. To compensate for such limitations, we need thus task/function-based tasks that are designed to elicit or collect more authentic IL data through interactions of IL learners with each other or a researcher/tester. These tasks include various kinds of task/function-based tasks, a set of writing tasks, oral interviews, and others.

① Task-based Tasks

A variety of task-based tasks have been used in language teaching and testing, and SLA research (Brown, 2001, 2004; Ellis, 1989; Pienemann, 1998), but this study introduces two types of task-based tasks: picture-cued tasks and picture-cued information-gap tasks.

There are many forms of picture-cued tasks (Brown, 2001, 2004). One form is a picture-description task. Test takers are presented a somewhat more complex picture showing, for example, a family going on a summer vacation. They are then asked to describe the picture using given structures or words. Another is a picture-cued story-telling task. Test takers are presented a series of four to eight pictures depicting a sequence of an event or a story showing, for example, a man who is sitting on a wet-painted park bench. They are then required to silently study the pictures for a given time, and to tell the story that the pictures show for a given time.

There are also a variety of picture-cued information-gap tasks which, unlike picture-cued tasks, create a two-way communication task (Pienemann, 1998; Ellis, 1989). One type is a

Kim, Yong-Myeong

64

picture-difference task in which test takers in pairs are given one picture each of the “Spot the Difference” variety, and they are then asked to find out the differences by asking questions and making positive or negative responses to each other. This task can be used to elicit negations and question forms from test takers (Pienemann, 1998). Another is a picture-sequencing task. Test takers in dyads are each given part of a sequence of pictures, which are lettered so that they could be identified for discussion. The parts collectively make up a story or an event. They are asked to sequence the pictures with the story or the event by asking questions and making responses. An example of this task is a story that involved a man being assaulted by three different people on his way home from work (Pienemann, 1998). A third is a picture-cued information-gap task. Two pictures are cut up and are given to each of test takers in pairs. Each test taker is asked to describe his or her pictures without showing them to the partner so that they can jointly work out the story, creating a two-way communicative task (Ellis, 1989).

② Function-based Tasks

Function-based Tasks requires test takers in pairs to perform a given communicative function (e.g., request, invitation, etc.). Various types of function-based tasks can also be constructed, but this study devises a card-cued function task. Test takers in dyads are given one card each from a pair of function cards, each of which the information of a function to be performed is written on (e.g., the interlocutors, the contents, and the contexts). On the basis of the given information, they are asked to carry out the function together, taking turns speech acts of the sequence specific to the function. Take an example of this task, “invitation”. One of the pair of the function cards is for an inviter, whose card contains the following information: the invitee (e.g., the status, the relation to the inviter, etc.), the content of the invitation (e.g., inviting a party, going on a picnic, attending a conference, etc.), and the context of it (e.g., the time, the place, etc.). The other is for an invitee, whose card includes the following information: the inviter (e.g., the status, the relation to the invitee, etc.), and the content and context of the invitation. The test takers in pairs are given one each of the pair cards. They are required to silently study the card for a given time, and then to jointly perform the function, “invitation”, asking questions about it, and answering them.

③ Others

Other task-based procedures are a leaving-message task, a habitual-action, and an informal interview. A leaving-message task requires test takers to leave a message containing the given information (i.e., the purpose, the time, the place, and the things to do) on the answering machine within a given time (30 seconds) (see TOP manual). A habitual-action task involves a set of pictures depicting “a day in the life of someone such


65

as a librarian or a police officer”. Test takers are asked questions such as “what does a librarian do every day?” (Pienemann, 1998).

3) The Diagnosing System

The Diagnosing system plays an executive function in implementing the DRS. On the

basis of IL learners’ test performance, the system makes diagnoses of their language ability and pinpoints areas of the deficiencies in their IL grammar according to the CMS system. Thus, the Diagnosing system in concert with the CMS enables us to give an answer to the question of when and how to provide what rules for IL learners.

(1) Implementing the Diagnosing System into IL-sensitive Tests

The implementation of the Diagnosing system into IL-sensitive Tests first involves analyzing IL learners’ performance elicited from administrating the IL-sensitive tests, and determining what stage the IL structures produced by the learners belong to on the PDS. This is followed by an assessment of their IL proficiency and a diagnosis of their deficiency in terms of the CMS system. Consider the following examples, which were excerpted from a sample through the administration of the SCT.

(1) a. Do you want to know why Jane decided to divorce her husband last year? (s5) b. Do you like to know who my mother wants me to meet? (s16) c. Did you ask Mary what she handed to John during the history test? (s63) (2) a. *Do you think who Mary expects him to invite to the reception? (s5) b. *Do you think where they went to have a drink after the show? (s16) c. *Do you think why John parted from his sweetheart last week? (s63) This data tells us the learners’ language ability at the current time. What they are able to

do at this point is what Pienemann (1998) calls the cancel inversion as seen in (1). More specifically, they have already acquired that when a root question (e.g., Why did Jane decide to divorce her husband last year?) becomes an embedded clause, SAI (Subject/Aux Inversion) has to be cancelled or reinverted. Simply, the operator did should be moved back to the original position, and the operator has to be substituted by an appropriate morphological form (i.e., -ed) to affix the lexical verb decide. On the other hand, what they are not able to do is what we call the cleft-mechanism as seen in (2). Specifically, when the matrix verb is a non-opinion verb (e.g., know) which asks a ‘Yes or No answer’ of the interlocutors, there is no need to move the wh-phrase from the initial position of the embedded clause to the initial position of the matrix clause as seen in (1). Yet, as seen in

Kim, Yong-Myeong

66

(2), when the matrix verb is an opinion verb (e.g., think) which asks ‘specific opinions’, the wh-phrase should be moved out of the embedded clause into the initial position of the matrix, thereby the clause being cleft.

Now, let’s determine what stage the IL structures belong to on the PDS. On the basis of the above analysis, the learners are able to produce indirect questions governed by the cancel-mechanism which requires the procedural mechanism, “[+SCM([-IFM]/[-COM])]”, the syntactic mechanism, “Cancel-FocP”, and unificational mechanism, “Intersentential Unification ([CP[CP···]])” in each sequence of the PDS. Yet, they are not able to produce long-distance questions at this point because of not having yet acquired the cleft-mechanism which needs [-SCM([-IFM]/[-COM])], Re-FocP, and Intersentential Unification ([CP[CP···]]) in each sequence. According to the PDS, the IL structures governed by the cancel-mechanism (e.g., indirect questions) belong to the parallel stage 4X2 on the PDS (see Table 1 (p. 52)).

Finally, let’s assess their IL proficiency and diagnose their deficiencies. Since the learners have not yet arrived at the stage 5X2 on the PDS, their grammatical proficiency is thus rated “Level 7” in terms of the CMS (see Table 1). Also, since they are still deficient in applying the cleft-mechanism belonging to the stage 5X2, according to the Teachability hypothesis, they can be taught the critical structures or rules of the stage 5X2, that is, the various types of long-distance questions and cleft-sentences governed by the cleft-mechanism, with treating various errors caused by misapplying this mechanism.

In sum, IL-sensitive tests are designed to be sensitive to each stage of the PDS, thereby helping us to make more accurate diagnoses and more timely remedies. Yet, they are less likely to capture interaction, and hence to produce authentic data. Thus, as mentioned above, IL-sensitive tests need to be complemented by task/function-based tasks that are likely to yield more authentic, interactive data, thereby getting out of the bandwidth fidelity dilemma (Cronbach, 1984).

(2) Implementing the Diagnosing System into Task/Function-based Tasks

In task/function-based tasks, the Diagnosing system are implemented drawing on the Profile Analysis Procedure (Clahsen, 1985), or, more recently, the Rapid Profile (Pienemann & Mackey, 1993). The procedure involves first collecting samples of IL production through the administration of various kinds of task-based tasks. This is followed by an analysis of the IL learner’s output using the procedure which matches the critical structures and the error patterns produced against the parallel stages on the PDS, and by an assessment of his or her language proficiency in terms of the CMS system. It is beneficial to review an example study relying on this procedure.

Pienemann (1998) conducted task variation research, in which six ESL learners carried out six different task-based tasks (i.e., the habitual action, the story completion, the


67

informal interview, the picture sequencing, the picture difference, and the meet-partners). Three out of six tasks involved one participant in interacting with one researcher, and the other tasks involved two participants in interaction with each other, creating a two-way communication task. Data was collected, recorded in full, and transcribed from six informants carrying out six different tasks. The data was analyzed according to the acquisition map provided by the Processability theory (Pienemann, 1998), using the Profile Procedure. Put simply, each individual’s IL production in each task was analyzed using the procedure, and recorded on a kind of profiling checklist, on which the structures of each developmental stage are listed in the left-hand column and the six types of tasks are displayed to the right in the topside. Each cell of the resulting checklist thus refers to the specific structure as it occurs in one informant’s IL in one specific task.

Although Pieneman’s task-variation research was designed to verify his Steady hypothesis, the IL profiles provided by the Profile Procedure can also be utilized in implementing the Diagnosing system. In other words, the IL profiles gained from IL learners’ performing communicative tasks enable us to assess their IL proficiency and pinpoint their deficiencies according to the CMS system. Thus, task/function-based assessment coupled with the Profile Procedure can also provide an answer to the question of when and how to provide what rules for IL learners. Furthermore, as Brindley (1998) suggested, this task/function-based assessment with the Rapid Profile enables us to investigate the relationship between task/function fulfillment and grammatical processing skills more thoroughly.

4) The Teaching System

The Teaching system functions to provide IL learners with the learning tasks that are

tailored for their proficiency levels (i.e., developmental stages) and focused on their deficiencies identified by the Diagnosing system. The system consists of IL-sensitive tasks and task/function-based tasks.

IL-sensitive tasks are presented to IL learners in the same way as are IL-sensitive tests in the Testing system, but they contain hints, possible answer keys, and error patterns, to each item of the IL-sensitive tasks. Thus, IL learners carry out each task by referring to these things, and comparing their responses with the answer keys, thereby self-diagnosing their errors. Task/Function-based learning tasks are also presented to IL learners in the same way as are task/function-based tests in the Testing system, but they include vocabulary and structures necessary to perform a given task or function, and possible answers or the revised ones performed by peers, to the task. Thus, the learners work out these tasks by turning to the glossary, and possible, revised answers, to each task or function.

In conclusion, review some studies on the effects of such tailored teaching on language

Kim, Yong-Myeong

68

development. Pienemann (1985, 1987, 1989) provided IL learners with instruction tailored for the critical structures, “inversions”, of stage 4X for two weeks. According to the results of the analyses of pre- and post-test data gained from their spontaneous speech, the learners at stages 1X or 2X when the instruction began were no better at “inversions” when it ended. Conversely, the learners at stage 3X when the instruction began did apply “inversions” at higher frequency and in a wider range of linguistic context. Furthermore, they reached the stage 4X more rapidly than naturalistic acquires who seemed to take at least six months to move from stage 3X to 4X. Ellis (1989) also reported that the classroom learners reached higher levels of acquisition in a shorter period, and appeared to be more successful than the naturalistic learners.

V. CONCLUSION AND IMPLICATIONS On the basis of the Parallel Developmental Sequence (PDS) Model, Kim (2006c)

developed a Common Metric Scale (CMS) Model in terms of the IL approach. The IL approach incorporating the Testability and the Bounded-domain hypothesis converts the PDS into the CMS, thereby the scales, the scale descriptors, and the hierarchies of task difficulty being all in an implicational relationship. To retest the CMS, this study carried out additional research to Kim (2006c). The results of the study reconfirmed the validity of the CMS, thereby indicating that the CMS on the PDS can function as an IL-sensitive or acquisition-oriented scale for identifying IL learners’ current developmental stages. That is, both CMS and PDS can pinpoint the deficiencies in their IL grammar and predict what rules or structures will be learnable or teachable next. Thus, the CMS can provide a viable solution to the principal question of recent research on Form-focused Instruction: When and how can we provide what rules for IL learners? (Doughty & Williams, 1998; Robinson, 1996).

Given these possibilities, how can we implement the CMS into language teaching and testing, and SLA research? In other words, how can we determine when and how to teach which structures or rules for IL learners? To answer this question, this study has proposed the Diagnosis and Remedy Systems (DRS), an executive system of the CMS. The DRS consists of four subsystems (i.e., the CMS, the Testing, the Diagnosing, and the Teaching system), which are interlinked and cyclical. The DRS is implemented as follows: Suppose we are an English teacher in a secondary school and we intend to diagnose students’ language ability. We first enter the Testing system and have students take some tests provided in the system. Then, passing onto the CMS and the Diagnosing system, on the basis of results from these tests, we can assess their developmental stages and pinpoint their deficiencies in terms of the CMS system. This enables us not only to obtain their IL


69

profiles at this point but also to predict which structures will be teachable next. Finally, passing onto the Teaching system, on the basis of the diagnoses and the remedies, we can provide them with learning tasks that are tailored for their developmental stages and focused on their deficiencies, with employing teaching methods that are well suited to the characteristics of learning tasks to be presented, their learning styles, and teaching situations, etc. Or, we may place them into one of several levels according to their language proficiency. After tailored teaching done, we may ask them to take some tests again and analyze their performance to determine whether they progress to the next higher stages. If they pass the tests, they progress, and if not, they are given additional remedial work in the areas in which they are still deficient. This is the way the DRS is implemented into language teaching and testing, and the way it provides an answer for the question of how to provide a “timely remedy and tailored teaching” for IL learners.

The DRS can also provide some useful implications for language teaching and testing, and SLA research. More than anything else, the DRS can shed a new insight into the relationship between forms and functions. As discussed earlier, the DRS utilizes two different, but complementary elicitation procedures: IL-sensitive tasks and task/function-based tasks. This enables us to investigate the relationship between grammatical processing skills and task/function fulfillment more fully. For the sake of argument, suppose we are an SLA researcher. Following the DRS, we have participants take a few tests of IL-sensitive tasks in the Testing system. On the basis of their performance on the tests, we can learn at what stages they are on the PDS. Then, we ask the same participants to carry out some tasks of task/function-based tasks. On the basis of their profiling checklists gained from the analyses of their IL samples using the Rapid Profile, we can come to find out where they are now on the PDS. Pienemann (1998) got off here, but we can go on one step further. Specifically, we can also analyze and assess their overall IL samples in terms of the degree of their completion or fulfillment of the tasks or the functions. Or, if necessary, we can have native speakers rate the IL samples. This enables us to relate the degrees of the task/function fulfillment with the developmental stages of the PDS. Thus, we can infer which tasks or functions they can complete or fulfill when they are at what stage on the PDS. This should assist in clarifying the rather murky relationship between forms and functions, and in constituting a developmental sequence of functions as a counterpart of the PDS.

Kim, Yong-Myeong

70

REFERENCES

Bachman, L. F. (1989). Language testing-SLA research interface. Annual Review of Applied Linguistic, 9, 193-209.

Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford: Oxford University Press.

Bachman, L. F., & Cohen, A. D. (1998). Language testing-SLA research interface: An update. In L.F. Bachman & A. D. Cohen (Eds.), Interface between second language acquisition and language testing research (pp. 1-31). Cambridge: Cambridge University Press.

Bachman, L. F., & Palmer, A. S. (1983). Oral interview test of communicative proficiency in English. Los Angeles, CA: Photo-offset.

Bachman, L. F., & Palmer, A. S. (1996). Language testing in practice: Designing and developing useful language tests. Oxford: Oxford University Press.

Brindley, G. P. (1998). Describing language development? In L.F. Bachman & A. D. Cohen (Eds.), Interface between second language acquisition and language testing research (pp. 112-140). Cambridge: Cambridge University Press.

Brown, H. D. (2001). Teaching by principles: An interactive approach to language pedagogy (2nd ed.). White Plains, NY: Longman.

Brown, H. D. (2004). Language assessment: Principles and classroom practice. White Plains, NY: Longman.

Clahsen, H. (1984). The acquisition of German word order: A test case of cognitive approaches to L2 development. In R. Anderson (Ed.), Second language: A crosslinguistic perspective (pp. 219-242). Rowley, MA: Newbury House.

Clahsen, H. (1985). Profiling second language development: A procedure for assessing L2 proficiency. In K. Hyltenstam & M. Pienemann (Eds.), Modeling and assessing second language acquisition (pp. 283-325). Clevedon: Multilingual Matters.

Doughty, C., & Williams, J. (1998). Focus on form in classroom second language acquisition. New York: Cambridge University Press.

Douglas, D. (1998). Testing methods in context-based SL research. In L.F. Bachman & A. D. Cohen (Eds.), Interface between second language acquisition and language testing research (pp. 141-155). Cambridge: Cambridge University Press.

Douglas, D., & Selinker, L. (1985). Principles for language tests within the “discourse domains” theory of interlanguage: Research, test construction and interpretation. Language Testing, 2, 205-226.

Douglas, D., & Selinker, L. (1991). SPEAK and CHEMSPEAK: Measuring the English speaking ability of international teaching assistance in chemistry. Paper presented at the Language Testing Research Colloquium, Prinston, NJ, March.

Douglas, D., & Selinker, L. (1993). Performance on a general versus a field-specific test


71

of speaking proficiency by international teaching assistants. In D. Douglas & C. Chapelle (Eds.), A new decade of language testing research (pp. 235-254). Alexandria, VA: TESOL.

Ellis, R. (1989). Are classroom and naturalistic acquisition the same? Studies in Second Language Acquisition, 11(3), 305-328.

Ellis, R. (1994). The study of second language acquisition. Oxford: Oxford University Press.

Gregg, K. R. (1996). The logical and developmental problems of second language acquisition. In W. C. Ritchie & T. K. Bhatia (Eds.), Handbook of second language acquisition (pp. 49-81). San Diego: Academic Press.

Hatch, E., & Lazaraton, A. (1991). The research manual: Design and statistics for applied linguistics. New York: Newbury House.

Hudson, T. (1993). Nothing does not equal zero: Problems with applying developmental sequence findings to assessment and pedagogy. Studies in Second Language Acquisition, 15, 461-493.

Ingram, D. (1985). Assessing proficiency: An overview on some aspects of testing. In K. Hyltenstam & M. Pienemann (Eds.), Modeling and assessing second language acquisition (pp. 215-276). Clevedon: Multilingual Matters.

Kim, Y. M. (2006a). A unificational mechanism for morphological developmental sequence. Language Research, 42(1), 139-159.

Kim, Y. M. (2006b). An extended syntactic developmental sequence. Studies in Generative Grammar, 16(2), 405-423.

Kim, Y. M. (2006c). A common metric scale (CMS) on the parallel developmental sequence model. English Teaching, 61(4), 77-107.

Kim, Y. M., & Kwon, O. (2005). A parallel developmental sequence in SLA. Paper presented at the 5th International Symposium on Processability, Second Language Acquisition and Bilingualism. Melbourne, Australia.

Kim, Y. M., & Kwon, O. (2006). What mechanism operates on interlanguage development of Procedural Knowledge?: SPM Functions. English Teaching, 61(1), 111-134.

Kim, Y. M., & Kwon, O. (2007). A parallel developmental sequence in Second Language Acquisition. In F. Mansouri (Ed.), Second language acquisition research: Theory-construction and testing (pp. 239-275). Newcastle: Cambridge Scholars Press.

Meisel, J., Clahsen, H., & Pienemann, M. (1981). On determining developmental stages in natural second language acquisition. Studies in Second Language Acquisition, 3(2), 109-135.

Miller, G. (1956). The magical number seven, plus or minus two: Some limits on our capacity of processing information. Psychological Review, 63, 81-97.

Omaggio, H. C. (2001). Teaching language in context (3rd ed.). MA: Heinle & Heinle.

Kim, Yong-Myeong

72

Pienemann, M. (1985). Psychological constraints on the teachability of languages. Studies in Second Language Acquisition, 6, 186-214.

Pienemann, M. (1987). Determining the influence of instruction on L2 speech processing. Australian Review of Applied Linguistics 10, 83-113.

Pienemann, M. (1989). Is language teachable? Applied Linguistics 10, 52-79. Pienemann, M. (1998). Language processing and second language development: Processability

theory. Amsterdam: John Benjamins. Pienemann, M., & Johnston, M. (1987). Factors influencing the developmental of

language proficiency. In D. Nunan (Ed.), Applying second language acquisition research (pp. 45-141). Adelaide, Australia: National Curriculum Resource Center.

Pienemann, M., & Mackey, A. (1993). An empirical study of children’s ESL development and Rapid Profile. Sydney: NLLIA Language Acquisition Research Center, University of Sydney.

Robinson, P. (1996). Consciousness, rules, and instructed second language acquisition. New York: Peter Long.

Shohamy, E. (1998). Language testing and SLA. In L.F. Bachman & A. D. Cohen (Eds.), Interface between second language acquisition and language testing research (pp. 156-176). Cambridge: Cambridge University Press.

Vainikka, A., & Young-Scholten, M. (1998). The initial state in the L2 acquisition of phrase structure. In S. Flynn, G. Martohardjono & W. O’Neil, (Eds.), The generative study of second language acquisition (pp. 17-34). Mawha, NJ: Lawrence Erlbaum.

White, L. (1989). Universal Grammar and second language acquisition. Amsterdam: John Benjamins.

APPENDIX A The Critical Sentence Set

1. Parallel Developmental Stage 1X 2. Parallel Developmental Stage 2X 1) Do you wash the dishes everyday? 2) Who opened the windows on this cold day? 3) What happened at the party last night? 4) Who appeared on the concert last night? 5) Do you play tennis with your friends every weekend? 3. Parallel Developmental Stage 3X 1) Did you watch that news on TV last night?


73

2) Can you go camping with us this weekend? 3) Have you read an English novel recently? 4) Do you want to go there this weekend? 5) Would you like to have a snack after studying? 4. Parallel Developmental Stage 4X 1) When did you see her on the campus? 2) Where would you like to have a drink after the show? 3) What have you been writing since last week? 4) Who do you want to meet at the concert this Friday? 5) What did you give to Mary at the party last night? 5. Parallel Developmental Stage X2 5.1. Parallel Developmental Stage 1X2 1) Did you ask her to go for a drive last night? 2) What does he want you to do this summer vacation? 3) Who does she expect to hold a party next time? 4) Who does he want her to meet this Sunday? 5) I think that John is going with his sweetheart to the concert today. 5.2. Parallel Developmental Stage 2X2 1) Did you ask your daughter who liked her in her class? 2) The man who studies mechanics at college will investigate the car accident. 3) Do you know who taught English to this class last year? 4) Have you ever met the lady who was talking with John in the park last night? 5) Do you know who helped him to fix the broken TV? 5.3. Parallel Developmental Stage 3X2 1) John has proposed to the girl whom you longed to meet at college. 2) The girl whom you taught in college is going to marry my son next month. 3) The man whom Mary spoke to in the theater studies history at the college. 4) Have you ever met the man whom my father expects me to marry? 5) John employed a woman whom he thought to be honest at the job interview. 5.4. Parallel Developmental Stage 4X2 1) Does she know whom John came with to the party last night? 2) Do you want to know why Jane decided to divorce her husband last year? 3) Would you like to know who(m) my mother wants me to meet this Sunday? 4) Do you want to know who(m) Tom ordered to fire her after the audit? 5) Did you ask Mary what she handed John during history class? 5.5. Parallel Developmental Stage 5X2 1) Why do you think John parted from his sweetheart last week? 2) Where do you think they went to have drinks after the show?

Kim, Yong-Myeong

74

3) Who does she think John handed the note to during the English test? 4) Who do you think Mary expects him to invite to the reception? 5) Who do you suppose John wants to introduce her to the mayor? 6. Parallel Developmental Stage X3 1) Do you want to know who John thinks Mary gave the nice gift to at the party? 2) Who do you guess Mary thinks John loves at the college? 3) Do you know who Mary thinks John asked Alice to invite to the reception? 4) Who do you believe John thinks Mary wants to introduce Alice to the president? 5) Who do you think John believes Mary told Alice to fire after the audit?

APPENDIX B The Implicational Scaling Scalogram Parallel Developmental Stages CMS TEPS Subjects

kX3 5X2 4X2 3X2 2X2 1X2 4X 3X 2X 1X Level GR/RC s01 + + + + + + + + + + 9 36/38 j08 + + + + + + + + + + 9 33/27 j29 + + + + + + + + + + 9 32/22 s02 - + + + + + + + + + 8 27/23 j01 - + + + + + + + + + 8 33/19 j20 - + + + + + + + + + 8 32/17 j11 - + + + + + + + + + 8 31/19 j25 - + + + + + + + + + 8 30/22 s07 - + + + + + + + + + 8 31/25 s17 - + + + + + + + + + 8 30/22 j05 - + + + + + + + + + 8 30/20 j10 - + + + + + + + + + 8 29/24 s27 - + + + + + + + + + 8 26/10 s39 - + + + + + + + + + 8 21/28 s78 - + + + + + + + + + 8 x/x s90 - + + + + + + + + + 8 31/15 s103 - + + + + + + + + + 8 27/25 s105 - + + + + + + + + + 8 30/24 s28 - (+) + (-) + + + + + + 7 27/17 s95 - (+) + (-) + + + + + + 7 31/24 j34 - (+) + (-) + + + + + + 7 24/15 s03 - - + + + + + + + + 7 x/x s05 - - + + + + + + + + 7 27/24 s16 - - + + + + + + + + 7 25/15 s26 - - + + + + + + + + 7 28/12 s40 - - + + + + + + + + 7 25/13 s42 - - + + + + + + + + 7 25/16 s63 - - + + + + + + + + 7 30/22 s106 - - + + + + + + + + 7 26/11 s107 - - + + + + + + + + 7 28/23 j06 - (+) (+) (-) + (-) + + + + 6 x/x s56 - - (+) (-) + + + + + + 6 22/19


75

s65 - - (+) (-) + + + + + + 6 27/24 j09 - - (+) (-) + + + + + + 6 24/28 s73 - - (+) (-) + + + + + + 6 24/20 s76 - - (+) (-) + + + + + + 6 22/12 j42 - - (+) (-) + + + + + + 6 21/13 s88 - - (+) + (-) + + + + + 6 27/13 s41 - - (+) + + (-) + + + + 6 27/13 s54 - - (+) + + (-) + + + + 6 23/15 j04 - - - + + + + + + + 6 26/18 s18 - - - + + + + + + + 6 22/13 s25 - - - + + + + + + + 6 23/16 s29 - - - + + + + + + + 6 24/13 s45 - - - + + + + + + + 6 22/16 s77 - - - + + + + + + + 6 23/20 s81 - - - + + + + + + + 6 23/21 s98 - - - + + + + + + + 6 21/13 s15 - - (+) (+) (-) (-) + + + + 5 27/19 s96 - - - (+) (-) + + + + + 5 22/16 s06 - - - (+) + (-) + + + + 5 29/12 s94 - - - (+) + (-) + + + + 5 20/20 s04 - - - - + + + + + + 5 18/16 s12 - - - - + + + + + + 5 25/16 s14 - - - - + + + + + + 5 28/12 j15 - - - - + + + + + + 5 25/17 j07 - - - - + + + + + + 5 23/15 s22 - - - - + + + + + + 5 20/15 s46 - - - - + + + + + + 5 22/20 s48 - - - - + + + + + + 5 16/08 s62 - - - - + + + + + + 5 25/20 s67 - - - - + + + + + + 5 21/14 s85 - - - - + + + + + + 5 30/16 s86 - - - - + + + + + + 5 23/16 s99 - - - - + + + + + + 5 19/14 j13 - - - - (+) + (-) + + + 4 23/13 s08 - - - - (+) (-) + + + + 4 22/12 s09 - - - - (+) (-) + + + + 4 23/11 s24 - - - - (+) (-) + + + + 4 21/17 s37 - - - - (+) (-) + + + + 4 18/10 s49 - - - - (+) (-) + + + + 4 22/07 s55 - - - - (+) (-) + + + + 4 21/17 s68 - - - - (+) (-) + + + + 4 23/10 s11 - - - - - + + + + + 4 18/08 s20 - - - - - + + + + + 4 23/15 j03 - - - - - + + + + + 4 22/17 j16 - - - - - + + + + + 4 21/13 j23 - - - - - + + + + + 4 18/14 s30 - - - - - + + + + + 4 18/12 s35 - - - - - + + + + + 4 19/07 s51 - - - - - + + + + + 4 19/14 s52 - - - - - + + + + + 4 21/13 s69 - - - - - + + + + + 4 15/12 s89 - - - - - + + + + + 4 19/17 s104 - - - - - + + + + + 4 18/13

Kim, Yong-Myeong

76

j12 - - - - - + + + + + 4 15/11 s108 - - - - - + + + + + 4 19/12 s87 - - - - - (+) (-) + + + 3 17/05 j02 - - - - - - + + + + 3 25/19 j41 - - - - - - + + + + 3 21/11 s21 - - - - - - + + + + 3 18/10 s32 - - - - - - + + + + 3 20/14 s33 - - - - - - + + + + 3 20/07 s36 - - - - - - + + + + 3 19/06 j24 - - - - - - + + + + 3 17/04 s38 - - - - - - + + + + 3 15/10 s43 - - - - - - + + + + 3 15/12 s47 - - - - - - + + + + 3 12/12 s53 - - - - - - + + + + 3 16/11 s57 - - - - - - + + + + 3 20/18 s75 - - - - - - + + + + 3 16/15 s79 - - - - - - + + + + 3 20/13 j38 - - - - - - + + + + 3 19/16 j14 - - - - - - + + + + 3 17/12 s80 - - - - - - + + + + 3 16/18 s84 - - - - - - + + + + 3 19/07 s100 - - - - - - + + + + 3 16/11 j17 - - - - - - - + + + 2 22/15 s19 - - - - - - - + + + 2 16/10 s23 - - - - - - - + + + 2 15/08 s34 - - - - - - - + + + 2 17/12 j26 - - - - - - - + + + 2 16/11 s50 - - - - - - - + + + 2 15/09 s66 - - - - - - - + + + 2 14/09 s70 - - - - - - - + + + 2 15/14 s72 - - - - - - - + + + 2 17/11 s74 - - - - - - - + + + 2 13/15 s82 - - - - - - - + + + 2 19/12 j36 - - - - - - - + + + 2 13/09 j30 - - - - - - - + + + 2 11/14 s91 - - - - - - - + + + 2 12/08 s92 - - - - - - - + + + 2 14/09 s97 - - - - - - - + + + 2 13/15 j21 - - - - - - - + + + 2 12/13

s101 - - - - - - - + + + 2 18/07 j37 - - - - - - - - + + 1 13/14 j31 - - - - - - - - + + 1 13/09 j32 - - - - - - - - + + 1 12/16 j33 - - - - - - - - + + 1 12/12 j35 - - - - - - - - + + 1 13/09 j40 - - - - - - - - + + 1 19/05 j22 - - - - - - - - + + 1 12/14 s10 - - - - - - - - + + 1 10/09 s44 - - - - - - - - + + 1 18/07 s58 - - - - - - - - + + 1 x/x s59 - - - - - - - - + + 1 11/11 s64 - - - - - - - - + + 1 12/09 s71 - - - - - - - - + + 1 13/11


77

j19 - - - - - - - - + + 1 10/04 j39 - - - - - - - - + + 1 x/x s93 - - - - - - - - + + 1 10/10 s102 - - - - - - - - + + 1 10/13 s13 - - - - - - - - - + 0 10/11 j27 - - - - - - - - - + 0 11/07 s31 - - - - - - - - - + 0 11/04 s60 - - - - - - - - - + 0 08/05 j28 - - - - - - - - - + 0 08/09 s61 - - - - - - - - - + 0 08/09 j18 - - - - - - - - - + 0 07/08 s83 - - - - - - - - - + 0 07/08

Total 147 3 128 22 109 41 108 42 80 70 75 75 45 105 25 125 8 142 0 150 +: Pass Stage, -: Non-pass Stage, (+)/(-): Non-parallel Stage, x: Non-attendance at TEPS

Applicable level: all learners from children to adults Key words: SLA, language testing, parallel developmental sequence, common metric scale,

interlanguage approach, teachability hypothesis, IL-sensitive tests, Acquisition- oriented tests, testability hypothesis, bounded-domain hypothesis

Yong-Myeong Kim Hankuk University of Foreign Studies 201-ho, Dukmoom-building, 106-1, Dongbinggo-dong, Yongsan-gu Seoul 140-811, Korea Email: [email protected] Received in February, 2007 Reviewed in March, 2007 Revised version received in May, 2007

Diagnosis and Remedy Syst ems (DRS) for Teaching English...

Documents

Transcript of Diagnosis and Remedy Syst ems (DRS) for Teaching English...