Testing Four Skills in Japan - 英国（イギリス）の公的 … Can use familiar everyday...

日本における4技能試験 Testing Four Skills in Japan Anthony Green アントニー·グリーン

University of Bedfordshire ベッドフォードシャー大学

Centre for English Language Learning and Assessment

LANGUAGE ASSESSMENT IN CONTEXT

Japanese language policies and purposes

Copyright © Anthony Green

September 2014

University of Bedfordshire 2

English Next Graddol 2007

• With growth in regional trade. English spreading as lingua franca for

Asia (but native-speaker norms becoming less relevant).

• English becoming a component of basic skills education in many

countries: A2/B1 for service roles, C1 and above for professionals

• ¾ of all international travel is between non-English speaking

countries need for foreign language learning and English as a

lingua franca.


September 2014


Language Policy in Japan: The Challenge of Change Gottlieb 2012 p.74

• ‘It is simply not enough to rely on just English’ - regional languages

are becoming increasingly important

• Growth of internal community languages as migrant numbers grow

• Greater ease of learning Asian languages like Korean

MEXT 2012 White Paper on Education

Internationalization and computerization… are the most important

issues confronting education

Policy aims:

• promoting international understanding, improving international

exchange and cooperation in government, private, academic sectors

• nurturing people who flourish in internationalized society, global

leaders who can compete internationally

• promoting internationalization initiatives at universities and high

schools, social activities and international exchange


September 2014


• Training human resources able to adapt to the global environment is

one of the keys for Japan to achieve future economic growth

Liberal Democratic Party’s education reform panel


September 2014


Visitors to Japan source: JNTO

300 million domestic trips per year

Visitors to Japan since 1990 source: JNTO


September 2014


• Annually 18 million Japanese travel overseas: Korea, China,

Taiwan, USA (esp. Guam, Hawaii) among top destinations

Foreign Residents Registered in Japan 2011 Source: Ministry of Justice


September 2014


Exports of goods and services 2013 (% of GDP) Source: the World Bank


September 2014


Japan’s main trading partners


September 2014


Academic exchange (tertiary) 2012 Source: UNESCO

150,617 (3.9%) international

students from:

• China 96,592

• Korea 24,171

• Vietnam 4,047

Studying overseas

Japan 33,751 (0.9%)

(over half to USA)


September 2014


International students in:

• Korea 59,472 (1.8%)

• China: 88,979 (0.3%)

• Germany 206,986 (7.0%)

Studying overseas

• Korea 123,674 (3.7%)

• China 694,365 (2.1%)

• Germany 117,576 (4.0%)

LANGUAGES IN SCHOOL Japanese language education


September 2014


Purposes for foreign language study Grammar/translation and the classical tradition Liberal: culture and self-improvement

It has been abundantly proved that there is no more effective means of

strengthening the mind than by the earnest pursuit of this branch of learning

[of studying grammar and translating classic texts].

D‘Ooge 1909 Latin for Beginners

The average schoolboy, at the end of his school career has but a poor and

superficial smattering of Latin and Greek. He has spent ten years in not

learning two languages. That this can be deemed a fine mental training for this

poor average boy is to me inconceivable.

Postgate 1902 Are the Classics to Go?


September 2014


Purposes for foreign language study Harold Palmer (advisor to Japanese Ministry of Education in 1920s)

Utilitarian: commercial, social, interpersonal interaction

English should be presented to our pupils… as nothing other than a

means of communication. Let them… realize that a foreign language

is… not a collection of rules, formulas and symbols, but something as

natural and as simple as their mother tongue.

Teachers… generally miss this fundamental point. They look upon the

language as a code. or as a subject or as a literature in short as

something to be learned or studied, whereas a language is rather

something to be used.


September 2014


What might Japanese learners do through foreign languages? Common European Framework for Languages. Council of Europe (2001)


September 2014


B1 Can deal with most situations likely to arise whilst travelling in an area where the language is spoken. Can describe experiences and events, dreams, hopes & ambitions and briefly give reasons and explanations for opinions and plans.

A2 Can communicate in simple and routine tasks requiring a simple and direct exchange of information on familiar and routine matters. Can describe in simple terms aspects of his/her background, immediate environment and matters in areas of immediate need.

A1 Can use familiar everyday expressions and very basic phrases aimed at the satisfaction of needs of a concrete type. Can introduce him/herself and others and can ask and answer questions about personal details. Can interact in a simple way provided the other person talks slowly and clearly and is prepared to help.

• Branch campus of an American University established

• All academic courses taught in English

• Entrance examination includes a 15 minute oral interview in English

• Academic level of students judged broadly equivalent to students on

the US campus

• Ability to write in English judged better than that of the US students

Where?

When?


September 2014


Internationalisation of Higher Education The international campus model

Sapporo

1876

Source: Duke (2009) The History of Modern Japanese Education:

Constructing the National School System, 1872-1890


September 2014


The European Survey on Language Competences: language abilities of 15yr olds Source: SurveyLang 2012 and MEXT online

FSI estimates c.240 hours to achieve ACTFL Intermediate Low (able to satisfy basic survival needs) for group 1, c.720 hours or more for group 4 languages.


September 2014


Test and Score Data Summary for the TOEFL iBT® test 2013 Source: www.ets.org/toefl

IELTS Test taker performance 2012 Source: www.ielts.org


September 2014


EF Index (www.ef.co.uk/epi): Japan, Korea moderate; China low English proficiency

LANGUAGE ASSESSMENT IN CONTEXT

Japanese language education


September 2014



September 2014


• reduce class sizes

• improve in-service teacher education

• increase freedom in textbook selection

• emphasise proficiency in the spoken language as a goal

• develop students’ ability to ‘think in English’ without recourse to

translation

• use native speaker teachers more effectively

• reform university entrance examinations to

• use more authentic language

• include tests of speaking and listening skills

When proposed?

Key proposals for reforming English language education in Japan

2nd Convention of the IRET, held at Aoyama Kaikan in Tokyo, Nov.1925

The Writings of Harold E. Palmer An Overview. Richard C. Smith (1999)

Five Proposals and Specific Measures for Developing English Proficiency Commission on the Development of Foreign Language Proficiency (2011)


September 2014


Foreign language abilities needed = to actively communicate with people of

different countries and cultures, to make logical and reasoned explanation of

one's own views, and to convince and persuade others

1. English ability required of students: Assessment and verification of

attainment level

2. Promoting students’ awareness of necessity of English in global society,

stimulating motivation for English learning

3. Providing students with more opportunities to use English through effective

utilization of ALTs, ICT etc.

4. Reinforcement of English skills and instruction abilities of English teachers.

Strategic improvement of English education at the level of schools and

communities

5. Modification of university entrance exams toward global society

Five Proposals and Specific Measures for Developing English Proficiency Commission on the Development of Foreign Language Proficiency (2011)


September 2014


Survey results for public junior high schools and combined junior high

and high schools

Target: STEP 3 (CEFR A1) at junior high graduation

Current position:

• 9.5% certified plus16.0% uncertified: total 25.5%

• 30% of 3rd year JHS students could not follow English classes

Target: STEP Pre-1 (CEFR B2) for teachers

Current position:

• 27.7% of teachers achieved this level

• 高等学校学習指導要領 (平成21年３月)

• 目標 = 英語を通じて，積極的にコミュニケーションを図ろうとする態度を育成するとともに，聞くこと，話すこと，読むこと，書くことなどの基礎的な能力を養う。

• Teaching centred on listening and speaking in primary school

• Well balanced teaching of 4 skills in middle school

• English classes should be conducted principally in English in high

school


September 2014


Course of Study Mandates balanced approach to four skills

MEASURING LEARNING MEASURING PROFICIENCY

Do we need tests of spoken language skills?


September 2014



Spoken and written language What is speaking?

Socially situated, often reciprocal oral interaction with another or others

It requires…

• lexico-grammatical knowledge

• phonological control

• knowledge of discourse

• pragmatic awareness

• strategic competence

These are different from written language


September 2014

Understanding speech Spoken sounds vary. A lot.


September 2014


Written language has standard spelling – no spoken equivalent.

Spoken language is “chunked” into phrases and clauses: can be difficult

to separate out the individual words

Sounds of words change according to context:

I met him /t/ I met you /t∫/

Sounds of words depend on gender, age, shape and size of vocal tract

of speaker.

Fast speech is harder for learners to follow

Those familiar with text book speech are often unable to keep up with a

more natural pace


What is proficient speech?

- Shorter and less complex units of speech

- More errors per unit of speech

- Less and more limited use of cohesive markers

- Limited to use of common words

- Pauses linked to language search

- Pauses within grammatical constituents

- More silent pause time

- Shorter ‘runs’ of speech between noticeable pauses

- Speed of delivery noticeably below native speaker rate

+ Longer and more complex (e.g. more embedded) units of speech

+ Fewer errors per unit of speech

+ More, and more varied, use of cohesive markers

+ Use of more sophisticated and idiomatic vocabulary

+ Pauses linked to content search

+ Pauses between grammatical constituents

+ Less silent pause time

+ Longer runs of speech between noticeable pauses

+ Speed of delivery not noticeably below native speaker rate

(Tonkyn and Wilson, 2004)


September 2014


What is proficient speech? The social dimension

Conversational rules and moves

• Opening and closing

• Turn taking

• Holding the floor

• Interrupting

Register

• Status and roles

• Degrees of formality

• Politeness

Communication strategies

• Paraphrase

• Avoidance

• Seeking help

• Asking for repetition

Language functions

• Macro-functions

• transmitting information

• social interaction

• persuading others

• learning and thinking

• Micro-functions

• inviting

• apologizing

• warning

Speech acts

• Locutionary

• Illocutionary

• Perlocutionary


September 2014


How should we assess Speaking?

The best way to test people’s speaking ability is to get them to speak

(Young & He, 1998)


September 2014

Getting them to speak An English lecturer blogs about a speaking test for university entrance…

After lunch, the interviews begin. It is obvious who have been drilled by their

high school teachers. They have been given different messages about eye

contact. Some, generally the boys, blink, swallow and stare into the distance

refusing to look us in the face. The girls glance and then look away shyly.

Next, the students read a short passage on music in English and answer two

questions. It immediately becomes obvious they cannot understand the

passage at all, and can’t answer either of the questions. I try to hit on a

question for which they have a prepared answer: “My favourite singer” and

“Why I like animals” (“Because they are cute”) hit their targets.

At the end of the day, the Japanese lecturer turns to me and says, “They don’t

speak English at all, do they?” I agree the standard is appallingly low for

students who have spent six years studying English in compulsory education.

“It is our education system,” admits the lecturer.


September 2014



Tests of spoken production and interaction

1. Elicit samples of spoken language under realistic

conditions

2. Simulate behaviour in the real world

Not like multiple-choice ‘objective’ tests

Not reading aloud, memorised dialogues

3. Observed and evaluated by an agreed judging

process


September 2014


How should we get people to speak?


September 2014


Why assess Speaking?

Practicality

Speaking tests are a challenge logistically

Speaking tests are often expensive

Reliability

Speaking tests can be difficult to score reliably

Validity

Speaking is an important element of language ability

Spoken language is not equivalent to written language

Impact

Testing Speaking in important tests may encourage teaching of

Speaking


September 2014

WASHBACK

High schools and entrance examinations


September 2014


Maximum Overlap: Studying for the test is to develop skills for the target domain Limited Overlap: Studying for the test develops skills for test and target domain No Overlap: Studying for the test does not develop skills for the target domain

Target Task Characteristics Test Task Characteristics

A simple model of washback: Overlap


September 2014


Washback variability

Individuals are differently affected by tests…

• Tests matter more to some people than to others

• Some people are better informed about tests than others

• Tests are more difficult for some people than for others

• Some people have more faith in tests than others

• Adopters

• Adapters ?

• Resisters

University of Bedfordshire 36 Copyright © Anthony Green

September 2014

A moer complex model of washback Green 2013 Exploring Language Assessment and Testing


September 2014


Washback by design

Key issues

• Target skills – what are the long-term learning goals?

• Test design features

• Test stakes and importance

• Participants’ knowledge and resources

Evidence

Processes: asking, watching

Products: will students really become better language

users if the entrance exams are reformed?

University of Bedfordshire 38 Copyright © Anthony Green

September 2014

Overlap

• How far does the design of university entrance tests

cover the language skills that students need for academic

study in English at Japanese universities?


Academic English Skills University

Entrance Test


September 2014

Method:

Anonymous questionnaire survey

Yes/No and Likert scale (1 to 6) format

Period:

September to October 2012

Stakeholders:

University teachers of English (n=19)

3rd year HS students at (75) schools affiliated with the university

(n=3868)

English teachers of the high schools (n=423)

TEAP Impact Study Phase 1

Method ②

1. Perceptions of university study and the English language skills

required at university

2. Current practices in Japanese high school English classes

3. The washback effects of the current university entrance exams

4. Effects of changes to the content of university entrance exams

5. Perceptions of the innovative features of TEAP

The TEAP Impact Study

Language skills required for university study High School students’ perspective 6 = definitely

42 Copyright © Anthony Green

September 2014

University of Bedfordshire

The Teaching of English in Japan

Koike 1978 p.iv

In general, reading comprehension skills are fostered, while hearing

and speaking skills are less stressed… Even today, because

university entrance examinations require English reading

comprehension, high schools often neglect the development of

hearing and speaking skills.


September 2014


Most time in HS English class devoted to…

あなたの学校の英語の授業において、次の4つの技能のうち最も時間がかけられているものはどれですか読む聞く話す書く

44

High school teachers n.332 High school students n. 2,784

75 schools

Reading

Listening

Speaking

Writing


September 2014

University of Bedfordshire

Result ④ Effects of changes to the content of university entrance exams 6 = definitely

高校生高校教師大学先生

a speaking test in university English

entrance exams would lead to an

increase in speaking activities in HS

English classes 5.00 5.19 4.58

a writing test in university English

entrance exams would lead to an

increase in writing activities in HS

English classes 5.06 5.24 4.74

Attitudes towards changes to the content of university entrance exams 6 = definitely 高校生高校教師大学先生

Testing only reading is sufficient for

university English entrance exams 3.22 1.93 1.58

entrance exams should include a

listening test 4.10 5.05 5.63

…should include a speaking test 3.51 4.16 4.26

…should include a writing test 4.15 4.85 5.05


September 2014


IMPACTS OF FOUR SKILLS TESTING

Will changing tests improve learning?


September 2014


Modification of university entrance exams The National English Ability Test (NEAT) NEAT to have positive impact on English learning

Lyle F. Bachman The Korea Times April 10 2013

Given the fact that it is based on the Korean national English curriculum, and covers

all four skills ― listening, speaking, reading, and writing, NEAT is likely to provide

positive washback on English teaching in Korea, which I would see as a very strong

beneficial consequence. It is to KICE's credit that they have designed this intended

beneficial consequence into the test from the very beginning of its development.


September 2014


Korean HS curriculum

NEAT

With speaking

and listening

KSAT

No speaking or listening

Washback direction

Modification of university entrance exams The National English Ability Test (NEAT) Billions blown on NEAT

Jung Min-ho, Jung Sung-eun The Korea Times May 21 2014

Teachers, students and parents have criticized the Ministry of Education for wasting

taxpayer's money on the development of a new college entrance English language test

that has now been scrapped.

The aim of introducing NEAT was to help schoolchildren improve their English

proficiency. But,,, critics point out that the NEAT formula was doomed to fail from the

start because few teachers were prepared to teach speaking and writing.


September 2014


Participant characteristics and values

- understanding of test demands

- resources to meet test demands

Washback variability

Four Skills Testing: Five Modest Proposals


September 2014


1. Make local tests that meet international standards - design tests

that reflect the curriculum and the language knowledge, skills and

abilities that Japanese language learners need

2. Collect evidence to inform decision making

3. Make continuous improvements to tests in line with best practice,

but avoid being too radical too soon

4. Involve teachers and other stakeholders in reform - don’t

overestimate the power of tests alone to improve education

5. Support in service training - do build assessment literacy into

teacher training

Testing Four Skills in Japan - 英国（イギリス）の公的 … Can use familiar everyday...

Documents

Transcript of Testing Four Skills in Japan - 英国（イギリス）の公的 … Can use familiar everyday...