Bifie – Bundesinstitut für Bildungsforschung, Innovation & Entwicklung des österreichischen...

bifie – Bundesinstitut für Bildungsforschung, Innovation & Entwicklungdes österreichischen Schulwesens

External Assessment in Austria

. © Gabi FRIEDL- LUCYSHYN (2009)


Educational context:


Standards on primary and lower secondary level

describe essential competences a student is supposed to sustainably have acquired in certain subjects at decisive stages in his/her school career

formulate the desired outcome of the learning process

aim at evidence-based quality assurance on classroom, school and system level


3 Functions of Standards

1. Accountability for outcome of teaching and learning

2. Individual coaching of students (diagnostic feedback)

3. System monitoring


Clear focus on teaching, not on testing No influence on assessment of individual student

Aims: gauging the degree to which expected goals are

achieved data for quality assurance teachers must assure the acquisition of defined

competences and coach – as best they can - students individually by continually comparing defined goals with actual performance

Support: diagnostic tools


The Assessments (regular monitoring) Psychometrically validated tests Subjects, test cycles Structure of feedback Design of feedack:

Context variables (questionnaires) allow for fair feedback (performance in relation to classes and schools in comparable contexts)

Feedback given in as differentiated a way as possible Trained experts can be called upon to support schools in interpreting data

and taking action


Challenges


School leaving exam


Context Basis: BIFIE has the mandate to design, develop, and

implement scientifically validated, competence-oriented written exams for all types of upper secondary schools

Subjects: Language of instruction, Modern Languages (CEF Levels B1 or B2), Latin/Greek, Maths

Aim: Validity of exam and assessment of competences

Key data: nationwide introduction 2013/14 (general secondary), 2014/15 (vocational colleges on secondary level)

© FRIEDL-LUCYSHYN 2009


Architecture: Basic concept:

Combination of individual student and school profile as well as standardisation in essential areas

3 phases: „pre-scientific“ paper (internal) 3 or 4 written examinations (external) 2 or 3 orals (internal)

Certificate: Exam results in each part + study programme of previous

4 years (subjects, number of hours, grades)



Written exam (external)

No external correction, but keys and justifications, assessment criteria und cut scores are binding

Various models of quality assurance under discussion

Online help desk and hotlines


Piloting in all subjects until 2013/14

Voluntary participation Modern languages: since 2008 (reading, listening,

language in use), 300 out of 350 schools on board; writing in 2011/12

Maths, language of instruction, Latin, Greek: from 2010/11


Test development by Austrian universities

Cooperation with/review by (inter)national tertiary institutions

Active and essential role of teachers as practitioners at all stages

Integration of learners (as target group) in validation process

Open-ended and close-ended item formats


Development of tasks, items and test papers – general procedure


How tests come into being(foreign languages) Designed and developed by teachers and methodologists (university

academics): Regional, professional and institutional diversity Training

Validation: Trialing of tasks and items (+ questionaires)

Processing, analysis and interpretation of stats (test-psychologists + psychometricians) Item-moderation Trialing ….

Standard setting (stakeholders)



How tests come into being (2)

Aim of validation process: Appropriateness of level of difficulty for student population Validity and quality of items Clarity of instructions Motivation

University of Innsbruck Cooperation/Review:

University of Lancaster (Prof. Charles Alderson, Dr. Rita Green): Training of item writers, item moderation, standard setting



The test paper in the foreign languages


Skills Listening Reading Language in Use

Number of tasks

4 -5 tasks 4 - 5 tasks 4 – 5 tasks

Test formate

•Multiple Choice•Multiple Matching•True/False•Short answer questions

•Multiple Choice•Multiple Matching•True/False/Not Given •Short answer questions

•Editing/error correction•Multiple choice cloze •Word formation


The test paper in the foreign languages Writing:

Part 1: argumentative text (essay); compulsory

Part 2: 2 tasks, student chooses one

Text types: letter, report, article, composition

Writings skills: argumentative, discursive, descriptive and narrative

Assessment scales B1/B2

Recommendations for orals


Thank you!

www.bifie.at



Results compared to internal tests in final year 2009

listening

11%7%

30%52%

comparable

better

worse

no comment


listening

13%4%

61%22%comparable

better

worse

no comment

Evaluation of exam results 1 (N = 444)


Evaluation of exam results 2


reading

2%

9%17%

72%comparable

better

worse

no comment


reading

15%11% 72%

2%

comparable

better

worse

no comment


Evaluation 3 (N-444)

020406080

percentage of teachers reporting changes in the amount of instruction

time caused by introduction of the new final assessment

hotline

7,9%

3,0%

19,8%

9,9%

59,4%listening

reading

language in use

reading, hearing

reading, hearing, LiU

Bifie – Bundesinstitut für Bildungsforschung, Innovation & Entwicklung des österreichischen...

Documents

Transcript of Bifie – Bundesinstitut für Bildungsforschung, Innovation & Entwicklung des österreichischen...