Academic Writing and Feedback Results from a Research Study

Olga Dysthe Department of Education, University of Bergen, Norway

eLC Research Update Meeting Tuesday, 22nd October 2013

Overview of lecture

•  Introduction: multidisciplinary e-feedback research

•  A qualitative research study at meso & micro level: “ Feedback practices and study design in a 1. year Master of Education programme “

•  Implications of findings •  Feedback between researchers

–  Authoritative and dialogic models of feedback –  Examples from my experiences of co-authorship

E-feedback research exists in several fields

•  Writing research (response) –  Rationale: Feedback is a crucial part of the writing process

•  Assessment research (formative assessment, ass. for learning) –  Feedback involves assessing student writing

•  Education (learning processes) –  i.e. John Hattie’s metatudies: feedback important for learning

•  Linguistics –  i.e. e-feedback as a specific text genre

•  Second language research L2 –  Students particularly dependent on teacher feedback

•  Technology enhanced learning

A case study of feedback practices & study design “What’s the purpose of f.b. when revision is not expected?”

Research questions: How is student learning afforded or constrained by the writing and feedback practices in on-line courses in 1. y Master of Ed.? Qualitity of feedback

–  What are characteristic traits of the teachers’ feedback? Use of feedback

–  To what extent and how is the feedback information being used by students?

Learning from feedback –  How does the study design influence student learning? –  Other factors?

Contextualization

•  Evaluation of our study programmes: •  2 year campus-based (hybrid) Master in Education program

–  Year one: 4 courses 1.  Educational Philosophy (on-line) – teacher feedback 2.  Knowledge and Communication 3.  Theory of Science & Methods (on-line) – teacher feedback 4.  Project plan seminar

–  Year two: research & master thesis

•  4-year web-based M.Ed.

Materials and methods Qualitative study: document analysis, text analysis + interviews

•  Data sources I. - Study plans and programme committee decisions

–  Assignments & teacher guidelines for each assignm.

II. - Student written texts (24) –  Feedback texts from 5 teachers

III. - Focus group interviews with 4 students - Written course evaluations

•  Model for analysis of feedback texts: Hattie & Timperley (2007)

Theoretical framework

•  Sociocultural perspective

–  The importance of contexts for giving and receiving feedback (meso level, i.e. study design)

–  The importance of dialogue for learning (Bakhtin 1986)

•  Cognitive perspectives –  Model of feedback to enhance learning (micro level)

(Hattie & Timperley, 2007) –  Self-regulation (Nichols & Macfarlane-Dick, 2006)

.

Gap How ??? Levels

Feedback is evidence about:

•  Where am I going? (Goals) FEED UP •  How am I going? (Performance) FEED BACK •  Where to next? (Specific advice) FEED FORWARD

Hattie & Timperley (2007)

Findings from analysis of feedback texts Qualitity of feedback

Finding 1: Lack of clear goals both in the assignments and feedback

Examples from assignments 1.  ”Discuss how different forms of knowledge are represented in

formal education” 2.  ”Use the literature as support and argue for… 3.  ”Use your own experiences as a point of departure for a text about

the value of education. Use literature as support”

Genre expectations? Writing-to-learn-to-write? (academic text) Writing-to-learn assignments?

Example of comment relating to the goal of writing •  ”[The goal of writing] is to develop analytical lenses, not to find the

correct answer, but to be more reflective regarding your own and others’ points of view” (group feedback/B)

Finding 2: Feedback primarily on Task level – few comments on Process level & Self-regulation

•  Task feedback: how well is the assignment performed ”Too much repetition of literature and too little understanding” ”Be clear about the difference between analysis and interpretation.”

•  Process feedback: i.e. writing process, strategies for error

detection or problem solving ”Let the first draft rest and revise before handing it in” ” Read other students’ papers to get other perspectives than your own”

•  Self-regulation feedback: i.e. self-monitoring, self-assessment, self-discipline

”Trust your own voice and your own opinions and reproduce less of other people’s words”

Discussion: Is task level feedback less transferable than feedback on process and self-regulation?

•  ”[Task feedback] is more powerful when it is about faulty interpretations, not lack of information. One of the problems with feedback at the task level is that it often does not generalize to other tasks” (H & T, p 91) Confirmed by findings in student interviews

•  Is the reason for lack of process feedback that HE

teachers in the disciplines don´t have knowledge of writing process and learning strategies?

•  Nicol & Macfarlane Dick (2006): Self-regulation main goal

of feedback: involving students in self & peer evaluation

When revision not expected – is another type of feedback needed?

Finding 3: Mostly implicit feed-forward

Implicit: ”Your conclusion is ’common sense’ ” ”Who talks in the text,- you or the literature?”

”Where is your argument” Do students recognize what to do? Do they know how to? (cf. Chanock, Hounsell )

Explicit: •  ”formulate your own controling questions” •  ”present your own viewpoint” •  ”write in your own voice”

Does feed-forward mean ’how to’ or just ’what to’ do?

Finding 4:

Clear, personal, confrontational, dialogic feedback and feed forward functions best for students

Examples of dialogic & personal fb: •  ”I think this is unnecessary, - but you may disagree?” •  ”In my view … What do you think?” •  ”I am not convinced by your argument here”

Discussion: The importance of •  a dialogic feedback style of teacher (micro level) •  dialogic spaces in the study design (meso level)

What did the students do with feedback? Use of feedback

Finding 5: Not much? (difficult to determine) • It did not improve the draft as they were not asked to revise • Little evidence of transfer to next assignments

Finding 6: The study design counteracted student learning from feedback

* The study design lacked •  clear goals for student writing •  clear purpose for assignments •  provision of dialogic spaces around feedback

•  The study design did not require revision •  Tacit expectation: transfer of learning from feedback would

take place to next assigment

Practical implications of study: What has changed in our study program?

•  Goals –  More presise goals for writing assignments

•  Distinguish more clearly between ’writing-to-learn’ assignments and writing in academic genres

•  Criteria •  Revision

•  Fewer assignments - revision expected •  Revision competence – part of writing courses

•  Feedback in hybrid courses (campus + net) –  More direct teaching –  Seminar for teachers about quality feedback –  Combination of written and oral feedback –  Reintroduction of peer feedback in short courses

Additional finding at micro level: Different fb styles

IMPERSONAL STYLE of FEEDBACK 1. Little explicit (genuine) praise -

only ´ritual´ praise

2. Implicit critique

3. Extensive use of’mirroring’ (telling what student has done, but few examples to learn from)

4. Extensive use of passive, inpersonal pronouns and lack of use of ”I”. ”It is necessary to ”,One should always” ”Descriptions are important…

PERSONAL & DIALOGIC STYLE 1. Explicit praise in marginal

comments ”This claim is well supported”

2. Explicit criticism, but explained

”You use theory normatively” + example

3. Examples from students’ texts

used to illustrate a point + challenge

4. Use of personal statements and

examples from own research

Some overarching issues raised by the study

Students’ use of feedback •  How to make sure that students benefit from the great

amount of work teachers put into giving feedback? –  Ask for revision - Teach revision –  Be clear about expectations

Study design •  What are pros and cons of demanding student revisions?

–  Revision means fewer writing assignments •  Is transfer from one assignment to another to be expected?

–  Does it require a different type of feedback? How to develop a writing culture with peer response in e-

courses? How to develop response competence?

Feedback between researchers

•  Our whole academic publishing system is dependent on peer review

•  Giving and utilizing feedback is therefore two crucial competences that all academics need

•  How to foster those competences at institutional level? –  Build in peer feedback in master & PhD courses –  Organise online writing groups –  Encourage co-authoring of conference papers & articles

How to give productive feedback?

Authoritative & Dialogic Models of feedback •  Authoritative model - the traditional perspective on fb:

–  teacher/tutor/peer reviewer as representative of the authoritative truth

–  Revision as correction –  Obviously not a model for feedback among researchers

•  Dialogic model – feedb as joint activity (M. Bakhtin)

Two aspects of Bakhtin´s dialogism

•  Meaning is created in the meeting point between different voices –  An alternative communication model to Sender-Receiver model

•  The productiveness of tension, disagreement, conflicting views

–  (cf. Piaget: socio-cognitive conflicts & Vygotsky´ intersubjectivity)

–  Eugene Matusov: ”I respectfully disagree”

5 principles of good feedback

•  Give specific, not ritual praise

•  Be clear about the overarching goal of the text

•  Remember ”Fokus – Form – Formalia”

•  Be challenging – but in a respectful manner

•  Invite dialogue: ”In my opinion …. What do you think ?”

Examples from my experience as co-author

The feedback process was very different •  With a close partner(s) at my own university

–  Genuine dialogues, even provocations, based on trust •  With junior partner at a different university in Norway

–  Clarify goals and roles; for example expertise in English •  With a senior partner at a university abroad

–  Most problematic –  My strategy: reformulations: ”Is this what we want to say?”

•  With junior partners at different institutions abroad –  ”walk the other mile” to give them ownership to text

How to ask for constructive feedback? How to give constructive feedback without offending?

Thanks for listening!

Stryn, Nordfjord, Norway

Literature 1:

•  Bakhtin, M. M. (1981). The dialogic imagination: Four essays by M. M. Bakhtin. Translated by C. Emerson and M. Holquist. Austin, TX: University of Texas Press.

•  Crisp, B. R. (2007) Is it worth the effort? How feedback influences students’ subsequent submission of assessable work. Assessment & Evaluation in Higher Education. Vol 32, No 5, 571-581.

•  Duncan, N. (2007) ‘Feed-forward’: improving students’ use of tutor’s comments. Assesment & Evaluation in Higher Education, Vol. 32, No.3, 271-283.

•  Dysthe, O. (2002) Professors as mediators of academic text culture. An interview study with advisors and master degree students in three disciplines in a Norwegian university. Written Communication, Sage publications: 2002, 19/4, 485-536.

•  Dysthe, O. (2009) "What is the Purpose of Feedback when Revision is not Expected?" A Case Study of Feedback Quality and Study Design in a First Year Master's Programme. Journal of Academic Writing 2011; Vol. 1.(1) s. 126-134

•  Dysthe, O. (2009) What factors influence the improvement of academic writing practices? A study of reform of undergraduate writing in Norwegian higher education. In C. Bazerman, et al. (Eds.), Traditions of writing research. New York, NY: Routledge/Taylor & Francis,

•  Dysthe, O., Lillejord, S., Vines, A., Wasson, B. (2011) Productive E-feedback in higher education – Some critical issues. Ludvigsen, S., Lund, A., Rasmussen, I. & Säljö, R Learning across sites: New tools, infrastructures and practices. Oxford, UK: Pergamon Press, pp. 243-258.

•  Dysthe, O. (2012) Multivoiced classrooms in higher education academic writing. In M. Castelló & C. Donahue (Eds.) University writing: Selves and Texts in Academic Societes. Emerald Group Publishers.

• 

Literature 2 •  Dysthe, O.& Lillejord, S. (2012) From Humboldt to Bologna: Using peer-feedback to foster productive

writing practices among online master students. International Journal of Web Based Communities 2012 ;Volum 4.(8) s. 471-485

•  Hattie, J. & Timperley, H. (2007) The power of feedback. Review of educational research 77(1): 81-112. •  Hounsell, D, et al. (2008) The quality of guidance and feedback to students. Higher Education Research

& Development, Vol 27, No 1, pp. 55-67. •  Kluger, A. N. & DeNisi, A (1996) The effect of feedback interventions on performance: A historical

review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119 (2), pp. 254-284.

•  Orsmond, P. et al. (2005) Biology students’utilization of tutor’s formative feedback: a qualitative interview study. Assessment & Evaluation in Higher Education, Vol 30. 30, No 4, pp.369-386.

•  Merry, S. ; Price, M.; Carless, D. & Taras, M. Eds. (2013) Reconceptualising feedback in higher education. London/Ny: Routledge.

•  Shute, V. J. (2008) Focus on formative feedback. Review of Educational Research, 78, 1, pp. 153-189. •  Wake, J. D., Dysthe, O., & Mjelstad, S. (2007). New and changing teacher roles in a digital age.

Educational Technology & Society 10(1). •  Vines. A. (2009) Multivoiced e-feedback in the study of law: Enhancing learning opportunities? In R.

Krumsvik: Learning in the network society and the digitized school. Nova Science Publishers •  Vines, A. & Dysthe, O. (2010) Productive learning in the study of law:The role of technology in the

learning ecology of a law faculty. L. Dirckinck-Holmfeld, C. Jones, B. Lindstrøm (Eds) Analysing networked learning practices in higher education and continuing professional development. Sense Publishers.

.