Peer � Marking � In � MoodleHanqing � Zhao

Overview / Project Content

16/5/23

• Assessment in higher education• Marking in COMP1710• Data analysis and Mark calculation in COMP1710• Drawbacks of this peer assessment approach• How to improve the approach (Software functions)• Research on Moodle• Limitations and Future Work

Assessment in higher education

16/5/23

• Formative Assessment (2 disadvantages)

• Summative Assessment

- Exclusiveness - Difficulty of enforcing

16/5/23

Marking in COMP1710

• Formative assessment – quiz on report writing

• Summative assessment – report specifications

• Summative assessment – peer assessment

Combination of both formative assessment and summative assessment:

The Peer Marking in COMP1710

References: HEAd2015ConferencePoster Sabrina B. Caldwell, Tamás D. Gedeon

16/5/23

16/5/23

The Peer Marking in COMP1710

Data analysis and Mark calculation

References: Optimising  Peer  Marking  with  Explicit  Training:  from  Superficial   to  Deep  Learning  S.B.  Caldwell,  T.D.  Gedeon

• S mark – mark from the students

• D mark – mark from the expert

• SqErr rank – Square difference between S rank and D rank

• SqErr mark – Square difference between S mark and D mark

Problems of this approach

16/5/23

• In order to divide reports into 5 different levels, lecturer and tutors have to mark all the reports first.

• Create unique permutation of five reports for each students

• The reports in the spreadsheets have to be anonymized

• One report should be marked by two students

Need lots of manual work by the teaching staff due to the following element:

The main problem:

16/5/23

How to improve the approach - Idea

One possible approach is: Spreadsheets automation

Two main things included in spreadsheets automation:1. The generation of Excel spreadsheets workbook 2. The selection and permutation of five embedded reports

16/5/23

• The generation of Excel spreadsheets

16/5/23

Technology: Apache POI - a Java API for Microsoft Documents

Environment: Java

Input: Anonymized reports in PDF format

Output: Generated spreadsheets for peer evaluation

How to improve the approach - Method

• The selection and permutation of five embedded reports

program

16/5/23

How to improve the approach - Method

The selection and permutation of five embedded reports

Permutation: Heap's algorithm

- generates all possible

permutations of 5 reports

- then we randomly choose

one permutation and add it in the spreadsheet

Selection: Set a calculator c for every PDF file name, c will increase one when it is chosen, the PDF with lowest c value has higher priority to be chosen.

Research on Moodle

16/5/23

• What is Moodle

• Why should you use a

Course Management

System(CMG)

• What make Moodle special:

- Free and open source

- Educational Philosophy

- Community

16/5/23

Limitations and Future Work

What can be done to extend this project in future:• Moodle Construction Practice• PHP programming

-- E.g. download the spreadsheets from Wattle automatically -- E.g. an Email distribution and collection system to automatically

send notifications to students who didn’t submit reports / spreadsheets

Limitations on the software:1. Not so “automatic” still need manual work: The amount of the spreadsheets that can generated is fixed.2. Haven’t tested in the real-time working environment.

Questions