Post on 13-Mar-2020
Learning analytics for Virtual Learning Environments
Pedro J. Muñoz-Merino, José A. Ruipérez-Valiente, Derick Leony, Carlos Delgado Kloos, Héctor Javier
Pijeira Díaz, Javier Santofimia, Diego Redondo
Twiter: @pedmume email: pedmume@it.uc3m.es
Universidad Carlos III de Madrid
2
Our Learning Analytics Research lines
● Technological issues
Effective processing of the data
Compatibility between different sources
● Useful indicators
● Useful visualizations
● Know the relationships among indicators. Know
the causes
● Evaluation of the learning process
● Actuators: Recommenders, adaptive learning,
etc.
3
Inference of indicators
● Pedagogical theories
● Best practices
● Adapted models from other systems
● Indicators that can predict other interesting
indicators
4
Examples of indicators I
Learning profiles
Effectiveness, efficiency,
interest
Behaviours
Skills
Emotions
Pedro J. Muñoz-Merino, J.A. Ruipérez Valiente, C. Delgado Kloos, “Inferring higher level learning information from low level data for the Khan Academy platform.” Proceedings of the Third International Conference on Learning Analytics and Knowledge, pp. 112–116. ACM, New York, (2013)
Example II: Inference of optional activities
José A. Ruipérez‐Valiente, Pedro J. Muñoz‐Merino, Carlos Delgado
Kloos, Katja Niemann, Maren Scheffel, “Do Optional Activities Matter in Virtual Learning Environments?”,European Conference on Technology Enhanced Learning, pp 331-344, (2014)
6
Example III: Inference of more precise indicators: Effectiveness
Pedro J. Muñoz-Merino, José A. Ruipérez-Valiente, Carlos Alario-Hoyos, Mar Pérez-Sanagustín, Carlos Delgado Kloos, "Precise effectiveness strategy for analyzing the effectiveness of students with educational resources and activities in MOOCs", Computers in Human Behavior, vol. 47, pp. 108–118 (2015)
7
Example IV: Inference of emotions
Derick Leony, Pedro J. Muñoz-Merino, José A. Ruipérez-Valiente, Abelardo Pardo, David Arellano Martín-Caro, Carlos Delgado Kloos, "Detection and evaluation of emotions in Massive Open Online Courses", Journal of Universal Computer Science (2015)
Example V: HMM for emotion detection
Each emotion has its corresponding HMM.
Events are normalized and used as observations for a HMM.
Two current approaches:
Probability of emotion HMM is mapped to emotion level.
Emotion with highest probability in HMM.
Derick Leony, Pedro J. Muñoz-Merino, Abelardo Pardo, Carlos Delgado Kloos, “Modelo basado en HMM para la detección de emociones a partir de interacciones durante el aprendizaje de desarrollo de software”, Jornadas de Ingeniería Telemática, 2013
Level of relationship
•30.3 % hint avoider •25.8 % video avoider •40.9 % unreflective user •12.1% of hint abuser
Hint
avoid.
Video
avoid.
Unrefl.
User
Hint
abuser
Hint avoidance 1 -0.186
Video avoid. 1 0.289 0.096
Unrefl. user 0.289 1
Analysis of optional activities •76.81% of students who accessed, did not use any optional activities
•Difference of use per course and depending on type of optional activities
•55 goals (50.9% completed)
•40 votes (26 positive, 13 indifferent, 1 negative)
Type of activity Percentage of activities accessed
0% 1-33 % 34-66% 67-99% 100%
Regular learning
activities 2.48% 51.55% 23.19% 18.84% 3.93%
Optional activities 76.81% 18.43% 4.14% 0.41% 0.21%
Level of relationship: optional activities vs learning activities
Optional activities
sig. (2-tailed) N = 291
Exercises accessed:
0.429*
(p=0.000)
Videos accessed:
0.419*
(p=0.000)
Exercise abandonment:
-0.259*
(p=0.000)
Video abandonment:
-0.155* (p=0.008)
Total time:
0.491* (p=0.000)
Hint abuse:
0.089 (p=0.131)
Hint avoider:
0.053
(p=0.370)
Follow recommendations:
-0.002
(p=0.972)
Unreflective user:
0.039
(p=0.507)
Video avoider:
-0.051
(p=0.384)
Proficient exercises sig. (2-tailed)
N = 291
Optional activities:
0.553*
(p=0.000)
Goal:
0.384* (p=0.000)
Feedback:
0.205* (p=0.000)
Vote:
0.243* (p=0.000)
Avatar:
0.415* (p=0.000)
Display badges:
0.418*
(p=0.000)
Clustering depending on effectiveness
Grouping types of students
Prediction models
Learning gains= 25.489 - 0.604 * pre_test + 6.112 * avg_attempts + 0.017 * total_time + 0.084 * proficient_exercises
José A. Ruipérez-Valiente, Pedro J. Muñoz-Merino, Carlos Delgado Kloos, “A predictive Model of Learning Gains for a Video and Exercise Intensive Learning Environment”, Artificial Intelligence in Education conference, pp. 760-763 (2015)
Evaluation of the learning process
Pedro J. Muñoz-Merino, José A. Ruipérez-Valiente, Carlos Delgado Kloos, Maria A. Auger, Susana Briz, Vanessa de Castro, Silvia N. Santalla, “Flipping the classroom to improve learning with MOOC technology: A case study with engineering students using the Khan Academy platform”, Submitted for publication
15
ALAS-KA: Khan Academy
José A. Ruipérez-Valiente, Pedro J. Muñoz-Merino, Derick Leony, Carlos Delgado Kloos, “ALAS-KA: A learning analytics extension for better understanding the learning process in the Khan Academy platform”, Computers in Human Behavior, vol. 47, pp. 139-148 (2015)
16
LA for Google Course Builder:
● Links
Source code:
https://github.com/roderickfanou/Learning_analytics_on_
GCB
Video: http://www.youtube.com/watch?v=b5vOmWHgmqg
Miriam Marciel, Foivos Michelinakis, Roderick Fanou, and Pedro J. Muñoz-Merino, “Enhancements to Google Course Builder: Assessments Visualisation, YouTube Events Collector and Dummy Data Generator”, SINTICE conference (2013), Madrid, España
Visualization of emotions Time-based
visualizations Contextualized
visualizations
Timeline for the level of each emotion.
Daily average of emotion by day. Emotion levels vs student grades.
Emotion levels vs use of tools.
Derick Leony, Pedro J. Muñoz-Merino, Abelardo Pardo, Carlos Delgado Kloos, “Provision of awareness of learners’ emotions through visualizations in a computer interaction-based environment”, Expert Systems With Applications, vol. 40, no. 3 (2013), pp. 5093-5100
18
ANALYSE: LA support for open edX
● General information and Github
http://www.it.uc3m.es/pedmume/ANALYSE/
https://github.com/jruiperezv/ANALYSE
● Evaluation
● Demos
https://www.youtube.com/watch?v=3L5R7BvwlDM&fe
ature=youtu.be
Héctor Pijeira, Javier Santofimia, José A. Ruipérez-Valiente, Pedro J. Muñoz-Merino, Carlos Delgado Kloos, Demo ANALYSE, EC-TEL 2015
19
MOOC of maths
● MOOC of maths:
Available at: http://ela.gast.it.uc3m.es
Topics: Units of measurement, algebra, geometry
High school education for adults
Generation of educational materials: CEPA Sierra
Norte de Torrelaguna: Diego Redondo Martínez
28 videos, 32 exercises
Configuration, support and personalization of the
MOOC platform at Univ. Carlos III de Madrid
Open for everyone
Flipped the classroom methodology
20
ANALYSE in the MOOC of maths
● Uses of ANALYSE in the experience
Self-reflection for students
Support for the flipped classroom
Evaluation of educational materials
Evaluation of students
Evaluation of the course
21
Example I: Self-reflection
22
Example II: Evaluation of students
23
Example III: Flipped classroom, evaluation of materials
24
Example IV: Evaluation of the course
25
Present and future work in ANALYSE
● Design and implementation of higher level
learning indicators and their correspondent
visualizations
● Scalability. Work with a big amount of users
● Recommender based on previous data analysis
we have performed on other experiences ->
clustering, prediction, relationship mining
● Integration with Insights
26
ANALYSE in the Spanish TV
● Part of the mapaTIC project:
“La Aventura del Saber”, La 2, TVE,
http://www.rtve.es/alacarta/videos/la-aventura-del-
saber/aventuramapatic/3163463/
ANALYSE and the MOOC of maths: 5:30-7:11
27
Adaptation (I)
28
Adaptation (I): ISCARE
29
ISCARE competition tool
30
ISCARE: Publications
Pedro J. Muñoz-Merino, Manuel Fernández-Molina, Mario Muñoz-Organero, Carlos Delgado Kloos, “Motivation and Emotions in Competition Systems for Education: An empirical study”, IEEE Transactions on Education vol. 57, no. 3 (2014), pp. 182-187
Pedro J. Muñoz-Merino, Carlos Delgado Kloos, Manuel Fernández Molina, “Analyzing Learning Gains in a Competition Intelligent Tutoring System”, Intelligent Tutoring Systems Conference (2014)
Pedro J. Muñoz-Merino, Manuel Fernández Molina, Mario Muñoz-Organero, Carlos Delgado Kloos, “An Adaptive and Innovative Question-driven Competition-based Intelligent Tutoring System for Learning”, Expert Systems With Applications, vol 39, no 8 (2012), pp. 6932-6948
ADAPTATION (II)
Repetici
ones
4
4
Aprobado
Suspenso
No realizado
No cuenta
Master Thesis: Alma Collado
Learner profile is augmented with affective information (AI).
Fila 1 Fila 2 Fila 3 Fila 4
0
2
4
6
8
10
12
Columna 1
Columna 2
Columna 3
Implementation in the ROLE Personal Learning Environment.
Adaptation (III): Document recommendation based on emotions
Adaptation (IV)
Pedro J. Muñoz-Merino, Carlos Delgado Kloos, Mario Muñoz-Organero, Abelardo Pardo, "A Software Engineering Model for the Development of Adaptation Rules and its Application in a Hinting Adaptive E-learning System", Computer Science and Information Systems, vol. 12, no. 1, pp. 203-231(2015)
34
Adaptation (V): FEST Tutor
35
Adaptation (V): FEST Tutor
Master Thesis: Belén del Campo
Learning analytics for Virtual Learning Environments
Pedro J. Muñoz-Merino, José A. Ruipérez-Valiente, Derick Leony, Carlos Delgado Kloos, Héctor Javier
Pijeira Díaz, Javier Santofimia, Diego Redondo
Twiter: @pedmume email: pedmume@it.uc3m.es
Universidad Carlos III de Madrid