Tell me what you want and I’ll

show you what you can have

Who drives design of technology for

learning?

Sue Cobb Human Factors Research Group, University of Nottingham, UK

Structure of presentation

• Introduction

• My background and research

• Description of VE/CVE technology

• Why we wanted to apply this technology for student learning

• Example projects and Design/development approach: UCD/PD

• Outcomes and Lessons learned

Sue Cobb

• 1992: Virtual Reality Applications Research Team (VIRART), Human

Factors Research Group, University of Nottingham

• Head of HFRG, Faculty of Engineering

• Background - Psychology and Human Factors

• Research interest – Applications of VR technology in education

– Specific interest from special educational needs community in how VR

technology could bring the world of learning into the classroom

– Need to work with technology that could be placed in schools (desktop VR

systems)

• Research approach = Methods for user-centred design and

evaluation of VR applications

• www.hfrg.nottingham.ac.uk

Virtual and mixed reality technologies

Virtual Real

User-Centred Design (UCD)

• Design centred around the user

• Understanding the needs of the user in order to inform design – Involve users as much as possible (recognise

limitations)

– Integrate knowledge and expertise from different disciplines

– Iterative testing to check design meets user requirements

Participatory Design

• Originated in Scandinavia 1970‟s

• Established PD methods

– System prototypes

– User evaluation

– Mutual learning between active participants

– All partners are equal: Users, managers,

systems designers, human factors experts

Working with the User Group

How can VR support learning?

• Constructivist learning – „direct experience‟ – Multi-sensory cues – 3D immersion – Frames of reference

• Constructionist learning – Students create or modify VE – Collaborative learning

• Situated learning – Interactive role play in CVEs

Moshell and Hughes (2002)

• Computers are fun!

• Motivational

• Increased learning opportunities

• Self-directed, active learning

• Self-paced

• Learning by doing

• Safe space for learning

• Scaffolded

Neale, Brown, Cobb and Wilson (1999) Cobb, Neale and Stewart (2001).

• The Virtual City Project

• Practice of procedural steps in everyday tasks

• 3 weeks of training improved task familiarisation

• Student-led activity in real world

Brown et al.(1998)

Life skills education

Main findings

• Students found it easier to use VEs than

mentors believed they would

• Shift of control-action and decision-making shift

from teacher initiated to student initiated

• Anecdotal reports of benefit to performance

• Student feedback on UCD method positive

• Usability issues raised

VEs for Social Skills Training

AS Interactive 3yr project: 2000-2003

Funded by Shirley Foundation

University of Nottingham

Human Factors/VIRART

Computer Science

Psychology

National Autistic Society

www.virart.nott.ac.uk/asi

Why Virtual Environments (VEs)?

• Ability to control input levels e.g. number of people

present, auditory inputs, non-verbal cues

• Practice skills safely without experiencing real world

problems (role play)

• Shared (visual) features between virtual and real worlds

may facilitate generalisation of skills from former to latter

Parsons & Mitchell (2002)

Participatory design

• Idea generation – Discussion and brainstorming

– Low tech prototyping

– Bodystorming

• Design modelling – Mock-ups (“Wizard of Oz”)

• Prototype evaluation – User testing

– Observation

– User notes and comments

Text is displayed here

VE content and viewpoints

Interface design

Research Questions

• What type of VEs?

• How do we replicate social situations?

• How do users understand and interpret these VEs?

• How can we encourage social interaction?

• How do we measure social skills?

• How can we support generalisation?

Research Questions

Single User Virtual Environments (SVEs)

A single user is guided through a

social interaction task and invited to

make choices about what to do and

what to say in specific situations

The user may be supported by a

teacher or training advisor seated

next to them at the PC

SVE Training Scenarios

Social Café

Catching a bus

• Use, understanding and learning (to a limited degree).

• Some indication of successful learning and potential

generalisation to other media

• Learning potential not confined to those with stronger

verbal abilities (Mitchell et al. 2007)

• However, limitations

– Visually „blocky and unrealistic‟

– Limited spontaneous interpersonal communication

• CVEs offer more flexibility for but technology not

sufficiently robust for use in schools (Cobb, 2007)

Main findings

http://cospatial.fbk.eu

Objective: to develop specific interactive technologies for school settings that may help to promote learning and social competence in children on the autism spectrum.

Project structure

SoA Review

CBT Framework

SAS Technology CVE Technology

Evaluation

Social competence training Technology for ASC

Social competence skills:

• cooperation/collaboration

• social conversation

Application of CBT techniques:

• learning

• experience

• Design Process • Usability • Barriers to implementation

• Effectiveness

Collaborative Virtual Environments

(CVEs)

• Several users, real time

communication

• Open-ended, more natural

• Less anxiety provoking

• Exploration

• Dynamic

• Perspective taking

• Facilitation

• Supportive environment for

rehearsing appropriate

behaviours

Design method

• Design team: research group + teachers

• Design workshops –

evaluation & discussion of

design concepts

• School visits

• Concept designs

• Prototype building

• Methods for user testing and evaluation

• User trials

Participatory design

CBT aspects of scenario What is the intervention focused general social goal? Cooperation / Collaboration

What are goal’s main social components? Choosing

Mutual planning

Mutual performance

What are goal’s sub social components? Compromise

Which CBT learning

techniques are most

suitable to be

implemented?

Indicate which are best implemented

via software

Problem solving

Indicate which are best implemented

via a human mediator

Concept clarification

(collaboration/cooperation)

Which CBT

experience

techniques are most

suitable to be

implemented?

Indicate which are best implemented

via software

Behavioural rehearsal (working

together)

Feedback reinforcement (players are

rewarded for completing the task)

Indicate which are best implemented

via a human mediator

Modelling practice

Homework (practice working together

on a task in a real world activity)

Teacher Involvement

• 3 schools; 3 teachers in core design team

• 5 schools; 40 teachers in stage 2 reviews

• 4 schools intervention study (22 pupils)

• Dissemination event: 120 delegates, 1000+ hits online

Agile

Design Team

WP5

Wider school setting (new schools & PD with ASD students)

Block Party Block Party

Block Party

Catch

the Sheep

Catch

the

Sheep

Catch

the

Sheep

Snack Tim

e

Design Concepts

Adapted methods: involving

children with ASC in design

Millen, Cobb, Patel and Glover (2012).

Millen, Cobb, and Patel (2011)..

CVE software

Talk2U: Two children work with a teacher who explains social conversation and helps them to practise key parts. Different topics can be chosen and talked about by pressing the button in the room.

Block Challenge: Two children work together to build a tower of blocks. Each has a different target, and they must talk to each other to select blocks and rotate them so that the side facing each player matches their own target.

Main findings

• Suitable for supporting collaboration and communication

• Encourages children to consider others‟ perspective and

task-related needs

• Facilitation can be tailored to the individual needs of

children

• Engaging for all children but too easy for most

• Overall, game does what we wanted it to do!

Follow-on project: Shape digital stories. See outcomes relating to COSPATIAL in videos „Collaboration by Stealth‟, „Outside the Box‟, and „Working Party‟. http://www.birmingham.ac.uk/research/activity/education/shape/digital-stories/index.aspx

Why can’t you have what

you want?

Lessons learned: how to improve

the participatory design approach

CVE development process

Concept evaluation & prototyping Formative evaluation & re-design

Background technology review Concept elaboration

User centred design process.

Iterative review of design stages

Identification of needs & requirements + CBT

Presentation of requirements to

different stakeholders

Multi-disciplinary research in development and

application of interactive technology

Multi-disciplinary design/development teams Stakeholders--- define objectives/interpret outcomes HF/HCI researchers --- interface design CS and technical research--- develop design prototypes Users --- inform and evaluate interface, usability

Understanding user needs and requirements Task description Interviews and observation Focus groups

Informing technology design and development

UCD and PD methods Low tech prototyping Focus group review Usability studies

Evaluation of outcomes User testing Observation and measurement of

performance Evaluation of effectiveness

Inclusive design toolbox of methods (Neale, Cobb and Kerr, 2003)

Applying Participatory Design to Develop Technology for Autism Evaluation and Treatment.

Online course in collaboration with Autism Speaks: http://pdtech.haifa.ac.il/login/index.php

The COSPATIAL sandwich

Top slice: theory-driven

from the literature and

disciplinary custom

Filling: technology requirements

affordances, representations,

control, input devices

Bottom slice: user-driven

participants, practices,

processes Parsons and Cobb (2014)

Lessons learned

Overall conclusions

• Development of VR for learning requires a multi-

disciplinary team

• A participatory design approach ensures contribution from

all stakeholders

• Development process difficult and time consuming

• Acknowledge tensions between design drivers -

• Utility still requires teacher-led facilitation

• Not easy for teachers to set up in schools conduct

design process within school context? (Newbutt, N. PhD Thesis 2013)

Acknowledgements

• Thanks to the students, parents, teachers and other

professionals who participated in these projects.

• The Virtual City project was supported by a National

Lotteries Grant.

• AS Interactive was supported by funding from the Shirley

Foundation. Visit www.virart.nott.ac.uk/asi for information

• COSPATIAL was supported by the Seventh Framework

Programme of the European Commission (Grant

agreement no. 231266). Visit http://cospatial.fbk.eu/ for

project information.

References Brown, D. J., Neale, H. R., Cobb, S. V. G., & Reynolds, H. (1999). Development and evaluation of the virtual city. International Journal

of Virtual Reality, 3(4), 27-38.

Cobb, SVG (2007). Virtual environments supporting learning and communication in special needs education. Topics in Language

Disorders, 27 (3), 211-225.

Cobb, S.V.G., Hawkins, T., Millen, L., and Wilson, J.R. (2014) Design and Development of Virtual Environments for Special

Educational Needs. Chapter 42 in: In K. Hale and K. Stanney (Eds.), Virtual Environment Handbook, 2nd Edition, Boca Raton: CRC

Press, 1075-1108.

Cobb, S.V. G., Neale, H.R. and Stewart, D. (2001). Virtual Environments - Improving accessibility to learning? Proceedings of 1st

International Conference on Universal Access and Human Computer Interaction, Aug 8-10, New Orleans, Lawrence Erlbaum

Associates, 783-787.

Millen, L., Cobb, S.V.G. & Patel, H (2011). Participatory design approach with children with autism. International Journal on Disability

and Human Development (IJDHD), 10(4), 289-294.

Millen, L., Cobb, S.V.G., Patel, H. and Glover, T. (2012). Collaborative virtual environment for conducting design sessions with students

with autism spectrum conditions. Proc.9th Intl Conf. on Disability, Virtual Reality and Assoc. Technologies. P.M. Sharkey, E. Klinger

(Eds), pp269-278. Laval, France, 10-12 Sept. 2012.

Neale, H., Cobb, S. and Kerr, S. (2003). An inclusive design toolbox for development of educational Virtual Environments. Presented

at: Include2003, Royal College of Art, London, 25-28 March 2003

Parsons, S. and Mitchell, P. (2002) The potential of virtual reality in social skills training for people with autistic spectrum

disorders. Journal of Intellectual Disability Research, 46, (5), 430-443.

Neale, H. R., Brown, D. J., Cobb, S. V. G., & Wilson, J. R. (1999). Structured evaluation of Virtual Environments for special needs

education. Presence: Teleoperators and Virtual Environments, 8(3), 264-282.

Parsons, S. and Cobb, S. (2014). Reflections on the role of the „users‟: challenges in a multidisciplinary context of learner-centred

design for children on the autism spectrum. International Journal of Research & Method in Education,

DOI:10.1080/1743727X.2014.890584

Parsons, S. and Mitchell, P. (2002) The potential of virtual reality in social skills training for people with autistic spectrum

disorders. Journal of Intellectual Disability Research, 46, (5), 430-443.