Human Factors Considerations When Setting up/ Expanding … · Human Factors Considerations When...

26
Human Factors Considerations When Setting up/ Expanding a Telemedicine and Telehealth program Patrick Boissy Ph.D Université de Sherbrooke ([email protected]) American Telemedicine Association Quality Healthcare Through Telecommunications Technology September 2013 Toronto, Ontario Canada

Transcript of Human Factors Considerations When Setting up/ Expanding … · Human Factors Considerations When...

Page 1: Human Factors Considerations When Setting up/ Expanding … · Human Factors Considerations When Setting up/ Expanding a Telemedicine and Telehealth program Patrick Boissy Ph.D Université

Human Factors Considerations When Setting up/ Expanding a Telemedicine and

Telehealth program

Patrick Boissy Ph.D Université de Sherbrooke

([email protected])    

American Telemedicine Association Quality  Healthcare  Through  Telecommunications  Technology  

September 2013 Toronto, Ontario

Canada

Page 2: Human Factors Considerations When Setting up/ Expanding … · Human Factors Considerations When Setting up/ Expanding a Telemedicine and Telehealth program Patrick Boissy Ph.D Université

What is Human Factors ?

PROCESS ‘Human factors engineering—the application of knowledge about human behavior, abilities, limitations, and other characteristics to the design of tools, machines, equipment, devices, systems, tasks, jobs, and environment to achieve productive, safe, comfortable, and efficient human use.’ DISCIPLINE AND A JOB ‘Human factors is the scientific discipline concerned with the understanding of interactions among humans and other elements of a system, and the profession that applies theory, principles, data and methods to design in order to optimize human well-being and overall system performance’ . - International Ergonomics Association

From: Eric Shaver (2009). The Many Definitions of Human Factors and Ergonomics.

Human factors

From: http://www.doubleimage.co.uk/the_simpson_posters.html

© BOISSY September 2013 Human Factors SIG 1

Goals: Ensuring that human interactions with systems enhance performance (usually measured as efficiency), increase safety and increase user satisfaction

Page 3: Human Factors Considerations When Setting up/ Expanding … · Human Factors Considerations When Setting up/ Expanding a Telemedicine and Telehealth program Patrick Boissy Ph.D Université

Scientific Disciplines & Methodological Approaches in HF

2 Human Factors SIG

So are the methodological approaches…

The field of human factors is diverse and multidisciplinary…

© BOISSY September 2013

Page 4: Human Factors Considerations When Setting up/ Expanding … · Human Factors Considerations When Setting up/ Expanding a Telemedicine and Telehealth program Patrick Boissy Ph.D Université

HF in Telehealth and Telemedecine Programs?

3 Human Factors SIG

•  Efficiency, safety and user satisfaction impact service delivery… and technology acceptance (TA)

•  TA can be seen as behavioral intention mediated by two key variables (perceived usefulness and perceived ease of use)

•  TA is a determining factor in use behavior and ultimately adoption

Degree to which a person has formulated conscious plans to perform or not perform some specified future behavior.

From : Venkatesh,V. and H.Bala (2008). Technology Acceptance Model 3 and a Research Agenda on Interventions. Decision Sciences Volume 39 Number 2, p 280.

Technology acceptance model 3 (TAM3)

Use and adoption are requirements for a

sustainable program !

© BOISSY September 2013

Page 5: Human Factors Considerations When Setting up/ Expanding … · Human Factors Considerations When Setting up/ Expanding a Telemedicine and Telehealth program Patrick Boissy Ph.D Université

Technological acceptance, use and adoption of EHR

4 Human Factors SIG

You can lead a horse to water, but you can't make it drink

•  Longitudinal survey of US hospitals

show that 44 percent of hospitals report having and using what can be define as at least a basic EHR system.

•  Although 42.2 percent meet all of the federal stage 1 "meaningful-use" criteria, only 5.1 percent could meet the broader set of stage 2 criteria.

•  Having a basic EHR and using an EHR is not the same …

‘Meaningful-use stages and criteria’  

Desroches, C. M., et al. (2013). "Adoption Of Electronic Health Records Grows Rapidly, But Fewer Than Half Of US Hospitals Had At Least A Basic System In 2012." Health Aff (Millwood) 32(8): 1478-1485.

© BOISSY September 2013

Page 6: Human Factors Considerations When Setting up/ Expanding … · Human Factors Considerations When Setting up/ Expanding a Telemedicine and Telehealth program Patrick Boissy Ph.D Université

7 Human Factors SIG

RESEARCH

ARTIC

LE

Ope

nAccess

Barriers

totheacceptan

ceof

electron

icmed

ical

recordsby

physicians

from

system

atic

review

totaxono

myan

dinterven

tions

Albe

rtBo

onstra*†,M

anda

Broe

khuis†

Abs

tract

Backgrou

nd:T

hemainob

jectiveof

thisresearch

isto

iden

tify,catego

rize,andanalyzebarrierspe

rceivedby

physicians

totheadop

tionof

Electron

icMed

icalRecords(EMRs)in

orde

rto

provideim

plem

enterswith

bene

ficial

interven

tionop

tions.

Metho

ds:A

system

aticliteraturereview

,based

onresearch

pape

rsfro

m1998

to2009,con

cerningbarriersto

the

acceptance

ofEM

Rsby

physicians

was

cond

ucted.

Four

databases,“Scien

ce”,“EBSCO

”,“Pub

Med

”and“The

Cochrane

Library”,w

ereused

intheliteraturesearch.Studies

wereinclud

edin

theanalysisifthey

repo

rted

onph

ysicians’p

erceived

barriersto

implem

entin

gandusingelectron

icmed

icalrecords.Electron

icmed

icalrecordsare

defined

ascompu

terized

med

icalinform

ationsystem

sthat

collect,store

anddisplaypatient

inform

ation.

Results

:The

stud

yinclud

estw

enty-twoarticlesthat

have

considered

barriersto

EMRas

perceivedby

physicians.

Eigh

tmaincatego

riesof

barriers,includ

ingatotalo

f31

sub-catego

ries,wereiden

tified.

Theseeigh

tcatego

riesare:

A)Financial,B)

Technical,C)

Time,D)Psycho

logical,E)

Social,F)Legal,G)Organizational,andH)Ch

ange

Process.

Allthe

secatego

riesareinterre

latedwith

each

othe

r.In

particular,C

ateg

oriesG(Organizational)andH(Chang

eProcess)seem

tobe

med

iatin

gfactorson

othe

rbarriers.By

adop

tingachange

managem

entpe

rspe

ctive,we

developsomebarrier-related

interven

tions

that

couldovercometheiden

tifiedbarriers.

Conc

lusion

s:Despite

thepo

sitiveeffectsof

EMRusagein

med

icalpractices,the

adop

tionrate

ofsuch

system

sis

stilllow

andmeets

resistancefro

mph

ysicians.Thissystem

aticreview

revealsthat

physicians

may

face

arang

eof

barrierswhe

nthey

approach

EMRim

plem

entatio

n.Weconclude

that

theprocessof

EMRim

plem

entatio

nshou

ldbe

treatedas

achange

project,andledby

implem

entersor

change

managers,in

med

icalpractices.The

quality

ofchange

managem

entplaysan

impo

rtantrole

inthesuccessof

EMRim

plem

entatio

n.Thebarriersandsugg

ested

interven

tions

high

lighted

inthisstud

yareintend

edto

actas

areferenceforim

plem

entersof

Electron

icMed

ical

Records.Acarefuld

iagn

osisof

thespecificsituatio

nisrequ

iredbe

fore

relevant

interven

tions

canbe

determ

ined

.

Backgrou

ndElectronicMed

ical

Records

(EMRs)

areco

mpu

terized

medical

inform

ationsystem

sthat

collect,store

anddis-

play

patien

tinform

ation.

The

yareameans

tocreate

legiblean

dorganizedreco

rdings

andto

access

clinical

inform

ationab

outindividu

alpa

tien

ts.Fu

rthe

r,EMRs

areintende

dto

replaceex

isting(often

pape

rba

sed)

med

ical

reco

rds

which

are

alread

yfamiliar

to

practition

ers[1].Patient

reco

rdsha

vebe

enstored

inpa

perform

forcenturiesan

d,ov

erthis

period

oftime,

they

hav

eco

nsu

med

increasing

spacean

dnotab

lyde

layedaccess

toeffic

ient

med

ical

care

[2].In

contrast,

EMRsstoreindividu

alpatien

tclinical

inform

ationelec-

tron

ically

anden

able

instan

tavailabilityof

thisinform

a-tion

toallprov

idersin

thehe

althcare

chainan

dso

shou

ldassist

inprovidingcoherent

andconsistent

care.

Electronic

Med

ical

Records

(EMRs)

andElectronic

HealthRecords

(EHRs)

areview

edas

intercha

ngeable

synon

ymsin

mosthe

alth

inform

atics.

Other

similar

expr

ession

sex

istalbe

itwith

aso

metim

esslightly

restricted

focu

s.W

hile

EMRsha

veagene

ralfocu

son

*Co

rrespon

dence:albe

rt.boo

nstra@

rug.nl

†Co

ntrib

uted

equally

Faculty

ofEcon

omicsandBu

siness,University

ofGroning

en,G

roning

en,The

Nethe

rland

sFulllistof

author

inform

ationisavailableat

theen

dof

thearticle

Boon

stra

andBroe

khuisBM

CHealth

Services

Research

2010

,10:23

1http://www.biomed

central.com

/147

2-69

63/10/23

1

©2010

Boon

stra

andBroe

khuis;licen

seeBioM

edCe

ntralLtd.Thisisa

nOpe

nAc

cessarticledistrib

uted

unde

rthe

term

softhe

Creative

Common

sAttribu

tionLicense(http://creativecom

mon

s.org/licenses/by/2.0),w

hich

perm

itsun

restricteduse,

distrib

ution,

and

reprod

uctio

nin

anymed

ium,p

rovide

dtheoriginal

workisprop

erlycited.

HF variables are part of the the barriers most cited for technological acceptance of EHR

© BOISSY September 2013

Page 7: Human Factors Considerations When Setting up/ Expanding … · Human Factors Considerations When Setting up/ Expanding a Telemedicine and Telehealth program Patrick Boissy Ph.D Université

expected to replace current product revenue by offering a better fit with current market needs.While convenient to diagram, decisions on whether to modify or retire products, when toretire products and which products to retire are difficult for firms to make.

Not all of the traits of artifacts are identical to those of living things. In contrast with liv-ing things, human-made systems do not typically show whether they are doing well or arehaving problems. It can be a challenge to know how well a system is performing. Are each ofthe elements that have been selected to be a part of the system performing at capacity? Is theconfiguration of those elements optimal? Can the system be optimized further, or is the sys-tem obsolete and actually living on borrowed time? The breadth, complexity andinterdependence of systems can make these difficult questions to answer.

1.4 The development process

Firms vary in the approach that each takes to research and development. Even so, there aresteps that account for the information and action that is typically necessary to accomplishdesign and human factors activity in the context of development.

1.4.1 Manufacturing

Figure 1.4 shows Robert Bateman’s view of the way that research and development activitiesfit into the life cycle of a system. Portions of the diagram which are shown in gray indicatemajor steps: define problem, gather data, determine feasibility, plan (define requirements,identify alternatives), analyze (evaluate alternatives), design (build prototype, test, install),implement, use (evaluate) and retire. The design, prototype and test cycle may be repeateduntil the solution meets requirements. Evaluation includes means to either plan a productmodification or to retire it. To create a follow-on product in time, it is necessary to completeplanning before retiring its predecessor.

16 Human factors practice

Figure 1.4 Development process—Robert Bateman describes the phases of product developmentwithin the context of product life cycles. To create a follow-on product in time, it is neces-sary to complete planning before retiring its predecessor. Major phases in the developmentcycle are shown as grey shaded arrows.

Source: Bateman, R. Reproduced by permission of the author.

DesignAnalyze

Plan

Use

Test

Install

Identify alternatives

Define requirements

Define problem Gather

data

Determine feasibility

Evaluate

Build prototype

Evaluatealternative

Implement

Retire

Contributions of Human Factors in dev. & Lifecycle of Product

DESIGN  AND  DEVELOPMENT  

PHASE  

USE  OF    PRODUCT    IN  REAL    WORLD  

6 Human Factors SIG

Who : Designers, users, managers, stakeholders When Design , procurement, Implementation and operation phases

From: Christopher P. Nemeth (2004). Human Factors Methods for Design Making Systems Human-Centered. CRC Press, Page 16.

© BOISSY September 2013

Page 8: Human Factors Considerations When Setting up/ Expanding … · Human Factors Considerations When Setting up/ Expanding a Telemedicine and Telehealth program Patrick Boissy Ph.D Université

Context and Goals of HF Evaluations / program perspective

7 Human Factors SIG

•  Understanding how people

work and what people will do with technology

•  Fitting the technology to the users and their clinical workflow

•  Supporting users in the field •  Evaluating and monitoring

technology use to optimize outcomes

•  Establish technological feasibility and requirement to scale up the activities

Real life use of the product in your context

HF

eval

uatio

ns to

opt

imiz

e te

chno

logy

us

e du

ring

key

phas

es

Goals of HF evaluations:

From: Christopher P. Nemeth (2004). Human Factors Methods for Design Making Systems Human-Centered. CRC Press, Page 16.

© BOISSY September 2013

Page 9: Human Factors Considerations When Setting up/ Expanding … · Human Factors Considerations When Setting up/ Expanding a Telemedicine and Telehealth program Patrick Boissy Ph.D Université

HF Evaluations processes / program perspective

8 Human Factors SIG

How: •  Contextual inquiries to

understand workflow of user and the factors involved

•  Creating a use case scenario for the technology

•  Identifying usability issues with the technology to find workarounds

•  Monitoring real time use, reliability and satisfaction metrics

© BOISSY September 2013

Page 10: Human Factors Considerations When Setting up/ Expanding … · Human Factors Considerations When Setting up/ Expanding a Telemedicine and Telehealth program Patrick Boissy Ph.D Université

•  Fly on the wall - Observe and record behavior within its context, without interfering with people’s activities.

•  A day in the life - Catalog the activities and contexts that users experience for an entire day.

•  Personal inventory - Document the things that people identify as important to them as a way.

•  Shadowing - Tag along with people to observe the use of similar devices in a similar scenario and understand their day-to-day routines, interactions, and contexts. The researcher asks questions to ensure that he or she understands what the users are doing and why they are doing it that way.

•  Interview and focus group –Methods to elicit information and insights from users in response to carefully designed questions, either one informant at a time or through interaction between the moderator and a group of informants.

Contextual Inquiries /

“Know the User”

9 Human Factors SIG

Walking in someone else

shoes…

© BOISSY September 2013

Page 11: Human Factors Considerations When Setting up/ Expanding … · Human Factors Considerations When Setting up/ Expanding a Telemedicine and Telehealth program Patrick Boissy Ph.D Université

•  Use case = method for describing from the user point of

view the interactions between a user and a device to accomplish a functional objective in a specific context.

•  Tasks are the fundamental unit of description of that interaction. A task is ‘a set of human actions that contributes to a specific functional objective and ultimately to the output goal of a system.’ (Drury et al. (1987:373).

•  A use case has multiple “paths” that can be taken by any user at any one time.

•  A use case scenario is a single path through the use case where a set of tasks that represent the optimal solution to accomplish a functional objective with a device in a specific context are identified and sequenced.

Creating a Use Case Scenario for the Technology

10 Human Factors SIG

The use case scenario is often used for usability testing and to create documentation for users… Task analysis is the foundation to establish a use case

scenario. © BOISSY September 2013

Page 12: Human Factors Considerations When Setting up/ Expanding … · Human Factors Considerations When Setting up/ Expanding a Telemedicine and Telehealth program Patrick Boissy Ph.D Université

•  There are various forms and levels of TASK ANALYSIS (TA) possible.

•  The form and level of a TA depends on the purpose for which the analysis is to be conducted.

•  For the creation of a use case scenario the most common form is HIERARCHICAL TASK ANALYSIS (HTA).

•  HTA is a mean to break down the use of a device into a

hierarchy of goals, operation and plans needed to accomplish a functional objective. •  Goals are the unobservable task goals associated

with operation of the device. •  Operations are the observable behaviours or

activities which accomplish the goals. •  Plans are unobservable decisions and planning on

behalf of the operator.

Creating a Use Case Scenario - Task analysis / HTA

11 Human Factors SIG © BOISSY September 2013

Page 13: Human Factors Considerations When Setting up/ Expanding … · Human Factors Considerations When Setting up/ Expanding a Telemedicine and Telehealth program Patrick Boissy Ph.D Université

Ex. of HTA with Task Flow diagram- Making Coffee

From:    Mark  R.  Lehto,  James  R.  Buck    (2008).  Introduction  to  Human  Factors  and  Ergonomics  for  Engineers  .  Taylor  &  Francis  Group,  pages  118,119.    

task about the accuracy of the description. The latter verification helps if thetask includes sensory or mental activities that are not always obvious fromdirect observation.

When the observed activities do not correspond to the TFD, direct ob-servational procedures can be used to develop the task description. Theseforms of direct observation may be continuous or periodic, and they may beperformed by a human observer making on-site observations or by using somerecording device such as digital video or photography.22 Here again, it is agood idea to take the resulting TFD from the observational study and askhuman operators performing the tasks about its accuracy.

Another technique that is sometimes used is called a Talk Aloud proce-dure, which asks the human operator to talk about what he or she is doingwhile performing the task. Use of the talk aloud procedure is controversial be-cause the acts of talking and performing the task can interfere with each other,and the result may be misleading, especially if performance time measuresare taken at the same time. No matter which combination of procedures isused to determine an initial TFD, the primary purpose is to answer the ques-tions: who, what, where, when, why, and how about the tasks and job.

Hierarchical Task Analysis (HTA)

As should be apparent from the earlier example for making coffee, tasksand subtasks normally fit into a natural hierarchy. In some instances there are

CHAPTER THREE Design to Fit Tasks, Processes, and People 119

22. Some approaches for taking such observations are discussed in chapter 10. Making a video recording of thetask is often the easy part. Anyone who has tried to analyze video tapes of task activity knows well thatthis can be time-consuming, tedious, and difficult. Fortunately, computer tools, such as the Interact soft-ware available from Mangold International (http://www.mangold-international.com) are now availableto make it easier to analyze video recordings of task activity. The latter software allows the analyst toview digital video recordings of the task on a PC, and provides a convenient interface for marking the startand end of particular activities, which are then automatically recorded in a convenient spread sheet format.The interface also provides the analyst the ability to rapidly move back and forth at different speedsthrough the sequence of activities to specific frames of interest, which is a great improvement over tryingto figure out exactly where a task starts or ends using a standard video player or movie projector!

A more refined view of making coffee. ! FIGURE 3.15

COFFEE POT ACTIVITIES1. Pick up the pot2. Grasp the handle of the pot with right hand3. Move the pot off the warmer4. Transport the pot to the kitchen sink5. Place the pot under the cold water faucet6. Fill the pot with water7. Turn on cold water faucet and let the water fill to the mark8. Turn off the cold water faucet9. Transport the filled pot to the coffee maker

10. Empty the pot into the receiver of the coffee maker11. Tip the pot and pour the total contents into the water receiver12. Untip the pot13. Carry the pot to the lower level part of the coffee maker14. Insert the pot back onto the warmer

ch03_8129_Lehto_LEA 5/9/07 5:40 PM Page 119

Operations  

A block diagram showing these three steps is shown in Figure 3.14. The read-ers’ likely feeling at this point is that the description is too coarse to be verybeneficial. An expanded list for the first set of activities might be as shown inFigure 3.15. Similarly expanded lists could be developed for the Filter holderactivities, and the Switch activities.

Note that each level of detail fits within the level of detail above, creat-ing a hierarchy of finer and finer details. If one used a hyper-link computerprogram, each level would fit within the one above so that a designer canmove down to the desired level of detail. Virtually any reader who has madecoffee with a similar type of coffeemaker will easily recognize how this par-ticular procedure works, although that reader may argue about the particularsubtasks, and he or she could easily point out that steps 1 and 2 could be donein either order.

Task Flow Diagrams

The activities performed in a task are often described with flow diagramsvery similar to those used to describe computer programs. When there is onlyone accepted way to perform the task, the Task Flow Diagram (TFD) is a sim-ple sequence of boxes denoting the sequence of activities (see Figure 3.14).More complex TFDs include decisions (see Figures 3.12 and 3.13). Whenthe design team starts with the currently existing procedure as the initial con-cept for finding improvements, the typical procedure is to first find the oldtask flow diagram (TFD). The degree this description fits the current situationis then evaluated. If there is enough time available, a person from the designteam may go to the site and observe people doing the task. If the observedactivities correspond to the description in the TFD, it can be assumed to becurrent. Further verification may involve asking the operator performing the

118 Introduction to Human Factors and Ergonomics for Engineers

! FIGURE 3.14 Task flow diagram for making coffee.

ch03_8129_Lehto_LEA 5/9/07 5:40 PM Page 118

Goals  

12 Human Factors SIG © BOISSY September 2013

Page 14: Human Factors Considerations When Setting up/ Expanding … · Human Factors Considerations When Setting up/ Expanding a Telemedicine and Telehealth program Patrick Boissy Ph.D Université

Ex. of use case scenario for teledermatology consult

13 Human Factors SIG

Patient  presents  with  Derm  

condition  

Primary  Care  Physician  

Primary  Care  nurse  

Teledermatologist  

Document  background    

in  chart  

Capture  photos  

Upload  photos  &  route  chart  

Review  photos  &  chart  

Appointment  

Needed?  

Route  chart  to  PCP  

End  Process  Communicate  to  patient  

MA  deletes  photos  

Document  Tx  Plan  

Request  appt  in  Derm  

ACME  Teledermatology  Flow  Chart  

© BOISSY September 2013

Page 15: Human Factors Considerations When Setting up/ Expanding … · Human Factors Considerations When Setting up/ Expanding a Telemedicine and Telehealth program Patrick Boissy Ph.D Université

16 Human Factors SIG

Operations to accomplish goals #2 (capture clinical photo)

1.  Obtain Camera & Setup Clinical Camera

2.   Capture Clinical Photos 3.  Setup Dermatoscope Camera &

Capture Dermatoscope Photos (if used)

4.  Provide Patient Instructions for Rapid Response (Preferred) or Deferred Response

5.  Enter Note in KPHC Visits Section 6.  Connect the Clinical Camera 7.  1mport/Acquire Clinical Camera

lmages 8.  Connect Dermatscope Camera 9.  Import Derrnatscope lmages 10.  Connect to PACScan 11.  Validate Photo Upload (System1) 12.  Validate Photo Upload (System 2)

List of goals for Primary care nurse to capture clinical photo and upload

photos & route chart

© BOISSY September 2013

Page 16: Human Factors Considerations When Setting up/ Expanding … · Human Factors Considerations When Setting up/ Expanding a Telemedicine and Telehealth program Patrick Boissy Ph.D Université

Identifying Usability Issues to Find Workarounds

15 Human Factors SIG

•  No new technology is perfect out of the box…

•  Often a disconnect between procurement people versus people using, supporting and managing technology

•  Will it be usable in

your context and environment with your users ?

© BOISSY September 2013

Page 17: Human Factors Considerations When Setting up/ Expanding … · Human Factors Considerations When Setting up/ Expanding a Telemedicine and Telehealth program Patrick Boissy Ph.D Université

•  "[Usability refers to] the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.” - ISO 9241-11

•  Usability has multiple components and is traditionally associated

with these five usability attributes: learnability, efficiency, memorability, errors, satisfaction. - Jakob Nielsen, Usability Engineering, 1993, p.26

What is Usability?

16 Human Factors SIG © BOISSY September 2013

Page 18: Human Factors Considerations When Setting up/ Expanding … · Human Factors Considerations When Setting up/ Expanding a Telemedicine and Telehealth program Patrick Boissy Ph.D Université

Usability Inspection and Testing Methods

Usability testing

‘ Refers to an observational research technique where representative end users are recruited to participate in realistic task scenarios in a simulated environment to detect design flaws and deficiencies’ (Rubin, 1994). •  Formative test: Focus = diagnose

problems and offer solutions, less formal data analysis techniques.

•  Summative test: Tests conducted in order to generate metrics or measure formal improvements, such as benchmark studies require more formal data analysis methods

17 Human Factors SIG

Usability Inspection

•  Heuristic evaluation & estimation •  Cognitive walkthrough •  Pluralistic walkthrough •  Feature inspection •  Consistency inspection •  Standards inspection •  Formal usability inspection

‘ Generic name for a set of methods that are all based on having evaluators inspect a user interface to identify usability problems in the design, though some methods also address issues like the severity of the usability problems and the overall usability of an entire system’ (Neilsen, 1995).

See  http://www.nngroup.com/articles/summary-­‐of-­‐usability-­‐inspection-­‐methods/  © BOISSY September 2013

Page 19: Human Factors Considerations When Setting up/ Expanding … · Human Factors Considerations When Setting up/ Expanding a Telemedicine and Telehealth program Patrick Boissy Ph.D Université

How is Usability Testing Done ?

•  Step #1 - Define the audience and the goals of the usability test.

•  Step #2 -Create and select task scenarios participants will have to realize to address goals of usability test.

•  Step #3- Choose the right metrics and methods to collect usability outcomes during the task scenarios.

•  Step #4- Establish qualification of test participants and recruit the right people to perform the task scenarios.

•  Step #5- Watch test participants and measure usability outcomes as they perform the task scenarios or once completed.

•  Step #6 Analyze usability outcomes in relation to usability goal.

18 Human Factors SIG © BOISSY September 2013

Page 20: Human Factors Considerations When Setting up/ Expanding … · Human Factors Considerations When Setting up/ Expanding a Telemedicine and Telehealth program Patrick Boissy Ph.D Université

•  Ease of learning - How fast can a user who has never seen the user interface before learn it sufficiently well to accomplish basic tasks?

•  Memorability - If a user has used the system before, can he or she remember enough to use it effectively the next time or does the user have to start over again learning everything?

•  Subjective satisfaction - How much does the user like using the system?

•  Efficiency of use - Once an experienced user has learned to use the system, how fast can he or she accomplish tasks?

•  Error frequency and severity - How often do users make errors while using the system, how serious are these errors, and how do users recover from these errors?

Metrics of Usability ?

19 Human Factors SIG © BOISSY September 2013

Page 21: Human Factors Considerations When Setting up/ Expanding … · Human Factors Considerations When Setting up/ Expanding a Telemedicine and Telehealth program Patrick Boissy Ph.D Université

Capturing Usability Metrics ?

User interactions are studied using: •  Live view with observation grid and

timing device (time and motion study) •  Video and audio recordings •  Data logging and screen capture of

the system operation •  Gaze tracking

20 Human Factors SIG

Video    camera  

Gaze    tracking    system  

Data    logging  

© BOISSY September 2013

Page 22: Human Factors Considerations When Setting up/ Expanding … · Human Factors Considerations When Setting up/ Expanding a Telemedicine and Telehealth program Patrick Boissy Ph.D Université

Capturing Usability Metrics ?

21 Human Factors SIG © BOISSY September 2013

•  Morea 3.3 Usability software suite http://www.techsmith.com/morae.html •  Silverback Usability testing software http://silverbackapp.com •  Online Screenshot Testing Software http://www.optimalworkshop.com/chalkmark.htm •  Visual heatmap of clicks on a HTML page http://www.labsmedia.com/clickheat/index.html

http://www.usabilitynet.org/tools/ r_questionnaire.htm

Software resources for capturing usability metrics from PC based technology

http://www.epiphan.com

Questionnaire resources for capturing usability metrics

Hardware for screen capture, audio and video capture

Page 23: Human Factors Considerations When Setting up/ Expanding … · Human Factors Considerations When Setting up/ Expanding a Telemedicine and Telehealth program Patrick Boissy Ph.D Université

•  Key consideration when buying, deploying a product/ technology in healthcare processes.

•  Part of the system development life cycle but it doesn’t mean that the product/technology you buy/deploy will work for all contexts and users.

•  Buying or going to the field without considering human factors is like jumping head first in a lake without knowing the depth of the lake.

•  Human factors is often a compromise between function and form.

Conclusions

Human factors :

21 Human Factors SIG © BOISSY September 2013

Page 24: Human Factors Considerations When Setting up/ Expanding … · Human Factors Considerations When Setting up/ Expanding a Telemedicine and Telehealth program Patrick Boissy Ph.D Université

Conclusions

Usability evaluation is:

•  Part of the triangulation process required to establish the complete picture of a product and its intended use in a specific environment.

•  When put in context of other data, such as project goals, user goals, user an experts feedback, and usage metrics, usability evaluation can help identify specific high priority problems.

•  Not an exact science. Recommendations and conclusions have to be interpreted with caution and confronted to the reality studied.

22 Human Factors SIG © BOISSY September 2013

Page 25: Human Factors Considerations When Setting up/ Expanding … · Human Factors Considerations When Setting up/ Expanding a Telemedicine and Telehealth program Patrick Boissy Ph.D Université

1.  http://www.usabilitynet.org/tools.htm 2.  http://www.upassoc.org/usability_resources/index.html 3.  Faulkner, L. (2003). Beyond the five-user assumption: benefits of increased sample sizes in usability

testing. Behav Res Methods Instrum Comput, 35(3), 379-383. 4.  FDA. (2011). Applying human factors and usability engineering to optimize medical device design. 5.  Jaspers, M. W. (2009). A comparison of usability methods for testing interactive health technologies:

methodological aspects and empirical evidence. Int J Med Inform, 78(5), 340-353. 6.  Kaufman, D. R., Patel, V. L., Hilliman, et al. (2003). Usability in the real world: assessing medical

information technologies in patients' homes. J Biomed Inform, 36(1-2), 45-60. 7.  Kushniruk, A., & Borycki, E. (2011). Exploring the relationship between usability and technology-induced

error: unraveling a complex interaction. Stud Health Technol Inform, 166, 48-56. 8.  Kushniruk, A. W., Borycki, E. M., Kuwata, S., & Kannry, J. (2011). Emerging approaches to usability

evaluation of health information systems: towards in-situ analysis of complex healthcare systems and environments. Stud Health Technol Inform, 169, 915-919.

9.  Kushniruk, A. W., & Patel, V. L. (2004). Cognitive and usability engineering methods for the evaluation of clinical information systems. J Biomed Inform, 37(1), 56-76

10. Liljegren, Erik. (2006). Usability in a medical technology context assessment of methods for usability evaluation of medical equipment. International Journal of Industrial Ergonomics, 36(4), 345-352.

11. Sears, Andrew , & Jacko, Julie A. (2007). The Human-computer interaction handbook: fundamentals, evolving technologies and 3merging applications: CRC; 1 edition (Sep 1 2002).

Usability evaluations

©  BOISSY  April  2013   23 Human Factors SIG

Web Resources & References

Page 26: Human Factors Considerations When Setting up/ Expanding … · Human Factors Considerations When Setting up/ Expanding a Telemedicine and Telehealth program Patrick Boissy Ph.D Université

Questions

©  BOISSY  April  2013   24 Human Factors SIG

[email protected]