WIAD 2016 Mexico City - UX afectiva: Retos del diseño de la experiencia del usuario con cómputo...
-
Upload
world-information-architecture-day -
Category
Education
-
view
318 -
download
2
Transcript of WIAD 2016 Mexico City - UX afectiva: Retos del diseño de la experiencia del usuario con cómputo...
2/21/16
1
01 WORLD IA DAY 2016 PRESENTATION TITLE HERE
HEADER OPTION SUB HEAD OR SHORT DESCRIPTION!
Some kind of explanatory text, reference or footnote can go here and wrap to two lines, if needed.
Some kind of illustration or image?
Affective UX"Challenges in UX design involving affective computing
Alfredo Sánchez
Universidad de las Américas Puebla
©2006-2014,AlfredoSánchez
h8p://www.udlap.m
x
Architecturing happiness Howdoweknowusersarehappy?
CaninteracIvesystemsdetecthappiness(orthelackthereof)?
CaninteracIvesystemsintenIonallyproducehappinesss?
Cantechnologyatleastnotgetintheway?
©2006-2016,AlfredoSánchez
Agenda
• TermsofaffecIon• AffecIvecompuIng
• DetecIon• Synthesis• MediaIon
• AffectinUX• Recentdevelopment• Openissues
Terms of affection
©2006-2014,AlfredoSánchez
Emotion
AreacIonexperiencedusuallytowardaspecificobjectandtypicallyaccompaniedbyphysiologicalandbehavioralchangesinthebody
©2006-2014,AlfredoSánchez
2/21/16
2
Mood
• Aconsciousstateofmind(typicallylasInglongerthananemoIon)
joy depression
©2006-2014,AlfredoSánchez
Sentiment
• AnaYtudeoropinion–astatethatresultsfroma8ributesassignedtoobjects,whicharegroupedintocategories
©2006-2014,AlfredoSánchez
The affect spectrum
emoIon
mood
senIment
may trigger
may prompt
Duration ( - ) ( + ) ©2006-2014,AlfredoSánchez
Affective computing
©2006-2014,AlfredoSánchez
(Computersytems)relatedto,arisingfrom,ordeliberatelyinfluencingaffecIvephenomena- MITAffec)veCompu)ngGroup
AffecIveCompuIngRosalindPicard1995
Affective computing (for engineers) J
Thestudyanddevelopmentofsystemsanddevicesthatcanrecognize,interpret,process,simulateandmediatehumanaffects.
©2006-2016,AlfredoSánchez
Why affective computing?
©2006-2014,AlfredoSánchez
2/21/16
3
Why affective computing?
©2006-2014,AlfredoSánchez
Ichoosenottofeel
h8p://anim
ala.mx/no
Icias/20
15/02/27
/adios-m
r-spock/
Why Affective Computing
• “Reasonis,andoughttobe,onlytheslaveofthepassions”
DavidHume,1711-1776• WithoutpassionswewouldlackallmoIvaIon,allimpulseordrivetoact,oreventoreason
©2006-2014,AlfredoSánchez
Why affective computing?
• Alove/haterelaIonshipwiththecomputerandITsingeneral
©2006-2014,AlfredoSánchez
h8ps://w
ww.you
tube
.com
/watch?v=ZcZBT
B-dkqQ
Why affective computing? (for engineers) J
• StudiesbymanyresearchersshowavitalroleofemoIoninmanyprocesses:
• PercepIon• Decisionmaking• CreaIvity• Comprehension• Memory• SocialinteracIon
©2006-2014,AlfredoSánchez
h8p://www.bintegra.com/w
p-conten
t/up
loads/2012/05/Fotolia_socila_u
sers_S.jp
g
Some application areas • EducaIon–intelligenttutoring• Health–monitoringpaIents• Safety–monitoringdrivers/pilots• Security–detecInglies• E-commerce–aYtudestowardsproductsorservices
• Videogames–producingengagingenvironments,enrichingUX
• Userstudies–incorporaIngemoIonintoevaluaIon
©2006-2014,AlfredoSánchez
Why affective computing (for our community)
EmoIonisthewaywecommunicateourexperience
©2006-2014,AlfredoSánchez
2/21/16
4
Main aspects of affective computing
perception
synthesis
mediation
©2006-2014,AlfredoSánchez
(ofemoIons,moodsandsenIments)
Affect detection
©2006-2014,AlfredoSánchez
Emotion detection: From facial cues
©2006-2014,AlfredoSánchez
h8p://npx.auIsmspot.com/facial_expression.jpg
Emotion detection: From postural cues
©2006-2014,AlfredoSánchez
h8p://instrid
ebod
ywork.com/w
p-conten
t/up
loads/2013/05/po
stures-of-e
moI
on1.jpg
Emotion detection: From voice cues
©2006-2014,AlfredoSánchez
h8p://pandodaily.files.wordpress.com/2013/07/whisper-in-ear.jpg?w=584&h=329
h8p://gigaom
2.files.wordp
ress.com
/2013/07/emoI
on-spe
ech.jpg?w=300&h=
200&
crop
=1
Emotion detection: From usage patterns
©2006-2014,AlfredoSánchez
h8p://web
.deepn
etsecurity.com/produ
cts2/typesen
se.asp
Keystrokedynamics NavigaIonpa8erns Eyetracking
2/21/16
5
Emotion detection: From brain activity
©2006-2014,AlfredoSánchez
Emo)
vEPOCse
nsor
Hond
aBC
Isystem
Emotion detection: From sensors of autonomic activity • Skinconductanceincreasesevenwithali8lesurpriseoranxiety
• Bloodvolume/pressureincreaseswithangerorstress
• Respiratoryrhythmvarieswithfear,sadness,happiness,anxiety,fear
• Muscletensionincreaseswithsurpriseandanxiety
©2006-2014,AlfredoSánchez
Affec)vasensor
NeXusse
nsor
Emotion detection: From sensors of autonomic activity
©2006-2014,AlfredoSánchez
“Knowyourinnerself”W/Me
“Hackhappiness”Feel(bySenIo)
Some general techniques to support affective computing
FordetecIon,synthesisandmediaIon
©2006-2014,AlfredoSánchez
Facial Action Coding System (FACS)
©2006-2016,AlfredoSánchez
h8p://www.th
eroadtosiliconvalley.com/tag/Im-roth/
FACS • FacialAcIonCodingSystem(FACS),verypopular• DefinesAcIonUnits(AUs),contracIonorrelaxaIonofoneormorefacemuscles
…
©2006-2014,AlfredoSánchez
2/21/16
6
FACS (for “universal” emotions)
h8p://www.paulekm
an.com
/facs/
©2006-2014,AlfredoSánchez
Posture, motion and gesture analysis
©2006-2014,AlfredoSánchez
Posture analysis
h8p://pe
ople.ict.u
sc.edu
/~gratch/CSCI534/ACII-H
andb
ook-Ge
stureSyn.pdf
©2006-2014,AlfredoSánchez
Posture and motion analysis
• E.g.armmovementsandemoIon:
©2006-2014,AlfredoSánchez
h8p://pe
ople.ict.u
sc.edu
/~gratch/CSCI534/ACII-H
andb
ook-Ge
stureSyn.pdf
Gesture analysis
©2006-2014,AlfredoSánchez
h8p://pe
ople.ict.u
sc.edu
/~gratch/CSCI534/ACII-H
andb
ook-Ge
stureSyn.pdf
MPEG-4
• MPEG-4(ISO144996)defines• 86FaceAnimaIonParameters(FAPs)• 196BodyAnimaIonParameters(BAPs)
©2006-2014,AlfredoSánchez
2/21/16
7
Some existing tools
• FaceReader,SHORE,eMoIon,RealEyes
©2006-2014,AlfredoSánchez
Some challenges
• Prototypicalvs.non-prototypicalemoIons• Purevs.combinedemoIons• MulIplesubjects• Partlycoveredfaces/bodies• Trackingfaceorbodymovement• LighIngcondiIons
• IntegraIngtechniques• SupporIngapplicaIondevelopment
©2006-2014,AlfredoSánchez
Emotion in UX evaluation
©2006-2014,AlfredoSánchez
Evaluating user experience with novel technologies – impact of emotion
©2006-2014,AlfredoSánchez[Magallanes, Molina, Méndez & Sánchez 2012]
Gestures + Contextual Menus Heuristic evaluation
• Problemsobservedinactualuse
©2006-2014,AlfredoSánchez
• IssuesidenIfiedbyexpertsID Issue Occurrences
P1 Movement of objects using two fingers. 13
P2 Size of the keyboard. 8 P3 Finding help for the user. 5 P4 Finding the toolbar associated
with each object. 4
P5 Navigating using the fan menu. 3 P6 Navigating between nodes. 3 P7 Identifying the icon for linking to
new pages 2
P8 Incomplete or inaccessible menus. 1
ID Task Issue
1 Insert two images, change their position and size. P1
2 Enter a text next to the images as a caption. P2
3 Obtain the toolbar for an image. P3, P4
4 Pop a menu and discover how it works. P5
5 Build a hierarchy of linked workspaces. P6, P7
6 Get the menu in different positions. Cause the menu to go off the screen.
P8
7 Insert two images, change their position and size. P1
8 Enter a text next to the images as a caption. P2
2/21/16
8
Additional feedback on emotions, based on Product Emotion Measurement Tool (PrEmo)
Disgust Indignation Contempt Dissatisfaction Disappointment Unpleasantsurprise
boredom
Desire Inspiration Admiration Satisfaction Fascination Pleasantsurprise
Amusement
[Desmet 2002]
ParIcipantswereaskedtoselectananimatedrepresenta:onincorrespondencewiththeiremoIonalreacIons
©2006-2014,AlfredoSánchez
[IncollaboraIonwithUniversidaddelCauca,Colombia]
Positive emotions in spite of problems!
©2006-2016,AlfredoSánchez!
! !PosiIveemoIonsassociatedwithnovelty?
NegaIveemoIonsassociatedwithknowninterfaceelements?[Magallanes, Molina, Méndez & Sánchez 2012]
Implications
• TechniquesforevaluaIonofusabilityanduserexperienceareneededthattakeaffectintoaccount
• FrameworkdevelopedatUniCauca,Colombia[Méndez,Collazos,Granollers,Sánchez2014;Méndez2015]
©2006-2014,AlfredoSánchez
EmoVal
• AframeworkforemoIonalevaluaIondesignedtoenrichexisIngUXevaluaIonmethods
• Threemajorcomponents• EmoVal-C:Behavioralevalua:on(facialexpressions,gaze,posture,movements)
• EmoVal-S:Self-reportofemo:onswhileusingasystem• EmoVal-F:Physiologicalsensing,mainlybrainacIvityduringsystemusage
• SupporIngwebsite(h8p://artemisa.unicauca.edu.co/~ymendal/EmoVal)
©2006-2014,AlfredoSánchez[Méndez2015]
EmoVal: Detailed support / facial expressions
©2006-2014,AlfredoSánchez[Méndez2015]
EmoVal: Detailed support / posture and movement
©2006-2014,AlfredoSánchez [Méndez2015]
2/21/16
9
Building capabilities for affect detection
©2006-2014,AlfredoSánchez
Current scenario: Extending an app to include more techniques is not straightforward
Facial&cue&analysis&Monitoring&pilot&alertness&
Lie&detec4on&
Usability&tes4ng&
Online&tutoring&
Posture&analysis&
Voice&analysis&
Keystroke&dynamics&
Eye&tracking&
Naviga4on&pa@erns&
ApplicaIondomain Technique Technologies
©2006-2014,AlfredoSánchez
Goal: Integration platform
Usability)tes+ng)
Eye)tracking)
Vikara)Integra+on)Services)
Vikara)Monitoring)Services)
Lie)detec+on)
Online)tutoring)
Alertness)monitor)
Naviga+on)pa@erns)
Keystroke)dynamics)
Voice)analysis)
Posture)analysis)
Facial)cue)analysis)
Visualiza+on)interfaces)
©2006-2014,AlfredoSánchez[Sánchezetal.2015]
Architecture for implementation
Database
Controller EmotionsController
View
Data Model
Home - Index - UploadUserContent
HomeController
SessionsController
Vikara Services Interface
Data entities
cshtml Session - Index - List
EmotionModel
UserContentController
FileController
Emotion
UserContent Session
MS SQL
Resources Google Chart Tools
WebGrease Modernizr
Android Emotion Interpreter
Windows Emotion Interpeter
Server
Visualization interfaces
Other Emotion Interpreters
Observer / administrator
End
user
s
©2006-2014,AlfredoSánchez[Sánchezetal.2015]
Simple data model and messaging protocol
!©2006-2014,AlfredoSánchez
[Sánchezetal.2015]
Prototypical apps: Kinect-based emotion detection
©2006-2014,AlfredoSánchez
14sorpresa
[Sánchezetal.2015]
2/21/16
10
Prototypical apps: Kinect-based emotion detection
©2006-2014,AlfredoSánchez[Sánchezetal.2015]
Prototype evaluation
!
©2006-2014,AlfredoSánchez[Sánchezetal.2015]
Prototypical apps: Android-based emotion self reporter
©2006-2014,AlfredoSánchez
![Sánchezetal.2015]
Prototype: Monitoring services
©2006-2014,AlfredoSánchez[Sánchezetal.2015]
UX for BCI
h8p://www.gadgeIfy.com/w
p-conten
t/up
loads/2014/09/15/epo
c.pn
g
Experimenting with EPOC+
2/21/16
11
What about sentiment?
©2006-2014,AlfredoSánchez
Sentiment analysis
• BydefiniIon,TheuseofNLPandcomputaIonallinguisIcstoidenIfyaLtudes
inspeakersorwriterswithrespecttospecificsubjects
©2006-2014,AlfredoSánchez
h8p://datameer2.datam
eer.c
om/blog/wp-conten
t/up
loads/2012/10/22nd
.png
©2006-2014,AlfredoSánchez
Applications of sentiment analysis
DetecIng/studying:• ResponsestomarkeIng/poliIcalcampaigns• UsersaIsfacIon• Biasinnews• Appropriatenessofcontentsfordifferentaudiences
©2006-2014,AlfredoSánchez
Some techniques
• MostlyheurisIcmethods• Socialmediaanalysis
• Keyword-based• SemanIcapproaches• Annotateddatasets• SupportVectorMachines(SVMs)forclassificaIon
©2006-2014,AlfredoSánchez
ExpressionArchitecture
AflexiblearchitectureenablesExpressiontolistenfromanydatasourceandhaveanindependentcomponent-basedfront-end.
Expression–SenImentAnalysisPlazormatUDLAPArchitecture
[Cervantes,GuIérrez,GuIérrez,CasIllo,Sánchez,Wan2015]
2/21/16
12
ThefrontendofExpressionisbasedonsen:mentscardsthatrepresenttopicsofinterest.
SenImentcards
h8ps://github.com/Innova4D/expression
ASen:mentCardgraphicallysummarizesopiniontrendsinsocialmedia
SenImentCard
Some challenges
• LanguagesubtleIes–Irony,sarcasm,abbreviaIons,misspellings,idioms
• RepresentaIveness–DissaIsfieduserstweetmoreacIvely• ContextdependentsemanIcs• Spam• ClassificaIonbeyondposiIveornegaIve
©2006-2014,AlfredoSánchez
Affect mediation
©2006-2014,AlfredoSánchez
Mediation: Conveying affect through ICTs
• Videoconferencing
• Immersiveenvironments
• AsynchronouscommunicaIon• Email,forums,blogs…
©2006-2014,AlfredoSánchez
Some issues and challenges • AudioandvideoqualityoverInternet
• QualityofmulImediaexperience(h8p://i-guide.org/)
• Metrics,assessmentmethods
• Gazeawareness
• 3Dmodeling• Issuessharedwithaffectsynthesis
©2006-2014,AlfredoSánchez
2/21/16
13
Emotion over low data rates: Sending codes instead of full video
IncollaboraIonwithUniversityofAgder,Norway
h8p://w
ww.pau
lekman.c
om/fac
s/
1. Sensing2. Encoding3. TransmiYng4. ReconstrucIng
[Lamei,Li,Sánchez2014]
Conveying mood
©2006-2014,AlfredoSánchez
Instant messaging • EmoIconsworkforemoIons,instantreacIonsassociatedtospecificstatements
• Moodislongerlas:ng,overarching• IdenIfyingmoodcanbeachallenge,evenforthepersoninthataffecIvestate
©2006-2014,AlfredoSánchez
Russell’s Circumplex Model can help identify mood
• Two-dimensional,valence-arousalmodel
• ConsistentspaIallocaIoninself-reportexperimentsacrossindividualsandacrosscultures/languages
©2006-2014,AlfredoSánchez
[Russell1983]
Extending Russell’s model - Intensity
©2006-2014,AlfredoSánchez
Extended, discrete model
©2006-2014,AlfredoSánchez
UsertesIngshowedaccuracyinaffectidenIficaIonButusingthemodeldirectlywouldresultinaclu8eredinterface
2/21/16
14
Gradual revelation
annoyed
happy
sad
calm
Representative affectives states per quadrant (according to user testing)
sleepy
aroused
upset satisfied
©2006-2014,AlfredoSánchez
Emoticon design
©2006-2014,AlfredoSánchez
Prototype (“Russkman”)
[Sánchezetal.2006]©2006-2014,AlfredoSánchez
Prototype
[Sánchezetal.2006]©2006-2014,AlfredoSánchez
Prototype
[Sánchezetal.2006]©2006-2014,AlfredoSánchez
Evaluation
• Chatsessionsamong2-3subjects• HypotheIcalsituaIons(upcomingexams,goodgrades,recentparty,lostwallet…)
• Subjectswereabletospecifytheirmoodrapidly• SubjectswereabletoidenIfymoodofotherusers(highcorrelaIon)• PersistentemoIconshelpedinkeepingawarenessofmoods
©2006-2014,AlfredoSánchez
2/21/16
15
Affect synthesis: Modeling and displaying
emotion and mood
Synthesizing emotions
• IfanemoIon(ormoodorsenIment)isrecognizedintheuser,thecomputer(oranadapIveenvironment)could(should?)respondaccordingly
• ApproachestoemoIonsynthesis• ReasoningaboutemoIons—givenasetofcondiIons,producespecificreacIons
• Characterswithpersonality(temperament,senIment)
©2006-2014,AlfredoSánchez
Emotion synthesis Characterswithanimatedfacialexpressions(facialcuesforsensingalsoapplyhere)
[CSLU]
©2006-2014,AlfredoSánchez
Responding to emotion
AffecIveCompanionTextfromautomaIcspeechrecognizerandEmoVoice(fortheuser)Naturallanguageunderstanding [Cavazzaetal.2010]
©2006-2014,AlfredoSánchez
FACS applied to emotion synthesis
FACSGe
n[Roe
schetal.2010]
©2006-2014,AlfredoSánchez
Affective animated avatars
• DesignusingpopulartoolsandareducedsetofFACS’AUs
• Availableath8p://chopis.com
• TesInginMéxicoandChina
[Sánchezetal.2014]©2006-2014,AlfredoSánchez
2/21/16
16
Some challenges
• Prototypicalvs.non-prototypicalemoIons• Purevs.combinedemoIons• IntegraIngfacial,postureandvoiceexpressions• OfferingserviceplazormsforappconstrucIon
©2006-2014,AlfredoSánchez
Perspectives of affective UX
• AffectiskeyinUXdesignandevaluaIon
• OpportuniIesforresearchanddevelopmenttoimprovedetecIon,synthesisandmediaIon
• WorkneededinmoodandsenImentanalysis
• OpportuniIesforinter-disciplinarycollaboraIon
©2006-2014,AlfredoSánchez
Affective computing becoming mainstream?
©2006-2014,AlfredoSánchez
“Pepper”,So}Bank’saffect-awarerobot,1,000unitssoldout,December2015,foraboutUSD1,800…InvolvesfacialrecogniIon,videoandaudioprocessing,sensors,machinelearning,…
But wait…
• Forward-lookingstatementsarebytheirnaturesubjecttovariousrisksanduncertain:es,including,withoutlimitaIon,adeclineingeneraleconomiccondiIons,generalmarketcondiIons,technologicaldevelopments,changesincustomerdemandforproductsandservices,increasedcompeIIon,risksassociatedwithinternaIonaloperaIons,andotherimportantfactors,eachofwhichmaycauseactualresultsandfuturedevelopmentstodiffermateriallyfromthoseexpressedorimpliedinanyforward-lookingstatement.[…]Useoforrelianceontheinforma:oninthispresenta:onisatyourownrisk.
h8p://webcast.so}bankmobile.co.jp/en/conference/20140605/index.html
©2006-2014,AlfredoSánchez
01 WORLD IA DAY 2016 PRESENTATION TITLE HERE
HEADER OPTION SUB HEAD OR SHORT DESCRIPTION!
Some kind of explanatory text, reference or footnote can go here and wrap to two lines, if needed.
Some kind of illustration or image?
Affective UX"Challenges in UX design involving affective computing
Alfredo Sánchez
Universidad de las Américas Puebla http://ict.udlap.mx
References • Cervantes,O.,GuIérrez,F.,GuIérrez,E.,CasIlloE.,Sánchez,J.A.,Wan,W.2015.Expression:UnderstandingaffecIvecontentinsocialmedia.7thLaInAmericanConferenceonHuman-computerInteracIon(CLIHC2015),Cordoba,ArgenIna,Nov.18-23.AvailablefromtheACMDigitalLibrary(h8p://dl.acm.org/citaIon.cfm?doid=2824893.2824903).
• Magallanes,Y.,Molina,A.,Méndez,Y.,Sánchez,J.A.2012.CombininggesturesandgraphicalelementsforcollaboraIonusingmulItouchsurfaces.Proceedingsofthe22ndInternaIonalConferenceonElectronics,CommunicaIonsandComputers(CONIELECOMP2012,Feb.).IEEEComputerSocietyPress,173-178.
• Magallanes,Y.,Molina,A.,Méndez,Y.,Sánchez,J.A.2012.TowardsanemoIonalvalidaIonofheurisIcapproachesforusabilityevaluaIon.ActaUniversitaria22,3,119-125.(h8p://www.actauniversitaria.ugto.mx/index.php/acta/arIcle/view/351).
• Méndez,Y.A.,Collazos,C.A.,Sánchez,J.A.,MerchánParedes,L.2012.CriteriosparalaevaluacióndesuperficiesmulItácIlesysusaplicaciones.Lámpsakos7(Jan.),64-68.(disponibleenh8p://www.funlam.edu.co/revistas/index.php/lampsakos/arIcle/view/847).
• Méndez,Y.A.2015.Marcodereferenciaparalacapturasemi-invasivayanálisisdelasemocionesenelprocesodeevaluaciónconusuariosdesistemasinteracIvos.Tesisdoctoral.UniversidaddelCauca,Colombia.Disponibleenh8p://www.unicauca.edu.co/doctoradoce/publicaciones/Monografia_Mendez.pdf
©2006-2014,AlfredoSánchez
2/21/16
17
References • Sánchez,J.A.,Hernández,N.P.,Penagos,J.C.,Ostróvskaya,Y.2006.ConveyingmoodandemoIonininstantmessagingbyusingatwo-dimensionalmodelforaffecIvestates.ProceedingsoftheSymposiumonHumanFactorsinComputerSystems(IHC2006,Natal,Brazil),66-72.AvailablefromtheACMDigitalLibrary(h8p://dl.acm.org/citaIon.cfm?id=1298033&CFID=582956970&CFTOKEN=61490837).
• Méndez,Y.A.,Collazos,C.A.,Granollers,T.,Sánchez,J.A.2014.EvaluaInginteracIvesystemsfromanemoIonalperspecIve.RevistaCien�ficaGuillermodeOckham12(1),43-49.(disponibleenh8p://www.redalyc.org/pdf/1053/105332478005.pdf).
• Sánchez,J.A.,Cortés,X.,Starostenko,O.,Cervantes,O.,Wan,W.2015.AnExtensiblePlazormforSeamlessIntegraIonandManagementofApplicaIonsforEmoIonSensingandInterpretaIon.JournalofAmbientIntelligenceandSmartEnvironments7(1),5-19.(availablefromh8p://iospress.metapress.com/content/y1g64422t78107v7).
• Starostenko,O.,Cortés,X.,Sánchez,J.A.2015.UnobtrusiveemoIonsensingandinterpretaIoninsmartenvironments.JournalofAmbientIntelligenceandSmartEnvironments7,59-83.(avalilablefromh8p://iospress.metapress.com/content/w2681p2827l2u22p/).
©2006-2014,AlfredoSánchez