Cognitive Assistant for the Blind
-
Upload
diannepatricia -
Category
Technology
-
view
292 -
download
1
Transcript of Cognitive Assistant for the Blind
Cognitive Assistant for the Blind�Chieko Asakawa,
IBM Fellow,
IBM Research
Kris Kitani,
Robotics Institute,
Carnegie Mellon University
1
© 2016 IBM Corporation
18th century, the mathematician Euler authored a leading textbook on mathematics after he became blind.
Captions for TV programs were invented for deaf people, but are now widely used in sports bars and to help immigrants, etc.
Based on Self-‐‑‒standing wheelchair.
2011, voice dialog technologies for smartphones
TelephoneKeyboard
Voice recognition
Voice dialogTV Captions
Voice synthesis and OCR
Modern Mathematics
Automatic vehicles
Segway
“Blind driving challenges”
Early use of keyboards was a to help people with hand-‐‑‒and-‐‑‒motor disabilities in writing
First consumer product was a reading assistant for the blind in the 1970s.
1876, Alexander Graham Bell invented the phone after teaching hearing impaired people.
Smart Machines
2
17th century, Dom Perignon. Champagne
Accessibility Needs Ignite Innovation
1960s Talking Typewriter
1960s Talking Typewriter
1975 1403 Braille Printer
1984 Talking 3270 Terminal
1990 VoiceType™
2008 Social Accessibility
1998 ViaVoice®
2000 Accessibility Center
1997 Home Page Reader
1988 ScreenReader/DOS
2004 aDesigner
2007 Eclipse Accessibility Tools Framework
2009 Rich Internet Accessibility Technologies
1914 First IBMer with disabilities
2012 Brazil Accessible Vocational Training Project
2011 Senior Workforce Project
1975 1403
Braille Printer
IBM History of Accessibility
3 © 2016 IBM Corporation
Progress in Information Accessibility
Exponential increase of accessible information (Conceptual chart)
1980 1990 2000 2010
Paper Braille
Digital Braille
Voice Web access
4
Am
ount
of a
cces
sibl
e in
form
atio
n
© 2016 IBM Corporation
5
Walk by myself Finding discount posters Finding restaurants Shopping
Many things are difficult…
Challenges of Real World Accessibility
© 2016 IBM Corporation
Chika
6
Chika from “Esper the Light Speed”
© 2016 IBM Corporation
Natural Language
Processing
Infer from Knowledge
Predict the Future
Natural Interaction
Learning Senses
Watson 1.0, 2.0, …
Apply Watson Modeling
and Simulation
Neural Chip
SyNAPSE Image
Processing Speech
Recognition Technology
Traffic Simulation
Dialog System
Question and Answering System
7
Challenges of “Cognitive Computer”
© 2016 IBM Corporation
8
Demonstration Video
© 2016 IBM Corporation
Localization Radio-wave
Vision Dead reckoning
Recognition Objects People
Environment
Knowledge Personal
Social media Encyclopedic
Interaction Speech Gesture
Sonification
Building Blocks of Cognitive Assistant
9 © 2016 IBM Corporation
Understanding the Functionality of the Environment
Nick Rhinehart and Kris Kitani, CVPR 2016
© 2016 IBM Corporation
When we observe a scene…
… we know how we can act in that environment
© 2016 IBM Corporation
When we observe a scene…
… we know how we can act in that environment
Walk
Walk
© 2016 IBM Corporation
When we observe a scene…
… we know how we can act in that environment
Sit Sit
Sit
© 2016 IBM Corporation
When we observe a scene…
… we know how we can act in that environment
Open
Open Open
© 2016 IBM Corporation
Can we teach a computer to
understand scene functionality by
observing people?
© 2016 IBM Corporation
Setup:
One wearable camera
Detect actions using computer vision
Learn the function of an environment
© 2016 IBM Corporation
© 2016 IBM Corporation
© 2016 IBM Corporation
© 2016 IBM Corporation
21
Information Accessibility Progress (revisited)
1980 1990 2000 2010
Paper Braille
Digital Braille
Voice Web access
Am
ount
of a
cces
sibl
e in
form
atio
n
Exponential increase of accessible information (Conceptual chart)
2020 2030
2016
© 2016 IBM Corporation
“Real-world” information access