Post on 15-Jan-2016
description
ShareCam: Interface, System Architecture, and Implementation of a
Collaboratively Controlled Robotic Webcam
Dezhen Song (TAMU) and Ken Goldberg (UC Berkeley)
IEEE/RSJ International Conference on Intelligent Robots and SystemsFinalist for New Technology Foundation (NTF) Award for
Entertainment Robots and Systems
Teleoperation: Related Work• Tesla, 1898• Goertz, ‘54• Mosher, ‘64• Tomovic, ‘69• Salisbury,Bejczy, ‘85• Ballard, ’86• Volz, ’87-• Sheridan, ‘92• Sato, ’94• Goldberg, ’94-• Presence Journal ‘92-• O. Khatib, et al. ’96
Internet
networkedrobot:
Taxonomy (Tanie, Matsuhira, Chong 00)
Multiple Operator, Single Robot (MOSR):
Single Operator, Single Robot (SOSR):
Single Operator, Multiple Robot (MOSR):
Pan, Tilt, Zoom robotic video camera
sharecam
Entertainment Applications
Playing Games
Related Work• Networked robots
– Tanie, K., Chong, N. et al(01)– Jia, S. and K. Takase (01)– Hu, H., Yu, L., Tsui, P., Zhou, Q (01)– Safaric, R. et al. (01)– Goldberg and Siegwart (02)– Coppin, P. and Wagner, M.D. (02)– Konukseven, I., Erkmen, A. et al (02)
• SOSR– Siegwart, R. and Saucy P. (99)– Paulos, E. and Canny, J. (99)– Tanie, K., Arai, H. et al. (00)– Lynch, K. and Liu, C. (00)– Fong, T., Thorpe, C., et al(01)
Related Work• SOMR
– Hu, Yu, Tsui, Zhou (01) – Jia, Takase (01)
• MOMR– Fukuda, Xi, Liu, Elhajj et al. (00,02)– Tanie, Chong, et al. (00)
• MOSR– Cinematrix (91)– Cannon, McDonald, et al. (97) – Goldberg, Chen, et al. (00, 01)
Sharecam: System Architecture
Users
Internet
ShareCam Server
Video Server
Sharecam Software
TCP/IP
TCP/IP
User database
RegistrationCore (with shared memory segments)
Apache module
Apache module
Apache module
Communication
Console/Log Login CGI
ShareCam web server
ShareCam applet InetCam applet
Client
RS232C
HTTP
Camera control
Calibration
Panoramic image generation
InetCam server
Video server
Canon VC-C3 Camera
Java
Gnu C++
PERL
MySQL
frame selection problem: given n requests, find optimal frame
Problem Definition
Requested frames: i=[xi, yi, zi], i=1,…,n
Problem Definition• “Satisfaction” for frame i: 0 Si
1
Si = 0 Si = 1
= i = i
•Symmetric Difference
•Intersection-Over-Union
SDArea
AreaIOU
i
i
1)(
)(
)(
)()(
i
ii
Area
AreaAreaSD
Similarity Metrics
Nonlinear functions of (x,y)
Intersection over Maximum:
),(
)(
),max(
)1,)/min(()/(),(
i
i
i
i
biiii
Max
Area
aa
p
zzaps
Requested frame i , Area= ai
Candidate frame
Area = api
• global satisfaction:
n
iii
n
i
biii
yxpyxS
zzapS
1
1
),(),(
)1,)/min(()/()(
for fixed z
find * = arg max S()
approximation
n)dd
whgO(
spacing resolution :
spacing lattice :
z2
zd
d x
y
d
Compute S(x,y) at lattice of sample points:
w, h : width and height,g: size range
error bound
run time:O(n / 3 )
c* Optimal frame
Optimal at latticec~
Smallest frame on lattice that encloses c*
c
)ˆ()~()( * cscscs
)(
)ˆ(
)(
)~(1
** cs
cs
cs
cs
zdz
z
2...
min
min
1
Processing Zoom Type Complexity
Centralized Discrete Exact O(n2)
Centralized Discrete Approx O(nk log(nk)), k=(log(1/ε)/ε)2
Centralized Contin. Exact O(n3)
Centralized Contin. Approx O((n + 1/3) log2 n)
Distributed Discrete Exact O(n), Client: O(n)
Distributed Contin. Approx O(n), Client O(1/3)
frame selection algorithms
ShareCam Application:Game based learning : global environment
robotic video cameras
motion sensors
timed checks
sensor networks
humans: amateurs and profs.
Collaborative Observatories for Natural Environments (CONEs)
Frank van der Stappen (CS, Utrecht)Vladlen Koltun (EECS, UC Berkeley)George Bekey (CS, USC)Karl Bohringer (CS, UW)Anatoly Pashkevich (Informatics, Belarus) Judith Donath (Media Lab, MIT) Eric Paulos (Intel Research Lab, Berkeley)Dana Plautz (Intel Research Lab, Oregon)Sariel Har-Peled (CS, UIUC)
Thank you.ShareCam: Interface, System Architecture, and Implementation of a
Collaboratively Controlled Robotic Webcam
Satellite Imaging
MIT Press, 2002
Networked Robots
• Tele-Operation• Internet Tele-Operation• Collaborative Tele-Operation• Tele-Actor• Co-Opticon• Co-Opticon Algorithms
www.ken.goldberg.net
Infiltrate
NetworkedRobots
internettele-robot:
RoboMotes: Gaurav S. Sukhatme, USC
Smart Dust: Kris Pister, UCB (Image: Kenn Brown)
NetworkedCameras
Where to look?
Sensornet detects activity• “Motecams”
• Other sensors:audio, pressure switches,
light beams, IR, etc
• Generate bounding boxes
and motion vectors
• Transmit to PZT camera
Activity localization
1. Network Standards: HTML, Browsers, Java
2. Infrastructure: Backbone, Routers, ADSL
3. Public Adoption
4. Bandwidth: 10 Mbps, 100Mbps, Gbps
5. Video/Audio Compression: MP2,3,4
Networked robots Systems that couple communication to control of one or more robots in a network that often includes sensors and remote human operators.
Two subclasses : 1) Tele-operated, where human supervisors send commands
and receive feedback via the network. Such systems support research, education, and public awareness by making valuable resources accessible to broad audiences.
2) Autonomous, where robots and sensors exchange data via the network. In such systems, the network extends the effective sensing range of individual robots.
Challenges:
1. Variable Time Delays, Congestion
2. Latency
3. Access Control, Security, Interfaces
4. Protocol Design
5. Noise, Error Detection and Recovery
6. Deployment, Dynamic Routing
7. Power Management
8. Hybrid Architectures
Conventional Security Cameras
• Immobile or Repetitive Sweep• Low resolution
Future Work
• Continuous zoom (m=)• Multiple outputs:
– p cameras – p views from one camera
• “Temporal” version: fairness– Integrate si over time: minimize accumulated
dissatisfaction for any user
• Network / Client Variability: load balancing
• Obstacle Avoidance
Outline
• Collaborative Teleoperation
• Cinematrix• Co-opticon• Tele-Twister
"In times of terror, when everyone is something of a conspirator,everyone will be in a situation where he has to play detective." -- Walter Benjamin (1938)
Statistics of Satellite Imaging
• 2.5 Billion Market in 2003• Increasing 300% per year
since 1999• Major clients
– Government / Military– Oil exploration– Weather Prediction– Agriculture
Ikonos, 1999
Intersection over Maximum: si( ,i)
si = 0.20 0.21 0.53
Requested frame i
Candidate frame
• Staircase Approximation– Exact algorithm O(n3/2 log3 n) [data
structure]
– Approximation Algorithm O(nk log(nk)), k=(log(1/ε)/ε)2
• Staircase approximation, large constant factors