Ground Veri�cation of the Feasibility of Telepresent

On-Orbit Servicing

Enrico Stoll

Ulrich Walter

Institute of Astronautics (LRT)Technische Unversität München

Garching, Germanye.stoll@mytum.dewalter@tum.de

Jordi Artigas

Carsten Preusche

Philipp Kremer

Gerd Hirzinger

Institute of Robotics and MechatronicsGerman Aerospace Center (DLR)

Oberpfa�enhofen, Germanyjordi.artigas@dlr.de

carsten.preusche@dlr.dephilipp.kremer@dlr.degerd.hirzinger@dlr.de

Jürgen Letschnik

LSE Space Engineering & Operations AGWessling, Germany

juergen.letschnik@lsespace.com

Helena Pongrac

Human Factors InstituteUniversity of the Bundeswehr

Neubiberg, Germanyhelena.pongrac@unibw.de

Abstract

In an ideal case telepresence achieves a state, where a human operator can nolonger di�erentiate between an interaction with a real environment or a tech-nical mediated one. This state is called transparent telepresence. The applica-bility of telepresence to on-orbit servicing (OOS), i.e. an unmanned servicingoperation in space, teleoperated from ground in real time, is veri�ed in thispaper.For that purpose, a communication test environment was set up on ground,which involved the Institute of Astronautics (LRT) ground station in Garching,Germany and the ESA ground station in Redu, Belgium. Both were connectedvia the geostationary ESA data relay satellite ARTEMIS. Utilizing the datarelay satellite, a teleoperation was accomplished, in which the human opera-tor as well as the (space) teleoperator was located on ground. The feasibilityof telepresent OOS was evaluated, using an OOS test bed in the Institute ofMechatronics and Robotics at the German Aerospace Center (DLR). The ma-nipulation task was representative for OOS and supported real time feedbackfrom the haptic-visual workspace. The tests showed that complex manipula-tion tasks can be ful�lled by utilizing geostationary data relay satellites.For verifying the feasibility of telepresent OOS, di�erent evaluation methods

were used. The properties of the space link were measured and related to sub-jective perceptions of participants, which had to ful�ll manipulation tasks. Anevaluation of the transparency of the system, including the data relay satellite,was accomplished as well.

1 Introduction

Spacecrafts are the only complex engineering systems without maintenance and repair in-

frastructure. Occasionally, there are space shuttle based servicing missions, starting with

the Solar Maximum Repair Mission (SMRM) in 1984, but there are no routine procedures

foreseen for individual spacecrafts. Most malfunctioning spacecrafts require only a minor

maintenance operation on orbit, a so-called On-Orbit Servicing (OOS) mission, to continue

operational work. Instead, they have to be replaced due to the lack of OOS opportunities.

The accomplishment of OOS missions would, similar to terrestrial servicing procedures, be

of great bene�t for spacecraft operators, since a wide spectrum of use cases exists as e.g.

spacecraft assembly, orbit transfer, maintenance and repair, resupply, or even safe deorbiting.

1.1 Motivations

Following the SMRM, there were several Space Transportation System (STS) based ser-

vicing missions (e.g. Intelsat VI (F-3) and the Hubble Space Telescope). These incipient

OOS missions did not only demand a complex and cost intensive shuttle mission, but

also the application of Extra-Vehicular Activities (EVA). Astronauts had to leave the safe

environment of the space station in order to retrieve and repair the spacecrafts in outer

space, only protected by their suits.

Based on the criticality of an EVA, concepts of robotic applications have been developed

that can be controlled by astronauts. However, such devices like the Special Purpose

Dexterous Manipulator (SPDM) by the Canadian Space Agency (Mukherji et al., 2001) or

the robotic astronaut (Robonaut) (Peters II and Campbell, 1999), developed by the National

Aeronautics and Space Administration (NASA), are in situ teleoperated by astronauts (from

the Space Shuttle or the International Space Station) and demand still the use of an STS.

In contrast to this manned OOS missions, there are options to accomplish OOS missions

unmanned. As it will be seen in section 1.2, unmanned spacecrafts (servicer satellites) are

foreseen to accomplish OOS operations at a target satellite. For that purpose the explorative

and manipulative possibilities of robots will be exploited to dock the servicer satellite with

the malfunctioning target satellite and execute complex operations, remotely controlled

from ground.

While there is much research undertaken on spacecraft autonomy, there are only a few space

projects considering a telepresent control of the spacecraft, which is of special interest for the

work, presented here. A telepresent control includes a human operator in a ground station

controlling the robotic application and receiving instantaneous (visual and haptic) feedback

from the spacecraft to the actions. This work analyzes, whether the concept of telepresence

(control) is applicable to OOS. For that purpose a test environment, which focuses on space

applications in Low Earth Orbit (LEO) was developed and set up on ground.

OOS in LEO is a special problem, since direct contact between a ground station and the

servicing spacecraft is only given in small time intervals. However, the feasibility of OOS

operations is highly dependent on whether and how long a communication link between

the controlling ground station and the servicer spacecraft can be established. Tab. 1 will

clarify the reason. The space shuttle based OOS missions of the Hubble Space Telescope

(HST) are listed. Each of them required several EVAs resulting in a total EVA time of more

than 24 hours. An OOS mission, which is telepresently controlled from ground demands an

equivalent amount of contact time.

flight year number of EVAs total EVA time

STS - 61 1993 5 35 h 28 min

STS - 82 1997 5 33 h 11 min

STS - 103 1999 3 24 h 33 min

STS - 109 2002 5 35 h 55 min

Table 1: Space Shuttle based Hubble Space Telescope OOS missions

Using direct communication in LEO would accrodingly require several weeks or a complex

ground station network. Since the HST orbits the Earth at approximately 570 km, 4-8

orbit revolutions per day exist, in which a human operator could steer a robotic servicer for

maximum 10 minutes per orbit revolution (Lundin and Stoll, 2006). Thus, for accomplishing

complex OOS operations, the use of geostationary satellites is proposed, which increases the

mean acquisition time of the spacecraft in LEO up to more than 1 hour per orbit revolution.

The use of geostationary data relay satellites in turn, increases the round trip delay of the

signal, that is, the time between operator action and spacecraft feedback. The main goal of

this work is to prove that the utilization of geostationary (GEO) data relay satellites for

OOS is reconcilable with a telepresent control of the servicer spacecraft.

1.2 State of the art OOS technology demonstrators

This section gives a brief overview of OOS technology demonstrators that were brought to

orbit, that are still orbiting Earth, or pending. The emphasize is hereby, placed on the com-

munication architecture (direct or relayed contact) and the manner of control (telepresence

or autonomy).

The �rst robot in space, which has been remotely controlled from ground, was the Robot

Technology Experiment (ROTEX ) aboard the space shuttle Columbia in 1993. The operatio-

nal modes were tele-sensor-programming (learning by showing), automatic (pre-programmed

on ground), and teleoperation on-board (an astronaut controlled the robot using a stereo

monitor). Further, a teleoperation by a human operator from ground, using predictive com-

puter graphics, was performed. (Hirzinger et al., 2004)

ETS VII is a Japanese Engineering Test Satellite (ETS) capable of demonstrating bilateral

teleoperation in space (Imaida et al., 2004). The spacecraft, consisting of a pair of satellites,

was launched in 1997. Autonomous capturing of the smaller target satellite, inspection pro-

cedures and a series of manipulation operations was demonstrated (Oda, 2000).

The Robotic Component Veri�cation aboard the ISS (Rokviss) is a German space technology

experiment, which was installed in 2005 outside the International Space Station (ISS) at the

Russian service module (Landzettel et al., 2006). Rokviss is a two joint robotic manipulator,

controlled by a human operator via a direct radio link from the ground station in Weilheim,

Germany (Preusche et al., 2006).

The Experimental Satellite System 10 (XSS-10 ) (Davis, 2005) was developed by the US Air

Force. The space mission was launched in 2003 and the mission objectives included autono-

mous navigation and proximity operations.

The Experimental Satellite System 11 (XSS-11 ) (AFRL, 2007) was a micro satellite of

approximately 100 kilograms. Launched in 2005, XSS-11 has been designed for testing auto-

nomous technologies necessary for the inspection of malfunctioning satellites.

All operations of the Demonstration of Autonomous Rendezvous Technology (DART ) (Rum-

ford, 2003) mission were developed to be autonomous. Launched to verify hardware and soft-

ware for rendezvous and proximity operations, the main objectives were the demonstration

of station keeping and collision avoidance maneuvers. However, when DART approached the

target, it overshot an important waypoint, and thus, the pre-programmed transition to the

target satellite, and collided with it. A premature retirement of DART was the consequence.

The mission plan of Orbital Express (BOEING, 2007) foresaw the validation of software for

autonomous mission planning, rendezvous, proximity operations, and docking. Further tests

of robotic OOS scenarios included fuel and electronics transfer, deployment of, and opera-

tions with a micro-satellite. The Swedish Space Cooperation (SSC), together with Kayser-

Threde, Germany and Sener, Spain is developing the SMART Orbital Life Extension Vehicle

(SMART-OLEV ) (Tarabini et al., 2007). It aims at extending the operational life time of

geostationary satellites.

The Deutsche Orbitale Servicing Mission (DEOS ) (Sommer, 2003) will demonstrate diverse

OOS scenarios such as rendezvous, inspection, formation �ight, capture, stabilization, and

controlled de-orbiting of the target and servicer compound. In this connection two modes for

commanding the servicer are foreseen. On the one hand, there will be active ground control

via telepresence, i.e. a control with instantaneous feedback to the human operator. On the

other hand, it will also be possible to passively monitor autonomous operations from ground.

The ranger robotics program started in 1992 as the Ranger (Parrish and Akin, 1996) Te-

lerobotic Flight Experiment (RTFX) at the University of Maryland, USA. The goal was to

develop a dexterous extravehicular space telerobot with four robot manipulators and a free-

�ight capability in space. In 1996 the program got redirected as a shuttle launch payload

to the Ranger Telerobotic Shuttle Experiment (RTSX), was �nanced till 2001, and never

advanced from an engineering model.

The Spacecraft for the Universal Modi�cations of Orbits (SUMO) (Bosse et al., 2004) is a

project by the Naval Research Laboratory and funded by DARPA. The main goal is to deve-

lop a spacecraft, capable of demonstrating future OOS technologies. A series of autonomous

rendezvous, grapple, and servicing experiments are planned.

Fig. 1 shows a possible classi�cation of the OOS demonstrators. The classi�cation is based

on two criteria. Firstly, the options for communication with the OOS demonstrator (direct or

relay to LEO or direct to GEO)in the respective orbit are considered. Secondly, the round trip

delay between operator command and received feedback is considered. Since a telepresent

operation demands an instantaneous feedback to the operator's action, di�erent natures of

round trip delay are de�ned. They are a �rst indication for the telepresence capability of the

system.

The ideal or transparent telepresence (TTP) is, encouraged by the Rokviss experiments,

here de�ned as being in the magnitude of approximately up to 0.1 s. This allows the user

to obtain instantaneous feedback of the telecommands, and technical means will enable the

human operator to feel present in the removed environment. It is followed by telepresence

(TP), being in the magnitude of approximately up to 1.0 s. This is of special interest when

considering applications, which are either located in GEO or of which the communication

has to be relayed via the GEO. The large physical distance, which the communication si-

gnal has to bypass in both cases, causes a minimum round trip time larger than 0.24 s

and 0.48 s, respectively. Therefore, the round trip delay can be found in the telepresence

branch. Transparent telepresence cannot be achieved for GEO or relay to LEO applications

as emphasized in Fig. 1.

The third branch of the round trip delay criteria, covering round trip delays of approxima-

tely up to the 10.0 s magnitude, is de�ned as telerobotics (TR). This de�nition, encouraged

by the ROTEX experiment, speci�es an operation, in which the operator has to cope with

comparable large time delays. While working in a virtual reality with a 3D model of the real

environment, the human operator receives instantaneous simulated (predicted) feedback to

the actions while they are executed in space a few seconds later and synchronized with the

virtual reality afterwards. It is evident, that this approach is not usable for applications in

which the environment is not su�ciently known, i.e. for an application, of which either the

3D model or its dynamics is not known in detail.

Autonomy and supervisory control schemes, opposing direct telepresence, are considered as

a �eld where the round trip delay is not of importance since the operator on ground is not

actively involved in the operations.

Figure 1 � A classi�cation of OOS demonstrators

1.3 Work overview

As Fig. 1 shows, most OOS demonstrators, which are already on orbit utilized either the

concept of TR or autonomy. Both are for the addressed reasons not reconcilable with the

concept of telepresence. The preconditions for TTP do not hold for relayed communication,

which in turn is necessary for complex and time demanding operations in LEO. This work

considers the telepresence branch in connection with a relay approach as labeled in Fig. 1.

To the authors' knowledge, no telepresence control with haptic feedback was executed via

a geostationary relay satellite before. Thus, the work presents �eld experiments on Earth

that demonstrate a telepresent teleoperation to a servicer in LEO. For that purpose a

test environment was implemented, which included the ground station of the Institute of

Astronautics (LRT) at Technische Universität München, Germany and the ground station

of the European Space Agency (ESA) in Redu, Belgium. Both were connected via the

ESA geostationary relay satellite ARTEMIS to set up a realistic environment. The robotic

teleoperation, which was executed, comprised an OOS test bed developed by the Institute

of Robotics and Mechatronics at the German Aerospace Centre (DLR) in Oberpfa�enhofen.

Section 2 deals with the communication architecture of the test environment, including

the geostationary relay satellite. Section 3 describes in detail the robotic OOS test bed.

Section 4 depicts performance and results of the experiments and clari�es how telepresence

or the degree of telepresence can be evaluated. Section 5 closes the treatment of the ground

veri�cation of telepresence for OOS with concluding remarks and considers future direction

for continuing research.

2 The communication architecture

For verifying the feasibility of telepresent OOS in LEO, i.e. a telepresent servicing with fast

feedback and relayed communication, a representative test environment was setup on ground.

For that purpose the two ground stations in Germany and Belgium were connected via the

GEO ESA relay satellite ARTEMIS.

2.1 The geostationary relay satellite ARTEMIS

The Advanced Relay Technology Mission (ARTEMIS) (Moens et al., 2003) was chosen to be

the primary connector between human operator and teleoperator, both located on ground.

ARTEMIS is the �rst European relay mission. It aimed at demonstrating new telecommu-

nication techniques for data relay and mobile services. Being launched in 2001 and designed

for 10 years, it is now situated at approximately 21.45°E and is able to provide service to

Africa and Europe.

Figure 2 � ARTEMIS and its antennas (Moens et al., 2003) (left) and ARTEMIS linkde�nitions (right)

Besides an optical link terminal, an L-band payload and a Ku-band payload as depicted

in Fig. 2 (left), ARTEMIS features an S-and K-band Data Relay (SKDR) payload.The

latter was in the focus of this work since the already existing LRT ground station utilized

the S-band frequency range. Using the SKDR payload, ARTEMIS communicates with its

supporting ESA ground station in Redu, Belgium. Thereby, the feeder links (forward /

return) are in Ka-band and in K-band, respectively. The link de�nitions can be found in

Fig. 2 (right).

In forward direction ARTEMIS can only provide one inter orbit link (IOL) at one time, i.e.

either S-Band or Ka-Band communication is supported. In contrast, in return direction up

to four IOLs can simultaneously be supported. Three of these channels can be allocated

in K-band, while the fourth channel supports the S-band frequency range. Accordingly,

ARTEMIS cannot provide data relay between two communication end points, only using

S-band. The feeder link (FL) only works in K(a)-band. This fact is very important for the

development of the communication architecture, which follows in the next section.

2.2 The mirror approach

The ARTEMIS transponder speci�cations given in the previous section yielded very specia-

lized test setup of the telerobotic test environment (Stoll, 2008). The FL as well as the IOL

of ARTEMIS had to be utilized since ARTEMIS cannot operate the IOL only. The basic

idea was to use both the ARTMEMIS FL and the IOL for communication to ground. The

Institute of Astronautics operates a ground station in the S-band frequency range, which is

suitable for communicating via the IOL with ARTEMIS ; that is, it represents the spacecraft

(S/C) in orbit. Further, the ESA ground station in Redu and its K(a)-band equipment can

be used to establish the FL, as depicted in Fig. 3(left). The human operator (HO) send te-

lecommands (TC) via the data relay satellite (DRS) ARTEMIS to the teleoperator (TOP),

situated in the LRT ground station (GS). After being executed, sensor data of the TOP is

being transmitted as telemetry (TM) back to the OP.

This communication architecture is not suitable for fundamental telepresence experiments,

Figure 3 � Communication architecture : (left) ideal and (right) using the mirror approach

as there is a large spatial distance between human operator in Garching (near Munich),

Germany and teleoperator in Redu, Belgium. This causes an increase of system complexity.

Therefore, the existing experimental system was modi�ed.

The selected communication architecture featured a so-called signal (radio frequency) mirror

as Fig. 3 illustrates. All available radio links were used for transmitting TC data after it is

generated by the HO located in Germany. That way it was possible to locate the TOP in

Germany as well. That way the mirror was introduced in the Redu GS, which is basically

a local bypass at the communication system (in particular the IMBU (see Sec. 2.3). This

means that the TC signal is reinjected immediately after reception into the uplink chain

unchanged. Four hops 1 are used for the TC before the TOP receives and processes it. The

control loop between HO and TOP is closed locally (using TM data) with only one hop,

which features a negligible time delay compared to the length of the TC hops. Thus, the

round trip delays, which the operator perceives, are identical for the ideal communication

architecture and the mirror approach.

The advantage of this mirror approach is evident. The LRT ground segment can be used as

the GS and the S/C of the experiment. Identical communication equipment can be used for

both, since the same communication path (the IOL), with one frequency each for up- and

downlink, is utilized. Further, no complex equipment, which has to be remotely con�gured,

had to be installed at Redu.

2.3 The communication setup

As Fig. 3 shows, the LRT ground segment has to feature all functionalities of a ground

station, as well as of the spacecraft. This means that in forward and return direction

(almost) identical parameter (modulation, packet length et cetera) for con�guring the

communication link had to be used.

Traditional space missions show in contrast a very asymmetrical behaviour considering

downlink and uplink capacity. TM and TC data, usually being in the range of a few kilobits

per second (kbps), is transmitted using a narrowband. Additionally, the data acquisition of

a satellite payload (e.g. synthetic aperture radar applications for Earth observation) may

necessitate the utilization of a broad band with the capability to download several Megabits

per second (Mbps). Tab. 2 exemplarily shows typical maximum uplink / downlink data

rates of space systems. Rokviss is an example, which indeed shows the asymmetry, but not

to such a degree as for example Cryosat. The reason is that the robotic TC data has to be

sent at a very high sampling rate.

For the mirror approach experiments, presented here, this asymmetry had to be repealed.

1. In computer or communication networks one hop is the path between two communication nodes, e.g. betweenrouters on Earth or between a S/C and a GS, when considering space applications.

Uplink and downlink data rate are of the same value since they carry identical data. The

haptic-visual feedback was realized locally, in order to not add delay.

Satellite max. uplink / downlink

Terrasar-X 4 kbps / 300 Mbps

Cryosat 2 kbps / 100 Mbps

SMOS 4 kbps / 18,4 Mbps

Rokviss 256 kbps / 4 Mbps

Table 2: Maximum data rates in uplink / down-link of exemplary space missions

Hence, an Integrated Modem and Baseband Unit (IMBU), as the core element of the

communication, was custom made by Satellite Services BV for the speci�c requirements to

the test environment. Fig. 4 shows a basic block diagram of the communication architecture.

The DLR Institute of Robotics and Mechatronics, the LRT mission control centre, and the

LRT ground station are highlighted. Transmitting and receiving telepresence data at the

Figure 4 � The communication setup of the telerobotic test environment

LRT ground station was realized by utilizing specialized radio frequency up- and downcon-

verter in the S-band frequency range. The converters feature an adjustable oscillator, with

which the received or transmitted signal was mixed. This guaranteed a conversion between

the intermediate frequency (70 MHz), which the Integrated Modem and Baseband Unit

(IMBU) utilized and the frequencies of the receiver and the transmitter, respectively.

As seen in Fig. 3 the ARTEMIS IOL forward and return links were used for S-band

communication with the LRT ground station. Thus, frequencies had to be coordinated

with the German federal network agency. The frequencies that have been allocated for the

ARTEMIS service were 2076.5 MHz for IOL return (ground station uplink) and 2255.0

MHz for IOL forward (ground station downlink). Since these frequencies are usually used

for communication in orbit, they were not usable on Earth without further regulations.

Thus, the federal network agency allocated these frequencies only temporarily to LRT in

the framework of an experimental radio communication license.

Broadband power ampli�ers (High Power Ampli�er / HPA, Low Noise Ampli�er / LNA)

had to be implemented into the test environment. The HPA was crucial for amplifying

the upconverter signal to 20 W. This transmitting power results from the link budget and

ensures that the requirements of an ampli�cation of minimum +53.0 dB and a power output

(at 1 dB compression point) of minimum +43.0 dBm were met. An according ampli�er was

not available o�-the-shelf and was custom-built by MITEQ.

Further, the antenna feed system (see Fig. 4) had to be modi�ed for the experiments since it

had not only to support the communication with ARTEMIS in GEO, but also conventional

satellites in LEO. A �exible polarization switch was implemented into the setup. It allowed

switching between the left handed circular polarization (LHCP) of the IOL frequencies

and the right handed circular polarization (RHCP), which is used for ground to satellite

communication. A gain to noise ratio G/T > 4 dB/K was realized by the setup and was

su�cient for the telepresence experiments via the DRS.

ARTEMIS is a transparent satellite, i.e. received data is transmitted immediately after a

frequency conversion. In contrast to so-called bent pipe technology. There is no demodula-

tion, decoding, data correction, coding and modulation done as for regenerative satellites.

This makes the ARTEMIS less �exible compared to regenerative satellites, but results in

less processing time of data and accordingly supports the telepresence requirements of

minimum round trip delays.

3 The bilateral control architecture of the OOS test bed

A test bed for telepresent OOS has been coupled to the LRT mission control center

as illustrated in Fig. 4. Located in the DLR institute of Robotics and Mechatronics in

Oberpfa�enhofen, Germany, the test bed comprises two DLR light weight robots (LWR)

as a bilateral (force coupled) master-slave system, and a target satellite dynamic emulator

with an operations platform based on another LWR (Artigas et al., 2006).

As seen in Fig. 5 a 7DoF LWR-III is used as haptic man-machine interface. Further, a head

mounted display with stereo visualization is used to obtain an immersive implementation

of the visual channel. A DLR LWR-II is used as teleoperator robot which has been further

equiped with an industrial gripper system in order to interact with the environment.

The OOS test bed has been designed to analyze di�erent bilateral control strategies to

cope with the presence of di�erent varying round trip delays. The remote environment

demonstrates a non-cooperative, malfunctioning satellite emulated by means of another

LWR-II with a satellite test board attached to the end-e�ector of the robot. Featuring

a de�ciently working attitude control system, the satellite is tumbling in orbit and the

operator has to ful�ll two major tasks :

a) Docking : The satellite platform is to be caught by means of the gripper. Once the

task is achieved, the target satellite is considered to have no relative motion to the

servicer satellite.

b) Servicing : A series of servicing possibilities are given on target satellite side, which

the operator has to solve. The manipulation of a bayonet nut connector and a series

of cables is foreseen.

The motion and force commands are measured and transmitted via the data relay satellite

to the distant slave manipulator. This in turn tracks the commands of the master mani-

pulator and feeds the interaction forces with the remote environment back to the master

manipulator system. The human operator is thus energetically coupled to the remote

environment through the electromechanical elements.

Figure 5 � DLR OOS Test bed

The inclusion of the human operator in the closed loop and the presence of the varying and

comparably large time delay in the communication channel represents a challenging task.

Most approaches dealing with time delayed haptic telepresence describe the system by means

of power network elements, which are either designed to be passive (Anderson and Spong,

1989; Niemeyer, 1996) or will be adaptive in nature to keep passivity on the time domain (i.e.

passivity not as a design constrain (Hannaford and Ryu, 2000; Artigas et al., 2007; Artigas

et al., 2008)).

One of the most remarkable approaches is the classical method of the Scattering transforma-

tion or its Wave Variables formulation introduced in (Anderson and Spong, 1989; Niemeyer,

1996). By using the electrical-mechanical analogy, the wave variables transformation uses

the power conjugated variables of force and velocity to de�ne wave variables, the same way

voltages and currents are related to energy waves in transmission lines. A wavy system has

the interesting feature that passivity is preserved in the presence of time delay. Thus, the

two-port network created by a communication channel with time delay described in terms of

wave variables is a passive system which will not alter stability for any amount of constant

time delay.

The bilateral control method used here is based on the wave variables approach. The next

subsections review the wave transformation and its passivity aspects, and introduces a me-

thod to cope with the variation of the delay and the package loss.

Figure 6 � Wave-based position-force teleoperation control scheme.

3.1 Wave variables

Wave variables present an extension to the theory of passivity and are based on the concepts

of power and energy. The transformed variables, u and v, are an algebraic relation of the

power conjugated variables x and F , velocity and force respectively. u is the wave variable

traveling from master to slave. v is the returning wave, from slave to master.

3.1.1 Position-Force system with constant time delay

Fig. 6 shows a wave variables -based position-force scheme. Two PI controllers at master and

slave sites are responsible for minimizing the end e�ector position errors between master and

slave, hence generating the required manipulation force on slave site and haptic interaction

forces on master site. Note that the channel is isolated from the rest of the system by means

of the wave transformers on each side of the channel, which in turn separate power variables

domain from wave domain. The wave transformation equations for this scheme are (1) for

master site and (2) for slave site, denoted by m and s, respectively. Here u represents the

wave, sent from the master to the slave and v the returning wave from the slave to the

master.

um(t) =bxmd(t) + Fm(t)√

2b, vm(t) =

bxmd − Fm√2b

. (1)

us(t) =bxsd(t) + Fs(t)√

2b, vs(t) =

bxsd(t)− Fs(t)√2b

. (2)

Here b is the wave impedance constant, Fm and Fs are local controller force commands and

xmd and xsd are desired motion velocities of the master and slave respectively.

For the complete wave formulation please refere to (Niemeyer, 1996).

3.2 Passivity condition

De�ning Pin, the power entering a system, as the scalar product between input vector x and

output vector y, and Estore as the stored energy of the system, the system is passive if and

only if ∫ t

0

Pindτ =

∫ t

0

xTydτ ≥ Estore(t)− Estore(0). (3)

Without loss of generality, it can be assumed that the initial stored energy in the channel

is equal to zero, Estore(0) = 0. Further the forward and backward delays are assumed to be

constant as a �rst estimation. Then the shift of wave signal can be formulated as follows.

um(t− Tfwd) = us(t), vm(t) = vs(t− Tbwd). (4)

In the wave domain, condition (3) leads to

1

2

[∫ t

t−Tfwd

u2m(τ)dτ +

∫ t

t−Tbwd

v2s(τ)dτ

]≥ 0, (5)

which holds true as long equation (4) (constant time delay and guarantee of signal delivery)

does. Accordingly, the stability of the teleoperator system is guaranteed.

3.2.1 Varying time delay and package loss

As previously seen in Chapter 2 the radio link from Garching - ARTEMIS - Redu imposes

a set of characteristics which produce a direct impact upon the bilateral control. The pre-

vious section assumed an ideal delayed communication, with a constant time delay and zero

package loss and bit error rates. In real scenarios, however, the delay cannot be considered

constant and considerable package loss and bit error rates will be presented.

UDP is used for network connection of LRT and DLR due to its fast exchange rate (the

smaller the delay, the better the operation performance and transparency), but this protocol

is subject to packet drops and reordering because the transmission is not controlled in the

network layers.

Time-varying delay

Let forward and backward delays by de�ned by the time functions Tfwd(t) and Tbwd(t).

Equation (4) can be reformulated :

um(t) = us(t+ Tfwd(t)), vm(t+ Tbwd(t)) = vs(t).

Tfwd(t) and Tbwd(t) functions are unknown at time t for the master and slave sent waves um(t),

vs(t) respectively. These functions can be identi�ed only after transmission and delivery.

Therefore the passivity condition in form of equation (5) cannot be obtained. However, the

passivity condition can be conserved by keeping the decoupled forward and backward channel

lines passive, as follows.

Ecomm,fwd =1

2

∫ t

0

[u2m(τ)− u2

s(τ)]dτ ≥ 0, (6)

Ecomm,bwd =1

2

∫ t

0

[v2s(τ)− v2

m(τ)]dτ ≥ 0. (7)

Packet loss

Using the Internet as the communication medium will induce occasional packet drops and

therefore information loss in the exchange line, in addition to jitter (variations of delay). The

same is valid for satellite based communication where the time delay function is smoother

and the data loss rate is lower. Both mediums are discrete time systems where senders and

receivers sample and pick the data with a certain frequency. An increase in time delay leads to

the empty sampling instances for the receiver, which is usually called signal stretching in the

continuous domain. The decrease in time delay leads to instances where more than one data

packet exists to be sampled at the receiver site. This is called signal compression in continuous

domain. However only one signal can be sampled and the other will be discarded. This

leads to information loss. It is important to notice that the wave information has energetic

meanings. Another source of packet-loss is the reordering of the sent signals where a later

sent packet arrives earlier than an earlier sent packet. Typically empty sampling instances

are solved by either inserting a zero value (Null packet or Zeroing strategy) or conveying

the last valid signal, also known as Hold-last-sample (HLS). It has been shown that none of

them is su�cient for performance or stability conditions and modi�cation should be applied

to them (Hirche and Buss, 2004). The Null packet strategy is lossy, overly conservative and

leads to poor performance as well as wear and noise in mechanical parts. On the other hand,

HLS strategy keeps the transmitted wave form well, but cannot guarantee the passivity of

the channel and may lead to instability of the operation.

3.3 Proposed method

For the sake of simplicity the loss and time delay variations of the backward channel are

neglected. Only the forward channel characteristics (dealing with backward link as ideal)

are considered. After reformulating the continuous time expression in (6) for discrete time,

the below relation is obtained.

2∆E(i)/Ts =k=i∑k=0

u2m(k)−

k=i∑k=0

u2s(k), (8)

where∑k=i

k=0 u2m(k) is the sum of the square of the right moving wave variables and represents

the power input to the forward communication block,∑k=i

k=0 u2s(k) is the sum of the recei-

ved wave squares which is the power output of the communication and Ts is the sampling

time. The overall sent energy is calculated and its value is sent through the time delayed

communication channel to the remote site (here slave). Hence, the remote receiver at time t

receives knowledge on the overall energy sent up to a certain time instance (t∗). t = nTs is

the current time and t∗ = n∗Ts is the time stamp of the arrived packet. These two times are

related by :

nTs = n∗Ts + Tfwd(n∗Ts). (9)

To ensure the passivity, it is su�cient to state

Esent(n∗Ts)− Ereceived(nTs) ≥ 0. (10)

Equation (10) is the online forward observed energy equation. The passivity of the channel

has to be checked, comparing the overall energy input and output at the same time instances,

but in practice the passivity condition cannot be measured online. Due to accumulative

property of the sum of the sent wave energies, the following relation holds :

Esent(n) ≥ Esent(n∗) where n > n∗ . (11)

Considering the forward channel and renaming Esent by Em and Ereceived by Es, and based

on (11), the following relation is valid

Em(n)− Es(n) ≥ Em(n∗)− Es(n). (12)

The left hand side of (12) is the passivity condition and the right hand side is the online for-

ward observed energy equation. Keeping the inequality in (10) leads to ful�lling the following

overall passivity condition.

Em(n)− Es(n) ≥ 0 (13)

The online forward observed energy equation is used for online passivity checks. As it is appa-

rent, using this equation for passivity observation yields the system safety limits against the

activity and the potential instability. A packet generator is implemented at each sender site

putting the algorithm's required information into a data structure. The information put into

the sending packets are sending time stamp (n∗Ts), current sending wave value (um(n∗)), pa-

cket reception acknowledgment �ag, summation of the overall sent wave (Ts

∑n∗

0 um(n∗Ts))

and the wave energy sent overall up to the current instance (12Ts

∑n∗

0 u2m(n∗Ts)). The algo-

rithm performs the following tasks to ensure the passivity. At each sampling time the packet

reader on the receiver side checks the �ag for arrival of data. When the �ag indicates data

arrival and its time stamp is bigger than the last recorded time stamp the packet data will be

processed. This process consists of one-step-ahead energetic checks and if the online energy

observer violated the passivity condition, the current wave is modi�ed to dissipate required

energy and keep the passivity. The program checks the theoretical energetical impacts if the

current wave was conveyed using equation (14). Ecurr is the energy that the latest delivered

wave could transmit to the system during one time sample.

Ecurr(n) =1

2u2

m(n∗)Ts (14)

The forward energy observer (Efeo) checks the safety of conveying the current arrived wave

in equation (15). Esoutput(n−1) is the energy output of the packet processor up to the current

time, which is fed back to the algorithm and Esent(n∗) is the sum of the energy sent from

the master up to n∗ instance.

Efeo(n) = Esent(n∗)− Esoutput(n− 1)− Ecurr(n) (15)

If (15) is bigger or equal to zero, passivity would be kept by conveying um(n∗). If (15) is

negative, um(n∗) has to be modi�ed. In (17) the arrived um(t∗) has to be changed in a way

that dissipates the activity of Efeo(n). If u2s(n) ≥ 0 is satis�ed, a real answer exists and

using wave modi�cation in one sample the sensed activity can be dissipated. We can refer to

um(n∗) as us(n). The modi�cation is applied to us(n), based on the following energy balance

equation.1

2u2

s(n)Ts + Efeo(n) =1

2u2

s(n)Ts (16)

Based on this, us(n) has to be modi�ed to us(n) following

u2s(n) = u2

s(n) +2Efeo(n)

Ts

, (17)

to dissipate the Efeo(n) activity. The sign of the wave can be interpreted as push or pull

command. Even though wrong signature selection does theoretically not a�ect the energetic

behavior of the channel, the correct solution is important. This is more signi�cant when a

black-out occurs in the line and high number of consequent losses are detected. The trend

of the sent command (push or pull) is observable by comparing the currently arrived sum of

the sent waves with the last valid recorded arrived sum in (18). nlvr is the last-valid-recorded

time instance of data arrival and is smaller than n∗.

S =i=n∗∑i=0

um(i)−i=nlvr∑

i=0

um(i) (18)

When S is positive or negative the trend indicates more push commands and more pull

commands, respectively. Thus, the conveyed signal after a black out can be calculated.

us(n) =S

|S|

√u2

s(n) +2Efeo(n)

Ts

(19)

For simple empty instances (due to reordering, occasional losses or stretching) selection of

the signature, simply based on the current wave is correct but equation (19) generalizes these

conditions as well.

If u2s(n) < 0 holds, the wave is replaced with a Zero, which is the maximum energy that can be

dissipated in one instance. The remained undissipated energy is recorded to be dissipated in

the next samples if necessary. As the activity detection in forward observed energy equation is

conservative and preemptive and is based on the worst possible case (sending zero command

Figure 7 � Scheme of the forward link packet processing and energy compensation.

from counterpart), correction of these undissipated excess energy (remained after Zeroing)

might not be necessary in the next steps. However, it is decided by the algorithm based on

the received new data from the sender.

When the time stamp of the arrived packet is older than the last valid recorded time, the

packet is discarded (The algorithm has already dealt with this packet's absence energetically

as another later-stamped-earlier-arrived packet informed the receiver of the energy sent in

between). There can be also instances that simply no packets are delivered. In both cases

the algorithm has to handle the empty instance. The algorithm assumes that the current

wave should be equal to the last valid signal (HLS strategy). With this assumption the HLS

signal should be checked in the previous algorithm for energy considerations. If necessary

the same modi�cations introduced before will be applied on the waves. The scheme of the

proposed method and the integration into the forward link is depicted in Fig.7.

4 Performance and results of the experiments

An objective evaluation of telepresence is di�cult since the feeling of being immersed into

the remote environment is dependent on the subjective perception of the individual. The

immersion of the user is in�uenced by the modality of the feedback from the teleoperator.

The modality in turn depends on the properties of the link, which is used to transmit the

feedback. Usually quality of service (QoS) criteria are used for evaluating terrestrial net-

works. Based on that, it has been shown (Chen, 2005; Park and Kenyon, 1999; Tfaily, 2003)

that the QoS criteria round trip delay, jitter, and packet loss, in the context of space com-

munications often expressed in terms of bit error rate can in�uence the task performance of

a teleoperation.

The task performance of the human operator is directly related to the immersion of the hu-

man operator into the system, and thus a measure for telepresence. Out of the above criteria,

the round trip delay is the one considered as most critical for teleoperation since time delay

can destabilize a telepresence system

QoS criteria will initially be used for evaluating the telepresence capability of the system

and related to the task performance of the human operator. For that purpose participants

were asked to manipulate the OOS test bed under psychological instruction. The focus was

laid on the task performance depending on the round trip delay characteristics of the space

link, i.e. the progression of the round trip delay over time.

Finally, for a numerical determination of the telepresence capability of the system, an eva-

luation of the transparency of the system has been undertaken.

4.1 Link characteristics depending task performance

Using the mirror approach, the signal from the master manipulator at DLR traveled to the

LRT ground station, was transmitted to ARTEMIS, forwarded to Redu, was mirrored, and

was analogously redirected back to the DLR slave manipulator. According round trip delay

characteristics were generated. The characteristic involves the behavior of the round trip

delays over time. Round trip delay characteristics with a mean round trip delay of tRTD =

622 ms were obtained. The characteristic in Fig. 8 shows a periodic behavior of the round

trip delay. Every approximately 1000 packets the round trip delay increases abruptly and de-

creases slowly afterwards. This behavior is superimposed by �uctuations, featuring a smaller

span. Additionally, high peaks (> 2000 ms) occurred, which are caused by the comparable

high amount of lost packets (≈ 5.8%). This in turn originates mostly from errors in the satel-

lite modem, as a lot of received packets had incorrect checksums and to some minor degree

from the use of UDP in the connection with the terrestrial WAN, since the space link itself

featured < 0.1 % of lost packets. For evaluating the jitter of a signal, i.e. the variation in the

round trip delay, it has been reverted to the sample standard deviation st since the samples

can be considered as statistically (quasi) independent. The sample standard deviation has

been calculated to st = 66 ms, which means that approximately 68.3 % of all data samples

are located in the interval 622 ms ± 66 ms.

Bit error ratios were measured within a dedicated BER test session. Usually the DRS service

of ARTEMIS stops after an hour for a couple of minutes. The reason is that the SKDR

payload has to be recalibrated. Thus, BER tests were usually limited to approximately one

hour. However, ESA arranged an uninterrupted test session for 135 minutes, in which the

BER could be measured. A BER of < 10−5 is regarded as standard acceptable for satellite

telemetry and < 10−6 for the telecommand system. Thus the evaluated 8.3∗10−6 , can be

considered as su�cient for satellite communication.

Large round trip delays in the haptic and visual channel worsen task performance and tele-

presence feeling (Pongrac, 2008). This e�ect is in particular more distinct if the round trip

delays are varying. The tests, conducted within the framework of this work, second this fact.

Figure 8 � Round trip delay characteristics of the telerobotic manipulation

Using DRS for teleoperation increases the round trip delay, which worsens the telepresence

capability of a system in general. For an initial evaluation of the telepresence capability of

the system a group of participants was asked to participate in a psychological evaluation,

consisting of one exercise and two experimental trials. Because of the limited availability of

the ARTEMIS link the group only consisted of six participants, who were completely unfa-

miliar with the system. Thus, the evaluation started with an exercise course. For becoming

acquainted with the system, the bayonet nut connector (see Fig. 5) had to be opened several

times by means of the robotic manipulator under 0 s round trip delay. The exercise was re-

garded as �nished when the practice criterion was met, which was to ful�ll the task within 30

s or in 25 % of the time, needed in the �rst trial. Afterwards, the experimental trials started

anew with 0 s round trip delay. The remote environment, i.e. the non-cooperative, malfunc-

tioning satellite started moving and the participants had to grip it via the slave manipulator.

Once the handhold on the test board was caught, the requirements for a successful docking

operation were considered as met. The time for performing the �rst task was measured. The

satellite was brought to initial position and no relative movement between manipulator and

satellite was further introduced. The anew opening of the bayonet nut connector was the

second task to be ful�lled. This is quite a complex task for a teleoperation, since the partici-

pants had to grip the respective part of the connector and execute a turn to release it from

the second part. For a �nal disconnection of both parts, the extraction had to be ful�lled

exactly parallel to the board. Otherwise the parts cant and a disconnection is not possible.

The performance time was again logged.

The following second experimental trial was identical with the one, executed earlier (0 ms),

except for the round trip delay. The two di�erent tasks were performed using the ARTEMIS

link and thus, with a comparably high and not constant round trip delay (see Fig. 8). The

participants had to answer three questions after the experimental trials.

• How natural did the interaction with the environment feel (telepresence feeling) ? The scale

ranged from 1 (very arti�cial) to 7 (very natural).

• How deeply did you feel immersed into the remote environment (immersion into the sys-

tem) ? The scale ranged from 1 (very weak) to 7 (very deeply).

• In your opinion, of what magnitude was the round trip delay ?

The task performance of the group of participants that was measured, is depicted in Fig. 9.

The �gure shows that the task performance of the human operator decreases due to the

use of the data relay satellite. While the participants required (after a training phase) a

mean of 15.0 s (with a sample standard deviation σ = 13.1 s) to grip the satellite without

time delay, using the master manipulator, the gripping time increased to 44.8 s (σ = 23.0

s) in the presence of the DRS. The mean time for opening the bayonet nut connector more

than tripled from 31.0 s (σ = 15.1 s) to 111.6 s (σ = 94.7 s). Further, the results show

that the participants systematically overestimate the round trip delay, even in the 0 ms

case (estimated mean 226 ms). By using the ARTEMIS link the participants estimated

the round trip delay in mean to 952.6 ms(σ = 587.0 ms). The mean telepresence feeling

(rating 1) dropped by 1.75 from 5.46 (σ = 1.38) to 3.71 (σ = 1.35) , which corresponds

from the psychological point of view a rating decrement of 214 %. The mean immersion into

the system (rating 2) dropped by 0.55 points from 5.17 (σ = 1.52) to 4.62 (σ = 1.47 s),

which corresponds to a rating decrement of 130 %. Additional tests showed that there was

no signi�cant correlation between the demographical data of the participants (age, gender,

handedness etc.) and the experimental data. Therefore, the logged values were most likely

only depending on the experimental setup. Even though participants were small in number,

the tests clearly showed that all participants were able to ful�ll the assigned tasks via the

Figure 9 � Mean task performance of participants

relay satellite in the presence of round trip delay. This will be the basis for additional tests.

4.2 Transparency evaluation

A haptic MMI can be approximated as a device, which generates mechanical impedance

(Colgate and Brown, 1994). It represents the dynamic relationship between displacement (or

velocity) and force. An ideal haptic interface would be capable of generating any impedance,

which is required to represent the remote environment realistically to the human operator.

Accordingly, transparency can be considered as the accuracy in rendering the remote envi-

ronment to the human operator (Hashtrudi-Zaad and Salcudean, 2001).

After stability, the major goal of any haptic telepresence system is transparency. Previous

works (Lawrence, 1993; Yokokohji and Yoshikawa, 1994) exhaustively show that the pursuit

of stability compromises transparency once the system constraints are established. In par-

ticular, the presence of delay in the communication channel leads to a conservative design

of the control architecture, lowering the system transparency and hence, the telepresence

feeling in general. For a numerical con�rmation of the above psychological evaluation of

the teleoperation via ARTEMIS, a transparency evaluation of the master slave system was

conducted, which was based on the haptic feedback channel. This evaluation was indepen-

dently conducted from the psychological evaluation and did not require any participants. For

evaluating the performance of a teleoperated system, the Z-width concept was used.

4.3 The Z-width concept

The Z-width is de�ned as the achievable range of impedance which the system can stably

present to the operator (Colgate and Brown, 1994). Here the impedance to human Ztoh(s)

is de�ned by

Ztoh(s) =Fh(s)

Xm(s). (20)

The Z-width range is delimited by frequency dependent lower and upper bounds. The lower

bound of Z-width, Zmin(s), is calculated for Ze(s) → 0, while the upper bound, Zmax(s), is

calculated for Ze(s)→∞. That is

Free Motion : Zmin(s) = Ztoh(s)|Ze(s)→0

(21)

Constrained Motion : Zmax(s) = Ztoh(s)|Ze(s)→∞ . (22)

Consequently, the Z-width is given by

Zwidth(s) = Zmax(s)− Zmin(s). (23)

This Z-width allows to compare di�erent control schemes quantitatively. Since the bandwidth

of human actuation and sensing capabilities are limited (Lawrence et al., 2004), the analysis

of the Z-width in this paper will be restricted to the relevant frequency range.

4.4 Z-width measurement procedure

If the human force and master position can be measured, then the human perceived impe-

dance can be obtained from these measured signals using Least-Squares Input/Output (LS

I/O) Identi�cation method (Hirche et al., 2003). The goal of the identi�cation method is to

�nd the parameters of the transfer function of the human perceived impedance that minimize

the squared error between the real output and the output of the identi�ed transfer function.

The following procedure is followed in order to measure the Z-width out from the measured

human percieved impedance :

1. Measurement of the human operator force fh and the master position xm in the free

movement case.

2. I/O system identi�cation of the transfer function Ztoh =fhfree

xmfree. This is the measured

minimum limit of the Z-Width, Zmin.

3. Measurement of the human operator force fh and the master position xm in the case

of wall contact.

4. I/O system identi�cation of the transfer function Ztoh =fhwall

xmwall. This is the measured

maximum limit of the Z-Width, Zmax .

5. The measured Z-Width, Zwidth, is the di�erence between the two identi�ed impedances

Zwidth = Zmax − Zmin.

Note that this evaluation was only done in 1-dimensional space for obtaining scalar values

for force and displacement. The two cases free movement and wall contact were generated by

randomly moving the slave in free space and by moving the slave into a rigid wall (satellite),

respectively. The progression of master displacement xm and slave displacement xs (in the

time domain) is illustrated in Fig. 10. The position of the slave (dashed) follows the master

(solid) after a round trip delay, which is caused by the relay via the DRS. A wall is located

at zero position. The free environment is indicated by a negative sign of the displacements.

It can be seen that initially a movement in free environment has been conducted, followed

by a movement into the virtual wall. The master displacement at that point in time becomes

positive, whereas the slave motion is constrained by the wall. Thus, its displacement is al-

Figure 10 � Displacement of master and slave over time

Figure 11 � Force of master and slave over time

most constant at zero position.

Fig. 11 shows master and slave force in the time domain. The slave force is delayed by the

DRS and features an opposite sign compared to the master force. Their absolute value is

approximately identical in free environment and the absolute value of the slave force tends

to be smaller. In contrast, for the wall condition the slave force increases to a multiple of

the master force, considering absolute values. The values obtained that way were transferred

Figure 12 � Absolute values for Zmin and Zmax via ARTEMIS

into the frequency domain. Zmin and Zmax were derived and can be accessed from Fig. 12.

Since the impedances are complex values, they were plotted over the frequencies using a 20

log |Z| scale to be compliant with a Bode representation. For approximating the transfer

functions in Fig. 12 appropriate time intervals for wall contact and free environment from

Fig. 10 and Fig. 11 were selected.

The corresponding Zwidth is depicted in Fig. 13. The Z-width concept is not an absolute indi-

cator for system transparency. It rather can be utilized to evaluate the system transparency

with respect to another system setup. For that reason the reference Zwidth for 0 ms is also

plotted in Fig. 13. It can be seen that the increasing delay decreases the system transparency

(in the low frequency range), analogously to the task performance of the human operator.

This is compliant with theory. In the low frequency range the graphs can be assumed to be

constant as a �rst approximation. It follows a peak (for the graphs with DRS), which is not

of practical relevance because of the limits of the approximation.

This limits originate from the fact that the Z-width is commonly derived by using a Padé

series of �nite order N to describe the time delay system element Dt. By using a �rst order

series (N = 1)

Dt = e−sT ≈1− T

s

1 + Ts

, (24)

the Padé approximation is only valid for frequencies ωlim < 1/(3T ) (Hirche et al., 2005).

Therefore, the peaks are of no practical relevance, since the Z-width graph is only valid for

the low frequency range.

The system frequency assigned to the axis of abscissa can be interpreted as frequency,

with which the human operator steers the master manipulator. This is constrained by the

limiting frequency and amounts in the given example to ωlim ≈ 0.5 Hz. This limitation is

of practical interest since it labels a region, in which the system cannot react fast enough

anymore, due to the occurring round trip delay, to a given input. Accordingly, the user must

not be allowed (by technical means, as e.g. inertia or damping) to execute teleoperations

above this limiting frequency.

5 Summary and future directions

This work focused on utilizing the concept of telepresent control to on-orbit servicing. In

particular, the applicability to OOS missions in low Earth orbit, for which the communi-

Figure 13 � Absolute value of the Z-width for zero delay and via ARTEMIS

cation has to be relayed via a DRS in geostationary orbit, was considered. This section

summarizes the results and gives suggestions for future work.

5.1 Summary

The term telepresence describes a control concept, in which a teleoperator is separated

from the controlling human operator by a barrier. Telepresence demands that the human

operator can hardly di�erentiate, whether his impressions of feedback result from direct

interaction with the environment or from technical means.

Utilizing telepresence for OOS operations promises a number of advantages for astronautics.

On the one hand, the human operator is located on ground and thus, cost intensive and

critical EVAs can be avoided. On the other hand, it enables the human operator a real

time response to unforeseen incidents, which is not possible considering autonomous OOS

missions.

Contrary to terrestrial applications, the contact time to the teleoperator is limited, especially

in LEO. Direct contact from the ground station to the servicing spacecraft is only given for

a few of minutes, which limits the data acquisition time. Hence, the use of geostationary

satellites for data relay was considered, which increases the acquisition time to a multiple,

compared to direct contact. However, the disadvantage of this approach is the increase of

round trip delay, jitter and package loss.

It could be shown that using the appropriate bilateral control strategy this increase in round

trip delay and non-idealities does not con�ict with the concept of telepresence control, for

which a realistic feedback is demanded to some extend. A test environment was developed,

which was representative for telepresent OOS using a DRS and involved the Institute of

Astronautics, the European Space Agency, and the German Aerospace Center. A robotic

OOS test bed was teleoperated via the ESA satellite ARTEMIS. The implementation of

ARTEMIS and the setup on ground enabled obtaining realistic measurements, in order to

evaluate the feasibility of telepresent OOS.

The obtained round trip delays with a mean of 622 ms enabled an accomplishment of the

telepresent manipulation tasks, even under the in�uence of additional network delays.

For validating the above statement, a psychological evaluation was conducted. The feeling

of telepresence or immersion into the system is a subjective perception. Thus, a number

of participants were asked to steer the robotic scenario via ARTEMIS. The results show

that it is possible to execute complex OOS maneuvers via a geostationary DRS and

maintain the feeling telepresence. This was corroborated by an evaluation of the system

transparency. Based on that, the conclusion is drawn that telepresent on-orbit servicing

with haptic feedback via a geostationary relay satellite is feasible. Nonetheless, the number

of participants was limited due to the availability of the satellite link. Thus, further testing

will be needed to obtain a comprehensive human-machine evaluation.

5.2 Future directions

Network delays are part of the considered round trip delays. They can be minimized, when

realizing a telepresent space mission, either by locating the human operator in proximity to

the ground station or prioritizing the network link.

Future research has to consider the synchronization of system elements. The clock frequency

of signal generating equipment and the communication equipment (e.g. IMBU) has to be

adjusted with respect to each other in order to avoid peaks in round trip delay. The necessity

of system bu�ers and a su�cient bu�er management will be of further interest. Optimizing

the control parameters of the system holds even more potential for enhancing the system

transparency than a minimized round trip delay. The master-slave control architecture has

to be adapted to the respective OOS operation to increase the telepresence capability.

By steering the robotic application via the DRS, the S-band link met its limits. The master

sample rate had to be decreased to 500 Hz instead of the common 1000 Hz. The stereo

feedback (telemetry) had to be transmitted locally. Higher band width is indispensable for

space missions. The Ka-band frequency range is a possible alternative. This would also allow

transmitting additional sensor data from the haptic-visual work space, which enhances the

telepresence capability of the system. This in turn will support the immersion of the human

operator into the system.

Acknowledgments

This work is supported in part by German Research Foundation (DFG) within the colla-

borative research center SFB453 "High-Fidelity Telepresence and Teleaction". The authors

would like to express their sincere gratitude to the ESA ARTEMIS team for providing the

opportunity to use the ARTEMIS relay satellite for telepresence experiments. In particular,

special thanks to Benoit Demelenne and Damien Dessoy for prompt help and unbureau-

cratic scheduling concerning the data relay service. Further, we would like to thank LSE

Space Engineering, DLR MORABA, Rhode & Schwarz, and DOMO TV for supporting the

development of the LRT ground station.

References

AFRL (2007). Xss-11 micro satellite - fact sheet. Technical report.

Anderson, R. and Spong, M. (1989). Bilateral control of teleoperators with time delay.Automatic Control, IEEE Transactions on, 34(5) :494�501.

Artigas, J., Kremer, P., Preusche, C., and Hirzinger, G. (2006). Testbed for telepresenton-orbit satellite servicing. In in Proceedings of the Human-Centered Robotic SystemsConference (HCRS), Munich, Germany.

Artigas, J., Preusche, C., and Hirzinger, G. (2007). Time domain passivity for delayedhaptic telepresence with energy reference. In proc. IEEE/RSJ International Conferenceon Intelligent Robots and Systems (IROS'07)., San Diego, CA, USA.

Artigas, J., Preusche, C., Hirzinger, G., Borghesan, G., and Melchiorri, C. (2008). Bilateralenergy transfer in delayed teleoperation on the time domain. In proc. of IEEE/RSJInternational Conference on Robotics and Automation (ICRA'08), Pasadena, CA, USA.

BOEING (2007). Orbital express - mission updates. Technical report.

Bosse et al., A. (2004). Sumo : spacecraft for the universal modi�cation of orbits. In proc.of SPIE - The International Society for Optical Engineering, vol. 54 iss.19, Aug. 2004.

Chen, L. (2005). E�ects of network characteristics on task performance in a desktop cvesystem. In proc. of 19th IEEE International Conference on Advanced Information Net-working and Applications, Tamkang, Taiwan, Mar. 2005.

Colgate, J. and Brown, J. (1994). Factors a�ecting the z-width of a haptic display. In Roboticsand Automation Proceedings, IEEE International Conference on, pages 3205�3210 vol.4.

Davis, T. (2005). Xss-10 micro-satellite �ight demonstration. In proc. of Georgia Institute ofTechnology Space Systems Engineering Conference, paper no. GT-SSEC.D.3, Nov. 2005.

Hannaford, B. and Ryu, J.-H. (2000). Time Domain Passivity Control of Haptic Interfaces.IEEE Transactions on Robotics and Automation.

Hashtrudi-Zaad, K. and Salcudean, S. E. (2001). Analysis of control architectures for te-leoperation systems with impedance/admittance master and slave manipulators. TheInternational Journal of Robotics Research, 20(6) :419�445.

Hirche, S., Bauer, A., and Buss, M. (2005). Transparency of haptic telepresence systemswith constant time delay. In proc. of IEEE Conference on Control Applications, Toronto,Canada, 2005.

Hirche, S. and Buss, M. (2004). Packet loss e�ect in passive telepresence systems. In 43rdIEEE Conference on Decision and Control, pages 4010�4015.

Hirche, S., Stanczyk, B., and Buss, M. (2003). Transparent exploration of remote environ-ments by internet telepresence. In proc. of the International Workshop on High-FidelityTelepresence and Teleaction jointly with the IEEE conference HUMANOIDS, (Munich,Germany), 2003.

Hirzinger, G., Landzettel, K., Brunner, B., Fischer, M., Preusche, C., Reintsema, D., Albu-Schä�er, A., Schreiber, G., and Steinmetz, M. (2004). DLR's robotics technologies foron-orbit servicing. Advanced Robotics � Special Issue Service Robots in Space, 18(2) :139�174.

Imaida, T., Yokokohji, Y., Doi, T., Oda, M., and Yoshikawa, T. (2004). Ground-spacebilateral teleoperation of ets-vii robot arm by direct bilateral coupling under 7-s timedelay condition. Robotics and Automation, IEEE Transactions on, 20(3) :499�511.

Landzettel, K., Preusche, C., Albu-Schä�er, A., Reintsema, D., Rebele, B., and Hirzinger,G. (2006). Robotic On-Orbit Servicing - DLR's Experience and Perspective. In proc. ofthe International Conference on Intelligent Robots and Systems (IROS), Peking, China.

Lawrence, D. (1993). Stability and transparency in bilateral teleoperation. Robotics andAutomation, IEEE Transactions on, 9(5) :624�637.

Lawrence, D., Pao, L., White, A., and Xu, W. (2004). Low cost actuator and sensor for high-�delity haptic interfaces. In Haptic Interfaces for Virtual Environment and TeleoperatorSystems, 2004. HAPTICS '04. Proceedings. 12th International Symposium on, pages74�81.

Lundin, R. and Stoll, E. (2006). Coverage time variation in a near-earth data relay satellitesystem. In 57th International Astronautical Congress, Valencia, Spain, Oct. 2006.

Moens, C., Absolonne, F., and Lezy, C. (2003). Artemis data relay payload iot results asmeasured from redu in feb/mar 2003. Technical report.

Mukherji, R., Rey, D., Stieber, M., and Lymer, J. (2001). Special purpose dexterous mani-pulator (spdm) advanced control features and development test results. In proc. of the6th International Symposium on Arti�cial Intelligence and Robotics and Automation inSpace (i-SAIRAS), St-Hubert, Canada, Jun. 2001.

Niemeyer, G. (1996). Using Wave Variables in Time Delayed Force Re�ecting Teleopera-tion. PhD thesis, Massachusetts Institute of Technology, Department of Aeronautics andAstronautics Engineering, Cambridge, MA.

Oda, M. (2000). Experiences and lessons learned from the ets-vii robot satellite. In Robo-tics and Automation, 2000. Proceedings. ICRA '00. IEEE International Conference on,volume 1, pages 914�919 vol.1.

Park, K. and Kenyon, R. (1999). E�ects of network characteristics on human performancein a collaborative environment. In proc. of IEEE International Conference on VirtualReality, Houston, USA, Mar. 1999.

Parrish, J. and Akin, D. (1996). The ranger telerobotic �ight experiment : Mission, techno-logies, and programmatics. In proc. of Conference on Robotics for Challenging Environ-ments, Albuquerque, USA, Jun. 1996.

Peters II, R. and Campbell, C. (1999). Robonaut task learning through teleoperation. Inproc of IEEE Inter-national Conference on Robotics and Automation, Taipei, Taiwan,September, 2003.

Pongrac, H. (2008). Gestaltung und Evaluation von virtuellen und Telepräsenzsystemen anHand von Aufgabenleistung und Präsenzemp�nden. PhD thesis, Human Factors Institute,Universität der Bundeswehr, 2008.

Preusche, C., Reintsema, D., Landzettel, K., and Hirzinger, G. (2006). ROKVISS � Preli-minary Results for Telepresence. In proc. of the International Conference on IntelligentRobots and Systems (IROS), Peking, China.

Rumford, T. (2003). Demonstration of autonomous rendezvous technology (dart) projectsummary. In proc. of Space Systems Technology and Operations Conference, Orlando,USA, Apr. 2003.

Sommer, B. (2003). Automation and robotics in the german space program - unmanned on-orbit servicing (oos) and the tecsas mission. In proc. of 55th International AstronauticalCongress, Vancouver ; Canada, Oct. 2004.

Stoll, E. (2008). Ground Veri�cation of Telepresence for On-Orbit Servicing. PhD thesis,Institute of Astronautics, Technische Universität München, 2008.

Tarabini et al., L. (2007). Ground guided cx-olev rendezvous with uncooperative geosta-tionary satellite. Acta Astronautica ; Selected Proceedings of the 57th IAF Congress,Valencia, Spain, Oct. 2006, 61(1-6).

Tfaily, R. e. a. (2003). Haptic virtual environment performance over ip networks : A casestudy. In proc. of 7th IEEE International Symposium on Distributed Simulation andReal-Time Applications, Delft, The Netherlands, Oct. 2003.

Yokokohji, Y. and Yoshikawa, T. (1994). Bilateral control of master-slave manipulators forideal kinesthetic coupling-formulation and experiment. Robotics and Automation, IEEETransactions on, 10 :605�620.