EE788 Robot Cognition and Planning, Prof. J.-H. Kim
Lecture 6. Cognitive Robot ArchitectureLecture 6. Cognitive Robot Architecture
Robot Intelligence Technology Lab.
Contents
1. INTRODUCTION2. MULTIAGENT-BASED COGNITIVE ROBOT ARCHITECTURE
- Reflective Processes- Cognitive architecture- Self Agent- Machine Consciousness
3. MEMORY STRUCTURE- Short-Term Memory: The Sensory EgoSphere
2Robot Intelligence Technology Lab.
- Short-Term Memory: The Sensory EgoSphere- Long-Term Memory: Procedural, Episodic, and Declarative Memories- Working Memory
4. COGNITIVE CONTROL AND CENTRAL EXECUTIVE AGENT- Cognitive Control- Central Executive Agent
5. CURRENT COGNITIVE CONTROL EXPERIMENT6. CONCLUSIONS
1. INTRODUCTION
n The new generation of robots should be able to recognize and deal with situations in which its traditional reactive and reasoning abilities fall short of meeting complex task demands.
n A cognitive robot as a robot that knows what it is doing and reflects on past experience to deal with new situations.
3Robot Intelligence Technology Lab.
n Cognitive control in humans is the ability to “consciously manipulate thoughts and behaviors using attention to deal with conflicting goals and demands”
1. INTRODUCTION
n Cognitive modeling: approaches and architecturesl Neural network and symbolic approaches
n Computational modelsl Low-level cognitive processes: perception, attention and
memory
4Robot Intelligence Technology Lab.
l Higher-level cognitive processes: language and reasoning
n Major issues and controversiesl Should computational models be thought of as theories?
l Is the architecture of cognition symbolic or subsymbolic?
l How is the world reflected in the mind?
1. INTRODUCTION
n Why emotions?l Rationality and emotionality are not opposed or
even contradictory. Emotionality is a prerequisite of rational behavior.
l Humans treat computers like persons. If people have emotional relations to computers, why not
5Robot Intelligence Technology Lab.
have emotional relations to computers, why not make the computers recognize these emotions or make them express emotions?
l In computer animation, to give synthetic actors some autonomy, personality models have to be developed in which emotions play a major role.
1. INTRODUCTION
n Cognitive humanoid robot, ISACl Cognitive robot architecture: three distinctive memory structure
- Short-term memory- A sparse data structure called “Sensory Egosphere”
containing spatio-temporal sensory data acquired within a recent time frame
6Robot Intelligence Technology Lab.
- Long-term memory- Composed of behaviors, semantic knowledge and
past experience- Working memory
- Allows attention on the most relevant features of the current task
1. INTRODUCTION
7Robot Intelligence Technology Lab.
Fig 1.1 Robotic system with structured intelligence
1. INTRODUCTION
8Robot Intelligence Technology Lab.
2. MULTIAGENT-BASED COGNITIVE ROBOT ARCHITECTURE
9Robot Intelligence Technology Lab.
2.1 Introduction
n Three key cognitive abilities a robot must have in order to effectively interact with humans:l Self-awareness: the robot must be able to reason about
the status of its own internal processing.
l Awareness of other: the robot must be able to reason about
10Robot Intelligence Technology Lab.
l Awareness of other: the robot must be able to reason about the states of the humans with which it interacts.
l Self-reflection: it allows the robot to reason about its own abilities, cognitive processes, and knowledge.
2.1 Introduction
n Earlier hybrid architectures to build machines capable of perceiving and interacting with the world around them:l explicit, often logical, knowledge representation schemes and
formally justified techniques for manipulating internal representations. l dynamic, and sometimes complex, interactions between modular
primitive reactive processes and the world.à Robots being both fluent in routine operations and capable of
11Robot Intelligence Technology Lab.
à Robots being both fluent in routine operations and capable of adjusting their behavior in the face of unexpected situations or demands.
n A fully cognitive robot should be able to recognize situationsin which its reactive and reasoning abilities fall short of meeting task demands, and it should be able to make reasoned modifications to those abilities in hopes of improving the situation.
n DARPA has recently announced a new research initiative in cognitive systems – systems that possess levels of autonomy and reasoning far beyond those of today’s systems. l Cognitive systems will literally be systems that know what
they are doing.They will include both reactive and deliberative processes and
2.1 Introduction
12Robot Intelligence Technology Lab.
l They will include both reactive and deliberative processes and will also incorporate mechanisms for self-reflection and adaptive self-modification (see Figure 2.1).
n A humanoid robot ISAC (Intelligent Soft-Arm Control)l To work with humans as an assistant in a variety of settingsl A multi-agent software architecture for parallel and distributed
robot control and a robust human-robot interface
2.1 Introduction
13Robot Intelligence Technology Lab.
Figure 2.1 A Cognitive System Architecture
2.1 Introduction
n The development and maintenance of complex orlarge-scale software systems can benefit from domain-specific guidelines that promote code reuse and integration through software agents.
n Information processing in ISAC
14Robot Intelligence Technology Lab.
n Information processing in ISACl From perception through action executionl Integrated into a multi agent-based software architecture
based on the Intelligent Machine Architecture (IMA)l The IMA was designed to provide such guidelines and
allows for the development of subsystems capable of environmental modeling and robot control through the collections of IMA agents and associated memories.
2.2 Reflective Processes
Reflection in Multi-Agent Systems
n Reactive systems:l Collections of interacting reactive agents can adapt in
a sometimes surprising way to varying environments.
l The systems often rely on the emergence of useful dynamic
15Robot Intelligence Technology Lab.
patterns from the complex interaction of simple components.
l Quite difficult to discover a functional decomposition and a coupling of agents that reliably produces adaptive behavior across a wide range of contexts.
Reflection in Multi-Agent Systems
n Deliberation processesl Common sentential knowledge representation schemes
support the design process - specific bits of knowledge (e.g., rules) can be incorporated without danger of catastrophic interactions with the rest of the knowledge base.
16Robot Intelligence Technology Lab.
base.
l While deliberation may be too inefficient to guide the system through stereotyped behaviors, it does provide many benefits when circumstances are unusual and unexpected.
Reflection in Multi-Agent Systems
n Reflective processesl reason about the internal information processing processes
of the system and the knowledge and mechanisms that guide the selection of actions.
l Note that deliberative components of the system reason about the external world and the actions that can be taken in it.
l A cognitive system will need to perform such reflective
17Robot Intelligence Technology Lab.
l A cognitive system will need to perform such reflective reasoning, analyzing and modifying its own reactive and deliberative mechanisms.
l Reflection allows a system to "know what it knows" and, potentially, to "know what it needs to find out.“
l Reflective processes can both guide autonomous learning and provide a means for performance improvement through interaction with humans.
Reflection in Multi-Agent Systems
n Multiple software agents based architecturel A greater challenge with regard to a reflective process's ability
to reason about the way in which the collection of agents collectively process information.
l Placing critical knowledge in shared databases addresses part of this challenge.
18Robot Intelligence Technology Lab.
n Ways to address this challenge, characterized by the answers to the following interrelated design questions:l HOW/WHAT/WHEN does the reflective process come to know
about the information processing behavior of agents and collections of interacting agents?
Self-Reflective Processes in ISAC
n Reflective processes, interacting with the Self Agent, will continuously gather statistics about the distribution of inputs and outputs observed for each individual agent.
n When an impasse arises, these statistics will be used as heuristic guides for identifying the potential source of the problem.
19Robot Intelligence Technology Lab.
n At the point of impasse, reflective processes will experiment with the existing collection of agents, running them in “simulation mode” in order to predict the likely outcome of various control manipulations.
Self-Reflective Processes in ISAC
n These predictions will then be fed into a utility-based analysis in order to select a new collection of control parameters, which will then be used to address the impasse.
n Finally, the reflective processes will compare the actual behavior of the system to that which was “simulated” during experimental
20Robot Intelligence Technology Lab.
of the system to that which was “simulated” during experimental reflection, recording statistics on this accuracy for use when estimating the reliability of future simulations.
2.3 MULTIAGENT-BASED COGNITIVE ROBOT ARCHITECTURE
21Robot Intelligence Technology Lab.
ISAC’s Cognitive Architecture
n The Intelligent Machine Architecture (IMA)l A multi-agent based software architecture
l An agent-based software system that permits robot control through collections of cooperating software agents
l Information processing in ISAC, from perception through action selection and execution, is integrated into.
22Robot Intelligence Technology Lab.
action selection and execution, is integrated into.
l For the development of subsystems capable of environmental modeling and robot control through the fabrication of collections of IMA agents and associated memories.
l An architecture for concurrently executing IMA agents on separate machines that perform as a group through extensive inter-agent communication.
ISAC’s Cognitive Architecture
23Robot Intelligence Technology Lab.
Figure 2.2 Multi-Agent Based Cognitive Architecture
ISAC’s Cognitive Architecture
n Atomic IMA agentsl Autonomous entities that have one or more execution
threads.l Typically, an atomic agent cannot perform useful activity
independently. l Collections of IMA agents interact to complete tasks
by providing capabilities, such as sensing the external
24Robot Intelligence Technology Lab.
by providing capabilities, such as sensing the external environment.
n Various types of IMA agents for robot controll Hardware agents for accessing sensors and actuators, environment
agents for abstracting object and environment interactions. l Within IMA, the robot itself is abstracted as a self-agent (SA), and
the state of external entities, such as people, is abstracted in the form of human agents.
ISAC’s Cognitive Architecture
n A Human Agent and Sensory EgoSphere (SES) l To represent the external environment
n The Self Agent and the Long Term Memory (LTM) l To provide memory and internal control mechanisms
25Robot Intelligence Technology Lab.
for ISAC.l The self-agent uses a memory database structure,
consisting of short-term and long-term structures, to determine the appropriate situational control commandsfor the robot.
2.4 The Human Agent
n The Human Agentl To determine the current state of interacting humans from
observations and from explicit communications with those humans
l The Human Agent’s assessment of how to interact with a local human is also passed on to the Self Agent, where it can
26Robot Intelligence Technology Lab.
human is also passed on to the Self Agent, where it can influence deliberations concerning the robot’s own intentions.
l The Self Agent then interprets this suggestion in the context of its own current state, e.g. current intention, status, tasks, etc.
l This processing guides the robot to choose socially appropriate behaviors, leading toward an ultimate goal of completing tasks with (or for) people.
2.5 The Self Agent
n The Self Agent: a cognitive agentl For monitoring sensor signals, agent communications, and
high-level (i.e. cognitive) decision-making aspects of ISAC.
l Integrates failure information from sensors and maintains information about the task-level status of the humanoid.
27Robot Intelligence Technology Lab.
l The cognitive aspects: include recognition of the human intention from the Human Agent and coordination of appropriate actions by activating meta-level behaviors within the LTM.
2.5 The Self Agent
Emotion Agent
28Robot Intelligence Technology Lab.
Figure 2.3 Structure /Functions of Self Agent in IMA
2.5 The Self Agent
n The Central Executive Controller (CEC)l To coordinate the various Procedural Memory structures stored
in the Long Term Memory that encapsulate ISAC's behaviors. l These behaviors may be invoked as atomic steps in plans
constructed by the CEC to perform various novel tasks.l The goal of each generated plan is determined by input from
the Intention Agent.
29Robot Intelligence Technology Lab.
the Intention Agent. l Constructed plans are put into action by activating appropriate
atomic agents. l Component actions are molded to the current task situation
through the adjustment of parameters to meta-level behaviors, guided by the descriptive memory residing in the LTM, sensory information received from atomic agents, and input from the SES.
2.5 The Self Agent
n The Intention Agentl To determine the intended action of the humanoid based on
the intention of the human, the state of the humanoid, and whether this action would conflict with the humanoid's current activities.
30Robot Intelligence Technology Lab.
n The Emotion Agent l Emotional state of the robot describes what the robot feels toward
the task and the environment based on past experience.l Adding an Emotion Agent to the Self Agent to conduct cognitive
control experiments.
2.5 The Self Agent
n Anomaly Detection Agent (ADA): l A reflective process
l monitoring the inputs and outputs of the atomic agents in the system and will gather statistics concerning the distributions of these inputs and outputs, conditioned on the current intention maintained by the Self Agent
31Robot Intelligence Technology Lab.
Figure 2.4 Anomaly Detection Agent
2.5 The Self Agent
n Mental Experimentation Agent (MEA): l Another reflective processl When an impasse is raised, it will be initially assumed that
the fault involves the selection of an improper action sequence. Thus, the MEA initially invokes the Central Executive Controller to produce an action sequence appropriate for current task conditions and the current intention.
32Robot Intelligence Technology Lab.
2.5 The Self Agent
n If both the planning mechanism of the Central Executive Controller and the control parameter search of the MEA fail to resolve the impasse, this reflective process will prompt a query to nearby humans, orchestrated by the Human Agent, in hopes that human assistance will result in the removal of the problem.
33Robot Intelligence Technology Lab.
the problem.
2.5 The Self Agent
n This initial attempt to integrate self-reflective reasoning into ISAC will result in a system capable of increased flexibility through self-diagnosis and self-correction.
n This approach focuses exclusively on the diagnosis of problems that are immediately detected – that
34Robot Intelligence Technology Lab.
of problems that are immediately detected – that immediately produce an impasse.
n It is likely that some form of episodic memory will have to be provided to ISAC in order to allow it to fully reflect on its own limitations.l It cannot search the sequence of recently performed actions
for the cause of a task failure, because it does not record these actions or even the states through which it passes.
2.5 The Self Agent
n Machine Consciousnessl A software agent called the Self Agent that uses emotion, attention
and cognitive control to model “consciousness” for ISAC.
l Through a set of tightly-coupled atomic agents trying to achieve a common goal of which concept was inspired by Minsky’s work in the Society of Mind
35Robot Intelligence Technology Lab.
l Phenomenal consciousness, sometimes called sentience, or subjective (first person) experience
l Self-consciousness, that is being aware of oneself. This often includes one’s self image.
2.5 The Self Agent
l Functional consciousnessif its architecture and mechanisms allow it a number of followings and, perhaps, other functions:
- It helps us deal with novel or problematic situations for which we have no automatized response.
- It makes us aware of potentially dangerous situations.
36Robot Intelligence Technology Lab.
- It alerts us to opportunities presented by the environment.
- It allows us to perform tasks that require knowledge of location, shape, size or other features of objects in our environments.
- And there are a number of other functions of consciousness.
2.5 The Self Agent
n Self Agentl represents the sense of self through monitoring the robot’s
own internal state as well as the progress of task executionvia sensor signals, agent communications and working memory.
l The internal representation of the robot’s self should continually be updated and enhanced to allow the system
37Robot Intelligence Technology Lab.
continually be updated and enhanced to allow the system to reason and act based on its status and the context of assigned tasks.
l responds to commands given by humans through the Human Agent and is responsible for controlling task execution.
Self Agent
38Robot Intelligence Technology Lab.
3. MEMORY STRUCTURE
39Robot Intelligence Technology Lab.
3.1 Introduction
n An adaptive working memory: l For robot control and learning, closely tied to the learning,
execution of tasks, decision making capabilities by focusing on essential task information and discarding distractions
n To integrate the memory structure into a robot to explore the issues of task learning in a physical embodiment. l This leads to a complex but realistic system involving
40Robot Intelligence Technology Lab.
l This leads to a complex but realistic system involving perceptual systems, actuators, reasoning, and short-term and long-term memory structures.
l inspired by the hypothesized contribution that the prefrontal cortex (PFC) of the human brain makes to working memory and cognitive control.
n Planned experiments intended to evaluate the utility of the adaptive working memory.
3.1 Introduction
n The goal is to support the robot with the followings:l Focus attention on the most relevant features of the current task.
l Support generalization of tasks without explicitly programming the robot.
l Guide perceptual processes by limiting the perceptual search space.
41Robot Intelligence Technology Lab.
space.
l Provide a focused short-term memory to prevent the robot from being confused by occlusions, i.e., to avoid the out of sight, out of mind, problem.
l Provide robust operation in the presence of distracting irrelevant events.
3.2 MEMORY STRUCTURE
n Short-Term Memory (STM), Long-Term Memory (LTM), and the Working Memory System (WMS).
n The STM holds sensory information about the current environment.
n The LTM holds learned and taught behaviors, semantic knowledge, and past experience.
42Robot Intelligence Technology Lab.
knowledge, and past experience. n The WMS holds task-specific STM and LTM information and
streamlines the information flow to the cognitive processes during the task
3.2.1 Short-Term Memory: The Sensory EgoSphere
n A sparse sensory data structure, called the Sensory EgoSphere (SES) to hold STM data.
n The SESl Inspired by the egosphere concept, defined by Albus, serves as
a spatio-temporal STM for a robot.Structured as a geodesic sphere centered between the robot's
43Robot Intelligence Technology Lab.
l Structured as a geodesic sphere centered between the robot's cameras and indexed by azimuth and elevation.
l Each vertex of the geodesic sphere contains a database representing a detected stimulus at the corresponding angle
l Combines a geodesic dome interface with a database to store sensory events that occur in the robot’s environment.
l The manager queries the database of the SES to register data, to retrieve data and to remove old data.
Short Term Memory and Attention
n A definition for visual attention:
Fig. 3.1 SES
44Robot Intelligence Technology Lab.
n A definition for visual attention:l Attention is the allocation of resources. l It is a mechanism that chooses which part of the image will
get the computational resources.
n An attention network in the SES: l To focus ISAC’s resources on the most significant area of
the environment: - Bottom-up (or stimulus-driven) and Top-down (or goal-directed)
Bottom-up architecture for attention
45Robot Intelligence Technology Lab.
3.2.2 Long-Term Memory
n LTM including PM, DM, EM, stores information such as skills learned and experiences gained for future retrieval.
n Procedural memory (PM) contains generic behaviors:l Motion primitives and behaviors needed for movement
- An example of PM is “how to reach and grasp an object.”l To generate such behaviors,
46Robot Intelligence Technology Lab.
- A spatial-temporal Isomap approach- Motion data are collected from teleoperation then segmented into
a set of motion primitives. Then S-T Isomap dimension reduction, clustering and interpolation methods are applied to the motion segments to produce motion primitives and behaviors
- The Verbs and Adverbs method- Another data structure containing declarative memory within the LTM.
Long-Term Memory
47Robot Intelligence Technology Lab.
Fig. 3.2 Derivation of Procedural Memory through human-guided motion stream.
Long-Term Memory
n Declarative memory (DM):l A data structure about objects in the environment. l Goals and task sequences are stored as DM units.
n Episodic memory (EM):l To store past experience.
48Robot Intelligence Technology Lab.
l To store past experience.
STM and LTM
n The short-term memory (STM)l uses the Sensory EgoSphere (SES) to represent short-term
external events in the environment. l The SES is a data structure that provides a short-term-memory
to store events, such as the state of external human agents.l To provide an estimate of the current external state for
determining appropriate task-level intentions for the robot.l Based on these intentions, the self-agent uses procedures
49Robot Intelligence Technology Lab.
l Based on these intentions, the self-agent uses procedures in LTM to provide control commands to accomplish the robot's intentions.
n The long-term memory (LTM)l contains information about procedures considered intrinsic
to the robot. l A self-agent uses derived behaviors as procedures to produce
motion control for achieving robot objectives.
3.2.3 Working Memory
n Working memory:l What allows you to remember the telephone number that
the directory assistance operator just recited, retaining it only long enough to dial it yourself.
l What allows you to stay focused on the search for a specific product on the shelves of a grocery store
50Robot Intelligence Technology Lab.
a specific product on the shelves of a grocery store after looking up from your shopping list.
l What allows you to keep track of the particular location in which you last had a clear view of a stalking wolf.
Working Memory System
n Closely tied to task learning and executionn Represents a limited-capacity store
l for retaining information over the short term and l for performing mental operations on the contents of
this store.
n To develop a more complex, but realistic robotic
51Robot Intelligence Technology Lab.
n To develop a more complex, but realistic robotic learning system involving perceptual systems, actuators, reasoning, attention, emotion, and short-and long-term memory structures.
n The backboard of the mindn Includes the Central Executive Agent and short- and
long-term working memories.
The Role of Working Memory
n Short-term memoryl For visual-spatial information and for speech-based information
(Baddeley 1986).
n Long-term memoryl To include somewhat segregated systems for semantic
knowledge and for specific episodes (Tulving 1972)l Memory in general has been found to have both implicit
52Robot Intelligence Technology Lab.
Memory in general has been found to have both implicit (i.e., inaccessible to consciousness) and explicit components(Warrington and Weiskrantz 1968).
n Working memory:l The memory that actively maintains transient information that is
critical for successful decision-making in the current context.l A mechanism that protects a small number of informational
“chunks” from interference and distraction and places them in a position to directly influence behavior (Goldman-Rakic ’87).
The Role of Working Memory
l Updated quickly, with its contents dynamically manipulated at sub-second rates, while many longer-term memory systems are much slower to adapt (Waugh and Norman 1965).
l Its contents are seen as readily available to deliberative and executive control processes (Norman and Shallice 1986).
l Its limited capacity constrains the amount of information immediately available to explicit reasoning and executive control processes.
53Robot Intelligence Technology Lab.
à Its contents must be carefully selected.l Capable of representing and actively maintaining different types
of information, including information about spatial locations, about recently viewed objects, about sought or expected objects, and about rules for action, and also the encoding of verbal forms of information (Demb et al. 1995).
l The activity of prefrontal cells (PFC) specifically capture information that is critical to the performance of the current task (Rainer et al. 88).
Initial Working Memory Efforts
n Its small size and tight coupling with deliberation mechanisms alleviates the need for costly memory searches or retrievals. l Information needed to fluently perform the current task
is temporarily kept “handy” in the working memory store.
54Robot Intelligence Technology Lab.
n The primary challenge for such a working memory system l Determining whether a given chunk of information should
be maintained in working memory or not.
l Learning when to store a particular chunk of informationin working memory.
Initial Working Memory Efforts
n The problem of learning when to update the contents
of working memory
l The PFC receives dense inputs from neurons in the midbrain
that communicate using a neurotransmitter called dopamine
for changes in expected future reward.
55Robot Intelligence Technology Lab.
l These dopamine neurons are involved in a kind of
reinforcement learning called temporal difference (TD)
learning, since this class of learning algorithms depends
critically on a measure of change in expected future reward.
Initial Working Memory Efforts
n Initial efforts towards integrating an adaptable working memory into robotic systems that have the ability to interact physically with their environment.
n Using a model of working memory based on temporal difference learning, as inspired by evidence in
56Robot Intelligence Technology Lab.
difference learning, as inspired by evidence in cognitive neuroscience, a working memory structure for robotic systems is being implemented.
4. COGNITIVE CONTROL AND THE CENTRAL EXECUTIVE AGENT
57Robot Intelligence Technology Lab.
4.1 Cognitive Control
n A goal of cognitive robotl To be fluent in routine operations and capable of adjusting
behaviors in the face of unexpected situationsn Cognitive control in humans is the ability to “consciously
manipulate thoughts and behaviors using attention to deal with conflicting goals and demands”
n Cognitive control is thought to be useful for a cognitive robotduring the action selection process
58Robot Intelligence Technology Lab.
during the action selection processl It guides the robot through the search for component behaviors
that might be combined and used efficiently to execute routine tasks as well as to appropriately respond in novel situations.
n As levels of human behavioral processes range from reactive to full deliberation, cognitive control must be able to switch between these levels to cope with the demand of task and performance, particularly in novel situations.
4.1 Cognitive Control
n Biological evidence of cognitive control in humansl can be found in the function of the basal ganglia
thalamocortical system, the prefrontal cortex (PFC), and the anterior cingulated cortex (ACC).
l Basal ganglia are involved in the planning and execution of complex motor and cognitive acts.
59Robot Intelligence Technology Lab.
complex motor and cognitive acts.
l The PFC is involved in guiding these actions by supportingrepresentations of relevant information from interference due to competing information.
l The ACC is involved in detecting and helping to resolve response conflicts during a task performance.
4.1 Cognitive Control
n Cognitive control in human is performed through the working memory in the pre-frontal cortex (PFC).
n Furthermore, attention and emotion play an important role in human’s decision and task execution.
n One classical model of working memory by Baddely and Hitch in which the control of executive processes
60Robot Intelligence Technology Lab.
and Hitch in which the control of executive processes is done by a component called the Central Executive.
n Cognitive control in ISAC is implemented using the Central Executive Agent (CEA) that interfaces with the WMS which allows task related information to be actively maintained during a task execution.
4.2 Central Executive Agent
n CEA’s functions: Task planning, action selection, and action execution
n Goal-oriented behavior selection and execution is being done in a modular fashion. l Upon receiving a command, the CEA associates a set of
behaviors based on past experience and places them in the WMS.
61Robot Intelligence Technology Lab.
n State estimators produce estimated states to calculate task relevancies of each behavior according to the goal.
n The behavior selector computes time-varying weights wibased on task relevancies to combine behaviors to generate the final action.
n Results from the task execution are used to calculate expected rewards for the TD-Learning.
4.2 Central Executive Agent
62Robot Intelligence Technology Lab.
Fig. 4.1 Interaction between the CEA and WMS during a task execution
4.3 Enabling Robotic Components
n Central Executive (CE)l For high-level executive functions such as a goal-directed
action selection process called cognitive control.
l The CE and the WM System perform the role of cognitive control.
63Robot Intelligence Technology Lab.
n Cognitive (or executive) controll “The ability of the brain to identify possible goals and figure out
how to achieve them … and ignore the distractions and impulses that would derail our goal directed efforts” (Miller, 03)
4.3 Enabling Robotic Components
64Robot Intelligence Technology Lab.
Fig. 4.2 MultiAgent-based Cognitive Robot Architecture
4.3 Enabling Robotic Components
65Robot Intelligence Technology Lab.
4.3 Enabling Robotic Components
n All behaviors taught or learned are stored in the Long-Term Memory (LTM) (Erol et al. 2003).l A behavior controller and a state estimator are assigned to
each behavior. l When a new command or goal is given, appropriate behaviors
are loaded into the WMS based on TD-Learning.The CE then selects behaviors based on the task relevancy.
66Robot Intelligence Technology Lab.
l The CE then selects behaviors based on the task relevancy. Task relevancy for a behavior, Bi, is defined as
where the error, ei is defined as the difference between the current state and the estimated state.
4.3 Enabling Robotic Components
n The state estimator is a component of a behavior module that has been loaded into the WMS. l The state estimator predicts the state of the system at time t+1
for its assigned module. l This estimated state is used to calculate task relevancy for the
system to evaluate movements most relevant to the goal.
67Robot Intelligence Technology Lab.
n The role of the TD-Learning systeml To select the behaviors that will be appropriate for the task at hand. l A new reward is calculated for each behavior module after
attempting the task with that module. l Sets of behaviors that are selected most often when performing
a task and sets of behaviors that complete the task most quickly will be rewarded.
4.3 Enabling Robotic Components
68Robot Intelligence Technology Lab.
4.4 Spatial Reasoning
n The capability of modeling spatial relations to support innate spatial cognition as well as linguistic, human-robot communication based on this cognition
n Interactive Spatial Languagel To provide capabilities of performing interactive experiments
to test the working memorye.g., “Find the object to the right of the cup.”
69Robot Intelligence Technology Lab.
e.g., “Find the object to the right of the cup.”l To be used by the robot to describe its environment, such as
“There is a cup on the table to the left of the telephone.”Human: “How many objects do you see?”Robot: “I see 4 objects.”Human: “Where are they located?”Robot: “There are two objects in front of me, and one object on my right.”Human: “The nearest object in front of you is a telephone. Place the cup
to the left of the telephone.”
4.4 Spatial Reasoning
n Spatial Representationsl The position of recognized objects will be stored in a robot-centric
frame called the Sensory Ego Sphere (SES) (Peters et al. 2001).l The SES is a database implementation of Albus’s proposed
egosphere (1991). This spatial database provides an egocentric view of the world which is consistent with the robot’s viewing perspective.
l The SES structure is a geodesic dome with a default frequency
70Robot Intelligence Technology Lab.
l The SES structure is a geodesic dome with a default frequency of 13, yielding a resolution of about 5 degrees with 1680 hexagonally-connected triangles.
l Each vertex can be labeled with an object identifier; some objects span multiple vertices.
l Objects may be retrieved using azimuth and elevation angles as indices into the database.
l An azimuth and elevation may also define a starting point in a search, for example, to look for an object in a specified region.
5. CURRENT COGNITIVE CONTROL EXPERIMENT
71Robot Intelligence Technology Lab.
5. CURRENT COGNITIVE CONTROL EXPERIMENT
n An integrated cognitive system experiment based on the CEA, attention, emotion and the adaptive working memory system:
1. ISAC is trained to learn specific object using voice, vision, attention. (Learn by association)
2. ISAC is asked to point to one of the learned objects. (Use of short-term memory of the object and long-term procedural
72Robot Intelligence Technology Lab.
(Use of short-term memory of the object and long-term procedural memory) (Figure 5.1)
3. ISAC is asked to visually track the object held by a human. (Color tracking)
4. A person enters the room and yells "Fire!" ISAC using attention, emotion and cognitive control, suspend the current tracking task and warn everyone to exit the room (Cognitive control).
5. CURRENT COGNITIVE CONTROL EXPERIMENT
73Robot Intelligence Technology Lab.
Fig. 5.1 ISAC is asked to point to one of the learned objects.
Fig. 5.2 Cognitive Control Experiment.
5. CURRENT COGNITIVE CONTROL EXPERIMENT
n In Step 4, ISAC's cognitive control must• pay attention to new stimulus and• use emotion to activate cognitive control.
n This experiment is being done through integrating the WM and cognitive control with the existing IMA
74Robot Intelligence Technology Lab.
the WM and cognitive control with the existing IMA agentsl to demonstrate that "The artificial cognitive machine is not
governed by any programs and therefore will not execute any preprogrammed decision commands like the IF-THEN ones.”
5. CURRENT COGNITIVE CONTROL EXPERIMENT
75Robot Intelligence Technology Lab.
6. CONCLUSIONS
n The next grand challenge will be in the integration of body and mind.
n This lecture has summarized the challenge for the realization of a cognitive robot using cognitive control, attention, emotion, and an adaptive working memory system, where multiagent-
76Robot Intelligence Technology Lab.
and an adaptive working memory system, where multiagent-based cognitive approach is an attempt to capture brain-style computation without necessarily committing to the neural-level details.
REFERENCES
6.1 A Multi-Agent Approach to Self-Reflection for Cognitive Robotics
6.2 A Biologically Inspired Adaptive Working Memory for Robots
6.3 The Sensory Ego-Sphere as a Short-Term Memory for Humanoids
77Robot Intelligence Technology Lab.
6.4 Modular Behavior Control for a Cognitive Robot
6.5 Development of a Robot with a Sense of Self
Top Related