Gennaro Cordasco , Rosario De Chiara Ugo Erra and … › ~cordasco › papers › ICUMT09 ›...
Transcript of Gennaro Cordasco , Rosario De Chiara Ugo Erra and … › ~cordasco › papers › ICUMT09 ›...
Gennaro Cordasco1, Rosario De Chiara1,
Ugo Erra2 and Vittorio Scarano1
1ISISLab-Dipartimento di
Informatica ed Applicazioni
Università degli Studi di Salerno
Italy
2Dipartimento di
Matematica e Informatica
Università degli studi della Basilicata
Italy
Outline
� Massive Simulation Environment (MSE)
� Motivation and Challenge
� Distributed Massive Simulation Environment (DMSE)
Issues in designing DMSEs� Issues in designing DMSEs
� Workload Balancing: The Prediction Binary Tree (PBT)
� Our Proposal
� Conclusion
International Conference on Ultra Modern Telecommunication (ICUMT 2009) 2
Massive Simulation Environment (MSE)
� 3D environment simulating a group of characters (aka
actors) whose behaviors are related to social
interactions and rules among group members
� MSEs have been investigated since the 1980s. (Military � MSEs have been investigated since the 1980s. (Military
Simulations)
� Early works take inspiration from particles system [R83]
� Trends:
� Larger scale
� More realistic simulation
International Conference on Ultra Modern Telecommunication (ICUMT 2009) 3
Massive Simulation Environment (MSE)
Flock Simulation:� [Reynolds87] Steering Behaviors
For Autonomous Characters
� [DEST04] Massive simulation
using GPU of a distributed
behavioral model of a flock with
Separation Alignment Cohesion
4
behavioral model of a flock with
obstacle avoidance
� [EFS09] A GPU-based Method for Massive Simulation of Distributed Behavioral Models with CUDA
Boid model� The boid model can be expanded by assembling basic
behaviors to obtain more complex behaviors:
Seek and Flee Pursuit and Evade Obstacle avoidance
International Conference on Ultra Modern Telecommunication (ICUMT 2009) 5
Seek and Flee Pursuit and Evade Obstacle avoidance
Alignment, Separation,
Cohesion
Flee, Pursuit, Offset Pursuit,
Seek, Evade
Alignment, Separation,
Cohesion
Flee, Pursuit, Offset Pursuit,
Seek, Evade
Alignment, Separation,
Cohesion
L/R flank, L/R Face, March
forward, at ease
Enemy pursuit, shooting,
evasion
Boid model
behaviors
Basic
behaviors
Directional
commands
Combat
commands
[Reynolds 1987] [Reynolds 1999 ] [BDS09]
Massive Simulation Environment (MSE)
� [BDS09] Massive Battle: Coordinated Movement of
Autonomous Agents Battle
6
Platoons are able to march following a path;
7
MVE vs MSE
World of Warcraft
8
Massive Battle
International Conference on Ultra Modern Telecommunication (ICUMT 2009)
DMSE Challenges
� Scalability
A single machine is not able to manage
thousands of player at the same time
(Massive battle is able of simulating at
DMSE
(Distributed
Massive (Massive battle is able of simulating at
most 5000 units)
� Responsiveness
� Consistency
� Reliability
International Conference on Ultra Modern Telecommunication (ICUMT 2009) 9
Massive
Simulation
Environment)
P2P-DMSE
� Each peer has two
subsystems:
� Simulation engine
� Visualization engine
Simulation
Engine
Visualization
Engine� Visualization engine
� Each engine defines its AOI (Area of interest)
� Simulation AOI (Region simulated and neighborhood)
� Visualization AOI (Field of view of the visualization
camera)
International Conference on Ultra Modern Telecommunication (ICUMT 2009)
Engine
10
P2P-DMSE
Simulation
Engine
Visualization
EngineEngine
Issues in designing DMSEs
� World Partitioning
� Load Balancing
� World State Propagation
International Conference on Ultra Modern Telecommunication (ICUMT 2009) 12
Similarity with Parallel Computing
� Minimize the completion time
� Decomposition: divide the whole job into tasks
� Tasks granularity
Load
Balancing
Communication
Costs
Locality of
Interaction
Spatial
Coherence
World Partitioning
� Mapping: allocate tasks to workers (peers) aiming to:
� Balance the overall load distribution
� Minimize tasks inter-processor dependency
International Conference on Ultra Modern Telecommunication (ICUMT 2009) 13
Balancing Costs Interaction Coherence
Fine-grained� xxxx xxxx xxxx
Coarse-grained xxxx � � �
Load Balancing
World State Propagation
World PartitioningUsing a DHT we can easily partition the environment map:
The peer whose ID is the closest to the region ID become the region
master for the region. Region masters are responsible to:
� Simulate all the actors which belong to the region;
� Deliver the state of the region (that is the state of each actor which
belongs to the region) to the peers whose AOI overlaps with the
region; region;
� Handle handovers of actors between regions.
14
Underlying networkDHT ID space
Peer
Region
Peer
Region
World Partitioning
Issues:
� Partitioning
� Static
� Dynamic
� Granularity
� Fine
Load balancing
Intertask communication
Locality of interaction
� Coarse
15
Underlying networkDHT ID space
Peer
Region
Peer
Region
Locality of interaction
World State Propagation
Publish/Subscribe design pattern:
� a multicast channel is assigned to each region
� users then simply subscribe to the channels associated
with the regions which overlap with their AOIwith the regions which overlap with their AOI
� the granularity of the World partitioning is again an
issue
International Conference on Ultra Modern Telecommunication (ICUMT 2009) 16
Interaction between zone
� A graph G=(N,E) is used to model zone and their
dependencies:
� Each node v ∈ N represents a zone to be simulated
� Edge among nodes represents zone dependencies� Edge among nodes represents zone dependencies
International Conference on Ultra Modern Telecommunication (ICUMT 2009)
17
Tiled Mapping Strategies
� We are interested in tiled mapping strategies where the
whole map is partitioned into Regions (contiguous 2-
dimensional blocks of zones).
� Tiled mapping allows to exploit the local dependencies � Tiled mapping allows to exploit the local dependencies
among zones:
� Locality of interactions
� Spatial coherence
International Conference on Ultra Modern Telecommunication (ICUMT 2009) 18
Step-wise computations
� Simulations are performed in successive phases
� Temporal coherence: The amount of time required by a
Region r in phase f is comparable to the amount of time
required by r in phase f+1
Phase f Phase f+1
International Conference on Ultra Modern Telecommunication (ICUMT 2009) 19
Phase f Phase f+1Region r
The Prediction Binary Tree [CCDES08]
� A semi-static coarse-grained decomposition/mapping
strategy for parallel mesh-like computation which exploits
temporal coherence
� semi-static: decisions are made before each computing � semi-static: decisions are made before each computing
phase
� coarse-grained: to reduce communication and exploit the
local dependencies among tasks
� mesh-like: the strategy is based on tiled mapping
� temporal coherence is exploited in order to balance task
sizes
International Conference on Ultra Modern Telecommunication (ICUMT 2009) 20
The Prediction Binary Tree (PBT)
� The PBT is in charge directing the tiling-based mapping
strategy:
� Each computing phase is split into m tiles
� Each leaf of the PBT represents a tile
International Conference on Ultra Modern Telecommunication (ICUMT 2009) 21
The Prediction Binary Tree (PBT)
� Before each new phase tile sizes are adjusted according
to the previous phase
International Conference on Ultra Modern Telecommunication (ICUMT 2009) 22
light tasks
heavy task
The Prediction Binary Tree (PBT)
� Before each new phase tile sizes are adjusted according
to the previous phase
International Conference on Ultra Modern Telecommunication (ICUMT 2009) 23
Improved balancing
Our Proposal
Hybrid approach
Peer Subsystems:
� Simulation engine
provides computing resource to a distributed simulation � provides computing resource to a distributed simulation
environment
� uses a DHT as a Data Storage and for World states propagation
� Visualization engine
� works on its own
International Conference on Ultra Modern Telecommunication (ICUMT 2009) 24
Our Proposal: Load Balancing
� Two level approach:
� Coarse grained static decomposition of the whole Map
Environment into Regions. Regions are assigned to nodes
(region masters) using a DHT scheme(region masters) using a DHT scheme
25
Underlying networkDHT ID space
Peer
Region
Peer
Region
Our Proposal: Load Balancing
� Two level approach:
� Coarse grained static decomposition of the whole Map
Environment into Regions. Regions are assigned to nodes
(region masters) using a DHT scheme(region masters) using a DHT scheme
� Each region is dynamically partitioned exploiting the PBT
idea
26International Conference on Ultra Modern Telecommunication (ICUMT 2009)
Our Proposal: Load Balancing
27
Underlying network
DHT ID space
Region dynamic partition
Peer
Static region
Dynamic sub-region
Peer
Static region
Dynamic sub-region
PBT
Our Proposal
Underlying networkDHT ID space
Region dynamic partition
Peer
Static region
Dynamic sub-region
Peer
Static region
Dynamic sub-region
PBT
� A forest of PBT maintained by Region masters: After each
computing phase the PBT is updated
� Region master is also responsible of keep interested
nodes aware about the state of the PBT
� The simulation is performed by the nodes which are
responsible of the PBT leaves
International Conference on Ultra Modern Telecommunication (ICUMT 2009) 28
Region dynamic partition
Conclusion
� MSEs due to their scalability requirement appear to be a
natural application for P2P architectures
� As any distributed computing platform, DMSEs based on
P2P approach present several issues (Load balancing /
Communication trade-off).Communication trade-off).
� We presented a P2P architecture which exploits the
feature of both DHT schemes and a novel data structure
(PBT) in order to provide a balanced workload without
introducing excessive communication.
� We are working on the implementation of this
architecture for a distributed version of Massive Battle
International Conference on Ultra Modern Telecommunication (ICUMT 2009) 29
Salerno
ITALY
Gennaro Cordasco, Rosario De Chiara, Ugo Erra, Vittorio Scarano
http://www.isislab.it
References[R83] Reeves W. T., “Particle Systems-A Technique for Modeling a Class of Fuzzy Objects”. In ACM
Transactions on Graphics, V2–2, April 1983. and reprinted in Computer Graphics. V17–3, July
1983, (ACM SIGGRAPH ’83 Proceedings), pp. 359-376.
[R87] Reynolds C., “ Flocks, herds and schools: a distributed behavioral model”. In SIGGRAPH’87:
Proceedings of the 14th Annual Conference on Computer Graphics and Interactive
Techniques, ACM, New York, NY, USA, 1987.
[R99] Reynolds C., “Steering behaviors for autonomous characters”. In Game Developers
Conference, Miller Freeman Game Group, San Francisco, CA, USA, 1999.
[DEST04] De Chiara R. , Erra U., Tatafiore M., and Scarano V., “Massive simulation using GPU of a [DEST04] De Chiara R. , Erra U., Tatafiore M., and Scarano V., “Massive simulation using GPU of a
distributed behavioral model of a flock with obstacle avoidance ”. In Proceedings of Vision,
Modeling, and Visualization 2004 (VMV 2004) pp. 233-240.
[R06] Reynolds C., “ Big fast crowds on PS3”. In Sandbox’06: Proceedings of the 2006 ACM
SIGGRAPH Symposium on Videogames, ACM, New York, NY, USA, 2006.
[BDS09] Boccardo A., De Chiara R., and Scarano, V., “Massive Battle: Coordinated Movement of
Autonomous Agents”. In Proceedings of the Workshop on 3D Advanced Media In Gaming And
Simulation (3AMIGAS), 2009
[CCDES09] Cordasco G., Cosenza B., De Chiara R., Erra U., and Scarano, V., “Experiences with
Mesh-like computations using Prediction Binary Trees”. In Scalable Computing: Practice and
Experience, Scientific international journal for parallel and distributed computing (SCPE), Vol.
10, ISSN: 1895-1767, pages 173-187, June 2009.
International Conference on Ultra Modern Telecommunication (ICUMT 2009) 31