Communication Architectures Split-screen Console - Limited players All peers equal -Easy to extend -Doesn’t scale (LAN only) One node server - Clients only to server -Server may be bottleneck Server pool -Improved scalability -More complex
Distributed Interactive Application (DIA) Networked – components on different machines Collaborative – multiple users working together Distributed – parts of environment on different machines allows a group of users connected via a network to interact synchronously with a shared application state. DIS is the name of a family of protocols used to exchange information about a virtual environment among hosts in a distributed system that are simulating the behavior of objects in that environment. It was developed by the US DoD to implement systems for military training, rehearsal, and other purposes.
DIA Consistency – Every entity must have the same view of the global state as every other entity in the entire network Scalability – An increase in users does not affect the efficiency of the network Security – No node can have advantage over another node Robustness – A failure of any participant has no effect on any other participant Availability – The network is perpetually accessible Real-Time – Processes are delivered no later than the time needed for effective control
Interactive Gaming common requirements: low latency (200 ms end-end) loss tolerant potentially large scale many-many, most “players both send and receive group structure (e.g., locality in communication) among players applications: distributed interactive simulation virtual reality distributed multi-player games
Consistency Consistency is the similarity of the view to the data in the nodes belonging to a network. Inverse: Responsiveness is the delay it takes for an update event to register throughout the network. Both are affected by bandwidth How do we maintain consistency and responsiveness with limited bandwidth? – Bandwidth reduction – Bucket Synchronization – Dead Reckoning
Consistency Why do we need consistency? Why do we get inconsistency? Example for Delay-Induced Inconsistency
Distributed Simulation: Vehicle Example Virtual environment simulation containing two moving vehicles One vehicle per simulator Each vehicle simulator must track location of other vehicle and produce local display (as seen from the local vehicle) Approach 1: Every 1/30th of a second: – Each vehicle sends a message to other vehicle indicating its current position – Each vehicle receives message from other vehicle, updates its local display
Consistency: Bandwidth LAN – 10 Mbps to 10 Gbps – Limited size and scope WANs – tens of kbps from modems, to 1.5 Mbps (T1, broadband), to 55 Mbps (T3) – Potentially enormous, Global in scope Number of users, size and frequency of messages determines bandwidth use
Communication Requirements: Vehicle Example Multiple players on 10 Mbits/sec Ethernet LAN DIS: each packet contains 144 bytes (1152 bits) Each vehicle generates position update every 1/30 second – 34,560 bits per second Upper bound: support 289 entities Above is extremely optimistic – Cannot utilize all of the Ethernet’s bandwidth – Entities generate other packets (e.g., weapon fires) – Multiple entities per human player (synthetic forces) 56Kbits/sec modem: at best, only one vehicle! Player 1 Player 2 Player 3 Player N …
Communication Issues Requires generating many messages if there are many vehicles – we need to economize on communication bandwidth Position information corresponds to location when the message was sent – doesn’t take into account delays in sending message over the network Need to address lack of information (missing or intermittent packets) – Bandwidth reduction – Interest Managment – Bucket Synchronization – Dead Reckoning
Reducing Bandwidth Packet compression – reduces the number of bits needed to represent particular information. – A lossless technique preserves all information while a – lossy technique leaves out less relevant information so that when data is reconstructed Packet Aggregation – merges several packets and transmits content in one larger packet, resulting in lower overhead caused by packet headers.
Reducing Bandwidth Interest Management – only distribute packets to nodes who are interested in them. – comprised of a player’s aura subspace where interaction occurs, so when two players’ auras intersect, they need to be aware of each other’s actions. – In gaming, aura is further divided into focus and nimbus, which translate into a player’s perception and perceptivity. While player A may see player B, player B does not have to see A. – nodes transmit changes to a subscription manager that holds all nodes’ information interests. The subscription manager is responsible for transmitting only relevant information to nodes. reduces bandwidth, it also increases processing time.
Interest Management: Focus and Nimbus -nimbus must intersect with focus to receive -Example above: hider has smaller nimbus, so seeker cannot see, while hider can see seeker since Seeker’s nimbus intersects hider’s focus
Addressing Latency: Bucket Synchronization All calculations are delayed until the end of each cycle The bucket cycles are typically 100ms (bucket frequency) Bucket frequency is set as a constant value which is equal to the rate that a human vision perceives smooth motion.
Incomplete Data: Dead Reckoning If a packet is lost or received too late, dead reckoning is used to estimate the “most probable” state or position of the object. The success of Dead Reckoning is based on the intelligence of the algorithm design There is inconsistency between the actual and expected states.
Dead Reckoning Send position messages less frequently DRM predicts the position of remote entities between updates – based on last position and velocity 10001050 1000 (1050,1000) last reported state: position = (1000,1000) traveling east @ 50 feet/sec predicted position one second later – When are updates sent? – How does the DRM predict vehicle position? Image Generator Dead reckoning model visual display system terrain database “infrequent” position update messages get position at frame rate
Re-synchronizing the DRM Compare DRM position with exact position, and generate an update message if error is too large Generate updates at some minimum rate, e.g., 5 seconds (heart beats)
t2t2 generate state update message true position DRM estimate of true position state update display update message E t1t1 Dead Reckoning Example Potential problems: Discontinuity may occur when position update arrives; may produce “jumps” in display Does not take into account message latency B C A DRM estimates position D receive message just before screen update
Time Compensation Taking into account message latency Add time stamp to message when update is generated (sender time stamp) Dead reckon based on message time stamp update with time compensation D E t2t2 t1t1 true position DRM estimate of true position state update display update message A B C
Smoothing Reduce discontinuities after updates occur “phase in” position updates After update arrives – Use DRM to project next k positions – Interpolate position of next update interpolated position D extrapolated position used for smoothing E t2t2 t1t1 true position DRM estimate of true position state update display update message – Accuracy is reduced to create a more natural display A B C
Dead Reckoning Summary Managing communications is a major issue in implementing distributed simulations Dead reckoning model (DRM) – Extrapolate current position based on past updates – Send update messages when DRM error becoming too large – Reduces interprocessor communication DRM based on equations of motion Time compensation to account for message latency Smoothing to avoid “jumps” in display
Real-Time Case Study: Age of Empire Age of Empire study: 250 milliseconds of command latency was not noticeable Between 250 to 500 msec was playable People develop a 'game pace' or mental expectation. Users would rather have a constant 500msec command delay rather than one that alternates between fast and slow. In excited moments users would repeat commands which would cause huge spikes in the network demand so a simple filter was placed to prevent reissuing of commands
Centralized Centralized Pros: Simplified administration Ease of maintenance Ease of locating resources Cons: Difficult to scale High cost of ownership Little or no redundancy Single point of failure
Distributed Distributed Pros: Highly extensible and scalable Highly fault tolerant Dynamic addition of new resources Cons: Difficulty in synchronizing data and state Scalability overhead can be large Extremely difficult to manage all resources
Security and Cheating Unique to games – Other multi-person applications don’t have – In DIS, military not public and considered trustworthy Cheaters want: – Vandalism – create havoc (relatively few) – Dominance – gain advantage (more) Distributed applications are more prone to cheating than centralized due to the fact that there is no authority supervising the actions of the users Security bears a trade-off of efficiency vs. fairness
Packet and Traffic Tampering: Suppress-correct cheat Reflex augmentation - enhance cheater’s reactions – Example: aiming proxy monitors opponents movement packets, when cheater fires, improve aim Packet interception – prevent some packets from reaching cheater – Example: suppress damage packets, so cheater is invulnerable Packet replay – repeat event over for added advantage – Example: multiple bullets or rockets if otherwise limited
Information Exposure Allows cheater to gain access to replicated, hidden game data (i.e. status of other players) – Passive, since does not alter traffic – Example: defeat “fog of war” in RTS, see through walls in FPS Look ahead cheat : Players makes decision after receiving all updates from participating players. Cannot be defeated by network alone
Design Defects Distribution may be the source of unexpected behavior – Features only evident upon high load (say, latency compensation technique) Age of Empires example : – When both a villager and a farm selected, issue the Stop command. Because valid for a villager, it was allowed to go through, but listed both objects as target of command. – The villager would stop working & reset – The farm would also reset, something never normally done, & replenish its food supply. Half-Life example. – firefight with another player, both using the same weapon – opponent was able to reload much more quickly
Cheating Solutions Install a mechanism in the game that verifies that each player is using the same program and data files. Changing from a game engine that issues commands to one that issues command requests Each player's machine creates a status summary of the entire game simulation on that computer. The status is in the form of a series of flags, CRCs, and checksums Look for hacking side-effects: Can that player see the object he just clicked on?" Synchronization strategies
Cheating Solutions Lockstep Protocol : No host receives the state of another host before the game rules permit 1.Player decides but does not announce its turn t + 1 2.Each player announces a Cryptographically secure one-way hash of its decision as a commitment. 3.After all players have announced their commitments, players reveal their decisions. 4.Each host can verify revealed decisions by comparing hashes.
Cheating Solutions Asynchronous Synchronization : Asynchronous Synchronization : Relaxes requirements of lockstep synchronization by decentralizing game clock 1.Player determines its decision for the turn and announces the commitment of the decision to all players. 2.Commitments that are one frame past the last revealed frame of a remote player are accepted. 3.Before revealing its commitment, the local player must determine which remote players it is waiting for. intersection with the SOI dilated from the last revealed frame of the remote player. The SOI is calculated using the base radius of the last known position plus a delta radius. 4.If no remote hosts are in the wait state: 1. the local host reveals its state turn for turn t, 2.updates its local entity model of each other player with their last known state 3.advances to the next turn.
Cheating Solutions AS with Packet Loss : players can skip missing packets and accept new, out-of-order packets from other players when the missing packets represent state outside a SOI intersection. Missing packets that represent intersection of SOI cannot be dropped or skipped. AS represents a performance advantage over lockstep, rather than contact every player every turn, players need only contact players that have SOI intersection. Downside of all previous protocols: Performance Penalty: All nodes must slow down to the speed of the slowest user.
Conclusion Overview of problems with MOGs – Networking resources – Distribution architectures – Compensation techniques – Security
Robustness -Users can join or leave the network at any time, without having any negative effects on other nodes. - A failure of any participant has no affect on any other participant. - Participants joining an ongoing session have missed the data that has previously been exchanged by the other session member. What to do? Late Join Algorithms -
Late Join Algorithms Necessary algorithms to distribute the current state of the session to new users. Two Approaches: Transport protocol. Application based Transport Protocol: Request ALL previous session information (rollback) Pros: Robust Application Independent Cons: Inefficient The state of some applications can’t be reconstructed. (Networked action games)
Late Join (cont.) Application based: The late join algorithm varies by the type of application. (e.g.- networked games vs. whiteboard) Efficient – Only need the current state of the session Lack of reusability Setup for Late Join: 1.New node must determine the priority of the subcomponents of the state (e.g. – You want to transfer the most recent page for a whiteboard) 2.New node (client) needs to select one or more of the existing nodes as a server. 3.Information must be transmitted to the new node.
Late Join (cont.) Late join policy differs based on the application Different proposed policies: 1.No late join 2.Immediate late join 3.Event-triggered late join 4.Network-capacity-oriented
Late Join (cont.) Distribution of Data: 1.One network group (base group) – Broadcast the state to the whole group. Unnecessary packets get sent to existing nodes. (Beneficial if the ratio of late joins to the existing users is very high) 2.Two network groups – All late join clients join the client group. 3.Three network groups – In addition to the two network groups, the late join servers form an additional multicast group. Problems: Who should be selected to act as a server??
Availability -Like robustness, no single point of failure will affect the entire network. - This is one of the major advantages over centralized networks, where the failure of the server causes the entire network to fail. - If a node fails, it gets disconnected from the network, but game/session continues with remaining nodes. - After N packets of a failed node are not received, other nodes determine that this node got disconnected, and stops using dead reckoning on its messages. -