1 Fault-tolerant Paths ISRG Retreat Z. Morley Mao 1/11/2000 Services Paths

2 Example path application: Jukebox/cell-phone application Ninja Jukebox: – service providing real-time streaming audio data from a collection of CDs in the network GSM Cell-phone: –12kbps data, 13kbps voice –communicates with BTS Jukebox Path :operator:connector

3 What is a path? A way to compose services to create customizable complex services Goals: –composability –accessibility –availability, fault-tolerance –scalability –security

4 Overall path construction process –a continuous optimization process with feedback: Path Instantiation, Execution, Maintenance Logical Path Creation Physical Path Creation Path Tear-down

5 Logical path creation: Path matching algorithm Formulated as shortest path graph search –Operators ===> edges –Data format/type ===> nodes Dijkstra’s shortest path algorithm –O(v 2 ) Difficulty: expressing constraints and optimization variables

6 Path maintenance: Partial Path Repair (PPR) APC(Automatic Path Creation) service guarantees robustness and fault-tolerance Two ways of monitoring: –active checking of operator status –operators notify APC of neighboring operators’ failure

7 Performance measurements (4 operators, Jukebox/cell-phone app) Logical/Physical path creation time: 264ms Path instantiation time: 215ms –operator instantiation: 70ms –connector creation: 64ms –start operator running: 81 ms Path recovery: one operator fails –Time to detect failure of operator: 2ms –Time to repair one failed operator: 400ms Path tear-down time: 289ms

8 Open design issues Wide area considerations Improved path reliability model Path performance modeling Path resource management framework Flexible path control –control path, path migration, dynamic adaptation Applications for paths Metrics for evaluation

9 Wide area path design APC SAN WAN service Hierarchical APC Service

10 Step-by-step WAN path creation for Jukebox/Cell-phone application 1.End-user using cell-phone requests access to Jukebox service QoS needs: delay-sensitive, reliable service 2.APC uses graph search algorithm finds the logical path 3.APC searches for the physical path 1.Finds relevant parameters affecting QoS, determines the reliability model

11 Step-by-step WAN path creation for Jukebox/Cell-phone application 2.Obtains resource information from resource management framework 3.Uses queuing model to evaluate choices 4.APC selects the optimal choice 5.APC dynamically adjusts the decision given feedback from the resource monitoring tools

12 Operator placement decisions Depend on –operator computational requirement –software/hardware requirement –output/input properties data location, data volume, delay-sensitivity, degradation properties –network characteristics bandwidth, latency, jitter, packet loss

13 Path resource management framework develop network monitoring tools –to obtain network statistics Available resources – computational, memory, network etc. Make trade-offs due to interdependencies among resources resource allocated per path basis

14 Path resource management framework A high-level global hierarchical resource manager Local resource manager per SAN Runtime resource monitoring tools monitor/discover resource changes during the lifetime of paths

15 Applications for paths Operators: –content transcoding operators: text-to-speech, mp3-to-PCM, PCM-to-GSM web search tools, filtering, aggregation, personalization Microsoft COM objects, existing Web services... Document conversion services –protocol translation operators: serial socket, security transcoder, RMI Lite Any serviceAny device

16 Measurement metrics Path creation time –logical/physical path creation, instantiation, execution Scalability –number of paths created per amount of time Fault-recovery time Control, ease-of-use, programmability of paths Ease of transparent path migration, adaptation to resource changes

17 Conclusion Recent work: –APC prototype built for SAN with reasonable performance, Partial Path Repair Future work: –focus on WAN, scalable path design –WAN test plan: campus-wide millennium cluster –support for continuous path optimization and adaptation

18 For more information Please send comments/questions to Z. Morley Mao Slides will be available at: –

19 Extra Slides

20 Flexible path control: control path Control path –Definition: make changes of operators, connectors independent of data path highly-available, fault-tolerant Proposed design: –replicated control paths: neighboring operators have control over each other APC has complete control over localized operators

21 Flexible path control: path migration Two paths of different quality running –migrate from the fast-to-startup, lower-quality one to slow-to-startup, higher-quality one Transparent migration of paths –dynamic fusion of operators –dynamic deletion, addition, replacement of operators/connectors –Goal: adapt changes in resources and locations of end points

22 Wide area considerations Goal: scalable, network-partition-tolerant proposed design: –replicated APC service instances –state of paths partitioned and replicated –operators are soft-state –continuous monitoring of operators and connectors by APC service instances –a few localized path components hooked together over wide area