Presentation is loading. Please wait.

Presentation is loading. Please wait.

Overview of the OMG Data Distribution Service Douglas C. Schmidt & Jeff Parsons

Similar presentations


Presentation on theme: "Overview of the OMG Data Distribution Service Douglas C. Schmidt & Jeff Parsons"— Presentation transcript:

1 Overview of the OMG Data Distribution Service Douglas C. Schmidt & Jeff Parsons Professor of EECS Vanderbilt University Nashville, Tennessee

2 Tutorial on DDS Douglas C. Schmidt 2 Past R&D Successes: Platform-centric Systems Legacy DoD systems are designed to be: Stovepiped Proprietary Brittle & non-adaptive Expensive to develop & evolve Vulnerable From this design paradigm… Air Frame AP Nav WTS GPS IFF FLIR Cyclic Exec Problem: Small changes can break nearly anything & everything

3 Tutorial on DDS Douglas C. Schmidt 3 …and this operation paradigm… Real-time QoS requirements for platform-centric DoD systems: Ensure end-to-end QoS, e.g., Minimize latency, jitter, & footprint Bound priority inversions Allocate & manage resources statically Utility Resources Utility Curve Broken Works Harder Requirements Problem: Lack of any resource can break nearly anything & everything Past R&D Successes: Platform-centric Systems

4 Tutorial on DDS Douglas C. Schmidt 4 …to this design paradigm… Past R&D Successes: Network-centric Systems Event Channel Replication Service GPSIFFFLIR Object Request Broker Air Frame APNavWTS Todays leading-edge DoD systems are designed to be: Layered & componentized More standard & COTS Robust to expected failures & adaptive for non-critical tasks Expensive to evolve & retarget Vulnerable Problem: Unanticipated changes can break many things

5 Tutorial on DDS Douglas C. Schmidt 5 …and this operational paradigm… Past R&D Successes: Network-centric Systems Endsystem Applications MiddlewareMiddleware Applications Mechanism & Property Managers Sys Cond Interceptor Local Resource Managers Sys Cond { } QoS Contracts Network latency & bandwidth Workload & Replicas CPU & memory Connections & priority bands Network latency & bandwidth Workload & Replicas CPU & memory Connections & priority bands Local Resource Managers Adaptive Real-time Middleware

6 Tutorial on DDS Douglas C. Schmidt 6 …and this operational paradigm… Past R&D Successes: Network-centric Systems Problem: Network-centricity is an afterthought in todays systems Resources Utility Desired Utility Curve Working Range Softer Requirements

7 Tutorial on DDS Douglas C. Schmidt 7 New Demands on Enterprise DRE Systems Key challenges in the solution space Enormous accidental & inherent complexities Continuous evolution & change Highly heterogeneous platform, language, & tool environments Key challenges in the problem space Network-centric, dynamic, very large- scale systems of systems Stringent simultaneous quality of service (QoS) demands Highly diverse & complex problem domains Mapping & integrating problem artifacts to solution artifacts is very hard

8 Tutorial on DDS Douglas C. Schmidt 8 Coordination Of Multiple UAVs Dynamic Mission Replanning Feedback & Control Image Processing & Tracking Case Study: QoS-enabled Publish/Subscribe Technologies for Tactical Information Management DARPA PCES Capstone demo, 4/14/05, White Sands Missile Range

9 Tutorial on DDS Douglas C. Schmidt 9 Coordination Of Multiple UAVs Dynamic Mission Replanning Feedback & Control Image Processing & Tracking Case Study: QoS-enabled Publish/Subscribe Technologies for Tactical Information Management Synchronization Memory Management Physical Memory Access Asynchronous Event Handling Scheduling Asynchronous Transfer of Control Modeling ToolsModel CheckingReal-time JVMsReal-time ORBs Aspect Languages

10 Tutorial on DDS Douglas C. Schmidt 10 Limitations with Demo d PCES Information Management Technologies Real-time Event Service Tactical Network & RTOS Object Request Broker Real-time Info to Cockpit Shooter information management was based on platform-centric Real-time CORBA & Real-time CORBA Event Service Well-suited for point-to-point & static pub/sub command processing over wireline networks e.g., statically provisioned QoS policies Poorly suited for dynamic pub/sub info dissemination to multiple nodes in MANETs e.g., too many layers, excessive time/space overhead, inflexible QoS policies & pub/sub model, etc. Problem: Net-centricity is afterthought in platform-centric technologies

11 Tutorial on DDS Douglas C. Schmidt 11 Limitations with Demo d PCES Information Management Technologies C2 & C4ISR information management based on J2EE & Java Messaging Service (JMS) Best suited for operational enterprise environments e.g., large data centers connect via wireline networks Poorly suited for tactical environments e.g., lack of QoS policies & RTOS integration, extremely high time/space overhead Java Messaging Service Enterprise Network & OS J2EE Middleware Track Processing Problem: Enterprise technologies are ill suited for tactical systems

12 Tutorial on DDS Douglas C. Schmidt 12 Our R&D Goal: Evaluate QoS-enabled Info Brokering for Tactical Systems Solutions must function properly where Communication bandwidth is limited/variable Connectivity is intermittent Connections are noisy Processing & storage capacity are limited Power & weight limits affect usage patterns Unanticipated workflows are common Dynamic network topology & membership changes are frequent Ideally, solutions should be COTS, standard, & integrate with heterogeneous legacy systems Goal is better than best-effort, subject to resource constraints…

13 Tutorial on DDS Douglas C. Schmidt 13 Data Reader R Data Writer R PublisherSubscriber Topic R Overview of the Data Distribution Service (DDS) Tactical Network & RTOS DDS Pub/Sub Infrastructure RT Info to Cockpit & Track Processing DDS is an highly efficient OMG pub/sub standard e.g., fewer layers, less overhead

14 Tutorial on DDS Douglas C. Schmidt 14 Data Reader R Data Writer R PublisherSubscriber Topic R NEW TOPIC NEW SUBSCRIBER Overview of the Data Distribution Service (DDS) DDS is an highly efficient OMG pub/sub standard e.g., fewer layers, less overhead DDS provides meta-events for detecting dynamic changes NEW PUBLISHER

15 Tutorial on DDS Douglas C. Schmidt 15 DDS is an highly efficient OMG pub/sub standard e.g., fewer layers, less overhead DDS provides meta-events for detecting dynamic changes DDS provides policies for specifying many QoS requirements of tactical information management systems, e.g., Establish contracts that precisely specify a wide variety of QoS policies at multiple system layers Data Reader R Data Writer R PublisherSubscriber S1 S2 S3 S4 S5 S6 S7 S6S5S4S3S2S1 Topic R S7 X HISTORY RELIABILITY COHERENCY RESOURCE LIMITS LATENCY Overview of the Data Distribution Service (DDS)

16 Tutorial on DDS Douglas C. Schmidt 16 DDS is an highly efficient OMG pub/sub standard e.g., fewer layers, less overhead DDS provides meta-events for detecting dynamic changes DDS provides policies for specifying many QoS requirements of tactical information management systems, e.g., Establish contracts that precisely specify a wide variety of QoS policies at multiple system layers Move processing closer to data Data Reader R Data Writer R PublisherSubscriber S1 S2 S3 S4 S5 S6 S7 Topic R SOURCE FILTER DESTINATION FILTER Overview of the Data Distribution Service (DDS) TIME-BASED FILTER

17 Tutorial on DDS Douglas C. Schmidt 17 Promising Approach: The OMG Data Distribution Service (DDS) Application Global Data Store read write DDS provides flexibility, power, & modular structure by decoupling: Location – anonymous pub/sub Redundancy – any number of readers & writers QoS – async, disconnected, time-sensitive, scalable, & reliable data distribution at multiple layers Platform & protocols – portable & interoperable Network latency & bandwidth Workload & Replicas CPU & memory Connections & priority bands

18 Tutorial on DDS Douglas C. Schmidt 18 DDS Architectural Elements Data-Centric Publish-Subscribe (DCPS) –The lower layer APIs apps can use to exchange topic data with other DDS- enabled apps according to designated QoS policies Data Local Reconstruction Layer (DLRL) –The upper layer APIs that define how to build a local object cache so apps can access topic data as if it were local DDS spec only defines policies & interfaces between application & service Doesnt address protocols & techniques for different actors implementing the service Doesnt address management of internal DDS resources

19 Tutorial on DDS Douglas C. Schmidt 19 DDS Application Architecture DCPS Application DLRL Application DLRL Application DLRL Application DLRL Communication The Application {

20 Tutorial on DDS Douglas C. Schmidt 20 DDS Domains & Domain Participants DomainParticipant Node Domain 1 Domain 2 Domain 3 Node The domain is the basic construct used to bind individual applications together for communication Like a VPN

21 Tutorial on DDS Douglas C. Schmidt 21 DCPS Entities DCPS Entities include –Topics Typed data –Publishers Contain DataWriters –Subscribers Contain DataReaders –DomainParticipants Entry points Data can be accessed in two ways –Wait-based (synchronous calls) –Listener-based (asynchronous callbacks) Sophisticated support for filtering –e.g., Topic, Content-FilteredTopic, or MultiTopic Configurable via (many) QoS policies Topic Data Reader Data Writer Data Reader Data Writer Subscriber Publisher Subscriber Data Domain Domain Participant

22 Tutorial on DDS Douglas C. Schmidt 22 QoS Policies Supported by DDS DCPS entities (i.e., topics, data readers/writers) configurable via QoS policies QoS tailored to data distribution in tactical information systems Request/offered compatibility checked by DDS –DEADLINE Establishes contract regarding rate at which periodic data is refreshed –LATENCY_BUDGET Establishes guidelines for acceptable end-to-end delays –TIME_BASED_FILTER Mediates exchanges between slow consumers & fast producers –RESOURCE_LIMITS Controls resources utilized by service –RELIABILITY (BEST_EFFORT, RELIABLE) Enables use of real-time transports for data –HISTORY (KEEP_LAST, KEEP_ALL) Controls which (of multiple) data values are delivered –DURABILITY (VOLATILE, TRANSIENT, PERSISTENT) Determines if data outlives time when they are written –… and many more …

23 Tutorial on DDS Douglas C. Schmidt 23 Best Effort Reliability QoS in DDS Publisher Subscriber QoS Reliability = BEST_EFFORT Notification of new data objects no notificationnotificationtime time-based filter deadline timeout Very predictable Data is sent without reply Publishers and subscribers match and obey QoS Deadline policy settings Time-based Filter QoS policy gives bandwidth control

24 Tutorial on DDS Douglas C. Schmidt 24 Reliable QoS in DDS QoS Reliability = RELIABLE Topic R Data Reader R Data Writer R PublisherSubscriber history S1 S2 S3 S4 S5 S6 S7 S6S5S4S3S2S1 Ordered instance delivery is guaranteed

25 Tutorial on DDS Douglas C. Schmidt 25 Tradeoff Between Reliability & Determinism X X X QoS Reliability = RELIABLE QoS Reliability = BEST_EFFORT Cant have reliability and determinism. Best effort mode for streaming data. No retries of dropped packets. Minimizes latency. Reliable mode for commands & events. Retry dropped packets until timeout. Every packet received in order. Speculative cache mechanism. DDS reliability is tunable. Can be adjusted per subscription.

26 Tutorial on DDS Douglas C. Schmidt 26 All QoS Policies in DDS Deadline Destination Order Durability Entity Factory Group Data History Latency Budget Lifespan Liveliness Ownership Ownership Strength Partition Presentation Reader Data Lifecycle Reliability Resource Limits Time-Based Filter Topic Data Transport Priority User Data Writer Data Lifecycle

27 Tutorial on DDS Douglas C. Schmidt 27 Single vs. Multiple Domain Systems

28 Tutorial on DDS Douglas C. Schmidt 28 Data Writers & Publishers Data Writers are the primary access point for an application to publish data into a DDS data domain The Publisher entity is a container to group together individual Data Writers User applications –Associate Data Writers with Topics –Provide data to Data Writers –Data is typically defined using OMG IDL It can therefore be as strongly or weakly typed as you desire

29 Tutorial on DDS Douglas C. Schmidt 29 Data Readers & Subscribers A Data Reader is the primary access point for an application to access data that has been received by a Subscriber Subscriber is used to group together Data Readers Subscribers & Data Readers can have their own QoS policies User applications –Associate Data Readers with Topics –Receive data from Data Readers using Listeners (async) or Wait-Sets (sync)

30 Tutorial on DDS Douglas C. Schmidt 30 Types & Keys A DDS Type is represented by a collection of data items. –e.g. IDL struct in the CORBA PSM struct AnalogSensor { string sensor_id; // key float value; // other sensor data }; A subset of the collection is designated as the Key. –The Key can be indicated by IDL annotation in CORBA PSM, e.g., #pragma DDS_KEY AnalogSensor::sensor_id Its manipulated by means of automatically-generated typed interfaces. –IDL compiler may be used in CORBA PSM implementation A Type is associated with generated code: –AnalogSensorDataWriter // write values –AnalogSensorDataReader // read values –AnalogSensorType // can register itself with Domain

31 Tutorial on DDS Douglas C. Schmidt 31 Topics A DDS Topic is the connection between publishers & subscribers. A Topic is comprised of a Name and a Type. –Name must be unique in the Domain. –Many Topics can have the same Type. Provision is made for content- based subscriptions. –MultiTopics correspond to SQL join –Content-Filtered Topics correspond to SQL select. Domain ID35 Topic nameMySensor Type stringsensor_id [ key ] float value AnalogSensorname

32 Tutorial on DDS Douglas C. Schmidt 32 Topic-Based Publish/Subscribe DataWriter is bound (at creation time) to a single Topic DataReader is bound (at creation time) with one or more topics (Topic, ContentFilteredTopic, or MultiTopic) –ContentFilteredTopic & MultiTopic provide means for content-based subscriptions & joins, respectively Pressure Temperature

33 Tutorial on DDS Douglas C. Schmidt 33 Subscription = Topic + DataReader Topic 1 application Topic 2Topic n Data Reader Data Reader Data Reader subscriber QoS

34 Tutorial on DDS Douglas C. Schmidt 34 Content-based Subscriptions ContentFilteredTopic (like a DB-selection) –Enables subscriber to only receive data-updates whose value verifies a condition. –e.g. subscribe to Pressure of type AnalogData –where value > 200 MultiTopic (like a DB-join operation) –Enables subscription to multiple topics & re-arrangement of the data- format –e.g. combine subscription to Pressure & Temperature & re- arrange the data into a new type: struct { float pres; float temp; };

35 Tutorial on DDS Douglas C. Schmidt 35 DDS Content-Filtered Topics Content-Filtered Topic Topic Filter Expression: Value > 260 Expression Params Topic Instances in Domain Instance 1 Instance 2 Instance 3 Instance 4 Instance 5 Instance 6 Instance 7 Value = 249 Value = 230 Value = 275 Value = 262 Value = 258 Value = 261 Value = 259 Filter Expression and Expression Params determine which Topic instances the Subscriber receives.

36 Tutorial on DDS Douglas C. Schmidt 36 DDS MultiTopic Subscriptions Topic MultiTopic Data Reader Subscriber MultiTopics can combine, filter, and rearrange data from multiple Topics. Domain Participant

37 Tutorial on DDS Douglas C. Schmidt 37 Example: Create Domain Participant // used to identify the participant DomainId_t domain_id =...; // get the singleton factory instance DomainParticipantFactory_var dpf = DomainParticipantFactory::get_instance (); // create domain participant from factory DomainParticipant_var dp = dpf->create_participant (domain_id, PARTICIPANT_QOS_DEFAULT, NULL); DomainParticipant object acts as factory for Publisher, Subscriber, Topic and MultiTopic entity objects

38 Tutorial on DDS Douglas C. Schmidt 38 Example: Create Topic …… // register the data type associated with the topic FooDataType foo_dt; foo_dt.register_type (dp, Foo); // create a topic Topic_var foo_topic = dp->create_topic (Foo_topic, //topic name Foo, // type name TOPIC_QOS_DEFAULT, // Qos policy NULL); // topic listener

39 Tutorial on DDS Douglas C. Schmidt 39 Example: Create Subscriber and DataReader …… // create a subscriber from the domain participant SubscriberQos sQos; dp->get_default_subscriber_qos (sQos); Subscriber_var s = dp->create_subscriber (sQos, NULL); // create a data reader from the subscriber // and associate it with the created topic DataReader_var reader = s->create_datareader (foo_topic.in (), DATAREADER_QOS_DEFAULT, NULL); FooDataReader_var foo_reader = FooDataReader::_narrow (reader.in ());

40 Tutorial on DDS Douglas C. Schmidt 40 Example: Create Publisher and DataWriter …… // create a publisher from the domain participant PublisherQos pQos; dp->get_default_publisher_qos (pQos); Publisher_var p = dp->create_publisher (pQos, NULL); // create a data writer from the publisher // and associate it with the created topic DataWriter_var writer = p->create_datawriter (foo_topic.in (), DATAWRITER_QOS_DEFAULT, NULL); // narrow down to specific data writer FooDataWriter_var foo_writer = FooDataWriter::_narrow (writer); // publish user-defined data Foo foo_data; foo_writer->write (foo_data);

41 Tutorial on DDS Douglas C. Schmidt 41 How to Get Data (Async Listener-based) Listener_var subscriber_listener = new MyListener(); foo_reader->set_listener(subscriber_listener); MyListener::on_data_available(DataReader reader) { FooSeq_var received_data; SampleInfoSeq_var sample_info; reader->take(received_data.out (), sample_info.out (), ANY_SAMPLE_STATE, ANY_LIFECYCLE_STATE); // Use received_data …… }

42 Tutorial on DDS Douglas C. Schmidt 42 How to Get Data (Sync Wait-based) Condition_var foo_condition = reader->create_readcondition(ANY_SAMPLE_STATE, ANY_LIFECYCLE_STATE); WaitSet waitset; waitset->attach_condition(foo_condition); ConditionSeq_var active_conditions; Duration_t timeout = {3,0}; waitset->wait(active_conditions.out (), timeout);... FooSeq_var received_data; SampleInfoSeq_var sample_info; reader->take_w_condition(received_data.out (), sample_info.out (), foo_condition); // Use received_data

43 Tutorial on DDS Douglas C. Schmidt 43 Benchmark Scenario Two processes perform IPC in which a client initiates a request to transmit a number of bytes to the server along with a seq_num (pubmessage), & the server simply replies with the same seq_num (ackmessage). –The invocation is essentially a two-way call, i.e., the client/server waits for the request to be completed. The client & server are collocated. DDS & JMS provides topic-based pub/sub model. Notification Service uses push model. SOAP uses p2p schema-based model.

44 Tutorial on DDS Douglas C. Schmidt 44 Testbed Configuration Hostname blade14.isislab.vanderbilt.edu OS version (uname -a) Linux version _FC4smp GCC Version g++ (GCC) (Red Hat Linux fc4) CPU info Intel(R) Xeon(TM) CPU 2.80GHz w/ 1GB ram

45 Tutorial on DDS Douglas C. Schmidt 45 Test Parameters Average round-trip latency & dispersion Message types: –sequence of bytes –sequence of complex type Lengths in powers of 2 Ack message of 4 bytes 100 primer iterations 10,000 stats iterations // Complex Sequence Type struct Inner { string info; long index; }; typedef sequence InnerSeq; struct Outer { long length; InnerSeq nested_member; }; typedef sequence ComplexSeq;

46 Tutorial on DDS Douglas C. Schmidt 46 Roundtrip Latency Results (1/2)

47 Tutorial on DDS Douglas C. Schmidt 47 Roundtrip Latency Results (2/2)

48 Tutorial on DDS Douglas C. Schmidt 48 Roundtrip Latency Results Analysis From the results we can see that DDS has significantly better performance than other SOA & pub/sub services. Although there is a wide variation in the performance of the DDS implementations, they are all at least twice as fast as other pub/sub services.

49 Tutorial on DDS Douglas C. Schmidt 49 Encoding/Decoding Benchmarks Measured overhead and dispersion of –encoding C++ data types for transmission –decoding C++ data types from transmission DDS3 and GSOAP implementations compared Same data types, platform, compiler and test parameters as for roundtrip latency benchmarks

50 Tutorial on DDS Douglas C. Schmidt 50 Encoding/Decoding Results (1/2)

51 Tutorial on DDS Douglas C. Schmidt 51 Encoding/Decoding Results (2/2)

52 Tutorial on DDS Douglas C. Schmidt 52 Encoding/Decoding Results Analysis Slowest DDS implementation is compared with GSOAP. DDS is faster. –Almost always by a factor of 10 or more. –GSOAP is encoding XML strings. Difference is larger for byte sequences. –DDS implementation has optimization for byte seq. Encodes sequence as a single block – no iteration. –GSOAP always iterates to encode sequences. Jitter discontinuities occur at consistent payload sizes.

53 Tutorial on DDS Douglas C. Schmidt 53 subscribers publishers Summary of DCPS features Efficient Publish/Subscribe interfaces QoS suitable for real-time systems –deadlines, levels of reliability, latency, resource usage, time-based filter Listener & wait-based data access suits different application styles Support for content-based subscriptions Support for data-ownership Support for history & persistence of data-modifications DDS Information consumer subscribe to information of interest Information producer publish information DDS matches & routes relevant info to interested subscribers

54 Tutorial on DDS Douglas C. Schmidt 54 Data Local Reconstruction Layer (DLRL) Track 3D_Track Track_Topic3D -Track DLRL DCPS Cache

55 Tutorial on DDS Douglas C. Schmidt 55 Goals of DLRL Data Local Reconstruction Layer (DLRL) model is local to an application Object-oriented access to data Application developers can –describe classes with their methods, data fields, & relations –attach some of those data fields to DCPS entities –manipulate these objects (i.e., create, read, write, delete) using native language constructs –activate attached DCPS entities to update objects –have these objects managed in a cache Like a mapping or binding (intuition only)

56 Tutorial on DDS Douglas C. Schmidt 56 DLRL Objects DLRL objects are (native) language objects with: –data members and methods Only the data members are relevant to data distribution; they can be: –attributes, i.e., values –relations, i.e., reference other objects –plain local data members (i.e., not known or involved in data distribution) are also supported DLRL classes can be organised by inheritance DLRL objects maps to one or more DCPS Topics

57 Tutorial on DDS Douglas C. Schmidt 57 DLRL Object Examples

58 Tutorial on DDS Douglas C. Schmidt 58 Oid 2 z 300 T3D_TOPIC Oid 11 RADAR_TOPIC Oid 1 2 x y TRACK_TOPIC Class Track Track3D radar 11 COMMENTS_TOPIC Oid 1 1 comments a comment another comment Class Track index 0 1 R_Oid 11 RADAR_TRACKS_TOPIC T_Oid 1 2 T_Class Track Track3D index 0 1 DLRL Radar Example Track x : real y : real comments [*] : string w : integer Track3D z : real Radar x : real y : real comments [*] : string z : real tracksa_radar *0..1 w : integer

59 Tutorial on DDS Douglas C. Schmidt 59 Evaluating DDS Pros Low overhead & efficient use of transport bandwidth –e.g., less features/overhead than CORBA in the main request path Dynamically scalable & highly flexible –e.g., can receive notification about all sorts of meta-events, such as new topics, publishers, subscribers, etc. Supports one-to-one, one-to-many, many-to-one, & many-to-many communication Large number of configuration parameters & QoS policies that give developers extensive control of message transmission & reception Cons DDS is not well suited to request-reply services, file transfer, & transaction processing The data-distribution paradigm is not optimized for sending a request & waiting for a reply Implementations dont yet cover the entire spec Lack of interoperability between different vendors implementations

60 Tutorial on DDS Douglas C. Schmidt 60 Comparing CORBA with DDS Distributed object Client/server Remote method calls Reliable transport Best for Remote command processing File transfer Synchronous transactions Distributed data Publish/subscribe Multicast data Configurable QoS Best for Quick dissemination to many nodes Dynamic nets Flexible delivery requirements DDS & CORBA address different needs Complex systems often need both…


Download ppt "Overview of the OMG Data Distribution Service Douglas C. Schmidt & Jeff Parsons"

Similar presentations


Ads by Google