Presentation is loading. Please wait.

Presentation is loading. Please wait.

Financial Services Technology Expo Microsoft StreamInsight for Financial Services A Microsoft Point of View Presentation Hilton New York Hotel New York,

Similar presentations


Presentation on theme: "Financial Services Technology Expo Microsoft StreamInsight for Financial Services A Microsoft Point of View Presentation Hilton New York Hotel New York,"— Presentation transcript:

1 Financial Services Technology Expo Microsoft StreamInsight for Financial Services A Microsoft Point of View Presentation Hilton New York Hotel New York, NY June 22-24, 2010

2 2 Microsoft StreamInsight for Financial Services Torsten Grabs Lead Program Manager Microsoft StreamInsight

3 Microsoft StreamInsight Operational Intelligence Platform Sources Devices, Sensors Web servers Stock tickers & News feeds Data Bus Caching Processing Visualization Distribution Static Reports Mining, Validation, “What-If” Scenarios Operational Dashboard (Ticking - Snapshot) Reporting Dashboard (Refreshed) Message Bus Operational Analytics Automated Decisions In-memory Database Intra-Day Cubes Historic Cubes ETL Reference Data CacheCache Refresh (Push) Re-compute (Pull) Service Broker

4 Operational Analytics Queries designed over offline/historical data “Backtesting” of models Ability to work over vast amounts of offline data Queries deployed to online/live feeds Tens of thousands of events/messages per sec Continuous processing – standing queries Timely results – low latency Queries reason about time Windows over time Sequences of events

5 The Need for an Event-Driven Platform 5 Query results need to reflect important changes in business reality immediately and enable responses to them with minimal latency Database ApplicationsEvent-driven Applications Query Paradigm Ad-hoc queries or requests Continuous standing queries LatencySeconds, hours, daysMilliseconds or less Data RateHundreds of events/secTens of thousands of events/sec or more Query Semantics Declarative relational analytics Declarative relational and temporal analytics request response Event output stream input stream

6 Overview: Microsoft StreamInsight 6 StreamInsight Engine Output Adapters Input Adapters Event Standing Queries Event sources Event targets Devices, Sensors Web servers Event stores & Databases Stock tickers & News feeds Event C_IDC_NAMEC_ZIP Event stores & Databases Pagers & Monitoring devices KPI Dashboards, SharePoint UI Trading stations Event Application at Runtime Static reference data.NET C# LINQ Application Development

7 Event Types StreamInsight events in use the.NET type system Events are structured and can have multiple fields Fields are typed using the.NET framework types StreamInsight engine provisioned timestamp fields capture all the different temporal event characteristics Event sources populate time stamp fields All calculations done based on “business time” Timestamps /Metadata Long pumpID String Type String Location Double flow Double pressure ………………

8 Event Streams & Adapters A stream is a sequence of events Defined over a.NET type Possibly infinite Stream characteristics: Event/data arrival patterns (steady, bursty) Out of order events: Order of arrival of events does not match the order of their application timestamps Adapters Receive/get events from the data source Enqueue events for processing in the engine Insertions of new events Changes to event durations 8

9 StreamInsight Query Features Operators over streams Calculations (PROJECT) Correlation of streams from different data sources (JOIN) Check for absence of activity with a data source (EXISTS) Selection of events from streams (FILTER) Stream partitioning (GROUP & APPLY) Aggregation (SUM, COUNT, …) Ranking and heavy hitters (TOP-K) Temporal operations: hopping window, sliding window Extensibility – to add new domain-specific operators

10 LINQ Query Examples LINQ Example – GROUP&APPLY, WINDOW: from e3 in MyStream3 group e3 by e3.i into SubStream from win in SubStream.HoppingWindow( FiveMinutes,ThreeSeconds) select new { i = SubStream.Key, a = win.Avg(e => e.f) }; LINQ Example – GROUP&APPLY, WINDOW: from e3 in MyStream3 group e3 by e3.i into SubStream from win in SubStream.HoppingWindow( FiveMinutes,ThreeSeconds) select new { i = SubStream.Key, a = win.Avg(e => e.f) }; LINQ Example – JOIN, PROJECT, FILTER: from e1 in MyStream1 join e2 in MyStream2 on e1.ID equals e2.ID where e1.f2 == “foo” select new { e1.f1, e2.f4 }; LINQ Example – JOIN, PROJECT, FILTER: from e1 in MyStream1 join e2 in MyStream2 on e1.ID equals e2.ID where e1.f2 == “foo” select new { e1.f1, e2.f4 }; Join Filter Project Grouping Window Project & Aggregate

11 StreamInsight Licensing Guidance Data rate: Tens of thousands of events every second Latency: Sub-second Standard and Enterprise Datacenter Data rate: Thousands of events per second Latency: seconds StreamInsight is part of Microsoft SQL Server 2008 R2 Standard & Enterprise edition for embedding in many ISV applications Standard & Enterprise edition for deployments on the edge Data Center edition for server deployments Data Sources Aggregation & Correlation Devices Sensors Web servers Feeds Stream Insight Complex Analytics & Mining Actual mileage will vary

12 Recap: Microsoft StreamInsight 12 StreamInsight Engine Output Adapters Input Adapters Event Standing Queries Event sources Event targets Event C_IDC_NAMEC_ZIP Event Static reference data CEP Application Development Development experience with.NET, C#, LINQ and Visual Studio 2008 or 2010 CEP platform from Microsoft to build event- driven applications Event-driven applications are fundamentally different from traditional database applications: queries are continuous, consume and produce streams, and compute results incrementally Flexible adapter SDK with high performance to connect to different event sources and sinks The CEP platform does the heavy lifting for you to deal with temporal characteristics of event stream data

13 For More Information StreamInsight main page & download : complex-event.aspx complex-event.aspx complex-event.aspx StreamInsight blog: StreamInsight MSDN documentation: us/library/ee362541(SQL.105).aspx us/library/ee362541(SQL.105).aspx us/library/ee362541(SQL.105).aspx StreamInsight E-clinics on Microsoft e-learning https://www.microsoftelearning.com/eLearning https://www.microsoftelearning.com/eLearning

14 14 More Microsoft Sessions  Tuesday June 22 nd, 2:30 PM Madison Room: Cloud Computing In Financial Services  Wednesday June 23 rd, 8:00 AM Green Room – 4 th Floor Breakfast: Microsoft CRM for Financial Services  Wednesday June 23 rd, 2:15 PM Petit Trianon Room – Real Time Risk Management with Microsoft’s StreamInsight  Meet with our Cloud Computing, StreamInsight and HPC Product teams at the Green Room, 4 th Floor  Follow us on  Visit us at

15 More Microsoft Sessions  Let us know about any topic you would like us to follow up on!  Write the topic on the back of your business card  Drop it off in the bowl on your way out of the session  You will be automatically entered to win a 16GB Zune HD!* *Winner need not be present to win. Winners notified at the end of the SIFMA conference.


Download ppt "Financial Services Technology Expo Microsoft StreamInsight for Financial Services A Microsoft Point of View Presentation Hilton New York Hotel New York,"

Similar presentations


Ads by Google