SQL Server 2000 Notification Services Bart De Smet [MVP]

Agenda What is SQL Server Notification Services? Notification applications defined Introducing Notification Services Notification Services Explained Instances versus applications Development cycle Architectural overview Troubleshooting and tips Demos, demos, demos

Let’s show you first

What are SQL Server Notification Services? Notification applications defined

Notification Applications Definition Web News ERPCRM Notification Application My Device of choice Personalized and timely information sent to any device = what people want

Notification applications Timely delivery of information Stock feeds Stock feeds News services News services Custom applications Custom applications LOB applications LOB applications Etc. Etc. ActivityGenerationRouting End Points Information flow A solution may use some or all of these parts SQL Server Notification Services SQL Server Notification Services.NET Alerts.NET Alerts LCS LCS SMS Aggregators SMS Aggregators Exchange (SMTP) Exchange (SMTP) Outlook Outlook Windows Messenger Windows Messenger Messenger for Mac Messenger for Mac Pocket PC Pocket PC Smart Phone Smart Phone SMS capable phone SMS capable phone

Notification Applications A high-level view Source System s Subscriber Devices and Systems Subscriptions (millions) Notifications (millions) Match Subscribers Events

What are SQL Server Notification Services? Introducing Notification Services

SQL Server 2000 NS Platform A platform for developing and deploying rich and highly scalable notification applications A declarative programming model Based on XML and T-SQL An efficient and scalable execution engine Based on SQL Server and.NET Framework Developer writes Notification Application Notification Services hosts and executes it

Notification Services releases NS 2.0 SP1: Web download for SQL Free addition to SQL 2000 licensees Standard and Enterprise Editions available Future: SQL Server 2005 Integrated component (cf. installation) Management tools support Analysis Services event provider Programming model enhancements Smooth upgrade from NS 2.0

Developer benefits Build and deploy applications quickly Reduce time-to-market Build your application within a rich Notification Services framework Focus on application, not basic services Performance and Richness “Internet scale” matching: millions of events against millions of subscriptions Queue management, time zones, multi- language, scheduled subscriptions, fall behind, retry logic, multi-device…

Administrator benefits Notification Services applications are easy to deploy and manage Predictable, reliable tools Deployment can be automated Use standard mechanisms Windows performance counters, Event log, SQL reporting stored procedures Proven, tested reliability Rigorous testing at Microsoft Patterns and practices used over and over in real applications

Notification Services Explained Instances versus applications

Instances and applications Definition Instance A single named configuration of Notification Services Shared repository for subscriber data (not subscriptions!) Application An individual functional entity for collecting events, matching them with subscriptions, and delivering notifications Part of an instance

Instances and applications Definition One instance can host multiple applications Applications in an instance share subscribers Applications have their own events, subscriptions, and notifications Subscribers Application 1 ESN Application 2 ESN

Instances and applications Scalability Multiple instances can exist on the same machine Separate security and trust boundaries Each instance can be a different version of Notification Services Each instance has its own Windows Service An instance can span multiple machines Scale out Each machine runs the Windows Service Various roles can be spread across machines

Instances and Applications Declaration & configuration “Instance Configuration File” (ICF) Defines an instance SQL Server used by the instance Applications Protocols and Delivery Channels available “Application Definition File” (ADF) Defines the schema, logic, and topology of a single application Both are XML documents

The ICF and ADF

Notification Services Explained Development cycle

XMLFiles SQL-NS Compiler SQL Server Databases WindowsService

Nscontrol.exe SQL Server Notification Services Prompt Used to create, register, enable instances and applications No “Visual Studio.NET project” (!)

Development process Design Prototype Connect Deploy Define events, subscriptions, notifications Design matching logic Implement application core Test with sample data Add event providers and delivery protocols Build subscription management interface Move application to production hardware Implement monitoring and maintenance procedures

Compilation and deployment

Notification Services Explained Architectural overview

The big picture Subscription Management Application Data Change s ExternalDeliveryExternalDelivery Event Provider Generator Distributor SQL Server

Subscription Management Subscription Management Application Data Change s ExternalDeliveryExternalDelivery Event Provider Events Subscribers Subscriber Devices Subscriptions Notifications GeneratorDistributor

Subscription Management App Allows subscribers to manage subscriptions API in SQL-NS Managed: ASP.NET, VB.NET, C#, etc Unmanaged: ASP, VB6, C++, etc

Managing subscriptions

Event Providers Subscription Management Application Data Change s ExternalDeliveryExternalDelivery Event Provider Events Subscribers Subscriber Devices Subscriptions Notifications GeneratorDistributor File System Watcher SQL Server provider Custom EP Event Provider Host

Event Providers Event Providers collect data changes and submits them as Events in Event Batches Notification Applications have one or more Event Providers Notification Services ships with two standard providers File System Watcher Event Provider (testing) SQL Server Event Provider

Built-in Event Providers Events XML File System Watcher Provider Event Provider Host Events Data Source SQL Server provider Event Provider Host Query PostQuery Schedule Changes XML event file

Custom Event Providers Hosted Event Provider Run in-proc with Notification Services Windows Service Less custom code by leveraging Initialize, Run, Terminate, Schedule Starts and Stops with Notification Services Windows Service Independent Event Provider Can run on alternate platform or remote system Starts and Stops independently from Notification Services Windows Service Three Event Submission Methods Event Object XMLLoader Object Event Submission SQL Server Stored Procedures

Adding event providers

Generator Subscription Management Application Data Change s ExternalDeliveryExternalDelivery Event Provider Events Subscribers Subscriber Devices Subscriptions Notifications GeneratorDistributor SQL Match Rule Executes Stored Procs File System Watcher SQL Server provider Custom EP Event Provider Host

Generator Matches Events and Subscriptions to generate Notifications Write “match rules” with full power of SQL (Transact-SQL, sp, transactions…) Subscribers Subscriber Devices Subscriptions Events Generator Notifications

Rules Classification  Match subscriptions with events  Multiple per Subscription Class  Time Zones  Recurrence  Example: Send MSFT price at 1PM Mon-Fri  Match subscriptions with events  Multiple per Subscription Class  Executed when the generator fires  Fired before scheduled rules  Example: Notify when MSFT crosses $70 Event chronicle rules Subscription event rules Subscription scheduled rules  Maintain Event state  One per event class  Fire for each incoming event  Always fired first  Example: Latest stock prices

Writing the rules

Distributor Subscription Management Application Data Change s ExternalDeliveryExternalDelivery Event Provider Events Subscribers Subscriber Devices Subscriptions Notifications GeneratorDistributor SQL Match Rule Executes Stored Procs File System Watcher SQL Server provider Custom EP Event Provider Host XSLT CF Custom CF XSLT Custom DP .NET Alerts * File HttpExt

Distributor Periodically scans for notification batches Formats raw notification data to message body Assembles formatted notifications into protocol packets Delivers the protocol packet Handles delivery failures

Content Formatting Format raw SQL Notification data Language-CountryRegion and DeviceType Language-CountryRegion and DeviceType Specified per NotificationClass Stock notifications are formatted differently from Flight notifications Built-in XSLT Content Formatter Applies a XSLT to Notifications Custom Content Formatter Written as Managed Class

Delivery Richness  Combine notifications sent to the same user  Send me a daily summary of my portfolio  Send me a summary of all the documents that have been modified since the last time I connected  Choice of content formatting (XSLT or custom content formatter)  Device specific (mail, SMS, IM, etc)  Language specific (localized notifications) Rich Formatting Digestnotifications  Same message to many people  Format once, deliver to many  Weather report for the Brussels area  Traffic on E40 ‘Multicast’notifications

Get it delivered

The picture that tells it all... Subscription Management Application Data Change s ExternalDeliveryExternalDelivery Event Provider Events Subscribers Subscriber Devices Subscriptions Notifications GeneratorDistributor SQL Match Rule Executes Stored Procs File System Watcher SQL Server provider Custom EP Event Provider Host XSLT CF Custom CF XSLT Custom DP .NET Alerts * File HttpExt

Troubleshooting and tips About debugging, manageability, performance, scalability and deployment

Detecting Problems Event log Shows error and warning messages Check and doublecheck permissions Stored Procedure Richness Views installed at runtime Manual submission of events Management op subscriptions Performance Counters Provide throughput data for various parts of the NS pipeline

Building For Performance NS generates notifications faster than any protocol can deliver! NS scales with the hardware platform SQL Server  Data, logs, tempdb on dedicated disks  Pre-allocate database to prevent auto-grow  Disk subsystem vs. # CPU Subscribers, Subscriptions  Single / Multi-Threaded  Bulk load using multiple clients Generator  Beware of rule complexity; Install indexes on tables  ThreadPoolSize and parallel rule firing  Optimize here! Distributors  Use multiple machines  Notification batch size

Deployment Scenarios SQL NS Single server NSSQL Separate DB SQL NS Highly scalable distribution SQL SQL NS Highly available

SummarySummary Useful resources, books, summary, Q&A

Resources Information and download TechNet nol/sql/2000/evaluate/sqlnsto.mspx nol/sql/2000/evaluate/sqlnsto.mspx nol/sql/2000/evaluate/sqlnsto.mspx Books online oductdoc/default.asp oductdoc/default.asp oductdoc/default.asp.NET Alerts (30-days free trial) rd/default.aspx rd/default.aspx rd/default.aspx

Notification Services Books Microsoft SQL Server 2000 Notification Services (Shyam Pather) The Rational Guide to SQL Server Notification Services (Joe Webb)

Summary Notification Services is a platform Enables rapid development of robust scalable Notification Applications Easy to use Declarative Programming Model Free addition for SQL Server 2000 licensees Get prepared for next releases

Q&A

Developer & IT Pro Days February 1 and 2 at Heysel, Brussels Opening Keynote by Bill Gates +30 Pure Technical Sessions Separate Tracks For: Developers IT Professionals Special Community Happening Early Bird (till 26 December): EUR 249 Full Price: EUR 299

Common Scenario 1 “Manual polling” Web News ERPCRM BrowserInbox... Request Response No service, so do it yourself… Or … waste valuable time No service, so do it yourself… Or … waste valuable time

Common Scenario 2 Subscribe and... SPAM Web News ERPCRM My Inbox SPAM Push Non-personalized information sent by mail = bad service

Application Model Events & Subscriptions Model events as rows in a table Model subscriptions as rows in a table The most efficient way to do the match is to use SQL JOIN technology