1. But It Sounded So Simple! Building a Messaging System in Service Broker Matt Wigdahl, ScriptPro LLC

2. Introduction  Matt Wigdahl, Director of Application and Database Development at ScriptPro LLC  Over 20 years of development experience, over 15 years of SQL Server experience  Email: mlwigdahl@gmail.commlwigdahl@gmail.com  Twitter: @mlwigdahl  LinkedIn: https://www.linkedin.com/in/mlwigdahl https://www.linkedin.com/in/mlwigdahl

3. What the Heck is Service Broker?  Lightweight Enterprise Service Bus  Powerful  Mature – 10 years old (!)  Obscure

4. But What Does It Do?  Asynchronous messaging  Persistence  Ordering  Full transactional support  Useful for apps that require inter-database connectivity  Single- or multi-instance

5. Aren’t There Other Options? YYes! MMSMQ – Microsoft Message Queueing OOld (NT 4.0) UUses file-based queues and MSDTC SService Bus AAzure or Windows Server NNew FFull ESB architecture HHarder

6. So Why SSSB?  Happy medium LESB  Powerful but relatively simple  Plays to strengths of DB  Persistence  Transactional processing  Existing connectivity

7. But Mostly… AActivation! EEvent-driven processing IInternal (proc) or External (app)

8. What Would I Use It For?  Workload processing for distributed apps  Wigxpedia.com customers reserve hotel rooms, cars, and airline flights  Asynchronous triggers  Replacement for SQL Agent for Express  Messaging system for data and app events

9. Getting to Know Service Broker  Many types of components that work together  Message types  Empty  Free-form  “Well-formed” XML  Schema-bound XML

10. Getting to Know Service Broker Some More  Services  Logical endpoints of communication  Contracts and Conversations  Patterns of message exchange between two services  Reference message types and services  Queues  Physical repositories for messages  Specialized tables

11. Service Broker Overview Service A Queue A App A Service B Queue B Message (XML) Contract* *(Initator to Target using Message, followed by Target to Initator using Message) InitiatorTarget App B

12. The Project: Basic Messaging System  Desktop applications need to update proactively when data changes.  Interaction is via a single SQL Server instance.  Messaging is one-way (publish- subscribe).  Multiple receivers can subscribe to a given message.

13. The Message Type  Could include several pieces of data:  Table name  Type of modification  Key data  Routing information  WELL_FORMED_XML  XPath / XQuery to process data

14. Message Code

15. The Contract

16. The Services and Queues SSituation is slightly more complex here 11->N with routing means we need: SSomething to process messages SSomething to perform routing and dispatch PPerfect case for internal activation TTwo stages – pre- and post-dispatch

17. Service and Queue Layout Initiator Start Service Initiator Start Queue Initiator End Service Initiator End Queue Activation Procedure Receiver Start Service Receiver Start Queue Initiator End Service Initiator End Queue Initiator End Service Initiator End Queue Receiver End Service Receiver End Queue App A App B

18. Queue Code

19. Service Code

20. Routing Infrastructure  Need something to drive message routing  Map apps to services  Determine the messages each app wants  Use tables:

21. The Activated Procedure  Plain old stored procedure  Reads messages off queue and processes them  Figures out where it needs to send each message  Fires off new copies when there is more to do

22. Sending Messages  Format up some XML:  Generate conversation between services using proper contract  Send XML as the message payload

23. Receiving Messages  Use RECEIVE FROM (usually with WAITFOR):  Can embed in procedure – general procedure will need dynamic SQL.

24. Putting It All Together

25. The Devil in the Details  Error Processing  Security  Scaling and Optimization  Diagnostics

26. Errors and Error Handling TTwo major types of errors: TT-SQL errors during message processing EErrors received as messages FFirst Rule of Message Processing PProcess in a transaction IIf they fail, you can roll back and try again SSome errors (XML) are unrecoverable

27. Poison Messages  What happens when a message keeps failing?  Could spin forever  Would waste resources  Auto deactivate queue after 5 failures  Can turn off in 2008 R2+, but don’t  Better: Detect up front / Remove manually

28. Security  2 types of security  Dialog  At conversation level  Master key required  Remote service bindings / certificates  Transport  At instance level  Negotiated between endpoints

29. Scaling and Optimization  Standard guidelines:  Touch as little data as possible  Do as little work as possible  Small messages, minimal processing  Bottleneck analysis (activated procedures)

30. Service Broker Specific Tips SService Broker specific: RReuse conversations DDon’t just use one conversation RReceive multiple messages at once TThe “150 trick” (look it up)

31. Diagnostics RRoll your own LLog at various stages QQuery queues, Service Broker tables, DMVs sssbdiagnose CConfiguration and error analysis CCan handle more complex setups

32. More Resources MMSDN SStack Overflow RRemus Rusanu WWrote large amounts of Service Broker rrusanu.org