1. MSG332 Developing Transport Event Messaging Solutions for Windows and Exchange Max Ciccotosto Program Manager - Exchange Microsoft Corporation

2. Agenda Sinks Overview and Basics SMTP Service What can you do with events? Event model Programming Language Support Advanced Content Conversion Event registration Interface descriptions How to create transport and protocol event sinks

3. Exchange 2003 Vs. 2000 No changes in the Transport Infrastructure Same events Same interfaces Exchange 2000 Sinks will work on 2003. Added support for Retry Improved documentation

4. Add recipient to messages from Internet

5. Windows SMTP Service Store and forward relay Mailing list (Distribution List or ‘DL’) expander “Deliver” to Drop directory “Pickup” from Pickup directory

6. Protocol And Transport Events The Goals To allow the creation of a mail system on top of the Windows 2000 SMTP Service To allow the extension of Exchange 2000 and Windows 2000 SMTP Service

7. Exchange Exchange uses Windows SMTP Service Exchange depends on SMTP Service Exchange extends SMTP Service with events Adds store (mailboxes), store driver (local delivery to store), a whole lot more All messages go through events, even X.400 and RPC inbound/outbound

8. Protocol And Transport Events Access to all message properties 821, 822 and Transport specific properties Many places in the code to “hook in” COM based, C++ only Support for Synch and Asynch events Fast!

9. Event Model Protocol Events Extend SMTP protocol (ESMTP) Modify existing SMTP protocol commands and responses Transport Events Extend message processing system (address resolution, restrictions processing, message routing) Code runs on each message Read/Write access to envelope and body Executes in-process with IIS/SMTP

10. Uses For Protocol Events Billing and charge-back based on number of connections, time of connections Monitoring system via SMTP, implement new ESMTP commands Provide new SMTP authentication methods Provide RCPT checking Custom anti-spam scripts

11. Uses For Transport Events Forward all mail for a domain to a mailbox Scan all inbound mail from the Internet for keyword, subject, attachment filename Processing spans multiple internal mail systems Virus checking Spam (UCE) checking Stop mail with all numeric From: address Stop mail with keywords in subject or body Add a disclaimer to all outbound mail to the Internet Apply custom restrictions

12. Event Overview Protocol Command Events Transport Submission Pre Categorizer Categorizer Post Categorizer Routing Store Driver

13. Transport Core SMTP Service PreCatEventPostCatEvent Routing Event SMTP Service Message Categorizer Routing Engine (IMessageRouter) Store Submission MTS Out MTA CategorizerEvents Advanced Queuing PreCatSinkPostCatSink OnSubmissionEvent OnSubmissionSink MTA Store Local Delivery MTS Out MTS In In

14. Exchange Event Sinks Routing Sink Exchange Store Driver Protocol sinks “Filter list”, Link State update, Authentication Categorizer Resolves recipients against Active Directory to find Exchange mailbox server Applies restrictions Uses Exchange 2000 shared cache

15. Language Support C++ CDO_onArrival OnSubmission Event Scriptable, dual-interfaced CDO Message object gives easy access to message fields, attachments Managed Wrappers Wrappers for all Transport Objects Interop abstraction http://msdn.microsoft.com/library/default.asp? url=/library/en- us/dnsmtps/html/writingmngsinks.asp

16. CDO Sink

17. Content Conversion

18. Exchange Content No Content Conversion SMTP Inbound from Internet SMTP Outbound to Internet Content Conversion MAPI-Submitted TNEF Outlook 2000 Outlook Web Access

19. Bifurcation Recipients:Content:Default MailMsg Internal User External User Body Recipients:Content:S-TNEF Internal User MailMsg Recipients:Content:MIME External User Body MailMsg MAPIClient Pre Cat SMTP Post Cat

20. Exchange Issues Not easy to modify content of MAPI/OWA- submitted messages UNLESS it is going to “The Internet” Design architecture so you don’t need to modify TNEF Much easier to write Store Event sink to modify internal messages Transport Events best suited for “the border” Bottom Line

21. Modifying TNEF You can modify the TNEF Hook PostCat sink Decode TNEF – MAPI32.DLL function Modify MAPI properties Encode TNEF – MAPI32.DLL function Save TNEF to message Caveats Message has to leave E2K server via SMTP (no X.400, RPC) If internal (S-TNEF), this is tricky!

22. Disclaimer Sink

23. Event Registration

24. Metabase Used as configuration storage in IIS ACLs, key/value based, remote-able Inheritable tree structure SMTP Configuration all stored in Metabase Active Directory is not required by SMTP Service

25. Server Event Bindings Stored in Metabase /smtpsvc/ /EventManager Event dispatcher (AQ) uses bindings to match sinks to events Rules (“filter”) Priority for each event Event sinks executed in order of priority Same priority events executed in random order

26. Event Rules Limit event call to certain messages Case-insensitive Can filter on MAIL FROM=, RCPT TO=, and EHLO= Available on Submission, PreCat, PostCat events Exchange tip To catch MAPI-submitted messages, must have no filter rule

27. Event Rule Example MAIL FROM= user1@domain;RCPT TO=user2@domainuser1@domain TO=user2@domain Makes sink only fire on messages from user1@domain OR to user2@domain user1@domainuser2@domain Can use wildcards: *@domain, user@foo**@domain Can have multiple RCPT TO:s, multiple MAIL FROM:s, multiple EHLOs; each is OR’d

28. Creating Bindings 1. Use smtpreg.vbs available in the Exchange SDK. 2. Do it yourself: SEO used to modify Metabase to create bindings Tools for registration, de-registration, debugging (list binding) VBScript, or can make sink self-registering via regsvr32 entry points Troubleshooting: list bindings to see what is installed

29. Registering Sinks: SMTPREG

30. Protocol Events

31. Protocol Event Sinks Sink implements one or more sink interfaces Each sink fired based on what category the session is in Inbound On Inbound Command, On Server Response Outbound Session Start, Message Start, Per-Recipient, Before Data, Session End

32. Protocol Event Sinks ISmtpInCommandSink Called when inbound SMTP command is received ISmtpOutCommandSink Called when outbound SMTP command is to be sent ISmtpServerResponseSink Called when server responds to previously sent command

33. Protocol Event Interfaces General Interface Property Bags IServer - Information about the SMTP Virtual Server ISession – Information about the current SMTP Session IMailMsgProperties – Information about the current message being received Context Interface System-implemented interface Sink can call to get and set information about commands and responses, set state

34. Outbound Protocol Commands Built-in outbound commands are PRIO_DEFAULT priority Can add commands before, in place of, after commands Session Start: EHLO Message Start: MAIL FROM: Per Recipient: RCPT TO: Before Data: DATA Session End: QUIT

35. Protocol Event Contexts ISmtpInCommandContext command that was received ISmtpOutCommandContext command to be sent out ISmtpServerResponseContext response to a command previously sent

36. Transport Events

37. IMailTransportSubmission OnSubmission event Recipients should be added here (before categorizer-related events) IMailTransportOnPreCategorize Should be used to check recipient lists All recipients should be present IMailTransportOnPostCategorize DLs will be expanded – actual recipients will be listed

38. Transport Events Related Functions IMailTransportNotify Sink may return asynchronously, return MAIL_TRANSPORT_S_PENDING That message will be halted and wait for callback via this function Dispatcher will allow other messages to be processed until the thread pool limit

39. Transport Events Related Functions IsCacheable If not defined, sink instance deleted after exit Overhead of CoCreateInstance() for each event raised if not cache-able Sink must be re-entrant/thread-safe No shared data without proper locking Protocol events also can implement

40. Transport Events AQ IMailTransportRouterReset System implemented Routing Engine calls this to cause AQ to recompute next hop for all queues AQ then recomputes by calling into Routing Engine

41. Transport Events Categorizer Default Categorizer can resolve against Active Directory Features of Categorizer sink Can look up attributes with almost no performance cost Can replace default LDAP lookup engine More complicated than PreCat/PostCat sinks; more powerful

42. IMailMsg OnSubmission/PreCat/PostCat sinks get object IMailMsg IMailMsgProperties object MapContent() – maps content of message into memory CopyContentToFile() – copies content of message to file GetProperty()/PutProperty() – reads and writes properties Many methods

43. Modifying Recipient List Recipient list is not thread safe Only one thread at a time may have access to it How to add to IMailMsg recipient list Allocate new recipient list Copy original recipients to new list AddPrimary – Replaces a matching recipient with the one being added (recipients have properties that can be overwritten) AddSecondary – will not replace existing recipient (continued …)

44. Modifying Recipient List (… continued) 3. Add new recipients with AddPrimary/AddSecondary 4. Call WriteList – Replaces original recipient list

45. Transport Events Routing IMessageRouter Sink implements GetNextHop Major Routing Engine interface Input: destination address Output: next hop address

46. Transport Events Routing IMailTransportSetRouterReset Called by SMTP or MTA when link down to request computation of new path Will later call AQ via IMailTransportRouterReset to recompute next hop for all queues

47. Transport Events Routing IMailTransportRouterSetLinkState Allows a routing sink to set link state, e.g. for scheduled connections Each Link in AQ has a Schedule ID Each Link can be set to a specific link state Pending scheduled connection Pending retry Force connection now Connect even if no messages

48. Transport Events Routing IMessageRouterLinkStateNotification Routing sink implements this interface Sink implements, system calls this function when the link state changes Exchange 2000 routing sink uses to mark a connector as “up” or “down” in internal link state table

49. Subsequent Sinks Sink returns S_OK Other sinks at same or lower priority are called Sink returns S_FALSE Other sinks at same or lower priority are not called

50. Recipe For Creating Transport Event Sink

51. Creating Transport Sink 1. Implement sink interface method 2. Register event using smtpreg.vbs 1. Register COM interface of sink in registry 2. Register interface in event binding database in metabase via SEO 3. Send messages through, watch it fire!

52. CDO OnArrival Event

53. CDO OnArrival CDO Event called by transport: ISMTPOnArrival VB shell Implements CDO.ISMTPOnArrival Public Sub ISMTPOnArrival_OnArrival ( byVal Msg As IMessage, pEventStatus as CdoEventStatus ) ' Event sink code here End Sub

54. Subsequent Sinks OnArrival determines if subsequent sinks (lower priority) will run by returning CdoEventStatus enum cdoRunNextSink0 Run the next sink cdoSkipRemainingSinks1 Skip any remaining sinks

55. CDO Message Envelope IMessage.EnvelopeFields collection arrivaltime clientipaddress messagestatus (read/write) recipientlist Senderemailaddress

56. CDO Benefits Dual interface Can be used with C++, C, VB, VBScript, JScript Headers parsed MIME parser built-in

57. Creating CDO Sink 1. Implement CDO OnArrival function 2. Register sink in metabase via SEO to use CDO.SS_SMTPOnArrivalSink class 3. Send messages through, watch it fire!

58. Managed Sink

59. References http://msdn.microsoft.com/exchange http://msdn.microsoft.com/library/ Revised content on MSDN (Sept) New whitepapers Disclaimer Whitepaper (coming) How to customize NDRs (coming) Managed Wrappers Catch-all Sink: Q324021

60. Ask The Experts Get Your Questions Answered Talk one-on-one with a community of your peers Community Experts: Microsoft product teams, consultants and Tech*Ed speakers Resources: whiteboards, internet, etc. Location: in the middle of the Exhibit Hall Hours: at least 12-3:30p every day I will be at the ATE after this session

61. Community Resources http://www.microsoft.com/communities/default.mspx Most Valuable Professional (MVP) http://www.mvp.support.microsoft.com/ Newsgroups Converse online with Microsoft Newsgroups, including Worldwide http://www.microsoft.com/communities/newsgroups/default.mspx User Groups Meet and learn with your peers http://www.microsoft.com/communities/usergroups/default.mspx

62. Suggested Reading And Resources The tools you need to put technology to work! TITLE Available Microsoft® Exchange Server 2003 Administrator's Companion: 0- 7356-1979-4 9/24/03 Microsoft® Windows® Server 2003 Administrator's Companion: 0-7356-1367-2 Today Microsoft Press books are 20% off at the TechEd Bookstore Also buy any TWO Microsoft Press books and get a FREE T-Shirt

63. Thank You!!! massici@microsoft.com Please Fill Out The Evaluation Form

64. General Comments

65. CDO Msg Object Use in Transport Event Sinks  Not just OnArrival event  Combines flexibility of Transport Event sources with ease of CDO Message wrapper  Uses CDO’s IMessage interface in C++ Transport Event Sink  Lower performance, much simpler to write  Use especially for sinks that don’t fire often or on low traffic servers

66. Things To Watch Out For Only use on Internet border Envelope available even on internal messages Attachments and body on internal messages may be within TNEF Events can slow down server performance Message is stuck until events are finished with it Event sink doing I/O can cause message to “hang in queue”

67. Performance Comparison Can be very similar performance Biggest bottleneck is using CDO Message object Both C++ and Script Script itself has relatively low overhead Implement IsCacheable in order to be cached for later use by subsequent calls (both CDO and Transport Events) Making a sink asynchronous makes the biggest difference to throughput CDO OnArrival vs COM event sinks

68. CPU Utilization In performance tests, adding event sinks mostly add CPU overhead Most Exchange servers operate with low CPU utilization, due to I/O bottleneck Test Note: test is not designed to test max throughput, just relative speed with and without sink No Sink registered VBScript Disclaimer ½ HTML, ½ plain text Throughput (msgs/sec) 67.666.9 CPU Utilization19.1%51.8%

69. CPU Utilization Conclusion: You may have CPU to spare to allow your border gateway to run with a CDO sink with no performance degradation

70. Deciding C++ or Script Do you need flexibility of PostCat or other interfaces? Need C++ Do you want the highest performance possible? Need C++ Do you need to do file I/O or remote calls in the sink? Should use C++ so can complete asynchronously

71. Community Resources http://www.microsoft.com/communities/default.mspx Most Valuable Professional (MVP) http://www.mvp.support.microsoft.com/ Newsgroups Converse online with Microsoft Newsgroups, including Worldwide http://www.microsoft.com/communities/newsgroups/default.mspx User Groups Meet and learn with your peers http://www.microsoft.com/communities/usergroups/default.mspx

72. evaluations evaluations

73. © 2003 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.