1. Web 303 Data Access Patterns
Richard Turner
Program Manager
XML Enterprise Services
Microsoft Corporation

2. Objectives Data services in the SOA Passing data between services
Basic ideas
Design practice
Passing data between services
Options and practice
Data services as data access patterns
Best practice for R/O and R/W data

3. Agenda Services and Data Services How To Pass Data
Services, Tiers, and Data Services
Coupling and Near/Far
Kinds of Data Services
How To Pass Data
Simple Data Services
Transactional Data Services

4. Before We Get Started Layer: Architectural Unit Tier: Deployment Unit
Presentation == Facade
Tier: Deployment Unit
Service: Functional Module
We used to have one service per tier
Tiers used to blend architecture and deployment

5. public class MyBusinessComponent { ... }
What is a Service?
More Flexible
App
GUI
HTML
COM
W.S.
Public Interface
Data
Presentation
Presentation
public class MyBusinessComponent { ... }
Application Logic
Data
Resource Access
Less Flexible
SQL MQ
W.S.
App
COM

6. Tiers and Services Data Access App Logic Presentation Data Client Data
Data Logic
Data Access
App Logic
Presentation
App Logic
Data
Client
Data
Client
App Service
Data Service
Data Access
Resource Access
Data Access

7. Coupling Interrelatedness of client and server
Message patterns
Message schema
Protocols
Shared Environment
Versioning
Coupling is a matter of degree….

8. Near vs. Far You’re Near If…. Otherwise, You’re Far
You’re on closely related machines and
You’re in the same organization and
You’re connected reliably and
You’re in the same trust domain and
You’re not interoperating
Otherwise, You’re Far

9. Near Data Services Frequent, smaller, “chatty” messages
Assume Microsoft on both sides
Parts of same application
Assume shared trust, good LAN, etc….
Technologies
Enterprise Services
.NET Remoting
ADO.NET / SQL Server

10. Far (Web) Data Services
Larger, less frequent messages
Interoperability
Decoupled application logic
Only related through contract
Technologies
ASMX / WSE
Less common than near data services

11. Agenda Services and Data Services How To Pass Data
Types of Contracts
Recommendations
Simple Data Services
Transactional Data Services

12. Open Contract Examples Ineffably hip: X(ml)Treme programming
COM IUnknown, System.Object, Xsd:AnyType
Ineffably hip: X(ml)Treme programming
Particularly compelling for tool builders
Not particularly productive yet
Potentially dangerous / awkward
May be useful as a versioning tool
How can you do this?
SOAP extension (can be typed)
XMLElement, Object, String parameters
Potentially dangerous since it demands programmatic validation, handling & manipulation of data.

13. Simple Types & Structures
Examples
MSMQ / QC / System.Messaging
ASMX / WSE
Simple and fast
You publish Contract (i.e., WSDL)
You publish Schema (i.e., XSD in WSDL)
You send messages or…
… Your method calls become messages

14. Untyped and Typed Datasets
Untyped Dataset
Dynamically typed access to data
Schema in contract is very open
Detailed schema is sent at runtime
Typed Dataset
Strictly typed access to data
Schema is limited to specific elements
Detailed schema available at design-time

15. Dataset as a Pattern How does it work Considerations
You send a data-filled tool
Known schema (runtime/design-time)
Helper methods guide use
Considerations
Shifts load away from data sources
Constrains use modes
Interop can be problematic

16. CLR Application Types Instance with methods as parameter
Considerations
Serialization
Shared app types
Behavioral coupling
Versioning
Marshal-By-Ref problems
Common in Remoting, E.S.
Impossible in ASMX

17. Recommendations Great!: Simple Types & Data Structures
Yikes!: Open Contract
Near Data Services
Use Typed Datasets
Roll your Dataset definitions as needed
Avoid application types & MBR
Web Data Services
Use Simple types & structures
Consider App instead of Data Services
Avoid application types (classes with methods etc) for data passing. Use data containers without methods (like “C” structs)

18. Agenda Services and Data Services How To Pass Data
Simple Data Services
R/O Data Service Patterns
R/W Data Service Patterns
Data Caching
Transactional Data Services

19. R/O vs R/W Data: Northwind
Territories
Region PK
TerritoryID PK
RegionID
TerritoryDescription
RegionDescription
EmployeeTerritories
FK1
RegionID
Employees PK
EmployeeID
PK,FK1
EmployeeID
Suppliers
PK,FK2
TerritoryID I1
LastName PK
SupplierID
FirstName
Title I1
CompanyName
Orders
TitleOfCourtesy
ContactName
BirthDate PK
OrderID
Customers
HireDate
ContactTitle
Products
Address PK
CustomerID
Address
FK1,I1,I2
CustomerID
City PK
ProductID
City
Order Details
FK2,I4,I3
EmployeeID
Region I2
CompanyName
Region I5
OrderDate I2
PostalCode I3
ProductName I2
PostalCode
PK,FK1,I2,I1
OrderID
RequiredDate
ContactName
FK2,I5,I4
SupplierID
Country
PK,FK2,I4,I3
ProductID
Country I6
ShippedDate
ContactTitle
HomePhone
FK1,I2,I1
CategoryID
Phone
FK3,I7
ShipVia
Address
Extension
QuantityPerUnit
Fax
UnitPrice I1
City
HomePage
Quantity
Freight
ShipName I4
Region
Photo
UnitPrice
UnitsInStock
Discount I3
PostalCode
Notes
ShipAddress
Country
FK1
ReportsTo
UnitsOnOrder
ShipCity
Phone
PhotoPath
ReorderLevel
ShipRegion
Discontinued I8
ShipPostalCode
Fax
Categories
ShipCountry PK
CategoryID
Shippers I1
CategoryName
Description PK
ShipperID
Picture
CompanyName
Static
Near Static
Dynamic
Phone
Hot
almost never
hardly ever
time to time
pretty often

20. General R/O Pattern Pass Copies Replication Indexing Bring On-Machine
Like Instances
Replication
Indexing
Bring On-Machine
Update sometimes
Think timeliness M2 M3

21. Pooled R/O Copies asmx application service
Stateless instances
EnterpriseServices data service
Fetches dataset from pool
Pass smart copies to app service
app svc
data svc
Can’t change anything through R/O copy.
Don’t care what copy you have
Data location is hidden
Update policy hidden; updates happen automatially
Data

22. Caching: Near Data Service
Dataset stored in ASPNET session state if sessions are necessary
Data Service hands out datasets
Data Service hands out Enterprise Services pooled objects containing datasets
App Svc
Data Svc
Data
Machine A

23. Caching: Far Data Service
Dataset stored in ASPNET session state if sessions are necessary
Data Service hands out Enterprise Services pooled objects containing datasets
Data Svc
Data Svc
Data
Far Data Service distributes its data as web-standard messages
Near Data Service distributes its data via a object containing a dataset

24. Caching and Updates Data App Svc Data Svc R/W Svc
During next fetch, new instances are created and pooled
Enterprise Services Object Pools are an excellent choice here
Data Service may know to refresh cache by policy (i.e., 5pm daily)
Data Service may sense writes directly (triggers, notification, etc)
Data Service invalidates existing instances

25. Abstracting Sources Data Data Data Svc Other Svcs
Data Service hides location and structure of data from apps
Data Service provides custom, simpler, well-scoped datasets for each purpose
Locally, machine seldom hits database, apps do not notice additional machines
Other services go to Data Service frequently, use smart customized copies of data
Data Service aggregates many database views into many app views
Data Service goes to database infrequently; holds copies of data locally
Data Service retrieves large datasets containing all relevant data for all apps
Number of machines can grow; each machine holds only temporary database connection

26. Basic R/W Pattern Data App Svc
A Bulk Service might present cached aggregated data for R/O
App Svc
Data
One service per database; one instance per record
Reads of individual records come here, too
Bring R/O data to work; Send R/W work to data

27. R/O Patterns Revisited
Do R/O Patterns carry over to R/W?
Caching: Coherency becomes costly.
Replication: Conflicts hard to resolve.
Aggregation: Send work to data.
Security: Still works!

28. Writable Service Instances
Writing “Own” (isolated) Data
Bank, Hotmail
Writing “Inventory” (shared) Data
Online Stores
App Svc
R/W Svc
Data
Queue in front of work can fix contention
Locks on Service Suffice

29. Security App Svc Data Svc User Runs in low-privilege (ASPNET) account
Requires higher privileges
Identify, Authorize
App Svc
Data Svc
HTTP Auth.
User
Identity transition,
caller identification, authorization
through
Enterprise Services

30. Indexing Large Databases
Near Indexing
Dataset queries powerful
Far Indexing
Dedicated servers like Sharepoint
Special protocols (XMLA, etc)
Write service to front end DB

31. Data Isolation Patterns
Simple access patterns
Bring R/O data to you: copies OK
Send work to R/W data: copies not OK
Follow the lead of SQL language
Deal with R/O data in bulk
Deal with R/W data in isolation
Design Pattern: One Record/One DB

32. Transactional Data Services
Transactions help you:
Keep your data clean & consistent
Handle failures effectively & gracefully
Two main types of transaction:
2 Phase Commit
Typically a near service
Largely provided by platform & infrastructure
Compensation
Typically for far or disconnected environments
Requires more code

33. The Problem of Errors
Uh! What now? ?

34. Near: Two-Phase Commit
Yes!
Yes!
Transaction Manager
Prepare!
Abort! ?
No!
Yes!

35. Far: Sagas & Compensation
Coordinator
Coordinator

36. Summary Carefully architect your data access patterns
Use these patterns consistently and regularly
Updating data comes at a price
Updating shared data introduces serialization
Updating isolated data minimizes contention etc.
Consider impact on scale when architecting solution
A successful, effective data access strategy can make or break a product/project

37. Ask The Experts Get Your Questions Answered
Talk one-on-one with a community of your peers
Community Experts: Microsoft product teams, consultants and Tech*Ed speakers
Resources: whiteboards, internet, etc.
Location: in the middle of the Exhibit Hall
Hours: at least 12-3:30p every day fc

38. Community Resources Community Resources
Most Valuable Professional (MVP)
Newsgroups
Converse online with Microsoft Newsgroups, including Worldwide
User Groups
Meet and learn with your peers

39. evaluations

40. © 2003 Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.