1. Troubleshooting Exchange ActiveSync devices
Ingo Gegenwarth

2. Ingo Gegenwarth IT Principal MCM Exchange 2010 Office Server and Services MVP Blog:

3. Agenda IIS logs EWS for calendar items Get-DatabaseEvent
MessageTracking
ActiveSyncDebugLogging & MailboxLogParser
Q&A

4. Why would you troubleshoot EAS devices?
Mobile devices are very in common and cloud services helps being productive.
But there is also a downside as these devices can cause a variety of issues, which are frustrating for end-users as well as for administrators:
Account logouts
Calendar issues (Cancelations, date/time modification….)
Notifications sent-onbehalf of the organizer …

5. IIS logs

6. IIS logs
Since Exchange 2013 you will find two directories of IIS logs:
Front End (Client Access) related logs:
C:\inetpub\logs\LogFiles\W3SVC1
Back End (Mailbox) related logs:
C:\inetpub\logs\LogFiles\W3SVC2

7. IIS logs
The Client Access server/services are responsible for authentication and routing to correct mailbox server
Good tracing for:
Authentication issues
Incoming requests statistics
Contains ONLY issued EAS Cmd
Mailbox server logs all details related to Exchange ActiveSync protocol

8. Example
Field
Value
User
Ingo
DeviceID
UBDKD3QGCL3E7BHS4JJATJVFGA
DeviceType
iPhone
Cmd
Sync
Ver1
160
HH(requested host)
mail.fabrikam.com
SmtpAdrs
Ty (type)
Em ( )
Filter1 (time range)
3 (1 week)
DevOS
iOS E304 As
Allowedl
User=ingo&DeviceId=UBDKD3QGCL3E7BHS 4JJATJVFGA&DeviceType=iPhone&Cmd=Syn c&Log=PrxFrom: _Ver1:160_HH:mail. fabrikam.com_SmtpAdrs:ingo.gegenwarth% 40fabrikam.com_FldrC1:1_Fid:12_Ty:Em_Filter1 :3_St:S_Sk: _Sks: _SSK b1:15_SsCmt:15_TotSvC:1_ColdSvC:1_TotLdC:1_ MR:0_PSyncType1:ICS_GetChgsIter:1_GetChg sTime:3_Pfs:1_BR:1_BPR:0_Ers:1_Fet:86_Pk: _DevOS:iOS E304_SC1:1_A s:AllowedI_Mbx:fabex01.fabrikam.local_Cafe:f abex02.fabrikam.local_Dc:fabdc0014.fabrika m.local_Throttle:0…

9. Example As an example you can decode and split the log entry.
This makes it easier to read and extract data from.
PowerShell is fast and almost always available.

10. IIS Logs
Field
Description
PrxTo
ProxyingTo
PrxFrom
ProxyingFrom
RdirTo
RedirectTo
PrxUser
ProxyUser
Ver
ProtocolVersion HH
Host
SmtpAdrs
UserSmtpAddress
Fid
FolderId Ty
FolderDataType
Filter
FilterType
FiltSms
SmsFilterType St
SyncType Sk
ClientSyncKey
Sks
ServerSyncKey
PSyncType
ProviderSyncType
FstSyncTime
Sync0Time
GetChgsIter
GetChangesIterations
GetChgsTime
GetChangesTime
Cli "{0}a{1}c{2}d{3}f{4}e{5}s{6}fs"
PerFolderClientAdds; PerFolderClientChanges; PerFolderClientDeletes; PerFolderClientFetches; PerFolderClientFailedToConvert; PerFolderClientSends; PerFolderClientFailedToSend
Srv "{0}a{1}c{2}d{3}s{4}e{5}r{6}A{7}sd"
PerFolderServerAdds; PerFolderServerChanges; PerFolderServerDeletes; PerFolderServerSoftDeletes; PerFolderServerFailedToConvert; PerFolderServerChangeTrackingRejected; PerFolderServerAssociatedAdds; PerFolderSkippedDeletes BR
BodyRequested
BPR
BodyPartRequested E
NumErrors Io
NumItemsOpened Hb
HeartBeatInterval
Rto
RequestTimedOut …
Exchange logs a vast amount of data in the IIS logs (cs-uri-query).
This table shows only a subset, but for analysis the most important fields.

11. Exchange Web Services (EWS)

12. Calendar Versioning
In Exchange 2010 Calendar Versioning was introduced and in each version evolved:
Keeps track of each change for each item
Versions are kept for 120 days
Versions are stored in folder Calendar Logging (it depends)

13. Calendar logging Using EWS to search one or multiple mailboxes for calendar items
Search filter items
Search scope
Search filters properties
Subject
StartDateLastModified
EndDateLastModified
CleanGlobalObjectID
GlobalObjectID
AllFolders
CalendarOnly
AllItemProps

14. Calendar logging Prerequisites: Permissions: Software: FullAccess
ApplicationImpersonation
Software:
EWS managed API
PowerShell

15. Calendar logging
This is one of the most important MAPI properties as here the client, which modified the item , is logged.
Depending on Exchange version, this property is not always set.
When EAS was used, a subset of the IIS log field cs-uri-query is extracted.
In all other cases the value contains only something similar than cs(User-Agent) in IIS logs.
E.g.: Client=MSExchangeRPC

16. Calendar logging
In this screenshot you can see the gathered information from an example. First an appointment was created, which was than changed to a meeting request. Location and body was also changed.

17. Get-DatabaseEvent

18. Get-DatabaseEvent
In Exchange 2007 a new table in each database was introduced. It’s called EventHistory. Within this table Exchange tracks modifications. The way to access and query this table was made public in 2013, June.
Prerequisites:
At least Exchange Administration Tools
PSSnapin needs to be loaded manually

19. Get-DatabaseEvent #load PSSnapin
Add-PSSnapin Microsoft.Exchange.Management.Powershell.Support
#get information from mailbox
Get-Mailbox < alias > | Format-List ExchangeGUID,ServerName

20. Get-DatabaseEvent
The database event entries contains detailed information for any item

21. Get-DatabaseEvent
ClientCategory tells you, which protocol was used to modify an item.
In a delegate scenario you will see that PrincipalName and PrincipalSid have values.
Note: In Exchange 2013 there is a bug, which causes PrincipalName and PrincipalSid are not populated.

22. MessageTracking

23. Get-MessageTrackingLog Those logs contains high detailed information and are easy to parse
SourceContext
EventData
MDB
Mailbox
Event
MessageClass
CreationTime
ClientType
SubmissionAssistant
…..
MailboxDatabaseGuid
ItemEntryId
DeliveryPriority
AccountForest
FirstForestHop
E2ELatency
MsgRecipCount
…..

24. Get-MessageTrackingLog
From the SourceContext we can definitely identify, which client and what MessageClass was submitted.

25. MailboxLogParser

26. MailboxLogParser
MailboxLogparser is a tool, which helps you analyzing Exchange ActiveSync mailbox logs. It presents the logs in a human readable format and provides capabilities to search these logs and/or export the Grid into a CSV file for further examination.
Download:
Hurry up: CodePlex is shutting down by December 15th, 2017!

27. MailboxLogParser #Enable logging for a mailbox
Set-CASMailbox < alias > -ActiveSyncDebugLogging:$true
#Check if logging is enabled
Get-CASMailbox < alias > -ActiveSyncDebugLogging | Format-List *debug*
Note: Don’t forget the parameter ActiveSyncDebugLogging!

28. MailboxLogParser #Retrieve MailboxLog
Get-MobileDeviceStatistics -Mailbox < alias > -GetMailboxLog
# Retrieve MailboxLog to an additional recipient
Get-MobileDeviceStatistics -Mailbox < alias > -GetMailboxLog –Notification Addresses <additional recipient>

29. MailboxLogParser Limitations: Total number of logs 5000 MaxAge 72hours
Logsize
10MB (size >5MB compressed)

30. MailboxLogParser Q: Why should we care about the limits?
A: When you enable tracing for a mailbox to collect logs, Exchange will disable logging once a limit was hit  you might end-up with no logs when it happens.
Solution:
Use a scheduled task to make sure logging is enabled and to pull logs.

31. Where to start?
It really depends on the issue you are troubleshooting.
Calendar
First choice would be EWS as gathering data is much faster than parsing several GB of IIS logs
Lockouts/No sync at all
IIS logs in W3SVC1 folder across all servers
FullResync loop
Mailbox logging might help you to identify an item, which causes a client to choke on

32. Combination
Sometimes you need to exam logs from multiple sources to find the main conclusion.
In this example the MAPI property, which represents the client was not populated.
The IIS logs shows an issues command SendMail.
As a sanity check MessageTracking was used to check the value of SourceContext.

33. Combination

34. Summary IIS logs Exchange Web Services and Calendar Logging
Get-DatabaseEvent
MessageTracking
MailboxLogParser
Combination of previous mentioned technics

35. Q&A