OnBase Troubleshooting Greg Wheeler, Customer Care Team Manager

What went wrong? Something is misconfigured Bad data causes recurring errors Your specific environment causes a strange problem with the software A performance bottleneck causes errors Custom Code is failing You may have discovered a bug in the software

Two sample scenarios Documents stopped appearing in a Workflow queue “Problem with the environment” Unity Client error/ performance troubleshooting “Problem with the software’s behavior” (based on actual Customer Care Team solved cases)

Scenario #1: Where are my files? Users report that a Workflow queue that normally contains documents is empty These Files are images that are released from another system Just started this morning!

Uh, where are they? ?

Where do we start? Most problems are way too big—we need to make them smaller and more manageable/testable

Data Collection We need to know what the user, application, script or service was doing We need to try to differentiate between a coding, data or environmental cause Can it be reproduced at will? What about in another environment? What do the logs say?

Narrowing an issue down Is it a problem with thick client, Core Services, or both? What servers can we take out of the equation? What modules can we eliminate? What functionality can we exclude?

First Step: Is Workflow…working? Is it limited to this queue/lifecycle in Workflow, or are there others affected? ?

Thoughts We see timers are working – We see other documents moving around We can look at the timer machine to make sure THIS timer is still running Let’s trace further back

Is DIP…you know, DIPping?

What to check? DIP is importing documents as a service on the DIP machine To troubleshoot DIP, we would stop the service, then fire up a client with the SCHED switch

If you see this…it’s not OnBase

If you see this…there’s a problem

Where did it fail?

Sys – Verification Reports

Let’s look at the directory

ProcMon Demo

Process Monitor ProcMon can be used to see behind-the-scenes Windows calls and access requests http://technet.microsoft.com/en-us/sysinternals/bb896645.aspx

Solution

Issue #2: Unity Performance Users are experiencing slow client performance first thing every morning Workflow queues taking a long time to navigate between documents Sometimes happens in custom queries, document retrieval All users affected

Options Observe Unity directly on Application Server (still slow?) What about testing the thick client? (Verbose) Look at Application Server performance (perfmon…or resmon) Look at connectivity to DB/Diskgroups (ping/WireShark) Look at Diagnostics Console

Let’s run Unity from the AppServer

Sometimes Thick Client is valid…

Test the Database Choose a SELECT query and run it directly on the database server. Compare times to complete Is the query coming back quickly enough?

Perfmon

Consider ResMon

Diagnostics Console - Service Tab Change to “True”

What can ResMon tell you?

Test Connectivity Application Server > Database Application Server > Diskgroup

Diagnostics Console “What is OnBase doing right now?” “What went wrong?” “What queries are running, and how fast?”

Advice Beware the Database tab in Production Consider logging to a file to review on another machine Log the least amount of information needed to troubleshoot

The rest of the story Grabbed some queries from the Database tab from around the same time Ran them in SQL Server Management Studio Slow there too! Unoptimized indexes

The Smoking Gun

Your Issues!