0. OIA Tools & Technology Evaluation Methodology
Prepared for State of Ohio, Office of Budget & Management
Deloitte Consulting
April 18, 2008

1. Content Overview Objectives OIA Technology Needs & Requirements
Current Vendor & Tools Landscape
Software Selection Methodology
Next Steps

2. Objectives
Identify high level Office of Internal Audit (OIA) business needs, and technical and functional requirements for software tools
Perform high level review of existing software tools and functionality related to the OIA’s needs and requirements
Develop a software evaluation methodology, to include a criteria evaluation approach that could be used in drafting a Request for Information (RFI) to software vendors

3. OIA Technology Needs & Requirements

4. Current Needs & Requirements
Per discussion with key stakeholders within OBM, OIT, and the HB166 Advisory Committee, the following high level business and technology needs and requirements were identified for OIA:
Needs:
Tools to support the management of OIA, to include time reporting, accountability, reporting and billing
Tools to support the management, maintenance, and retention of audit workpapers
Tools to support the risk assessment and audit universe maintenance function
Tools to organize and drive governance, risk, and compliance initiatives (e.g. CSA’s by IT Departments)
Tools to support the auditing of key financial and operational controls for critical application systems
Process mapping capabilities for documentation and education purposes
Continuous and automated monitoring of controls
Requirements:
Solutions must be able to support a large and distributed workforce (approx 125 auditors in various locations w/ remote capability (web-based), custom reporting, & centralized QA monitoring needs)
Solutions must be cost effective
Solutions would benefit by incorporating existing OIA tools and technology (e.g. Sharepoint, web architecture, etc.)
Solutions must have adequate security measures to safeguard audit evidence
Web-based for customer usage (e.g. comment tracking) and improved transparency for the public (e.g. final reports)
Facilitates record retention requirements & timely public record requests
Software vendor must have stability & provide adequate training/support

5. Current Vendor & Tools Landscape

6. Current Situation Analysis
Internal audit departments have growing pressure for increased oversight and assurance
With mounting pressure and increased workloads, internal audits are overwhelmed with manual and decentralized processes
Organizations are now looking to reduce the cost and improve efficiency associated with their internal audit departments
Focus has shifted from reliance on “error-prone” manual controls to automating and monitoring the execution of those controls
Interim solutions provide adequate controls repository functionality but offer little in improving the approach
Tools are able to assist with enterprise wide risk assessment, compliance, planning, scheduling, control automation, control monitoring, review, report generation, trend analysis, and storage.
Flexibility of the software allows OIA to customize and scale the tool according to their business or changes in IT infrastructure and platforms
Tools help assess various risks (financial, environmental, health & safety, IS) under a consistent risk methodology

7. Decisions to Consider
Organizations are assessing the value/cost of implementing a tools solution to help with internal audit processes
Focus on a broader Governance, Risk and Compliance (GRC) program
Understand your current process & issues and focus on improvement
Timing, cost and level of effort to implement, to include initial and ongoing training needs, software licensing, and maintenance
How best to enhance the existing audit program into a more sustainable, repeatable process
Improve documentation version control
Consider self-assessment procedures
Improve efficiency of the process (i.e. workflow)
Enhance reporting features to improve effectiveness
Understanding the vendor landscape for the next generation of audit technologies
Find the tool that best fits your needs
GRC is the “hot” label today
Many boutique vendors in this space today to address cost concerns and unique needs

8. Software Vendors
In the past, there was a clear definition between Internal Audit software and GRC software for controls auditing. Presently, these competencies are merging in new releases of software products from numerous vendors
Internal Audit Software
Focus on audit procedures and maintenance
Workpaper creation and maintenance, management signoff, audit planning and scheduling, audit budget
Centralized data repository, online checkout functionality, best practice/knowledgebase repository
Compliance with SOX
GRC Software
Focus on governance, risk, and compliance initiatives
Policy management, incident management, asset management
Risk assessment, threat management, risk dashboards
Internal audit components incorporated

9. Market Trend – Internal Auditor Software Survey
Survey respondents:
21% Government industry
Majority of respondents from small audit shops (reason for Excel’s stronghold)
Large departments rely on specialty products (TeamMate, AutoAudit)
Software related concerns noted:
#1 – Ability to find software that meets the department’s specific needs
#2 – Cost
Source: Grey, Glen. “An Array of Technology Tools.” Internal Auditor August 2006:

10. Market Trend – Internal Auditor Software Survey
Among companies who use a risk management analysis tool (beyond Excel), TeamMate (6%) and AutoAudit (3%) showed largest market share.
70% of Government agencies use audit management and risk management software tools
Source: Grey, Glen. “An Array of Technology Tools.” Internal Auditor August 2006: 10

11. Software Selection Methodology

12. Selection Methodology
A three phased approach is recommended for effectively selecting a software solution:
Phase I
Planning and Requirements Definition
Phase I will begin by defining the scope for OBM’s tool selection process, to include business needs and requirements and setting a timeline for the process. Next, an extensive list of requirements will be reviewed and weighted according to the specific needs of OIA. A preliminary list of potential vendors will be gathered.
Phase II
Request for Proposal Development and Execution
During this phase of the selection process, the project team will take the information learned during phase I and trim the vendor list to only the most viable candidates. A request for information (RFI) will be sent to each vendor, responses will be compiled and analyzed and a demo list of 2-3 vendors will be created. Phase II will be completed by facilitating the vendor demonstration process, scoring and compiling of results.
Phase III
Final Analysis and Recommendation
During the final phase of the selection process, the project team finalizes the selection process, presents the compliance system recommendation to executive management and facilitates next steps toward solution implementation.

13. Sample Timeline / Milestones

14. Selection Process – Phase I
Tasks
Initiate Project and Establish Team Roles and Responsibilities
Establish project objectives, scope, priorities, and determine key milestones.
Assist OIA in identifying project team members, formalize team structure and reporting responsibilities, and develop detailed project work plan.
Identify Unique Functional and Technical Requirements
Develop the unique business and system requirements for the software tools. These requirements will be used in Phase II as a basis for determining which tool provides OIA the best fit to their requirements.
Establish Critical Success Factors
Meetings focused on communicating and affirming issues and critical success factors, understanding specific project expectations, and identifying how these will impact OIA’s organization and the selection project.
Functional and technical requirements coupled with the critical success factors will serve as a detailed checklist to guide and facilitate vendor demonstrations.
Tool Selection Process
Phase III
Final Analysis and Recommendation
Phase II
Request for Proposal Development and Execution
Phase I
Planning and Requirements Definition
Key Deliverables
Detailed Project Work Plan
Defined and Weighted Functional and Technical Requirements List

15. Requirements
Weight business and technical requirements according to OIA’s needs

16. Selection Process – Phase II
Tasks
Develop Short List of Vendor Candidates
Gather knowledge within the marketplace to identify a short list of 4-5 potential candidates.
Phase III
Final Analysis and Recommendation
Phase I
Planning and Requirements Definition
Tool Selection Process
Phase II
Request for Proposal Development and Execution
Develop RFI/Scorecard and Solicit Vendor Bids
The Request for Information (RFI) will require each vendor to provide a sample implementation schedule, pricing, warranty, references and other pertinent guidelines for the bidders to follow. OIA to populate the vendor scorecard.
Vendors will be contacted and bids solicited from them.
Facilitate Comparative Analysis of RFI Responses
Review requirements and institute a ranking system to evaluate the vendor proposals.
Collect vendor RFI responses and prepare a comparative analysis report.
Utilize comparative analysis report to further condense candidate list to 2-3 vendors for demonstration.
Facilitate Vendor Demonstration Sessions
Invite top candidates to OIA to present their system and to answer/clarify specific questions related to their RFI response.
Develop demo scripts to evaluate vendors
Key Deliverables
Short List of Vendor Candidates
Request for Information (RFI)
Vendor Scorecard
Comparative Analysis Report

17. Request for Information
Sections:
Company (OIA) background and overview, description of desired solution, benefits sought
Response directions - OIA contact information, response due date, target demo dates
Vendor information – company profile and tool, including financial solvency & market share
Requirements – customized questions according to OIA requirements, response from vendor limited to 500 characters.
Customer references – list of customers of similar size/requirements
Disclaimer – RFI solely for informational and planning purposes

18. Selection Process – Phase III
Tasks
Facilitate Vendor Reference Checks
Utilize a customized questionnaire to evaluate each vendor’s performance at comparable clients. The questionnaire will be customized to OIA’s specific requirements and interests.
Prepare Final Selection & Recommendation Report
The project team will work together to put the finishing touches on the selection report.
Provide guidance for executive-level presentation.
Select the Best Internal Audit Tool (or Tools)
Conduct a detailed review session to reach agreement that OIA’s requirements are met by the chosen vendors and solutions.
Finalize Selection & Recommendation Report
Provide recommendation report template for OIA to complete and present.
Support OIA in Vendor Contract Negotiations
Present OIA with applicable rate and licensing information based on existing vendor relationships.
Phase II
Request for Proposal Development and Execution
Phase I
Planning and Requirements Definition
Tool Selection Process
Phase III
Final Analysis and Recommendation
Key Deliverables
Reference Check Summary
Final Business Case
Sample System Requirements