1. September ‘13 Webinar System Redesign: Lessons Learned

2. New Jersey MATRX Project Susan Matthews MATRX Project Director New Jersey Motor Vehicle Commission Susan.Matthews@dot.state.nj.us Deb Gerrow Senior Manager Mathtech dgerrow@mathtechinc.com

3. Indiana STARS Program Kent Schroder Chief Information Officer Indiana Bureau of Motor Vehicles kschroder@bmv.in.gov

4. Kansas MOVRS Project Toni Roberts Project Manager Kansas Department of Revenue Information Systems Toni.Roberts@kdor.ks.gov

5. Facilitator Sheila Prior Director of Member Support, Regions III & IV AAMVA sprior@aamva.org

6. Project Status’ New Jersey IndianaKansas 30 year old mainframe Oversight vendor (MathTech) hired in 2006 HP (Saber) awarded contract in 1/09 Plan for 3 phase implementation 1.Infrastructure upgrade / document scanning – in production 1/11 2.VRT functions, customer manager & web transaction – late summer ‘14 3.Driver & business licensing, driver history, case management – TBD – will be big bang Awarded contract to Unisys in 2000. STARS phases: 1.Upgrade legacy system to Windows Timeline: 8/00 to 12/01 2. Create and install STARS Planned Timeline: 6/01 to 7/03. Actual Timeline: 6/01 to 10/04 Branch piece unusable but accepted as contract did not allow the state to reject. 3. Create and install STARS application in home office Planned Timeline: 6/02 to 7/04. Actual Timeline: 1/05 to 7/06. Total Cost: $34,408,060.78 Contract finalized and letter of intent signed with 3M on 7/1/09 to purchase/install new T&R and DL system 5/7/12 – T&R component implemented statewide Drivers system planning is underway

7. Planning New JerseyIndianaKansas Define project visionIdentify needsComplete feasibility study Get executive sponsorIdentify funding optionsDetermine funding source Set-up governanceGain project approval $4 surcharge on reg Figure out how to backfill staff Do not leave planning to vendor Develop RFP - functional and technical requirements Limit/stop changes on existing system Need internal or external project management Develop project charter Define current rules, processes, side systems Take time in detailing workplan Get Governor/Legislature on board Include key project staff in planning Develop technical training plan Don’t accept “generic tasks” Define implementation strategy Make sure workplan is not a payable deliverable

8. Data Clean-Up New JerseyIndianaKansas Can’t start early enough / start before project Determine data that needs to be cleaned up addresses, VIN’s, SSNs Identify problems and do clean-up in legacy system Establish staging database Some data can be cleansed in new system Legacy mainframe application had 3 different databases: Drivers, Titles and Registrations. STARS system is customer centric Poor plan for data conversion The appropriate approach would have been to modify existing legacy mainframe system to make necessary connections well ahead of data conversion Allow sufficient time for data-clean-up Develop conversion rules; apply to mapping and provide error reports to business users for review 50 million records took 30 mths. to map/convert Final conversion took 75 hours; practiced many times / mock conversions Select correct resources to review error reports Command 10 utilized / transactions incomplete Complete transactions and cleanse data well in advance

9. Project Organizations New JerseyIndianaKansas Need dedicated project staff from ops and IT PM / contract mgmt infrastructure operational staff – define requirements & participate in testing training IT staff – must be dedicated and not pulled off project The right fit was to create our own Project Management Office (PMO) in 2006 following PMBOK standards The creation of the department caused a culture shift over time. At first only major software changes were treated as projects. Now every software change is treated in this manner. The entire Agency participates and commits resources for the entire project lifecycle. Separate project team created that included business units, training, communications, technical staff and project management

10. Organizational Change New JerseyIndianaKansas Set-up team to collect changes and prepare materials Informal sessions to introduce project / update periodically Utilize newsletters to provide project status Provide org change sessions for front-line & supervisory staff Plan for “change” training before new system training Pre-training sessions prior to formal training involve HR / Human Dev & Training groups Indiana had a very difficult time with this initially; system changes were viewed as an “IT System” Business users were not engaged in the testing process Business users were required to conduct regular job duties in addition to the new system design and testing. The organization must have strong leadership at the top Change agent network created early to communicate with stakeholders Changes communicated at monthly/bi-monthly meetings Sandbox created for users to practice transactions prior to go- live System users were required to take and pass instructor lead training before user credentials issued On-going training offered

11. Schedule Management New JerseyIndianaKansas Base schedule on deliverables / pay points Need combined milestone to management view Require weekly reports Real progress wasn’t made until BMV took over management of the project including managing the schedule. Have seen the same success with other projects (DDL, central issuance of plates and registrations, etc.). Senior management needs to see the major milestones but the project management and development teams need to be focused on the detailed deliverables Utilize MS project with detail level tasks that arrive at specific milestones Make sure detail level tasks are accurately described, estimated and reported against Both vendor and state must be honest about status of tasks Task completion rate will indicate if task information is accurate and ensure you are measuring against baseline task information

12. Scope Management New JerseyIndianaKansas Ensure all existing functionality available in new system Establish a change board & meet regularly Limit / stop changes to existing system until cost/impact defined twice the work / twice the cost Need a well defined process & tool to identify, assess & manage requests for changes Make prototyping/hands on part of requirements session Detailed design sessions with committed resources can reduce this type of issue. Even with that very strong leadership is a must to keep a focused scope. Scope can change due to new laws, work with legislature Include contract language that requires vendor to implement system compliant with federal law at the time of implementation For changes not driven by law, create change control processes; change control board must approve, deny or defer change requests Include SLA for delivery

13. Testing Strategies New JerseyIndianaKansas Team involved in defining requirements needs to be involved in testing / verification of those requirements Determine if current staff have skill sets to administer, manage and execute comprehensive UAT; if not outsource Pick tools the state team can use and will carry forward after project Pick best and brightest to write scripts/UAT Get executive buy-in You can’t start too early to identify and prepare for testing This is another change Indiana has made since the implementation. During the development / initial implementation of STARS, IN relied on Unisys to provide all system testing Since then a formal testing area has been created for the IT department. Indiana now requires formal user testing and signoff of all software changes. This is a much better approach as the end users see the system as their own and not an “IT System”. Create test plan & utilize test scripts Have multiple testers for same scenario Follow standard bug reporting protocol Complete regression testing for code changes Test reports Age test system Run mock batch jobs Don’t accept too many work arounds Allow testers time to adequately test system Categorize/prioritize bug fixes (DMV must set) Clear documentation / keep copies

14. Production Roll-Out New JerseyIndianaKansas Don’t do big bang of all functions Define data synchronization plan to support phased roll out Include significant production pilot time Set-up help desk during testing Establish “field team” to be in field for go live Define how you will time training and help staff to keep new skills May need temp staff or office closing Attempted big bang Branch piece was to run for a year before back office implementation / data conversion. The delayed schedule did not allow this to happen. The back office piece was to be in production by 7/04 but development had barely been started in 1/05 By correcting the system rather than starting over, in a much shorter time, the State was able to position itself for future successes. Utilized big bang approach Considered LE along with conversion of data Need ability to track transactions across one system County offices closed five days during the conversion; backlog built before offices opened for first day

15. Internal and External Communication New JerseyIndianaKansas Internal info sessions with all staff before project start newsletter updates senior management meeting updates External provide general project info to stakeholders hold info sessions with common groups periodic communication to stakeholders press releases legislative updates demonstrations to stakeholders Indiana did not have a communication plan for all stakeholders. External partner communication was particularly lacking. One of the major problems was with Law Enforcement. Indiana relied on Indiana State Police for this communication. Unknown to Indiana BMV at the time was the fact that ISP also changed their system at the same time. Different inquiry options in the ISP system caused confusion about BMV data being returned to ISP. External A list manager was created to push communications to specific groups for quick communication. A project website was created for communication & storage of historical newsletters & documents for future reference. Internal Project mtgs./emails Communications lead to handle public inquiries or notices.

16. Join us next time... Legalization of Marijuana - Highway Safety Implications Wednesday, October 16 2 p.m. – 3 p.m.

17. Panelist Contact Info Susan Matthews, NJ MVC – Susan.Matthews@dot.state.nj.us Susan.Matthews@dot.state.nj.us Deb Gerrow, Mathtech – dgerrow@mathtechinc.com dgerrow@mathtechinc.com Kent Schroder, IN BMV – kschroder@bmv.in.gov kschroder@bmv.in.gov Toni Roberts, KS DOR – Toni.Roberts@kdor.ks.gov Toni.Roberts@kdor.ks.gov Sheila Prior, AAMVA – sprior@aamva.org sprior@aamva.org

18. System Redesign Webinar The following slides contain additional details on lessons learned from system redesign projects in New Jersey, Indiana and Kansas.

19. Project Status – New Jersey The NJ Motor Vehicle Commission is in process of a complete modernization of all its primary computer systems including foundation technologies such as database, content management, rules engine, interface and print engines as well as all core functionality including: Driver Licensing, Driver (Event) History, Title & Registration, Business Licensing, Finance, and Case Management. The legacy system is a 30 year old mainframe/COBOL system with all of the typical flexibility, expandability, and maintenance problems. The State decided to engage an Oversight Vendor to assist with planning and management of this project. An RFP was issues and Mathtech was awarded a contract in 2006. An implementation RFP was developed and HP (Saber) was awarded the contract in January 2009.

20. Project Status, New Jersey, continued The project, called MATRX – Motor Vehicle Automated Transaction System, is planned to be implemented in three (3) phases. Phase 1 was infrastructure upgrades and the rollout of document scanning to all field offices – this phase is in production as of Jan 2011. Phase 2 is the implementation of Vehicle Title and Registration functions, Customer Manager, and Web Transaction Center. This phase is currently planned for rollout in Late Summer 2014 Phase 3 is the implementation of Driver and Business Licensing, Driver (Event) History, and Case Management. Final Schedule in negotiations. This will be a big bang implementation.

21. Project Status, New Jersey, continued Firm Fixed Price Contract with Saber (HP) -- 51 Deliverables - $36,850,167 Firm Fixed Price for Hardware/Software - $14,789,781 ( NJ maintained the option to purchase from HP or to use other NJ Hardware/Software contracts) Total $51,639,958. A Bond was issue to cover the costs of the Contract. Annual Operating funds are appropriated based on project needs for that particular fiscal year.

22. Project Status - Indiana Indiana awarded its system redesign to Unisys in 2000. The new STARS system involved 3 phases: – Phase 1 Upgrade legacy system to run in Windows environment Cost: $2,308,971.00 Timeline: August 2000 – December 2001 – Phase 2 Create and install STARS application in all branches Cost: $14,028,638.53 Planned Timeline: June 2001 – July 2003 Actual Timeline: June 2001 – October 2004 The branch piece as delivered was unusable but accepted by the state (the contract did not allow the state to reject) and payment was made in 2004

23. Project Status, Indiana, continued – Phase 3 Create and install STARS application in the home office Cost: $13,100,000.00 Planned Timeline: June 2002 – July 2004 Actual Timeline: January 2005 – July 2006 – Support Unisys Support Cost: $2,731,970.00 PC Replacement: $1,488,481.25 – Computers in branches purchased for STARS became obsolete before implementation Database Server Upgrade: $750,000.00

24. Project Status, Indiana, continued – Total Cost: $34,408,060.78 2000-2004: – Development Paid: $30,189,579.53 2005: – Development: $180,000.00 – Hardware Upgrades: $2,238,481.25 – Warranty and Support: $1,800,000.00

25. Project Status, Indiana, continued Indiana spent 10 months in 2005 correcting bugs in the branch code and rewriting poorly performing queries with in- house staff to make the system operable. This created difficulties for Unisys when completing the back office software. It is one system and there is shared code between the branch and back off pieces. Unisys had to merge the fixes made by the State into their code base while completing the work for the back office. The result was reintroduction of bugs into the branch piece. Indiana had no Database Administrator (DBA) on staff to approve system design, review code and monitor performance before 2005. The BMV has had two DBAs on staff since 2005. They continue to review and approve queries and stored procedures and monitor performance today.

26. Project Status, Indiana, continued Once the system was fully delivered the BMV worked with in- house staff to correct problems in the code for the remainder of 2006 (here again, the contract did not allow for rejection and full payment was made). Beginning in 2007, the BMV began rewriting many pieces of the system. By the end of 2008 the code for Registrations, Suspensions, Reinstatement and Finance were redesigned and written by BMV staff. Only about 20% of the delivered product still exists.

27. Project Status - Kansas The Kansas DMV Project contract was finalized and a letter of intent was signed with 3M on July 1, 2009 to purchase and install a new titles & registration and driver license system. On May 7, 2012 the title and registration component (MOVRS) was implemented statewide. The (DRIVS) System planning is underway.

28. Planning – NJ/Mathtech New Jersey’s RFP for the modernization was a complete revamp of all systems. If we had to do it all over again we would break it up into smaller projects. The project included re writing existing automated systems and business operations that were primarily manual. These manual business operations created the greatest challenge for requirements gathering and were high risk for scope creep. Prior to any requirement gathering activity we would recommend business process reviews and re-establishment of goals and objectives for the business operation. The Project needs an Executive Sponsor - to champion your project across state agencies, run interference, set priorities – it should be at the highest Level in the organization.

29. Planning, NJ/Mathtech, continued Establish a governance teams with representatives from the top three levels of the organization that: Recommend/Approve Project Management decisions regarding changes to schedule, scope and budget Recommend/Approve decisions relating to Policy or Operational procedure changes Review request for IT services that may impact on the project or on IT resources dedicated to MATRX Address issues, risks and change orders Review Request for IT services outside of MATRX or other MVC major IT projects

30. Planning, NJ/Mathtech, continued Ensure you have the right technical skills and the adequate number of staff to implement the project The vendor will be making assumptions as to the skill levels and number of staff available to participate in the project. Clarify the limitations in the RFP. Take stock of the capacity and life expectancies of your architecture, servers, network, wireless services, security, operating systems, peripherals, what other software programs (DDL, Financials, Surcharge systems) that are dependent on your existing system.

31. Planning, NJ/Mathtech, continued Ensure you have documentation of your rules and business processes – the more obscure the rule the greater chance it will be missed Capture what improvements (policy changes, process changes) you want the new system to have – implement an approval process to document these improvements so that they are included in the requirements documentation

32. Planning, NJ/Mathtech, continued If this is a large scale project and if there is a requirement is to have vendor staff at site to do development ensure that you plan for adequate space, conference rooms, AV equipment etc. If you are implementing a new report writer – review current reports eliminate what is no longer necessary If you are implementing a new print engine – review current notices/correspondence – eliminate unnecessary communication, update/correct notices.

33. Planning, NJMVC Define Project Vision upfront – define it at an exec level and get it approved – will drive the other planning and activities Get an Executive Sponsor - to champion your project across state agencies, run interference, set priorities – it should be an Operational person – while these projects are “technical” – Operations must own it Setup governance – define criteria for decision making/approvals for each level You don’t have enough staff; Figure out how to backfill/prep back up staff now - you need good people to support the project to get a good result - need staff from operations and IT.

34. Planning, NJMVC, continued Limit/Stop changes on existing system – it’s hard to build a new system when still changing the current system – get Legislature/Governor on notice/on board Define what you have now – what are the rules, processes? What side systems are people using? Print screens from current systems to use as a guide for making sure all requirements are identified Develop a technical training plan for your IT staff if you are planning on new technologies – get them prepared ahead of the project (i.e establish some base skills – do a small “practice” project) Define your implementation strategy – what will you do vs what you want the vendor to do (ex: training, data conversion etc)

35. Planning - Indiana Indiana left the planning to Unisys. No project management existed within the BMV and no external project management was contracted. Not having this skill set put Indiana at a distinct disadvantage and unable to challenge any Unisys plans. Clearly, without internal expertise, Indiana should have hired a project management team independent of Unisys to ensure success of the project. It should have started at RFP and continued through the contracting, development and implementation phases. Indiana now has a Project Management Office and a Procurement and Contracting team.

36. Planning - Kansas Complete a Feasibility Study to document your current business requirements, processes, technical environment and stakeholders. The Feasibility Study compares various solutions and obtains marketplace information and allowed us to arrive at a recommended solution along with an estimated cost range for replacement. Funding was obtained through legislation “Vehicles Modernization Fund” which applied a $4.00 surcharge on each vehicle being registered.

37. Planning, Kansas, continued The procurement process was a project all by itself. The Request for Proposal (RFP) had to be developed which included both functional and technical requirements. Include the right people for evaluation of the bids and bring the vendors in and ask questions. – Be sure to include all stakeholders – Allow plenty of time to develop your requirements – Obtain sign-off from those stakeholders in order to obtain buy-in – Don’t use generic terms or language that can be left for interpretation, BE SPECIFIC – List all interfaces by name describe how they function and what they touch – Document everything –

38. Planning, Kansas, continued Once the vendor is selected, a Project Charter is key and must contain: The Project Overview (description, vision, objective) – Scope (Description of Work) – Project Organization (organizational chart /roles and responsibilities), – Governance (Executive Sponsors and Steering Committee) – Risk, Assumptions and Constraints – Communication (website, e-mail, newsletter)

39. Planning, Kansas, continued Miscellaneous – When detailing out the workplan take your time or you will pay for it 5 times over later – Include all the key project staff in the planning – Obtain realistic estimates for the work to be completed – Don’t accept “generic tasks” such as Deliver Code to UAT (what functionality is specifically being delivered?) – Make sure you and your staff understand the workplan and are included in the development process, don’t allow your vendor to take this over for you – Make sure the workplan is not a payable deliverable

40. Data Clean Up – NJ/Mathtech Start early. Think of data that should be cleaned up – addresses, VIN numbers, social security numbers etc.

41. Data Clean-Up, NJMVC Can’t start early enough - Start data planning before project – id problems and do cleanup in legacy system Establish a staging database to help with data cleansing/validation Some data could be cleansed as it is processed in the new system – depends on volume and processing impact

42. Data Clean Up - Indiana Indiana’s legacy mainframe application had 3 different databases: Drivers, Titles and Registrations. The STARS system is customer centric with customer IDs on each record. The Mainframe Titles and Registrations databases did not have clean data to convert to the new STARS system. Indiana had a poor plan for data conversion. The original idea was to run the branch piece for a year which would include a cycle of vehicle registrations and connecting customer records to vehicles. This did not address Titles at all and did not account for customers who did not renew on time or at all.

43. Data Clean Up, Indiana, continued As a result, 7% of Registrations and 10% of Titles did not have a customer connected to them at conversion. The missing Registration records caused production issues after conversion. Special programming was created to allow the branch associate to connect Registrations to customers. The appropriate approach would have been to modify the existing legacy mainframe system to make these connections well ahead of the data conversion.

44. Data Clean Up - Kansas The data clean-up process started 18 months prior to the start of the project by collecting the DL numbers when completing a title or registration transaction. The legacy data was cleansed by in house staff. There is nobody that knows your data better. Kansas did combine the data clean-up effort as part of the roll out of the new system. The techniques used for identifying bad data included developing conversion rules, the rules were then applied to the mapping and resulted in error reports. The error reports were provided back out to the business users for review. Once reviewed correction or new rules were established and implemented.

45. Data Clean Up, Kansas, continued The entire legacy DB contained 50 million records which took approximately 30 months to map, create conversion scripts and test the conversion process. The final conversion process took 75 hours which was practiced many times with a mock conversion process. Lesson learned - Make sure you select the correct resources when reviewing the error reports. Many records that were businesses were converted to individuals. Command 10 (mainframe force) was utilized within our County offices and left the transaction or record in a pending state as the required documentation or update of the transaction was not complete. Lesson learned – Complete Transactions and cleanse your data well in advance.

46. Procurement & Contracts, Kansas A deliverable based contract was used in Kansas. A documented change request process was established for any functionality that was deemed out of scope. Penalties and liquated damages were included.

47. Procurement & Contracts, Indiana Indiana had no internal expertise at RFPs or Contracts. The Legal Department lacked effective leadership. As a result, the Unisys contact had no performance incentives or penalties. All Unisys had to do was deliver code and after 30 days, if not accepted, was assumed accepted. If rejected, the clock could not be reset and the 30 day window expired and was deemed accepted. Payments were required by the State.

48. Project Organization – NJ/Mathtech Project Management/Contract management – need authority to manage state staff and speak to the contractual requirements also need to serve as the communication link for the project with Operation/Administrative staff and visa versa. Operational staff dedicated to project to define requirements and participate in testing Information Technology staff dedicated to project so that they are not pulled off project and given other unrelated projects

49. Project Organization, NJMVC Need dedicated project staff from operations and IT (I’ve seen anywhere from 5 – 15); big areas that should be covered: – PM/Contract management – need authority to manage state staff and speak to the contractual requirements – Infrastructure – need to understand and monitor standards and regular upgrades – Operational staff - defining requirements, testing/validation – Training – need to support non system training required (ex: policy/procedure changes) and be prepared to take over eventually – IT – need to support architecture, data conversion, testing, sysadmin, reporting, rules, design/code review etc…

50. Project Organization - Indiana For Indiana the right fit was to create its own Project Management Office (PMO) in 2006 following PMBOK standards. The creation of the department caused a culture shift over time. At first only major software changes were treated as projects. Now every software change is treated in this manner. The entire Agency participates and commits resources for the entire project lifecycle.

51. Project Organization - Kansas In Kansas, a separate project team was created that included, business unit representation, training staff, communication staff, technical staff and Project Management.

52. Organizational Change – NJ/Mathtech Introduction to the Project through informational sessions – continue periodic updates Departmental newsletters providing status of project Interview with staff who are participating in Requirement gathering sessions, testing etc. Depending on how large the project and impact on staff provide organization change sessions for front line staff and supervisory staff Prior to formal training – Pre-training sessions which include screen shots, what will be new, what won’t be there, what the formal training will include etc.

53. Organizational Change, NJMVC Setup a team to collect these changes and prep materials Plan for ‘change training” before new system training – involve HR/Human Dev/Training groups

54. Organizational Change - Indiana Indiana had a very difficult time with this initially. The system changes were viewed by the organization as an “IT System” despite the fact that IT would never actually use it. Business users were not engaged in the testing process. Business users were required to conduct their regular job duties in addition to participate in the new system design and testing. Design sessions frequently became make the new system work like the old system. Users were more concerned about getting back to their desks and completing their day to day tasks than designing a system that met their needs and was an improvement on old processes. The organization must have strong leadership at the top.

55. Organizational Change - Kansas Early on a Change Agent Network was created to communicate with our Stakeholders to build rapport. As changes were identified, they were communicated out at bi- monthly and then monthly meeting. A “Sandbox” environment was created for our business users to practice in prior to go-live so the changes they would be expected to make wouldn’t be a surprise. System users were required to take and pass instructor led training before user credentials were issued. On-going training is offered as a refresher or for new system users.

56. Contract Management, Indiana Unfortunately, the contract had been nearly executed completely before an administration change occurred.

57. Contract Management, Kansas Contracts with these types of endeavors are very large and are very difficult to insure the vendor is held accountable to fulfilling their obligations. Create a checklist for compliance. Make sure the vendor lists their key resources and require that you are notified when there is a change and you approve the replacement – take this down to the lowest level possible. (Many hours of experience can be lost.) Deliverable based contracts are easier to manage for payments. A Statement of Work document is key for defining responsibilities and expectations.

58. Schedule Management – NJMVC Based on deliverables/pay points – drives focus Need combined milestone to mgmt. view – each team needs the details Need weekly reports – late tasks, milestone status/changes, upcoming tasks for next several weeks, high level timeline

59. Schedule Management - Indiana For Indiana, real progress wasn’t made until it took over management of the project including managing the schedule. It has seen the same success with other projects (DDL, central issuance of plates and registrations, etc.). Senior management needs to see the major milestones but the project management and development teams need to be focused on the detailed deliverables.

60. Schedule Management - Kansas The best approach to manage your schedule is utilizing MS project with detail level tasks that arrive at specific milestones. The key is to make sure your detail level tasks are accurately described, estimated and reported against. Both the State and Vendor need to be honest about tasks completed or tasks incompleted. The Task completion rate will tell you alot if the task information is accurate and you are measuring against BASELINE TASK INFORMATION.

61. Scope Management – NJ/Mathtech Keep it simple – get it done. Ensure that all existing functionality is available in the new system; try to keep enhancements aligned with your primary functions. If you are building a refrigerator don’t expect it to microwave your frozen bagel. Ask the question “Do we absolutely need this day 1 of production? Limit/Stop changes on existing system – it’s hard to build a new system when still changing the current system – get Legislature/Governor on notice/on board. If you build it in your old system – you will have to build it in your new system. Twice the work, twice the cost.

62. Scope Management, NJMVC Need to establish a change board (operations, IT, project staff) and meet regularly Any change to the existing/legacy system should not be approved until impact/cost on modernization project is defined – all approved together Need a well- defined process and tool to identify, assess and manage requests for changes – include roles & responsibilities for all parties Make prototyping/hands on part of requirements sessions – much clearer than a paper document Assess the skill sets of your IT team to see if they can maintain the system, what tier level can you maintain, identify the service gaps and what identify what needs to be outsourced

63. Scope Management - Indiana Detailed design sessions with committed resources can reduce this type of issue. Even with that very strong leadership is a must to keep a focused scope.

64. Scope Management - Kansas Your scope can change simply due to new laws that are passed and go into effect. Work with your legislature the best you can. Sometimes it’s unavoidable, you have to update the mainframe system in order to be in compliance with the law and you must implement the change within the new system being built. Include a within your contract some language that requires the vendor to implement the system that is compliant with Federal law at the time of implementation. In addition, for changes that are not driven by law create a change control process and board that has the authority to approve, deny or defer change requests. Make sure your change control documentation includes a service level agreement that includes a timeline for delivery and any resulting consequences for non-delivery.

65. Performance & Vendor Management, Kansas Document everything, assign the correct staff and make the vendor accountable for delivery or non-delivery but be sure to have the correct documentation to back it up. Make sure and include legal staff in the writing of the contract and specifically surrounding non-conformance and define what that is. Define system acceptance upfront well in advance.

66. Performance & Vendor Management, Indiana This is exactly what Indiana faced. Lack of project management expertise and large project expertise overall led to a poorly designed and implemented system. There was a belief within the Indiana BMV that Unisys was the “expert” at these types of implementations and as a result their plans were rarely challenged. Unisys developers were offsite for most of the project. In the second half of 2005 many of those developers were required to be onsite. Management became much easier and productivity increased after the move. When the project was awarded to Unisys they lacked resources with sufficient.Net knowledge. The result was less optimized source code.

67. Testing Strategies – NJ/Mathtech Assess if your current staff have the skill sets to administer, manage and execute a comprehensive User Testing Acceptance. If not outsource. Secure the best and the brightest of the operational staff to participate in script writing and UAT execution Gain Executive commitment to providing staff for this endeavor.

68. Testing Strategies, NJMVC The team involved in defining the requirements needs to be involved in testing/verification of those requirements Pick tools the state team can use and would want to carry forward after the project You can’t start too early to identify and prep for testing

69. Testing Strategies - Indiana This is another change Indiana has made since the implementation. During the development and initial implementation of STARS, Indiana relied on Unisys to provide all system testing. Since then a formal testing area has been created for the Indiana IT department. Indiana now requires formal user testing and signoff of all software changes. This is a much better approach as the end users see the system as their own and not an “IT System”.

70. Testing Strategies - Kansas Creating a test plan and using test scripts as a guide for our testing efforts for system functionality is key. This is a way to organize and document what is to be tested, what has been tested, who tested it and what the results were. Creating test plans for changes and enhancements in the same manner. Having multiple testers test the same scenarios helps in confirming results. When issues/bugs are found, they are reported through the same standard protocol by every tester. Reported bugs are communicated to team members so that everyone understands the issue at hand. If further clarification is needed for any reason (amongst our team or our vendor) meetings are scheduled to discuss more in depth.

71. Testing Strategies, Kansas, continued Regression testing of system functionality is necessary anytime there is a change to the code set to ensure new issues were not created from bug fixes, changes, or enhancements. Test the reports you plan on using Age test the system for specific correspondence or events Run mock batch jobs for title printing and renewal notice mailers Don’t accept too many work-arounds that impact your end users Allow your testers the time necessary to test the system adequately – DON’T rush the process

72. Testing Strategies, Kansas, continued Include your outside parties (interfaces/end users) in the testing process Categorize/prioritize the bugs you find appropriately and don’t allow the vendor to set those categories or priorities for you. Be clear with your documentation and keep your own copies.

73. Production Roll-Out – NJ/Mathtech Don’t do Big Bang of all functions Big Bang is a major risk and too much change to support all at once Define a data synchronization plan to support a phased rollout Include significant production pilot time Setup Helpdesk during testing – good practice for them Establish “Field Teams” to be onsite at field offices for go live – involve them in testing so they are well prepared Define how you will time training and help staff keep new skills (practice system available from field sites? Just in time training?) You may need temporary staff or plan to close offices depending on the timing of training and the rollout schedule.

74. Production Roll-Out, NJMVC Don’t do Big Bang of all functions (DL, VTR, Driver History, Business Licensing/dealers etc…) Big risk and too much change to support all at once Define a data synchronization plan to support a phased rollout Include significant production pilot time Setup Helpdesk during testing – good practice for them Establish “Field Teams” to be onsite at field offices for go live – involve them in testing so they are well prepared Define how you will time training and help staff keep new skills (practice system available from field sites? Just in time training?) You may need temporary staff or plan to close offices depending on the timing of training and the rollout schedule.

75. Production Roll-Out - Indiana Indiana attempted a combo phased/big bang approach. The phases were split between branch and back office. However, the modules for each were installed at the same time. The original timeline called for the branch piece to run in the branches for a year before implementation of the back office piece and data conversion. The delayed schedule did not allow this to happen. Indiana was rolling out the branch piece while making bug fixes and tuning database queries. At the same time Unisys was writing the back office piece of the system. Unisys attempted to merge these changes into the back office piece. However, with a shortened timeline old and new bugs were introduced at final conversion in July of 2006.

76. Production Roll-Out, Indiana, continued Keep in mind that the back office piece was supposed to be in production by July 2004 but development had barely been started January of 2005. The money had already been spent and the decision was made to salvage the code rather than starting over. There was no way contractually to get the money back from Unisys. By correcting the system rather than starting over, in a much shorter time, the State was able to position itself for future successes.

77. Production Roll-Out - Kansas Kansas used a big bang approach for the title and registration system as there was no way to implement a partial Title and Registration System. Law enforcement had to be considered along with the conversion of data. In addition, we had to have the ability to track transactions across one system. Our county offices were closed (5 days) during the conversion process so a backlog of work built up before we even opened the doors for the first day.

78. Communication – New Jersey Internal Communication Information sessions regarding the project before it starts should be given to all staff Newsletters with updates, interview with staff involved in project should be issued periodically Periodic updates at Senior Management meetings

79. Communication, New Jersey, continued External Communication Provide general project information to stakeholders, what will be new and improved, what will not change Hold informational sessions with common groups; law enforcement, third party vendors, other state agencies to provide status of project when necessary to address specific impacts (new access, training etc). Periodic communication to stakeholders Press releases Legislative updates Demonstrations to stakeholders

80. Communication - Indiana Indiana did not have a communication plan for all stakeholders. External partner communication was particularly lacking. One of the major problems was with Law Enforcement. It turned out that many local Law Enforcement departments were not aware of the changes. Indiana relied on Indiana State Police for this communication. Unknown to Indiana BMV at the time was the fact that Indiana State Police (ISP) also changed their system at the same time. Different inquiry options in the ISP system caused confusion about BMV data being returned to ISP.

81. Communication - Kansas External Communication The list manager was created to push communications to specific groups for quick communication. The website was created more for pull communications and storage of historical newsletters and documents for reference. Internal Communication – In person project meetings & e-mails – Assign a Communications lead to handle public inquiries or notices.