Presentation on theme: "1 Session 3: Benefits Definition and Measurement v4.0 May 2009 eHealth Benefits Management Toolkit."— Presentation transcript:
1 Session 3: Benefits Definition and Measurement v4.0 May 2009 eHealth Benefits Management Toolkit
Benefits Management 2 Session 3 Outline Benefits measurem ent categorisat ion Benefits profile template Benefits Realisation Plan Transition to business as usual
Benefits Management 3 At the end of the session you will have an understanding of: what methods can be used to define quantifiable benefits and estimate their value; how to finalise detailed benefit definitions using SMARTT and ensure the benefits are measurable; how to pull all the outputs of the previous sessions together to create the Benefits Realisation Plan; and what further activities need to take place at the end of the project. Session 3 purpose
Benefits Management 4 Benefits Measurement Categorisation Benefits measurem ent categorisat ion Benefits profile template Benefits Realisation Plan Transition to business as usual
Benefits Management 5 Specific Definitions of Benefits Benefits should be: clearly and succinctly defined aligned with the projects objectives and the overall NHS strategy focused on improving service and patient care, not focused on IT Effective measurement and monitoring will: indicate the extent to which benefits are being realised; give early warning of potential problems; create the opportunity to adapt the benefits or changes to enable the overall objectives; and ensure that achieved benefits are measured, reported and communicated.
Benefits Management 6 Measurable Outcomes The next step is to: devise measures that will show when the benefits have been delivered and to what degree; determine which benefits can be quantified; determine the financial value of benefits where possible; investigate additional data capture activities required; and establish baseline, targets and benchmarking figures.
Benefits Management 7 Defining Measures Use existing measures where possible Agree all new measures with those concerned Ask yourself: –Is it achievable? –Is it an appropriate and valid indication that the benefit has been delivered? –Is it directly attributable to a particular change in the project? Consider the effort and cost of capturing and collating the data Decide who will be responsible for ensuring that the data is available Ensure the measures encourage the right type of behaviour Define when and how often the measurements should be taken
Benefits Management 8 Benefits Categorisation (JW) Degree of explicitness of measures Financial By applying a cost, price or other valid financial formula to a quantifiable benefit, a financial value can be calculated. Quantifiable Sufficient evidence exists to forecast how much improvement / benefit should result from the changes. Measurable This aspect of performance is currently being measured or an appropriate measure could be implemented. But it is not possible to estimate by how much performance will improve. Observable By use of agreed criteria, specific individuals or groups will decide to what extent the benefits has been realised, based on their experience and judgement.
Benefits Management 9 Non Quantifiable Benefits Observable benefits lowest category of benefits in term of robust measuring often the softer / people benefits can help get buy-in essential to the implementation of changes and the realisation of their resulting benefits agree realisation criteria and who will say whether they are met Measurable benefits performance is easily measured; but potential improvement won’t be known until after the event take baseline measurements before any changes are made for comparison
Benefits Management 10 Quantifiable Benefits Quantifiable benefits aspect of performance easily and consistently measured improvement can be predicted and targets defined usually applicable to embedded processes where performance is well documented and predictable Financial benefits highest category of benefits in term of robust measuring focus on benefits that can easily be translated in terms of financial gain or reduced cost: if it is too complicated or not robust don’t assign a monetary value to it don’t confuse value with savings
Benefits Management 11 Progressing Non Quantifiable Measures to Quantifiable Measures The following methods can be used to get from measurable to quantifiable measures: Modelling or simulation: extrapolate expected benefits from existing data. It may be necessary to establish trends/times series. Benchmarking: evaluate changes in relation to “best practices” in the comparable organisations. References sites: can also be used to establish a comparative base but assess relevance and feasibility in your own organisation. Pilots: test the new way of working and system on a small scale.
Benefits Management 12 Examples for Benefits Template Category Benefit Descriptions and NumbersMeasure Value / Improvem ent Level Financial e.g. “increase utilisation of theatres resources by allowing patient-cancelled appointments to be filled at short notice from reserve list” expected increase in utilisation = 1 in 3 cancellations filled = 200 slots = £1000 per slot = £200,000 cancellations filled x resource value (£1,000) £200,000 Quantifiable e.g. “elimination of referrals from SCI gateway being printed onto paper for transfer to secondary care system” number of referrals, and therefore those being printed, known before implementation; integration will eliminate these % of referrals processed through new system 100% Measurable e.g. “reduce unnecessary x-rays by avoiding repeat request” not known how many x-ray requests are repeats, but assumption is that these will be eliminated, reducing overall number of x-ray requests number of x- ray requests N/A Observable e.g. “reduce staff frustration due to need for double data entry” staff satisfaction survey with regard to duplicate data entry % of satisfied or very satisfied N/A
Benefits Management 13 Benefits Profile Template Benefits measurem ent categorisat ion Benefits profile template Benefits Realisation Plan Transition to business as usual
Benefits Management 14 SMARTT Benefits Detailed benefit descriptions contain the following categories: Benefit Characteristic DefinitionExample Specific Description of the benefitsNumber of re-xray forms completed. Quality dimensions enabledSafety, efficiency, timeliness MeasureableVariable to be evaluatedNumber of forms filled in per month Achieveable How high a priority is the benefit? Scored on a scale of 1-5 (where 5 is the highest priority) How likely is the benefit to be realised? Scored on a scale of 1-5 (where 5 is the highest likelihood) Realistic What is the benefit dependent on? Strategic direction, process, information management, skills/ behaviours, organisation/ roles, infrastructure/ physical environment, technology integration Time-bound Realisation timeframe E.G. Measure benefit from 3 months pre-go-live until 1 year post go-live Measurement interval recommendation E.G. Per minute, per hour, per week, etc. Frequency of measurements E.G. One week out of each month, one month out of each year, etc. Targeted BenchmarkE.G. 20 forms/month TargetE.G. 20% reduction
Benefits Management 15 Filling in the Benefit Profiles ACTION: Expand on the benefits information from the previous workshops in the template for each priority benefit identified: Utilise measurements that are already being used by the Health Board where possible Take into account pre-existing government targets of efficiency estimates when setting benchmarks and targets for benefits Ensure that dependencies are taken into account when setting realisation timelines Benefit ID Enter ID number from BDN Benefit description Enter benefit description Benefit owner Enter the name of the person or group who will receive the benefit Benefit sponsor Enter name and role of the person who will ensure that the benefit is achieved and instigate remedial actions if required Safe Tick if relevant Effective Tick if relevant Timely Tick if relevant Efficient Tick if relevant Patient-centred Tick if relevant Equitable Tick if relevant Measure/Variable to be evaluated Enter description of measure to be collected Benefit explicitness Enter category: financial (high), quantifiable, measurable, observable (low) Value/Level of improvement Only applicable for quantifiable and financial benefits Impact How important is the benefit? Scale of 1 (low) to 5 (high) Likelihood How likely is the benefit to be realised? 1 (low) to 5 (high) Strategic direction Enter ID and description of objective(s) the benefit contributes to Changes required Enter ID and owner of required changes (see change profile for details) IM&T required Enter ID and description of required IM&T capabilities Benefit realisation date Enter date the benefit will be realised (or dates and degree of realisation for staged roll-out) Measurement timeframe Enter start and end dates of monitoring and reporting Measure interval Enter description of the period of time the measurement covers Baseline Enter value, source and date if applicable Benchmark or target Enter value, source and date if applicable
Benefits Management 16 Preparing to Measure Are there pre-set requirements for benefits realisation timelines? Who is going to be collecting the measurements? How will the measurement be collected? (data sources) Who will oversee the benefits collection? Where will the results be reported to? Who will conduct benefits reviews? Who will be responsible for communicating about the benefits side of the project?
Benefits Management 17 Preparing for Benefits Monitoring In order that the benefits monitoring is as productive as possible: define a robust mechanism to monitor progress and ensure that reports are distributed to the appropriate people; ensure that the Benefits Management templates are up to date; hold regular benefits reviews to track the progress of benefits realisation against the project and programme objectives; use benefits reviews as an opportunity to highlight areas where benefits are not being achieved, and identify the barriers to achieving them; for those benefits that are under-realised, refer to the dependencies shown in the BDN for potential root causes; assign responsibilities for taking and coordinating corrective actions where benefits are not being met; and identify and communicate successes too!
Benefits Management 18 Benefits Realisation Plan Benefits measurem ent categorisat ion Benefits profile template Benefits Realisation Plan Transition to business as usual
Benefits Management 19 Formalising it The Benefit Realisation Plan provides evidence to: Justify investment decisions Prioritise competing work in context of finite resources Establish clear ownership / governance arrangements Realise benefits Demonstrate value and effective use of resources Store the Benefit Realisation Plan with the other key project management work.
Benefits Management 20 Benefits Realisation Plan (BRP) When drafting the Benefit Realisation Plan remember to: Consider the dependencies identified between benefits, changes and IS&T capabilities; Identify timeline and milestones that take those dependencies into account; Work with stakeholders towards minimising the negative impact of identified disbenefits; and Extend the plan beyond the system implementation date (often viewed as the end of the project) to schedule regular reviews of benefits realised and consider potential further benefits.
Benefits Management 21 BRP Content The Benefit Realisation Plan should include: the completed Benefits Dependency Network (BDN) the completed benefit profiles, including dates for capture of baseline figures at an early stage the completed change templates the completed stakeholders analysis Prioritise the benefits so that the most important always has the most focus. This ensures that the project makes the greatest impact. Identify dates for expected delivery of the benefits.
Benefits Management 22 Benefits Strategy A benefits strategy should also be prepared for the business case, including: the rationale linking the drivers, objectives and benefits defined roles and responsibilities a mechanism for regular measurements and reporting a mechanism for regular reviews and corrective action arrangements for the transfer of responsibilities at close of project
Benefits Management 23 Providing all this, the business case will demonstrate that: benefits outweigh disbenefits; a shared vision is held which is strategically aligned; the BDN clearly shows how the shared vision will be achieved; the change required, when considered in relation to the overall schedule of change planned, is not overwhelming for any stakeholders; and, the benefits realised will be worth the effort required to achieve them. It will also show that: there is a clear process for identifying, monitoring and realising the benefits; and the baseline benefits position has been recorded to enable comparison with projected targets for monitoring the achievement of benefits. Linking Business Case & Benefits Management
Benefits Management 24 Transition to Business as Usual Benefits measurem ent categorisat ion Benefits profile template Benefits Realisation Plan Transition to business as usual
Benefits Management 25 Close of project activities Following full implementation of information systems, the achievement of the business case should be reviewed. The review will produce 3 reports: formal end-of-project review lessons learned potential further benefits
Benefits Management 26 End of Project Review The end of project review will: assess to what extend the project was successful in delivering the benefits; decide what further actions are required, by whom and when they should be reviewed; and detail handover arrangements for any tasks outstanding (including benefits monitoring and reporting). The Benefits Plan should be updated as relevant to ensure a reference document is available to handover to the “business as usual” team.
Benefits Management 27 Lessons Learned The purpose of the “lessons learned report” is to: reflect on the lessons learned for future investments; disseminate useful lessons learned during the project for the benefit of other projects, internal and external; provide a useful control as part of the functions of an independent quality assurance group; consider how much effort was required to complete the benefits management documentation and monitor the benefits; and make recommendations for future enhancement or modification of the benefits management method.
Benefits Management 28 Further Benefits The Review Team should also consider and communicate: what opportunities exist for the identification and realisation of additional benefits; and who will take the business case for them forward. Additional benefits should be defined, planned and implemented following the same methodology as the initial project benefits.
Benefits Management 29 Transition to Business As Usual Leave no loose ends: each outstanding activity needs to be allocated to a named individual Have end dates for decommissioning of legacy systems to ensure the new IM&T are used – this will ensure new processes are used too and related benefits are delivered Monitor the implementation of change over a period: –people may revert to the “old ways” once the initial enthusiasm has worn off –people may find work-arounds where the technology or new processes are not working instead of reporting the problems –address teething problems
Benefits Management 30 Summary of Benefits Management Lifecycle Project lifecycleBenefits management lifecycleOutputs Project initiation Identify and structure benefitsBenefits Dependency Network (BDN) Identify change activities required to realize the benefits Prepare benefit plan and business caseBenefit profiles Change profiles Stakeholder analysis Run first benefits analysisBaseline data Delivery management Use benefit plan to inform project management, system development and change management activities Regularly review project progress against benefits analysis Updated figures Make adjustment to system capabilities, business changes or benefit definitions as required Amended templates and BDN Close Run end of project benefits analysis (formal review) Formal report to Board Business as usual Identify need for further reviews and potential for further benefits
Benefits Management 31 References and Further Reading The Clinical Indicators Team has produced a presentation on statistical process control (SPC): HM Treasury’s “Green Book” provides examples of methods used to estimate the financial value of activities or resources: No Delays Scotland have devised a methodology for measuring benefits: s/IT211_Methodology_for_measuring_benefits.aspx Statistics for NHS England and Wales for benchmarking on: (cont’d)
Benefits Management 32 References and Further Reading(cont’d) Benchmark statistics (Scotland) are available from ISD: Quarterly and annual figures on patient safety are published by NPSA: reports/ The Department of Health publishes annual and quarterly healthcare performance statistics: ataandstatistics/index.htm Population numbers by NHS Boards – births, deaths, totals, by age, by gender – are available from GRO: estimates/index.html
Benefits Management 33 Congratulations on completing the benefits management programme and on all your efforts to ensure that maximum benefit is gained from eHealth Systems.