1. Using Performance Information for Management and Budgeting - Canada’s Experience
Lee McCormack, Executive Director
Results-based Management
OECD Presentation, May 3, 2006

2. Overview The Canadian context
Problems, priorities and implementation challenges
Lessons learned

3. The Canadian context

4. 32 million citizens in 5 ½ time zones
Canada is a big country with relatively few people and a highly decentralized government
32 million citizens in 5 ½ time zones
Decentralized federation with 10 provinces and 3 territories
Provinces are equal to the federal government and have real power
Hundreds of federal-provincial agreements and federal programming is weighted towards transfer payments
Over 90 departments and agencies in the federal government
46 Crown corporations
public servants
Eight consecutive federal surpluses
New Conservative minority government

5. The majority of federal expenditures are transfer payments…
Total net expenditures in were $172.8 B, of which 56% ($96.8B) were transfer payments (including major transfers & grants and contributions)
Total Net Expenditures: $172.8 B
Grants and Contributions: $22.5B
Transfer payments

6. The federal expenditure management system is also decentralized
Element
Responsibility
Developing an annual Budget and a multi-year fiscal framework
Department of Finance
Establishing annual departmental reference levels
Treasury Board Secretariat and departments – approved by Treasury Board
New policy or program approval phase
Privy Council Office supporting Cabinet
Allocating and reallocating to ensure alignment with priorities and aggregate expenditure control
Departments routinely – plus three central agencies in special cases
Seek Parliamentary approval of spending plans
Treasury Board Secretariat
Reporting to Parliament on spending plans, actual expenditures, and results achieved
Treasury Board Secretariat and Departments

7. Cabinet Review of Priorities Cabinet Review of Budget Strategy
The Finance department sets the Fiscal Framework and manages the annual budget process
Budget Speech
Budget Bill
Main Estimates
Final Decisions on Budget
Cabinet Review of Priorities
February
Focus on the “increment”
No formal mechanism requiring incorporation of performance information into Budget process
Not many targets
Jan-Feb
Aug-Sept
Cabinet Review of Budget Strategy
Release of Budget Consultation Papers (Fall Update)
Budget Consultation Process
Dec-Jan
October
Oct-Dec

8. Treasury Board Secretariat monitors the ongoing “A-Base” and seeks Parliamentary supply
Main Estimates
Interim Supply
A-Base Perspective Input to Budget
Supplementary Estimates and Supply Bill
February
Jan-Feb
Feb-Mar
Substantial planning and performance information given to Parliament
Much performance information available, especially on G and C programs
Room to develop more systematic cyclical review
Pressure Management Dept Baseline Funding
Reports on Plans and Priorities
Full Supply Bill
Oct-Jan
March-June
Supplementary Estimates and Supply Bill
A-Base Perspective Input to New Policy Proposals
Departmental Performance Reports, Canada’s Performance and Public Accounts
Nov-Dec
Aug-Nov
Aug-Nov

9. All major departments and agencies
Canada has a relatively mature performance measurement and reporting infrastructure
All major departments and agencies
Have internal audit and evaluation units
Publish their audits and evaluations
Produce and explain to Parliament upon request their reports on plans and priorities and departmental performance reports
Are developing outcome frameworks that are consistently based across government
Treasury Board Secretariat
Reviews program and management performance, focusing primarily on G and C programs
Has a strong policy centre for results based management
Produces an annual performance report – Canada’s Performance
Assesses management performance across government, including results-based management capacity
Office of the Auditor General
Produces performance audits of departments and agencies

10. Although there is long results management history …
Management, Resources and Results Structures – regaining detailed program level knowledge
Possible Federal Accountability Act 2006 – cyclical evaluation of all transfer payment programs
Improved Reporting to Parliament – moving to whole of government planning and reporting
Management Accountability Framework – assessing management performance across government
Canada’s Performance – linking programs to societal performance
Results for Canadians – results-based management as a stated priority of government
Transfer Payment Policy – performance frameworks and evaluations reviewed by TBS
Modern Comptrollership – investment in management practices and controls
Program Review – dealing with a large deficit through major expenditure cuts
Departmental Reporting to Parliament – moving to results-based plans and reports on performance
Planning Reporting and Accountability Structures – unsuccessful try at linking resources to results
First Program Evaluation Policy – government-wide implementation
Today’s Agenda
2000
Mid 90s
Early to Mid 80s
… some real challenges remain

11. Problems, priorities and implementation challenges

12. Our agenda focuses on four main problems that are limiting us from getting performance information into our budget and management processes
To regain our detailed whole-of-government understanding of the ongoing program base we are implementing the Management, Resources and Results Structure (MRRS)
To have better ongoing measurement, periodic evaluation and use of performance information we are renewing the evaluation function
To help Parliamentarians fulfil their role of holding the Government to account we are transforming the existing reporting regime through the Improved Reporting to Parliament Project (IRPP)
To ensure the right management incentives are in place we are implementing the Management Accountability Framework (MAF)
Agenda
MRRS
Evaluation
IRPP
MAF

13. How do you regain a detailed understanding of the program base?
MRRS
Evaluation
IRPP
MAF
How do you regain a detailed understanding of the program base?
To manage its programs, the government needs to know:
What they are
How they align to intended outcomes
What they cost
What results they achieve
This should support more informed decisions on value for money, program effectiveness and possible ways of improving
Treasury Board approved the Management, Resources and Results Structure policy (MRRS) to remedy this situation
The policy is mandatory for appropriated organizations and entering the second year of implementation

14. MRRS
Evaluation
IRPP
MAF
The approach requires consistent, structured information for every department and agency
Program Activity Architecture (PAA) Financial and non financial information
Program Activity Architecture
D - 1
D - 2
D - 3
D - 4 D
Sub - Sub – Sub – Program Activity
Sub - Sub – Program Activity
Sub – Program Activity
Program Activity
Departmental Strategic Outcome M
Department or Agency
Planned and Actual Results
Planned & Actual Financial information
Governance
Required for all
elements and
all levels
of the PAA
Required for the highest or “program activity” level
Recoding of departmental information systems will be necessary
Central Expenditure Management Information System (under development)

15. Budget Office Function
MRRS
Evaluation
IRPP
MAF
MRRS is the basis for planning, resource allocation and reporting across the government
DEPARTMENTS
Management Tool
Improve program performance
Better alignment of resources & priorities
Easier reallocation
More evidence based reporting
More logical & consistent basis for interaction
PARLIAMENT
Reporting Tool
Stronger accountability
on spending and results
Common basis for planning & reporting inside & outside
Finance
Privy Council Office
Budget
Plan
Budget Office Function
Informed decisions and investment choices based on priorities and
value for money
MRRS

16. Implementation faces four main challenges
MRRS
Evaluation
IRPP
MAF
Implementation faces four main challenges
Cultural
Some departments have had difficulty in adopting a performance measurement culture
Many tend to identify too many measures and actually measure too few
Transparency leads to accountability and that can be scary
Estimate it will take 3 – 5 years to establish a government wide performance measurement framework
Institutional
Departments are managed in vertical, organizational silos and some are having difficulty converting to logic based program structures
Over the last 2 years 30% of departments have made changes to the basic program activity architecture – not unexpected
Performance measurement capacity – demographic concerns
3-5 years to stabilize to long term and enduring structures

17. Implementation faces four main challenges
MRRS
Evaluation
IRPP
MAF
Implementation faces four main challenges
Definitional
What is a program?
How do you measure the effectiveness of corporate services?
How do you treat overheads consistently?
What is the lexicon of program descriptions? (dozens of other questions)
Technical
Developing a system capable of collecting, sorting and linking information
Making sure it talks to other systems – annual budget, public accounts and others
Recoding of departmental systems is a large job – pilot projects

18. … and this will put a strain on evaluation capacity
MRRS
Evaluation
IRPP
MAF
The Federal Accountability Act, if passed, will increase scrutiny of program performance …
Auditor General to inquire into the use of funds by third parties under any funding agreement with the federal department
Parliamentary Budget Office to provide analysis to Parliament about the nation’s finances and trends in the national economy
Departments to evaluate, at least once every five years, the relevance and effectiveness of its grants and contributions programs
… and this will put a strain on evaluation capacity

19. There are some real challenges for the evaluation function
MRRS
Evaluation
IRPP
MAF
There are some real challenges for the evaluation function
Quality, timeliness and strategic focus issues make it difficult to use evaluation to support decision-making:
often focused on small programs and not strategic
can take too long to complete and are difficult to understand
can be self serving when funded by program managers
Government wide capacity issues have made it difficult to deliver
Lack of trained evaluators
No consistent competencies for those who lead the evaluation function
Definition of the evaluation “product” hasn’t changed much in 20 years

20. Step #1 is to renew our policy so that evaluation is…
MRRS
Evaluation
IRPP
MAF
Step #1 is to renew our policy so that evaluation is…
Better recognized as a core function of public management
Embedded in the decision-making process
Linked to budgeting and expenditure management
A neutral and independent function, aligned to government management priorities
Timely in its production of a quality product

21. Step #2 is to zone in on implementation
MRRS
Evaluation
IRPP
MAF
Step #2 is to zone in on implementation
Make sure the function is properly resourced
Build capacity by re-establishing training and recruitment programs and partnering with universities on certification
Ensure objectivity – evaluators fund evaluations, period
Redefine the “product” to promote relevance and better coverage:
Strategic policy evaluation
Impact evaluation
Value-for-money assessments
Implementation evaluation
Treasury Board Secretariat to assess quality and set out clear competencies for Heads of evaluation and evaluators
Introduce a Government wide evaluation plan

22. Workload – Will be difficult to manage without a realistic plan
MRRS
Evaluation
IRPP
MAF
Canadian evaluators are ready for change but they will face real implementation challenges
Demographic - High turnover rates will need to be met with targeted whole of government recruitment
Workload – Will be difficult to manage without a realistic plan
Skills – Will take time to rebuild training programs and partner with universities
Jurisdictional – Continually reinforcing the distinction between internal audit and evaluation
Relevance – Good quality, timely products need to be used to inform allocation and reallocation decisions

23. Parliamentarians want better public performance reporting, ideally …
MRRS
Evaluation
IRPP
MAF
Parliamentarians want better public performance reporting, ideally …
Simpler, more integrated information
More context and analysis (trends, graphs, including reporting on horizontal issues)
High level overviews with the ability to drill-down to more detail
Clearer logic between planning and performance reporting documents
More balanced reporting
Better links between programs, resources and results
More clarity between parliamentary reports and estimates votes

24. Whole of Government planning and reporting is part of the solution …
MRRS
Evaluation
IRPP
MAF
Whole of Government planning and reporting is part of the solution …
Government could better communicate its overall “strategic plan”
This “strategic plan” could be akin to a whole of Government Report on Plans and Priorities:
Outlining plans and priorities for the coming fiscal year
Linking planned results to planned expenditures
Electronic, in every parliamentary office
Logically, a fall whole of Government performance report could present the results and actual spending vis-à-vis such a Plan
… but this solution depends on a stable, whole of Government framework

25. MRRS
Evaluation
IRPP
MAF
This framework has been tested in Canada’s Performance over the past few years

26. MRRS
Evaluation
IRPP
MAF
This agenda depends on better management overall: the Management Accountability Framework
Public Service Values
By their actions departmental leaders continually reinforce the importance of PS Values and Ethics in the delivery of results to Canadians (e.g.: democratic, professional, ethical and people values).
Learning, Innovation and Change Management
The department manages through continuous innovation and transformation, promotes organizational learning, values corporate knowledge, and learns from its performance.
Results and Performance
Relevant information on results (internal, service & program) is gathered and used to make departmental decisions, and public reporting is balanced, transparent, and easy to understand.
Stewardship
The departmental control regime (assets, money, people, services, etc.) is integrated and effective, and its underlying principles are clear to all staff.
Policy and Programs
Departmental research and analytic capacity is developed and sustained to assure high quality policy options, program design and advice to Ministers.
Accountability
Accountabilities for results are clearly assigned and consistent with resources, and delegations are appropriate to capabilities.
Citizen Focused Service
Services are citizen-centred, policies and programs are developed from the ‘outside in’, and partnerships are encouraged and effectively managed.
People
The department has the people, work environment and focus on building capacity and leadership to assure its success and a confident future for the Public Service of Canada.
Risk Management
The executive team clearly defines the corporate context and practices for managing organizational and strategic risks proactively.
Governance & Strategic Direction
The essential conditions – internal coherence, corporate discipline and alignment to outcomes -- are in place for providing effective strategic direction, support to the Minister and Parliament, and the delivery of results.

27. So far, so good but there are some implementation issues:
MRRS
Evaluation
IRPP
MAF
The Management Accountability Framework is results based and demands attention
All departments are assessed against a set of indicators including the quality of Management, Resources and Results Structures, program evaluation and reports to Parliament
Discussions with senior officials identify management improvement priorities
Treasury Board Secretariat feeds this input into performance appraisals for the most senior officials
So far, so good but there are some implementation issues:
Process takes time
Resource intensive
Debates on ratings with departments

28. Lessons Learned

29. There are several lessons learned from recent Canadian experience
There is no end point on results based management – persistence over many years is required and you never get it “right”
You need central leadership to build capacity – someone with authority must set the game plan, make and enforce policies and invest
A detailed understanding of the program base is essential – it is very easy to lose and not easy to get back
Everybody knows about cultural barriers but don’t underestimate the challenges with systems
You need a common whole of government planning and reporting framework if you want to do real strategic planning and reporting
There is no substitute for evaluation but you need to give it some backbone and support
Parliamentarians and the external auditor demand better public performance information and this is good
There is no managing for results without sound management practices period – you need clear expectations and annual assessment

30. Useful resources
Government of Canada -
Treasury Board Secretariat -
Finance (Budget) -
Reports on Plans and Priorities ( ) -
Departmental Performance Reports ( ) -
Canada’s Performance -
Results-Based Management (MRRS, Evaluation, Improved Reporting to Parliament)
Management Accountability Framework -
EMIS -