1. A New Approach in University Evaluation: the Balanced Scorecard
Angela Maria D’Uggento, Massimo Iaquinta,Vito Ricci
Università degli Studi di Bari
Direzione Analisi Statistica – Area Studi, ricerche e programmazione
Staff di Rettorato
Statistical modelling for University Evaluation: an International Overview
Università di Foggia- Mattinata (FG), 5th-6th September 2008

2. A New Approach in University Evaluation: the Balanced Scorecard
Balanced scorecard: a definition
The balanced scorecard (BSC) is a strategic planning and management system used to align business activities to the vision and strategy of the organization and to monitor organization performance against strategic goals. It embodies a set of measures that gives top managers a quick but exhaustive vision of the business; in fact, it complements the financial measures with operational measures on: 1) customer satisfaction; 2) internal processes 3) organizational innovation; 4) improvement activities (Morard et Stancu, 2005). It was created by Kaplan and Norton as a performance measurement framework that added strategic non-financial measures to traditional financial ones (Kaplan, Norton 1992).

3. A New Approach in University Evaluation: the Balanced Scorecard
The BSC has evolved from its early use to a full strategic planning and management system (Kaplan, Norton 1996).
Born in the business managerial field, in the last decades it has drawn the attention of public administrators, becoming a useful tool to carry on evaluation during years.
Some foreign Universities have adopted the BSC: Edinburgh (UK), San Diego, California (USA), Bond (Australia).
Any Italian University has not a BSC but it’s currently used either by private companies or by Public Administration (Health Care, Local Government, etc).

4. A New Approach in University Evaluation: the Balanced Scorecard
Balanced scorecard: the 4 traditional Perspectives
Financial: it concerns with economic and financial aspects;
Stakeholders: this perspective allows the Organization to fit outcome metrics to “customers” needs and expectations;
Internal process: it’s the perspective of operational processes;
d) Learning and Growth: it includes employee training, empowerment, informative systems.

5. A New Approach in University Evaluation: the Balanced Scorecard
BSC: traditional perspectives’ relations
A New Approach in University Evaluation: the Balanced Scorecard

6. A New Approach in University Evaluation: the Balanced Scorecard
Balanced scorecard: main steps
Specification of strategic objectives and goals
Definition of a Strategical Map to highlight cause and effect relationships among strategic objectives
Selection of the KPA (key performance actions) and the KPI (key performance indicators)
Implementation of the BSC dashboard for monitoring and evaluation

7. A New Approach in University Evaluation: the Balanced Scorecard
The Balanced scorecard of the University of Edinburgh/1
Four perspectives and performance indicators

8. A New Approach in University Evaluation: the Balanced Scorecard
The Balanced scorecard of the University of Edinburgh/2
Four perspectives and performance indicators

9. The Balanced scorecard of the University of Edinburgh/3
Specification of the single indicator (1)

10. The Balanced scorecard of the University of Edinburgh/4
Specification of the single indicator (2)

11. A New Approach in University Evaluation: the Balanced Scorecard
A BSC for an Italian University: the revised Perspectives
(Del Sordo et al, 2007) d
Students (Academic Courses)
(needs and expectations)
Community (Research)
Learning and Growth
(employee training, empowerment,
informative systems.)
Internal processes
(operational processes)
Economic and financial
(efficency & efficacy)
Stakeholders
Learning &
Growth
Internal
Processes
Financial

12. A New Approach in University Evaluation: the Balanced Scorecard
A BSC for University: Our proposal
Introduction of MiUR five Guidelines and Indicators about University Planning (MiUR, DM n.362/07 e DM n. 506/07) into the BSC standard frame;
2) Translation of the six guidelines of Bari University Planning into the four BSC traditional dimensions.

13. A New Approach in University Evaluation: the Balanced Scorecard
A BSC for University: Our proposal
The perspective Stakeholders would be divided into 4 sections (Figure 1):
Academic courses,
Development of scientific research,
Relationship with territory,
Internationalization programs
so representing all the possible “players”: students and their families, researchers, local, domestic and foreign Institutions and firms.

14. A New Approach in University Evaluation: the Balanced Scorecard
Figure 1 - BSC Perspectives and Guidelines of University planning

15. A New Approach in University Evaluation: the Balanced Scorecard
A BSC for University: MiUR Guidelines & Evaluation Fields
(from DM 506/2007)
Academic courses (3 indicators)
Scientific Research (5 indicators)
Services to Students and Researchers (5 indicators)
Internationalization Programmes (4 indicators)
Teaching and non-teaching staff policies (4 indicators)
Relationships with territory (Bari Plan )

16. A New Approach in University Evaluation: the Balanced Scorecard
A BSC for Bari University
Strategic objectives, Initiatives/actions & targets Key Performance Indicators
University Plan
21 MiUR indicators +
21 Performance indicators

17. A New Approach in University Evaluation: the Balanced Scorecard
A BSC for University: some performance indicators of Bari University

18. A New Approach in University Evaluation: the Balanced Scorecard
A BSC for University: work in progress
If implemented in Italian Universities, BSC could represent:
a standardized model and as a reporting tool for decision-makers and stakeholders;
a framework to measure the multidimensional performance of the University (intellectual & relational capital), whose core business is producing and offering knowledge intensive services and to maximize students’ satisfaction;
a modern and integrated approach, particularly appreciated by the University management which, sometimes, hardly succeeds in monitoring, at the same time, several variables which are usually untied;
an instrument to rationalize data collection and to minimize information overload

19. A New Approach in University Evaluation: the Balanced Scorecard
A BSC for University: work in progress
There is no doubt that one criticism of the BSC is its construction. Sometimes, the conceptualization is very hard to apply in a university environment, but a modeling of the system is required as a starting point (how many Italian universities have “mapped” their internal processes ?). As to the methodological approach, in recent years the research directions include the structural equation modeling, encompassing Partial Least Squares models, and the multivariate approach, in addition to the traditional statistical techniques.

20. A New Approach in University Evaluation: the Balanced Scorecard
A BSC for University: work in progress
Structural equation model provides the estimation of multiple interrelated dependent relations between variables and also the investigation of latent variables. PLS regression, generalizing and combining features from principal component analysis and multiple regression, is particularly useful to predict a set of dependent variables from a very large set of independent variables.

21. A New Approach in University Evaluation: the Balanced Scorecard
A BSC for University: further developments
the introduction of a BSC concerning the whole Italian Universities System (using mean or median of single Universities’ indicators with also standard deviation). It could be a useful tool for the Italian Ministry of Higher Education;
a standardized instrument of benchmarking among Italian Universities;
after measuring and managing performances, an instrument to evaluate them, either at the whole University level or at Faculty or Department level;
to apply a structural equation modeling in a rationalization tentative of BSC (Morard & Stancu, 2005)