1. Best Paper Award
Transparency of Hospital Productivity Benchmarking in Two Finnish Hospital Districts (Research-in-Progress)
S. Laine, Department of Computer Science and Engineering, Aalto University E. Niemi, School of Economics, Aalto University
Link to Full Paper
the 29th annual Patient Classification Systems International (PCSI) Conference

2. The Finnish Hospital Productivity Benchmarking has a long history – but it is not used in decision making
Data
Results
The benchmarking results are produced by National Institute for Health and Welfare (THL) on annual basis.
The background, implementation and future plans of the BMS have been described earlier by Linna and Häkkinen.
They noted that policymakers and managers do not regularly use efficiency analyses and the main reason appears to be concern about data quality.
Linna, M. and Häkkinen, U. (2007) Benchmarking Finnish Hospitals. In Evaluating Hospital Policy and Performance: Contributions from Hospital Policy and Productivity Research , pp

3. Benchmarking claimed significant productivity differences in neurology specialty
Pirkanmaa Hospital District
Hospital District of Southwest Finland
1,12
0,71

4. Finnish Hospital Productivity Benchmarking
Information Production Process (IPP) consists of three phases based on Total Quality Management (TQM)
DATA SUPPLY
DATA MANUFACTURING
DATA CONSUMPTION
Analyses and reports data
Builds data sets for secondary use
Enters data for primary purpose
Interprets data and makes decisions
Scripts produce internal reports
Internal Service Reports
Electronic Patient Record
Scripts
collect data
Data Warehouse
Scripts produce data external sets
Scripts collect data
Medical Imaging System
Finnish Hospital Productivity Benchmarking
Data Warehouse
Scripts produce external reports
Wang, R. Y., Lee, Y. W., Pipino, L. L. and Strong, D. M. (1998) Manage Your Information as a Product. Sloan Management Review, 39, 4, pp

5. The QUALIDAT project ( ) uses three complementary research approaches to study the same Information Production Process (IPP)
The management approach studies the information management and governance best practices
The usability approach studies the users and their hands-on work situations in care pathways
The data analytics approach tracks the entire information flow from data entry to data utilization

6. Finnish Hospital Productivity Benchmarking
Productivity figures are complex combination of care pathways and information production processes!
DATA SUPPLY
DATA MANUFACTURING
DATA CONSUMPTION
Financial Issues
Analyses and reports data
Builds data sets for secondary use
Enters data for primary purpose
Interprets data and makes decisions
Care pathways
Scripts produce internal reports
Internal Service Reports
Electronic Patient Record
Scripts
collect data
Data Warehouse
Scripts produce data external sets
Scripts collect data
Medical Imaging System
Productivity Formula
Finnish Hospital Productivity Benchmarking
Data Warehouse
Scripts produce external reports
Information production process

7. Fragmentation bias rewards splitting and heterogeneity
Episode 1 …
DRG A
DRG B
DRG C
Episode A
DRG X
Hospital District 3 2
Episode A
DRG X
More production but less health for same money!
Less production but more health for same money!

8. Systematic biases in measurement
The systematic biases have many undesirable consequences for validity of benchmarking results, quality of healthcare, and incentives of hospital management.
By systematic bias we mean unintended or undesirable inherent characteristics that have effects to the benchmarking results.
We identified two types of potential systematic biases: fragmentation and case-mix.
Complex fragmentation of care pathways can be built-in system mechanisms in the technical software systems, hospital business models and clinical work practices.
The NordDRG-system weights used in the productivity calculations have been said to fail in capturing fully case-mix differences.
Fragmentation bias is problematic for our case organizations, because they have different service model compared to other Finnish hospital districts.
Focused on health problems and patient processes
Others have traditional specialty based organization

9. Finnish Hospital Productivity Benchmarking
IPP has three problem themes: Human Errors, Software Features and Obscurity
DATA SUPPLY
DATA MANUFACTURING
DATA CONSUMPTION
Obscurity
Obscurity
Obscurity
Analyses and reports data
Builds data sets for secondary use
Scripting Error
Data Entry Errors
Enters data for primary purpose
Interprets data and makes decisions
Scripts produce internal reports
Internal Service Reports
Electronic Patient Record
Scripts
collect data
Application Feature Bias
Architecture Bias
Data Warehouse
Scripts produce data external sets
Scripts collect data
Scripting Error
Medical Imaging System
Finnish Hospital Productivity Benchmarking
Obscurity
Data Warehouse
Scripts produce external reports

10. Transparent Care Pathway
More information about the entire benchmarking system should be made visible for all stakeholders to avoid organizational silos and hidden bias
Transparency should cover all significant influencing factors in result themes: productivity formula, healthcare service production, information production process and financial issues
Details are important!
A small detail can have huge impacts to the factors in other themes.
Transparency should open all IPP phases in and between all participating organizations: data supply, manufacturing and consumption
Semantic error can occur in any phase!
Errors can increase, diminishing or change direction depending on the internal calculations! Black boxes are unpredictable.
Transparent Care Pathway
Transparent Information Production Process

11. More information about the entire benchmarking system should be made visible for all stakeholders to avoid organizational silos and hidden bias
Only in this way, one can evaluate the validity of decisions in patient care, hospital administration, policy making, and medical research.
Transparency should cover all significant influencing factors in result themes: productivity formula, healthcare service production, information production process and financial issues
Details are important!
A small detail can have huge impacts to the factors in other themes.
Transparency should open all IPP phases in and between all participating organizations: data supply, manufacturing and consumption
Semantic error can occur in any phase!
Errors can increase, diminishing or change direction depending on the internal calculations! Black boxes are unpredictable.
Transparent Care Pathway
Transparent Information Production Process

12. The Next Steps in QUALIDAT Project

13. Finnish Hospital Productivity Benchmarking
QUALIDAT tracks care pathways and data flows – to identify critical factors that affect productivity results!
DATA SUPPLY
DATA MANUFACTURING
DATA CONSUMPTION
Referral
User Interface Screen 1
Attribute X
Attribute X
Visit
Attribute X
Attribute X
User Interface Screen 2
Admission
Attribute X
Finnish Hospital Productivity Benchmarking
User Interface Screen 3
Attribute X
Discharge
Attribute X