1. Evaluation of the metrological quality of medico-administrative data for perinatal indicators: a pilot study in three French university hospitals
K.Goueslard1
A.Pierron3
M.Revert1
A. Vuagnat2
J.Cottenet1
E. Benzenine1
J. Fresson3
C. Quantin1,4
Thank you for giving me the opportunity to present our study about the metrological quality of medico-administrative data for perinatal indicators.
This study was coordinated by Pr C. Quantin.
1.Department of medical information-Dijon, France
2.DREES department of research and evaluation – Ministry of Health
3.Department of medical information-Nancy
4.Inserm UMR 1181 « Biostatistics, Biomathematics, Pharmacoepidemiology and Infectious Diseases » 1

2. Background Exhaustive and standardised information for Indicators
EUROPERISTAT
Public health studies
Medico-administrative database
800,000 deliveries/year
Comparison with National Perinatal Enquiry
In perinatal health, exhaustive and standardized information is required to compare indicators with those in other European countries and for public health studies.
The Medico-administrative database has particular interest in France since almost all births take place in hospital. We have already compared aggregated statistics from the medico- administrative database with National Perinatal Enquiry aggregated data and shown the interest of using such data, as the results are very close.
We would like now to undertake epidemiological studies using hospital data and need to have more information about the quality of individual data. 2

3. Pilot study Assess the metrological quality of
medico-administrative data (discharge abstracts) for core indicators in three university hospitals
Compare the medico-administrative data to the medical record considered as the Gold Standard
The aim of our study was to assess the metrological quality of the medico-administrative data for core indicators in three university hospitals.
We propose to compare hospital data with medical records which we consider the gold standard. 3

4. Selection of discharge abstracts
Three university hospitals:
Paris Port-Royal, Nancy, Dijon
Discharge abstracts (2012)
Selection of deliveries from French Common Classification of Procedures
Random selection of 100 live births per hospital
Nancy
Paris
Dijon
This pilot study took place in 3 university hospitals of a large, a middle and a small university town.
Selection of deliveries was conducted using procedures coded in discharge abstracts, according to the French Common Classification.
We developed a specific program to select at random one hundred live births by hospital among the eleven thousand fifteen eligible stays in these hospitals during the year two thousand and twelve. 4

5. Collection from medical records
Characteristics collected by midwives
Stay (length of stay, …)
Women (age, obesity, parity, …)
Pregnancy (diabetes, hypertensive disorders, …)
Delivery (mode, epidural, postpartum haemorrhage, …)
Children born (gender, weight)
In medical records, data were collected by midwifes from a standardized collection grid, which included characteristics of the hospital stay, the women, the pregnancy, the delivery and the children born. 5

6. Comparison discharge abstracts/medical records
Positive Predictive Value (PPV)
Sensitivity
Concordance rate
Medical record
Medico Adinistrative
Data + -
PPV FP FN
NPV
PPV: Positive Predictive Value FP: False Positive
NPV: Negative Predictive Value FN: False Negative
The positive predictive value and the sensitivity were calculated for dichotomous data. Continuous data were assessed by the concordance rate.
PPV: probability that the information in the discharge abstract was also present in the medical record.
Se: probability that the information in the medical record was also present in the discharge abstract.
Concordance rate: number of concordant cases between discharge abstracts and medical records. 6

7. Maternal characteristics
Medical record
Medico administrative data
PPV [CI 95%]
Sensitivity
Age
< 20 7
100
20-24 28 29
93.1 [ ]
96.4
25-29 88 91
94.4 [ ]
96.6
30-34
104
101
99.0 [ ]
96.2
35-39 52
≥ 40 20 DM 1
Parity
Primiparous
109
116
91.4 [ ]
97.3
Multiparous
126
118
96.6 [ ]
90.5 2
As regards the average maternel age, the concordance rate was 93%. If we consider age classes, we can also compute the positive predictive value and the sensitivity which were within between 93 and 100%. In 2012, parity was only available for vaginal deliveries. The PPV was ninety-one point four (91.4) per cent for primiparous women and 96.6 per cent for multiparous women. 7

8. Medico administrative data
Maternal morbidity
Medical record
Medico administrative data
PPV [CI 95%]
Sensitivity
Diabetes 27 20
100
74.1
Pre-existing Diabetes 2
Gestational Diabetes 22 18
88.9[ ]
72.7
Diabetes, not specified 3 -
Obesity 33 32
81.3 [ ]
78.8
Hypertensive disorders 19
63.0
Gestational
Hypertension 4
66.7 [ ]
50.0
Pre-eclampsia
HELLP Syndrome 16 13
81.3
Postpartum Haemorrhage 41 38
90 [ ]
82.9
The results for maternal morbidity showed that the positive predictive value was 100% and the sensitivity was 74.1% for diabetes. The positive predictive value was only 88.9% for gestational diabetes.
In thirty-two hospital stays, the positive predictive value for obesity was 81.3%.
We studied hypertension disorders during pregnancy and found a positive predictive value of 100%. The positive predictive value for pre-eclampsia and HELLP syndrome was also 100% but the Se was only 81.3%.
Lastly, the positive predictive value for postpartum hemorrhage was 90%. 8

9. Delivery Characteristics
Medical record
Medico administrative data
PPV [CI 95%]
Sensitivity
Gestationnal age <37 33
93.9 [ ]
74.1
Simple birth
289
100
Twins birth 11
Epidural
211
209
96.2 [ ]
93.3
Vaginal delivery
197
198
99.5 [ ]
Instrumental extraction 40 39
97.5
Caesarean 62 63
98.4 [ ]
Caesarean in emergency 48 51
92.2 [ ]
97.9
Cephalic presentation
195
193 99
Breech presentation 3 5
60.0 [ ]
Concerning the delivery characteristics, the positive predictive value and sensitivity for simple and twin births were one hundred per cent.
As regards the mode of delivery, the positive predictive value was 99.5% for vaginal delivery, 100% for instrumental delivery, 98.4 for caesarean and 92.2 when the caesarean took place in an emergency.
The positive predictive value was 100% and the sensitivity was 99% for cephalic presentation among the 193 hospitals stays that provided data on this.
For some variables like gestational diabetes and obesity, the positive predictive value varied considerably between hospitals: PPV varied from 74,3% to 100% for gestational diabetes and from 67,8% to 94,8% for obesity. 9

10. Discussion Aggregated level: frequency of pathologies
Aggregate data: false negatives & false positives
Aggregated level: frequency of pathologies
Individual level: longitudinal studies
Professional practices: differences in coding practises
Postpartum Haemorrhage
Medical record: Hospital data: 38
We consider that this type of study to assess individual data is essential as aggregate data don’t assess the impact of false positives and false negatives which may balance each other out.
For example, postpartum hemorrhage presented similar numbers in the two databases but only a positive predictive value of 89,5%.
As a consequence, we think that, when the validation is performed at an aggregated level, medico- administrative data can be used to estimate the frequency of pathologies. However, for longitudinal studies, it is necessary to assess individual data.
It is also important to be careful when studying professional practices from hospital databases as the inter-hospital differences may only reflect differences in coding practices.
Our results differed of course from one indicator to another.
The indicators based on consensus definition and having an influence on the hospital budget (gestational age, simple or twin births, mode of delivery, presentation) were of course better collected.
Indicators with less standardized definitions were less documented. This was the case for pathologies for which the formulation of wordings in the medico-administrative database may differ from the terms used in clinical practice.
Specific training and validation seem to be needed to achieve the objectives of hospital financing, but also to meet the ever-increasing use of medico-administrative databases for epidemiological purposes. 10

11. Discussion Consensual definition: well-collected
Difference between indicators
Consensual definition: well-collected
Pathologies: less documented
Specific training and validation

12. Conclusion Medico-administrative data more reliable for two reasons ?
Importance for budgetary promotion in hospitals
Increasing use for statistical and epidemiological purposes
National study: 25 hospitals
In conclusion, since 2012 in France, medico-administrative data have become more reliable for two reasons:
- The importance of this data for budgetary purposes in hospitals
- The increasing use of this information for statistical and epidemiological purposes by research and government institutions.
This study produced preliminary results which should be confirmed by a national study which is financed by the National Agency for Research and will begin soon. It will include twenty-five hospitals and major indicators of perinatal health, such as prematurity and perinatal mortality. 12

13. 13

14. Medical record Administrative information Medical history
Pregnancy monitoring
Delivery report +/- operative report
Discharge letter

15. PPV: Probability that the information in discharge abstract was present in medical record
Sensitivity: Probability that the information in medical record was present in discharge abstract
Concordance rate
False positive (Type I error): Presence in hospital data since absence in medical record
False negative (Type II error): Absence in hospital data since presence in medical record

16. Selection of delivery Z37 may:
Be identified in postpartum stay
Take place in establishment other than delivery place
French common classification of procedures for delivery
All types of deliveries
Whatever presentation
Spontaneous vaginal deliveries
Instrumental vaginal deliveries
Asisted vaginal deliveries
Caesarean
Simple or multiple births

17. Maternal characteristics
PPV [CI 95%]
Sensitivity [CI 95%]
Age
< 20
100
20-24
93.1 [ ]
96.4 [ ]
25-29
94.4 [ ]
96.6 [ ]
30-34
99.0 [ ]
96.2 [ ]
35-39
≥ 40 DM
Parity
Primiparous
91.4 [ ]
97.3 [ ]
Multiparous
96.6 [ ]
90.5 [ ]
As regards the average maternel age, the concordance rate was 93%. If we consider age classes, we can also compute the positive predictive value and the sensitivity which were within between 93 and 100%. In 2012, parity was only available for vaginal deliveries. The PPV was ninety-one point four (91.4) per cent for primiparous women and 96.6 per cent for multiparous women. 17

18. Maternal morbidity Diabetes Obesity Hypertensive disorders
PPV [CI 95%]
Sensitivity [CI 95%]
Diabetes
100
74.1 [ ]
Pre-existing Diabetes
Gestational Diabetes
88.9[ ]
72.7 [ ]
Diabetes, not specified -
Obesity
81.3 [ ]
78.8 [ ]
Hypertensive disorders
63.0 [ ]
Gestational
Hypertension
66.7 [ ]
50.0 [ ]
Pre-eclampsia
HELLP Syndrome
81.3 [ ]
Postpartum Haemorrhage
90 [ ]
82.9 [ ]
The results for maternal morbidity showed that the positive predictive value was 100% and the sensitivity was 74.1% for diabetes. The positive predictive value was only 88.9% for gestational diabetes.
In thirty-two hospital stays, the positive predictive value for obesity was 81.3%.
We studied hypertension disorders during pregnancy and found a positive predictive value of 100%. The positive predictive value for pre-eclampsia and HELLP syndrome was also 100% but the Se was only 81.3%.
Lastly, the positive predictive value for postpartum hemorrhage was 90%. 18

19. Delivery Characteristics
PPV [CI 95%]
Sensitivity [CI 95%]
Gestationnal age <37
93.9 [ ]
74.1 [ ]
Simple birth
100
Twins birth
Epidural
96.2 [ ]
93.3
Vaginal delivery
99.5 [ ]
Instrumental extraction
97.5 [ ]
Caesarean
98.4 [ ]
Caesarean in emergency
92.2 [ ]
97.9 [ ]
Cephalic presentation
99 [ ]
Breech presentation
60.0 [ ]
Concerning the delivery characteristics, the positive predictive value and sensitivity for simple and twin births were one hundred per cent.
As regards the mode of delivery, the positive predictive value was 99.5% for vaginal delivery, 100% for instrumental delivery, 98.4 for caesarean and 92.2 when the caesarean took place in an emergency.
The positive predictive value was 100% and the sensitivity was 99% for cephalic presentation among the 193 hospitals stays that provided data on this.
For some variables like gestational diabetes and obesity, the positive predictive value varied considerably between hospitals: PPV varied from 74,3% to 100% for gestational diabetes and from 67,8% to 94,8% for obesity. 19