How to Organize Data Collection For Registers on ART What Data and Why? Istanbul. March 21, 2009 David Adamson, MD Director, Fertility Physicians of Northern California Clinical Professor, Stanford University Associate Clinical Professor, UCSF International Committee Monitoring ART

What Do We Need to Know? 1. Country/Region 2. ART Clinic 3. Patient demographics 4. Treatment 5. Outcome

Country/Region Why does it matter? 1.Race 2.Socioeconomic status 3.Cultural/religious uniqueness 4.Political agenda 5.Healthcare 1. Quality 2. Access 6.Submission to ICMART

ART Clinic Why does it matter? Public vs. private Public vs. private Academic vs. non-academic Academic vs. non-academic Large vs. small Large vs. small Urban vs. rural Urban vs. rural Types of services provided Types of services provided

ART Patient Demographics Why does it matter? Types of patients using ART Types of patients using ART Factors that affect pregnancy rates Factors that affect pregnancy rates Types of interventions in different types of patients Types of interventions in different types of patients Different outcomes in different types of patients Different outcomes in different types of patients

ART Treatment & Outcomes Why does it matter? Need to know which interventions are successful Need to know which interventions are successful Need to link intervention with patient type and outcome to determine benefit of intervention Need to link intervention with patient type and outcome to determine benefit of intervention Link with complications Link with complications

What is Success?* Percentage=NumeratorDenominator *ICMART Glossary does not include specific measures of “success”, which should take into consideration the wellbeing of babies as well as their mothers.

Possible Numerators 1. Number of oocytes 2. Number of 2 pronuclear zygotes 3. Number of healthy embryos 4. Number of embyros transferred 5. Number of embryos cyropreserved 6. Chemical pregnancy 7. Clinical pregnancy 8. Fetal heartbeat 9. Viable (ongoing) pregnancy 10. Third trimester pregnancy 11. Term pregnancy 12. Live birth(s) 13. Healthy child(ren) 14. Healthy child(ren) including frozen embryos 15. Healthy adult(s)

Possible Numerators The numerator that is chosen usually depends on the interest of the person doing the choosing: Patient Patient Infertility specialist Infertility specialist Scientist Scientist Obstetrician Obstetrician Pediatrician Pediatrician Policy-maker Policy-maker

Most appropriate numerator: Healthy Singleton Best for patient and family Best for patient and family Best for child Best for child Best for obstetrician Best for obstetrician Best for pediatrician Best for pediatrician Best for policy-maker Best for policy-maker ? Best for infertility physician ? Best for infertility physician

Problems With Healthy Singleton as Measure of Outcome Who collects the data? Who collects the data? What is definition of healthy? What is definition of healthy? Who evaluates baby? Who evaluates baby? When is evaluation done? When is evaluation done? How long is the follow-up? How long is the follow-up? What about healthy multiples? What about healthy multiples?

Possible Denominators ART is not a single procedure 1. 1.Decision to pursue ART 2. 2.Ovarian stimulation (“Intention to treat”) 3. 3.Egg retrieval 4. 4.Fertilization 5. 5.Embryo transfer 6. 6.Implantation (chemical pregnancy) 7. 7.Fetal development (clinical vs. heartbeat) 8. 8.Viable pregnancy (>20 wks vs. > 26 wks) 9. 9.Live-birth delivery Patient (i.e. more than one cycle, + frozen)

Problems Using “Patient” as Denominator Women do different numbers of cycles Women do different numbers of cycles Some have frozen embryo cycles, some not Some have frozen embryo cycles, some not Variation in decision to pursue additional cycles Variation in decision to pursue additional cycles Different populations in each round of cycles Different populations in each round of cycles Cycles performed different Cycles performed different –Years –Intervals –Clinics Important laboratory data may be overlooked Important laboratory data may be overlooked

USA Definition Pregnancy Rate= Live Births per number of ovarian stimulation procedures initiated

USA Dataset Clinic Clinic name, address, number Name of laboratory used* SART member ? Services for single women ? Donor egg available ? Donor egg sharing ? Total number of ART cycles

USA Dataset Patient Information Ethnicity, birth date, address U.S. Resident Prior pregnancy history Sterilization Months of infertility Prior ART cycles FSH/E2 levels

USA Dataset ART Cycle Information Reason(s) for ART Reason(s) for ART Cycle start date Cycle start date Suppression with GnRHa Suppression with GnRHa Stimulation drugs & dosage Stimulation drugs & dosage Intended treatment procedures Intended treatment procedures –Fresh/frozen –Patient/donor eggs/surrogacy –IVF, GIFT, ZIFT,TET –Cycle for embryo banking, research

USA Dataset ART Cycle Information Did cycle occur as intended? Did cycle occur as intended? –Cancelled, date, reason –Complications –Hospitalization Date of oocyte retrieval Date of oocyte retrieval Number of oocytes Number of oocytes Semen source, collection method Semen source, collection method

USA Dataset ART Cycle Information Use of ICSI, hatching Use of ICSI, hatching Transfer attempt & date Transfer attempt & date Number of fresh embryos transferred, cryopreserved Number of fresh embryos transferred, cryopreserved Number of thawed embryos transferred, re-frozen Number of thawed embryos transferred, re-frozen

USA Dataset ART Cycle Outcome Information Outcome Outcome –Not pregnant –Type of pregnancy Ultrasound Ultrasound –Date –Number of fetal hearts Induced reduction Induced reduction

USA Dataset ART Cycle Outcome Information Outcome of pregnancy Outcome of pregnancy Source of information Source of information Number of infants born Number of infants born Birth weight Birth weight Neonatal morbidity/mortality Neonatal morbidity/mortality

USA Dataset ART Cycle Outcome Information Birth Defects Birth Defects For each infant For each infant –Pregnancy termination Spontaneous Spontaneous Elective Elective Induced reduction (multifetal reduction) Induced reduction (multifetal reduction) –Stillbirth –Live birth Categorization of defect Categorization of defect Ascertainment issues and bias Ascertainment issues and bias Birth certificates unreliable* Birth certificates unreliable* Later death and diagnosis ? Later death and diagnosis ? *Gore et al. J Reprod Med 2002;47:

ART Outcomes Birth Defects Confidentiality issues (e.g., HIPAA) Linkage to other registries (e.g. birth registry) Definition inconsistency Data poor quality Methodology weaknesses Ascertainment bias Lack of controls Statistics complicated Wide confidence intervals Statistical vs. clinical significance

Summary What Data and Why? Many factors influence what data should be collected Many factors influence what data should be collected Need system to collect good data (GIGO!) Need system to collect good data (GIGO!) STANDARDIZATION STANDARDIZATION –Definitions (ICMART/WHO glossary) –Collection –Analysis

The Fertility Clinic Success Rate And Certification Act (FCSRCA) of 1992 (Wyden Law) Passed with support of SART and ASRM Passed with support of SART and ASRM Required: Required: –Annual reporting clinic-specific success rates –Listing of clinics that do not report –Development of model program for certification of embryo laboratories –Promulgation of criteria and procedures for approval of accreditation programs to inspect and certify labs First report published under the law in 1997 for 1995 cycles First report published under the law in 1997 for 1995 cycles

FCSRCA Clinic-SART-CDC Relationship CDC contracts with SART to obtain annually a copy of SART database CDC contracts with SART to obtain annually a copy of SART database Clinics submit data to SART Clinics submit data to SART 380 of 400 (95%) clinics are SART members 380 of 400 (95%) clinics are SART members All clinics that submit data to this CDC-supported SART system in compliance with FCSRCA All clinics that submit data to this CDC-supported SART system in compliance with FCSRCA SART keeps database of clinics (openings and closings) SART keeps database of clinics (openings and closings) Clinics must notify SART of personnel, address etc. changes Clinics must notify SART of personnel, address etc. changes Requirements published in the Federal Register 2000;65: Requirements published in the Federal Register 2000;65:

FCSRCA Data Collection SART distributes Clinical Outcomes Reporting System and instructions (SART-CORS) annually to clinics SART distributes Clinical Outcomes Reporting System and instructions (SART-CORS) annually to clinics Clinics abstract data from clinic records Jan 1-Dec 31 Clinics abstract data from clinic records Jan 1-Dec 31 Data entered using SART software Data entered using SART software ART cycle starts with ART cycle starts with –ovarian stimulatory drugs, or –ovarian monitoring with the intent of having embryos transferred Data file organized with one record per cycle Data file organized with one record per cycle Multiple cycles from one patient not linked (HIPAA) Multiple cycles from one patient not linked (HIPAA)

FCSRCA Data Quality During Collection Medical Director responsible Medical Director responsible –Submit data on time (one year following year being reported) –Verify by signature that data are accurate –All patient identifiers removed before submission to SART Prospective data submission Prospective data submission –Started with Year 2000 data –Within 3 days of cycle start (i.e. before outcome known) –Demographic data –Internet based

FCSRCA Data Quality During Collection SART compiles data and submits to CDC SART compiles data and submits to CDC –CDC cannot identify individual patients SART & CDC review, resolve inconsistencies SART & CDC review, resolve inconsistencies SART compiles and submits final, corrected dataset to CDC SART compiles and submits final, corrected dataset to CDC Individual cycle data compiled and analyzed Individual cycle data compiled and analyzed –Report clinic-specific results –Aggregated for national report

FCSRCA Data Quality Validation Sample of 8-10% of clinics chosen for on-site validation Sample of 8-10% of clinics chosen for on-site validation Randomly, and Randomly, and Pre-selected variables Pre-selected variables –50 randomly selected charts reviewed by 2-person team –CDC representative attends some visits as observer –Error rates calculated –SART & CDC review findings

FCSRCA Data Quality Validation Data validation process Data validation process –Primarily educational –Identify problem areas in data-collection process –Correct general data collection problems –Requires individual clinics to correct problems –Funded by government as of 1999 Validation results to date Validation results to date –All error rates for all clinics acceptable –Majority of errors minor –Errors have minimal impact on success rates

FCSRCA Data Analysis and Publication CDC primarily responsible CDC primarily responsible Report co-authored by ASRM, SART, RESOLVE Report co-authored by ASRM, SART, RESOLVE Results published hard copy and website Results published hard copy and website –National report –Individual fertility clinic tables –Appendices of associated information Detailed internal analysis and statistical validation Detailed internal analysis and statistical validation CDC editorial staff for final format and proofing CDC editorial staff for final format and proofing Publication by CDC Publication by CDC

Additional Recent Advances in USA Registry Model program for certification of laboratories Model program for certification of laboratories Publish scientific articles based on registry data (SART & CDC) Publish scientific articles based on registry data (SART & CDC) Obtain highest possible Confidentiality Status (308D): limits data access to CDC & SART Obtain highest possible Confidentiality Status (308D): limits data access to CDC & SART Bridge to SART CORS from clinic IT systems Bridge to SART CORS from clinic IT systems

What Have We Learned? ART surveillance system is not static ART surveillance system is not static Challenges Challenges –Data collection Large number of clinics, coordination difficult Large number of clinics, coordination difficult Deadlines necessary Deadlines necessary Individual clinic variability Individual clinic variability Standardization of definitions and practice difficult Standardization of definitions and practice difficult –Data presentation Focus groups Focus groups Other feedback Other feedback Simplification and explanation Simplification and explanation

The Future Surveillance system will continue to evolve Surveillance system will continue to evolve Clarification of definitions and ambiguities (ICMART) Clarification of definitions and ambiguities (ICMART) Collect better data on outcomes and confounding variables Collect better data on outcomes and confounding variables Stop collecting non-useful variables Stop collecting non-useful variables Link ART cycles in same patient=cumulative live births Link ART cycles in same patient=cumulative live births Prospective reporting, validation, analysis, audit, investigation, sanctions Prospective reporting, validation, analysis, audit, investigation, sanctions Focus on Focus on –Accuracy –Appropriate context for use of data Fairness to clinics Fairness to clinics Clarity to patients Clarity to patients Continued collaboration ASRM, SART, Clinics, RESOLVE, CDC Continued collaboration ASRM, SART, Clinics, RESOLVE, CDC

THANKYOU!