1. Constructing a Data Management System National Center for Immunization & Respiratory Diseases Influenza Division Regional Training Workshop on Influenza Data Management Phnom Penh, Cambodia July 27 – August 2, 2013 Pam Kennedy Analyst, McKing Consulting

2. Course Objectives Database Management System (DBMS)  What is it?  Essential functions Data collection forms Considerations in building a DBMS  Structure design  Data quality and control

3. Database Management System Definition  “…set of programs that enables you to store, modify, and extract information from a database,…database  … provides users with tools to add, delete, access, modify, and analyze data stored in one location …  ….provide the method for maintaining the integrity of stored data, running security … and recovering information if the system fails.” Basic database functions Date EntryStorage ModificationExtraction SearchingAnalysis http://en.wikipedia.org/wiki/Database_management_system

4. Considerations in Building a DBMS Essential Functions How will the data be used?  Understand the study objective  Types of data needed  Data relationships  Capture data collected from the questionnaire and study forms  Understand the data flow  Understand what output is visualized  Ask questions – no assumptions

5. Data Collection Forms Use data collection forms as the basis of the electronic database  Identify all collection forms  Understand the form sequence  Understand each question and desired output  Yes/No  Date field  Data lists  Eliminate redundant or unneeded information  Define interdependent information – Date of Birth vs. Date Of Hospital Admission Gender vs. Pregnant Date of Hospital Admission vs. Date of Hospital Discharge

6. Data Collection Forms Identify Data Rules  Identify variables that can be skipped – if any  If ‘Male’ then skip questions on pregnancy  Decide on variable options  Drop down lists  Yes/No fields  Option fields  Decide how to treat missing information  Not available vs. Unknown vs. Not applicable

7. Considerations in Building a DBMS Structure Design To increase effectiveness a good DBMS should have the following control functions enforced  Data access & relational functions  Security  Control access rights  Enforce data integrity  Relationship functions  Data accuracy review process  Database salvage functions  Backup and restore functions

8. Considerations in Building a DBMS Structure Design Questions to ask during design  How much data will be collected and stored?  How will data be analyzed?  Will year to year comparisons be conducted?  Will more than one person need access to data at same time?  Where will backup data be stored?

9. What is Data Quality (DQ)? Aspects of data quality include:  Accuracy  Date of birth expressed in day/months/years and not only years  Completeness  Missing information  Update status  Timeliness  Relevance  Data relevant for the purpose of the activity  Consistency across data sources  Data collection form to data management system  Reliability  Recorded temperature or respiratory rates within acceptable ranges

10. Methods to ensure data quality include:  Data validity checks  Review procedures  Limited access to enter and edit data once entered in system Documentation of changes/edits to system data Error log Standard operating procedures (SOPs) can aid in ensuring quality of data collected Data quality cannot be “fixed” one time and then left alone  Will revert to poor quality if not controlled  Issues will change over time What is Data Quality (DQ)? (c ont)

11. Quality Control Strategy Steps  Determine parameters (data) to be controlled  Establish criticality and whether control is needed before (data entry), during (data storage) or after results (reporting) are produced  Establish a specification which provides limits of acceptability – For example - range of acceptable temperatures (x to x)  Produce plans for control  Specify how to achieve data quality, variation detection and removal  Install a ‘validation check’ at an appropriate point in the process  Collect and transmit data to location for analysis  Verify the results and diagnose causes of variance  Propose remedies and decide on the action needed How to develop a Data Quality (DQ) Strategy? http://www.transition-support.com/Quality_control.htm

12. Identify possible sources of poor data quality  Data capture and entry procedures  Data collection tools  Poor or lack of training  Equipment calibration  Data transfer from form to computer/site to site Identify the responsible person(s)  Data source - surveillance and laboratory sites  Data transfer/entry level Data Quality (DQ) Actions

13. Develop methods to address data quality issues  Review of CIF/Lab results by a second reviewer to check for missing information, etc.  Identification of data “errors” at data entry level (missing field, data inconsistency)  Procedure to query source (sentinel site/laboratory) to correct data “errors” identified (missing field, data inconsistency)  Random check of records  Refer back to data sources (e.g. CIF/Lab report) to correct errors originated at data entry level  Double data entry Data Quality (DQ) Actions

14. Standard operating procedures are a systematic way of collecting, managing and storing data Standard operating procedures (SOPs) should include:  Review and documentation of entire data collection system  Identification of people/team responsible for DQ  Definition of roles and responsibilities for all data collection personnel  Methods to identify and address data quality problems Data Quality (DQ) Standard Operating Procedures

15. Validity Check (Example) When you enter an invalid value, an error message prompts you to correct before allowing you to move to next item Validity checks help identify errors at data entry

16. Double Data Entry identify errors at data entry level Data Entry Check (Example)

17. Data Quality Verification Suggested indicators that can be used for surveillance systems  Completeness  % of patients enrolled over total screened that meet the case definition in use (screening and enrollment logs)  Example:  Total screened = 1000  # of patients enrolled = 1100  % enrolled = 110%  % of enrolled patients with CIF  % of enrolled patients with laboratory results  % of available CIF fully completed  % of completeness for key variables

18. Data Quality Verification (cont)  Timeliness  % of CIF sent to central level within a defined time period  % of Specimens sent to central laboratory within a defined time period  % of CIF entered in the database from reception within a defined time period  % of laboratory results available from reception within a defined time period  Example:  Total lab results = 1000  # lab results available within 7 days = 200  % available within 7 days = 20%  % of laboratory results sent to site from testing within a defined time period

19. Remember!!! Understand the data and why you are collecting Collection forms should collect data you will use Define data rules and variable options Document process and ensure everyone is aware and understands process Develop SOPs Data quality problems can occur at many points in the data collection process To control data quality, you must control it at many different points If not controlled, data may become inaccurate and begin to hinder its usefulness

20. Questions???

21. For more information please contact Centers for Disease Control and Prevention 1600 Clifton Road NE, Atlanta, GA 30333 Telephone, 1-800-CDC-INFO (232-4636)/TTY: 1-888-232-6348 E-mail: cdcinfo@cdc.gov Web: www.cdc.gov The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention. THANK YOU National Center for Immunization & Respiratory Diseases Influenza Division