Data Migration- Two Separate Databases Michael Odawa.

Overview Migration is the movement of data from one database to one master database. What happens if there are instances where there are some people present in both databases? Do you continue with the migration? How do you identify these individuals? Merging (in this context) refers to the process of identifying matching records in two separate databases and consolidating the information for these matching records into one record.

Importance of Merging Data. The data contained in both databases are different and they may belong to the same patient. By merging the data, it becomes easier to analyse the patient level data since they will be in one consolidated file. Clinically, it is advisable to have all patient records in one file. Fragmented data storage of patient records results in easier tracking of the patient’s medical history. This assists in programme management since the total number of patients who need medical assistance will be centralized and as such this reduces the possibility of duplicated efforts in making drug orders and other program interventions for these patients for those patients who exist in both databases. To monitor and evaluate the efficiency of linkage to care for patients who are identified as HIV positive by the HCT Research Assistants in the field into the HIV Care system offered by AMPATH.

Summary of Activities Involved. FLOWCHART SHOWING THE MIGRATION PROCESS Set up the server to contain the Port Victoria HCT data Run a dedup query in the HCT Port Victoria server and in the AMRS Merge the matching records in the HCT Port Victoria Rerun the dedup report to generate the Port Victoria Import table Run queries to obtain the AMRS and Port import tables Create a flat file for analysis and run RecMatch on this Conduct manual review on the true matches Identify the uuids for the true and false matches Migrate demographics for the false matches Merge the demographics for the true matches Classify the gray area patients into true matches and false matches Migrate the encounters for all the patients Migrate the observations for all the patients

RecMatch This is a process by which the demographics from the two databases are joined together and then processed to get potential true matches, true matches and false matches. The tool that does this disaggregation is a stand-alone Java patient matching file. org.regenstrief.linkage.gui.RecMatch.jar

How does it work? The process involves measuring the agreement rates of specific fields between true and false matches, then calculating the likelihood of a given pair being a true match, and assigns a score based on how well the fields match. Various algorithms can be used to do the matching, eg the Jaro-Winkler Estimation, Levenshtein Least Common Subsequence or the Exact Matching algorithm.

When not in Use

When in use

Summary of Softwares Used FLOWCHART SHOWING THE MIGRATION PROCESS Set up the server to contain the Port Victoria HCT data Run a dedup query in the HCT Port Victoria server and in the AMRS Merge the matching records in the HCT Port Victoria Rerun the dedup report to generate the Port Victoria Import table Run queries to obtain the AMRS and Port import tables Create a flat file for analysis and run RecMatch on this Conduct manual review on the true matches Identify the uuids for the true and false matches Migrate demographics for the false matches Merge the demographics for the true matches Classify the gray area patients into true matches and false matches Migrate the encounters for all the patients Migrate the observations for all the patients OpenMRS MySQL SAS, R Statistical Programming Language, STATA, Excel OpenMRS- Yank Module

Blocker to completion of exercise. FLOWCHART SHOWING THE MIGRATION PROCESS Set up the server to contain the Port Victoria HCT data Run a dedup query in the HCT Port Victoria server and in the AMRS Merge the matching records in the HCT Port Victoria Rerun the dedup report to generate the Port Victoria Import table Run queries to obtain the AMRS and Port import tables Create a flat file for analysis and run RecMatch on this Conduct manual review on the true matches Identify the uuids for the true and false matches Migrate demographics for the false matches Merge the demographics for the true matches Classify the gray area patients into true matches and false matches Migrate the encounters for all the patients Migrate the observations for all the patients Help Required……

Sites to be reviewed Port Victoria- HCT-AMRS (Done apart from obstacles mentioned earlier) Port Victoria – HCT-PHCT-AMRS (Ongoing- Currently doing the Manual Data Review- hopefully to be done in 2-3 Weeks) Chulaimbo- HCT-PHCT-AMRS (Next to be tackled) Teso- HCT-PHCT-AMRS All other HCT-PHCT Data stored in stand alone databases.

Going Forward This process needs to be completed for the baseline HCT Data collected from the various sites from 2009 – Going forward, this data should ideally be sent directly to the AMRS but this can only be done successfully if the baseline data is already in the AMRS (for reasons mentioned in the importance section).

Thanks!! Questions??