1. Converting Local Portal Data to MySpace Anne L. Highsmith, Consortia Systems Coordinator & Prabha Acharya, Programmer Analyst II Texas A&M University

2. Converting Local Portal Data to MySpace ELUNA 2007 Points to Cover Background Data Conversion Issues Testing on a Cloned Database Conversion Process Results and Future Plans

3. Converting Local Portal Data to MySpace ELUNA 2007 Background (1) TAMU had a local Electronic Resources Locater (ERL) function to list ejournals and databases. Users searched ERL and clicked on links to access eresources Users could store the links in their MyPortal record on the library’s website

4. Converting Local Portal Data to MySpace ELUNA 2007 Background (2) Metalib went into production at TAMU in January 2006 as Search Now. TAMU had discussions with Ex Libris staff as to how MyPortal links could be converted to Metalib First proposal, for Ex Libris staff to do a major conversion, not feasible Decided to wait for Metalib 4.0 and X-Portal APIs to store data to MySpace

5. Converting Local Portal Data to MySpace ELUNA 2007 Rude awakening! Fall of 2006, Ex Libris staff informed TAMU there would be no API to store to MySpace Alternatives Margaritas? Convert them ourselves?

6. Converting Local Portal Data to MySpace ELUNA 2007 Conversion options Assumed conversion would have to be: User-initiated Title by title Discovered a way do the conversion directly, without involving the end user

7. Converting Local Portal Data to MySpace ELUNA 2007 Data Conversion Issues Patrons Journals Databases

8. Converting Local Portal Data to MySpace ELUNA 2007 Data Conversion Issues - Patrons How to match the MyPortal patron with the Metalib user? MyPortal patrons identified by uin (University ID Number) Our Metalib implementation uses uin as user login Needed to match the uin with the z312_source_id

9. Converting Local Portal Data to MySpace ELUNA 2007 Data Conversion Issues - Journals How to match MyPortal journals with Metalib? Basic journal data stored in z11 table Wrote Perl script ( journals.pl ) to extract journal identifier and issn from z11 Used MS Access to match MyPortal data, keyed by uin & issn, with Metalib data User journal data stored in z312_jnl_001 Wrote Perl script ( journals_by_patron.pl ) to extract z312_source_id (uin) and journal identifier from Metalib user record in z312 table

10. Converting Local Portal Data to MySpace ELUNA 2007 Data Conversion Issues - Databases How to match MyPortal databases with Metalib? During creation of Metalib IRDS, stored the ERL key in the IRD cataloger note Loaded IRD extract into MS Access Matched Metalib user extract with IRD extract Basic database data stored in z122 table Wrote Perl script ( databases_by_patron.pl ) to extract z312_source_id (uin) and Metalib identifier (TEX #)

11. Converting Local Portal Data to MySpace ELUNA 2007 Data Conversion Issues - Final As a result of matching ERL data and Metalib data, created a file that contained 1 or 2 entries for each user whose data had to be converted: Tab-separated record beginning with uin and list of database id numbers Tab-separated record beginning with uin and list of journal id numbers

12. Converting Local Portal Data to MySpace ELUNA 2007 Testing on a Cloned Database

13. Converting Local Portal Data to MySpace ELUNA 2007 Testing – Cloning the database (1) TAMU has a test Metalib DB on separate server Cloned production DB to test DB so could test in a realistic environment mpsync - documented in Appendix D in the System Administrators Guide & Knowledge Base entry #1240. exp_current_library & imp_current_library, documented in 7.8 in the SysAdmin Guide Further doc – “Working with Test/Failover Installations of MetaLib”

14. Converting Local Portal Data to MySpace ELUNA 2007 Cloning – Export from Production Created files on prod for export to test Run mpsync to copy IRDs TAMU has approx. 4500 IRDS Elapsed time – 58 sec. Run export current library for vir00 TAMU has 32,000+ user records Elapsed time – 1 hr. Run export current library for jnl01 TAMU has approx. 44,000 journal records Elapsed time – 2:25 min.

15. Converting Local Portal Data to MySpace ELUNA 2007 Cloning – Import into Test Import IRDS created by mpsync Elapsed time – 3 hrs. 688 new records added Matched existing keys. Appeared to create some duplicates Import User Data Elapsed time – 40 min. Drops & re-adds tables Import Journal Data Elapsed time – 3 min. Drops & re-adds tables

16. Converting Local Portal Data to MySpace ELUNA 2007 Conversion Process Quality control File preparation Loading new patrons Convert databases and ejournals

17. Converting Local Portal Data to MySpace ELUNA 2007 Conversion – Quality Control Wrote a Perl script ( pre_migration.pl ) to collect stats and set up QC measures. For each user it reports: Users who don’t have Metalib record # of databases and journals already in Metalib # of databases and journals to convert # of users that have more than maximum number of databases or ejournals

18. Converting Local Portal Data to MySpace ELUNA 2007 Conversion – File Preparation Testing demonstrated that It takes about 3 seconds per resource to be migrated. With about 13000 resources that would be about 10 hours So, wrote Perl script ( breakup_myportal_file.pl ) to break the input file into several files of about 250 records each.

19. Converting Local Portal Data to MySpace ELUNA 2007 Conversion – Loading new patrons Since TAMU uses external authentication rather than patron load, some older MyPortal patrons not in Metalib database Using file created during QC stage, create_xml_patron_load.pl formats user records for Metalib patron load utility Set expiry date to 1 year in future Based on a review of records created on that date, approx. 65 have since logged in.

20. Converting Local Portal Data to MySpace ELUNA 2007 Conversion - Convert databases and ejournals (1) Run databases_by_patron.pl to get current lists from user records for comparison

21. Converting Local Portal Data to MySpace ELUNA 2007 Conversion - Convert databases and ejournals (2) Using save_to_bookbag.cgi, for each user Log in to Metalib Send request to Metalib pds to get pds handle Send request to Metalib, using just obtained pds handle, to get session id Construct url of the form http://metalibhost.edu/V/sessionid?pds_handle=p dshandle&func=find-db-add- res&resource=resourceid

22. Converting Local Portal Data to MySpace ELUNA 2007 Conversion - Convert databases and ejournals (3) Set session id in cookie For each database Check to see if it duplicates db already stored If not, store db to user’s clipboard For each journal Store to user’s record

23. Converting Local Portal Data to MySpace ELUNA 2007 Conversion - Convert databases and ejournals (4) Do QC to make sure that all of the appropriate databases and ejournals are stored Exit the session Error logging revealed that for some users, 1 database or ejournal was not stored – these were re-run as an added input file

24. Converting Local Portal Data to MySpace ELUNA 2007 Final results Processing all files took most of the week, off and on Each file, limited to 250 records, took about 1 hour to process Only post-production problem had nothing to do with conversion – ezproxy error. Once users have logged into library portal, can now go directly to My Databases or My Ejournals list within Metalib

25. Converting Local Portal Data to MySpace ELUNA 2007 Future plans Conversion of Voyager My Bookbag to e- shelf

26. Converting Local Portal Data to MySpace ELUNA 2007 Contact & download info Anne L. Highsmith Consortia Systems Coordinator Texas A&M University hismith@tamu.edu URL to download perl code, cgi, etc. http://libraryasp.tamu.edu/users/ahighsmi/prese ntations/save_to_bookbag.zip