Presentation on theme: "Batch changes in Voyager an overview of the possibilities, processes and tools GUGM 2014 Adam Kubik, Clayton State University Susan Wynne, Georgia State."— Presentation transcript:
Batch changes in Voyager an overview of the possibilities, processes and tools GUGM 2014 Adam Kubik, Clayton State University Susan Wynne, Georgia State University
Caveat emptor ●The Strawn tools are very powerful and do lots of things ●Require a working ODBC connection to your database ●Operate directly on records in your database -- There is no undo button! ●Some tools are well documented, some not so well ●We don’t pretend to have tested, used or understand every possible option and setting ●All of these tools can be run in preview or test mode. Use it and start small
VgerSelect ●An alternative or complement to Access Reports ●Can output results as either: o Tab-delimited files of selected elements, or o Records in MARC or XML format (to work with in MarcEdit, etc.)
Filtering with VgerSelect ●You can also ask VgerSelect to filter its own output, but... o Jobs might run for a while o Fewer options than Excel ●If you want to play with the data in multiple ways it is faster to output all the data once and use Excel to manipulate it
Bib Delete Reports ●Problems--any records the program could not delete (usually will be suppressed) ●List of records modified and what was done (suppressed, unsuppressed, deleted) ●Report--total number of records read ●List of OCLC numbers of deleted records, in case you need to delete holdings in WorldCat
Recent uses of Bib Delete at GSU ●Suppressed most print gov docs records for a weeding project o Needed Location Changer for another part of this project ●Deleted obsolete Duke ebrary records due to change of platform ●Deleted a group of ETD records that had been re- harvested with better mapping & new OCLC control nos.
Recent uses of Bib Delete at Clayton State ●Suppressed Films on Demand titles from OPAC using vendor supplied sets of MARC records representing titles that had been removed from the collection ●Removed sets of locally bulk imported records when something went awry with the load (e.g., MFHD records not created as expected) ●For these jobs, we just used the set of bib records as the input file and the option to match the 001 in the vendor set against the 035 in Voyager
Location Changer ●Like Bib Delete, can (un)suppress or delete MFHDs, items and bibs ●Can also automate multiple changes to other data elements in MFHD and item records ●Can work on records one-by-one or in batches
Selecting records to change ●Select records to change by using criteria in the records: ○MFHD or item location (perm or temp) ○bib record type ○item status ○call number range ●Or, input a text file of barcodes, item IDs, MFHD IDs, or bib IDs
Selecting records to change Use record criteria Use an input file
Use in a weeding project at Clayton State ●Specify records with an input file of scanned barcodes generated by student assistant as items are boxed ●Run job to change to permanent location (in MFHD) to withdrawn, suppress holdings, and reset the item status to withdrawn
Use of Location Changer in weeding project ●If the last item attached to a MFHD is suppressed, the MFHD is also suppressed ●If the last MFHD attached to a bib is suppressed, the bib is also suppressed ●Location changer also generates a report of OCLC numbers from suppressed bib records so that holdings can be batch deleted from OCLC as well
Use of Location Changer in weeding project ●Since Location Changer can’t adjust MFHD holdings statements (866) -- or create new MFHDs -- some cleanup is still required ●Titles with complex holdings will need to be adjusted manually if volumes on a MFHD were only partially withdrawn
Recent uses of Location Changer at GSU ●Flipped a subset of the obsolete Rare location to a new Treasures location ●Flipped the rest of the Rare items to general Special Collections location ●Applied a temp location to a group of items being digitized ●Suppressed print MFHDs only for gov docs bibs with multiple locations/formats attached
URL Changer ●Change URL stems ●Change or delete 856 $x, $y, $z and/or $3 ●Works on URLs in both bibs and MFHDs!
Recent uses of URL Changer at GSU ●Changing proxy prefix ●Changing URL stem when the name of our institutional repository changed
Recent uses of URL Changer at Clayton State? ●Adding proxy prefixes and other URL changes made using a different method: validation rules in Cataloger’s Toolkit ●URL Changer is less flexible but much more user friendly
Strawn’s RDA Conversion tool ●Use to make batch changes to pre-RDA access points in bib (or authority) records o Changes are the same as those made during the “Phase 2” changes to the LC authority file ●Can also generate reports on other access points that may require review
Changes that can be made to access points ●Expanding abbreviations (“arr.” “acc.” “unacc.” “Dept.”) ●Expanding or deleting “O.T.” and “N.T.”, as required ●Replacing “violoncello” with “cello” ●Replacing “Koran” with “Qur’an” ●Changing “Selections” in $a or $t to “Works. $k Selections” ●Shifting $k “Selections” before $l and $f ●Adding or removing parentheses for certain instances of $c in personal name access points ●Expanding abbreviations of months ●Replacing “fl.” and “ca.” with “active” and “approximately” ●Replacing “b.” or “d.” with open hyphenated dates ●Expanding dates (e.g., changing “1090 or 91” to “1090 or 1091”)
Changes that can be made to access points ●The RDA Conversion tool allows some “options” for access points… ●But you will probably want to follow standard practice
Didn’t want to change dates in subject access points.
Reports generated for review ●Access points that might not be RDA compliant, or that need human review to appropriately correct o Musical ensemble terms in $m o “Polyglot” or “&” in $l o “Libretto” or “Text” in $s o Treaties o Conferences o Personal name access point $c terms that might be improper ●Access points flagged due to possible typos, or errors in subfield coding
Record Reloader ●An alternative to bulk import with webadmin for overlaying existing Voyager records ●Can use VgerSelect to output a MARC file, make changes, and then overlay with Record Reloader
Considerations for options & settings in Strawn programs ●Do you need to limit to one or more owning libraries? o Select the owning library/ies you’re interested in ●When making changes to the database, do you want the changes to be reflected in the Universal Catalog? o Choose the appropriate “happening”/ “cataloging” location
GSU’s experience with Global Data Change ●GSU and GSU Law have run a total of 3 successful data change jobs in 2013 o change 049 GLLM to GLLO o add 007 fields to DVD records lacking them o delete an obsolete 590 note from selected microform records ●GDC is not currently working for us! o “record export failed”--error logs o have tickets in with USG and Ex Libris
Basic steps for GDC ●Create a record set ●Create one or more data change rule sets ●Create a data change rule set group with one or more rule sets o keep it simple! ●Preview ●Run data change job o everything works until we run jobs--they fail
GDC preview of a “replace string with string” example
Cool things about GDC ●Save record sets, rule sets, rule set groups for reuse ●Multiple ways to create a record set: o From a list of record IDs previously identified o From a search of the same indexes as in the cataloging client o Scan an existing record set or the entire database for desired criteria (set up scan rules & run scan jobs within GDC) ●Preview
Not so cool things about GDC ●IT DOESN’T WORK (for us *right now*) ●Creating rules is not intuitive ●Takes some trial and error (this is why you preview) ●Mysterious “validation errors” o Missing 005? o Corrupted or obsolete data in LDR/008? o These errors don’t cause the job to fail--these records are just skipped in a successful job
●Validation rules are extremely versatile at making batch changes to bibs and MFHDs ●Like GDC, validation rules will make changes directly to your database ●Like GDC, this method can be unintuitive...
Using custom Validation Rules ●Rules are contained in plain text files that can be edited in a text editor ●Rules have their own peculiar syntax ●Cataloger’s toolkit comes with a default set of canned validation rules ●You can add new rules to the file or create a supplemental file
Sample Validation Rules ●A rule to add the specified 007 to every bib record that lacks an 007: #4=BDFMPSU 007! F ●A rule to perform a find and replace in every MFHD 856 $u: #4=H 856/u F
Defining the records to be changed ●Rules are normally run on sets of records specified by bib ID or MFHD ID in a text file ●These files can be generated with some other tool, such as VgerSelect
Examples of simple Toolkit validation changes ●Batch changes to e-resource vendor records o Adding and/or fixing 006-008 fields in bibs o Adding and/or fixing GMDs o Adding genre headings o Adding and/or fixing 007-008 fields in MFHDs o Standardizing SMD in 300 $a and deleting 300 $c ●Added proxy prefixes to our NetLibrary URLs
More complex projects using Toolkit validation rules ●Copied classification portion of call number from bib to MFHD for our ebooks ●Added local call numbers in the form “Streaming Video (Title)” to our streaming video bibs and MFHDs ●Adding genre headings to our feature films based on data compiled from IMDB by a student assistant
IMDB feature film genre heading project ●Used VgerSelect to extract bib IDs and all 655 fields from our VHS and DVD records ●Used Excel to filter by “Feature films” heading (1246 resulting bibs) ●Student assistant searched for titles in IMDB and added genre information to spreadsheet
●Used Excel to filter for particular genres ●Then filtered out records that already had the corresponding genre heading ●Ran a validation rule on the remaining bib IDs to add the corresponding heading, such as: #3=BDFMPSU 245 F
Possible uses of MarcEdit ●Edit a file received from a vendor or other source before batchloading ●Use the Connexion Bib File Reader plug-in to work with a group of WorldCat records ●Use the WorldCat APIs to search, update holdings, create or replace records, and more ●Extract a MARC file from Voyager with VgerSelect, make changes, and overlay
Recent uses of MarcEdit at GSU ●Clean up ETD records harvested from the IR into WorldCat before loading into Voyager o I have 15+ MarcEdit steps, but can pre-set up and save most of them in Task Lists! ●Add 9XX field(s) to most vendor batchloads to identify different e-resource sets, e.g., o SFX MARCit o YBPeBookApproval
ETDs & MarcEdit ●Create a Connexion local save file of harvested records since last Voyager load ●Use Connexion Bib File Reader plug-in to bring into MarcEdit ●Basic cleanup, including: o add 007 o delete unwanted fields generated during harvesting o add a 970 o RDA Helper to add 33X o create local headings for the discipline using Swap Field
=LDR 00000cam 22000003u 4500 =001 868162197 =005 20140226122043.0 =008 140214s2012 xx om 000 0 und d =040 \\$aGSU$cGSU$dGSU =042 \\$adc =049 \\$aGSUU =100 1\$aBrien, Spencer T. =245 10$aThree Essays on the Formation and Finance of Local Governments$h[electronic resource]. =260 \\$bScholarWorks @ Georgia State University$c2012-01-06T08:00:00Z =500 \\$aapplication/pdf =502 \\$aThesis / Dissertation ETD =653 \\$aproperty tax =653 \\$acontract cities =653 \\$alocal government =653 \\$atax relief =653 \\$agovernment outsourcing =653 \\$atax exemption =655 \4$atext =786 08$nPublic Management and Policy Dissertations =856 40$uhttp://scholarworks.gsu.edu/pmap_diss/37$3Item Resolution URL$xThis 856 field was generated using the WorldCat Digital Collection Gateway$yView online$iPut this Resolution URL in a web browser to view this item. =856 40$uhttp://scholarworks.gsu.edu/cgi/viewcontent.cgi?article=1036&context=pmap_diss =029 0\$aGSU$boai:scholarworks.gsu.edu:pmap_diss-1036$chttp://scholarworks.gsu.edu/do/oai/ publication:pmap_diss$tDGCNT Harvested ETD record “before” MarcEdit
=LDR 00000cam 22000003i 4500 =001 868162197 =005 20140226122043.0 =007 cr =008 140214s2012 gau om 000 0 eng d =035 \\$a(OCoLC)ocn868162197 =040 \\$aGSU$cGSU$dGSU$beng =042 \\$adc =049 \\$aGSU1 =100 1\$aBrien, Spencer T. =245 10$aThree Essays on the Formation and Finance of Local Governments. =260 \\$aAtlanta, Ga. :$bGeorgia State University,$c2012. =300 \\$a1 online resource. =336 \\$atext$btxt$2rdacontent =337 \\$acomputer$bc$2rdamedia =338 \\$aonline resource$bcr$2rdacarrier =500 \\$aapplication/pdf =502 \\$aThesis / Dissertation ETD =653 \\$aproperty tax =653 \\$acontract cities =653 \\$alocal government =653 \\$atax relief =653 \\$agovernment outsourcing =653 \\$atax exemption =690 \\$aDissertations $xPublic Management and Policy. =786 08$nPublic Management and Policy Dissertations =856 40$uhttp://scholarworks.gsu.edu/pmap_diss/37 =029 0\$aGSU$boai:scholarworks.gsu.edu:pmap_diss-1036$chttp://scholarworks.gsu.edu/do/oai/ publication:pmap_diss$tDGCNT =970 \\$aETDBulkMarch2014 Added 33X fields with RDA Helper (requires correct 007) Created a 690 from the 786 using Swap Field & Edit Subfield Created 035 from 001 using Swap Field & Edit Subfield Added 260 $a, edited $b with Edit Subfield, stripped extra characters from $c with a regex in Find/Replace Added basic 007 using Add/Delete Field Deleted unwanted 856 subfields with Edit Subfield Added 970 with Add/Delete Field (so I can identify records if things go wrong!) Harvested ETD record “after” MarcEdit
Don’t fear the RegEx ●Ask the MarcEdit list for help (or search the archives) ●My First RegEx! o I had a bunch of dates in 260 $ c like this: $c2013-05-01T07:00:00Z $c2011-05-07 o Total length varied, but I just wanted to keep the first four characters
MarcEdit recommendations ●Ask for help on the MarcEdit listserv ●Save your original file before making changes ●Save frequently after *successful* edits ●There is an Undo button, but it will only undo your last change ●MarcEdit is great when you need to apply multiple changes to a group of records ●Just. Try. It. (Trust me!)
An alternative to GDC ●Output a MARC file from Voyager with VgerSelect ●Make changes with MarcEdit ●Overlay the records with Record Reloader ●(VgerSelect & Record Reloader require ODBC drivers)
Putting it all together a Special Collections project at GSU
Splitting the Rare location ●Special Collections staff separated the *really* rare items from the existing Rare location for a new Treasures location ●Most items in Rare were integrated into the existing general Special Collections (Spec.) location ●~1000 records to Treasures ●~1200 records to Spec.
Lots of reports on the Rare location ●VgerSelect or Access Reports to generate bib IDs for all records with a Rare MFHD attached ●VgerSelect to retrieve the existing 590 notes and more ●Access Reports to find *all*the attached MFHDs o Some had existing Spec. copies or multiple Rare, requiring some extra attention to 590 notes o I created a new table of the bib IDs & ran query to retrieve MFHDs ●Also needed MFHD ids & suppression status o need Access Reports to get suppression status
VgerSelect + MarcEdit + Record Reloader to update bibs ●Output a MARC file with VgerSelect using the appropriate list of bib IDs ●MarcEdit o Edit Subfield to change location codes in 049 o For Rare to Spec lists, Add/Delete Field to add a standard 590: Special Collections copy: Rare Book Collection. ●Record Reloader to overlay bibs
Bibliographic maintenance projects ●Use VgerSelect and Excel to locate invalid, obsolete, contradictory or missing data, such as: o “N/A” (or other invalid codes/typos) in 008/Lang o Sound recordings, videorecordings, maps, microforms lacking 007s o records with a 240 $l and no 041 (or an 041 0X) o incompatible 008/DtSt, 008/Date, 260/264 $c combinations ●List of possible maintenance projects at: http://www.carli.illinois.edu/products-services/i- share/cat/Cat-maintpriority http://www.carli.illinois.edu/products-services/i- share/cat/Cat-maintpriority
Bibliographic maintenance projects ●RDA conversion of descriptive fields o Gary’s RDA conversion program o MarcEdit RDA Helper ●Upcoming changes to music subject headings o Musical forms will be moved from LCSH to LCGFT o LCMPT terms in 382? ●Changes needed to facilitate data migration?
Strawn’s RDA Conversion tool ●Can be used to make RDA style batch changes to data other than access points, as well o Generate 336, 337 and 338 fields from data in record -- or -- force 336, 337 and 338 to certain values for a given set of records o Expand certain abbreviations in certain descriptive fields (see Toolkit documentation) o Remove GMDs o Reformat a 502 into RDA style ●Tool reports when selected data cannot be converted o For example, if data such as 007 and SMD are missing, ambiguous or contradictory
MarcEdit RDA Helper ●Similar capabilities to Gary’s RDA Conversion tool ●Will also generate the new 34X fields ●Option to generate GMDs on RDA records if desired ●GSU uses to add 33X to ETDs o limited testing on other files o explore using on selected or all vendor batchloads?
Recommendations ●Start with easy programs, like Bib Delete or Location Changer ●Test on small groups of 5-10 records ●Get comfy with reports! (Access Reports and/or VgerSelect) o Generate lists of record numbers to work with o Check your results
Recommendations ●*Always* preview/test ●Give yourself time to experiment and learn ●Ask questions… o MarcEdit-L o Voyager-L for Strawn programs
Coming attractions in Voyager 9 ●Some Location Changer functionality incorporated into Pick and Scan ●Options to delete, suppress or unsuppress added to GDC ●Option to export record sets and files added to GDC ●See Voyager product update from ELUNA and Voyager 9 release notes (requires login to Ex Libris Documentation Center)Voyager product updateVoyager 9 release notes
Links ●Gary Strawn’s programsGary Strawn’s programs o His excellent documentation is also available here o RDA Conversion program RDA Conversion program ●Voyager-LVoyager-L ELUNA archivesELUNA archives (login required) o good source of presentations on GDC and Strawn programs MarcEdit download MarcEdit download ●MARCEDIT-L discussion listMARCEDIT-L discussion list ●Terry Reese’s YouTube videos on MarcEditTerry Reese’s YouTube videos on MarcEdit ●Fear No Longer Regular ExpressionsFear No Longer Regular Expressions
Contact Adam Kubik email@example.com Susan Wynne firstname.lastname@example.org https://www.facebook.com/susan.wynne.94 https://gsu.academia.edu/SusanWynne
Your consent to our cookies if you continue to use this website.