Presentation on theme: "MarcEdit: Doing more, but faster"— Presentation transcript:
1 MarcEdit: Doing more, but faster Terry ReeseGray Family Chair for Innovative Library Services
2 Making your metadata work for you Finding ways to use MarcEdit to merge and manipulate existing metadata in various formatsi.e., working with XML formats, delimited formats, Excel, AccessDealing with data in multiple charactersets as we transition to a Unicode worldLearning how to automate repetitive tasks, and understand what editing functions are available to youLeveraging webservices like OCLC WorldCat to provide automatic classifications
5 Marc Tools Built-in functions MarcBreaker – Tool used to convert MARC records to the MarcEdit mnemonic formatMarcMaker – Tool used to convert MarcEdit mnemonic format to MARCMARC=>MARC21XML – converts MARC to MARC21XMLAutomatically converts data from MARC-8 to UTF8MARC21XML=>MARC – converts MARC21XML to MARCDoesn’t automatically convert data from UTF8 to MARC8 – will leave data in UTF8
6 MARC Character Conversions Supports moving between any known Windows Characterset and MARC8.Can be run from the Breaker/Maker – or as its own standalone utility
7 MARCSplit/MARCJoinUtility used for joining large sets of MARC data to a single fileUtility used for splitting large MARC record sets into smaller files
8 Batch Record Processor Allows MarcEdit to process “lots” of files.Files can be processed against an entire folder’s contents or by file typeCan utilize any built-in or derived XML Function transformation
9 MarcEdit and bad records Two MARC breaking algorithmsStrict MARC algorithmLoose breaking algorithmLoose algorithm can heal MARC records (sometimes)Structural errorsMissing field or record markers
10 Delimited text translator Translates Tab, comma, pipe, Excel (Office ), Access (Office ) files into MARCCan save translation mapsCan create constant data
11 Delimited text translator Options Wizard-like interfaceSupports Unicode data (in excel or delimited file)Joining (relating) fieldsEditing global 008/LDR
12 Delimited Text Translator: Mapping format Map to: Field + subfieldIndicators: Indicator valuesTerm Punct.: Trailing punctuationArguments – Joining defined items (select and right click on items)Ability to save templates
13 Common Joining techniques When would I mark a field as repeatable?By default, when the Delimited Text translator encounters two like subfields on the same field, it creates a new field. For example: column 1: This is a note column 2: This is a note 2 if I mapped column 1 500$a and column 2 to 500$a, by default, MarcEdit would generate the following output: =500 \\$aThis is a note =500 \\$aThis is a note 2However….
14 Common Joining techniques When would I mark a field as repeatable?If I need to have multiple, like subfields on the same field, for example, like a subject field – we would mark the field as repeatable: column 1: Geology column 2: Oregon column 3: Corvallis If these fields were not marked as repeatable, the output would look like: =650 \0$aGeology$zOregon =650 \0$zCorvallis However, if these fields were marked as repeatable, the output would look like: =650 \0$aGeology$zOregon$zCorvallis
15 MARC Conversions This is really the heart of MarcEdit All utilities and functions interact with the MARCEngine in some fashion.
16 MarcEdit Crosswalking model MARC21XMLEADFGDCMODSMARCDublin Core
17 Finding and Contributing Crosswalks In MarcEdit 5.6, an option was added to allow users to search for crosswalksCurrently, these are crosswalks I or LC have createdHopefully, community members will submit crosswalks for inclusion into the registry
19 Harvesting Metadata MarcEdit includes a builtin OAI harvester Allows for direct XML=>MARC translationsAllows for custom modification of XSLT translation tables.
20 Harvesting Metadata Required data Host name: i.e.,Metadata TypeNatively supports MARCXML, Dublin Core, OAIMARC and MODSOptions to support conditional harvests, raw data harvests, and resumptive harvests.
24 Configuring New Paging Set in the Options dialog
25 Paging ExampleIf you load the full file, or turn the preview mode off
26 Editing MARC MarcEditor Supports a number of global editing functions: Edit Subsets of recordsFind/Replace functionalityGlobally Add/Delete MARC fieldsGlobally Edit Subfield dataConditionally add/remove field dataGlobally Edit Indicator dataGlobally Swap field dataRecord DeduplicationRecord SortingCall Number GeneratorMacrosBest way to think of the MarcEditor is like notepad for MARC. It has been designed to work specifically with MARC data.
27 Editing MARC – Find/Replace Works like a normal Find/Replace in most Textpad utilities.Unlike most Textpads, Replace supports UTF-8 (when working with UTF-8 files) and regular expressions.Replace all works great for handling regular find/replace operations but can also be used to:Change field tagsUsing regular expressions to move subfield information from on subfield to anotherUsing regular expressions to do complex find/replace operations.
28 Editing MARC – Find AllFind all function was designed for use with the Paging modeAllows users to find any text across all pagesGenerates a jump list that can be used to find individual records for editReplace all works great for handling regular find/replace operations but can also be used to:Change field tagsUsing regular expressions to move subfield information from on subfield to anotherUsing regular expressions to do complex find/replace operations.
29 Jump to Jump to…record: Jump to…page: Allows you to jump to any recordsJump to…page:Allows you to jump to any page
30 Editing MARC – Global Add/Delete Field Globally add fields to all MARC recordsAllows users to set insertion position.Globally delete fieldsAllows global deleteAllows conditional deleteSupports Regular ExpressionsThe function is primarily useful if you have a field that needs to go into every record. For example, OSU receives aggregator records for EBSCOHost and we insert a text string into every record so that we can easily identify these records using listing tools within our ILS system.Another example: in our ILS system, we use a 949 field to pass command-line options to the MARC loader. When doing database maintenance operations, I can automatically add a single 949 field to all records to define the load table and common arguments to be used when loading the record.
31 Editing MARC – Modifying subfield data Allows for the modification of variable MARC field subfield data (MARC fields >10)Allows for the modification of control field data by position or range of positionsAllows users to prepend and append data to subfields.Allows users to change subfield tagging.
32 Editing MARC – Modifying subfield data Allows users to insert new subfields and define subfield placement.Allows users to move field data from one field to another.Supports:UTF-8 with UTF-8 filesRegular ExpressionsAdding new subfields.
34 Editing MARC – Swapping Fields Swap parts of MARC Fields or entire MARC fieldsDefine field, indicator and subfields to move.Can move field data and delete the original field or clone the field data and move the clone to the new location.Can add data to an existing field.
35 Character Conversions within the MarcEditor MarcEditor allows users to convert character data between different charactersets.
36 Fixing Boo-boos MarcEdit’s Special Undo Allows you to step back one global change.
37 Sorting Fields MarcEdit provides multiple sorting types: Control NumberSorts record position within the fileTitleAuthorCall Number0xx FieldsSorts the 0xx fields within individual records (does *not* change record position within a file)All FieldsSorts all fields within individual records (does *not* change record position within a file)Custom SortSorts all defined fields within individual records (does *not* change record position within a file)
38 Record Deduplication MarcEdit provides a simple dedup tool that can: Dedup on a defined control field (any field)Dedup on a transaction field (or using an additional transaction field)OutputRemoves all duplications and saves the duplications to a filePrints just unique items within the file (i.e., those without a duplicate pair)
39 Field Counts Field Count Provides a quick count of fields Report of subfields used within a particular fieldDetailed reports of all fields/subfields used within a fileset.
40 Material Type Report Material Type Report Reports number of records by material typeBreaks down material type by sub-typesUtilizes the Leader, 008 and GMD to determine format types
41 Task Automation Tool Stacking Operations Task automation provides a way for non-programmers to create defined task lists that can then be executed automaticallyThe different between a task and a macro is that MarcEdit tasks essentially function like the user was calling specific functions within MarcEdit.Anything that you can do in the MarcEditor, you can automate as a task.
42 Task Automation Managing Tasks Task management works like macro managementYou canCreate new tasksClone tasksRename tasksDelete tasksEdit tasks
43 Task Automation Demo Additional Information: Youtube: Introduction to task automation:Introduction to new task automation functions:
44 OCLC Classify ServiceMarcEdit can leverage OCLC WorldCat to generate call numbers automatically for filesFields used:001010$a$z020$a$z022$a$z024$a$z1xx$a776$w$z
47 MarcEdit 5.9+ AACR2->RDA macros Merge Record Enhancements Low-hanging conversions to support batch data processingMerge Record EnhancementsAdding more data points and customized merge fieldsMore Automation supportAbility to turn Edit shortcuts into Automation tasksBatch OAI HarvestingCreate jobs that you can schedule and have automatically run for youBatch Set HoldingsUsing either crappy z39.50 or OCLC’s yet to be publically released API for holdings settings.
48 Getting Help Call/write me: Ask the list: email@example.com MarcEdit ListServ