2018-09-20 Metadata Management Meeting Notes

Laura E Daniels, Felix Hemme, Charlotte Whitt

Charlotte Whitt

Ann-Marie presented some slides concerning work that has been done on the Data Import so far.

Link to the PPT: https://drive.google.com/open?id=1iU9UKe73BaBUQ7h4jamy6A_HOJJbeYnT

• The app has been named "Data Import". The decision was to keep the name as generic as possible at least for the time being. This app deals with Acquisitions, Cataloging, and Inventory data.
• Data Import Process:
  1. Obtain File
  2. Apply a Job that contains match rules and actions rules. This job could be new or stored. The rules, both match and action, can be built with stored pieces.
  3. CRUD in Various FOLIO Modules will occur.
  4. Load Report (stats, errors, review log)
• Scope of the first version
  • Retrieving files automatically - No
  • UI to manage the data import process - Yes
  • Matching & Action profiles - Yes
  • Gathering and sequencing - Yes
  • Parsing the files - Yes
  • Create, Overlay/Update/Merge, Delete in Acquisitions, Cataloging, and Inventory.
  • Preview feature - Yes
  • In-app import reports/logs: success & errors - Yes
  • Types of FOLIO Records Affected (from PowerPoint). Note: Storage is the "Source of Truth". A record stored here cannot be accessed directly. It will be accessible via MARCcat. If the bibliographic data is coming from MARC, then the inventory record can only be edited by MARCcat.
    
    
    Duke (via chat): "We imagine a clear/easy way to move from Inventory to MARCCat for those 'simple' edits. Permissions pending."
    
    Jacquie pointed out that the place for editing will be easily accessible, no matter which app is the starting point.
    
    Items: An incoming MARC bibliographic record may create and item. Editing an item record will have to happen in Inventory.
    
    Acquisitions - The order lines and invoice lines could be affected by the import
    
    Duke (via chat): "

    "For Discovery, all data is availalble for an API/app to harvest and then display. That is, any inventory, MARC, order, circ, etc., etc. We asked this question of a developer in the recent past. Wayne said that all data is harvest-able."

• Types of Records
  • MARC
  • EDIFACT Invoice Files
  • MARC Holdings
  • MARC Authority
  • Delimited/flat files
  • Order API data - This will enable FOLIO to build an order from a connection to a vended system.
    
    
• MARC Details
  • Binary, XML, JSON can all be loaded
  • UTF-8 or MARC-8 can be ingested. Everything will be stored as UTF-8 in FOLIO
  • Workflow slides:
    
    
    
    
    
  • Duke question: "Ann-Marie, does this process apply to MARC authorities, as well? Just different storage?" - Ann-Marie - "Yes"
  • Christie: Will the users be able to create match rules for the API? Ann-Marie - To start it will be a "fixed mapping". Ultimately it will be configurable at the institution level.
  • Duke (via chat):

    "Will we be able to batch load item data (that is, separately from the approval plan process you used as your example where you load your bib and then create instance/holding/item)." Ann-Marie - You should be able to load item data as part of the bib record import. The bib record may not need to be saved. Once the information is harvested, the rest of the incoming MARC can be deleted.

• Authority records will only be in MARCcat to start. Future versions will allow the authority records to affect the Inventory app.

• Lisa (TAMU) via chat: "

  "So what happens in the Inventory when/if we delete a record from the MARC Bib storage?"

• This could be used for a single record import, but preferrably would be a more simple solution in MARCcat. The interim solution may be to use this Data Import
• "Rollback the load" - Interim solution is to have a detailed preview
• Scheduler, Queueing/Re-sequencing the loads - This will not be available at the start, but planned for later.
• Export - This will start development after Data Import is closer to completion.
• Large migration is not

In chat:

Charlotte: Hi Lisa, there will be no UI for the MARC Bib storage So deletion of a record will happen in e.g. MARCcat, or in Inventory if the record is not maintained in MARCcat

From Lisa TAMU to Everyone: 12:48 PM
"Note: Not being able to easily load single records into FOLIO (from OCLC) will be problematic for us. Our workflow is such that a cataloger creates an original record in OCLC, exports to local system, and then does local holdings, end processing, and sends the item to the stacks."

References:

• Wiki Page, Google Drive Folder, Slack channel
• Epic in JIRA
• No user or backend
• Folio Development Teams Home (See MARC4J Overview in particular)

Comments:

Jacquie said that the single record import is necessary for Duke, as well as Lisa from TAMU at the beginning.

Michael Knarr

Theodor Tolstoy (One-Group.se)