SPIKE: Update MARC authority records via Data import
MODDATAIMP-585 - Getting issue details... STATUS
Feature: UXPROD-2990 - Getting issue details... STATUS
Data Import Subgroup Requirements
Overview
Now the data-import is able to import Marc authority records, mod-inventory can generate Folio records, mod-search provides API for the search. In this story, we need to understand how to implement the Update for Marc authorities in data-import, what changes are needed to support the Search of the updated records via mod-search.
The Update is an action that is specified by the Action profile, so anytime the user needs to choose the Action profile if there is a need to update records. The Matching profile defines the types of incoming records and existing records, that need to be updated. Also, a Match profile contains the Match criteria, which specifies parameters for records comparison.
Reference page: Update SRS MARC Authority records via Data Import
Approach
MARC Authority update workflow in data-import
The user uploads the MARC file, chooses an Action profile & Matching profile on the UI application, starts the import process
| Step№ | Module | Task | DI Event |
|---|---|---|---|
| 1 | mod-data-import | Sends a batch of records to Kafka queue | DI_RAW_RECORDS_CHUNK_READ |
| 2 | mod-source-record-manager | Receives records, parses to the JSON, sends it to Kafka queue | DI_RAW_RECORDS_CHUNK_PARSED |
| 3 | mod-source-record-storage | Receives the event from SRM, Stores Authority records into the database, and returns the result back via Kafka queue. The SRS now can match records only by 001 and 999 ff $s, this logic is hard to expand. Matching 010 $a or other tags requires implementing a spike MODDATAIMP-583 - Getting issue details... STATUS | DI_PARSED_RECORDS_CHUNK_SAVED |
| 4 | mod-source-record-manager | Receives the event from SRS, creates JSON payload (containing parsed MARC, profile, mapping parameters) for further processing, and sends it to the queue | DI_SRS_MARC_AUTHORITY_RECORD_CREATED |
| 5 | mod-inventory | Receives the event from SRM, loads the Folio record that matches Marc tags, creates JSON payload with a matched Folio record, sends it to the queue. We have to implement AuthorityLoader on this step. The loader retrieves the Folio record from mod-inventory-storage that matches Marc tags 001, 010 $a, 999 ff $s , or use Folio record id | DI_INVENTORY_AUTHORITY_MATCHED DI_INVENTORY_AUTHORITY_NOT_MATCHED |
| 6 | mod-inventory | Receives the event from itself. Updates the existing Folio record. We have to implement UpdateAuthorityEventHandler on this step. The handler updates the existing Folio record by the matched record taken from the event payload | DI_INVENTORY_AUTHORITY_RECORD_UPDATED DI_COMPLETED |
| 7 | mod-inventory-storage | Receives HTTP PUT request with the record that needs to be updated, updates the record in the database, puts record to Kafka topic | |
| 8 | mod-search | Receives a record from Kafka topic on an update, deletes the existing record matching by id, inserts the new record |
Field protection is not needed at the beginning
User stories
| Module | Description | Jira | High-level rough est |
|---|---|---|---|
| mod-source-record-storage | Spike to extend matching | MODDATAIMP-583 - Getting issue details... STATUS | 5 |
| mod-data-import-converter-storage | Story to create default Action profile & Matching profiles | To be defined later | To be defined later |
| data-import-processing-core | Story to add new event types for the Update | MODDICORE-235 - Getting issue details... STATUS | 1 |
| mod-inventory | Story to implement the AuthorityLoader | MODINV-606 - Getting issue details... STATUS | 3 |
| mod-inventory | Story to implement the UpdateAuthorityEventHandler | MODINV-607 - Getting issue details... STATUS | 5 |
| mod-source-record-manager | Log the non-matched records update DataImportJournalConsumersVerticle and JournalParam | MODSOURMAN-653 - Getting issue details... STATUS | 2 |
| mod-source-record-storage | Set matched_id, external_id, and parsedRecord data | MODSOURCE-429 - Getting issue details... STATUS | 5 |
mod-source-record-storage: Update matching criteria for authorities to support 010 field
ui-data-import: Update Matching profiles to enable Authorities
ui-plugin-find-import-profile: https://github.com/folio-org/ui-plugin-find-import-profile does it require some work ???