Page tree
Skip to end of metadata
Go to start of metadata
RolePersonComments
Solution ArchitectConfirmed
Java Lead
UI Lead
Product OwnerStephanie Buck


UXPROD-2272 - Getting issue details... STATUS  - Connect SRS and SMRE for MARC Bibs (SMRE work)

Requirements

Simple MARC Records Editor (SMRE) must be able to access an existing SRS MARC Bib via a button in the Instance (e.g. Edit MARC) and edit the MARC Bib. Once the edits have been made via SMRE, then:

  • The edits are saved to the SRS MARC Bib (including updates to the size of the MARC record in MARC Leader bytes 00-04): (a) the corresponding updates are made to the MARCcat MARC bib automatically (once MARCcat and SRS are integrated); (b) the corresponding updates are made to the Inventory Instance automatically.
  • Current scope for SMRE would be editing existing SRS Bib records only. Not creating new MARC records from Instances or creating new MARC records from scratch.
  • SMRE will use a simple UI, rather than a text editor.
  • The only expected validation is the general structural validation that SRS currently does. Any more rigorous MARC validation would happen in MARCcat or external to FOLIO.
  • If displayed in the SMRE, data in MARC 001 and 999 ff fields would be visible, but not editable
  • Once SRS is expanded to include MARC holdings records, the SMRE may be expanded to allow for editing of MARC holdings also.
  • Since record update is performed asynchronously, SMRE must provide information about current updating status and last successful update.


Approach & Design

quickMARC application should be implemented as separate micro service in its own module mod-quick-marc. It allows to extend number of use cases in future, for example, for Data Import could involve editing MARC as part of those workflows. Two endpoints must be implemented to meet the requirements for the MARC editor:

MethodPathProvided permissionsRequestResponseDescriptionNotes
GET

records-editor/records?instanceId={instanceId}

records-editor.records.item.getNA200 OK (QuickMarcJson)Get MARC record in quickMARC-compatible representation

{instanceId} - corresponding instance id

PUTrecords-editor/records/{parsedRecordId}records-editor.records.item.putQuickMarcJson202 ACCEPTEDUpdate an existed MARC record{parsedRecordId} - UUID of a parsed MARC record

Module permissions: records-editor.all

The first endpoint is responsible for the delivery and transformation of the MARC record to the client in the required format. The second one is necessary to update the record after corresponding validation and transformation.

(A) Get MARC record

  1. Client makes request to quickMARC GET /records-editor/records/instanceId={instanceId}
  2. quickMARC proxies request to source-record-storage: GET /source-storage/records?query=externalIdsHolder.instanceId==={instanceId} / CM: GET change-manager/parsedRecords?instanceId={instanceId}
  3. quickMARC transforms request to appropriate format (accepted by UI)
  4. quickMARC returns response 200 OK

(B) Update MARC record

  1. Client makes request to inventory PUT /records-editor/records/{id}
  2. quickMARC transform record from UI-compatible format to marc-json and updates to the size of the MARC record in MARC Leader bytes 00-04
  3. quickMARC updates corresponding instance in CM
  4. quickMARC returns response 202 ACCEPTED

Integration with SRS/SRM

quickMARC should operate with two endpoints to retrieve and update existed MARC records:

1) GET request to retrieve existing record (errors should be processed on SRS/CM)

2) PUT to update existing (errors should be processed on CM)

Full QM-CM/SRS diagram is presented below. More details about SRM/SRS can be found on SRM/SRS Design Document.

Key Components

UI compatible schema for MARC record representation

PropertyTypeDefaultRequiredNotes
parsedRecordIdUUIDNYMARC record Id
parsedRecordDtoIdUUIDNYParsedRecordDto identifier
supressDiscoverybooleanfalseYFlag indicates if the record is displayed during a search
instanceIdUUIDNYIdentifier of external instance
leaderstringNYLeader field value
fieldscollection<object>NYAt least 3 items: 001, 008, 999f are required
updateInfoupdateInfoNYUpdate information sub-object - holds data about record update


updateInfo

PropertyTypeDefaultRequiredNotes
recordStatestringACTUALYState of source record in QM workflow
updatedDatedate-timeNN

Date and time of last successful record update

Converter QuickMarcJson ↔ ParsedRecordDto

Converters should be implemented as separate components implementing Converter<ParsedRecordDto, QuickMarcJson> and Converter<QuickMarcJson, ParsedRecordDto>. Since it is necessary to be able to integrate with SRS and CM, both of services should be implemented. Each of these services uses its own object types for MARC records representation: record.json (SRS) and parsedRecordDto.json (SRM). The quickMARC editor uses its own type quick_marc_json.json to represent the corresponding MARC record. The connection between the required IDs of is shown in the following diagram:

It should be noted that IDs population is responsibility of service but not converter because service is type-specific unlike converter. Converter operates only with ParsedRecord that doesn't contain information about its holder: record or parsedRecordDto.


Performance Testing

There are two different approaches for system load simulation:

  • Closed systems (control of the concurrent number of users);
  • Open systems (where you control the arrival rate of users).

Closed system are system where the number of concurrent users is capped. At full capacity, a new user can effectively enter the system only once another exits. On the contrary, open systems have no control over the number of concurrent users: users keep on arriving even though applications have trouble serving them. As far as it concerns FOLIO in general and the quickMARC in particular we deal with Open System. Therefore the most optimal load profile is gradually increasing number of concurrent users during fixed time period. For scenario with 1200 threads and the ramp-up period of 120 seconds, JMeter scenario will take 120 seconds to get all 1200 threads up and running. Each thread will start 1/10 (120/1200) second after the previous thread was begun. We tested both of quickMARC operations: GET MARC record by instanceId and PUT MARC record as sequential operations for each thread (user). Results of load tests are presented in the table below.

DateReportNotes

 

GET - direct integration with SRS;

PUT - simple stub.


Jmeter report generation: jmeter -n -t <TestPlan.jmx> -l <report.jtl> -e -o dashboard



https://github.com/marc4j/marc4j

*integration with SRS is planned by source-record-manager; API isn't ready yet  UXPROD-2273 - Getting issue details... STATUS


  • No labels

5 Comments

  1. Just a couple of comments.

    As we discussed and agreed it should be a separate microservice, not a part of Inventory. Inventory is not responsible for storing MARC records or any other types of SRS records in the future. Some of the use cases discussed for Data Import could involve editing MARC as part of those workflows (e.g. correcting errors). That would not be Inventory centric, nor even triggered from Inventory. So please correct your diagrams.

    The second point is that SRM/SRS will provide you MARC records with some additional metadata required to find an appropriate record to update when you send changes back to SRM. Just pay attention to it when designing data structures.

    1. Thanks Taras Spashchenko for clarifying that the quick MARC editor should not be inventory centric and not involve changing the public APIs of mod-inventory.

      I had noticed the intention to change mod-inventory in a couple of stories and wasn't sure about this approach. I hadn't had chance to raise it, given we are in the release week.

  2. Taras Spashchenko thank you!

    We will pay attention on mentioned by you point about needed metadata for SRS/SRM.

  3. After the releases are bedded down, Stephanie and I will get Firebird and Folijet together to discuss the quickMARC/SRS interaction. Important to remember that even though this is a separate app, it won't show in the app bar at the top of the FOLIO screen. It'll be involved from a specific Inventory Instance record.

  4. Viachaslau Khandramai it might make sense to create a "Cataloging" area in the "Tech Designs and Decisions" space and move this there