MODQM-311 Spike: Investigate feasibility of local authority source support

Owned by Shans Kaluhin
Last updated: Apr 12, 2023 by Pavlo Smahin
4 min read

MODQM-311 - Getting issue details... STATUS

Goals

  1. Determine complexity of allowing for configuration of 001 generation (prefix + starting value) 
  2. Adding the ability to select the desired authority source file at point of creation of new record (so the system would know how to generate the 001 per the configuration)

Approach 

  1. Create new endpoint to retrieve HRID by authority file id 
  2. Implement sequence number generation mechanism for authority local files
  3. On the UI user will select file name. And before creation UI should put to 001 field value from the HRID endpoint. 

Tickets

OverviewApproachModuleStory PointsJira ticket
Validation for authority local files
  1. New parameter 'startNumber' for authority source file creation is presented 
  2. Implement AuthoritySourceFileHridManager with Validation:
      - (if source = 'local' then codes.size = 1) else throw exception
      - (if source = 'local' and startNumber = null) set startValue = 1
      - prevent user to update "start with" and "prefix" after file creation
mod-inventory-storage3 sp


MODINVSTOR-1049 - Getting issue details... STATUS
APPROVED

Implement sequence number generation mechanism for authority local files (Sequences Approach)





  1. New endpoint to catch-up authority current hrid information is presented
  2. Extend AuthoritySourceFileHridManager to work with sequences
      - Create sequence for authority local file after validation
      - Drop sequence for authority local file after deletion
  3. Return hrid on request 
      - Retrieve next value from localFile sequence 
      - Format next value to NaturalId(HRID) format
mod-inventory-storage5 sp

MODINVSTOR-1050 - Getting issue details... STATUS

DECLINED

Implement sequence number generation mechanism for authority local files (Table with counter Approach)


  1. New endpoint to catch-up authority current hrid information is presented
  2. New table to work with authority files HRID presented
  3. Extend AuthoritySourceFileHridManager to work with table
      - All transactions should be done with SERIALIZABLE isolation
      - On request return current hrid value and increment it. 
      - Format value to NaturalId(HRID) format
mod-inventory-storage5 sp
Authority file id validation
  1. mod-quick-marc in scope of authority creation should convert 001 field value to UUID and retrieve available hrid for source file id.
  2. Replace 001 field value from authorityFileId to "prefix + hrid"
  3. If 001 value doesn't comply with UUID then skip it. And leave the original value
mod-quick-marc2 sp

MODQM-322 - Getting issue details... STATUS

Local authority file selection
  1. User should have the ability to look-up for local authority file
    (As permanent-location look-up for Holdings)

  2. UI should put authorityFileId to 001 field
ui-inventory

UXPROD-3910 - Getting issue details... STATUS

Ability to store authority local files

Settings application UI preview:

localauthoritydefinition.PNG

Endpoints:

POST: /authority-source-files/{id}/hrid
Description:  Endpoint returns current hrid value and increment it. 

Authority Source File HRID Example 
{
    "id": "cb58492d-018e-442d-9ce3-35aabfc524aa",
    "prefix": "loc",
    "hrid": "000000001"
}

POST: /authority-source-files
New query parameter: startNumber (default = 1)

startNumber parameter schema 
"startNumber": {
  "description": "The number from which to start generating HRIDs",
  "type": "integer",
  "minimum": 1,
  "maximum": 99999999999
}


Sequence number generation mechanism for authority local files

Sequences Approach

There is a similar approach that works already in the mod-inventory-storage. It generates Hrid numbers for Instances, Holdings, and Item records, and can be configured from UI (Settings → Inventory → Hrid handling);
The problem here is that we will create new sequences in Runtime for each new authority file. (format: "hrid_authority_local_file_{prefix}_seq") 

PROS

CONS

Auto increment Separate sequence should be created for each authority file
Atomic transaction. Safety works with asynchronous work



Table with counter Approach

New table should be presented

hrid_authority_source_file 
CREATE TABLE IF NOT EXISTS hrid_authority_source_file
(
  authorityFileId UUID NOT NULL,
  hridSeq         INT  NOT NULL,
  CONSTRAINT hrid_authority_source_file_pkey PRIMARY KEY (authorityFileId),
  CONSTRAINT hrid_authority_source_file_fkey FOREIGN KEY (authorityFileId)
    REFERENCES authority_source_file (id)
);


To work with table it's better to use SERIALIZABLE Isolation:
@Transactional(isolation = Isolation.SERIALIZABLE)

PROS

CONS

One table for all authority filesAll work with select and increment should be done manually.

Harder work with transactions. 
All transactions should be fulfilled in SERIALIZABLE Isolation level. So it can reduce performance



Select local authority file on the UI

  • User should select local file for 001 field
  • We also need to warn the user that field 001 may be overwritten by field 010.


Q/A section

QuestionStatusAnswerNotes
How many files can be per one tenant? Do we expect millions of files? Can we restrict amount of authority files per tenant?

OPEN

We can likely expect less than 10 local authority files to be created. Typically, records are created for local files for use cases such as:

  • A record does not yet exist in an external/national database (for example, a university library may have items in their collections written by alumni for which the Library of Congress does not have a name authority record)
  • A record exists in an external/national database but the headings do not align with the mission/objectives of the institution (such as headings for indigenous peoples)



Do we expect authority records matching with local authority file during DI process?

OPEN

Yes, it is possible for the library to want to update an existing local authority record via DI and would need to match on the existing file.