Technical Designs and Decisions
Space shortcuts
Page tree
Skip to end of metadata
Go to start of metadata

THIS DECISION HAS BEEN ARCHIVED and replaced by Data consistency and message driven approach. While the examples here may be illustrative and useful for gaining additional context, no further discussions or decisions should take place here.


Status

COMPLETE

Stakeholders

Front-end and back-end devs of non-core modules that hold references to records in core-modules

Outcome

UI modules   

Created date

  

Owner

Zak_Burke

NOTICE

This decision has been migrated to the Technical Council's Decision Log as part of a consolidation effort.  See:   DR-000024 - Deletion of core-module records may leave dangling references from non-core modules


Context

  • Deletion of records by core UI modules is problematic because it may leave dangling references to deleted records in non-core modules. Core modules are at the bottom of the hierarchy, unaware of the modules that sit above them in the hierarchy; this prevents core modules from issuing queries to identify such references. This is succinctly, if frustratingly, captured in the PR discussion related to UITEN-128.
  • Even if we can resolve this in the UI (see notes, below) the possibility remains that direct API requests to delete records may leave dangling references in other parts of the system.

Decision

  • Yikes.

Notes

  • It may be possible to mitigate this in the UI by using the "handler" app-type to function as an event bus (since the UI bundle is deployed as a monolith, events published in one module could, potentially, be acted upon by another) but we don't actually have anything like this in production yet. We do have handler-modules, but don't have an established protocol for apps to issue their own events. 

JIRAs

Analysis and options

Basically, 2 main alternatives can be suggested for the provided context:

Alternative optionBrief descriptionPotential use cases
Cascade deletiondeleting a code-module record leads to all non-core-modules records referring to it must also be deleted
  • cascade deletion of outdated records
Soft deletiona core-module record is not being physically deleted but is being rather marked with a specific dedicated flag (e.g. is_deleted = true / false)
  • manual deletion made by end users

It looks that both options can be viable for FOLIO depending on particular use case.

It worth also noting that UICR-125 - Getting issue details... STATUS case is slightly different and is about dangling references while not deletion but moving of a core-module record; neither cascade nor soft deletion is applicable here. At the same time, this case and cascade deletion case have one great similarity - in both, non-core-modules are to be notified about a change in core-module. In this context, a robust notification channel with guaranteed delivery to transfer change events from one module (say, module-source) to another module (say, module-recipient) is required.  More details are on Data consistency and message driven approach # Notification channel.

Soft deletion option

More details regarding this option.

  • Q: It requires that every client understands that deleted records might be present and when the API will provide them and when it will not.
    • A: Sure, every client should know the rules of the game. Though once the rules will be agreed and shared, it's just a platform-wide knowledge
  • Q: Let's say a loan references an item, that item is then soft-deleted, when the loan is presented in the UI, would the item information be presented or not?
    • A: Yes, the loan is available, and the item information still can be retrieved by a single endpoint. At the same time, loan UI should be able to notify end user that the item has been deleted (via a warning message, or via a dedicated UI field, or via disabling of editing particular values received from the item

High level actions required for core / non-core modules are shown below:

Core moduleAnother module referencing core module
  • add an is_deleted flag to schema via Liquibase with false as a default value
  • Create - new records are to be created with is_deleted=false
  • Read
    • collection endpoint only returns not-deleted records
    • single endpoint can return both deleted and not-deleted records
  • Update - add a check for every update - update of a record with is_deleted=false is only allowed
  • Delete - implement soft-deletion with setting is_deleted to true
  • add an UI element showing if a record from code module is deleted or not, or disable editing of fields came from core module, or add a message to end user with the warning that a core module record this record is referencing to, is deleted
  • No labels

3 Comments

  1. Raman Auramau

    Zak_Burke - Have you considered such option as soft deletion? It's when a record is not being physically deleted from database leaving all those dangling references but is being rather marked with a specific dedicated flag (e.g. is_deleted = true / false).

    This approach does physically keep a record presented in database meanwhile there are no dangling references, and the is_deleted flag is to be handled properly on application level (e.g. the record should not be visible in UI, it's not allowed to be updated etc.)

    The key feature of this approach is a minimization of changes on data layer (so that no /minimal affect on data consistency is expected) and shifting / encapsulating a new logic to handle this on Data Access or even UI layer.

    If this seems to be a viable solution I can try to drive this forward working through additional details.

    1. Marc Johnson

      Raman Auramau

      Have you considered such option assoft deletion?

      Folks have talked about this as an option.

      It's when a record is not being physically deleted from database leaving all those dangling references but is being rather marked with a specific dedicated flag (e.g.is_deleted=true/false).

      How would this work in practice? Would the single record endpoint would still provide deleted records however the collection endpoint would not include them?

      This approach does physically keep a record presented in database meanwhile there are no dangling references, and theis_deletedflag is to be handled properly on application level (e.g. the record should not be visible in UI, it's not allowed to be updated etc.)

      This is one of the trade offs with this approach, it requires that every client understands that deleted records might be present and when the API will provide them and when it will not.

      The key feature of this approach is a minimization of changes on data layer (so that no /minimal affect on data consistency is expected) and shifting / encapsulating a new logic to handle this on Data Access or even UI layer.

      By encapsulated at the data layer, do you mean that the records would still be there, yet the API would not present them in either the collection or single record endpoint responses?

      If this seems to be a viable solution I can try to drive this forward working through additional details.

      I think one of the other trade offs with this approach is that it weakens the decision that something has been deleted, whilst that still needing to be considered by clients of the API.

      As an example. Lets say that a loan references an item, that item is then soft-deleted, when the loan is presented in the UI, would the item information be presented or not?

      1. Raman Auramau

        Thanks for you comments Marc Johnson. I added some answers and details regarding this option in text above. Let's sync during next Tech Leads meeting.