Settings - Reference data decisions and load sequence for migration

Scope

This is not intended to be a complete introduction to reference data. It is aimed at providing a list of reference data decisions that should be made prior to migration and documenting the order in which records should be migrated from legacy systems. Also, because this list is based on Texas A&M University's (TAMU) migration, it is not exhaustive, as TAMU did not migrate all possible record types from its previous system.

Data records – Prerequisite reference data/data records, other instructions

Data record

Application

Prerequisite reference data or other data records (settings/... entries are relative to FOLIO baseurl)

Schema or other documentation

Notes

instancesinventory
  • Reference data
    • MARC bib-to-Instance map (optional; if desired to change default marc-to-instance mapping for all MARC records)
    • settings/inventory instances section (optional)
    • settings/inventory/ instances, holdings, items section statistical code types, statistical codes (optional)
https://github.com/folio-org/mod-source-record-manager/blob/master/mod-source-record-manager-server/src/main/resources/rules/rules.jsonInstitutions may wish to identify records with statistical codes for a variety of reasons. Example: grouping records supplied by a given ebook vendor.
holdingsinventory
  • Reference data:
    • settings/tenant-settings/location-institutions (required)
    • settings/tenant-settings/location-campuses (required)
    • settings/tenant-settings/location-libraries (required)
    • settings/tenant-settings/servicePoints (required)
    • settings/inventory Holdings section (optional)
    • settings/inventory/ instances, holdings, items section statistical code types, statistical codes (optional)
  • Data records
    • instances (required)
Institutions may wish to identify records with statistical codes for a variety of reasons. Example: grouping records supplied by a given ebook vendor.
itemsinventory
  • Reference data
    • settings/inventory/loantypes (required),
    • settings/inventory/materialtypes (required),
    • settings/inventory Items section (optional)
    • settings/inventory/ instances, holdings, items section statistical code types, statistical codes (optional)
  • Data records:
    • instances (required)
    • holdings and their prerequisites (required)
Schema: https://s3.amazonaws.com/foliodocs/api/mod-inventory-storage/p/item-storage.html#item_storage_items_postInstitutions may wish to identify records with statistical codes for a variety of reasons. Example: recording multiple item statuses from a previous system
usersusers
  • Reference data:
    • settings/users/groups (required)
    • settings/users/addresstypes (required)
Schema: https://github.com/folio-org/mod-user-import/blob/master/ramls/schemas/userdataimport.json


proxiesusers
  • Data records: users (required)
Schema: https://s3.amazonaws.com/foliodocs/api/mod-users/p/proxiesFor.html#proxiesfor_post
loans
  • Reference data:
    • settings/tenant-settings/servicePoints
    • settings/calendar
    • settings/circulation/loan policies
    • settings/circulation/request policies
    • settings/circulation/notices policies
    • settings/circulation/overdue fine policies
    • settings/circulation/lost item fee policies
    • settings/circulation/patron-notices
    • settings/circulation/rules
  • Data records:
    • users and their prerequisites (required)
    • instances, holdings, items and their prerequisites (required)

For migration purposes, calendar begin and end dates must include the earliest loan to be migrated and the latest possible due date based on loan policy.

Also for migration, institutions may choose to use storage endpoints or business logic endpoints to migrate loans. See Considerations for migrating loans for a comparison and contrast of these approaches and links to the relevant schema definitions.

requests (recall/hold/page)requests
  • Data records:
    • instances, holdings, items and their prerequisites (required)
    • users and their prerequisites (required)
    • loans and their prerequisites (required)

https://s3.amazonaws.com/foliodocs/api/mod-circulation/p/circulation.html#circulation_requests_post

https://s3.amazonaws.com/foliodocs/api/mod-circulation-storage/p/request-storage.html#request_storage_requests_post

Institutions have differed in their approach to migrating requests, some choosing to use business logic modules and others choosing storage modules. Both schema urls are listed for that reason.
fees and fines
  • Reference records:
    • settings/users/owners (optional)
    • settings/users/feefinestable (optional)
  • Data records:
    • users and their prerequisites (required)
    • instances, holdings, items and their prerequisites (required)

https://s3.amazonaws.com/foliodocs/api/mod-feesfines/p/accounts.html#accounts_post

https://s3.amazonaws.com/foliodocs/api/mod-feesfines/p/feefineactions.html#feefineactions_post

  • The settings/users/owners and settings/users/feefinestable must be set up if the institution chooses to migrate existing fees/fines as "not automatic" i.e. manual.
  • Institutions have taken different approaches to migrating fees and fines, some creating only account records and others creating account records plus feefineaction records


contacts (organizations)organizations
Schema: https://s3.amazonaws.com/foliodocs/api/mod-organizations-storage/p/contact.html#organizations_storage_contacts_post
  • urls in the contact record must begin with 'http://' or 'https://', otherwise the record will be rejected as invalid.
organizationsorganizations
  • Reference data:
    • settings/organizations/category (optional),
  • Data records: contacts (optional)
Schema: https://s3.amazonaws.com/foliodocs/api/mod-organizations-storage/p/category.html#organizations_storage_categories_post
  • urls in the organizations record must begin with 'http://' or 'https://', otherwise the record will be rejected as invalid.
purchase ordersorders
  • Reference data: 
    • settings/tenant-settings/location-institutions (required)
    • settings/tenant-settings/location-campuses (required)
    • settings/tenant-settings/location-libraries (required)
    • settings/tenant-settings/servicePoints (required)
    • settings/acquisition-units required if orders/financials must be separated/permissioned differently for different users within the same tenant
  • Data records:
    • instances (optional)
    • organizations (vendors) (required)
    • finance structure (optional)
Schema: https://s3.amazonaws.com/foliodocs/api/mod-orders/p/order.html#orders_composite_orders_post
piecesreceiving
  • Data records
    • purchase orders
https://s3.amazonaws.com/foliodocs/api/mod-orders/p/pieces.html#orders_pieces_post

"Pieces" records may be created to contain loose issues that have been checked in but not yet bound and therefore lack an item record in inventory. The piece record requires a uuid link to the titleId that appears in the po line, but that uuid doesn't exist until the purchase order and its lines are POSTed to the /orders/composite-orders endpoint, so it's necessary to post the order po lines and get the title ids before creating the piece records.

As an alternative to creating piece records, some institutions have chosen to update the "receivingHistory" data element in the inventory holdings record to contain information about loose issues.

Reference data (elements are grouped by the url in the settings app and then listed alphabetically)


Data element (relative to base url for instance)



Description



Prerequisites



Schema



Notes


settings/calendarsets the normal opening and closing hours of a given service point as well as exceptions to normal hours.settings/tenant-settings/servicePointshttps://s3.amazonaws.com/foliodocs/api/mod-calendar/p/calendar.html#calendar_periods__servicepointid__period_postAs of the Honeysuckle release, circulation staff have experienced problems editing calendars in the settings UI. Institutions may wish to fill out the hours in a json record and post the calendar to the appropriate endpoint.
settings/circulation/fine-policiesdefine overdue fine rates, maximum overdue fine, and related settings
https://s3.amazonaws.com/foliodocs/api/mod-feesfines/p/overdue-fine-policy.html#overdue_fines_policies_post
settings/circulation/loan-policiesused in circulation module to define basic loan/renewal/recall configuration. If the local tenant wishes to migrate retrospective loans to FOLIO and link them to existing policies and rules, loan-policies must be created before migration.
https://s3.amazonaws.com/foliodocs/api/mod-circulation-storage/p/loan-policy-storage.html#loan_policy_storage_loan_policies_post
settings/circulation/lost-item-fee-policydefines whether item "ages to lost" and when, default replacement fees and fines, and related policies
https://s3.amazonaws.com/foliodocs/api/mod-feesfines/p/lost-item-fee-policy.html#lost_item_fees_policies_post
settings/circulation/notice-policiesused in circulation module to define which types of notices are sent in various conditions and what their frequency will be.
  • settings/circulation/patron-notices
https://s3.amazonaws.com/foliodocs/api/mod-circulation-storage/p/patron-notice-policy.html#patron_notice_policy_storage_patron_notice_policies_post

settings/circulation/patron-notices

defines the html templates used to compose circulation notices
https://s3.amazonaws.com/foliodocs/api/mod-notify/p/patron-notice.html#patron_notice_post
settings/circulation/request-policiesdefines which types of request (recall, hold, page) are allowed within a given policy
https://s3.amazonaws.com/foliodocs/api/mod-circulation-storage/p/request-policy-storage.html#request_policy_storage_request_policies_post
settings/circulation/rulesused in circulation to determine combination of  policies that apply to a given circulation situation. 
  • settings/circulation/loan-policies
  • settings/circulation/request-policies
  • settings/circulation/notices-policies
  • settings/circulation/fine-policies
  • settings/circulation/lost-item-fee policy
https://s3.amazonaws.com/foliodocs/api/mod-circulation/p/circulation-rules.html#circulation_rules_putRules can be created through the Settings/Circulation/Circulation rules UI or by "putting" a text file to the endpoint. Local tenants with complex circulation rules may find it more convenient to update rules via the text file, although that method requires substituting uuids for text labels. When PUTting the text file to the /circulation/rules endpoint, do not include the uuid in the file.
settings/inventory/callNumberTypesused in inventory holdings records to identify type of call number. 
https://s3.amazonaws.com/foliodocs/api/mod-inventory-storage/p/call-number-type.html#call_number_types_postAs of Honeysuckle release, the default reference data does not include a call-number-type for 'blank' as defined by MARC 852 indicator 1. Local tenants may need to add the 'blank' value for migration purposes.
settings/inventory/formatsused in inventory instance records to identify record format. Identified in the inventory instance record UI and Settings/Inventory UI as "formats". If instance record is created via Data Import of a MARC record, format is assigned by mapping the MARC field 338 subfield $b. Default list provided in FOLIO based on RDA carrier type codes and names. 
https://s3.amazonaws.com/foliodocs/api/mod-inventory-storage/p/instance-format.htmlInstructions on how to modify mapping rules are available at: How to work with mapping rules.
settings/inventory/holdingsTypesused in inventory holdings records to identify type of holdings record.
https://s3.amazonaws.com/foliodocs/api/mod-inventory-storage/p/holdings-type.html#holdings_types_postIf the local tenant wishes to base their holdings-types codes on USMARC holdings records leader/06, then they will have to create an additional holding type for 'unknown'.
settings/inventory/holdingsNoteTypesused in inventory holdings records to identify types of note included in the record. 

https://s3.amazonaws.com/foliodocs/api/mod-inventory-storage/p/holdings-note-type.html#holdings_note_types_post

Can be used, for instance, to define a type of note that contains "Latest volume or edition in alternate location".
settings/inventory/identifierTypesused in inventory instance records to define type of identifier. 
https://s3.amazonaws.com/foliodocs/api/mod-inventory-storage/p/identifier-type.html#identifier_types_postLocal tenant may wish to define specific identifier type for USMARC field 001 to uniquely identify previous system record key.
settings/inventory/resourcetypesused in inventory instance records to identify record type. Identified in the inventory instance record UI and Settings/Inventory UI as "resource type". If instance record is created via Data Import of a MARC record, instance-type is assigned by mapping the MARC field 336 subfield $b. Default list provided in FOLIO based on RDA content type codes and names. If the MARC record lacks field 336 subfield $b, the instance record is assigned a default instance-type of 'unspecified'.
https://s3.amazonaws.com/foliodocs/api/mod-inventory-storage/p/instance-type.html#instance_types_post
settings/inventory/loantypes

used in inventory item records as one of several elements that determine appropriate loan policy. 


https://s3.amazonaws.com/foliodocs/api/mod-inventory-storage/p/loan-type.html#loan_types_postLocal tenant may wish to delete default loan types provided with FOLIO before defining their own.
settings/inventory/materialtypesrequired element used in item records to identify the item's type of material. Can be used as a defining data element in circulation-rules. 
https://s3.amazonaws.com/foliodocs/api/mod-inventory-storage/p/material-type.html#material_types_postLocal tenant may wish to delete default material-types provided with FOLIO.
settings/inventory/StatisticalCodeSettingsvalues for locally-defined codes that can be used in main inventory record types, instances, holdings, items
  • settings/inventory/statisticalCodeTypes
https://s3.amazonaws.com/foliodocs/api/mod-inventory-storage/p/statistical-code.html#statistical_codes_postInstitutions may wish to identify records with statistical codes for a variety of reasons. Example: grouping instance or holdings records supplied by a given ebook vendor, identifying an item record data element migrated from a previous system that has no direct counterpart in FOLIO.
settings/inventory/statisticalCodeTypestypes of locally-defined codes that can be used in main inventory record types, instances, holdings, items
https://s3.amazonaws.com/foliodocs/api/mod-inventory-storage/p/statistical-code-type.html#statistical_code_types_post
settings/organizations/category (organizations)used in organization records to identify classes of data, such as addresses, emails, urls, etc. 
https://s3.amazonaws.com/foliodocs/api/mod-organizations-storage/p/category.html#organizations_storage_categories_postLocal tenant may wish to delete default categories provided with FOLIO, using the Settings/organizations/category page in the UI.
settings/tenant-settings/addressesused in purchase orders for Bill to and Ship to addresses.
https://s3.amazonaws.com/foliodocs/api/mod-configuration/p/config.html#configurations_entries_post
settings/tenant-settings/location-campusesused in inventory holdings & item records to identify 2nd level in location hierarchy.
  • settings/tenant-settings/location-institutions
https://s3.amazonaws.com/foliodocs/api/mod-inventory-storage/p/locationunit.html#location_units_campuses_post
settings/tenant-settings/location-institutionsused in inventory holdings & item records to identify top level in location hierarchy.
https://s3.amazonaws.com/foliodocs/api/mod-inventory-storage/p/locationunit.html#location_units_institutions_post
settings/tenant-settings/location-librariesused in inventory holdings & item records to identify 3rd level in location hierarchy.
  • settings/tenant-settings/institutions
  • settings/tenant-settings/campuses
https://s3.amazonaws.com/foliodocs/api/mod-inventory-storage/p/locationunit.html#location_units_libraries_post
settings/tenant-settings/location-locationsused to identify shelf locations in inventory holdings & item records.
  • settings/tenant-settings/location-institutions
  • settings/tenant-settings/location-campuses
  • settings/tenant-settings/location-libraries
  • settings/tenant-settings/servicePoints
https://s3.amazonaws.com/foliodocs/api/mod-inventory-storage/p/location.html#locations_postLocation codes and location names must be unique within a tenant. For example, if the "main" library and "science" 2 each has a reference section, those locations cannot use the location code 'ref' even though the upper levels of the hierarchy identify the location uniquely. One would have to be 'mainref' and the other 'sciref'.
settings/tenant-settings/servicePointsused primarily in circulation apps to identify points where circulation actions take place. Local tenant may wish to delete default service points provided with FOLIO. As of Q4 2019, the local tenant will apparently have to do the delete using a curl command or other script that deletes from the API endpoint, as there doesn't appear to be a delete function for these in the Settings/Tenant/Service Points UI. Also, although the options to print slips at a specific Service Point appear to be selected by default, they do not become active unless Pickup location is set to 'Yes'.
https://s3.amazonaws.com/foliodocs/api/mod-inventory-storage/p/service-point.html#service_points_post
settings/users/addresstypesused in user records to identify type of address. Local tenant may wish to delete default address types provided with FOLIO. 
https://s3.amazonaws.com/foliodocs/api/mod-users/p/addressTypes.html#addresstypes_post
settings/users/feefinestabledefines manual (as opposed to automatic) charges that can be created in the system. Users can set default amount charged, default charge notice, and default action notice.
  • settings/users/owners
https://s3.amazonaws.com/foliodocs/api/mod-feesfines/p/feefines.html#feefines_post
settings/users/groupsused in user records to identify patron groups. 
https://s3.amazonaws.com/foliodocs/api/mod-users/p/groups.html#groups_postAs of the Honeysuckle release, the system loads reference data for groups, so local tenants will probably want to delete those groups before creating their own.
settings/users/ownersdefines how service units are grouped to create a fee/fine "owner". 
https://s3.amazonaws.com/foliodocs/api/mod-feesfines/p/owners.html#owners_postFee/fine "manual charges", "payment methods", and "transfer accounts" are differentiated by fee/fine owner