Action Items for Development

Folks,

We will shut down this page and collect all action items in this Backlog: Meeting Topic and Action Item Backlog - SysOps

This page is outdated as of 05/31/2019 and should not be edited anymore.

Priorities:

P1: Critical

P2: High - we can go live without it, but it hurts

P3: Medium

P4: Low

P5: Trivial

Item-IDTopic

Following

Discussion on

Description

Priority

JIRA-Ticket #Status

TAMU installation experiences03/01/2018Right now, difficult to build Stripes and run modules on different machines.
  • Okapi does not have to manage deployment, but can take module descriptors.
  • Possibly wind up constructing your own module descriptors, somewhat like DNS.
  • Still have to work this out in practice, have thought about but not worked it out.
  • EBSCO probably has most experience here.
    • Have their own CIC pipeline and register as code is committed,
    • regularly prune the list of descriptors.
    • Modify the module descriptor by appending a build or rev number, so can distinguish between builds.
    • kicking off their own Maven builds, have their own Docker images, not using much directly from the public Docker repo. Subscribed to git repos, commits kick off buids in Jenkins, can modify templates.
    • Probably not many sites hosting their own will be doing this workflow, but possible.
P2
is being all touched upon and hit - pretty much solved (Jason)

API-Consistency03/18/2018

has been addressed in Madrid to Nassib, needs specifications.

we can go without, but there is advantage in addressing it.



Should be logged in particular issues (Anton)
SYSOPS-001Documentation on Docker modules02/22/2018

Documentation belongs

a) in the README on github

b) on dockerhub

P2

There is a Ticket about that (Anton). Marc Stacy and David Crossley work on this. It is being worked on May/24/2019 

FOLIO-2003 - Getting issue details... STATUS

and FOLIO-2008 - Getting issue details... STATUS

SYSOPS-002Re-write LS-Tools in FOLIO04/12/2018

LS-Tools = A tool set at Cornell which allows technical services do library automation tasks through a common web interface.

It is essentially an ETL tool. It comprises cleanup jobs, export and import jobs. There is a specification written for each job that describes who owns it, how it is executed, what is needed, etc. The app also allows for searching of marc data with a highly granular selection capability. The app is a swiss army knife of technical services activities. It has meant a huge increase in productivity for technical services staff. Re-implementing this in Folio will be a challenge.

Some combination of the workflow engine and the marc batch loader could replace LS tools. The workflow engine is meant to be a platform to build these kinds of applications on.

P3
moved

Install-issues04/26/2018
  • an issue getting Okapi/Folio to connect to postgres in single node deployment 
  • use of a service discovery tool, rather than the current Okapi setup
  • Individual microservices not typically aware of system, except self
    1. Okapi is currently arbiter of microservices

    2. Application shouldn’t manage its own state

    3. Okapi has no sense of HA backend services

      1. Can’t tell it to spin up multiple copies of service

    4. Need backend module register itself with discovery service, at URL

      1. Okapi would talk to discovery service like etcd or console

      2. Would help with load balance abilities

  • Most orchestration tools assume service discovery tool, which Okapi doesn't currently have
      1. Proxied okapi request for circ data, what happens?

      2. How does Okapi know where that service URL is located?

      3. Okapi was designed to be agnostic to service discovery

      4. Wayne recommends current smoother path to use DNS based launch/deployment descriptors, rather than service discovery tool

        1. How much orchestration do we want Okapi to do?

        2. Difficult previously to have use-cases to give developers.

        3. What will it use other than URL for mod-circ?

  • We don’t want Okapi as orchestration tool, only a proxy; too dynamic
      1. Rather make Okapi work better w/ orchestration tools

      2. Wayne: Module registration is unavoidable in Okapi

      3. Give Okapi discovery service information ability

      4. Stephen: modules should be able to self-register to discovery service rather than posting to Okapi

      5. How do I know what I am supposed to run?

        1. Build front-end first, then back-end

        2. This is backwards

        3. However, in multi-tenant Front-end first may be more appropriate, according to Wayne

P2

UXPROD-748 - Getting issue details... STATUS

UXPROD-754 - Getting issue details... STATUS

UXPROD-755 - Getting issue details... STATUS

UXPROD-1429 - Getting issue details... STATUS

either being worked on or solved;

covered in Kubernetes Group.

Closed here.

SYSOPS-003Lightweight Ticketing App05/03/2018

APPIDEAS-4 - Getting issue details... STATUS

take "Trello" or something else - will be discussed in SysOps SIG on May 31st.
SYSOPS-004Action Items from the data migration subgroup05/29/2018

At Wolfcon we worked out a plan for how to architect data migration. Implementing institutions would extract and transform their legacy data into a json representation that conformed to the json schema for the apis of the different modules, or, optionally, for bibliographic marc data, into marcxml. FOLIO developers will write a high-speed loader which would then move the json data, or parse and move the marcxml data, into the appropriate Folio modules. User loads may be an exception to this scheme because the requirements for bulk loading from on-going data feeds may be sufficiently similar to the requirements for the initial migration of data that a single loader may be able to serve both purposes. Wayne is currently exploring a suitable design for a high-speed loader. This project should have a very high priority.

Storage modules will need to accommodate bulk operations. Initial bulk loaders for migration vs. ongoing.

P1

UXPROD-756 - Getting issue details... STATUS

will be covered in the DM subgroup
SYSOPS-005Data model revisions05/29/2018

The data migration sub-group is currently working on an analysis of legacy data from our member institutions that is currently not represented in the published data models of the various Folio modules. We are forwarding the unrepresented data elements that we discover to the various domain SIGs to be considered for inclusion in data model revisions on which they are working. This project to enhance the data models of the various modules should have a high priority because of all of the down-stream development that depends on them, both for developers to code new functionality, but also for member institutions to review and comment on the adequacy of the institutional data that they have migrated.

Gap analysis will flow through the SIGs for the data model. This is going on with all of the modules.

P1
Data model revisions are closed (Wayne). There is nothing pending (Dale).
SYSOPS-006UUID-issue01/18/2018 + follow-ups

https://discuss.folio.org/t/on-primary-identifiers-in-folio/1826

Functional requirements: We need human readable identifiers. Eye-readable. Can be read on the phone. It has to accept legacy IDs. New records do not have to be compatible with the legacy identifiers.

Preferred: Sequential identifiers. Sequential identifiers are related to the legacy identifiers. They leverage existing IDs by sequential numbers.

It doesn't make sense to fix it after having gone live.

P1

We have them for the inventory app.

UIIN-435 - Getting issue details... STATUS

Closed here.

SYSOPS-007Wishlist from Deployment & Orchestration Session5/17/2018CLI / UI for dependency resolution. An Okapi administrative tool.P1
moved
SYSOPS-008Wishlist from Deployment & Orchestration Session5/17/2018Standards / Requirements for module instrumentation. Being able to see what is going on in a given module for debugging, performance analysis.P2

UXPROD-756 - Getting issue details... STATUS

moved
SYSOPS-009Wishlist from Deployment & Orchestration Session5/17/2018Defined core set of modules. Frontend modules (stripes apps) and backend modules. What is the bare minimum set of modules to run FOLIO ? Distribution management is to be done. Establish an automated way that all modules work together.P2
done
SYSOPS-010Wishlist from Deployment & Orchestration Session5/17/2018Bootstrapping tool that
  • creates tenant
  • sets up admin user
A script. Might be consolidated into a single administrative toolkit.
P2
moved
SYSOPS-011Wishlist from Deployment & Orchestration Session5/17/2018Useful data sets
  • reference data
  • sample data
Having a large enough sample data set. Anonymous user data. Maintenance is involved.
P3

FOLIO-1519 - Getting issue details... STATUS

done
SYSOPS-012Wishlist from Deployment & Orchestration Session5/17/2018

Release Management

Right now: snapshots and git commits. Is not sustainable. Need to invest resource. Similar to distribution management.

P1

Solved in part by the Q Releases.

Other part: Ongoing work in the Kubernetes subgroup.

It's being worked on by the QA group : /wiki/spaces/DQA/pages/2656765 ; this is part of the Kubernetes subgroup.

Will develop playbooks. SysOps can pick up and follow.

Playbook: How to put together a Rancher system in AWS (is being developed).


covered in Kubernetes Subgroup

Loan data source element6/4/2018It is useful to have the source element ID for loan records in the data, not just "retrieved from FOLIO"
Maybe written by Ann Highsmith ?I (Anne Highsmith) don't remember raising this issue and I'm not sure what "source element ID" refers to.

Referential data in loans6/4/2018There is the need to store data in the loans record even after an item record may be removed. Source should be indicated. How is this data updated if it changes in the instance record or item record?

Again, not sure what "source" refers to.

Historical loan data6/4/2018How to represent historical data (e.g. circ_trans_archive)? Need to indicate both current circ data and historical sources. No place to represent compiled (collapsed) transactions for historical data. It might make sense to load history directly into a reporting system, rather than loading into the loans endpoint.

Refers to "historical" or "closed" loans. Should probably be referred to Reporting SIG as part of the discussion on how historical data will be handled.

UserId in loan data6/4/2018UserID is a required field – how are data anonymized when a transaction is closed? May need to make userId not required. Same privacy issue about PatronGroup and OperatorID.

This may refer to anonymization for loans kept in FOLIO after loan is closed. If yes, I think that would go to Emma Boettcher, the loans Product Owner.

In house use in loan data6/4/2018Need store also in-house use (browses) in loan data.

Should go to Emma Boettcher, the loans Product Owner, for a JIRA ticket for a task that would allow tracking of browses.

Integrations
List of Integrations

UXPROD-211 - Getting issue details... STATUS

moved