2018-09-07 - System Operations and Management SIG Agenda and Notes

• Determine note taker: Ingolf
• Review open action items
  • "Run Minikube on a Vagrant Box": Minikube and Vagrant are alternative solutions. It is not feasible to run Minikube on a Vagrant box. Item closed.
  • "Building a laundry list" is kept as an ongoing action item (this, or preparing for this, is what we are doing most of the time).

Experiences with the Rancher setup.

See Jason's TAMU Kubernetes and Rancher 2.0 Deployment Overview !

Ingolf's notes:

Jason reports, that they, TAMU, have one complete instance of folio running in a Rancher environment. They are running the Q2 release.They use a Stateful Set which was assigned a 10 GB NFS share, and two replicates of it. They use VM Ware integrated containers.

Use their own Docker images of Okapi and Stripes and their own Docker registry.

Deploying crunchy-postgres, Kubernetes uses the Helm package manager.

When you have your database set up, you can install Okapi. 3 Replicates of Okapi are running. Okapi is running in "clustered mode."  Set up custom variables. Kubernetes DNS keeps track of the port which is actually changing. The IP, OKAPI_CLUSTERHOST, never changes.

Deploy Stripes module. Input certificates inside of Rancher. Set up some kind of Ingress/Load Balancing: Two ingresses, one for Okapi, one for Stripes.

Deployed ca. 20 different modules. Rancher gives you the YAML file, with which you can spin up the next module.

Caveat: How do persistent volumes work ?

Using their own Module Description Files. Using a Docker file to create these directories. A script generates the deployment descriptors, "create-deploy". Using the same thing for populating the database. Working with a Alpine Linux Container, which runs a bunch of scripts (e.g. SQL commands).

Discussion

When a request comes in, you have to know "this request needs to be send to this cluster". Okapi is running in a clustered mode, but it is not clustering.

Need a recipe to deploy in a different order rather than having Okapi deploying it (Jakub and Jason agree). The current approach "can not be sustained" (Jakub). You want Okapi to be in control of controlling dependencies, otherwise you will have an expanding dependecy tree (Jakub).

You can't put an URL in a module descriptor. If we extend the functionality, then we can have our own module descriptor registry (Jakub). Then you could use the /install - Endpoint (Wayne). Procedure is: 1. Do a poll. 2. Send a list of modules to the tenant.

Discussion interrupted at 10:45 ---- This discussion should be continued in the next session !!

Also see the ongoing discussion on the sys-ops Slack channel.

Especially progress relevant for deplyoment.

Tabled. Should be covered next time.

Tod and Patty shadowing the Batch loader development process; there are only a few requirements that specifically pertain to initial migration. To get tickets written and access to development time, Sys Ops convener should write to An-Marie, Batch-load product owner, and ask if she can coordinate these additional user stories and prioritize and assign to FoliJet team. Once the architecture of the module is developed, Tod and Patty can write the user stories and submit them.

Meeting Notes

Patty suggests to have Ann-Marie Breaux, the PO for the Batch Loader, also coordinate the efforts to build a Migration Tool and to formally ask her to do so.

Harry argues, that we should not charge Ann-Marie with that before she will have finished her work on the Batch Loader. He points out that Ann-Marie would be aware of the differences between the Batch Loader and the Migration Loader anyway. Harry says that the requirements have already been transformed into issues under the Batch Importer Epic, UXPROD-47. He will check that. He also mentions, that Ann-Marie, or in general a person who occupies the role of the Product Owner, can not assign resources in the FoliJet team (or their respective team). This role is reserved to the developer teams or the development lead, resp.

Patty says, the overall point is to change (i.e. to raise) the priorities for the migration loader issues. Harry argues, that the prioritizations have already been done and the next "round" for prioritization is only a three month or so.

Ingolf thinks that the migration loaders should have a higher priority than the Batch Loader, beacause migration will be done first. Harry replies that batch loading will be done more often, migration loading only once, and therefore it should have a high priority. Ingolf is concerned about the timetable for the development of the features for the migration loaders. The development and tests must be in time for the implementation plans of the early implementors. Harry assures Ingolf that the current prioritization will match their implementation plans.

... and here are finally the three differences between the Batch Loader and the Migration Loader, that have been worked out (by Patty and Tod):

• Flat files. The migration loader must be able to load flat files. Discussion about what is a flat file. A flat file can be anything in which the data is not hierarchically structured. In particular, sequential MARC or MARC binary is a flat file. Not all loaders must accept flat files, but at least the loader for bibliographic data.
• A scripting tool. In contrast to Batch Loading, which might all be done via an API, it must be possible to perform migration loading from the command line. There should be a script / a set of scripts for that.
• Performance. Migration loading needs to have a higher perfomance than batch loading. This is especially true for loading millions of bibliographic records at once.

(Harry) UXPROD-47 states the following:

For Flat files:

• Must be able to accept bibliographic data, possibly with embedded holdings and items, in Standard MARC-21, MARCXML or already mapped JSON file. Flat files for item data must also be accepted. (Common with Batch Loader, need to be able to ingest MARCXML or binary MARC.) - UXPROD-665
• The loader must be able to map data from MARC tags and fields correctly to the relevant fields in FOLIO inventory module. In mapping MARC/MARCXML, we should use only one transformation method, common to migration and batch loading. (Common with Batch Loader)
• When the source data is extracted from the previous system as a separate record, whether in JSON or in the MARC Format for Holdings Data (MFHD), the loader must be able to map the holdings appropriately in the FOLIO system, preserving links among holdings, instances, and items. (Common with Batch Loader-preserve links among bibs, holdings, items)

For Scripting:

• The loader requires a CLI. The interface should allow the kick off or scheduling of a load process. (Common with Batch Loader)
• The loader should also provide User Interface and should allow the kick off or scheduling of a load process, view progress, see status, see errors and issues. (Common with Batch Loader)

For Performance:

• The loader must perform quickly and efficiently, so that [4,000,000 bibs] can be loaded [per hour] (Common with Batch Loader)
  
  
  

What do we want to focus on ? What is outstanding (maybe has been postponed) ? What is most urgent now ?

Dale will convene the next meeting, as Ingolf will have half a day off.