Overview

While troubleshooting data migration script issues, I found myself looking for an easy way to verify these scripts without having to run a full blown FOLIO instance.  Here's another option that requires only a database, and the ability to run the module under test locally.

Prerequisites

This is essentially one-time setup you'll need to do.

Database

First you'll need a database.  Here's a how to setup postgres w/ docker:

# pull the desired postgres image
$ docker pull postgres:10

# start the db
$ docker run --rm --name pg_local -e POSTGRES_PASSWORD=mysecretpassword -p 5432:5432 -d postgres:10

# create role/db
$ docker exec -it pg_local /bin/bash
root@0728dc56d41e:/# su postgres
postgres@0728dc56d41e:/$ createuser -l -i -r -s -d -P folio (enter password when prompted)
postgres@0728dc56d41e:/$ createdb -O folio folio
postgres@0728dc56d41e:/$ exit
root@0728dc56d41e:/# exit

# create a config file for connecting to your db
$ cat <<EOF > /tmp/postgres-conf.json
{
  "host":"localhost",
  "port":5432,
  "username":"folio",
  "password":"letmein",
  "database":"folio"
}
EOF

# optional - create an image of postgres w/ your changes.  This will save you from having to do the work above over and over again.
# next time you start your db, specify 'pg_folio' instead of 'postgres:10'
$ docker commit pg_local pg_folio

Clone and build modules

Now that you have a database, clone the git repo using the appropriate branch/tag and build with maven

$ git clone git@github.com:folio-org/mod-finance-storage -b v4.1.1 mod finance-storage-4.1.1
$ cd mod-finance-storage-4.1.1
$ mvn clean package

$ cd ..
$ git clone git@github.com:folio-org/mod-finance-storage -b v4.2.0 mod finance-storage-4.2.0
$ cd mod-finance-storage-4.2.0
$ mvn clean package

Run/Initialize the version you're upgrading from

Start the module up and hit the tenant API to initialize the database

$ cd mod-finance-storage-4.1.1

$ java -Dvertx.logger-delegate-factory-class-name=io.vertx.core.logging.SLF4JLogDelegateFactory -jar target/mod-finance-storage-fat.jar -Dhttp.port=8082 -Dlog.level=debug db_connection=/tmp/postgres-conf.json

$ curl "http://localhost:8082/_/tenant" -H "X-Okapi-Tenant: testtenant" -H "X-Okapi-URL: http://localhost:8082" -H "Content-type: application/json" -XPOST -d'
{
  "module_to": "mod-finance-storage-4.1.1",
  "parameters": [{
    "key": "loadSample",
    "value": "true"
  },{
    "key": "loadReference",
    "value": "true"
  }]
}'

Check that things are working

Load additional data

This step is optional.  If you want to add data to the system to exercise edge cases or see how things behave when migrating larger data sets, now is the time to do so.

Run the version you're upgrading to and perform the upgrade

Stop the old module before proceeding

$ cd ../mod-finance-storage-4.2.0

$ java -Dvertx.logger-delegate-factory-class-name=io.vertx.core.logging.SLF4JLogDelegateFactory -jar target/mod-finance-storage-fat.jar -Dhttp.port=8082 -Dlog.level=debug db_connection=/tmp/postgres-conf.json

$ curl "http://localhost:8082/_/tenant" -H "X-Okapi-Tenant: testtenant" -H "X-Okapi-URL: http://localhost:8082" -H "Content-type: application/json" -XPOST -d'
{
  "module_from": "mod-finance-storage-4.1.1",
  "module_to": "mod-finance-storage-4.2.0",
  "parameters": []
}'

Validate upgrade and data migrations

Other considerations