| 60 min | Discussion | Everyone | E2E Testing capabilities - Unsure about capabilities for removing a tenant via API
- Testrail tests are run once per release as a crowd-sourced event. Anton posts the link to the test rails instance. One website is running previous release, and other is running next release
- Environments are “bugfest-goldenrod.folio.ebsco.com” and bugfest-honeysuckle.folio.ebsco.com”
- Data is migrated from golden rod to honeysuckle
- In Q1 system is built from UChicago data set
- In Q2 (golden rod) we migrated the original dataset, then decommissioned
- In Q3 we will migrate Q2 over to to the Q3 release
- Next year, we will have three releases instead of four, but in both cases this test rail is the final QA event.
- Data is not refreshed. Main objective was to have an enormous data set.
- You cannot have real user data in the system at all, and so there may be a different class of bugs that arise based on an accumulation of usage data in the production environment, but we cannot lift the dataset from a library.
- Tenant that being used for TestRail test is nowhere close to real tenant, so more like a unit test.
Nightmare testing - platform-core/test/ui-testing/*.js contains most tests,
- Most is navigating around from one place to the next making sure the UI is ready
- Helpers from stripes testing library
- Hard to know when you’re in nightmare and when you’re in the browser
- Some cleanup happens, but not a whole lot
- Run as part of the nightly provisioning script, only run against nightly deploy (unless developing)
Questions / Solutions - One of the challenges that QA world has is that it is difficult to improve the tools
- Symbiotic relationship between development teams and QA.
- Test Rail tests are very declarative and share a form with BigTest tests
- Maybe generate the human driven tests off of the UI tests
- Develop workflows that center around tests that allow collaboration between QA and ui testers
- What are the requirements for the backend for automating the QA setup.
- The deployment aspect is not core to folio, so there’s very little we can assume
- We know that it’ll be a DOM app
- Want to assume the capabilities for deployment that we have now (EBSCO and AWS)
- Let’s look at 10 tests that are very valuable, and can we implement them with what we have today
- Take Aways
- Anton to give Frontside an account to Test Rails so that we can see the test plan
- Take section by section of tests to automate
- Implement the nightmare tests and see how many are covered in Test Rail
- How much can be handled with a single set of base data. E.g. if all of that circulation data is effectively read-only, we can run much more of our tests on that data.
|