Discuss the need for investigation/review of our UI testing strategy and recommendations/requirements based on current tech/framework and state of coverage, level of effort, etc.
- Broadly, there are two types of UI tests: integration tests (full-stack) that operate against a production UI build backed by a live backend, and functional or unit tests (UI-only) that operate against a mocked backend.
- integration tests are written in NightmareJS; it is not currently maintained and uses an outdated version of the Electron browser
- unit tests are mostly written in BigTest v1, a framework that is now officially deprecated
- We have substantial investment in both Nightmare and BigTest, but given their lack of maintenance, we are a bit stuck.
- Additional unit testing challenges we face regardless of the framework:
- We don't have a good way to manage mocks for unit tests in a way that keeps them in sync with backend changes.
- We don't have a good way to manage mocks for interfaces in a single place when multiple UI apps rely on the same interface. Currently, we maintain separate copies of mocks in separate UI repositories.
Two real questions for now:
1) Should we review the technology, process and expectations (80%) for creation Unit tests. Consensus is yes. We'll review the requirements for both UI and backendAnton Emelianovvolunteers to kick this effort off. Will follow up in 3 weeks.
2) What do we do with modules that don't meet the test strategy requirements?
- Note that MARCcat is being taken up by @Cult and Scanbit - the expectations are that they'll have 80% coverage for Unit tests.
- Any module that has less than 80% needs to have "special exception" noted by Tech Council in order to be included in community builds.
- We need to have a process to document that special exception.
- UI-Courses had a "grandfathered" special exception because we thought tests were coming in Q2. It will not be included in the community releases in Q3 unless its code coverage meets our standards.