Jest (UI Testing Tool Evaluation Criteria)
- Speed - Fast
- Reliability - High
- Relevance - High
- Mocking facility - library can be mocked or functions that make HTTP requests. or via libs like `nock` etc
- Cost to migrate/rebuild existing tests. based on module: it can be few weeks and months (e.g ui-users) for modules that follow Page patterns (huge components).
- no
- yes (3 files in parallel, but not test cases)
- yes
React Testing Library with jest
its props/cons is a reflection of jest pros/cons.
On top of that:
- is a light-weight solution for testing React components
- provides light utility functions in a way that encourages devs to write tests that closely resemble how web pages are used
- rather than dealing with instances of rendered React components, tests will work with actual DOM nodes.
- finding form elements by their label text (just like a user would), finding links, and buttons from their text (like a user would). It also exposes a recommended way to find elements by a
data-testid
as an "escape hatch" for elements where the text content and label do not make sense or is not practical.
Additional resources:
https://github.com/testing-library/user-event
https://testing-library.com/docs/react-testing-library/intro
https://blog.sapegin.me/all/react-testing-3-jest-and-react-testing-library/
Bigtest
Pros
- Used by FOLIO community
Cons
Look at Jest Pros
Example
import React from 'react';
import { MemoryRouter } from 'react-router-dom';
import { describe, beforeEach, it } from '@bigtest/mocha';
import {
interactor,
Interactor,
} from '@bigtest/interactor';
import { expect } from 'chai';
import faker from 'faker';
import sinon from 'sinon';
// eslint-disable-next-line
import { mountWithContext } from '@folio/stripes-components/tests/helpers';
import QuickMarcEditor from './QuickMarcEditor';
const QuickMarcEditorInteractor = interactor(class {
static defaultScope = '#quick-marc-editor-pane';
closeButton = new Interactor('[data-test-cancel-button]')
});
const getInstance = () => ({
id: faker.random.uuid(),
title: faker.lorem.sentence(),
});
const marcRecord = {
records: [{
tag: '001',
content: '$a as',
}, {
tag: '245',
content: '$b a',
}],
};
// mock components? seems no, babel-plugin-rewire should be added
describe('QuickMarcEditor', () => {
const instance = getInstance();
let onClose;
const quickMarcEditor = new QuickMarcEditorInteractor();
beforeEach(async () => {
onClose = sinon.fake();
await mountWithContext(
<MemoryRouter>
<QuickMarcEditor
instance={instance}
onClose={onClose}
onSubmit={sinon.fake()}
initialValues={marcRecord}
/>
</MemoryRouter>,
);
});
it('should be rendered', () => {
// eslint-disable-next-line
expect(quickMarcEditor.isPresent).to.be.true;
});
// new async action? new branch
describe('clsoe action', () => {
beforeEach(async () => {
await quickMarcEditor.closeButton.click();
});
it('onClose prop should be invoked', () => {
// eslint-disable-next-line
expect(onClose.callCount !== 0).to.be.true;
});
});
// another prop is requried? the second render
describe('disable', () => {
beforeEach(async () => {
await mountWithContext(
<MemoryRouter>
<QuickMarcEditor
instance={instance}
onClose={onClose}
onSubmit={sinon.fake()}
initialValues={{}}
/>
</MemoryRouter>,
);
});
// no async in cases
it('onClose prop should be invoked', () => {
// eslint-disable-next-line
expect(true).to.be.true;
});
});
});
Jest
Pros
- Recommended by React team https://reactjs.org/docs/testing.html#tools
- Good documentation with examples - easy to learn
- Big community
- Active development
- Doesn't require additional dependencies like chai and sinon for mocha, everything is already included
- Good API
- Mocking is easy
- Async test cases (in case of BigTest all async operations like render and actions should be done in before* blocks, with it we have many inner describe blocks and not required rerenders)
- It's fast
Cons
- common mocks are required as tests don't render in real browser
Example
import React from 'react';
import { MemoryRouter } from 'react-router-dom';
import { render, cleanup, act, fireEvent } from '@testing-library/react';
import { IntlProvider } from 'react-intl';
import faker from 'faker';
import '@folio/stripes-acq-components/test/jest/__mock__';
import QuickMarcEditorContainer from './QuickMarcEditorContainer';
const getInstance = () => ({
id: faker.random.uuid(),
title: faker.lorem.sentence(),
});
const match = {
params: {
instanceId: faker.random.uuid(),
},
};
const record = {
id: faker.random.uuid(),
leader: faker.random.uuid(),
fields: [],
};
const messages = {
'ui-quick-marc.record.edit.title': '{title}',
};
const renderQuickMarcEditorContainer = ({ onClose, mutator }) => (render(
<IntlProvider locale="en" messages={messages}>
<MemoryRouter>
<QuickMarcEditorContainer
onClose={onClose}
match={match}
mutator={mutator}
/>
</MemoryRouter>
</IntlProvider>,
));
describe('Given Quick Marc Editor Container', () => {
let mutator;
let instance;
beforeEach(() => {
instance = getInstance();
mutator = {
quickMarcEditInstance: {
GET: () => Promise.resolve(instance),
},
quickMarcEditMarcRecord: {
GET: jest.fn(() => Promise.resolve(record)),
PUT: jest.fn(() => Promise.resolve()),
},
};
});
afterEach(cleanup);
it('Than it should fetch MARC record', async () => {
await act(async () => {
await renderQuickMarcEditorContainer({ mutator, onClose: jest.fn() });
});
expect(mutator.quickMarcEditMarcRecord.GET).toHaveBeenCalled();
});
it('Than it should display Quick Marc Editor with fetched instance', async () => {
let getByText;
await act(async () => {
const renderer = await renderQuickMarcEditorContainer({ mutator, onClose: jest.fn() });
getByText = renderer.getByText;
});
expect(getByText(instance.title)).toBeDefined();
});
describe('When close button is pressed', () => {
it('Than it should invoke onCancel', async () => {
let getByText;
const onClose = jest.fn();
await act(async () => {
const renderer = await renderQuickMarcEditorContainer({ mutator, onClose });
getByText = renderer.getByText;
});
const closeButton = getByText('stripes-acq-components.FormFooter.cancel');
expect(onClose).not.toHaveBeenCalled();
fireEvent(closeButton, new MouseEvent('click', {
bubbles: true,
cancelable: true,
}));
expect(onClose).toHaveBeenCalled();
});
});
});
11 Comments
Zak_Burke
Look, the problem isn't that we have failed to recognize Jest and RTL as emerging/de facto standards in the React community. These are facts; I don't dispute them.
The problem is that we have 18+ months of investment in BigTest and pivoting to something else would cause us to lose that investment both in our codebase and in our team's knowledge. Honestly, if you can figure out a way to run Jest and BigTest against the same codebase and generate a single, accurate code-coverage report, I'd be all ears. It would allow us to take advantage of the work we have already done – 79% of 23k lines in ui-users, 82% of 35k lines in stripes-components, etc. – while also allowing us to take advantage of the maturity and widespread community support Jest and RTL enjoy. This would be great! Until then, however, I'm going to continue to advocate writing one more BigTest in ui-users to maybe get us to 80% coverage rather than writing the ~500 Jest tests it would likely require to achieve similar coverage when starting from scratch with any other testing framework.
Mikita Siadykh
Zak_Burke we don't need to get rid of BigTest, it sill can/will be used for integration/functional. ui-users and ui-inventory are good examples, components are super huge to cover them with unit tests and mock smth
as we discussed the idea of unit test is testing of small piece of code and IMO Jest is more powerful for this purpose.
We won't lose anything, we introduce an additional testing level
I still don't understand why we need to merge coverage of BigTest and Jest (or smth else). If you worry about sonarcube report so old repos can use integ tests coverage till unit tests coverage is < 70-80
Zak_Burke
"We won't lose anything" ... "old repos can use integ tests coverage till unit tests coverage is < 70-80"
These two statements directly contradict each other.
Mikita Siadykh
hm, what contradictions do you see? I don't see any
Marc Johnson
Mikita Siadykh
I think the contradiction that Zak_Burke is referring to is that implicit in the following statement, is that once unit test coverage reaches 80%, the integration tests will be removed, and hence we would
lose something
.Unless you are proposing that we maintain 80% coverage in both styles of tests?
My understanding from the following statement is that it is proposed that the Jest unit test coverage would become the test metric that FOLIO uses to measure coverage for UI modules / libraries. In which case, the coverage provided by integration tests becomes unimportant (which is where I think Zak's comment about combining coverage comes from).
Have I understood that correctly?
Mikita Siadykh
Marc Johnson
that's right
nope, no need to remove, mb to refactor a bit to cover scenarios
good question, actually with new approach code should be covered by both testing types, so I'd say yes
It's kind of what you have on BE side, pieces are covered by JUnit or smth like, and everything connected by API tests
Marc Johnson
Mikita Siadykh
Does that mean a combined coverage of 80% or each style of test needs to independently have 80% coverage?
Your previous reply to Zak_Burkesuggested that we would not need to combine coverage information, which would suggest two separate measures.
Have I understood this aspect of the proposal sensibly?
Aside: Hmm, at least on the modules I work with, the balance between unit and larger scale tests has been challenging. Many of the modules, especially storage, only really have larger scale integration tests.
Mikita Siadykh
yes, we don't need to combine, but both testing types should be implemented, their purpose is different https://martinfowler.com/articles/practical-test-pyramid.html
seems the main questions What to do with old modules like ui-users? mb for them it makes sense to combine reports (e.g https://github.com/bhovhannes/junit-report-merger can be used), but it will stop unit tests implementation

I don't know what modules you work with and how you have 80% coverage with only api tests, Firebird and Thunderjet do unit and api testing
Marc Johnson
Mikita Siadykh
I agree that the purpose of the different styles of tests are different and the emphasis (as encouraged by the pyramid) tends to be on unit tests.
I think the question is more general than that (unless you are suggesting that new modules are already using Jest and mostly unit tests?).
For me, the primary question is, which type(s) of tests should be measured for the 80% coverage expectation?
I think that should be consistent across different modules (of if not, exceptions be well documented).
Is that because if the coverage exceeds 80% combined you believe there would be no interest or incentive to move to a greater emphasis on unit tests?
I've mostly worked on modules that have been around for longer periods in FOLIO, so it is likely I've missed some experience from the newer modules.
Please could you share examples of modules that have an emphasis on unit tests over API tests? I'd like to understand better how folks are doing that in FOLIO
Mikita Siadykh
Anton Emelianov seems we can continue discussion in comments before the meet, mb page will be updated (depends on comments)
Aliaksei Chumakou
point Kind of standard for React testing could be expanded to recommended by React team with the link https://reactjs.org/docs/testing.html#tools