Jest (UI Testing Tool Evaluation Criteria)

  1. Speed - Fast
  2. Reliability - High
  3. Relevance - High
  4. Mocking facility - library can be mocked or functions that make HTTP requests. or via libs like `nock` etc
  5. Cost to migrate/rebuild existing tests. based on module: it can be few weeks and months (e.g ui-users) for modules that follow Page patterns (huge components).
  6. no
  7. yes (3 files in parallel, but not test cases)
  8. yes

React Testing Library with jest

its props/cons is a reflection of jest pros/cons.

On top of that:

  • is a light-weight solution for testing React components
  •  provides light utility functions in a way that encourages devs to write tests that closely resemble how web pages are used
  • rather than dealing with instances of rendered React components, tests will work with actual DOM nodes.
  • finding form elements by their label text (just like a user would), finding links, and buttons from their text (like a user would). It also exposes a recommended way to find elements by a data-testid as an "escape hatch" for elements where the text content and label do not make sense or is not practical.


Additional resources:

https://github.com/testing-library/user-event

https://testing-library.com/docs/react-testing-library/intro

https://blog.sapegin.me/all/react-testing-3-jest-and-react-testing-library/

Bigtest

Pros

  • Used by FOLIO community

Cons

Look at Jest Pros

Example

import React from 'react';
import { MemoryRouter } from 'react-router-dom';
import { describe, beforeEach, it } from '@bigtest/mocha';
import {
  interactor,
  Interactor,
} from '@bigtest/interactor';
import { expect } from 'chai';
import faker from 'faker';

import sinon from 'sinon';

// eslint-disable-next-line
import { mountWithContext } from '@folio/stripes-components/tests/helpers';

import QuickMarcEditor from './QuickMarcEditor';

const QuickMarcEditorInteractor = interactor(class {
  static defaultScope = '#quick-marc-editor-pane';

  closeButton = new Interactor('[data-test-cancel-button]')
});

const getInstance = () => ({
  id: faker.random.uuid(),
  title: faker.lorem.sentence(),
});

const marcRecord = {
  records: [{
    tag: '001',
    content: '$a as',
  }, {
    tag: '245',
    content: '$b a',
  }],
};

// mock components? seems no, babel-plugin-rewire should be added

describe('QuickMarcEditor', () => {
  const instance = getInstance();
  let onClose;

  const quickMarcEditor = new QuickMarcEditorInteractor();

  beforeEach(async () => {
    onClose = sinon.fake();

    await mountWithContext(
      <MemoryRouter>
        <QuickMarcEditor
          instance={instance}
          onClose={onClose}
          onSubmit={sinon.fake()}
          initialValues={marcRecord}
        />
      </MemoryRouter>,
    );
  });

  it('should be rendered', () => {
    // eslint-disable-next-line
    expect(quickMarcEditor.isPresent).to.be.true;
  });

  // new async action? new branch
  describe('clsoe action', () => {
    beforeEach(async () => {
      await quickMarcEditor.closeButton.click();
    });

    it('onClose prop should be invoked', () => {
      // eslint-disable-next-line
      expect(onClose.callCount !== 0).to.be.true;
    });
  });

  // another prop is requried? the second render
  describe('disable', () => {
    beforeEach(async () => {
      await mountWithContext(
        <MemoryRouter>
          <QuickMarcEditor
            instance={instance}
            onClose={onClose}
            onSubmit={sinon.fake()}
            initialValues={{}}
          />
        </MemoryRouter>,
      );
    });

    // no async in cases
    it('onClose prop should be invoked', () => {
      // eslint-disable-next-line
      expect(true).to.be.true;
    });
  });
});

Jest

Pros

  • Recommended by React team https://reactjs.org/docs/testing.html#tools
  • Good documentation with examples - easy to learn
  • Big community
  • Active development
  • Doesn't require additional dependencies like chai and sinon for mocha, everything is already included
  • Good API
  • Mocking is easy
  • Async test cases (in case of BigTest all async operations like render and actions should be done in before* blocks, with it we have many inner describe blocks and not required rerenders)
  • It's fast 

Cons

  • common mocks are required as tests don't render in real browser

Example

import React from 'react';
import { MemoryRouter } from 'react-router-dom';
import { render, cleanup, act, fireEvent } from '@testing-library/react';
import { IntlProvider } from 'react-intl';
import faker from 'faker';

import '@folio/stripes-acq-components/test/jest/__mock__';

import QuickMarcEditorContainer from './QuickMarcEditorContainer';

const getInstance = () => ({
  id: faker.random.uuid(),
  title: faker.lorem.sentence(),
});

const match = {
  params: {
    instanceId: faker.random.uuid(),
  },
};

const record = {
  id: faker.random.uuid(),
  leader: faker.random.uuid(),
  fields: [],
};

const messages = {
  'ui-quick-marc.record.edit.title': '{title}',
};

const renderQuickMarcEditorContainer = ({ onClose, mutator }) => (render(
  <IntlProvider locale="en" messages={messages}>
    <MemoryRouter>
      <QuickMarcEditorContainer
        onClose={onClose}
        match={match}
        mutator={mutator}
      />
    </MemoryRouter>
  </IntlProvider>,
));

describe('Given Quick Marc Editor Container', () => {
  let mutator;
  let instance;

  beforeEach(() => {
    instance = getInstance();
    mutator = {
      quickMarcEditInstance: {
        GET: () => Promise.resolve(instance),
      },
      quickMarcEditMarcRecord: {
        GET: jest.fn(() => Promise.resolve(record)),
        PUT: jest.fn(() => Promise.resolve()),
      },
    };
  });

  afterEach(cleanup);

  it('Than it should fetch MARC record', async () => {
    await act(async () => {
      await renderQuickMarcEditorContainer({ mutator, onClose: jest.fn() });
    });

    expect(mutator.quickMarcEditMarcRecord.GET).toHaveBeenCalled();
  });

  it('Than it should display Quick Marc Editor with fetched instance', async () => {
    let getByText;

    await act(async () => {
      const renderer = await renderQuickMarcEditorContainer({ mutator, onClose: jest.fn() });

      getByText = renderer.getByText;
    });

    expect(getByText(instance.title)).toBeDefined();
  });

  describe('When close button is pressed', () => {
    it('Than it should invoke onCancel', async () => {
      let getByText;
      const onClose = jest.fn();

      await act(async () => {
        const renderer = await renderQuickMarcEditorContainer({ mutator, onClose });

        getByText = renderer.getByText;
      });

      const closeButton = getByText('stripes-acq-components.FormFooter.cancel');

      expect(onClose).not.toHaveBeenCalled();

      fireEvent(closeButton, new MouseEvent('click', {
        bubbles: true,
        cancelable: true,
      }));

      expect(onClose).toHaveBeenCalled();
    });
  });
});
  • No labels

11 Comments

  1. Look, the problem isn't that we have failed to recognize Jest and RTL as emerging/de facto standards in the React community. These are facts; I don't dispute them.

    The problem is that we have 18+ months of investment in BigTest and pivoting to something else would cause us to lose that investment both in our codebase and in our team's knowledge. Honestly, if you can figure out a way to run Jest and BigTest against the same codebase and generate a single, accurate code-coverage report, I'd be all ears. It would allow us to take advantage of the work we have already done – 79% of 23k lines in ui-users, 82% of 35k lines in stripes-components, etc. – while also allowing us to take advantage of the maturity and widespread community support Jest and RTL enjoy. This would be great! Until then, however, I'm going to continue to advocate writing one more BigTest in ui-users to maybe get us to 80% coverage rather than writing the ~500 Jest tests it would likely require to achieve similar coverage when starting from scratch with any other testing framework.

    1. Zak_Burke we don't need to get rid of BigTest, it sill can/will be used for integration/functional. ui-users and ui-inventory are good examples, components are super huge to cover them with unit tests and mock smth
      as we discussed the idea of unit test is testing of small piece of code and IMO Jest is more powerful for this purpose.

      We won't lose anything, we introduce an additional testing level

      I still don't understand why we need to merge coverage of BigTest and Jest (or smth else). If you worry about sonarcube report so old repos can use integ tests coverage till unit tests coverage is < 70-80

      1. "We won't lose anything" ... "old repos can use integ tests coverage till unit tests coverage is < 70-80"

        These two statements directly contradict each other. 

        1. hm, what contradictions do you see? I don't see any

          1. Mikita Siadykh

            hm, what contradictions do you see? I don't see any

            I think the contradiction that Zak_Burke is referring to is that implicit in the following statement, is that once unit test coverage reaches 80%, the integration tests will be removed, and hence we would lose something.

            old repos can use  \[integration\] tests coverage till unit tests coverage is < 70-80

            Unless you are proposing that we maintain 80% coverage in both styles of tests? 

            My understanding from the following statement is that it is proposed that the Jest unit test coverage would become the test metric that FOLIO uses to measure coverage for UI modules / libraries. In which case, the coverage provided by integration tests becomes unimportant (which is where I think Zak's comment about combining coverage comes from).

            Have I understood that correctly?

            I still don't understand why we need to merge coverage of BigTest and Jest (or smth else). If you worry about sonarcube report so old repos can use integ tests coverage till unit tests coverage is < 70-80

            1. Marc Johnson

              unit test coverage would become the test metric that FOLIO uses to measure coverage for UI modules / libraries.  In which case, the coverage provided by integration tests becomes unimportant

              that's right

              the integration tests will be removed, and hence we wouldlose something.

              nope, no need to remove, mb to refactor a bit to cover scenarios

              Unless you are proposing that we maintain 80% coverage in both styles of tests? 

              good question, actually with new approach code should be covered by both testing types, so I'd say yes

              It's kind of what you have on BE side, pieces are covered by JUnit or smth like, and everything connected by API tests


              1. Mikita Siadykh

                good question, actually with new approach code should be covered by both testing types, so I'd say yes

                Does that mean a combined coverage of 80% or each style of test needs to independently have 80% coverage?

                Your previous reply to Zak_Burkesuggested that we would not need to combine coverage information, which would suggest two separate measures.

                Have I understood this aspect of the proposal sensibly?


                It's kind of what you have on BE side, pieces are covered by JUnit or smth like, and everything connected by API tests

                Aside: Hmm, at least on the modules I work with, the balance between unit and larger scale tests has been challenging. Many of the modules, especially storage, only really have larger scale integration tests.

                1. yes, we don't need to combine, but both testing types should be implemented, their purpose is different https://martinfowler.com/articles/practical-test-pyramid.html


                  seems the main questions What to do with old modules like ui-users? mb for them it makes sense to combine reports (e.g https://github.com/bhovhannes/junit-report-merger can be used), but it will stop unit tests implementation 

                  I don't know what modules you work with and how you have 80%  coverage with only api tests, Firebird and Thunderjet do unit and api testing (smile)

                  1. Mikita Siadykh

                    yes, we don't need to combine, but both testing types should be implemented, their purpose is different

                    I agree that the purpose of the different styles of tests are different and the emphasis (as encouraged by the pyramid) tends to be on unit tests.


                    seems the main questions What to do with old modules like ui-users?

                    I think the question is more general than that (unless you are suggesting that new modules are already using Jest and mostly unit tests?). 

                    For me, the primary question is, which type(s) of tests should be measured for the 80% coverage expectation?

                    I think that should be consistent across different modules (of if not, exceptions be well documented).

                    it will stop unit tests implementation 

                    Is that because if the coverage exceeds 80% combined you believe there would be no interest or incentive to move to a greater emphasis on unit tests?


                    I don't know what modules you work with and how you have 80%  coverage with only api tests, Firebird and Thunderjet do unit and api testing

                    I've mostly worked on modules that have been around for longer periods in FOLIO, so it is likely I've missed some experience from the newer modules.

                    Please could you share examples of modules that have an emphasis on unit tests over API tests? I'd like to understand better how folks are doing that in FOLIO

  2. Anton Emelianov seems we can continue discussion in comments before the meet, mb page will be updated (depends on comments)

  3. point Kind of standard for React testing could be expanded to recommended by React team with the link https://reactjs.org/docs/testing.html#tools