Test each layer

Which test for which layer

Frontend presentation tests

A test of a view checks two things:

1. Action → DTO. Do an action on the view. Then check the DTO that the store sends.
2. Event → HTML. Publish an event. Then check the HTML of the view.

A view publishes its events and follows its store, so you can test each view alone. testPresentation("<context>") gives the real event bus and the real store of the context. Its message bus is a fake: it records the DTOs in sent, and no handler runs. The test publishes the answers of the handlers itself.

// @vitest-environment happy-dom
import { afterEach, describe, expect, globalWith, mountTemplate, settle, test, testPresentation } from "@ingenioz-it/caits/testing/frontend/presentation";

afterEach(() => { document.body.innerHTML = ""; });

const setup = () => {
  const presentation = testPresentation("messages");
  const mount = async (answer: MessagesEvent = new MessagesLoaded([])) => {
    const mounted = await mountTemplate(MessageBoard, { $global: globalWith({ messages: presentation }) });
    presentation.events.publish(answer);   // the answer of the handler to the request of the view
    await settle();
    presentation.sent.length = 0;          // forget the DTO that the view sent as it loaded
    return mounted;
  };
  return { ...presentation, mount };
};

describe("MessageBoard action → DTO", () => {
  test("adding a message sends its text", async () => {
    const { sent, mount } = setup();
    const { container } = await mount();

    await add(container, "Hello");

    expect(sent).toStrictEqual([new AddMessageCommand("Hello")]);
  });
});

describe("MessageBoard store → HTML", () => {
  test("says why the messages could not be loaded", async () => {
    const { mount } = setup();
    const { container } = await mount(new MessagesLoadFailed("HTTP 503"));

    expect(alerts(container)).toEqual(["The messages could not be loaded: HTTP 503"]);
  });
});

• The first line runs the test in a page made by happy-dom: the default environment of Vitest has no page.
• add and alerts are small functions of the test file: they type in the field and submit the form, or read the alerts.
• settle() waits until the view has drawn the change.
• The example also tests the board for a keyboard and a screen reader: where the focus goes after each change, and the status that tells a change that succeeds.

A page places the views, and has no test of its own: the end-to-end tests cover it.

Frontend application tests

Each test tells a scenario: give a DTO to the message bus, then tell which events the handler must publish, with one expectItToPublish for each. The order does not count, and the handler can publish other events too. A handler of the frontend gives nothing back: the scenario runs when the test awaits it.

import { applicationScenario, describe, expect, test } from "@ingenioz-it/caits/testing/frontend/application";

test("adds the message through the gateway, then announces the change", async ({ app }) => {
  const gateway = app.use(MessagesGateway, new FakeMessagesGateway());   // your own fake: keep it, to check it

  await applicationScenario(app)
    .givenTheDto(new AddMessageCommand("Hello"))
    .expectItToPublish(new MessagesChanged());

  expect(await gateway.list()).toStrictEqual([new Message("message-1", "Hello")]);
});

• Each test gets a new app, with all the contexts. The app builds a context only when the test needs it, with the bindings of test.di.ts over those of di.ts.
• Each port has a token with its name: app.get(<Port>) gives the adapter of the test, and app.use(<Port>, fake) gives the app your own.
• The frontend of a test has no network. An adapter that takes the Fetcher fails at its first request, with the message A test frontend has no network: replace the dependency that asked for …. Bind a fake adapter of its port in test.di.ts (see An adapter that calls the API).

Infra tests

Each port has one contract: the tests that all its adapters must pass. The test of each adapter runs the contract, and can add tests for what only this adapter does. So the fake adapter of the tests behaves like the real one. Add a port shows a contract and its adapters.

An HTTP adapter runs the contract against a fake API: the test replaces fetch, and answers as the backend does.

afterEach(() => vi.unstubAllGlobals());

describeMessagesGatewayContract("Http", ({ messages, unavailable, refusal }) => {
  vi.stubGlobal("fetch", vi.fn(async (url: string, init?: RequestInit) => {
    // … answers as the API of the backend: the list, a refusal (422), or 204; a 503 to each request when it is unavailable
  }));
  return new HttpMessagesGateway();
});

Backend presentation tests

1. Request → Input DTO. Send a request to the route. Then check the Input DTO that the message bus receives.
2. Output DTO → response. Make the message bus answer with an Output DTO, or fail with an error. Then check the response.

The message bus of these tests is a fake: it records the Input DTOs, and gives the answers that the test tells. The request goes through the real route, so the route has no test of its own.

import { describe, expect, presentationScenario, test } from "@ingenioz-it/caits/testing/backend/presentation";

const problemHeaders = { "content-type": "application/problem+json" };

test("hands the text to the application, and answers with the message it added", async ({ app }) => {
  await presentationScenario(app)
    .givenTheRequest("POST", "/api/messages", { text: "Hello" })
    .expectItToSend(new AddMessageCommand("Hello"))
    .givenTheApplicationReplies(new AddMessageOutput("m1", "Hello"))
    .expectTheResponseToBe(201, { id: "m1", text: "Hello" });
});

test("reports an unexpected failure, and tells nothing of it", async ({ app }) => {
  await presentationScenario(app)
    .givenTheRequest("POST", "/api/messages", { text: "Hello" })
    .expectItToSend(new AddMessageCommand("Hello"))
    .givenTheApplicationFails(new Error("disk full"))
    .expectTheResponseToBe(500, { type: "urn:caits:problem:internal-server-error", title: "Internal server error", status: 500 }, problemHeaders);

  expect(app.reportedErrors).toEqual([new Error("disk full")]);
});

test("refuses a body that is not JSON", async ({ app }) => {
  await presentationScenario(app)
    .givenTheRequest("POST", "/api/messages", "{not json")
    .expectTheResponseToBe(400, { type: "urn:caits:problem:invalid-request-body", title: "Invalid request body", status: 400, detail: "Invalid JSON" }, problemHeaders);
});

• When the request is not valid, the controller answers at once, and the message bus receives nothing: the scenario has no expectItToSend.
• A body that is an object goes as JSON; a string goes as it is, as in the last test. Both go with the header content-type: application/json.
• An error that the controller does not know becomes a 500 that tells nothing of it, and CAITS reports it: app.reportedErrors holds it.

Backend application tests

Each test tells a scenario:

1. Input DTO → Output DTO. Give an Input DTO to the message bus. Then check the Output DTO, with expectTheOutputToBe, or the error, with expectTheErrorToBe.
2. Events. When the events are part of the behavior, tell which events the handler publishes, in their order, with expectItToPublish, or that it publishes none, with expectItToPublishNothing. Then the scenario checks that the handler publishes these events and no others.

When the state of a fake adapter is part of the behavior, check it after the scenario.

import { applicationScenario, describe, expect, test, type TestApplication } from "@ingenioz-it/caits/testing/backend/application";
// … the imports of the domain and of the DTOs

const kept = async (app: TestApplication) => (await app.get(MessageRepository).list()).map(({ id, text }) => [id, text]);

test.for([
  ["", "A message cannot be empty."],
  ["   ", "A message cannot be empty."],
  ["x".repeat(Message.MAX_LENGTH + 1), "A message has at most 280 characters."]
])("the domain refuses %j, and keeps nothing", async ([text, reason], { app }) => {
  await applicationScenario(app)
    .givenTheDto(new AddMessageCommand(text))
    .expectTheErrorToBe(new InvalidMessageError(reason));

  expect(await kept(app)).toEqual([]);
});

The contexts of these tests are apart: an event goes to no other context. The context map tests check the flows between contexts.

Context map tests

The context map turns an event of one context into the DTOs of other contexts. Its tests are next to it, in src/backend/context-map.test.ts:

1. Event → DTOs. Give an event to the map. Then check the DTOs that it gives.
2. Flow. Send a DTO to one context, and wait for all the reactions. Then check the state of the other contexts, with their own queries.

import { backendTest, ContextMap, expect, test } from "@ingenioz-it/caits/testing/backend/context-map";
import translations from "./context-map.js";
// … the imports of the events and the DTOs

const map = new ContextMap(translations);

test("Event → DTOs: an added message has its words counted", () => {
  expect(map.inputsFor(new MessageAdded("m1", "Hello world"), { eventId: "evt-1", correlationId: "req-1" }))
    .toEqual([new CountWordsCommand("Hello world")]);
});

backendTest("Flow: an added message has its words counted", async ({ app }) => {
  await app.send(new AddMessageCommand("Hello world"));
  await app.settled();   // wait for all the reactions

  expect((await app.send(new GetWordCountQuery())).count).toBe(2);
});

The example has one backend context, so it has no context map: the names of this test are those of Connect two contexts. GetWordCountQuery is the query of statistics that gives the count.

Integration tests

The integration tests run the backend as in production: with the adapters of each di.ts, and no test.di.ts. They check a few flows again, with the real files or the real database.

import { afterEach, describe, expect, productionBackend, test, vi } from "@ingenioz-it/caits/testing/backend/integration";

test("keeps the messages in the file messages.json of the folder DATA_DIR, and a new backend finds them there", async () => {
  directory = await mkdtemp(join(tmpdir(), "messages-"));
  vi.stubEnv("DATA_DIR", join(directory, "data"));   // the files go to a temporary folder
  const first = productionBackend();
  const { id } = await first.send(new AddMessageCommand("Hello"));
  // …
  const second = productionBackend();               // a new backend reads the same files
});

They also kill the mutants of the di.ts files: a test that reads messages.json fails when a mutant changes the file name of the binding. For a flow across two contexts, wait for the reactions with await backend.settled() before you check the result, as in the context map tests.

End-to-end tests

The end-to-end tests run in a real browser, with Playwright, on the production build: caits e2e builds the app, starts it on the port 4310 with a new DATA_DIR, then runs test/e2e/*.spec.ts. Name each test Given …, When …, Then …: the documentation of the tests shows it on three lines.

Each test starts from a known state. The server of the end-to-end tests keeps its data from one test to the next, so the tests of the messages remove them first, through the API:

// Each test starts with no message, also when it runs again: the server keeps the messages of the tests before.
test.beforeEach(async ({ request }) => {
  const { messages: kept } = (await (await request.get("/api/messages")).json()) as { messages: { id: string }[] };
  for (const { id } of kept) await request.delete(`/api/messages/${encodeURIComponent(id)}`);
});

test("Given a visitor on the page of the messages, When they add, edit and remove messages, Then the list shows each change, also after a reload", async ({ page }) => {
  await page.goto("/messages");
  await expect(page.getByText("No message yet.")).toBeVisible();

  await add(page, "Hello");
  await expect(messages(page)).toHaveText([/Hello/]);
  // …
});

The helpers

Each module gives the functions of Vitest too (describe, expect, vi…): a test imports everything from one place.

Mutation tests

A mutation test changes the code a little, such as > into >=, or a condition into true, then runs the tests. If a test fails, the mutant is killed: the tests saw the change. If all the tests pass, the mutant survived: a part of the code has no test that checks it. CAITS runs the mutation tests with Stryker, and asks for 100% of the mutants killed.

npm run test:mutation

When the score is under 100%, the end of its output lists the mutants that survived, with their file, their line and their change. npx caits mutation-summary --survivors lists them again, from the last run. For each one, write the test that the change breaks. When a check fails tells what to do when no test can kill a mutant.

The documentation of the tests

The tests are also the documentation of the code: each test is a line of it. npm run check writes the page reports/test-docs/index.html: the end-to-end tests first, then the other tests, by side, context, layer and folder, in the order of their files. A test that failed shows its error. The page has a search, and a filter for the failures.

npm run test:coverage and npm run test:e2e each write their part, then the page again: npm run check runs the two, so it gives the whole page.