Guide / Sample for Testing with Jest

[shirshak55]

It would be great to have a guide to using jest and typeorm together.

This guide would ideally have patterns for both using real connections for integration testing as well as mocking the TypeORM connection / repository / etc entirely.

Currently, if we mock jest.mock TypeORM the entity definitions stop working as expected.

---

Original Issue Description Below

---

Issue type:

[ X ] question
[ ] bug report
[ X ] feature request
[ ] documentation issue

Database system/driver:

[ ] cordova
[ ] mongodb
[ ] mssql
[ ] mysql / mariadb
[ ] oracle
[ X] postgres
[ ] cockroachdb
[ ] sqlite
[ ] sqljs
[ ] react-native
[ ] expo

TypeORM version:

[ X] latest
[ ] @next
[ ] 0.x.x (or put your version here)

Is there any standard way to test typeorm based model etc.? I don't mind direct connection to database for integration testing (especially mocking gets too complex)

I setup global connection like this.

export async function setupConn(drop=false) {
  conn = await createConnection({
    type: "postgres",
    synchronize: drop,
    dropSchema: drop,
    logging: false,
    url: "postgres://quantum@localhost/test",
    entities: [__dirname + "/../../src/entities/**/*.js"],
  })
  return conn
}

Then I tried to use global setup/ different environment calling setupConn function but the problem is:

Jest spawns worker for each test suites (different *.test.js) file.

So I always see this error

{"data": null, "errors": [[GraphQLError: Connection "default" was not found.]]}

but if i use same function at beforeAll at test suites it works for 1 test suite .

So is there any good and recommend way to test application developed using typeorm.

And thanks for making such a beautiful orm I really enjoy using it :)

[dan003400]

Same exact problem! Each suite will spin up the connection and start running all the migrations and other setups at the same time as other tests running, causing every test to fail with inconsistent errors like relationship does not exist, etc.

[shirshak55]

@dan003400 for now I think we are locked. Currently what i do is not use sync and createConnection for each test suites. And remember to use sync for only one time . You can use npm scripts. If its confusing let me know i am ready to help. I just don't want other to go same trouble i faced.

[dan003400]

@shirshak55 I was able to finally get it working, I had a similar idea last night, finally got something working this AM.

"pre:test:drop": "env-cmd -f ./.test.env ts-node ./node_modules/typeorm/cli.js schema:drop"
"pre:test:sync": "env-cmd -f ./.test.env ts-node ./node_modules/typeorm/cli.js schema:sync"
"test": "yarn pre:test:drop && yarn pre:test:sync && jest"

Let me know if this helps!

[shirshak55]

actually this is not the problem i faced.

[shirshak55]

I was facing issue that i was unable to use same connection in typeorm and needed to establish connection on each test suites. Making tests too slow.

[dan003400]

The problem was every suite was running the schema synchronization at the same time causing lots of issues, with this it will run the drop then sync then tests.

Yes, I am creating a new connection on every suite in the beforeAll callback and closing that connection in every afterAll callback.

[shirshak55]

ok i get it. I think u can do that more efficiently by this instead of doing env stuff.

con.ts

import { Connection, createConnection } from "typeorm"

let conn: Connection | null = null

export async function setupConn(drop = false) {
  conn = await createConnection({
    type: "postgres",
    synchronize: drop,
    dropSchema: drop,
    logging: false,
    url: "postgres://quantum@localhost/bs_test",
    entities: [__dirname + "/../../src/entities/**/*.js"],
  })

  return conn
}

export async function closeConn() {
  await conn?.close()
}

setupDb.ts

import { setupConn } from "./db"

setupConn(true).then((c) => {
  process.exit(0)
})

in Npm scripts

 "db:reset": "node dist/__tests__/utils/setupDb",
"test": "yarn db:reset && yarn jest"

So current problem is only how to pass connection properly so typeorm only uses 1 connection on multiple jest workers.

[DarkLite1]

I'm having the same issue. When the connection is created in jest globalSetup it cannot be found within the tests. It's like the tests are running in a sandbox mode. I've opened an issue at jest to ask what I'm doing wrong.

[guiaramos]

Solution for unit test with TypeORM (edited with @sarfata recommendations, thanks!)

it's working now, hope it helps you guys and please let me know if there is any improvement.

settings

package.json

{
  "scripts": {
    "test": "jest",
    "test:cache": "jest --clearCache",
    "test:update": "jest -u",
    "test:commit": "jest --bail —-findRelatedTests",
    "lint": "tsc --noEmit && eslint 'src/**/*.ts{x,}' --fix --quiet && prettier-check --ignore-path .gitignore \"{.,src/**}/{*.ts{x,},*.md,ts*.json,*.config.js}\"",
  },
  "dependencies": {
    "@drdgvhbh/postgres-error-codes": "0.0.6",
    "cors": "^2.8.5",
    "dotenv": "^8.2.0",
    "reflect-metadata": "^0.1.13",
    "tslib": "1.11.2",
    "typeorm": "^0.2.24",
  },
  "devDependencies": {
    "@types/cookie-parser": "^1.4.2",
    "@types/jest": "^25.2.3",
    "@types/node": "^13.9.0",
    "@typescript-eslint/eslint-plugin": "^2.23.0",
    "@typescript-eslint/parser": "^2.23.0",
    "eslint": "^7.3.1",
    "eslint-config-airbnb-typescript": "^7.0.0",
    "eslint-config-prettier": "^6.10.0",
    "eslint-import-resolver-typescript": "^2.0.0",
    "eslint-plugin-import": "^2.20.2",
    "eslint-plugin-jest": "^23.17.1",
    "eslint-plugin-prettier": "^3.1.3",
    "husky": "^4.2.5",
    "jest": "^26.1.0",
    "lint-staged": "^10.2.2",
    "prettier": "^2.0.5",
    "prettier-check": "^2.0.0",
    "pretty-quick": "^2.0.1",
    "ts-jest": "^26.1.1",
    "ts-node-dev": "^1.0.0-pre.44",
    "typescript": "^3.9.5",

  },
  "lint-staged": {
    "**/*.ts{x,}": [
      "npm run lint",
      "npm run test:commit"
    ]
  }
}

ormconfig.ts

import { ConnectionOptions } from 'typeorm';
import { loadEnv } from './src/libraries/loadEnv';

loadEnv();

const DATABASE_TYPE = 'postgres';
const DATABASE_ENTITIES = ['src/entities/**/**.postgres.ts'];

const connectionOptions: ConnectionOptions[] = [
  {
    name: 'default',
    type: DATABASE_TYPE,
    database: String(process.env.DATABASE_DATABASE),
    host: String(process.env.DATABASE_HOST),
    port: Number(process.env.DATABASE_PORT),
    username: String(process.env.DATABASE_USERNAME),
    password: String(process.env.DATABASE_PASSWORD),
    entities: DATABASE_ENTITIES,
    synchronize: true,
    // dropSchema: true,
    logging: true
  },
  {
    name: 'test',
    type: DATABASE_TYPE,
    database: String(process.env.DATABASE_DATABASE),
    host: String(process.env.DATABASE_HOST),
    port: Number(process.env.DATABASE_PORT),
    username: String(process.env.DATABASE_USERNAME),
    password: String(process.env.DATABASE_PASSWORD),
    entities: DATABASE_ENTITIES,
    synchronize: true,
    dropSchema: true,
    logging: false
  }
];

export = connectionOptions;
];

export = connectionOptions;

src/libraries/loadEnv.ts

import { isProd, isTest } from 'src/constants/Envoriment';
import * as dotenv from 'dotenv';

export const loadEnv = () => {
  const loadFile = () => {
    if (isProd) return '.env';
    if (isTest) return '.env.test';
    return '.env.dev';
  };

  return dotenv.config({ path: loadFile() });
};

jest.config.js

const { pathsToModuleNameMapper } = require('ts-jest/utils');
const { compilerOptions } = require('./tsconfig.json');

module.exports = {
  preset: 'ts-jest',
  testEnvironment: 'node',
  clearMocks: true,
  maxWorkers: 1,
  moduleNameMapper: pathsToModuleNameMapper(compilerOptions.paths, { prefix: '<rootDir>/' }),
  testMatch: ['**/?(*.)+(spec|test).[jt]s?(x)'],
  setupFilesAfterEnv: ['<rootDir>/src/test-utils/db-env.ts']
};

src/test-utils/db-env.ts

// This file is executed once in the worker before executing each test file. We
// wait for the database connection and make sure to close it afterwards.
import setupServer from 'src/server/server.factory';

process.env.NODE_ENV = 'test';

beforeAll(async () => {
  const t0 = Date.now();
  const connection = await setupServer.connectionPostgres.create('test');
  const connectTime = Date.now();
  await connection.runMigrations();
  const migrationTime = Date.now();
  console.log(
    ` 👩‍🔬 Connected in ${connectTime - t0}ms - Executed migrations in ${
      migrationTime - connectTime
    }ms.`
  );
});
afterAll(async () => {
  await setupServer.connectionPostgres.close();
});

server.ts

 connectionPostgres: {
    async create(connectionName: 'default' | 'test' = 'default'): Promise<Connection> {
      const connectionOptions = await getConnectionOptions(connectionName);
      const connection = await createConnection({ ...connectionOptions, name: 'default' });
      return connection;
    },

    async close(): Promise<void> {
      await getConnection().close();
    },

    async clear(): Promise<void> {
      const connection = getConnection();
      const entities = connection.entityMetadatas;

      await Promise.all(
        entities.map(async (entity) => {
          const repository = connection.getRepository(entity.name);
          await repository.query(`DELETE FROM ${entity.tableName}`);
        })
      );
    }
  },

Test

src/entities/Postgres/User/tests/User.test.ts

import setupServer from 'src/server/server.factory';
import User from '../User.postgres';

describe('User entity', function () {
  beforeAll(async () => {
    await setupServer.connectionPostgres.create();
  });
  afterAll(async () => {
    await setupServer.connectionPostgres.close();
  });

  it('should be empty', async function () {
    const count = await User.count();
    expect(count).toBe(0);
  });
});

Docker (optional - for development only, never use docker for db)

docker-compose.db.yaml

version: '3.3'

services:
  postgres:
    image: postgres:11
    environment:
      POSTGRES_USER: user
      POSTGRES_PASSWORD: password
      POSTGRES_DB: db
    ports:
      - 5432:5432

  postgres_tests:
    image: postgres:11
    environment:
      POSTGRES_USER: test
      POSTGRES_PASSWORD: test
      POSTGRES_DB: test
    ports:
      - 5433:5433
    command: -p 5433

[sarfata]

@guiaramos I believe with your solution each test file will redrop and recreate the database completely. This might take a bit of time.

Instead, you can define a globalSetup to run the migrations and connect (without executing migrations) in each of your tests.

This works for me and took me a very long time to figure out. I 👍 this issue, there should be clearer documentation on how to do this.

jest.config

  globalSetup: "./src/test-utils/setup-db.ts",
  setupFilesAfterEnv: ["./src/test-utils/db-env.ts"],

setup-db.ts

// Those first two require are very important - without them the typescript migrations did not work for me.
// See https://github.com/facebook/jest/issues/10178

// tslint:disable-next-line: no-var-requires
require("ts-node/register")
// tslint:disable-next-line: no-var-requires
require("tsconfig-paths/register")
import "dotenv/config"
import { createConnection } from "typeorm"
import { PostgresConnectionOptions } from "typeorm/driver/postgres/PostgresConnectionOptions"
import ormConfig from "../ormconfig"

/*
 * This file is executed by Jest before running any tests.
 * We drop the database and re-create it from migrations every time.
 */
export default async () => {
  // Force dropping the schema so that test run clean every time.
  // Note that we are not cleaning *between* tests.
  const testOrmConfig: PostgresConnectionOptions = {
    ...(ormConfig as PostgresConnectionOptions),
    dropSchema: true,
  }

  const t0 = Date.now()
  const connection = await createConnection(testOrmConfig)
  const connectTime = Date.now()
  await connection.runMigrations()
  const migrationTime = Date.now()
  console.log(
    ` 👩‍🔬 Connected in ${connectTime -
      t0}ms - Executed migrations in ${migrationTime - connectTime}ms.`
  )
}

db-env.ts

// This file is executed once in the worker before executing each test file. We
// wait for the database connection and make sure to close it afterwards.

import { getConnection } from "typeorm"
import { createDatabaseConnection } from "~/repositories/DBConnection"

beforeAll(async () => {
  await createDatabaseConnection()
})
afterAll(async () => {
  await getConnection().close()
})

And my createDatabaseConnection is just:

import { createConnection } from "typeorm"
import ormConfig from "../ormconfig"

export async function createDatabaseConnection() {
  return await createConnection(ormConfig)
}

[guiaramos]

@sarfata Great!! Thanks for the improvement I will apply it.

[good-idea]

@guiaramos this is very, very helpful, thank you. Do you have a complete example in a repo? Or, could you share the full code for server.factory?

[guiaramos]

@guiaramos this is very, very helpful, thank you. Do you have a complete example in a repo? Or, could you share the full code for server.factory?

@good-idea nice, I have this repo that I am currently working on:
https://github.com/guiaramos/ts-graphql-server

Actually, I have to update my previous comment to this issue according to the repo... anyway you can find a good representation of factory in this example also:
https://github.com/filipedeschamps/cep-promise/blob/master/src/cep-promise.js

[ginachoi]

@guiaramos @sarfata were you able to specify seeds in your ormconfig.ts file? I was able to add seeds in ormconfig.js but not in ormconfig.ts file.

seeds: ["src/**/database/seeds/*-seeder.js"]

[sarfata]

@ginachoi not using seeds - sorry.

[guiaramos]

@guiaramos @sarfata were you able to specify seeds in your ormconfig.ts file? I was able to add seeds in ormconfig.js but not in ormconfig.ts file.

seeds: ["src/**/database/seeds/*-seeder.js"]

@ginachoi sorry for late reply, I am not using seeds but recommend this lib https://github.com/w3tecch/typeorm-seeding

[ben-grandin]

What about that :
https://dev.to/caiulucas/tests-with-jest-and-typeorm-4j1l

[sarfata]

@BenGrandin this solution works but the migrations will be executed once for each test file. If you have many test files, this slows down everything considerably.

[sgentile]

The sample needed for jest is to be able to mock the entity manager in a 'unit test'- in particular using transactions. We are able to mock everything but this. The sample needed is one where a database is not required.

[Chetan11-dev]

I use nestjs and I was unable to implement the above solutions due to their complexity. and find a solution.
You should use 'npm run --runInBand' in case you want to run tests one after another and in each file
do

import { createConnection, getConnection, getRepository } from "typeorm";
import { mysqlConfig } from "../config";
beforeEach(() => createConnection(mysqlConfig));
afterEach(() => getConnection().close());

because due to --runinband all tests will run in a sequence(test1->test2->test3) hence you will get no database errors.

Also the speed of my tests have been really fast after using the above options :) increasing my developer productivity.

[mpvosseller]

Hi, A common suggestion for TypeORM tests that use the database is to serialize them (via Jest options --maxWorkers=1 or --runInBand). The downside of this approach is obvious: tests will run much more slowly than they otherwise might.

I recently was able to parallelize such tests by running multiple databases. The idea is to run each Jest worker with its own database. Tests within one worker run serially but each worker runs in parallel to each other with its own database.

This is quite easy to setup using Docker and Jest and cut my build / test times in half.

Here is a write up:
https://blog.mikevosseller.com/2021/11/25/how-to-run-jest-with-multiple-test-databases.html

[sgentile]

We just mock out the connection (and typeorm doesn’t make that easy) - it’s an anti pattern to require a database for unit tests. Typeorm biggest downfall to me is the lack of good testing support in the library

[mpvosseller]

@sgentile I generally have two sets of tests in my project. You might call the first set "unit tests" and they mock most things out (including the database). You might call the second set "integration tests" (or something else) and they do not mock most things out (including the database). I think both are highly valuable. My approach above applies to the latter - any tests (whatever you call them) that use the database.

[sgentile]

Yes that is why I said “unit tests”. This is where typeorm can be painful to mock out especially dealing with transactions

[JefferyHus]

I really hope that it becomes more and more testable 😞

[Hiroki111]

Is it possible to create a DB connection only once when running all the test files?

What I'm trying to achieve is to

1. create a DB connection only when setting up the test env
2. close the connection only when exiting test

Now I have the following files:

json.config.js

module.exports = {
  preset: 'ts-jest',
  testEnvironment: 'node',
  globalSetup: '<rootDir>/tests/global-setup.ts',
  setupFilesAfterEnv: ['<rootDir>/tests/db-env.ts'],
  // Here are other lines
};

<rootDir>/tests/global-setup.ts

export default async () => {
  await createConnection(testDbConfig);
  await getConnection().runMigrations();
};

<rootDir>/tests/db-env.ts

import { createConnection, getConnection } from 'typeorm';

beforeAll(async () => {
  await createConnection(testDbConfig);
});
afterAll(async () => {
  await getConnection().close();
});

Although it works, it's slow because a DB connection is created for each test file.

So, I updated jest.config.js into the following

module.exports = {
  preset: 'ts-jest',
  testEnvironment: 'node',
  globalSetup: '<rootDir>/tests/global-setup.ts',
  globalTeardown: '<rootDir>/tests/global-teardown.ts',  // THIS WAS ADDED
  setupFilesAfterEnv: ['<rootDir>/tests/db-env.ts'],
};

Then, the following file was added:

<rootDir>/tests/global-teardown.ts

export default async () => {
  await getConnection().close();
  console.log('Exiting test...');
  process.exit();
};

Then, I commented out beforeEach and afterEach in db-env.ts
<rootDir>/tests/db-env.ts

beforeAll(async () => {
 //  await createConnection(testDbConfig);
});
afterAll(async () => {
 //  await getConnection().close();
});

However, I started to see the following error:

<test case name>

    ConnectionNotFoundError: Connection "default" was not found.

      at new ConnectionNotFoundError (src/error/ConnectionNotFoundError.ts:8:9)
      at ConnectionManager.Object.<anonymous>.ConnectionManager.get (src/connection/ConnectionManager.ts:40:19)
      at Object.getConnection (src/index.ts:252:35)

The error message comes from getRepository function from typeorm. So, it seems that the DB connection that is created in global-setup.ts doesn't exit when actual test files are being executed.

Does this mean it's necessary to create a DB connection for each test file?

[david-minaya]

@Hiroki111 You should store the connection in a global variable in the globalSetup function and use it to stop the database in the globalTeardown:

globalSetup

export default async () => {
  const connection = await createConnection(testDbConfig);
  await connection.runMigrations();
  (globalThis as any).connection = connection;
};

globalTeardown

export default async () => {
  const connection = (globalThis as any).connection;
  await connection.close();
};

[Hiroki111]

@david-minaya

Thank you for your suggestion, but I'd avoid storing the DB connection to globalThis.
The reason is that, in my app, all the helper functions that persist data are directly using getRepository.

e.g.

export async function createMockService(param?) {
    return getRepository(ServiceEntity).save({ ...defaultParams, ...param });
}

If globalThis is suppose to store the DB connection, I need to rewrite all the functions like the one above:

export async function createMockService(param?) {
    return (globalThis as any).connection.getRepository(ServiceEntity).save({ ...defaultParams, ...param });
}

This may work, but there are many functions like this, so it'll be best NOT to save the DB connection and share it across files