Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: gajus/slonik
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v31.2.4
Choose a base ref
...
head repository: gajus/slonik
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: v31.2.5
Choose a head ref
  • 6 commits
  • 6 files changed
  • 1 contributor

Commits on Oct 2, 2022

  1. Copy the full SHA
    4817776 View commit details

Commits on Oct 5, 2022

  1. Copy the full SHA
    edcae15 View commit details
  2. Copy the full SHA
    844ddba View commit details
  3. Copy the full SHA
    6bb7c65 View commit details
  4. docs: warn about SELECT INTO

    gajus committed Oct 5, 2022
    Copy the full SHA
    6c30a28 View commit details
  5. fix: correct @volatile condition

    gajus committed Oct 5, 2022
    Copy the full SHA
    31630cf View commit details
Showing with 128 additions and 11 deletions.
  1. +10 −0 .README/MIGRATIONS.md
  2. +2 −0 .README/README.md
  3. +33 −5 .README/RECIPES.md
  4. +47 −5 README.md
  5. +0 −1 src/factories/createConnection.ts
  6. +36 −0 test/helpers/createIntegrationTests.ts
10 changes: 10 additions & 0 deletions .README/MIGRATIONS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
## Migrations

This library intentionally doesn't handle migrations, because a database client and migrations are conceptually distinct problems.

My personal preference is to use [Flyway](https://flywaydb.org/) – it is a robust solution that many DBAs are already familiar with.

The Slonik community has also shared their successes with these Node.js frameworks:

* [`node-pg-migrate`](https://github.com/salsita/node-pg-migrate)
* [`@slonik/migrator`](https://www.npmjs.com/package/@slonik/migrator)
2 changes: 2 additions & 0 deletions .README/README.md
Original file line number Diff line number Diff line change
@@ -77,6 +77,8 @@ Note: Using this project does not require TypeScript. It is a regular ES6 module

{"gitdown": "include", "file": "./ERROR_HANDLING.md"}

{"gitdown": "include", "file": "./MIGRATIONS.md"}

{"gitdown": "include", "file": "./TYPES.md"}

{"gitdown": "include", "file": "./DEBUGGING.md"}
38 changes: 33 additions & 5 deletions .README/RECIPES.md
Original file line number Diff line number Diff line change
@@ -48,19 +48,47 @@ Inserting data this way ensures that the query is stable and reduces the amount

### Routing queries to different connections

If connection is initiated by a query (as opposed to a obtained explicitly using `pool#connect()`), then `beforePoolConnection` interceptor can be used to change the pool that will be used to execute the query, e.g.
A typical load balancing requirement is to route all "logical" read-only queries to a read-only instance. This requirement can be implemented in 2 ways:

1. Create two instances of Slonik (read-write and read-only) and pass them around the application as needed.
1. Use `beforePoolConnection` middleware to assign query to a connection pool based on the query itself.

First option is preferable as it is the most explicit. However, it also has the most overhead to implement.

On the other hand, `beforePoolConnection` makes it easy to route based on conventions, but carries a greater risk of accidentally routing queries with side-effects to a read-only instance.

The first option is self-explanatory to implement, but this recipe demonstrates my convention for using `beforePoolConnection` to route queries.

Note: How you determine which queries are safe to route to a read-only instance is outside of scope for this documentation.

Note: `beforePoolConnection` only works for connections initiated by a query, i.e. `pool#query` and not `pool#connect()`.

Note: This particular implementation does not handle [`SELECT INTO`](https://www.postgresql.org/docs/current/sql-selectinto.html).

```ts
const slavePool = await createPool('postgres://slave');
const masterPool = await createPool('postgres://master', {
interceptors: [
{
beforePoolConnection: (connectionContext, pool) => {
if (connectionContext.query && connectionContext.query.sql.includes('SELECT')) {
return slavePool;
beforePoolConnection: (connectionContext) => {
if (!connectionContext.query?.sql.trim().startsWith('SELECT ')) {
// Returning null falls back to using the DatabasePool from which the query originates.
return null;
}

// This is a convention for the edge-cases where a SELECT query includes a volatile function.
// Adding a @volatile comment anywhere into the query bypasses the read-only route, e.g.
// sql`
// # @volatile
// SELECT write_log()
// `
if (connectionContext.query?.sql.includes('@volatile')) {
return null;
}

return pool;
// Returning an instance of DatabasePool will attempt to run the query using the other connection pool.
// Note that all other interceptors of the pool that the query originated from are short-circuited.
return slavePool;
}
}
]
52 changes: 47 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
@@ -149,6 +149,7 @@ Note: Using this project does not require TypeScript. It is a regular ES6 module
* [Handling `StatementTimeoutError`](#user-content-slonik-error-handling-handling-statementtimeouterror)
* [Handling `UniqueIntegrityConstraintViolationError`](#user-content-slonik-error-handling-handling-uniqueintegrityconstraintviolationerror)
* [Handling `TupleMovedToAnotherPartitionError`](#user-content-slonik-error-handling-handling-tuplemovedtoanotherpartitionerror)
* [Migrations](#user-content-slonik-migrations)
* [Types](#user-content-slonik-types)
* [Debugging](#user-content-slonik-debugging)
* [Logging](#user-content-slonik-debugging-logging)
@@ -1154,19 +1155,47 @@ Inserting data this way ensures that the query is stable and reduces the amount
<a name="slonik-recipes-routing-queries-to-different-connections"></a>
### Routing queries to different connections

If connection is initiated by a query (as opposed to a obtained explicitly using `pool#connect()`), then `beforePoolConnection` interceptor can be used to change the pool that will be used to execute the query, e.g.
A typical load balancing requirement is to route all "logical" read-only queries to a read-only instance. This requirement can be implemented in 2 ways:

1. Create two instances of Slonik (read-write and read-only) and pass them around the application as needed.
1. Use `beforePoolConnection` middleware to assign query to a connection pool based on the query itself.

First option is preferable as it is the most explicit. However, it also has the most overhead to implement.

On the other hand, `beforePoolConnection` makes it easy to route based on conventions, but carries a greater risk of accidentally routing queries with side-effects to a read-only instance.

The first option is self-explanatory to implement, but this recipe demonstrates my convention for using `beforePoolConnection` to route queries.

Note: How you determine which queries are safe to route to a read-only instance is outside of scope for this documentation.

Note: `beforePoolConnection` only works for connections initiated by a query, i.e. `pool#query` and not `pool#connect()`.

Note: This particular implementation does not handle [`SELECT INTO`](https://www.postgresql.org/docs/current/sql-selectinto.html).

```ts
const slavePool = await createPool('postgres://slave');
const masterPool = await createPool('postgres://master', {
interceptors: [
{
beforePoolConnection: (connectionContext, pool) => {
if (connectionContext.query && connectionContext.query.sql.includes('SELECT')) {
return slavePool;
beforePoolConnection: (connectionContext) => {
if (!connectionContext.query?.sql.trim().startsWith('SELECT ')) {
// Returning null falls back to using the DatabasePool from which the query originates.
return null;
}

return pool;
// This is a convention for the edge-cases where a SELECT query includes a volatile function.
// Adding a @volatile comment anywhere into the query bypasses the read-only route, e.g.
// sql`
// # @volatile
// SELECT write_log()
// `
if (connectionContext.query?.sql.includes('@volatile')) {
return null;
}

// Returning an instance of DatabasePool will attempt to run the query using the other connection pool.
// Note that all other interceptors of the pool that the query originated from are short-circuited.
return slavePool;
}
}
]
@@ -2734,6 +2763,19 @@ await pool.connect(async (connection0) => {
`TupleMovedToAnotherPartitionError` is thrown when [`affecting tuple moved into different partition`](https://github.com/postgres/postgres/commit/f16241bef7cc271bff60e23de2f827a10e50dde8).


<a name="user-content-slonik-migrations"></a>
<a name="slonik-migrations"></a>
## Migrations

This library intentionally doesn't handle migrations, because a database client and migrations are conceptually distinct problems.

My personal preference is to use [Flyway](https://flywaydb.org/) – it is a robust solution that many DBAs are already familiar with.

The Slonik community has also shared their successes with these Node.js frameworks:

* [`node-pg-migrate`](https://github.com/salsita/node-pg-migrate)
* [`@slonik/migrator`](https://www.npmjs.com/package/@slonik/migrator)

<a name="user-content-slonik-types"></a>
<a name="slonik-types"></a>
## Types
1 change: 0 additions & 1 deletion src/factories/createConnection.ts
Original file line number Diff line number Diff line change
@@ -71,7 +71,6 @@ const destroyBoundConnection = (boundConnection: DatabasePoolConnection) => {
}
};

// eslint-disable-next-line complexity
export const createConnection = async (
parentLog: Logger,
pool: PgPool,
36 changes: 36 additions & 0 deletions test/helpers/createIntegrationTests.ts
Original file line number Diff line number Diff line change
@@ -9,6 +9,7 @@ import {
import {
serializeError,
} from 'serialize-error';
import * as sinon from 'sinon';
import {
z,
} from 'zod';
@@ -146,6 +147,41 @@ export const createIntegrationTests = (
test: TestFn<TestContextType>,
PgPool: new () => PgPoolType,
) => {
test('re-routes query to a different pool', async (t) => {
const readOnlyBeforeTransformQuery = sinon.stub().resolves(null);
const beforeTransformQuery = sinon.stub().throws();

const readOnlyPool = await createPool(t.context.dsn, {
interceptors: [
{
beforeTransformQuery: readOnlyBeforeTransformQuery,
},
],
PgPool,
});

const pool = await createPool(t.context.dsn, {
interceptors: [
{
beforePoolConnection: () => {
return readOnlyPool;
},
beforeTransformQuery,
},
],
PgPool,
});

await pool.query(sql`
SELECT 1
`);

t.true(readOnlyBeforeTransformQuery.calledOnce);
t.true(beforeTransformQuery.notCalled);

await pool.end();
});

test('does not allow to reuse released connection', async (t) => {
const pool = await createPool(t.context.dsn, {
PgPool,