Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: gajus/slonik
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v31.4.2
Choose a base ref
...
head repository: gajus/slonik
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: v32.0.0
Choose a head ref
  • 1 commit
  • 7 files changed
  • 1 contributor

Commits on Oct 20, 2022

  1. Remove baked in result parser in favor of a middleware (#424) 

    * feat: remove default result parser interceptor
* docs: document how to validate results using an interceptor
* test: remove interceptor specific tests

BREAKING CHANGE: query results are not validated by default
    @gajus
    gajus authored Oct 20, 2022
    Copy the full SHA
    87d5e67 View commit details
Showing with 132 additions and 176 deletions.
  1. +63 −1 .README/RUNTIME_VALIDATION.md
  2. +66 −1 README.md
  3. +1 −43 src/routines/executeQuery.ts
  4. +2 −0 src/types.ts
  5. +0 −25 test/helpers/createIntegrationTests.ts
  6. +0 −69 test/slonik/connectionMethods/one.ts
  7. +0 −37 test/slonik/connectionMethods/oneFirst.ts
64 changes: 63 additions & 1 deletion .README/RUNTIME_VALIDATION.md
View file
Original file line number Diff line number Diff line change
@@ -2,7 +2,10 @@

Slonik integrates [zod](https://github.com/colinhacks/zod) to provide runtime query result validation and static type inference.

Runtime validation is added by defining a zod [object](https://github.com/colinhacks/zod#objects) and passing it to `sql.type` tagged template.
Validating queries requires to:

1. Define a Zod [object](https://github.com/colinhacks/zod#objects) and passing it to `sql.type` tagged template (see below)
1. Add a [result parser interceptor](#result-parser-interceptor)

### Motivation

@@ -146,4 +149,63 @@ t.deepEqual(result, {
y: 2,
},
});
```

### Result parser interceptor

Slonik works without the interceptor, but it doesn't validate the query results. To validate results, you must implement an interceptor that parses the results.

For context, when Zod parsing was first introduced to Slonik, it was enabled for all queries by default. However, I eventually realized that the baked-in implementation is not going to suit everyone's needs. For this reason, I decided to take out the built-in interceptor in favor of providing examples for common use cases. What follows is the original default implementation.

```ts
import {
type Interceptor,
type QueryResultRow,
SchemaValidationError,
} from 'slonik';

const createResultParserInterceptor = (): Interceptor => {
return {
// If you are not going to transform results using Zod, then you should use `afterQueryExecution` instead.
// Future versions of Zod will provide a more efficient parser when parsing without transformations.
// You can even combine the two – use `afterQueryExecution` to validate results, and (conditionally)
// transform results as needed in `transformRow`.
transformRow: (executionContext, actualQuery, row) => {
const {
log,
resultParser,
} = executionContext;

if (!resultParser) {
return row;
}

const validationResult = resultParser.safeParse(row);

if (!validationResult.success) {
throw new SchemaValidationError(
actualQuery,
row,
validationResult.error.issues,
);
}

return validationResult.data as QueryResultRow;
},
};
};
```

To use it, simply add it as a middleware:

```ts
import {
createPool,
} from 'slonik';

createPool('postgresql://', {
interceptors: [
createResultParserInterceptor(),
]
});
```
67 changes: 66 additions & 1 deletion README.md
View file
Original file line number Diff line number Diff line change
@@ -100,6 +100,7 @@ Note: Using this project does not require TypeScript. It is a regular ES6 module
* [Handling schema validation errors](#user-content-slonik-runtime-validation-handling-schema-validation-errors)
* [Inferring types](#user-content-slonik-runtime-validation-inferring-types)
* [Transforming results](#user-content-slonik-runtime-validation-transforming-results)
* [Result parser interceptor](#user-content-slonik-runtime-validation-result-parser-interceptor)
* [`sql` tag](#user-content-slonik-sql-tag)
* [Type aliases](#user-content-slonik-sql-tag-type-aliases)
* [Typing `sql` tag](#user-content-slonik-sql-tag-typing-sql-tag)
@@ -1237,7 +1238,10 @@ await connection.query(sql`

Slonik integrates [zod](https://github.com/colinhacks/zod) to provide runtime query result validation and static type inference.

Runtime validation is added by defining a zod [object](https://github.com/colinhacks/zod#objects) and passing it to `sql.type` tagged template.
Validating queries requires to:

1. Define a Zod [object](https://github.com/colinhacks/zod#objects) and passing it to `sql.type` tagged template (see below)
1. Add a [result parser interceptor](#user-content-result-parser-interceptor)

<a name="user-content-slonik-runtime-validation-motivation"></a>
<a name="slonik-runtime-validation-motivation"></a>
@@ -1397,6 +1401,67 @@ t.deepEqual(result, {
});
```

<a name="user-content-slonik-runtime-validation-result-parser-interceptor"></a>
<a name="slonik-runtime-validation-result-parser-interceptor"></a>
### Result parser interceptor

Slonik works without the interceptor, but it doesn't validate the query results. To validate results, you must implement an interceptor that parses the results.

For context, when Zod parsing was first introduced to Slonik, it was enabled for all queries by default. However, I eventually realized that the baked-in implementation is not going to suit everyone's needs. For this reason, I decided to take out the built-in interceptor in favor of providing examples for common use cases. What follows is the original default implementation.

```ts
import {
type Interceptor,
type QueryResultRow,
SchemaValidationError,
} from 'slonik';

const createResultParserInterceptor = (): Interceptor => {
return {
// If you are not going to transform results using Zod, then you should use `afterQueryExecution` instead.
// Future versions of Zod will provide a more efficient parser when parsing without transformations.
// You can even combine the two – use `afterQueryExecution` to validate results, and (conditionally)
// transform results as needed in `transformRow`.
transformRow: (executionContext, actualQuery, row) => {
const {
log,
resultParser,
} = executionContext;

if (!resultParser) {
return row;
}

const validationResult = resultParser.safeParse(row);

if (!validationResult.success) {
throw new SchemaValidationError(
actualQuery,
row,
validationResult.error.issues,
);
}

return validationResult.data as QueryResultRow;
},
};
};
```

To use it, simply add it as a middleware:

```ts
import {
createPool,
} from 'slonik';

createPool('postgresql://', {
interceptors: [
createResultParserInterceptor(),
]
});
```

<a name="user-content-slonik-sql-tag"></a>
<a name="slonik-sql-tag"></a>
## <code>sql</code> tag
44 changes: 1 addition & 43 deletions src/routines/executeQuery.ts
View file
Original file line number Diff line number Diff line change
@@ -8,9 +8,6 @@ import {
import {
serializeError,
} from 'serialize-error';
import {
type ZodTypeAny,
} from 'zod';
import {
TRANSACTION_ROLLBACK_ERROR_PREFIX,
} from '../constants';
@@ -20,7 +17,6 @@ import {
ForeignKeyIntegrityConstraintViolationError,
InvalidInputError,
NotNullIntegrityConstraintViolationError,
SchemaValidationError,
StatementCancelledError,
StatementTimeoutError,
TupleMovedToAnotherPartitionError,
@@ -45,7 +41,6 @@ import {
} from '../types';
import {
createQueryId,
sanitizeObject,
} from '../utilities';

type GenericQueryResult = QueryResult<QueryResultRow>;
@@ -65,34 +60,6 @@ type TransactionQuery = {
readonly values: readonly PrimitiveValueExpression[],
};

const createParseInterceptor = (parser: ZodTypeAny): Interceptor => {
return {
transformRow: (executionContext, actualQuery, row) => {
const {
log,
} = executionContext;

const validationResult = parser.safeParse(row);

if (!validationResult.success) {
log.error({
error: serializeError(validationResult.error),
row: sanitizeObject(row),
sql: actualQuery.sql,
}, 'row failed validation');

throw new SchemaValidationError(
actualQuery,
sanitizeObject(row),
validationResult.error.issues,
);
}

return validationResult.data as QueryResultRow;
},
};
};

const retryQuery = async (
connectionLogger: Logger,
connection: PgPoolClient,
@@ -219,6 +186,7 @@ export const executeQuery = async (
poolId: poolClientState.poolId,
queryId,
queryInputTime,
resultParser: slonikSqlRename.parser,
sandbox: {},
stackTrace,
transactionId: poolClientState.transactionId,
@@ -382,18 +350,8 @@ export const executeQuery = async (

// Stream does not have `rows` in the result object and all rows are already transformed.
if (result.rows) {
const {
parser,
} = slonikSqlRename;

const interceptors: Interceptor[] = clientConfiguration.interceptors.slice();

if (parser) {
interceptors.push(
createParseInterceptor(parser),
);
}

for (const interceptor of interceptors) {
if (interceptor.transformRow) {
const {
2 changes: 2 additions & 0 deletions src/types.ts
View file
Original file line number Diff line number Diff line change
@@ -268,6 +268,7 @@ export type IntervalInput = {
* @property poolId Unique connection pool ID.
* @property queryId Unique query ID.
* @property queryInputTime `process.hrtime.bigint()` for when query was received.
* @property resultParser A Zod function that parses the query result.
* @property sandbox Object used by interceptors to assign interceptor-specific, query-specific context.
* @property transactionId Unique transaction ID.
*/
@@ -278,6 +279,7 @@ export type QueryContext = {
readonly poolId: string,
readonly queryId: QueryId,
readonly queryInputTime: bigint | number,
readonly resultParser?: ZodTypeAny,
readonly sandbox: Record<string, unknown>,
readonly stackTrace: readonly CallSite[] | null,
readonly transactionId: string | null,
25 changes: 0 additions & 25 deletions test/helpers/createIntegrationTests.ts
View file
Original file line number Diff line number Diff line change
@@ -28,9 +28,6 @@ import {
import {
Logger,
} from '../../src/Logger';
import {
type SchemaValidationError,
} from '../../src/errors';

const POSTGRES_DSN = process.env.POSTGRES_DSN ?? 'postgres@localhost:5432';

@@ -260,28 +257,6 @@ export const createIntegrationTests = (
await pool.end();
});

test('validates results using zod (fails)', async (t) => {
const pool = await createPool(t.context.dsn, {
PgPool,
});

const error = await t.throwsAsync<SchemaValidationError>(pool.one(sql.type(z.object({
foo: z.number(),
}))`
SELECT 'bar' foo
`));

if (!error) {
throw new Error('Expected SchemaValidationError');
}

t.like(error.issues[0], {
code: 'invalid_type',
});

await pool.end();
});

// We have to test serialization due to the use of different drivers (pg and postgres).
test('serializes json', async (t) => {
const pool = await createPool(t.context.dsn, {
69 changes: 0 additions & 69 deletions test/slonik/connectionMethods/one.ts
View file
Original file line number Diff line number Diff line change
@@ -6,7 +6,6 @@ import {
z,
} from 'zod';
import {
type SchemaValidationError,
DataIntegrityError,
NotFoundError,
} from '../../../src/errors';
@@ -93,71 +92,3 @@ test('describes zod object associated with the query', async (t) => {
foo: 1,
});
});

test('uses zod transform', async (t) => {
const pool = await createPool();

pool.querySpy.returns({
rows: [
{
foo: '1,2',
},
],
});

const coordinatesType = z.string().transform((subject) => {
const [
x,
y,
] = subject.split(',');

return {
x: Number(x),
y: Number(y),
};
});

const zodObject = z.object({
foo: coordinatesType,
});

const query = sql.type(zodObject)`SELECT '1,2' as foo`;

const result = await pool.one(query);

expectTypeOf(result).toMatchTypeOf<{foo: {x: number, y: number, }, }>();

t.deepEqual(result, {
foo: {
x: 1,
y: 2,
},
});
});

test('throws an error if property type does not conform to zod object (invalid_type)', async (t) => {
const pool = await createPool();

pool.querySpy.returns({
rows: [
{
foo: '1',
},
],
});

const zodObject = z.object({
foo: z.number(),
});

const query = sql.type(zodObject)`SELECT 1`;

const error = await t.throwsAsync<SchemaValidationError>(pool.one(query));

if (!error) {
throw new Error('Expected SchemaValidationError');
}

t.is(error.issues.length, 1);
t.is(error.issues[0]?.code, 'invalid_type');
});
37 changes: 0 additions & 37 deletions test/slonik/connectionMethods/oneFirst.ts
View file
Original file line number Diff line number Diff line change
@@ -6,7 +6,6 @@ import {
z,
} from 'zod';
import {
type SchemaValidationError,
DataIntegrityError,
NotFoundError,
UnexpectedStateError,
@@ -107,39 +106,3 @@ test('describes zod object associated with the query', async (t) => {

t.is(result, 1);
});

test('throws an error if object does match the zod object shape', async (t) => {
const pool = await createPool();

pool.querySpy.returns({
rows: [
{
foo: '1',
},
],
});

const zodObject = z.object({
foo: z.number(),
});

const query = sql.type(zodObject)`SELECT 1`;

const error = await t.throwsAsync<SchemaValidationError>(pool.oneFirst(query));

t.is(error?.sql, 'SELECT 1');
t.deepEqual(error?.row, {
foo: '1',
});
t.deepEqual(error?.issues, [
{
code: 'invalid_type',
expected: 'number',
message: 'Expected number, received string',
path: [
'foo',
],
received: 'string',
},
]);
});