Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: correctly compute update changes for json / jsonb columns with transformers #6929

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
4 changes: 3 additions & 1 deletion src/persistence/SubjectChangedColumnsComputer.ts
Expand Up @@ -61,9 +61,11 @@ export class SubjectChangedColumnsComputer {

// if there is no database entity then all columns are treated as new, e.g. changed
if (subject.databaseEntity) {
// skip transform database value for json / jsonb for comparison later on
const shouldTransformDatabaseEntity = column.type !== "json" && column.type !== "jsonb";

// get database value of the column
let databaseValue = column.getEntityValue(subject.databaseEntity, true);
let databaseValue = column.getEntityValue(subject.databaseEntity, shouldTransformDatabaseEntity);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean that transformers won't run? Perhaps we want to instead change how we do the comparisons?

I think disabling the transformation outright could lead to undesired behavior.

What do you think - where is this comparison failing that we need to not transform it here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From my understanding transformation here is done only for comparison purposes, if you observe the changes originally made in #5700, this was false by default so I don't expect it will affect anything.

As mentioned, the comparison is failing in the json / jsonb case where the code path is different, for all other cases the normalized entity value is again transformed before comparing, I initially thought to do the same for the 'json' / 'jsonb' case but figured transforming JSONs might incur a performance penalty so if I can spare that then why not?
However I open to suggestions :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you link to where the comparison is taking place that you're talking about? I'd like to step through this to verify I understand.

Also, two questions that will help with understanding:

  • So this does not prevent us from running transformers against JSON/JSONB fields?
  • The JSON/JSONB comparisons that you're talking about are doing a deep comparison and not a by-reference comparison?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I understand this does not affect transformation of JSON / JSONB values and only affects comparison used to compute changes for update statements.
You can see the comparison being performed as a deep comparison in lines 103 of the same file, also if you look at the original fix #5700 you might gain insight as to what the original intention was and how not handling JSON fields seems like an oversight in the original PR

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry for the late response, idobh2 is correct here, the transformation here is only use for computing changes, in the end what's being saved as the changed value is the value on the column, the transformation I altered only affects the current database value used for comparison.
As he also mentioned, you can find the JSON / JSONB case comparison at line 103, before the original fix in #5700 what was being compared are the non-transformed versions of the changed value and DB value, following that fix what's being compared is the entity value and transformed entity value.

This would cause (as the test case shows) the comparison for JSON / JSONB columns with transformers to always show a change, causing an unnecessary update to be generated for such columns


// filter out "relational columns" only in the case if there is a relation object in entity
if (column.relationMetadata) {
Expand Down
@@ -0,0 +1,17 @@
import { Entity } from "../../../../src/decorator/entity/Entity";
import { Column } from "../../../../src/decorator/columns/Column";
import { PrimaryGeneratedColumn } from "../../../../src/decorator/columns/PrimaryGeneratedColumn";
import { VersionColumn } from "../../../../src";
import { testTransformer } from "../test-transformer";

@Entity()
export class DummyJSONEntity {
@PrimaryGeneratedColumn()
id: number;

@VersionColumn()
version: number;

@Column({ type: "json", transformer: testTransformer })
value: Record<string, any>;
}
@@ -0,0 +1,17 @@
import { Entity } from "../../../../src/decorator/entity/Entity";
import { Column } from "../../../../src/decorator/columns/Column";
import { PrimaryGeneratedColumn } from "../../../../src/decorator/columns/PrimaryGeneratedColumn";
import { VersionColumn } from "../../../../src";
import { testTransformer } from "../test-transformer";

@Entity()
export class DummyJSONBEntity {
@PrimaryGeneratedColumn()
id: number;

@VersionColumn()
version: number;

@Column({ type: "jsonb", transformer: testTransformer })
value: Record<string, any>;
}
@@ -0,0 +1,14 @@
export const testTransformer = {
to(data: any) {
if ("secretProperty" in data) {
data.secretProperty = `secret-${data.secretProperty}`;
}
return data;
},
from(data: any) {
if ("secretProperty" in data) {
data.secretProperty = data.secretProperty.split("-")[1];
}
return data;
}
};
@@ -0,0 +1,53 @@
import "reflect-metadata";
import { createTestingConnections, closeTestingConnections, reloadTestingDatabases } from "../../utils/test-utils";
import { Connection } from "../../../src/connection/Connection";
import { expect } from "chai";
import { DummyJSONEntity } from "./entity/json-entity";
import { DummyJSONBEntity } from "./entity/jsonb-entity";

describe("other issues > correctly compute change for transformed json / jsonb columns", () => {

let connections: Connection[];
before(async () => connections = await createTestingConnections({
entities: [__dirname + "/entity/*{.js,.ts}"],
schemaCreate: true,
dropSchema: true,
enabledDrivers: ["postgres"]
}));
beforeEach(() => reloadTestingDatabases(connections));
after(() => closeTestingConnections(connections));

it("should not update entity if transformed JSON column did not change", () => Promise.all(connections.map(async connection => {
const repository = connection.getRepository(DummyJSONEntity);

const dummy = repository.create({
value: {
secretProperty: "hello"
},
});

await repository.save(dummy);

await repository.save(dummy);

const dummyEntity = await repository.findOneOrFail(dummy.id);
expect(dummyEntity.version).to.equal(1);
})));

it("should not update entity if transformed JSONB column did not change", () => Promise.all(connections.map(async connection => {
const repository = connection.getRepository(DummyJSONBEntity);

const dummy = repository.create({
value: {
secretProperty: "hello"
},
});

await repository.save(dummy);

await repository.save(dummy);

const dummyEntity = await repository.findOneOrFail(dummy.id);
expect(dummyEntity.version).to.equal(1);
})));
});