New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: correctly compute update changes for json / jsonb columns with transformers #6929
Merged
AlexMesser
merged 1 commit into
typeorm:master
from
dimitryvolkov:fix-json-transform-update-compute
Mar 27, 2021
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
17 changes: 17 additions & 0 deletions
17
test/other-issues/transformed-json-column-update-compute/entity/json-entity.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
import { Entity } from "../../../../src/decorator/entity/Entity"; | ||
import { Column } from "../../../../src/decorator/columns/Column"; | ||
import { PrimaryGeneratedColumn } from "../../../../src/decorator/columns/PrimaryGeneratedColumn"; | ||
import { VersionColumn } from "../../../../src"; | ||
import { testTransformer } from "../test-transformer"; | ||
|
||
@Entity() | ||
export class DummyJSONEntity { | ||
@PrimaryGeneratedColumn() | ||
id: number; | ||
|
||
@VersionColumn() | ||
version: number; | ||
|
||
@Column({ type: "json", transformer: testTransformer }) | ||
value: Record<string, any>; | ||
} |
17 changes: 17 additions & 0 deletions
17
test/other-issues/transformed-json-column-update-compute/entity/jsonb-entity.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
import { Entity } from "../../../../src/decorator/entity/Entity"; | ||
import { Column } from "../../../../src/decorator/columns/Column"; | ||
import { PrimaryGeneratedColumn } from "../../../../src/decorator/columns/PrimaryGeneratedColumn"; | ||
import { VersionColumn } from "../../../../src"; | ||
import { testTransformer } from "../test-transformer"; | ||
|
||
@Entity() | ||
export class DummyJSONBEntity { | ||
@PrimaryGeneratedColumn() | ||
id: number; | ||
|
||
@VersionColumn() | ||
version: number; | ||
|
||
@Column({ type: "jsonb", transformer: testTransformer }) | ||
value: Record<string, any>; | ||
} |
14 changes: 14 additions & 0 deletions
14
test/other-issues/transformed-json-column-update-compute/test-transformer.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
export const testTransformer = { | ||
to(data: any) { | ||
if ("secretProperty" in data) { | ||
data.secretProperty = `secret-${data.secretProperty}`; | ||
} | ||
return data; | ||
}, | ||
from(data: any) { | ||
if ("secretProperty" in data) { | ||
data.secretProperty = data.secretProperty.split("-")[1]; | ||
} | ||
return data; | ||
} | ||
}; |
53 changes: 53 additions & 0 deletions
53
...r-issues/transformed-json-column-update-compute/transformed-json-column-update-compute.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,53 @@ | ||
import "reflect-metadata"; | ||
import { createTestingConnections, closeTestingConnections, reloadTestingDatabases } from "../../utils/test-utils"; | ||
import { Connection } from "../../../src/connection/Connection"; | ||
import { expect } from "chai"; | ||
import { DummyJSONEntity } from "./entity/json-entity"; | ||
import { DummyJSONBEntity } from "./entity/jsonb-entity"; | ||
|
||
describe("other issues > correctly compute change for transformed json / jsonb columns", () => { | ||
|
||
let connections: Connection[]; | ||
before(async () => connections = await createTestingConnections({ | ||
entities: [__dirname + "/entity/*{.js,.ts}"], | ||
schemaCreate: true, | ||
dropSchema: true, | ||
enabledDrivers: ["postgres"] | ||
})); | ||
beforeEach(() => reloadTestingDatabases(connections)); | ||
after(() => closeTestingConnections(connections)); | ||
|
||
it("should not update entity if transformed JSON column did not change", () => Promise.all(connections.map(async connection => { | ||
const repository = connection.getRepository(DummyJSONEntity); | ||
|
||
const dummy = repository.create({ | ||
value: { | ||
secretProperty: "hello" | ||
}, | ||
}); | ||
|
||
await repository.save(dummy); | ||
|
||
await repository.save(dummy); | ||
|
||
const dummyEntity = await repository.findOneOrFail(dummy.id); | ||
expect(dummyEntity.version).to.equal(1); | ||
}))); | ||
|
||
it("should not update entity if transformed JSONB column did not change", () => Promise.all(connections.map(async connection => { | ||
const repository = connection.getRepository(DummyJSONBEntity); | ||
|
||
const dummy = repository.create({ | ||
value: { | ||
secretProperty: "hello" | ||
}, | ||
}); | ||
|
||
await repository.save(dummy); | ||
|
||
await repository.save(dummy); | ||
|
||
const dummyEntity = await repository.findOneOrFail(dummy.id); | ||
expect(dummyEntity.version).to.equal(1); | ||
}))); | ||
}); |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does this mean that transformers won't run? Perhaps we want to instead change how we do the comparisons?
I think disabling the transformation outright could lead to undesired behavior.
What do you think - where is this comparison failing that we need to not transform it here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From my understanding transformation here is done only for comparison purposes, if you observe the changes originally made in #5700, this was false by default so I don't expect it will affect anything.
As mentioned, the comparison is failing in the json / jsonb case where the code path is different, for all other cases the normalized entity value is again transformed before comparing, I initially thought to do the same for the 'json' / 'jsonb' case but figured transforming JSONs might incur a performance penalty so if I can spare that then why not?
However I open to suggestions :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you link to where the comparison is taking place that you're talking about? I'd like to step through this to verify I understand.
Also, two questions that will help with understanding:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As far as I understand this does not affect transformation of JSON / JSONB values and only affects comparison used to compute changes for update statements.
You can see the comparison being performed as a deep comparison in lines 103 of the same file, also if you look at the original fix #5700 you might gain insight as to what the original intention was and how not handling JSON fields seems like an oversight in the original PR
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sorry for the late response, idobh2 is correct here, the transformation here is only use for computing changes, in the end what's being saved as the changed value is the value on the column, the transformation I altered only affects the current database value used for comparison.
As he also mentioned, you can find the JSON / JSONB case comparison at line 103, before the original fix in #5700 what was being compared are the non-transformed versions of the changed value and DB value, following that fix what's being compared is the entity value and transformed entity value.
This would cause (as the test case shows) the comparison for JSON / JSONB columns with transformers to always show a change, causing an unnecessary update to be generated for such columns