Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: correctly pass table with schema when using comment #1139

Merged
merged 4 commits into from
May 1, 2024
Merged

Conversation

Shinigami92
Copy link
Collaborator

fixes #939

@Shinigami92 Shinigami92 added this to the v7.x milestone Apr 30, 2024
@Shinigami92 Shinigami92 self-assigned this Apr 30, 2024
Copy link

github-actions bot commented Apr 30, 2024

Coverage Report

Status Category Percentage Covered / Total
🟢 Lines 95.59% (🎯 90%)
🟰 ±0%
6746 / 7057
🟢 Statements 95.59% (🎯 90%)
🟰 ±0%
6746 / 7057
🟢 Functions 95.4% (🎯 90%)
🟰 ±0%
249 / 261
🟢 Branches 89.88% (🎯 85%)
🟰 ±0%
800 / 890
File Coverage
File Stmts % Branch % Funcs % Lines Uncovered Lines
Unchanged Files
src/db.ts 87.95% 88.46% 87.5% 87.95% 66-68, 98, 101-113, 153-155
src/index.ts 100% 100% 100% 100%
src/migration.ts 75.32% 78.04% 58.33% 75.32% 63, 66-68, 70-82, 111-117, 122-158, 215-216, 241-244, 252-253, 270-273, 302-303
src/migrationBuilder.ts 97.56% 92.3% 87.5% 97.56% 354-358, 497-502, 526-527
src/runner.ts 74.22% 58.92% 80% 74.22% 41, 69-70, 79-80, 83-91, 129-130, 172-175, 184-188, 201, 205-218, 237, 239-245, 248, 261, 273-286, 289-292, 302-303, 312-314, 323, 325-334, 346-353
src/sqlMigration.ts 90.19% 100% 80% 90.19% 45-49
src/types.ts 100% 100% 100% 100%
src/operations/sql.ts 100% 100% 100% 100%
src/operations/casts/createCast.ts 100% 100% 100% 100%
src/operations/casts/dropCast.ts 100% 100% 100% 100%
src/operations/casts/index.ts 100% 100% 100% 100%
src/operations/domains/alterDomain.ts 100% 100% 100% 100%
src/operations/domains/createDomain.ts 100% 100% 100% 100%
src/operations/domains/dropDomain.ts 100% 100% 100% 100%
src/operations/domains/index.ts 100% 100% 100% 100%
src/operations/domains/renameDomain.ts 100% 100% 100% 100%
src/operations/domains/shared.ts 100% 100% 100% 100%
src/operations/extensions/createExtension.ts 100% 100% 100% 100%
src/operations/extensions/dropExtension.ts 100% 100% 100% 100%
src/operations/extensions/index.ts 100% 100% 100% 100%
src/operations/extensions/shared.ts 100% 100% 100% 100%
src/operations/functions/createFunction.ts 100% 100% 100% 100%
src/operations/functions/dropFunction.ts 100% 100% 100% 100%
src/operations/functions/index.ts 100% 100% 100% 100%
src/operations/functions/renameFunction.ts 100% 100% 100% 100%
src/operations/functions/shared.ts 100% 100% 100% 100%
src/operations/grants/grantOnSchemas.ts 100% 100% 100% 100%
src/operations/grants/grantOnTables.ts 100% 100% 100% 100%
src/operations/grants/grantRoles.ts 100% 100% 100% 100%
src/operations/grants/index.ts 100% 100% 100% 100%
src/operations/grants/revokeOnSchemas.ts 100% 100% 100% 100%
src/operations/grants/revokeOnTables.ts 100% 100% 100% 100%
src/operations/grants/revokeRoles.ts 100% 100% 100% 100%
src/operations/grants/shared.ts 98.63% 85.71% 100% 98.63% 71
src/operations/indexes/createIndex.ts 98.03% 95.23% 100% 98.03% 74-75
src/operations/indexes/dropIndex.ts 100% 100% 100% 100%
src/operations/indexes/index.ts 100% 100% 100% 100%
src/operations/indexes/shared.ts 91.17% 86.95% 100% 91.17% 22, 32-35, 47
src/operations/materializedViews/alterMaterializedView.ts 100% 100% 100% 100%
src/operations/materializedViews/createMaterializedView.ts 100% 100% 100% 100%
src/operations/materializedViews/dropMaterializedView.ts 100% 100% 100% 100%
src/operations/materializedViews/index.ts 100% 100% 100% 100%
src/operations/materializedViews/refreshMaterializedView.ts 100% 100% 100% 100%
src/operations/materializedViews/renameMaterializedView.ts 100% 100% 100% 100%
src/operations/materializedViews/renameMaterializedViewColumn.ts 100% 100% 100% 100%
src/operations/materializedViews/shared.ts 100% 87.5% 100% 100%
src/operations/operators/addToOperatorFamily.ts 100% 100% 100% 100%
src/operations/operators/createOperator.ts 100% 100% 100% 100%
src/operations/operators/createOperatorClass.ts 100% 83.33% 100% 100%
src/operations/operators/createOperatorFamily.ts 100% 100% 100% 100%
src/operations/operators/dropOperator.ts 100% 100% 100% 100%
src/operations/operators/dropOperatorClass.ts 100% 100% 100% 100%
src/operations/operators/dropOperatorFamily.ts 100% 100% 100% 100%
src/operations/operators/index.ts 100% 100% 100% 100%
src/operations/operators/removeFromOperatorFamily.ts 100% 100% 100% 100%
src/operations/operators/renameOperatorClass.ts 100% 100% 100% 100%
src/operations/operators/renameOperatorFamily.ts 100% 100% 100% 100%
src/operations/operators/shared.ts 90% 75% 100% 90% 24-25, 37-38
src/operations/policies/alterPolicy.ts 100% 100% 100% 100%
src/operations/policies/createPolicy.ts 100% 100% 100% 100%
src/operations/policies/dropPolicy.ts 100% 100% 100% 100%
src/operations/policies/index.ts 100% 100% 100% 100%
src/operations/policies/renamePolicy.ts 100% 100% 100% 100%
src/operations/policies/shared.ts 100% 100% 100% 100%
src/operations/roles/alterRole.ts 100% 100% 100% 100%
src/operations/roles/createRole.ts 100% 75% 100% 100%
src/operations/roles/dropRole.ts 100% 100% 100% 100%
src/operations/roles/index.ts 100% 100% 100% 100%
src/operations/roles/renameRole.ts 100% 100% 100% 100%
src/operations/roles/shared.ts 98.97% 76.92% 100% 98.97% 78
src/operations/schemas/createSchema.ts 100% 100% 100% 100%
src/operations/schemas/dropSchema.ts 100% 100% 100% 100%
src/operations/schemas/index.ts 100% 100% 100% 100%
src/operations/schemas/renameSchema.ts 100% 100% 100% 100%
src/operations/sequences/alterSequence.ts 96.87% 75% 100% 96.87% 23
src/operations/sequences/createSequence.ts 100% 100% 100% 100%
src/operations/sequences/dropSequence.ts 100% 100% 100% 100%
src/operations/sequences/index.ts 100% 100% 100% 100%
src/operations/sequences/renameSequence.ts 100% 100% 100% 100%
src/operations/sequences/shared.ts 87.67% 68.75% 100% 87.67% 41, 43-44, 49-50, 63-64, 69-70
src/operations/tables/addColumns.ts 100% 80% 100% 100%
src/operations/tables/addConstraint.ts 100% 100% 100% 100%
src/operations/tables/alterColumn.ts 92.5% 76.47% 100% 92.5% 75, 82-89
src/operations/tables/alterTable.ts 100% 100% 100% 100%
src/operations/tables/createTable.ts 90.47% 66.66% 100% 90.47% 44-48, 65, 75-76
src/operations/tables/dropColumns.ts 100% 100% 100% 100%
src/operations/tables/dropConstraint.ts 100% 100% 100% 100%
src/operations/tables/dropTable.ts 100% 100% 100% 100%
src/operations/tables/index.ts 100% 100% 100% 100%
src/operations/tables/renameColumn.ts 100% 100% 100% 100%
src/operations/tables/renameConstraint.ts 100% 100% 100% 100%
src/operations/tables/renameTable.ts 100% 100% 100% 100%
src/operations/tables/shared.ts 88.47% 78.46% 80% 88.47% 137-138, 141-142, 195-199, 224, 228-229, 248-249, 274-275, 278-285, 288-289, 426-451
src/operations/triggers/createTrigger.ts 90.83% 68.18% 100% 90.83% 53-54, 66-67, 70-71, 74-77, 113
src/operations/triggers/dropTrigger.ts 100% 100% 100% 100%
src/operations/triggers/index.ts 100% 100% 100% 100%
src/operations/triggers/renameTrigger.ts 100% 100% 100% 100%
src/operations/triggers/shared.ts 100% 100% 100% 100%
src/operations/types/addTypeAttribute.ts 100% 100% 100% 100%
src/operations/types/addTypeValue.ts 100% 100% 100% 100%
src/operations/types/createType.ts 100% 100% 100% 100%
src/operations/types/dropType.ts 100% 100% 100% 100%
src/operations/types/dropTypeAttribute.ts 100% 100% 100% 100%
src/operations/types/index.ts 100% 100% 100% 100%
src/operations/types/renameType.ts 100% 100% 100% 100%
src/operations/types/renameTypeAttribute.ts 100% 100% 100% 100%
src/operations/types/renameTypeValue.ts 100% 100% 100% 100%
src/operations/types/setTypeAttribute.ts 100% 100% 100% 100%
src/operations/views/alterView.ts 100% 100% 100% 100%
src/operations/views/alterViewColumn.ts 100% 100% 100% 100%
src/operations/views/createView.ts 100% 100% 100% 100%
src/operations/views/dropView.ts 100% 100% 100% 100%
src/operations/views/index.ts 100% 100% 100% 100%
src/operations/views/renameView.ts 100% 100% 100% 100%
src/operations/views/shared.ts 100% 66.66% 100% 100%
src/utils/PgLiteral.ts 96.49% 100% 66.66% 96.49% 37-38
src/utils/StringIdGenerator.ts 100% 100% 100% 100%
src/utils/createSchemalize.ts 100% 100% 100% 100%
src/utils/createTransformer.ts 100% 100% 100% 100%
src/utils/decamelize.ts 100% 100% 100% 100%
src/utils/escapeValue.ts 100% 100% 100% 100%
src/utils/formatLines.ts 100% 100% 100% 100%
src/utils/formatParams.ts 100% 100% 100% 100%
src/utils/getMigrationTableSchema.ts 100% 100% 100% 100%
src/utils/getSchemas.ts 100% 100% 100% 100%
src/utils/identity.ts 100% 100% 100% 100%
src/utils/index.ts 100% 100% 100% 100%
src/utils/intersection.ts 100% 100% 100% 100%
src/utils/makeComment.ts 100% 100% 100% 100%
src/utils/quote.ts 100% 100% 100% 100%
src/utils/toArray.ts 100% 100% 100% 100%
src/utils/types.ts 100% 100% 100% 100%
Generated in workflow #729

@Shinigami92 Shinigami92 added c: bug Something isn't working p: 1-normal Nothing urgent labels Apr 30, 2024
@Shinigami92 Shinigami92 requested a review from osdiab April 30, 2024 15:51
@Shinigami92 Shinigami92 changed the title test(constraints): add comment fix(constraints): fix table with schema when using comment Apr 30, 2024
@Shinigami92 Shinigami92 changed the title fix(constraints): fix table with schema when using comment fix(constraints): correctly pass table with schema when using comment Apr 30, 2024
@Shinigami92 Shinigami92 marked this pull request as ready for review April 30, 2024 15:52
@Shinigami92
Copy link
Collaborator Author

@osdiab Would you like to do a first review with approval? 😊

Copy link
Collaborator

@osdiab osdiab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me—just a question about how the tests functionally work in the project. Thanks and ringing in my first review! 💪

@@ -383,7 +383,7 @@ export function parseConstraints(
comments.push(
makeComment(
`CONSTRAINT ${name} ON`,
literal(tableName),
literal(table),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking purely at the code, this looks sound, given that the literal function passed in should be a function that resolves the fully qualified name this looks like it would do the trick. Reviewing the actual issue, since it's just lacking the table schema, this would do the job. Thanks!

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, an alternative (I did not wanted to go in this PR) would be to check for potential quotes and a dot . and if there is some, then normalize it.

e.g. literal "just" takes a string, and so if e.g. there was already a literal-processed value (literal(literal({ schema: "myschema", name: "mytable" })) => literal("myschema"."mytable")) would currently result in double-quoted ""myschema"."mytable"", which is obviously an error.
So instead the literal function itself could e.g. remove all quotes, and if there is a dot, then use schema + name.

is_remark: 'boolean',
}
);
pgm.addConstraint(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do these tests just make sure that the migrations themselves don't error? or is there some notion of checking that the resulting schema comes out how was intended? I see that the integration tests invoked in the postgres-test.yml action runs these migrations, but a little surprised there's no assertions/expectations in the tests themselves.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PostgreSQL itself would fail the migration. Look here for an example of a previous commit that failed: https://github.com/salsita/node-pg-migrate/actions/runs/8897624981/job/24432928126?pr=1139#step:8:321

Example failed action
### MIGRATION 014_add_constraint (UP) ###
ALTER TABLE "t1"
  ADD CONSTRAINT "chck_nmbr" CHECK (nmbr < 30);
COMMENT ON CONSTRAINT "chck_nmbr" ON "t1" IS $pga$nmbr must be less than 30$pga$;
CREATE SCHEMA "payroll_reports";
CREATE TABLE "payroll_reports"."upload_headers" (
  "id" serial PRIMARY KEY,
  "paycode_type" text,
  "aggregate_paycode_type" text,
  "meta_field_type" text,
  "is_remark" boolean
);
ALTER TABLE "payroll_reports"."upload_headers"
  ADD CONSTRAINT "chk_only_one_header_type" CHECK ((       CASE WHEN paycode_type IS NOT NULL THEN 1 ELSE 0 END +       CASE WHEN aggregate_paycode_type IS NOT NULL THEN 1 ELSE 0 END +       CASE WHEN meta_field_type IS NOT NULL THEN 1 ELSE 0 END +       CASE WHEN is_remark THEN 1 ELSE 0 END      ) <= 1);
COMMENT ON CONSTRAINT "chk_only_one_header_type" ON "upload_headers" IS $pga$If no type present/truthy, it's ignored; cannot have more than one active at once$pga$;
INSERT INTO "public"."pgmigrations" (name, run_on) VALUES ('014_add_constraint', NOW());


Error executing:
ALTER TABLE "payroll_reports"."upload_headers"
  ADD CONSTRAINT "chk_only_one_header_type" CHECK ((       CASE WHEN paycode_type IS NOT NULL THEN 1 ELSE 0 END +       CASE WHEN aggregate_paycode_type IS NOT NULL THEN 1 ELSE 0 END +       CASE WHEN meta_field_type IS NOT NULL THEN 1 ELSE 0 END +       CASE WHEN is_remark THEN 1 ELSE 0 END      ) <= 1);
COMMENT ON CONSTRAINT "chk_only_one_header_type" ON "upload_headers" IS $pga$If no type present/truthy, it's ignored; cannot have more than one active at once$pga$;
error: relation "upload_headers" does not exist

> Rolling back attempted migration ...

The integration tests are running against real PostgreSQL docker containers in all currently PostgreSQL supported versions => https://www.postgresql.org/
image

@Shinigami92 Shinigami92 changed the title fix(constraints): correctly pass table with schema when using comment fix: correctly pass table with schema when using comment May 1, 2024
@Shinigami92 Shinigami92 merged commit 83e87dc into main May 1, 2024
35 checks passed
@Shinigami92 Shinigami92 deleted the issue-939 branch May 1, 2024 09:00
@osdiab
Copy link
Collaborator

osdiab commented May 2, 2024 via email

@Shinigami92
Copy link
Collaborator Author

That makes sense to me, to have literal basically behave idempotently, though the mechanism would need to be constructed with a little care.

It seems that Postgres schema and table names can in fact have periods in them. So if you had "schema.with.dots"."table.with.dots" then if you stripped the quotes first, you'd end up with "schema"."with.dots.table.with.dots" which isn't right. So if the literal function is made to be aware of this aspect of Postgres, it probably should handle detecting a schema first.

But that would also mean that if you actually passed in as a user, the name 'my.table' and you didn't want my to be interpreted as a schema but rather that this would be the table name, then the correct behavior would be ambiguous. That said, this seems rare and it doesn't seem like it would be a problem just to put in the docs that if you want to have periods in your table name, you need to use the object syntax. Sound good?

😲 I did not know that dots . are possible in names.
Yeah in this case, I think it is not a good idea yet, because the implementation would be much more complex.
Instead it can be just solved by just passing the correct think to the literal function as we did in this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c: bug Something isn't working c: test p: 1-normal Nothing urgent
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Adding a comment to addConstraint for a table in a schema fails
2 participants