Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix dependency query for PostgreSQL 15 #3456

Merged
merged 4 commits into from
Nov 29, 2022
Merged

Conversation

Domm98CZ
Copy link
Contributor

@Domm98CZ Domm98CZ commented Nov 9, 2022

Impact

  • Bug fix (non-breaking change which fixes expected existing functionality)
  • Enhancement/New feature (adds functionality without impacting existing logic)
  • Breaking change (fix or feature that would cause existing functionality to change)

Description

Invalid datytype caused sql error and then 0 changes detected.
Hoewer command return success code.

Additional Context

Error in console with --log-level=DEBUG

[2022-11-09 13:28:32] FINE [liquibase.executor] Executing with the 'jdbc' executor
[2022-11-09 13:28:32] FINE [liquibase.diff] Cannot get object dependencies: Error executing SQL WITH RECURSIVE preference AS (
    SELECT 10 AS max_depth  -- The deeper the recursion goes, the slower it performs.
         , 16384 AS min_oid -- user objects only
         , '^(londiste|pgq|pg_toast)'::text AS schema_exclusion
         , '^pg_(conversion|language|ts_(dict|template))'::text AS class_exclusion
         , '{"SCHEMA":"00", "TABLE":"01", "CONSTRAINT":"02", "DEFAULT":"03",
      "INDEX":"05", "SEQUENCE":"06", "TRIGGER":"07", "FUNCTION":"08",
      "VIEW":"10", "MVIEW":"11", "FOREIGN":"12"}'::json AS type_ranks),
               dependency_pair AS (
                   WITH relation_object AS ( SELECT oid, oid::regclass::text AS object_name  FROM pg_class )
                   SELECT DISTINCT                          substring(pg_identify_object(classid, objid, 0)::text, E'(\\w+?)\\.') as referenced_schema_name,                          CASE classid
                              WHEN 'pg_constraint'::regclass THEN (SELECT CONTYPE FROM pg_constraint WHERE oid = objid)
                              ELSE objid::text
                              END AS CONTYPE,
                         CASE classid
                              WHEN 'pg_attrdef'::regclass THEN (SELECT attname FROM pg_attrdef d JOIN pg_attribute c ON (c.attrelid,c.attnum)=(d.adrelid,d.adnum) WHERE d.oid = objid)
                              WHEN 'pg_cast'::regclass THEN (SELECT concat(castsource::regtype::text, ' AS ', casttarget::regtype::text,' WITH ', castfunc::regprocedure::text) FROM pg_cast WHERE oid = objid)
                              WHEN 'pg_class'::regclass THEN rel.object_name
                              WHEN 'pg_constraint'::regclass THEN (SELECT conname FROM pg_constraint WHERE oid = objid)
                              WHEN 'pg_extension'::regclass THEN (SELECT extname FROM pg_extension WHERE oid = objid)
                              WHEN 'pg_namespace'::regclass THEN (SELECT nspname FROM pg_namespace WHERE oid = objid)
                              WHEN 'pg_opclass'::regclass THEN (SELECT opcname FROM pg_opclass WHERE oid = objid)
                              WHEN 'pg_operator'::regclass THEN (SELECT oprname FROM pg_operator WHERE oid = objid)
                              WHEN 'pg_opfamily'::regclass THEN (SELECT opfname FROM pg_opfamily WHERE oid = objid)
                              WHEN 'pg_proc'::regclass THEN objid::regprocedure::text
                              WHEN 'pg_rewrite'::regclass THEN (SELECT ev_class::regclass::text FROM pg_rewrite WHERE oid = objid)
                              WHEN 'pg_trigger'::regclass THEN (SELECT tgname FROM pg_trigger WHERE oid = objid)
                              WHEN 'pg_type'::regclass THEN objid::regtype::text
                              ELSE objid::text
                              END AS REFERENCED_NAME,
                          substring(pg_identify_object(refclassid, refobjid, 0)::text, E'(\\w+?)\\.') as referencing_schema_name,                           CASE refclassid
                              WHEN 'pg_namespace'::regclass THEN (SELECT nspname FROM pg_namespace WHERE oid = refobjid)
                              WHEN 'pg_class'::regclass THEN rrel.object_name
                              WHEN 'pg_opfamily'::regclass THEN (SELECT opfname FROM pg_opfamily WHERE oid = refobjid)
                              WHEN 'pg_proc'::regclass THEN refobjid::regprocedure::text
                              WHEN 'pg_type'::regclass THEN refobjid::regtype::text
                              ELSE refobjid::text
                              END AS REFERENCING_NAME
                   FROM pg_depend dep
                            LEFT JOIN relation_object rel ON rel.oid = dep.objid
                            LEFT JOIN relation_object rrel ON rrel.oid = dep.refobjid, preference
                   WHERE deptype = ANY('{n,a}')
                     AND objid >= preference.min_oid
                     AND (refobjid >= preference.min_oid OR refobjid = 2200) -- need public schema as root node
                     AND classid::regclass::text !~ preference.class_exclusion
                     AND refclassid::regclass::text !~ preference.class_exclusion
                     AND COALESCE(SUBSTRING(objid::regclass::text, E'^(\\\\w+)\\\\.'),'') !~ preference.schema_exclusion
                     AND COALESCE(SUBSTRING(refobjid::regclass::text, E'^(\\\\w+)\\\\.'),'') !~ preference.schema_exclusion
                   GROUP BY classid, objid, refclassid, refobjid, deptype, rel.object_name, rrel.object_name
               )
 select referenced_schema_name,
    (CASE
      WHEN position('.' in referenced_name) >0 THEN substring(referenced_name from position('.' in referenced_name)+1 for length(referenced_name))
      ELSE referenced_name
    END)  AS referenced_name,
   referencing_schema_name,
   (CASE
      WHEN position('.' in referencing_name) >0 THEN substring(referencing_name from position('.' in referencing_name)+1 for length(referencing_name))
      ELSE referencing_name
    END)  AS referencing_name from dependency_pair where REFERENCED_NAME != REFERENCING_NAME  AND ( REFERENCED_NAME like 'public.%' OR REFERENCED_NAME NOT LIKE '%.%')
 AND (CONTYPE::text != 'p' AND CONTYPE::text != 'f')
 AND referencing_schema_name is not null and referencing_name is not null: ERROR: CASE types text and "char" cannot be matched
  Position: 939
[2022-11-09 13:28:32] INFO [liquibase.diff] changeSets count: 0
[2022-11-09 13:28:32] INFO [liquibase.diff] No changesets to add.
Liquibase command 'diff-changelog' was executed successfully.

Unchanged sql runned directly on database:
image

Changed sql runned directly on database:
image

Thanks.

@nvoxland nvoxland changed the title PostgreSQL 15, bug in datatype Fix dependency query for PostgreSQL 15 Nov 9, 2022
@nvoxland
Copy link
Contributor

nvoxland commented Nov 9, 2022

Pre-Review Notes

Change makes sense, improves the postgresql query to handle the datatype in an internal query better

Questions I have:

  • None

Potential risks:

  • It should work on older postgresql versions, automated tests will tell us for sure

What could make the full review difficult:

  • Nothing

@filipelautert filipelautert added the SafeToBuild Indicates that a particular PR contains changes which are safe to build using GitHub actions label Nov 18, 2022
@github-actions
Copy link

github-actions bot commented Nov 18, 2022

Unit Test Results

  4 752 files  ±0    4 752 suites  ±0   33m 4s ⏱️ - 4m 39s
  4 721 tests ±0    4 488 ✔️ ±0     233 💤 ±0  0 ±0 
55 668 runs  ±0  50 340 ✔️ ±0  5 328 💤 ±0  0 ±0 

Results for commit b342432. ± Comparison against base commit 344f471.

♻️ This comment has been updated with latest results.

@filipelautert filipelautert added SafeToBuild Indicates that a particular PR contains changes which are safe to build using GitHub actions and removed SafeToBuild Indicates that a particular PR contains changes which are safe to build using GitHub actions labels Nov 18, 2022
Copy link
Collaborator

@filipelautert filipelautert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests passed. Besides that I ran the changed SQL on Postgresql 9.6 and 12 and it worked fine.

@filipelautert filipelautert added this to the 1NEXT milestone Nov 18, 2022
Copy link
Contributor

@XDelphiGrl XDelphiGrl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR fixes a datatype mismatch between text and char when querying pg_constraint.contype, which returns a char. Liquibase was inconsitently casting pg_constraint.contype to text, resulting in the error. This is fixed.

  • Fix is specific to Postgres.
  • Automated functional and test harness executions passing.
  • No additional testing required.

APPROVED

@suryaaki2 suryaaki2 merged commit b97e22e into liquibase:master Nov 29, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
SafeToBuild Indicates that a particular PR contains changes which are safe to build using GitHub actions sprint2022-38 TypeBug
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

None yet

6 participants