Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Oracle to Spark SQL. Failure to unnest queries during optimisation #3448

Closed
yesemsanthoshkumar opened this issue May 10, 2024 · 0 comments · Fixed by #3464
Closed

Oracle to Spark SQL. Failure to unnest queries during optimisation #3448

yesemsanthoshkumar opened this issue May 10, 2024 · 0 comments · Fixed by #3464
Assignees

Comments

@yesemsanthoshkumar
Copy link

yesemsanthoshkumar commented May 10, 2024

Followup from slack message
https://tobiko-data.slack.com/archives/C0448SFS3PF/p1715277495692029?thread_ts=1715277495.692029&cid=C0448SFS3PF

Fully reproducible code snippet

from sqlglot import parse_one
from sqlglot.optimizer import optimize

query = """
select
 pt.c2,
 t2.person_id AS person_id,
 (
  SELECT
  MAX(t3.full_name) AS _col_0
  FROM
    db.third_table AS t3
  WHERE
    t3.person_id = t2.person_id
    AND t3.effective_start_date <= TRUNC(t2.creation_date)
    AND t3.effective_end_date >= TRUNC(t2.creation_date)
  ) AS person_full_name
from
 parent_table as pt,
 second_table as t2
where
 pt.c1 = 'something'
 and pt.second_col_id = t2.second_col_id
"""

tree = parse_one(query, dialect="oracle")
optimize(tree)
print(tree.sql("spark", pretty=True))

Python version: 3.11.2
SQLGlot version: 23.14.0

Expected output
Nested query to be moved to a CTE and referenced to the parent select under a join clause.

The flow goes through when the inner query has a single filter in the where clause. But fails with more than one.

@yesemsanthoshkumar yesemsanthoshkumar changed the title Oracle to Spark SQL. Failure to unnest queries. Oracle to Spark SQL. Failure to unnest queries during optimisation May 10, 2024
@georgesittas georgesittas self-assigned this May 11, 2024
georgesittas added a commit that referenced this issue May 13, 2024
georgesittas added a commit that referenced this issue May 13, 2024
… suite (#3464)

* Fix(optimizer): fix multiple bugs in unnest_subqueries, clean up test suite

* Fix AttributeError issue with subquery projections

* Fix #3448
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants