Getting Databricks error using an Oracle jdbc URL #5721
Comments
|
ok, after further testing various combinations I found problems have existed since the Databricks 1.0.1 extension was released, but got much worse with extension 1.1.2. liquibase 4.26 (and 4.25.0) and databricks 1.1.2 - Hard Fail Liquibase 4.26 (and 4.25.0) and databricks 1.0.1 - Works but with large delay and Databricks ext messages "INFO [liquibase.ext] Sequences are not supported by Databricks" |
|
Hi @doakd, thank you for reporting this issue. I've tested this using Liquibase version 4.26.0, liquibase-databricks versions 1.1.2 and 1.1.3, using the changeset you've provided. In my case, the changeset runs with no issues and the sequence is created. On the performance side of things it's clearly something we should look into, but it isn't blocking the use of Liquibase on my part. Can you confirm that it's that changeset that's causing the error? If you find a different one, I might be able to replicate the error and better understand what's going on. Thank you, |
|
Running Liquibase 4.26 with Databricks 1.1.2.
Getting this (error lines repeat 100s of times, and the run count number is garbase, only 8 changesets executed): |
|
ok, I discovered something new. It looks like it's the precondition that is causing the error. Both custom sql and the change-type fail when there is a precondition, and both work fine when the precondition is removed. |
|
@doakd thank you for providing more info. I still can't replicate what you're reporting, but it's known that Liquibase extensions can sometimes get in the way of executions.
Please try running your project with 1.1.3 and let me know if it works for you or not. Thank you, |
|
Current configuration: Running against an Oracle 19c database, the preconditions are now working, no longer getting the "Sequences are not supported by Databricks" error. But now I'm getting the following databricks extension error even though the oracle changesets are deployed successfully: |
|
Hello @doakd ! Does the release tag fix works to fix this issue too? |
|
@filipelautert Changeset: Deployment to Oracle: |
|
@tati-qalified can you review this one again? thanks! |
|
@filipelautert I'm still not getting errors that block execution, but the extension is crowding the logs and taking up resources. |
|
So if you add a changeset that use the sequenceExists precondition (as provided above), you have the databaricks extension installed, and you run an oracle deployment, you don't get the error I pasted above? |
|
@doakd I don't, so there's probably some mix of configurations or some collation in your environment that makes the liquibase-databricks extension fail. Still, it shouldn't have to matter as you're not using Databricks. |
|
We are using DataBricks, but this execution that gets the error was for Oracle. It is blocking the execution. The sequenceExists precondition has to be replaced with a sqlCheck precondition in order to workaround the issue. |
Search first
Description
Using xml changelog:
<changeSet id="create_sequence" author="doaksec"> <preConditions onFail="MARK_RAN"> <not> <sequenceExists sequenceName="text_archive_seq" /> </not> </preConditions> <sql> CREATE SEQUENCE text_archive_seq start with 1 increment by 1 minvalue 1 maxvalue 999999999999999999; </sql> </changeSet>(note: it does work if the createSequence change-type is used)
Getting the following error:
Liquibase Version: 4.26.0
[2024-03-21 19:56:59] INFO [liquibase.ui] Liquibase Version: 4.26.0
Liquibase Open Source 4.26.0 by Liquibase
[2024-03-21 19:56:59] INFO [liquibase.ui] Liquibase Open Source 4.26.0 by Liquibase
[2024-03-21 19:56:59] INFO [liquibase.integration] Starting command execution.
[2024-03-21 19:57:00] INFO [liquibase.changelog] Reading from DMARCHIVE.DATABASECHANGELOG
[2024-03-21 19:57:01] INFO [liquibase.lockservice] Successfully acquired change log lock
[2024-03-21 19:57:01] INFO [liquibase.command] Using deploymentId: 1051021154
[2024-03-21 19:57:01] INFO [liquibase.changelog] Reading from DMARCHIVE.DATABASECHANGELOG
Running Changeset: src/main/resources/db/changelog/db-change-14.xml::36::beckem7
[2024-03-21 19:57:01] INFO [liquibase.ui] Running Changeset: src/main/resources/db/changelog/db-change-14.xml::36::beckem7
[2024-03-21 19:57:01] INFO [liquibase.ext] Sequences are not supported by Databricks
[2024-03-21 19:57:01] INFO [liquibase.snapshot] Creating snapshot
UPDATE SUMMARY
Run: 4
Previously run: 40
Filtered out: 0
Total change sets: 44
[2024-03-21 19:57:01] INFO [liquibase.util] UPDATE SUMMARY
[2024-03-21 19:57:01] INFO [liquibase.util] Run: 4
[2024-03-21 19:57:01] INFO [liquibase.util] Previously run: 40
[2024-03-21 19:57:01] INFO [liquibase.util] Filtered out: 0
[2024-03-21 19:57:01] INFO [liquibase.util] -------------------------------
[2024-03-21 19:57:01] INFO [liquibase.util] Total change sets: 44
[2024-03-21 19:57:01] INFO [liquibase.util] Update summary generated
[2024-03-21 19:57:01] INFO [liquibase.lockservice] Successfully released change log lock
[2024-03-21 19:57:01] INFO [liquibase.command] Command execution complete
[2024-03-21 19:57:01] SEVERE [liquibase.integration] java.lang.StackOverflowError
java.lang.StackOverflowError
at liquibase.ext.databricks.snapshot.jvm.SequenceSnapshotGeneratorDatabricks.getSelectSequenceStatement(SequenceSnapshotGeneratorDatabricks.java:42)
at liquibase.ext.databricks.snapshot.jvm.SequenceSnapshotGeneratorDatabricks.getSelectSequenceStatement(SequenceSnapshotGeneratorDatabricks.java:59)
at liquibase.ext.databricks.snapshot.jvm.SequenceSnapshotGeneratorDatabricks.getSelectSequenceStatement(SequenceSnapshotGeneratorDatabricks.java:59)
at liquibase.ext.databricks.snapshot.jvm.SequenceSnapshotGeneratorDatabricks.getSelectSequenceStatement(SequenceSnapshotGeneratorDatabricks.java:59)
Steps To Reproduce
JAVA_OPTS="-Xmx1024m" /opt/liquibase/liquibase --changeLogFile=src/main/resources/db/changelog/db-changelog-master.xml --sql-log-level=INFO --showBanner=false --log-level=INFO --defaultsFile=/tmp/tmpliw8rs_j --classpath=/tmp/tmpu6su81ug update
defaultsFile contains:
username: XXXX
password: XXXX
url: jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=XXXXX)(PORT=XXXX)))(CONNECT_DATA=(UR=A)(SERVICE_NAME=XXXXXX)))
Expected/Desired Behavior
Executes using Oracle code path in Liquibase, not Databricks extension.
Liquibase Version
4.26
Database Vendor & Version
Oracle 19c
Liquibase Integration
CLI
Liquibase Extensions
liquibase-oracle 4.26, liquibase-teradata 4.26, liquibase-databricks 1.1.2
OS and/or Infrastructure Type/Provider
debian linux 11
Additional Context
This worked fine with Liquibase 4.25.
This is a major roadblock for us.
Are you willing to submit a PR?
The text was updated successfully, but these errors were encountered: