generate-changelog Fails Without Identifying View or Creating Changelog File Due to Invalid String Encoding Error

[lukas-grigis]

Search first

• I searched and no similar issues were found

Description

I am encountering an issue with the Liquibase generate-changelog command while trying to generate a changelog for a large Oracle schema. Despite following the standard command usage with the necessary parameters (URL, username, password, changelog-file and schemas), the process does not complete successfully.

The operation halts with an UnexpectedLiquibaseException related to an "Invalid string encoding on createView.remarks." According to the error message, the resolution involves removing the invalid character from the database and attempting the operation again. The exact console output is as follows:

ERROR: Exception Details
ERROR: Exception Primary Class:  UnexpectedLiquibaseException
ERROR: Exception Primary Reason: Invalid string encoding on createView.remarks. To resolve, remove the invalid character on the database and try again in changeSet liquibase/initial-changelog.xml::1710407787340-327::root (generated).

Additionally, the initial-changelog.xml file is not created, making it difficult to diagnose which view or remark is causing the issue or review the mentioned changeset. Even after setting the log level to DEBUG, the logs do not reveal the name of the problematic view. Given the large number of views in the schema, pinpointing the offending character without further details is challenging.

Steps To Reproduce

1. Execute the liquibase generate-changelog command with the parameters for connecting to an Oracle database schema.
2. Observe the error regarding invalid string encoding on createView.remarks.

Expected/Desired Behavior

I anticipated that the generate-changelog command would successfully create an initial changelog file for the Oracle schema, or at least provide detailed information about any errors to facilitate troubleshooting.

Liquibase Version

4.26.0

Database Vendor & Version

Oracle 19c

Liquibase Integration

Docker

Liquibase Extensions

None

OS and/or Infrastructure Type/Provider

Docker 4.26.0 #1141 built at 2024-02-06 21:31+0000

Additional Context

No response

Are you willing to submit a PR?

• I'm willing to submit a PR (Thank you!)

[tati-qalified]

Hi @lukas-grigis, thank you for reaching out to us.
I need some more information about your situation in order to test this more efficiently:

1. When you say "a large Oracle schema", do you mean a lot of tables, a lot of data, many database objects, complex procedures, or a mix of all of these (or others)?
2. Have you tried setting diff types to reduce the amount of data that needs to be processed? If so, do you get the same error?
3. There are some known problems with Liquibase when using special characters. Are you using any in your views?

Thank you,
Tatiana

[lukas-grigis]

Hi @tati-qalified

1. When you say "a large Oracle schema", do you mean a lot of tables, a lot of data, many database objects, complex procedures, or a mix of all of these (or others)?

Yes, it's a mix of all of these. What I mean by "Oracle Schema" is a user under Oracle. The schema consists of ca. 100 Tables several (~10) GBs of data and roundabout 7'000 database objects.

2. Have you tried setting diff types to reduce the amount of data that needs to be processed? If so, do you get the same error?

Yes, I tried the diff types to only create tables where the error did not occur. It appears only when generating the views.

3. There are some known problems with Liquibase when using special characters. Are you using any in your views?

No, there are no (known) special characters used within the view declaration.

[tati-qalified]

Looking at the code, this error is probably coming from liquibase.change.core.CreateViewChange#331:

List<Class<?>> databaseSupportsViewComments = Arrays.asList(OracleDatabase.class, PostgresDatabase.class, MSSQLDatabase.class, DB2Database.class,
                SybaseASADatabase.class);
        boolean supportsViewComments = databaseSupportsViewComments.stream().anyMatch(clazz -> clazz.isInstance(database));

        if (supportsViewComments && (StringUtil.trimToNull(remarks) != null)) {
            SetViewRemarksStatement remarksStatement = new SetViewRemarksStatement(catalogName, schemaName, viewName, remarks);
            if (SqlGeneratorFactory.getInstance().supports(remarksStatement, database)) {
                statements.add(remarksStatement);
            }
        }

The generated statement may be adding some special character, or not processing something correctly.

Would you be willing to submit a PR to resolve this?

Thank you,
Tatiana