Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The table field type in the SQL Server database is ntext and synchronized to the Oracle database, resulting in invalid column types in the corresponding NCLOB #4583

Open
austinuMyaccount opened this issue Apr 29, 2024 · 0 comments
Labels
status: waiting-for-triage Issues that we did not analyse yet type: bug

Comments

@austinuMyaccount
Copy link

Please do a quick search on Github issues first, there might be already a duplicate issue for the one you are about to create.
If the bug is trivial, just go ahead and create the issue. Otherwise, please take a few moments and fill in the following sections:

Bug description
The table field type in the SQL Server database is ntext and synchronized to the Oracle database, resulting in invalid column types in the corresponding NCLOB,resulting in data insertion failure

Environment
source DB Microsoft SQL Server 2008 R2 (RTM) - 10.50.1600.1 (X64) Apr 2 2010 15:48:46
target DB Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
Synchronization Tool : spring-batch-core-4.2.2.RELEASE

Steps to reproduce
Steps to reproduce the issue.

 GfDatasource sourceDs = (GfDatasource) this.sTdbMap.get("sourceDs");
        GfDatasource targetDs = (GfDatasource) this.sTdbMap.get("targetDs");
        Boolean isSqlServer = false;
        Boolean isOracle = false;
        if (sourceDs != null) {
            if (StringUtils.isNotBlank(sourceDs.getUrl())) {
                isSqlServer = sourceDs.getUrl().contains("sqlserver");
            }
        }

        if (targetDs != null) {
            if (StringUtils.isNotBlank(targetDs.getUrl())) {
                isOracle = targetDs.getUrl().contains("oracle");
            }
        }
        if (isSqlServer && isOracle) {
            for (Map.Entry<String, Object> entry : item.entrySet()) {
                Object value = entry.getValue();
                if (value instanceof ClobImpl) {
                    ClobImpl clob = (ClobImpl) value;
                    Connection conn = (Connection) this.sTdbMap.get("oracleConn");
                    OracleConnection oracleConn = null;
                    if (conn instanceof OracleConnection) {
                        oracleConn = (OracleConnection) conn;
                    } else {
                        throw new RuntimeException("conn不是OracleConnection");
                    }

                    Reader reader = clob.getCharacterStream();
                    // 创建字符缓冲区
                    StringBuilder stringBuilder = new StringBuilder();
                    // 读取字符流并存储到字符缓冲区中
                    char[] buffer = new char[1024];
                    int length;
                    while ((length = reader.read(buffer)) != -1) {
                        stringBuilder.append(buffer, 0, length);
                    }
                    // 将字符缓冲区中的字符转换为字节数组
                    String str = stringBuilder.toString();
                    byte[] bytes = str.getBytes(“UTF-16LE”);

                    NCLOB nClob = new NCLOB(oracleConn, bytes);
                    char[] charArray = str.toCharArray();
                    nClob.setPrefetchedData(charArray);
                    nClob.setChunkSize(8132);

                    NClob nClob1 = oracleConn.createNClob();
                    int i = nClob1.setString(1, str);


                    // 关闭 Reader
                    reader.close();

                    entry.setValue(nClob);
                }
            }
        }

If I use Oracle to synchronize to Oracle, then the field corresponding to the NCLOB type is successfully synchronized, and it is a byte array,the byte array like this :

 byte[] tmpBytes = {0, 84, 0, 1, 4, 76, 0, 0, 0, 2, 0, 0, 0, 1, 0, 0, 10, 34, -108, -77, 0, 8, -39, -34, 0, 8, -39, -35, 0, 2, 0, 2, 7, -48, 0, 1, 78, 10, 109, 119, 94, 2, 0, 0, -23, 122, -38, 54, 0, 0, 0, 0, 54, -32, 70, -63, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, -39, -35, 1, -117, -21, -113, 0, 0};

But when using SQL Server to synchronize to Oracle, the field type is ntext to NCLOB, and the corresponding type object for ntext is ClobImpl. Therefore, it is necessary to save the byte array stored in this object to the NCLOB object.The byte array obtained from the ClobImpl object looks like this:

        byte[] bytes = {10, 78, 119, 109, 2, 94};

Expected behavior
My goal is to save NCLOB type fields in tables in the SQL Server database that can be synchronized to the Oracle database, such as nvarchar (max) or ntext. I hope to be able to save the field values corresponding to this type normally, or the byte array mentioned above can be successfully saved.

Minimal Complete Reproducible example
Please provide a failing test or a minimal complete verifiable example that reproduces the issue.
Bug reports that are reproducible will take priority in resolution over reports that are not reproducible.

                step = stepBuilderFactory.get(row.getConfName())
                        .listener(new StepExecutionListener(row.getConfName()))
                        .startLimit(1)
                        .allowStartIfComplete(false)
                        .chunk(10000)
                        .reader(initJdbcPagingItemReader(row))
                        .processor(initItemProcessor(map, sTdbMap))
                        .writer(initJdbcBatchItemWriter(row))
                        .exceptionHandler(new DefaultExceptionHandler())
                        .build();

here is

    @Override
    public Map process(Map<String, Object> item) throws Exception {
        GfDatasource sourceDs = (GfDatasource) this.sTdbMap.get("sourceDs");
        GfDatasource targetDs = (GfDatasource) this.sTdbMap.get("targetDs");
        Boolean isSqlServer = false;
        Boolean isOracle = false;
        if (sourceDs != null) {
            if (StringUtils.isNotBlank(sourceDs.getUrl())) {
                isSqlServer = sourceDs.getUrl().contains("sqlserver");
            }
        }

        if (targetDs != null) {
            if (StringUtils.isNotBlank(targetDs.getUrl())) {
                isOracle = targetDs.getUrl().contains("oracle");
            }
        }
        if (isSqlServer && isOracle) {
            for (Map.Entry<String, Object> entry : item.entrySet()) {
                Object value = entry.getValue();
                if (value instanceof ClobImpl) {
                    ClobImpl clob = (ClobImpl) value;
                    Connection conn = (Connection) this.sTdbMap.get("oracleConn");
                    OracleConnection oracleConn = null;
                    if (conn instanceof OracleConnection) {
                        oracleConn = (OracleConnection) conn;
                    } else {
                        throw new RuntimeException("conn不是OracleConnection");
                    }
                    Reader reader = clob.getCharacterStream();
                    StringBuilder stringBuilder = new StringBuilder();
                    char[] buffer = new char[1024];
                    int length;
                    while ((length = reader.read(buffer)) != -1) {
                        stringBuilder.append(buffer, 0, length);
                    }
                    String str = stringBuilder.toString();
                    byte[] bytes = str.getBytes("UTF-16LE");
                    NCLOB nClob = new NCLOB(oracleConn, tmpBytes);
                    char[] charArray = str.toCharArray();
                    nClob.setPrefetchedData(charArray);
                    nClob.setChunkSize(8132);

                    reader.close();
                    entry.setValue(nClob);
                }
            }
        }
        item.put("SEND_BATCH_NO",map.get("SEND_BATCH_NO"));
        return item;
    }
@austinuMyaccount austinuMyaccount added status: waiting-for-triage Issues that we did not analyse yet type: bug labels Apr 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status: waiting-for-triage Issues that we did not analyse yet type: bug
Projects
None yet
Development

No branches or pull requests

1 participant