Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When JSON contains special escape characters, some characters will be lost after parsing #4323

Closed
1 task done
git-chenhao opened this issue Jan 16, 2024 · 8 comments
Closed
1 task done
Labels
need-test-case To work on issue, a reproduction (ideally unit test) needed

Comments

@git-chenhao
Copy link

git-chenhao commented Jan 16, 2024

Search before asking

  • I searched in the issues and found nothing similar.

Describe the bug

Version Information

2.16.1

Reproduction

public static void main(String[] args) throws JsonProcessingException {
        String body= "{\"name\":\"\\u2028澜\"}";
        System.out.println(body);
        ObjectMapper objectMapper = new ObjectMapper();
        Map map = objectMapper.readValue(body, Map.class);
        String json = objectMapper.writeValueAsString(map);
        System.out.println(json);
        System.out.println(body.equals(json));
    }


// {"name":"\u2028澜"}
// {"name":"
澜"}
// false

Expected behavior

No response

Additional context

No response

@git-chenhao git-chenhao added the to-evaluate Issue that has been received but not yet evaluated label Jan 16, 2024
@pjfanning
Copy link
Member

There is no Jackson class that I know of called Jsons.

@git-chenhao
Copy link
Author

There is no Jackson class that I know of called Jsons.

updated

@git-chenhao
Copy link
Author

@pjfanning Can you help me take a look

@pjfanning
Copy link
Member

Unicode 2028 is a line separator - have you proved that the char in // {"name":"
澜"} is not unicode 2028? Your OS may display unicode 2028 as simple whitespace. There is no obligation for Jackson internals to escape the unicode chars in \u2028 format when it outputs them.

@cowtowncoder
Copy link
Member

@git-chenhao Please do NOT use System.out.println() for testing equality; what gets printed may or may not look the same.

In addition, there is no guarantee that Unicode escaping would be somehow preserved when reading: what matters are logical String values. So test would need to show that decoded String is not what it should be -- basically, that if you read String produced by mapper.writeValueAsString(value), it should result in same Map as the original String.

At this point a unit test would be needed to show incorrect handling of Unicode escapes.

@cowtowncoder cowtowncoder added need-test-case To work on issue, a reproduction (ideally unit test) needed and removed to-evaluate Issue that has been received but not yet evaluated labels Jan 17, 2024
@git-chenhao
Copy link
Author

@git-chenhao请不要用于System.out.println()测试相等性;打印出来的内容可能看起来相同,也可能不一样。

此外,无法保证在读取时会以某种方式保留 Unicode 转义:重要的是逻辑 String 值。因此,测试需要表明解码后的 String 不是它应该的样子 - 基本上,如果您读取由 生成的 String ,它的结果应该与原始 Stringmapper.writeValueAsString(value)相同。Map

此时需要进行单元测试来显示 Unicode 转义的错误处理。

@cowtowncoder
Don't worry about System. out. println(), because body.equals(json) is false
I found that there is a JsonpCharacterEscapes class used to handle Unicode characters' 0x2028 'and' 0x2029 '. Will using this have any other effects?

@cowtowncoder
Copy link
Member

If the goal is to influence which characters are and which are not escaped, CharacterEscapes is the mechanism and yes, JsonpCharacterEscapes in particular might work here.
Its only effect should be added escaping, and possibly some minor performance overhead. Otherwise no effect.

@cowtowncoder
Copy link
Member

No test to actually reproduce the problem (printing out results is not reliable means to show an issue), closing.
May be re-opened/re-filed with a reproduction.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
need-test-case To work on issue, a reproduction (ideally unit test) needed
Projects
None yet
Development

No branches or pull requests

3 participants