Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spring unable to decode aggregated JSON content #31747

Closed
filipovskid opened this issue Dec 4, 2023 · 0 comments
Closed

Spring unable to decode aggregated JSON content #31747

filipovskid opened this issue Dec 4, 2023 · 0 comments
Assignees
Labels
in: web Issues in web modules (web, webmvc, webflux, websocket) status: backported An issue that has been backported to maintenance branches type: bug A general bug
Milestone

Comments

@filipovskid
Copy link

filipovskid commented Dec 4, 2023

Affects: Spring Framework 6.0.5+


Issue

Spring fails to decode a JSON payload whose content has been aggregated and represented by at least 2 DataBuffer components, causing the error below to be thrown. This happens only when we have a HttpObjectAggregator in the Netty ChannelPipeline and the content is transformed to a Flux publisher, regardless of whether we block or not when retrieving the data.

com.fasterxml.jackson.core.JsonParseException: Still have 1940 undecoded bytes, should not call 'feedInput'

Even though this is a specific case where we are "bypassing" the reactive nature of WebClient, and I can workaround the issue, I see this as a bug. Please let me know if I am misusing the client and Spring is not supposed to support this use-case.

How to reproduce the issue

Sample Spring Boot project with a data-source application that serves the data and a data-client application where the issue occurs while reading the data To reproduce this issue start both applications and invoke the /get-data-objects endpoint on the client application.

Minimal code

WebClient configuration.

HttpClient httpClient = HttpClient.create()
     .doOnChannelInit((observer, channel, remoteAddress) -> channel.pipeline()
         .addAfter(NettyPipeline.HttpCodec, NettyPipeline.HttpAggregator, new HttpObjectAggregator(1048576)));

ReactorClientHttpConnector connector = new ReactorClientHttpConnector(httpClient );
WebClient webClient = WebClient.builder().baseUrl(BASE_URL)
        .clientConnector(connector)
        .build();

Endpoint invocation.

webClient.get()
     .uri(uriBuilder -> uriBuilder.path("/data").build())
     .retrieve()
     .bodyToFlux(Object.class)
     .collectList()
     .block();

Debugging

I suspect the issue was introduced with the changes to org.springframework.http.codec.json.Jackson2Tokenizer in #29943.

@spring-projects-issues spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged or decided on label Dec 4, 2023
@filipovskid filipovskid changed the title Spring unable to decode aggregated HTTP content messages Spring unable to decode aggregated HTTP content messages to JSON Dec 4, 2023
@filipovskid filipovskid changed the title Spring unable to decode aggregated HTTP content messages to JSON Spring unable to decode aggregated HTTP content messages to Flux Dec 4, 2023
@poutsma poutsma self-assigned this Dec 5, 2023
@poutsma poutsma added type: bug A general bug in: web Issues in web modules (web, webmvc, webflux, websocket) and removed status: waiting-for-triage An issue we've not yet triaged or decided on labels Dec 6, 2023
@poutsma poutsma changed the title Spring unable to decode aggregated HTTP content messages to Flux Spring unable to decode aggregated Jackson content Dec 6, 2023
@poutsma poutsma changed the title Spring unable to decode aggregated Jackson content Spring unable to decode aggregated JSON content Dec 6, 2023
@poutsma poutsma added this to the 6.1.2 milestone Dec 6, 2023
@poutsma poutsma added the for: backport-to-6.0.x Marks an issue as a candidate for backport to 6.0.x label Dec 6, 2023
@github-actions github-actions bot added status: backported An issue that has been backported to maintenance branches and removed for: backport-to-6.0.x Marks an issue as a candidate for backport to 6.0.x labels Dec 6, 2023
poutsma added a commit that referenced this issue Dec 6, 2023
This commit ensures that we process after each fed buffer in
Jackson2Tokenizer, instead of after all fed buffers.

See gh-31747
Closes gh-31772
@poutsma poutsma closed this as completed in 0e6c17f Dec 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
in: web Issues in web modules (web, webmvc, webflux, websocket) status: backported An issue that has been backported to maintenance branches type: bug A general bug
Projects
None yet
Development

No branches or pull requests

3 participants