Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Write Skip Count Not Updated in Chunk Processing #4514

Open
thilotanner opened this issue Dec 14, 2023 · 0 comments
Open

Write Skip Count Not Updated in Chunk Processing #4514

thilotanner opened this issue Dec 14, 2023 · 0 comments
Labels
status: waiting-for-triage Issues that we did not analyse yet type: bug

Comments

@thilotanner
Copy link

thilotanner commented Dec 14, 2023

Bug description
It seems, the write skip count is not incremented / updated if items are removed from chunks during write (in case an item cannot be successfully written)

Environment
Spring Batch 5.0.3 / Spring Boot 3.1.5 / Java 17

Steps to reproduce

public class MemberWriter implements ItemStreamWriter<String> {

    @Override
    public void write(Chunk<? extends String> memberChunk) {
        Chunk<? extends String>.ChunkIterator iterator = memberChunk.iterator();
        ...
        iterator.remove(new RuntimeException());
    }

Expected behavior
According to documentation of class Chunk, the writeSkipCount in the corresponding StepExecution object should be incremented for every item removed from ChunkIterator:
Encapsulation of a list of items to be processed and possibly a list of failed items to be skipped. To mark an item as skipped clients should iterate over the chunk using the iterator() method, and if there is a failure call Chunk.ChunkIterator.remove() on the iterator. The skipped items are then available through the chunk.

Workaround
It's possible to circumvent the issue by creating a combined SkipListener / StepExecutionListener:

public class SkipTestListener implements SkipListener<String, String>, StepExecutionListener {
    private StepExecution stepExecution;

    @Override
    public void beforeStep(StepExecution stepExecution) {
        this.stepExecution = stepExecution;
    }

    @Override
    public void onSkipInWrite(String item, Throwable t) {
        stepExecution.setWriteSkipCount(stepExecution.getWriteSkipCount() + 1);
        log.warn("Skipped item: {}", item);
    }
}

and register it accordingly in the step:

        SkipTestListener testListener = new SkipTestListener();

        new StepBuilder("process-step", jobRepository)
                .<String, String>chunk(chunkSize, transactionManager)
                .reader(reader)
                .processor(processor)
                .writer(writer)
                .faultTolerant()
                .listener((StepExecutionListener) testListener)
                .listener((SkipListener<? super String, ? super String>) testListener)
                .build();
@thilotanner thilotanner added status: waiting-for-triage Issues that we did not analyse yet type: bug labels Dec 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status: waiting-for-triage Issues that we did not analyse yet type: bug
Projects
None yet
Development

No branches or pull requests

1 participant