BUG: Initialize the full nditer buffer in case of error #18922
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
charris
added
00 - Bug
component: numpy._core
08 - Backport
This is necessary because in some rare cases (reductions), we may not actually use the full buffer. In that case, the cleanup-on-error code would have to grow smart enough to handle these cases. It seems much simpler to just always initialize the full buffers, even if we may not end up using them. Admittedly, the old logic might have skipped the buffer clearing (especially the full buffer) in a fair bit of cases, but since this is only relevant for `object` dtype, I assume this is fine.
Buffer must always either contain NULL or valid references (assuming the dtype supports references) in order to allow cleanup in case of errors. It is thus unnecessary to clear buffers before every copy, if they contain NULL, all is fine. If they contain non-NULL, we should also DECREF those references (so it would be incorrect as well). Buffers thus need the memset exactly once: directly upon allcoation (which we do now). After this any transfer from/to the buffer needs to ensure that the buffer is always in a good state.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport of #18813.
This is necessary because in some rare cases (reductions), we may
not actually use the full buffer. In that case, the cleanup-on-error
code would have to grow smart enough to handle these cases.
It seems much simpler to just always initialize the full buffers, even
if we may not end up using them.
Admittedly, the old logic might have skipped the buffer clearing
(especially the full buffer) in a fair bit of cases, but since
this is only relevant for object dtype, I assume this is fine.