Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: Make mask_invalid consistent with mask_where if copy is set to False #22046

Merged
merged 3 commits into from Oct 5, 2022

Conversation

cmarmo
Copy link
Contributor

@cmarmo cmarmo commented Jul 26, 2022

This pull requests makes mask_invalid consistent with mask_where when copy is set to False.
Fixes #19332.

I'd rather make the two functions consistent than change the documentation.
Also from the documentation

Only applies to arrays with a dtype where NaNs or infs make sense

I added a test checking that an error is thrown when isinfinite is not applicable.

Thanks for considering it.

@cmarmo cmarmo changed the title Make mask_invalid consistent with mask_where if copy is set to False BUG: Make mask_invalid consistent with mask_where if copy is set to False Aug 1, 2022
numpy/ma/core.py Outdated
Comment on lines 2360 to 2363
try:
return masked_where(~(np.isfinite(getdata(a))), a, copy=copy)
except TypeError:
raise
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was there a particular reason for this try/except, or is it a debug remnant?

Suggested change
try:
return masked_where(~(np.isfinite(getdata(a))), a, copy=copy)
except TypeError:
raise
return masked_where(~(np.isfinite(getdata(a))), a, copy=copy)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, I can't recall why I put the try/except. Suggestion applied. Thanks!

with pytest.raises(TypeError,
match="not supported for the input types"):
np.ma.masked_invalid(a)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We were looking at this in the triage meeting as well. I guess the test is great (I honestly still need to parse it fully).
But we are missing an additional test for the successful path that was fixed I think: Checking that the array was indeed modified in-place with copy=False?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added a new test in test_masked_array_no_copy() ... just to explain myself: I didn't add it in the first place because mask_invalid becomes a straight call of masked_where which is already tested. But I guess the more tests we have the better?
Thanks!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, thanks. It is true that masked_where is tested, but it is also just nice to see the fix in action in the PR and we mainly have integration tests anyway.

A test too much cannot hurt :).

I find it odd that we consider inf invalid by default, but that is not a change.

Thanks @cmarmo, lets get this in!

@seberg
Copy link
Member

seberg commented Sep 22, 2022

I suppose we should do this just to align things anyway...

Although, now I actually suspect that this may have been intentional (Chesterton's fence greets): The function always replaces the mask, but does not always copy the data, which is a pattern to the madness that does make sense, the docs are probably just fuzzy on that intention (if it was the intention).
Sorry, long shot, but @ahaldane do you happen to have a gut feeling on this?

@InessaPawson
Copy link
Member

@mhvk Do you have any thoughts on this?

@mhvk
Copy link
Contributor

mhvk commented Oct 5, 2022

Not really. Overall, to me this PR makes sense: do as the doc states and just call masked_where. I've never understood why if data is kept, masks are not, though clearly it was designed that way.

@seberg
Copy link
Member

seberg commented Oct 5, 2022

Thanks @cmarmo and Marten, lets give this a shot then!

@seberg seberg merged commit 02b68f1 into numpy:main Oct 5, 2022
@charris
Copy link
Member

charris commented Oct 5, 2022

Should probably add a release note for this.

@seberg
Copy link
Member

seberg commented Oct 5, 2022

Hmmm, maybe better. @cmarmo are you interested in having a look at that, the instructions are in numpy/doc/release/upcoming_changes/README.txt (you basically add a file in that folder with a 22046.change.rst as name).

Otherwise, hopefully I will remember to follow up and do it :).

@bsipocz bsipocz added the 56 - Needs Release Note. Needs an entry in doc/release/upcoming_changes label Oct 5, 2022
@bsipocz
Copy link
Member

bsipocz commented Oct 5, 2022

Hmm, if the label is used consistently, a script could help to look for missing release notes before release time?

@cmarmo
Copy link
Contributor Author

cmarmo commented Oct 5, 2022

@cmarmo are you interested in having a look at that, the instructions are in numpy/doc/release/upcoming_changes/README.txt (you basically add a file in that folder with a 22046.change.rst as name).

I can take care of that if it can wait until the week-end. :)

@cmarmo cmarmo deleted the masked-invalid branch October 8, 2022 07:01
seberg pushed a commit that referenced this pull request Oct 14, 2022
This pull request add the changelog for #22046.
seberg added a commit to seberg/numpy that referenced this pull request Dec 19, 2022
This is the minimal solution to fix numpygh-22826 with as little change
as possible.
We should fix `getdata()` but I don't want to do that in a bug-fix
release really.

IMO the alternative is to revert numpygh-22046 which would also revert
the behavior noticed in numpygh-22720  (which seems less harmful though).

Closes numpygh-22826
charris pushed a commit to charris/numpy that referenced this pull request Dec 19, 2022
This is the minimal solution to fix numpygh-22826 with as little change
as possible.
We should fix `getdata()` but I don't want to do that in a bug-fix
release really.

IMO the alternative is to revert numpygh-22046 which would also revert
the behavior noticed in numpygh-22720  (which seems less harmful though).

Closes numpygh-22826
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
00 - Bug 56 - Needs Release Note. Needs an entry in doc/release/upcoming_changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BUG: ma.masked_invalid(a, copy=False) does not modify a
6 participants