BUG(?): Tests print 'uh-oh, unmatched shift_free(ptr, 8) but allocated 720'

[WarrenWeckesser]

Describe the issue:

When I run the full test suite (main branch, Python 3.9.1 on Linux), the last few lines of output are:

==== 18140 passed, 863 skipped, 19 xfailed, 3 xpassed in 346.86s (0:05:46) =====
uh-oh, unmatched shift_free(ptr, 8) but allocated 720
uh-oh, unmatched shift_free(ptr, 8) but allocated 720
uh-oh, unmatched shift_free(ptr, 8) but allocated 720
 A=123  
 B=123  
 C=123
 D=123
 CHANGE A,B,C,D
 A=A23  
 B=B23  
 C=C23
 D=D23

The lines that begin with uh-oh, unmatched shift_free are from code generated in test_mem_policy.py.

Is that output expected? It starts with "uh-oh", so it looks like it is indicating a bug.

Reproduce the code example:

n/a

Error message:

See above.

NumPy/Python version information:

1.22.0.dev0+1643.gbc087bb2b 3.9.1 (default, Dec 11 2020, 14:32:07)
[GCC 7.3.0]

[seberg]

It is not really a bug, the full test suite can't avoid running into these "warnings" for certain calls. Not sure if we could stop printing them somehow (by counting them instead?!). Maybe we should just start with changing the message to look less dangerous though.

[WarrenWeckesser]

Yeah, I just found this comment

The uh-oh, unmatched shift_free(ptr, 8) but allocated 720, as noted in this comment, comes from this pattern np.empty((2, 2, 0, 2, 2)). It calculates the bytes to allocate by multiplying the shape as-if the 0 is 1, but frees 8 bytes.

which links to a similar comment.

If it is not problem, can the printed output be removed or suppressed somehow? Spurious output in the tests is a nuisance--a minor nuisance, maybe, but still a nuisance.

[mattip]

I think we should change the calculation of the bytes-to-free to match the calculation of bytes-to-allocate.

[mattip]

Just to be clear: the problem in the bytes-to-allocate is that the calculation is done in a loop, and if fa->dimensions[i] == 0 it skips that value.

bytes-to-free uses PyArray_NBYTES, which does not skip 0.

[seberg]

Adding 1.22 milestone, but I don't think it is important enough to worry about with respect to an RC release.

[rgommers]

Adding 1.22 milestone, but I don't think it is important enough to worry about with respect to an RC release.

Agreed, not needed for RC1 - but the message should be removed for the final 1.22.0 release. It's just an fprintf statement in our own test suite, so there should be no problem in just deleting it if it cannot be fixed.

[mattip]

See #15788 which is a more complete solution but is still in draft form.

[charris]

Pushing this off to 1.22.1. It isn't a blocker, just a bit annoying. It arrives when it arrives.

[charris]

Pushing off to 1.21.2. It is annoying, but not a blocker.

[charris]

I cleared the milestone. This is still a problem, but not a blocker of anything. @mattip do you have the intention of doing something about this someday?

[seberg]

This should have been closed with the above linked PR.