Occasional broken instance when Python class inherits from .NET interface

[larkby]

Environment

• Pythonnet version: 3.0.2/3.0.1
• Python version: 3.11.4
• Operating System: Windows 11
• .NET Runtime: .NET Framework 4.8.1

Details

When a Python class inherits from a .NET interface, I'm very occasionally getting broken instances (once every few 10,000 to 100,000 instantiations):

import traceback
import clr
import System

class Foobar(System.IDisposable):
    __namespace__ = "System"

    def __init__(self):
        pass

    def Dispose(self):
        pass

first = True
i = 0

while True:
    instance = Foobar()
    #System.GC.Collect()
    try:
        str(instance)
    except Exception as exc:
        if first:
            traceback.print_exc()
            first = False
        print(i)
        i = 0
    i += 1

This will print the exception raised by str(instance) on one of the broken instances, and then print the number of good instances between each broken one. Output:

> python .\repro.py
Traceback (most recent call last):
  File "repro.py", line 21, in <module>
    str(instance)
TypeError: invalid object
38401
42235
122213
68406
244677
28940
5270
268394
259390
53728
248909
34220
...

In my production code, the issue also occurs if I pass an instance from Python to native C#/CLR code without doing anything to it in Python (i.e. not triggering a Python exception the way str(instance) does in the repro above). E.g:

instance = Foobar()
# public void SomeMethod(IDisposable arg_foo) { ... }
NativeCode.SomeMethod(instance)

In this case, the C# code raises an exception something like this, once in a blue moon:
System.InvalidCastException: 'Foobar' value cannot be converted to System.IDisposable

Notes:

• IDisposable was just something I grabbed at random for the repro; the issue seems to happen with any interface

• With the System.GC.Collect() line in the repro code uncommented, the issue doesn't seem to occur.

• In my production code, at least, adding assert System.GC.TryStartNoGCRegion(1) before the affected code seems to make the issue occur more frequently

• I found Python.exe crashes if __pycache__ directories exist (garbage collection trashed) #481 to be vaguely similar, at least to this outsider

[filmor]

The exact line that produces this error is

pythonnet/src/runtime/Types/ClassBase.cs

Line 281 in 22d07dd

return Exceptions.RaiseTypeError("invalid object");

This is in our tp_str implementation (as expected) and can only come from this function returning null:

pythonnet/src/runtime/Types/ManagedType.cs

Lines 17 to 33 in 22d07dd

	/// <summary>
	/// Given a Python object, return the associated managed object or null.
	/// </summary>
	internal static ManagedType? GetManagedObject(BorrowedReference ob)
	{
	if (ob != null)
	{
	BorrowedReference tp = Runtime.PyObject_TYPE(ob);
	var flags = PyType.GetFlags(tp);
	if ((flags & TypeFlags.HasClrInstance) != 0)
	{
	var gc = TryGetGCHandle(ob, tp);
	return (ManagedType?)gc?.Target;
	}
	}
	return null;
	}

I put a few debug prints in and what seems to happen here is that the GC handle's target is empty in this case, which is very odd. I'll see whether I can debug this further.