BinaryFormatter serialization and deserialization warning which throws an error in .net8

[RobertClapham]

Environment

• Pythonnet version: 3.0.3
• Python version: 3.9
• Operating System: Windows 11
• .NET Runtime: .net8 (release version 16/11/23)

Details

• When shutting down the the PythonEngine via a Dispose method on the impermentation class, the error occurs at the PythonEngine.Shutdown(); method and throws the error System.NotSupportedException: BinaryFormatter serialization and deserialization are disabled within this application (see trace below). The same code worked in .net7 without any issues, the error message leads to this description

• https://learn.microsoft.com/en-us/dotnet/standard/serialization/binaryformatter-security-guide

• Proir to the Shutdown error the pythonnet works correctly and calls python functions and returns the expected results

• The resolution is to add true to the .csproj file

• <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
    <EnableUnsafeBinaryFormatterSerialization>true</EnableUnsafeBinaryFormatterSerialization>
   </PropertyGroup>
  

  TODO

• _logger.Log is to to trace the point of failure

• _logger.Log(LogLevel.Error, "Shutting down Python engine"); runs

• it crashes at the PythonEngine.Shutdown(); method

• _logger.Log(LogLevel.Error, "Python engine shut down"); does not run

  private void Dispose(bool disposing)
  {
  _logger.Log(LogLevel.Error, "Dispose");
  if (_disposed) return;
  if (disposing)
  {
  _logger.Log(LogLevel.Error, "Shutting down Python engine");
  PythonEngine.Shutdown();
  _logger.Log(LogLevel.Error, "Python engine shut down");
  }
  _disposed = true;
  }

    print('TODO')

• System.NotSupportedException: BinaryFormatter serialization and deserialization are disabled within this application. See https://aka.ms/binaryformatt...

System.NotSupportedException
BinaryFormatter serialization and deserialization are disabled within this application. See https://aka.ms/binaryformatter for more information.
at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Serialize(Stream serializationStream, Object graph)
at Python.Runtime.RuntimeData.Stash()
at Python.Runtime.Runtime.Shutdown()
at Python.Runtime.PythonEngine.Shutdown()
at EcgAi.Training.Infrastructure.Python.PythonEnvironment.Dispose(Boolean disposing) in C:\Data\Rider\EcgAi.Training\Src\EcgAi.Training.Infrastructure.Python\PythonEnvironment.cs:line 113
at EcgAi.Training.Infrastructure.Python.PythonEnvironment.Dispose() in xxxxxxxxxxxxxxxxxxxxxxxxx

    print('TODO')

[TimIrwin26]

The issue happens in RuntimeData.Stash --

    var runtimeStorage = new PythonNetState
    {
        Metatype = MetaType.SaveRuntimeData(),
        ImportHookState = ImportHook.SaveRuntimeData(),
        Types = TypeManager.SaveRuntimeData(),
        Classes = ClassManager.SaveRuntimeData(),
        SharedObjects = SaveRuntimeDataObjects(),
    };

    IFormatter formatter = CreateFormatter();
    var ms = new MemoryStream();
    formatter.Serialize(ms, runtimeStorage);

   internal static IFormatter CreateFormatter()
   {
       return FormatterType != null ?
           (IFormatter)Activator.CreateInstance(FormatterType)
           : new BinaryFormatter();
   }

While there is an option to set another formatter, it's not used and expected to be of type IFormatter which is problematic.

[fsz1987]

Pythonnet use BinaryFormatter. and it is deprecated in .net 8.
create an instance of BinaryFormatter will cause an error.

you can set RuntimeData.FormatterType to your custom formatter to get it work.

if you dont want write a custom formatter, then add this line

AppContext.SetSwitch("System.Runtime.Serialization.EnableUnsafeBinaryFormatterSerialization", true);

before you call PythonEngine.Shutdown to temporary enable BinaryFormatter.

after PythonEngine.Shutdown. call

AppContext.SetSwitch("System.Runtime.Serialization.EnableUnsafeBinaryFormatterSerialization", false);

to disable BinaryFormatter

[amishratnasthapit]

Facing same issue and using work around currently. But .NET 9 is getting rid of Binary Formatter altogether. So this needs to be fixed.