Changing zoom mode (drag) resets the zoom level

[francescopenna]

I am using a chart with this zoom configuration (pan is disabled):

zoom: {
  zoom: {
    mode: "xy",
    drag: {
      enabled: true,
      treshold: 100,
      drawTime: "afterDatasetsDraw",
    },
  }
}

I noticed that after a first zoom if I try to change the mode, for example with chart.config.options.plugins.zoom.zoom.mode = "y", the mode is succesfully changed, but the zoom level gets resetted.
I made a sandbox showcasing this issue. In the example, try to change the zoom mode, zoom out, then change the zoom mode again to see the issue.
Is this intended? And if so, is there a workaround for this?
Thanks in advance