Fix memory leak in ESPHome disconnect callbacks

[bdraco]

needs #104146

Proposed change

There were two problems here:

• The disconnect callbacks were not cleared (A new list was created instead) after they were called so the references were still held which means anything holding a reference to them could end up holding it forever.

• In the BLE client we did not remove the callback when the ble device disconnected, only when the ESP itself disconnected. This prevented used BLE clients from being garbage collected until the ESP disconnected.

Type of change

• Dependency upgrade
• Bugfix (non-breaking change which fixes an issue)
• New integration (thank you!)
• New feature (which adds functionality to an existing integration)
• Deprecation (breaking change to happen in the future)
• Breaking change (fix/feature causing existing functionality to break)
• Code quality improvements to existing code or addition of tests

Additional information

• This PR fixes or closes issue: fixes #
• This PR is related to issue:
• Link to documentation pull request:

Checklist

• The code change is tested and works locally.
• Local tests pass. Your PR cannot be merged unless tests pass
• There is no commented out code in this PR.
• I have followed the development checklist
• I have followed the perfect PR recommendations
• The code has been formatted using Black (black --fast homeassistant tests)
• Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

• Documentation added/updated for www.home-assistant.io

If the code communicates with devices, web services, or third-party tools:

• The manifest file has all fields filled out correctly.
  Updated and included derived files by running: python3 -m script.hassfest.
• New or updated dependencies have been added to requirements_all.txt.
  Updated by running python3 -m script.gen_requirements_all.
• For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.
• Untested files have been added to .coveragerc.

To help with the load of incoming pull requests:

• I have reviewed two other open pull requests in this repository.

[home-assistant]

Hey there @OttoWinter, @jesserockz, @kbx81, mind taking a look at this pull request as it has been labeled with an integration (esphome) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of esphome can trigger bot actions by commenting:

• @home-assistant close Closes the pull request.
• @home-assistant rename Awesome new title Renames the pull request.
• @home-assistant reopen Reopen the pull request.
• @home-assistant unassign esphome Removes the current integration label and assignees on the pull request, add the integration domain after the command.

[bdraco]

 17 22:50:37 homeassistant homeassistant[548]: 2023-11-17 12:50:37.457 DEBUG (MainThread) [homeassistant.components.esphome.bluetooth.client] upstairsdeskgli89proxy [c8:f0:9e:d1:32:68]: LI0 - 98:1B:B5:71:40:6D: ESP device disconnected
Nov 17 22:50:37 homeassistant homeassistant[548]: 2023-11-17 12:50:37.457 DEBUG (MainThread) [homeassistant.components.esphome.bluetooth.client] upstairsdeskgli89proxy [c8:f0:9e:d1:32:68]: LI0 - 98:1B:B5:71:40:6D: BLE device disconnected
Nov 17 22:50:39 homeassistant homeassistant[548]: 2023-11-17 12:50:39.576 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved
Nov 17 22:50:39 homeassistant homeassistant[548]: Traceback (most recent call last):
Nov 17 22:50:39 homeassistant homeassistant[548]:   File "/usr/src/homeassistant/homeassistant/components/esphome/manager.py", line 167, in on_stop
Nov 17 22:50:39 homeassistant homeassistant[548]:     await cleanup_instance(self.hass, self.entry)
Nov 17 22:50:39 homeassistant homeassistant[548]:   File "/usr/src/homeassistant/homeassistant/components/esphome/manager.py", line 762, in cleanup_instance
Nov 17 22:50:39 homeassistant homeassistant[548]:     for disconnect_cb in data.disconnect_callbacks:
Nov 17 22:50:39 homeassistant homeassistant[548]: RuntimeError: Set changed size during iteration

need to make a copy

[bdraco]

objgraph looks good.

leaks are fixed

[frenck]

Thanks, @bdraco 👍

../Frenck

[bdraco]

Thanks