Resource Manager: UX and improvements.

[ajnavarro]

Description

I identified two main problems that we have to fix on Kubo regarding Resource Manager integration:

Resource Manager is not backward-compatible

We can say that using Resource Manager is not backward-compatible with protocols that are using libp2p, because they must handle these resource manager errors. We have to fix that on our two main protocols: Bitswap and the DHT:

• Make DHT protocol Resource Manager aware. libp2p/go-libp2p-kad-dht#806 :
  • Avoid fire and forget actions. Handle all errors and not just log them.
  • Avoid spamming unlimited goroutines on processes.
• https://github.com/ipfs/go-bitswap/issues/599 :
  • Put limits on requests and responses. Avoid requests of hundred of thousand of CIDs
  • Avoid fire and forget actions.
• Make identify protocol Resource Manager aware. libp2p/go-libp2p#2013

No way from the user side to specify to use all the available resources

We are setting Connections and Streams defaults based on available memory. The number of connections allocated is based on magic numbers and not on real available resources.

Limiting memory and file descriptors will be easier to understand for the user. Defaults will be always ok, and experienced users can still tweak the Resource Manager configuration to limit by Streams or Connections if they need it. The work needed to make this possible is defined below:

• rcmgr: Connections and Streams should also reserve memory libp2p/go-libp2p#2010
• Resource Manager: Default limits only limit the max memory and FDs and set everything else to unlimited. #9602

Other improvements

One of the things that are also confusing people is the Resource Manager's name. It is not managing resources, but only counts resources in use by libp2p and other processes that want to use it:

• rcmgr: Rename resource manager to resource accountant libp2p/go-libp2p#2011
• feat: rcmgr: Export resource manager errors libp2p/go-libp2p#2008
• ipfs swarm limit all: returns empty "" Service and Protocol name #9577
• Resource Manager: Remove ResourceMgr.Limits configuration from Kubo and use the default libp2p JSON file for it #9603
  • or at least ipfs swarm limit all --reset does not remove user overrides from Swarm.ResourceMgr.Limits #9576
• Add examples for a custom go-libp2p-resource-manager with go-libp2p defaults applied. libp2p/go-libp2p#1628

cc @MarcoPolo

[BigLep]

In the latest round of consolidating/organizing the resource manager/accountant work, I'm closing this in favor of #9650