Make default bootstrappers used by IPFS/libp2p be a more robust list.

[BigLep]

Problem

Right now, the default config in IPFS/libp2p nodes use bootsrappers hosted by PL. We should be looking into ways we can make the default bootstrapping more robust.

Ideas

(1) remember peers across restarts, decreasing impact of bootstrappers going dark (leveraging prior art from bittorrent and tor projects)
(2) include bootstrappers maintained by different organizations, decreasing risk of Facebook-like catastrophic bootstrapper misconfiguration

[BigLep]

Filecoin slack discussion (https://filecoinproject.slack.com/archives/C025ZN5LNV8/p1636550102021000 ):

Lidel
In short, you already run libp2p node, so it would be only a pledge to run specific PeersID under a specific multiaddr (ideally ipv4/ipv6, but dns is fine too) for long periods of time.

Thibault Meunier 2 months ago
I would be curious about what is the performance requirements of these nodes (at least at PL), and their number of peers and bitswap requests. At first thought, I think we could operate a bootstraper at Cloudflare (we are already using specific peerid under a specific multiaddr)

aschmahmann - Protocol Labs 2 months ago
It looks like it's about 2k connections per peer and the highest resource usage I've seen on any of our bootstrappers using raw machine metrics over the past year is: 28GB storage, 8GB RAM, system load of 3 on an 8 core machine. These are probably more resources than it should take to run a bootstrapper, but aren't too terrible. If they're problematic it's something we can look into.
Note that the machines don't need to support Bitswap since they just need to be good DHT servers. If the inbound requests become a problem we can just build a slimmer binary that only has the pieces we need. Also, given that the nodes shouldn't be storing much a bloom filter should largely prevent any Bitswap related syscalls.

boris 2 months ago
@lidel interested, but we also need DMCA filters before we can run this

aschmahmann - Protocol Labs 2 months ago
@boris what types of filters would you think are needed here? AFAIK we have received no take down requests for anything related to the DHT (bootstrappers, hydra nodes, etc.)

boris 2 months ago
oh, sorry, I was reading this as also needing to run an open http gateway

aschmahmann - Protocol Labs 2 months ago
nope, this is just about running DHT bootstrapper nodes. It'd be great to have https://github.com/ipfs/go-ipfs-config/blob/master/bootstrap_peers.go#L17-L24 look a little more like https://github.com/filecoin-project/lotus/blob/eede19fb0b2d3a7726f35392d0028b9e2ff56ce9/build/bootstrap/mainnet.pi#L1-L15 in terms of diversity of bootstrappers.

boris 2 months ago
Yes, definitely! Happy to run a bootstrap node in that case.
🙌

lidel 2 months ago
Yes, this is only about running bootstrap node, you can run it without gateway or with NoFetch: true to be sure you dont host third party data.
Ideally it would run a suite of transports: TCP+QUIC+Secure WebSockets (with TLS cert for use in browsers) to maximize utility, but TCP+QUIC alone are also good (edited)

boris 2 months ago
yeah, we run WebSockets already, and I’d love to have browser js-ipfs be able to connect to the main network

[BigLep]

One approach: Write up the policy – perhaps creating a ipfs/public-bootstrappers repo with readme that includes the policy and the list of operators who joined the program?

[hacdias]

Update on this: we have merged ipfs/kubo#8856, which added a backup list of bootstrap peers based on your currently connected peers.