feat: bootstrapper ready alpha use/testing in deployed env #135

SgtPooki · 2024-05-07T21:27:02Z

Title

Description

Related #18

Various fixes while debugging on a host I'm planning to deploy this to.

Notes & open questions

I debugged this by running directly with node dist/src/index.js --config /root/bootstrapper-config.json --enable-kademlia --enable-autonat and will start debugging with docker.

When I get this working with docker I'm switch from Draft mode.

Open questions

Why is webrtc not showing as a listening address?
Is there a better way to show what addresses the node is listening on instead of using setTimeout/setInterval? Is there an event I can use?
Should we enable "ping" service? other bootstrappers have it enabled.
are there other optimizations we can make?

Change checklist

I have performed a self-review of my own code
I have made corresponding changes to the documentation if necessary (this includes comments as well)
I have added tests that prove my fix is effective or that my feature works

…nect

SgtPooki · 2024-05-07T21:36:38Z

regarding bootstrappers configs..

This bootstrapper

╰─ ✔ ❯ ./vole libp2p identify /ip4/147.75.198.229/tcp/4003/ws/p2p/12D3KooWQgrExcg6dkCdiTER3G3ARe14PZ4cLhinKtRcLHsvnk1Q
PeerID: "12D3KooWQgrExcg6dkCdiTER3G3ARe14PZ4cLhinKtRcLHsvnk1Q"
Protocol version: "ipfs/0.1.0"
Agent version: "libp2p/1.5.1 UserAgent=v20.13.0"
Listen addresses:
	- "/ip4/147.75.198.229/tcp/4001/p2p/12D3KooWQgrExcg6dkCdiTER3G3ARe14PZ4cLhinKtRcLHsvnk1Q"
	- "/ip4/147.75.198.229/tcp/4003/ws/p2p/12D3KooWQgrExcg6dkCdiTER3G3ARe14PZ4cLhinKtRcLHsvnk1Q"
Protocols:
	- "/ipfs/id/1.0.0"
	- "/ipfs/id/push/1.0.0"
	- "/ipfs/kad/1.0.0"
	- "/libp2p/autonat/1.0.0"
	- "/libp2p/circuit/relay/0.2.0/hop"
	- "/webrtc-signaling/0.0.1"

Other bootstrappers

╰─ ✔ ❯ JSON_FILE=/Users/sgtpooki/code/work/protocol.ai/libp2p/js-libp2p-amino-dht-bootstrapper/prod-bootstrapper-config.json


╰─ ✔ ❯ for node in $(jq -r '.Bootstrap[]' "$JSON_FILE"); do
./vole libp2p identify "$node"
done
PeerID: "QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa"
Protocol version: "ipfs/0.1.0"
Agent version: "rust-libp2p-server/0.12.3"
Listen addresses:
	- "/dns4/ny5.bootstrap.libp2p.io/tcp/443/wss/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa"
	- "/dns6/ny5.bootstrap.libp2p.io/tcp/443/wss/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa"
	- "/ip4/139.178.65.157/tcp/4001/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa"
	- "/ip4/139.178.65.157/udp/4001/quic-v1/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa"
	- "/ip6/2604:1380:45d2:8100::1/tcp/4001/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa"
	- "/ip6/2604:1380:45d2:8100::1/udp/4001/quic-v1/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa"
Protocols:
	- "/ipfs/id/1.0.0"
	- "/ipfs/id/push/1.0.0"
	- "/ipfs/kad/1.0.0"
	- "/ipfs/ping/1.0.0"
	- "/libp2p/circuit/relay/0.2.0/hop"
PeerID: "QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb"
Protocol version: ""
Agent version: "kubo/0.28.0/e7f0f34"
Listen addresses:
	- "/dns4/am6.bootstrap.libp2p.io/tcp/443/wss/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb"
	- "/dns6/am6.bootstrap.libp2p.io/tcp/443/wss/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb"
	- "/ip4/147.75.87.27/tcp/4001/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb"
	- "/ip4/147.75.87.27/udp/4001/quic-v1/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb"
	- "/ip4/147.75.87.27/udp/4001/quic-v1/webtransport/certhash/uEiAQI9fwsOm5CuYA5iyGRiNxA8_J_IXgRK-c1vuMjzmv8g/certhash/uEiD3Q2mfd5EYt6Y2M9rsge_nna4hVyCgUVlfSz3IjAK8ew/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb"
	- "/ip6/2604:1380:4602:5c00::3/tcp/4001/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb"
	- "/ip6/2604:1380:4602:5c00::3/udp/4001/quic-v1/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb"
	- "/ip6/2604:1380:4602:5c00::3/udp/4001/quic-v1/webtransport/certhash/uEiAQI9fwsOm5CuYA5iyGRiNxA8_J_IXgRK-c1vuMjzmv8g/certhash/uEiD3Q2mfd5EYt6Y2M9rsge_nna4hVyCgUVlfSz3IjAK8ew/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb"
Protocols:
	- "/ipfs/bitswap"
	- "/ipfs/bitswap/1.0.0"
	- "/ipfs/bitswap/1.1.0"
	- "/ipfs/bitswap/1.2.0"
	- "/ipfs/id/1.0.0"
	- "/ipfs/id/push/1.0.0"
	- "/ipfs/kad/1.0.0"
	- "/ipfs/lan/kad/1.0.0"
	- "/ipfs/ping/1.0.0"
	- "/libp2p/circuit/relay/0.2.0/hop"
	- "/libp2p/circuit/relay/0.2.0/stop"
	- "/libp2p/dcutr"
	- "/x/"
PeerID: "QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt"
Protocol version: ""
Agent version: "kubo/0.28.0/e7f0f34"
Listen addresses:
	- "/dns4/sg1.bootstrap.libp2p.io/tcp/443/wss/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt"
	- "/dns6/sg1.bootstrap.libp2p.io/tcp/443/wss/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt"
	- "/ip4/145.40.118.135/tcp/4001/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt"
	- "/ip4/145.40.118.135/udp/4001/quic-v1/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt"
	- "/ip4/145.40.118.135/udp/4001/quic-v1/webtransport/certhash/uEiChu3_xNjKCQ8u59wYjGc4H1e6irJTGzAI2OYtDs-OQOw/certhash/uEiBENNd9IIPnU1cTGoVATo6cly-O2fQAjKpIyFi0msoJew/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt"
	- "/ip6/2604:1380:40e1:9c00::1/tcp/4001/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt"
	- "/ip6/2604:1380:40e1:9c00::1/udp/4001/quic-v1/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt"
	- "/ip6/2604:1380:40e1:9c00::1/udp/4001/quic-v1/webtransport/certhash/uEiChu3_xNjKCQ8u59wYjGc4H1e6irJTGzAI2OYtDs-OQOw/certhash/uEiBENNd9IIPnU1cTGoVATo6cly-O2fQAjKpIyFi0msoJew/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt"
Protocols:
	- "/ipfs/bitswap"
	- "/ipfs/bitswap/1.0.0"
	- "/ipfs/bitswap/1.1.0"
	- "/ipfs/bitswap/1.2.0"
	- "/ipfs/id/1.0.0"
	- "/ipfs/id/push/1.0.0"
	- "/ipfs/kad/1.0.0"
	- "/ipfs/lan/kad/1.0.0"
	- "/ipfs/ping/1.0.0"
	- "/libp2p/circuit/relay/0.2.0/hop"
	- "/libp2p/circuit/relay/0.2.0/stop"
	- "/libp2p/dcutr"
	- "/x/"
PeerID: "QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ"
Protocol version: ""
Agent version: "kubo/0.28.0/e7f0f340c"
Listen addresses:
	- "/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ"
	- "/ip4/104.131.131.82/udp/4001/quic-v1/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ"
	- "/ip4/104.131.131.82/udp/4001/quic-v1/webtransport/certhash/uEiDkpp8bHz_biKhgy6gWAqnRjyXAF3hJ_cPoZZXQ4Jq95Q/certhash/uEiCIMEw_vvBwFLEqWbOj_wx7I90HmfMabSyVZ9Vn5srjPA/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ"
Protocols:
	- "/ipfs/bitswap"
	- "/ipfs/bitswap/1.0.0"
	- "/ipfs/bitswap/1.1.0"
	- "/ipfs/bitswap/1.2.0"
	- "/ipfs/id/1.0.0"
	- "/ipfs/id/push/1.0.0"
	- "/ipfs/kad/1.0.0"
	- "/ipfs/lan/kad/1.0.0"
	- "/ipfs/ping/1.0.0"
	- "/libp2p/autonat/1.0.0"
	- "/libp2p/circuit/relay/0.2.0/hop"
	- "/libp2p/circuit/relay/0.2.0/stop"
	- "/libp2p/dcutr"
	- "/x/"
PeerID: "QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ"
Protocol version: ""
Agent version: "kubo/0.28.0/e7f0f340c"
Listen addresses:
	- "/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ"
	- "/ip4/104.131.131.82/udp/4001/quic-v1/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ"
	- "/ip4/104.131.131.82/udp/4001/quic-v1/webtransport/certhash/uEiDkpp8bHz_biKhgy6gWAqnRjyXAF3hJ_cPoZZXQ4Jq95Q/certhash/uEiCIMEw_vvBwFLEqWbOj_wx7I90HmfMabSyVZ9Vn5srjPA/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ"
Protocols:
	- "/ipfs/bitswap"
	- "/ipfs/bitswap/1.0.0"
	- "/ipfs/bitswap/1.1.0"
	- "/ipfs/bitswap/1.2.0"
	- "/ipfs/id/1.0.0"
	- "/ipfs/id/push/1.0.0"
	- "/ipfs/kad/1.0.0"
	- "/ipfs/lan/kad/1.0.0"
	- "/ipfs/ping/1.0.0"
	- "/libp2p/autonat/1.0.0"
	- "/libp2p/circuit/relay/0.2.0/hop"
	- "/libp2p/circuit/relay/0.2.0/stop"
	- "/libp2p/dcutr"
	- "/x/"
PeerID: "QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN"
Protocol version: ""
Agent version: "kubo/0.28.0/e7f0f34"
Listen addresses:
	- "/dns4/sv15.bootstrap.libp2p.io/tcp/443/wss/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN"
	- "/dns6/sv15.bootstrap.libp2p.io/tcp/443/wss/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN"
	- "/ip4/139.178.91.71/tcp/4001/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN"
	- "/ip4/139.178.91.71/udp/4001/quic-v1/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN"
	- "/ip4/139.178.91.71/udp/4001/quic-v1/webtransport/certhash/uEiBjfv8QG07rVwvXaeuxN2Aoi0zqnW5B55B0FW3bZ3A9pQ/certhash/uEiDYGZMqjz8wsz59DHA4iJin4nqTUfuJhq9AeAZlHBrmvg/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN"
	- "/ip6/2604:1380:45e3:6e00::1/tcp/4001/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN"
	- "/ip6/2604:1380:45e3:6e00::1/udp/4001/quic-v1/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN"
	- "/ip6/2604:1380:45e3:6e00::1/udp/4001/quic-v1/webtransport/certhash/uEiBjfv8QG07rVwvXaeuxN2Aoi0zqnW5B55B0FW3bZ3A9pQ/certhash/uEiDYGZMqjz8wsz59DHA4iJin4nqTUfuJhq9AeAZlHBrmvg/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN"
Protocols:
	- "/ipfs/bitswap"
	- "/ipfs/bitswap/1.0.0"
	- "/ipfs/bitswap/1.1.0"
	- "/ipfs/bitswap/1.2.0"
	- "/ipfs/id/1.0.0"
	- "/ipfs/id/push/1.0.0"
	- "/ipfs/kad/1.0.0"
	- "/ipfs/lan/kad/1.0.0"
	- "/ipfs/ping/1.0.0"
	- "/libp2p/circuit/relay/0.2.0/hop"
	- "/libp2p/circuit/relay/0.2.0/stop"
	- "/libp2p/dcutr"
	- "/x/"

some note-able services we may want to enable

dcutr - allow dialers to upgrade?

things we don't want to enable?

lan - no other nodes we want to peer with locally (*yet. that I know of)
ping - i dont think we need/want liveness checks, correct me if i'm wrong
bitswap - we don't want to do any data transferring as this is only a libp2p bootstrapper to help with connecting peers (peer discovery)
circuit-relay-stop - not sure what this is
/x/ - not sure what this is.

SgtPooki · 2024-05-07T22:16:55Z

without --network host I cannot get the libp2p node inside the docker container to provide any listening multiaddrs.

I tried with docker run -p 8888:8888/tcp -p 4001:4001/tcp -p 4003:4003/tcp -p 4002:4002/udp --publish-all -v /root/bootstrapper-config.json:/config.json -it amino --config /config.json --enable-kademlia --enable-autonat and it just doesn't work.

It would be nice to get this working without needing --network host but that's what i'm going to go with for now..

SgtPooki · 2024-05-08T16:46:24Z

Regarding protocol support, here is a command that gives all the protocols & count of bootstrappers supporting them:

$ JSON_FILE="/Users/sgtpooki/code/work/protocol.ai/libp2p/js-libp2p-amino-dht-bootstrapper/prod-bootstrapper-config.json"

jq -r '.Bootstrap[]' "$JSON_FILE" | \
xargs -I {} ./vole libp2p identify "{}" | \
ggrep -oP '^\s+-\s+"/.+"$' | \
sed -E 's/^\s+-\s+//' | \
sort | uniq -c

# ... some cruft of multiaddrs being matched.

   5 	- "/ipfs/bitswap"
   5 	- "/ipfs/bitswap/1.0.0"
   5 	- "/ipfs/bitswap/1.1.0"
   5 	- "/ipfs/bitswap/1.2.0"
   6 	- "/ipfs/id/1.0.0"
   6 	- "/ipfs/id/push/1.0.0"
   6 	- "/ipfs/kad/1.0.0"
   5 	- "/ipfs/lan/kad/1.0.0"
   6 	- "/ipfs/ping/1.0.0"
   2 	- "/libp2p/autonat/1.0.0"
   6 	- "/libp2p/circuit/relay/0.2.0/hop"
   5 	- "/libp2p/circuit/relay/0.2.0/stop"
   5 	- "/libp2p/dcutr"
   5 	- "/x/"

note: I use ggrep for GnuGrep because mac comes with grep not supporting standard options

…om host

SgtPooki added 10 commits May 7, 2024 12:16

tmp: trying to see what addresses we're listening on and if I can con…

66ae26f

…nect

fix: cli arguments match readme

3463869

fix: filter out private ips when announcing

c451af9

fix: allow dialing insecure websockets

70d6a38

chore: debugging logs

97e1f7c

feat: use identifiy and identifyPush

37bba4b

feat: add webrtc

2bd0569

chore: log multiaddr once

14e75a8

chore: create example config

d0af8e2

fix: ensure webrtc is listening

41ccd07

SgtPooki marked this pull request as draft May 7, 2024 21:27

SgtPooki mentioned this pull request May 7, 2024

feat: deploy js-libp2p bootstrapper #18

Open

SgtPooki added 5 commits May 10, 2024 17:35

chore: minor changes and updates when running locally

7d3e761

deps: use @next libp2p deps

a4b0a35

feat: get docker compose working locally

19a1c2d

feat: api endpoint for nodejs gc and heapdump

7e86e0a

feat: ensure snapshots from docker container are easily accessible fr…

2d38cec

…om host

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: bootstrapper ready alpha use/testing in deployed env #135

feat: bootstrapper ready alpha use/testing in deployed env #135

SgtPooki commented May 7, 2024 •

edited

SgtPooki commented May 7, 2024 •

edited

SgtPooki commented May 7, 2024

SgtPooki commented May 8, 2024

feat: bootstrapper ready alpha use/testing in deployed env #135

Are you sure you want to change the base?

feat: bootstrapper ready alpha use/testing in deployed env #135

Conversation

SgtPooki commented May 7, 2024 • edited

Title

Description

Notes & open questions

Open questions

Change checklist

SgtPooki commented May 7, 2024 • edited

This bootstrapper

Other bootstrappers

some note-able services we may want to enable

things we don't want to enable?

SgtPooki commented May 7, 2024

SgtPooki commented May 8, 2024

SgtPooki commented May 7, 2024 •

edited

SgtPooki commented May 7, 2024 •

edited