Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

issue trying to get multiple nodejs peers to sync #1183

Open
endomorphosis opened this issue May 10, 2024 · 5 comments
Open

issue trying to get multiple nodejs peers to sync #1183

endomorphosis opened this issue May 10, 2024 · 5 comments

Comments

@endomorphosis
Copy link

I was working on trying to use orbitdb to shard and serve a K nearest neighbors search engine using orbitdb as the key:value store, with the math part being handled by FAISS, and try to benchmark whether or not it is feasible with 8 million records in chunks of 4096 key-value pairs.

I have two nodes working on the local area network, and I've tried to throw the kitchen sink of different networking and peer discovery methods, but I cant seem to get the two different nodes to talk to each other on different PCs.

code
https://github.com/endomorphosis/orbitdb-benchmark/blob/master/src/orbitv3-slave-swarm.js
https://github.com/endomorphosis/orbitdb-benchmark/blob/master/src/orbitv3-master-swarm.js

node /home/devel/orbitdb-benchmark/src/orbitv3-master-swarm.js --ipAddress=127.0.0.1 --port=60001 --swarmName=caselaw --chunkSize=8 --index=1
node /home/barberb/orbitdb-benchmark-1/src/orbitv3-slave-swarm.js --ipAddress=127.0.0.1 --port=50001 --swarmName=caselaw --chunkSize=8 --index=1

@haydenyoung
Copy link
Member

Are the 2 pcs on the same local network?

@endomorphosis
Copy link
Author

yes

@haydenyoung
Copy link
Member

Have you tried using mdns for peer discovery? If not, try to configure it in the you Libp2p settings for Helia, E.g.

import { mdns } from '@libp2p/mdns'

export default {
  peerDiscovery: [
    mdns()
  ],
  /* other settings */
}

This should allow peers to discover one another on the local network without explicitly dialling one another.

@endomorphosis
Copy link
Author

Have you tried using mdns for peer discovery? If not, try to configure it in the you Libp2p settings for Helia, E.g.

import { mdns } from '@libp2p/mdns'

export default {
  peerDiscovery: [
    mdns()
  ],
  /* other settings */
}

This should allow peers to discover one another on the local network without explicitly dialling one another.

https://github.com/endomorphosis/orbitdb-benchmark/blob/master/src/orbitv3-master-swarm.js#L63

@haydenyoung
Copy link
Member

I've tried launching the peers in the above example and they are able to find one another. Using a simple peer:connect confirms this:

ipfs.libp2p.addEventListener("peer:connect", event => {
    console.log('connected', event.detail)
})

Can you dial master from slave explicitly?

I.e.

await ipfs.libp2p.dial(multiaddr(masterAddress))

If the problem persists, Libp2p may be better able to guide you in getting the peers to communicate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants