Increase FID lock array to 256 elements

[loriopatrick]

When benchmarking with a WIP update, increasing the FID lock array size drastically improved performance and consistency in benchmarking runs. However this change does not make a meaningful impact on performance on the main branch. It does lower the number of lock contentions which could affect performance in some scenarios. Tested on a fast NVME media.

Through measurements this change does lower the number of contentions but doesn't affect performance in the main branch or the improved prune patch.

Old Message

Benchmarking with replayed production gossip messages this change significantly speeds up processing.

~30 seconds => ~10 seconds to replay ~3,000 gossip messages

When measured I got 2,605 FID locks before the change and 470 after. This would likely not be as big of an issue if the ThreadPool was larger. Having a massive fid_locks array shouldn't hurt anything or take up any measurable memory.

I suggest FID_LOCKS_COUNT gets increased when or if the threadpool size is increased to continue to keep these conflicts less likely.

Motivation

Describe why this issue should be fixed and link to any relevant design docs, issues or other relevant items.

Change Summary

Describe the changes being made in 1-2 concise sentences.

Merge Checklist

Choose all relevant options below by adding an x now or at any time before submitting for review

• PR title adheres to the conventional commits standard
• PR has a changeset
• PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore)
• PR includes documentation if necessary.
• All commits have been signed

Additional Context

If this is a relatively large or complex change, provide more details here that will help reviewers

---

PR-Codex overview

This PR increases the number of file locks and updates the lock initialization method in Store.

Detailed summary

• Increased FID_LOCKS_COUNT from 4 to 256 in store.rs
• Updated the initialization of fid_locks to dynamically create locks based on FID_LOCKS_COUNT

✨ Ask PR-Codex anything about this PR by commenting with /codex {your question}

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
hub-monorepo	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	May 13, 2024 7:52pm

[changeset-bot]

⚠️ No Changeset found

Latest commit: 1b7932c

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[sanjayprabhu]

I believe we later concluded this was not actually providing a significant speed up? Let me know if I'm misremembering.

[loriopatrick]

That is correct. It didn't provide a noticeable improvement. If in the future you decide to increase the number of workers in the threadpool a change like this should be looked into.