Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extremely slow serialization with bincode v2 #40

Closed
mhluska opened this issue Aug 23, 2023 · 1 comment
Closed

Extremely slow serialization with bincode v2 #40

mhluska opened this issue Aug 23, 2023 · 1 comment

Comments

@mhluska
Copy link
Contributor

mhluska commented Aug 23, 2023

I'm noticing throughput of several MB per second when saving to file with some solutions (others work fine, yet others are slow at the beginning and fast towards the end of serialization). I'm using high end EC2 machines so it's unlikely to be a hardware limitation. I also have compression disabled.

I'm pretty sure the issue is the bincode v2 dependency: bincode-org/bincode#618

When I add this to the Cargo.toml file, things serialize much faster:

[profile.release]
lto = true
@b-inary
Copy link
Owner

b-inary commented Sep 30, 2023

Thank you for the comment. In general, LTO sometimes makes things dramatically faster, but sometimes it makes things worse. To understand why, we need to dive deep into Rust's complex build system.

I don't think there is much I can do here. Maybe we can switch to a different serialization library, but I think bincode is a very popular and reasonable library for our use case.

@mhluska mhluska closed this as completed Oct 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants