Skip to content
This repository has been archived by the owner on Sep 14, 2023. It is now read-only.

Speed up boolean encoding/decoding #68

Open
mre opened this issue Sep 21, 2019 · 1 comment
Open

Speed up boolean encoding/decoding #68

mre opened this issue Sep 21, 2019 · 1 comment
Labels
enhancement New feature or request good first issue Good for newcomers hacktoberfest help wanted Extra attention is needed

Comments

@mre
Copy link
Owner

mre commented Sep 21, 2019

From our benchmarks we can see that we are consistently slower than everyone else when serializing/deserializing boolean values. We should fix that.

orjson is using an unsafe block to create a reference to a boolean:
https://github.com/ijl/orjson/blob/03d55e99a953ce93cedc05f03e4b63b0bcbbcc7a/src/decode.rs#L81-L96

This avoids additional allocations.
For comparison, this is our code at the moment:

hyperjson/src/lib.rs

Lines 475 to 480 in ded13b4

fn visit_bool<E>(self, value: bool) -> Result<Self::Value, E>
where
E: de::Error,
{
Ok(value.to_object(self.py))
}

I wonder if we could achieve comparable performance without using unsafe.
@konstin, any idea? Maybe there was a recent development in pyo3 that we could leverage here?

@mre mre added enhancement New feature or request good first issue Good for newcomers hacktoberfest help wanted Extra attention is needed labels Sep 21, 2019
@konstin
Copy link
Collaborator

konstin commented Sep 22, 2019

It's weird that this slower than what orjson does. That's the implementation of ToObject:

unsafe {
    PyObject::from_borrowed_ptr(
        py,
        if *self {
            ffi::Py_True()
        } else {
            ffi::Py_False()
        },
    )
}

And that's the implementation of from_borrowed_ptr:

debug_assert!(
    !ptr.is_null() && ffi::Py_REFCNT(ptr) > 0,
    format!("REFCNT: {:?} - {:?}", ptr, ffi::Py_REFCNT(ptr))
);
ffi::Py_INCREF(ptr);
PyObject(NonNull::new_unchecked(ptr))

So in theory this should compile down to the same as orjson. You could try using PyBool::new and see if that makes a difference, but otherwise I don't know enough about inspecting assembly to debug that.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request good first issue Good for newcomers hacktoberfest help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

2 participants