Nesting types implementing De/SerializeAs with serde_as

[spikespaz]

I am attempting to use serde_as to chain serialization and deserialization in a strange way. I have a hard time explaining it so I think code may make more sense than words here.

#[serde_as]
#[derive(Clone, Debug, Serialize, Deserialize)]
struct TestType {
    #[serde_as(as = "JsonString<Vec<Base62Encoded<u64>>>")]
    field: Vec<u64>,
}

The type Base62Encoded is defined as follows:

use std::str::FromStr;

use serde::{Deserialize, Deserializer, Serialize, Serializer};
use serde_with::{DeserializeAs, DeserializeFromStr, SerializeAs, SerializeDisplay};

#[derive(Clone, Debug, PartialEq, SerializeDisplay, DeserializeFromStr)]
pub struct Base62Encoded<T>(T);

impl<T> std::fmt::Display for Base62Encoded<T>
where
    T: Clone + Into<u128>,
{
    fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        formatter.write_str(&base62::encode(self.0.clone()))
    }
}

impl<T> FromStr for Base62Encoded<T>
where
    u128: TryInto<T>,
{
    type Err = base62::DecodeError;

    fn from_str(other: &str) -> Result<Self, Self::Err> {
        match base62::decode(other)?.try_into() {
            Ok(n) => Ok(Base62Encoded(n)),
            Err(_) => Err(Self::Err::ArithmeticOverflow),
        }
    }
}

impl<T> SerializeAs<T> for Base62Encoded<T>
where
    T: Clone + Into<u128>,
{
    fn serialize_as<S>(source: &T, serializer: S) -> Result<S::Ok, S::Error>
    where
        S: Serializer,
    {
        Base62Encoded(source.clone()).serialize(serializer)
    }
}

impl<'de, T> DeserializeAs<'de, T> for Base62Encoded<T>
where
    u128: TryInto<T>,
{
    fn deserialize_as<D>(deserializer: D) -> Result<T, D::Error>
    where
        D: Deserializer<'de>,
    {
        Base62Encoded::deserialize(deserializer).map(|this| this.0)
    }
}

This does appear to do the job as expected. Now, this works fine if I use #[serde_as(as = "Vec<Base62Encoded<u64>>")] on a field that is of the type Vec<u64>. I then re-implemented JsonString intending to wrap the type to get a JSON-encoded array of base-62-encoded integers:

use std::marker::PhantomData;

use serde::de::{DeserializeOwned, Error as DeserializeError, Visitor};
use serde::ser::Error as SerializeError;
use serde::{Deserialize, Deserializer, Serialize, Serializer};
use serde_with::{DeserializeAs, SerializeAs};

#[derive(Clone, Debug, PartialEq)]
pub struct JsonString<T>(T);

impl<T> Serialize for JsonString<T> {
    fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
    where
        S: Serializer,
    {
        serializer.collect_str(&serde_json::to_string(self).map_err(SerializeError::custom)?)
    }
}

impl<'de, T> Deserialize<'de> for JsonString<T>
where
    T: DeserializeOwned,
{
    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
    where
        D: Deserializer<'de>,
    {
        struct JsonStringVisitor<T>(PhantomData<T>);

        impl<'de, T> Visitor<'de> for JsonStringVisitor<T>
        where
            T: DeserializeOwned,
        {
            type Value = JsonString<T>;

            fn expecting(&self, formatter: &mut std::fmt::Formatter) -> std::fmt::Result {
                formatter.write_str("a value that can be serialized as a JSON string")
            }

            fn visit_str<E>(self, value: &str) -> Result<Self::Value, E>
            where
                E: DeserializeError,
            {
                Ok(JsonString(
                    serde_json::from_str(value).map_err(DeserializeError::custom)?,
                ))
            }
        }

        deserializer.deserialize_str(JsonStringVisitor(PhantomData))
    }
}

impl<T> SerializeAs<T> for JsonString<T>
where
    T: Clone,
{
    fn serialize_as<S>(source: &T, serializer: S) -> Result<S::Ok, S::Error>
    where
        S: Serializer,
    {
        JsonString(source.clone()).serialize(serializer)
    }
}

impl<'de, T> DeserializeAs<'de, T> for JsonString<T>
where
    T: DeserializeOwned,
{
    fn deserialize_as<D>(deserializer: D) -> Result<T, D::Error>
    where
        D: Deserializer<'de>,
    {
        JsonString::deserialize(deserializer).map(|this| this.0)
    }
}

Using all of these in combination tells me that the compiler is expecting a totally different type:

error[E0308]: mismatched types
  --> src/endpoints.rs:16:24
   |
16 | #[derive(Clone, Debug, Serialize, Deserialize)]
   |                        ^^^^^^^^^ expected struct `base62::Base62Encoded`, found `u64`
   |
   = note: expected reference `&std::vec::Vec<base62::Base62Encoded<u64>>`
              found reference `&'__a std::vec::Vec<u64>`
   = note: this error originates in the derive macro `Serialize` (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0308]: mismatched types
  --> src/endpoints.rs:16:35
   |
16 | #[derive(Clone, Debug, Serialize, Deserialize)]
   |                                   ^^^^^^^^^^^ expected `u64`, found struct `base62::Base62Encoded`
   |
   = note: expected struct `std::vec::Vec<u64>`
              found struct `std::vec::Vec<base62::Base62Encoded<u64>>`
   = note: this error originates in the macro `try` (in Nightly builds, run with -Z macro-backtrace for more info)

I apologize for not asking a clearly defined question.

[jonasbb]

Hi, if you want to get into more complicated implementations, I encourage you to read the documentation of SerializeAs. It contains some examples which show how to implement the code.

The problem is that you are asking serde_json to deserialize a Vec<Base62Encoded<u64>>, but you need to produce a Vec<u64>. You need to mark in your type signature, that the T inside JsonString is different than the T you return. You turn the JsonString into a container-like type.

I would be happy to accept a PR which changes the JsonString in this crate to allow for this added flexibility.

pub struct JsonString<T>(PhantomData<T>);

impl<T, TAs> SerializeAs<T> for JsonString<TAs>
where
    TAs: SerializeAs<T>,
{
    fn serialize_as<S>(source: &T, serializer: S) -> Result<S::Ok, S::Error>
    where
        S: Serializer,
    {
        // `collect_str` will needlessly call `Display` again.
        // Since you already have a `String` `serialize_str` is better
        serializer.serialize_str(
            &serde_json::to_string(&serde_with::ser::SerializeAsWrap::<T, TAs>::new(source))
                .map_err(SerializeError::custom)?,
        )
    }
}

---

A couple of notes on your code:

JsonString(source.clone()).serialize(serializer)

This looks like unbounded recursion to me. To use SerializeAs your types also do not need to implement Serialize.

The Base62Encoded is also quite complicated, going from SerializeAs to Serialize to Display. You can have a look here, for a similar type and see how it is implemented.
https://docs.rs/serde_with/1.14.0/serde_with/base64/struct.Base64.html

[spikespaz]

Ignoring previous attempts, I am now further along.

I don't know how to properly investigate this type mismatch.

cargo test --package awaur --lib --all-features -- serde_with::json_string::tests::test_wrapping

error[E0308]: mismatched types
   --> src/serde_with/json_string.rs:194:33
    |
194 |             #[derive(Serialize, Deserialize)]
    |                                 ^^^^^^^^^^^ expected `u64`, found struct `base62::wrapper::Base62`
    |
    = note: expected struct `Vec<u64>`
               found struct `Vec<base62::wrapper::Base62<u64>>`
    = note: this error originates in the macro `try` (in Nightly builds, run with -Z macro-backtrace for more info)

This is in the integration test that I wrote to see if wrapping works. It does not.
https://github.com/spikespaz/awaur/blob/master/src/serde_with/json_string.rs#L194

[jonasbb]

You use a different implementation for JsonString than I posted in my original reply. Any reason you choose not to use this working implementation?

[spikespaz]

Because I am failing to understand why/how SerializeAsWrap and DeserializeAsWrap work, and am trying to do it without just to experiment. I know that my current code is wrong, but I do have impl<T, U> SerializeAs<T> for JsonString<U>. Is U inferred to be u64, or is it inferred to be Base62<u64>? I don't know what the compiler talks about when it says "expected" and "found".

The trait is analogue to the serde::Serialize trait, with the same meaning of input and output arguments. It can and should the implemented using the same code structure as the Serialize trait. As such, the same advice for implementing Serialize applies here.

The bold text is also my reason for wanting to use the exact same implementation (awaur::serde_with::json_string::with::{serialize, deserialize}) for the inner type T. I think I'm getting confused because T in Serialize/SerializeAs is a different type from T in Deserialize/DeserializeAs. I wanted to use awaur::serde_with::{json_string, base62}::with::deserialize so that the same Visitor is used.

Any reason you choose not to use this working implementation?

I don't copy/paste code unless I totally understand it, so I'm still messing around with it.

[spikespaz]

SerializeAsWrap requires that U: SerializeAs<T> meaning that types that only implement Serialize do not work as inner types of JsonString<U>.

[jonasbb]

The trait is analogue to the serde::Serialize trait, with the same meaning of input and output arguments. It can and should the implemented using the same code structure as the Serialize trait. As such, the same advice for implementing Serialize applies here.

The bold text is also my reason for wanting to use the exact same implementation (awaur::serde_with::json_string::with::{serialize, deserialize}) for the inner type T. [...]

The quoted code does not permit using the exact same implementation. It says the same structure will work in both cases. And that remains true. You will need to do adjustments like using SerializeAs instead of Serialize, after all the two traits serve slightly different purposes. But overall, you can often very much mirror the implementation. For example, compare impl<T> Serialize for Option<T> with impl<T, U> SerializeAs<Option<T>> for Option<U>. Maybe that example also shows better why SerializeAsWrap is needed.

SerializeAsWrap requires that U: SerializeAs<T> meaning that types that only implement Serialize do not work as inner types of JsonString<U>.

That is indeed how Rusts trait bounds work. If you only use SerializeAs in the bound, you cannot use Serialize. I fail to see how this is a problem though.