dedup items in Vec<T> field

[nklhtv]

Hi,

How to deduplicate vector entries of a field?
I've tried to use FromInto<HashSet> but that did not work out.
I've also tried to impl<'de, T, U> DeserializeAs<'de, Vec<T>> for Dedup<U> where Dedup is struct Dedup<T: Hash + Eq>(PhantomData<T>); but could not complete that either.

#[derive(Hash, PartialEq, Eq)]
struct A {
    id: String,
    name: String,
}

#[derive(From, Into)]
struct B(A);

impl Hash for B {
    fn hash<H: Hasher>(&self, state: &mut H) {
        self.0.id.hash(state);
    }
}

impl PartialEq for B {
    fn eq(&self, other: &Self) -> bool {
        self.0.id == other.0.id
    }
}

impl Eq for B {}

struct MyStruct {
    #[serde_as(as = "Dedup<B>")]
    a: Vec<A>,
}

Any suggestions?
Eventually i will need to sort the vec, but i cannot use HashSet or BTreeSet

[jonasbb]

I've also tried to impl<'de, T, U> DeserializeAs<'de, Vec<T>> for Dedup<U> where Dedup is struct Dedup<T: Hash + Eq>(PhantomData<T>); but could not complete that either.

Could you show what you tried so far? And please also include the serialized form of your data and how you expect the deserialized form to look like.

I've tried to use FromInto<HashSet> but that did not work out.

That is because there is no impl<T> From<HashSet<T>> for Vec<T>. You could work with IntoIterator and FromIterator, but there is nothing premade.

[nklhtv]

Hi @jonasbb You can check this commit to see my current implementation.
i want to deduplicate manifest catalogs by id & type when deserializing and serializing, while preserving the original order.
example:

{catalogs: [{"id": "top", type: "movie", ... }, {"id": "top", type: "series", ...}, {"id": "top", type: "movie", ...}] }

must be deserialized as

vec![ManifestCatalog { id: "top", r#type: "movie", ... }, ManifestCatalog { id: "top", r#type: "series", ... }]

[jonasbb]

I'm sorry. My comment about the type bounds is indeed entirely wrong. I seemed to have confused myself quite badly. I'm glad to hear that everything is working for you. The refactoring is quite more readable. I'm wondering if something like this might be worthwhile.

https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=6c401f4f984395d5b7085f831dde4435

[nklhtv]

@jonasbb Thank you for the proposal! It looks a lot better now, i just applied it.
I was also trying to remove the need of cloning (.to_owned()) in the impl UniqueVecAdapter blocks, but couldn't figure out the lifetimes.
My idea was to return (&String, &String) but that might be impossible :/
Any ideas?

[jonasbb]

@nklhtv Maybe something like this? I don't know what constraints the unique_by function requires.

pub trait UniqueByAdapter<'a> {
    type Original;
    type Comparable: Eq + Hash;

    fn call(t: &'a Self::Original) -> Self::Comparable;
}

impl<'a> UniqueByAdapter<'a> for ManifestCatalogUniqueByAdapter {
    type Original = ManifestCatalog;
    type Comparable = (&'a str, &'a str);

    fn call(value: &'a Self::Original) -> Self::Comparable {
        (&value.id, &value.r#type)
    }
}

[nklhtv]

@jonasbb Thats the signature of unique_by
Thats my attempt to integrate your last proposal, but did not work out.
One of the errors is very wierd. idk how to solve it:

the type parameter `U` is not constrained by the impl trait, self type, or predicates
  --> src/types/serde_as_ext.rs:32:13

the other error, from rust-analyser, is in the DeserializeAs impl:

cannot infer an appropriate lifetime for lifetime parameter `'a` due to conflicting requirements

note: expected `<A as types::serde_as_ext::UniqueVecAdapter<'de>>::Input`
         found `<A as types::serde_as_ext::UniqueVecAdapter<'_>>::Input`

because item param in the unique_by callback has shorter lifetime than 'de :/

Maybe i should just keep using the owned values in the Adapter

[nklhtv]

I will keep using the owned values in the Adapter for now. Thank you for your help @jonasbb