Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Serde flatten #26

Open
glademiller opened this issue Dec 19, 2018 · 12 comments 路 May be fixed by #40
Open

Serde flatten #26

glademiller opened this issue Dec 19, 2018 · 12 comments 路 May be fixed by #40

Comments

@glademiller
Copy link

馃悰 Bug description

Using the flatten attribute from serde almost works but breaks in the case of non string values in flattened structs. In this case config always parses size as a string. However, if I put the size attribute directly in Config then everything works.

馃 Expected Behavior

The usize value in the flattened struct should parse

馃憻 Steps to reproduce

#[derive(Deserialize)]
struct Config {
   #[serde(flatten)]
   pub subconfig: Subconfig
}

#[derive(Deserialize)]
struct Subconfig {
   pub size: usize
}

馃實 Your environment

nightly-x86_64-unknown-linux-gnu (default)
rustc 1.33.0-nightly (a8a2a887d 2018-12-16)

envy version:
latest

@xoac
Copy link

xoac commented Jan 22, 2019

I have also noticed this bug with bool variables.

softprops added a commit that referenced this issue Apr 13, 2019
@softprops softprops linked a pull request Apr 13, 2019 that will close this issue
@blechatellier
Copy link

blechatellier commented Apr 22, 2019

@glademiller @xoac running into the same issue, did you find a workaround?

@neysofu
Copy link

neysofu commented Aug 16, 2019

Also looking for a solution to this.

@softprops
Copy link
Owner

I'm open to pull requests to add this feature

@neysofu
Copy link

neysofu commented Aug 16, 2019

I tried giving it a look but couldn't even locate the bug, any tips?

@jaboatman
Copy link

I spent some time trying to trace down where the issue is with this. Turns out #[serde(flatten)] deserializes the sub-structure as a map, which means it only works well for self-describing formats (e.g. JSON), since it will always defer to deserialize_any.

I haven't been able to come up with a good workaround yet.

@Pzixel
Copy link

Pzixel commented Apr 19, 2021

Faced this isssue today, went with manual deserializing:

use serde::de;
use std::{fmt, fmt::Display, marker::PhantomData, str::FromStr};

pub fn deserialize_stringified_any<'de, D, T>(deserializer: D) -> Result<T, D::Error>
where
    D: de::Deserializer<'de>,
    T: FromStr,
    T::Err: Display,
{
    deserializer.deserialize_any(StringifiedAnyVisitor(PhantomData))
}

pub struct StringifiedAnyVisitor<T>(PhantomData<T>);

impl<'de, T> de::Visitor<'de> for StringifiedAnyVisitor<T>
where
    T: FromStr,
    T::Err: Display,
{
    type Value = T;

    fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
        formatter.write_str("a string containing json data")
    }

    fn visit_str<E>(self, v: &str) -> Result<Self::Value, E>
    where
        E: de::Error,
    {
        Self::Value::from_str(v).map_err(E::custom)
    }
}

And then:

#[derive(Deserialize)]
struct Config {
   #[serde(flatten)]
   pub fluentd_config: FluentdConfig 
}

#[derive(Deserialize, Debug)]
pub struct FluentdConfig {
    pub fluentd_host: String,
    #[serde(deserialize_with = "deserialize_stringified_any")]
    pub fluentd_port: u16,
    pub fluentd_environment: String,
    pub fluentd_tag: String,
}

Not ideal but works to me

@nazar-pc
Copy link

I worked around by creating non-flattened struct and mapping that to desired after parsing environment variables, seemed easier than writing custom deserializers.

@SeedyROM
Copy link

SeedyROM commented Mar 19, 2022

Any new advice besides implementing deserialize_stringified_any and it's visitor? Thanks @Pzixel for the quick fix, did you just know that would work because you know serde well or is that some kind of escape hatch?

@SeedyROM
Copy link

SeedyROM commented Mar 19, 2022

I decided to go with not making my root configuration struct deserializable, and then using a prefix for each nested struct inside something like this:

#[derive(Deserialize, Serialize, Debug)]
pub struct DBConfig {
    host: String,
    port: u16,
    user: String,
    pass: String,
}

#[derive(Debug)]
pub struct Env {
    pub db: DBConfig,
}

impl Env {
    pub fn from_env() -> Result<Self, Report> {
        let db = envy::prefixed("PG").from_env::<DBConfig>()?;

        Ok(Self { db })
    }
}

(eg.) I pass in values as standard PostgreSQL envs and get back the correct values!: PGPASS=***

2022-03-19T09:56:16.719152Z  INFO example_envy: env=Env { db: DBConfig { host: "localhost", port: 5432, user: "psql", pass: "psql" } }

I think @nazar-pc might have been hinting at something similar?

@bhoudebert
Copy link

Any update on that one, it seems like if the nested type is always expected to be a string,

Tried to put an u64 but got the parsing error actual String not a u64.

@rollo-b2c2
Copy link

@Pzixel

Couldn't we just call that on every field? I get it'd be slow, but if your bottleneck is parsing envars you probably have another problem

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.