Possible issues around file encoding

[weiznich]

This is mostly a second hand report. We got an issue report at the diesel repo about issues where the .env file uses a different encoding that UTF-8. See here for the original report.

I got the same issue and managed to also fix it using VSCode to create the file. The difference between the files was that the file created from command line had UTF-16 LE encoding and the other one was UTF-8. It should probably be mentioned in the getting started page or to fix parsing the file to accept different encodings.
I am on also on Windows 10 and using powershell.

Not sure if that counts a bug in dotenvy or if that should be handled at application level.

[allan2]

Could you provide example .env files?

[weiznich]

Unfortunately I do not have these issues myself. I only got reports without actual example files. But there is at least that in the linked issue:

echo DATABASE_URL=postgres://<omitted-username>:<omitted-password>@<omitted-host>/<omitted-db> > .env

to create the .env file on windows 10 with powershell to reproduce? the issue

[allan2]

It works for me. PowerShell/echo created the file in UTF-8 on my system.

use dotenvy::dotenv;
use std::{env, error::Error};

fn main() -> Result<(), Box<dyn Error>> {
    dotenv()?;
    println!("DATABASE_URL: {}", env::var("DATABASE_URL")?);
    Ok(())
}

[petzki]

echo DATABASE_URL=test.db > .env
Produced the attached file. I had to rename it to be able to upload it. File size is 46 bytes.
env.txt

[allan2]

Using my earlier example, you would get this error:

thread 'main' panicked at 'Error: Io(Error { kind: InvalidData, message: "stream did not contain valid UTF-8" })', src\main.rs:5:14

If you do need to handle .env files that may or may not be UTF-8 encoded, the encoding_rs_io crate is useful.

The following example works with your provided file:

use encoding_rs_io::DecodeReaderBytes;
use std::{
    env, error, fs,
    io::{self, Read},
};

const DOTENV_FILE: &str = "env.txt";

fn main() -> Result<(), Box<dyn error::Error>> {
    if let Err(e) = dotenvy::from_path(DOTENV_FILE) {
        match e {
            dotenvy::Error::Io(io_err) => match io_err.kind() {
                io::ErrorKind::InvalidData => {
                    // This is the error when the stream doesn't contain valid UTF-8

                    let bytes = fs::read(DOTENV_FILE)?;
                    let mut decoder = DecodeReaderBytes::new(&bytes[..]);
                    let mut dest = Vec::new();

                    // Must read to end to ensure that the stream is fully decoded
                    decoder.read_to_end(&mut dest)?;
                    dotenvy::from_read(&dest[..])?;
                }
                _ => return Err(io_err.into()),
            },
            _ => return Err(e.into()),
        }
    }

    println!("DATABASE_URL: {}", env::var("DATABASE_URL")?);
    Ok(())
}

Supporting additional encodings is out of scope for this small crate, but I can add this example to this repo to help future users.

Hope this helps!

[weiznich]

Adding an example sounds like a great idea 👍