Serialize & Deserialize

[nathanhammond]

There was a straightforward PR thrown up for discussion over at #65, effectively looking for a Serialize and Deserialize implementation for this library. The requests from @epage:

I think more of a case is needed for why a CLI flag needs serialization, particularly the fact that this was done in a raw way (separate verbose, quiet as u8s) - #65 (comment)

More design discussion is needed on this because exposing separate verbose and quiet fields that are u8 does not make sense. - #65 (comment)

---

Use Cases

Instrumentation

If an entire args object is being passed into tracing it is quite nice to have a Serialize so that derive works. I don't want to have to impl Serialize for Args for this reason. If I don't do some serialization then I end up with Debugand that is super-verbose.

use clap::Parser;
use serde::Serialize;
use svix_ksuid::{Ksuid, KsuidLike};
use tracing::instrument;

mod telemetry;

#[derive(Serialize, Parser, Debug)]
#[command(author, version, about, long_about = None)]
struct Args {
    // #[command(flatten)]
    // verbose: clap_verbosity_flag::Verbosity,
    #[arg(short, long, default_value_t = 1)]
    count: u8,
}

impl Args {
    fn to_json(&self) -> String {
        return serde_json::to_string(&self).expect("");
    }
}

// Main is a very small setup shim, and then gets out of the way.
fn main() -> proc_exit::ExitResult {
    let id = Ksuid::new(None, None);

    telemetry::init(id);
    let args = Args::parse();

    let result = run(id, args);
    proc_exit::exit(result);
}

#[instrument(name = "run()", fields(args = args.to_json(), id = id.to_string()) ret, err)]
fn run(id: Ksuid, args: Args) -> proc_exit::ExitResult {
    proc_exit::Code::SUCCESS.ok()
}

Args From <Elsewhere>

This some-cli gui command spins up a server and opens a browser to an HTML page. Eventually that HTML page makes a network request back to the server, which contains a serialized form of the arguments that would have been passed in. That's going to be JSON (cause it's coming from the web), deserializing into Args (from the above example).

Starts the server:

$ some-cli gui

(Complicated configuration happens in a browser.)

Data from a "configuration finished!" network request passed back in to the CLI entry point:

let args: Args = serde_json::from_str(args)?;
let result = run(id, args);

---

Design Space

There are basically only two reasonable options here to represent configuration since it is a count.

As Separate u8s

exposing separate verbose and quiet fields that are u8 does not make sense

Given that we have -q and -v[vvv] exposed as separate args this separation doesn't seem particularly egregious. Further, since the struct can be constructed with clap_verbosity_flag::Verbosity::new(2, 4), any serialization that doesn't give you both of those values is lossy.

Being able to represent the error conditions (error: the argument '--quiet...' cannot be used with '--verbose...') that can be triggered by command line usage is reasonable—even though typically it would never get through ::parse().

As A Single i16

Since we have the conflicts_with lockout from the primary clap use case, we could choose to serialize them into a single value, in spite of it being lossy. Take a page from the log mapping, and do quiet as -(count).

An i16 gives us way more bits than we need, but there isn't an i9.

...
-4 => -qqqq
-3 => -qqq
-2 => -qq
-1 => -q
0 => None
1 => -v
2 => -vv
3 => -vvv
4 => -vvvv
...

This approach can't support { quiet: 4, verbosity: 4 } like a manually constructed struct can.

---

I'm in favor of the separate u8s for representation fidelity reasons and nominate reopening and accepting #65.

[nathanhammond]

Here is what a remote derive implementation looks like, a lot of hassle and hackery:

[derive(Serialize, Deserialize)]
#[serde(remote = "clap_verbosity_flag::Verbosity")]
struct VerbosityDef {
    #[serde(getter = "clap_verbosity_flag::Verbosity::verbose")]
    pub verbose: u8,
    #[serde(getter = "clap_verbosity_flag::Verbosity::quiet")]
    pub quiet: u8,
}

pub trait VerbosityRemoteGetters {
    fn verbose(&self) -> u8;
    fn quiet(&self) -> u8;
}

impl VerbosityRemoteGetters for clap_verbosity_flag::Verbosity {
    fn verbose(&self) -> u8 {
        let debug_string = format!("{:#?}", self);
        let re = Regex::new(r"verbose: (?<verbose>[\d]+)").unwrap();
        let output = re.captures(&debug_string).unwrap();
        let string_value = output.name("verbose").unwrap().as_str();
        let calc = string_value.parse::<u8>().unwrap();
        calc
    }
    fn quiet(&self) -> u8 {
        let debug_string = format!("{:#?}", self);
        let re = Regex::new(r"quiet: (?<quiet>[\d]+)").unwrap();
        let output = re.captures(&debug_string).unwrap();
        let string_value = output.name("quiet").unwrap().as_str();
        let calc = string_value.parse::<u8>().unwrap();
        calc
    }
}

impl From<VerbosityDef> for clap_verbosity_flag::Verbosity {
    fn from(def: VerbosityDef) -> clap_verbosity_flag::Verbosity {
        clap_verbosity_flag::Verbosity::new(def.verbose, def.quiet)
    }
}

#[derive(Serialize, Deserialize, Parser, Debug)]
#[command(author, version, about, long_about = None)]
struct Args {
    #[serde(with = "VerbosityDef")]
    #[command(flatten)]
    verbose: clap_verbosity_flag::Verbosity,
}

---

Or, I could just copy/paste out the core of the implementation in fewer lines:

clap-verbosity-flag/src/lib.rs

Lines 64 to 90 in f44eadc

	#[derive(clap::Args, Debug, Clone, Default)]
	#[command(about = None, long_about = None)]
	pub struct Verbosity<L: LogLevel = ErrorLevel> {
	#[arg(
	long,
	short = 'v',
	action = clap::ArgAction::Count,
	global = true,
	help = L::verbose_help(),
	long_help = L::verbose_long_help(),
	)]
	verbose: u8,
	#[arg(
	long,
	short = 'q',
	action = clap::ArgAction::Count,
	global = true,
	help = L::quiet_help(),
	long_help = L::quiet_long_help(),
	conflicts_with = "verbose",
	)]
	quiet: u8,
	#[arg(skip)]
	phantom: std::marker::PhantomData<L>,
	}

[epage]

To be clear, are those your use cases or theoretical use cases?

As for the serialization format, I consider any non-textual serialization format to be a non-starter. As a user, entering random numbers in two fields and dealing with how those fields interact makes no sense and I would expect to just say "trace" or similar. The CLI is designed for user convenience / following convention and we hide the individual fields because that is the level of abstraction people should be working with.

[nathanhammond]

I have both of those use cases in the project I'm working on. You're looking at a trimmed down main.rs from a project that I'm just starting in on.

https://www.wispbuild.com

Designing a query for particular nodes in a huge DAG is hard; it's way easier to present them to the user for clicking/visual feedback of what is selected. They can run multiple times from the GUI context until it is right, and then serialize out the command string for later use.

Sample output for each option in the GUI makes it easy to figure out which options are in use, and applies to both verbose and quiet.

[nathanhammond]

I don't have super-strong opinions here (I've since dropped this as a dep), but I feel like there is a lot of value in standardizing the underlying tooling layer for the CLI hello world because it creates consistent UX outcomes for users.

The lack of Serialize and Deserialize caused me to eject from that vision.

---

Not exposing the numerics means that displaying the outcome as a range slider in the HTML GUI still requires mapping it myself from string, making the i16 approach more attractive. But that's not super-materially different since I need the labels anyway. (The web UI would never be two fields, the actual serialization format remains an implementation detail not presented to the end-user.)