Add support for merging config into option values, and get+set option value source

[shadowspawn]

Pull Request

Work in progress.

Problem

People would like to load configuration, from files or elsewhere.

Resolves: #1584

Config file/hash: #151 #328 #665

At a lower-level:

• people ask about how to tell whether options were specified on the command-line. This is not easy when option takes optional value or has default value.
• people ask about managing priority of default/config/env/cli for rolling their own solutions

Related: #1394 #1541 #1595

Solution

There are many different potential config file formats and file locations and patterns of usage. The goal here is to provide the tools to support author implementing their own desired config support.

1. low level: expose the "source" of option values. i.e. default/config/env/cli

2. medium level: add method to merge configuration values. The priority order is default < config < env < cli. The merge is quite subtle to support full Commander functionality including option types and custom parsing.

The actual reading of a configuration file is left to author to support JSON/js/es/yaml/ini/TOML or package.json property or whatever they wish to offer.

ChangeLog

[shadowspawn]

I am happy with this as a proof of concept of merging in config option-arguments in a rigorous way for one workflow:

• text values, going through custom option processing (same functionality as command-line, no more, no less)
• supports the various option types: required, optional, boolean, negated
• supports variadic, and choices, and custom processing
• respects priority default < config < env < cli

The hardest part was calling the option-argument processing correctly, and that could also be made available separately if it helps provide solutions for future problems.

However, this PR does not provide support for other things which comes up in config discussions, such as loading default values, or merging in native javascript values which do not go through custom processing, or a pattern for separating options by subcommand.

I'm not currently confident this will help enough people without raising more questions.

Going to focus for now on making the low-level "source" routines available so people have the extra information needed to roll their own solutions (#1613). This was identified as a current blocker in discussions in #1584.

low level: expose the "source" of option values. i.e. default/config/env/cli

[shadowspawn]

Copy of the example use of this PR I posted in #1584 (comment)

const { program, Option } = require('commander');

program
  .option('-d, --debug')
  .option('-c, --color')
  .option('-C, --no-color')
  .option('-o, --optional [value]')
  .option('-r, --required <value>', 'string', 'default string')
  .option('-f, --float <number>', 'number', parseFloat)
  .option('-f', 'number', parseFloat)
  .option('-l, --list <values...>')
  .addOption(new Option('-p, --port <number>').env('PORT'))
  .option('--use-config')
program.parse();

// Note: the multiple keys for a single option are to demonstrate variations, not a real use case!
if (program.opts().useConfig) {
  const config = {
    debug: true,
    color: true,        // same as CLI --color
    color: false,       // same as CLI --no-color
    '--no-color': true, // same as CLI --no-color
    '-o': true,         // same as CLI -o
    'optional': 'config string',
    'required': 'config string',
    'float': '3.4',
    'list': ['apple', 'pear'],
    'port': '443'
  };
  program.mergeConfig(config);
}

console.log(program.opts());

$ node merge.js                                
{ required: 'default string' }

$ node merge.js --use-config                   
{
  required: 'config string',
  useConfig: true,
  debug: true,
  color: false,
  optional: 'config string',
  float: 3.4,
  list: [ 'apple', 'pear' ],
  port: '443'
}

$ PORT=9000 node merge.js -r RRR --use-config -o OOO            
{
  required: 'RRR',
  useConfig: true,
  optional: 'OOO',
  port: '9000',
  debug: true,
  color: false,
  float: 3.4,
  list: [ 'apple', 'pear' ]
}

[shadowspawn]

Research links

• Add support for config files #1584
• https://github.com/yargs/yargs/blob/master/docs/api.md#config
  • .config([key], [description], [parseFn])
• https://github.com/yargs/yargs/blob/master/docs/api.md#pkgconfkey-cwd
  • .pkgConf(key, [cwd])
• https://docs.python.org/3/library/argparse.html#fromfile-prefix-chars
  • @args.txt, one option per line
• https://github.com/davidtheclark/cosmiconfig (used by gluegun, looks in lots of places)
• https://github.com/dominictarr/rc (looks in lots of places)
• https://github.com/spf13/viper (c.f. cobra)
• https://eslint.org/docs/user-guide/configuring/configuration-files#cascading-and-hierarchy
• https://mochajs.org/#configuring-mocha-nodejs
• https://stackoverflow.com/a/5826167/1082434 (popular argparse example)