cmd: configure all the things through CLI (#98)

* cmd: configure all the things through CLI This PR addresses all the gaps in CLI flags vs the `.yamlfmt` configuration file. All parts of the `yamlfmt` can now be adjusted entirely through command line flags. This PR also adds a huge amount of documentation which has been sorely lacking in the past. * fix table formatting * pre-commit suggest v0.8.0 * pre-commit docs correct revision
google · Mar 25, 2023 · b26cfb8 · b26cfb8
1 parent 07d5d97
commit b26cfb8
Show file tree

Hide file tree

Showing 9 changed files with 224 additions and 86 deletions.
diff --git a/README.md b/README.md
@@ -10,21 +10,27 @@
 
 ## Maintainers
 
-This tool is not yet officially supported by Google. It is currenlty maintained solely by @braydonk.
+This tool is not yet officially supported by Google. It is currenlty maintained solely by @braydonk, and unless something changes primarily in spare time.
 
 ## Installation
 
 To download the `yamlfmt` command, you can download the desired binary from releases or install the module directly:
 ```
 go install github.com/google/yamlfmt/cmd/yamlfmt@latest
 ```
+This currently requires Go version 1.20 or greater.
+
 NOTE: Recommended setup if this is your first time installing Go would be in [this DigitalOcean blog post](https://www.digitalocean.com/community/tutorials/how-to-build-and-install-go-programs).
 
-You can also simply download the binary you want from releases. The binary is self-sufficient with no dependencies, and can simply be put somewhere on your PATH and run with the command `yamlfmt`.
+You can also download the binary you want from releases. The binary is self-sufficient with no dependencies, and can simply be put somewhere on your PATH and run with the command `yamlfmt`.
+
+You can also install the command as a [pre-commit](https://pre-commit.com/) hook. See the [pre-commit hook](./docs/pre-commit.md) docs for instructions.
+
+## Basic Usage
 
-## Usage
+See [Command Usage](./docs/command-usage.md) for in depth information and available flags.
 
-To run the tool with all default settings, simply run the command with a path argument:
+To run the tool with all default settings, run the command with a path argument:
 ```bash
 yamlfmt x.yaml y.yaml <...>
 ```
@@ -39,70 +45,7 @@ yamlfmt -dstar **/*.{yaml,yml}
 ```
 See the [doublestar](https://github.com/bmatcuk/doublestar) package for more information on this format.
 
-## Flags
-
-The CLI supports the following flags/arguments:
-
-* Format (default, no flags)
-	- Format and write the matched files
-* Dry run (`-dry` flag)
-	- Format the matched files and output the diff to `stdout`
-* Lint (`-lint` flag)
-	- Format the matched files and output the diff to `stdout`, exits with status 1 if there are any differences
-* Quiet mode (`-quiet` flag)
-  - Affects `-lint` and `-dry` so they only output the filepaths with differences rather than the full diff contents
-* Stdin (just `-` or `/dev/stdin` argument, or `-in` flag)
-	- Format the yaml data from `stdin` and output the result to `stdout`
-* Custom config path (`-conf` flag)
-	- If you would like to use a config not stored at `.yamlfmt` in the working directory, you can pass a relative or absolute path to a separate configuration file
-* Doublestar path collection (`-dstar` flag)
-	- If you would like to use 
-
-(NOTE: If providing paths as command line arguments, the flags must be specified before any paths)
-
 # Configuration File
 
-The `yamlfmt` command can be configured through a yaml file called `.yamlfmt`.
-
-## Include/Exclude
-
-If you would like to have a consistent configuration for include and exclude paths, you can also use a configuration file. The tool will attempt to read a configuration file named `.yamlfmt` in the directory the tool is run on. In it, you can configure paths to include and exclude, for example:
-```yaml
-include:
-  - config/**/*.{yaml,yml}
-exclude:
-  - excluded/**/*.yaml
-```
-
-## Line Ending
-
-The default line ending is `lf` (Unix style, Mac/Linux). The line ending can be changed to `crlf` (Windows style) with the `line_ending` setting:
-```yaml
-line_ending: crlf
-```
-This setting will be sent to any formatter as a config field called `line_ending`. If a `line_ending` is specified in the formatter, this will overwrite it. New formatters are free to ignore this setting if they don't need it, but any formatter provided by this repo will handle it accordingly.
-
-## Formatter
-
-In your `.yamlfmt` file you can also specify configuration for the formatter if that formatter supports it. To change the indentation level of the basic formatter for example:
-```yaml
-formatter:
-  type: basic
-  indent: 4
-```
-If the type is not specified, the default formatter will be used. In the tool included in this repo, the default is the [basic formatter](formatters/basic).
-
-For in-depth configuration documentation see the [config docs](docs/config.md).
-
-# pre-commit
-
-Starting in v0.7.1, `yamlfmt` can be used as a hook for the popular [pre-commit](https://pre-commit.com/) tool. To include a `yamlfmt` hook in your `pre-commit` config, add the following to the `repos` block in your `.pre-commit-config.yaml`:
-
-```yaml
-- repo: https://github.com/google/yamlfmt
-  rev: v0.7.1
-  hooks:
-    - id: yamlfmt
-```
-
-When running yamlfmt with the `pre-commit` hook, the only way to configure it is through a `.yamlfmt` configuration file in the root of the repo or a system wide config directory (see [Configuration File](#configuration-file)). 
+The `yamlfmt` command can be configured through a yaml file called `.yamlfmt`. This file can live in your working directory, a path specified through a [CLI flag](./docs/command-usage.md#operation-flags), or in the standard global config path on your system (see docs for specifics).
+For in-depth configuration documentation see [Config](docs/config.md).
diff --git a/cmd/yamlfmt/config.go b/cmd/yamlfmt/config.go
@@ -7,6 +7,8 @@ import (
 	"os"
 	"path/filepath"
 	"runtime"
+	"strconv"
+	"strings"
 
 	"github.com/braydonk/yaml"
 	"github.com/google/yamlfmt"
@@ -179,6 +181,20 @@ func makeCommandConfigFromData(configData map[string]any) (*command.Config, erro
 		return nil, err
 	}
 
+	// Parse overrides for formatter configuration
+	if len(flagFormatter) > 0 {
+		overrides, err := parseFormatterConfigFlag(flagFormatter)
+		if err != nil {
+			return nil, err
+		}
+		for k, v := range overrides {
+			if k == "type" {
+				config.FormatterConfig.Type = v.(string)
+			}
+			config.FormatterConfig.FormatterSettings[k] = v
+		}
+	}
+
 	// Default to OS line endings
 	if config.LineEnding == "" {
 		config.LineEnding = yamlfmt.LineBreakStyleLF
@@ -202,5 +218,46 @@ func makeCommandConfigFromData(configData map[string]any) (*command.Config, erro
 		config.Include = flag.Args()
 	}
 
+	// Append any additional data from array flags
+	config.Exclude = append(config.Exclude, flagExclude...)
+	config.Extensions = append(config.Extensions, flagExtensions...)
+
 	return config, nil
 }
+
+func parseFormatterConfigFlag(flagValues []string) (map[string]any, error) {
+	formatterValues := map[string]any{}
+	flagErrors := []error{}
+
+	// Expected format: fieldname=value
+	for _, configField := range flagValues {
+		if strings.Count(configField, "=") != 1 {
+			flagErrors = append(
+				flagErrors,
+				fmt.Errorf("badly formatted config field: %s", configField),
+			)
+			continue
+		}
+
+		kv := strings.Split(configField, "=")
+
+		// Try to parse as integer
+		vInt, err := strconv.ParseInt(kv[1], 10, 64)
+		if err == nil {
+			formatterValues[kv[0]] = vInt
+			continue
+		}
+
+		// Try to parse as boolean
+		vBool, err := strconv.ParseBool(kv[1])
+		if err == nil {
+			formatterValues[kv[0]] = vBool
+			continue
+		}
+
+		// Fall through to parsing as string
+		formatterValues[kv[0]] = kv[1]
+	}
+
+	return formatterValues, errors.Join(flagErrors...)
+}
diff --git a/cmd/yamlfmt/flags.go b/cmd/yamlfmt/flags.go
@@ -17,6 +17,7 @@ package main
 import (
 	"flag"
 	"fmt"
+	"strings"
 
 	"github.com/google/yamlfmt/command"
 )
@@ -30,8 +31,33 @@ operation without performing it.`)
 	flagConf       *string = flag.String("conf", "", "Read yamlfmt config from this path")
 	flagDoublestar *bool   = flag.Bool("dstar", false, "Use doublestar globs for include and exclude")
 	flagQuiet      *bool   = flag.Bool("quiet", false, "Print minimal output to stdout")
+	flagExclude            = arrayFlag{}
+	flagFormatter          = arrayFlag{}
+	flagExtensions         = arrayFlag{}
 )
 
+func bindArrayFlags() {
+	flag.Var(&flagExclude, "exclude", "Paths to exclude in the chosen format (standard or doublestar)")
+	flag.Var(&flagFormatter, "formatter", "Config value overrides to pass to the formatter")
+	flag.Var(&flagExtensions, "extensions", "File extensions to use for standard path collection")
+}
+
+type arrayFlag []string
+
+// Implements flag.Value
+func (a *arrayFlag) String() string {
+	return strings.Join(*a, " ")
+}
+
+func (a *arrayFlag) Set(value string) error {
+	values := []string{value}
+	if strings.Contains(value, ",") {
+		values = strings.Split(value, ",")
+	}
+	*a = append(*a, values...)
+	return nil
+}
+
 func configureHelp() {
 	flag.Usage = func() {
 		fmt.Println(`yamlfmt is a simple command line tool for formatting yaml files.

diff --git a/cmd/yamlfmt/main.go b/cmd/yamlfmt/main.go
@@ -30,6 +30,7 @@ func main() {
 }
 
 func run() error {
+	bindArrayFlags()
 	configureHelp()
 	flag.Parse()
 

diff --git a/command/command.go b/command/command.go
@@ -35,9 +35,9 @@ const (
 	OperationStdin
 )
 
-type formatterConfig struct {
-	Type              string                 `mapstructure:"type"`
-	FormatterSettings map[string]interface{} `mapstructure:",remain"`
+type FormatterConfig struct {
+	Type              string         `mapstructure:"type"`
+	FormatterSettings map[string]any `mapstructure:",remain"`
 }
 
 type Config struct {
@@ -46,7 +46,7 @@ type Config struct {
 	Exclude         []string               `mapstructure:"exclude"`
 	Doublestar      bool                   `mapstructure:"doublestar"`
 	LineEnding      yamlfmt.LineBreakStyle `mapstructure:"line_ending"`
-	FormatterConfig *formatterConfig       `mapstructure:"formatter,omitempty"`
+	FormatterConfig *FormatterConfig       `mapstructure:"formatter,omitempty"`
 }
 
 type Command struct {

diff --git a/docs/command-usage.md b/docs/command-usage.md
@@ -0,0 +1,96 @@
+# Command Usage
+
+The `yamlfmt` command supports 3 primary modes of operation and a number of command line configuration options.
+
+## Basic Usage
+
+The most basic usage of the `yamlfmt` command is to use the default configuration and provide a single path to format.
+
+```bash
+yamlfmt x.yaml
+```
+
+You can also provide a directory path, which yamlfmt will search recursively through for any file with a `yml` or `yaml` extension.
+
+```bash
+yamlfmt .
+```
+
+You can make use of the alternative Doublestar path mode for more dyanmic patterns.
+
+```bash
+yamlfmt -dstar "**/*.yaml"
+```
+(Technically, providing the same pattern in bash without quotes in standard path mode will do standard bash glob expansion and work similarly.)
+
+See [Specifying Paths](./paths.md) for more details about the path collection behaviour.
+
+## Three Modes of Operation
+
+The command supports three modes of operation: Format, Lint, and Dry Run.
+
+### Format
+
+This is the default operation (no flag specified). It will collect all paths that match the include patterns and run them straight through formatting, rewriting each file with the results.
+
+### Format stdin
+
+This mode will read input from stdin and output formatted result to stdout. By using the paths `-`, `/dev/stdin`, or using the `-in` flag, `yamlfmt` will only read from stdin and write the result to stdout.
+```bash
+cat x.yaml | yamlfmt -
+cat x.yaml | yamlfmt /dev/stdin
+cat x.yaml | yamlfmt -in
+```
+(Despite `/dev/stdin` and `-` being Linux/Bash standards, this convention was manually implemented in a cross-platform way. It will function identically on Windows. Would you want to use the Linux standards like this on Windows? I don't know, but it will work :smile:)
+
+### Dry Run
+
+This mode is enabled through the `-dry` flag. This will collect all paths that match the include patterns and run them through formatting, printing each file that had a formatting diff to stdout. This mode is affected by the `-quiet` flag, where only the paths of the files with diffs will be printed.
+
+### Lint
+
+This mode is enabled through the `-lint` flag. This will collect all paths that match the include patterns and run them through formatting, and will exit with code 1 (fail) if any files have formatting differences, outputting the diffs to stdout. This mode is also affected by the `-quiet` flag, where only the paths of the files with diffs will be printed.
+
+## Flags
+
+All flags must be specified **before** any path arguments.
+
+### Operation Flags
+
+These flags adjust the command's mode of operation. All of these flags are booleans.
+
+| Name       | Flag     | Example                    | Description |
+|:-----------|:---------|:---------------------------|:------------|
+| Help       | `-help`  | `yamlfmt -help`            | Print the command usage information. |
+| Dry Run    | `-dry`   | `yamlfmt -dry .`           | Use [Dry Run](#dry-run) mode |
+| Lint       | `-lint`  | `yamlfmt -lint .`          | Use [Lint](#lint) mode |
+| Quiet Mode | `-quiet` | `yamlfmt -dry -quiet .`    | Use quiet mode. Only has effect in Dry Run or Lint modes. |
+| Read Stdin | `-in`    | `cat x.yaml | yamlfmt -in` | Read input from stdin and output result to stdout. |
+
+### Configuration Flags
+
+These flags will configure the underlying behaviour of the command.
+
+The string array flags can be a bit confusing. See the [String Array Flags](#string-array-flags) section for more information.
+
+| Name             | Flag          | Type     | Example                                                   | Description |
+|:-----------------|:--------------|:---------|:----------------------------------------------------------|:------------|
+| Config File Path | `-conf`       | string   | `yamlfmt -conf ./config/.yamlfmt`                         | Specify a path to read a [configuration file](./config-file.md) from. |
+| Doublstar        | `-dstar`      | boolean  | `yamlfmt -dstar "**/*.yaml"`                              | Enable [Doublstar](./paths.md#doublestar) path collection mode. |
+| Exclude          | `-exclude`    | []string | `yamlfmt -exclude ./not/,these_paths.yaml`                | Patterns to exclude from path collection. These add to exclude patterns specified in the [config file](./config-file.md) |
+| Extensions       | `-extensions` | []string | `yamlfmt -extensions yaml,yml`                            | Extensions to use in standard path collection. Has no effect in Doublestar mode. These add to extensions specified in the [config file](./config-file.md) 
+| Formatter Config | `-formatter`  | []string | `yamlfmt -formatter indent=2,include_document_start=true` | Provide configuration values for the formatter. See [Formatter Configuration Options](./config-file.md#basic-formatter) for options. Each field is specified as `configkey=value`. |
+
+#### String Array Flags
+
+String array flags can be provided in two ways. For example with a flag called `-arrFlag`:
+
+* Individual flags
+    - `-arrFlag a -arrFlag b -arrFlag c`
+    - Result: `arrFlag: [a b c]`
+* Comma separated value in single flag
+    - `-arrFlag a,b,c`
+    - Result: `arrFlag: [a b c]`
+* Technically they can be combined but why would you?
+    - `-arrFlag a,b -arrFlag c`
+    - Result: `arrFlag: [a b c]`
diff --git a/docs/config.md → docs/config-file.md b/docs/config.md → docs/config-file.md
@@ -20,21 +20,11 @@ The command package defines the main command engine that `cmd/yamlfmt` uses. It
 |:-------------------------|:---------------|:--------|:------------|
 | `line_ending`            | `lf` or `crlf` | `crlf` on Windows, `lf` otherwise | Parse and write the file with "lf" or "crlf" line endings. This global setting will override any formatter `line_ending` options. |
 | `doublestar`             | bool           | false   | Use [doublestar](https://github.com/bmatcuk/doublestar) for include and exclude paths. (This was the default before 0.7.0) |
-| `include`                | []string       | []      | The paths for the command to include for formatting. See [Specifying Paths](#specifying-paths) for more details. |
-| `exclude`                | []string       | []      | The paths for the command to exclude from formatting. See [Specifying Paths](#specifying-paths) for more details. |
-| `extensions`             | []string       | []      | The extensions to use for standard mode path collection. See [Specifying Paths](#specifying-paths) for more details. |
+| `include`                | []string       | []      | The paths for the command to include for formatting. See [Specifying Paths][] for more details. |
+| `exclude`                | []string       | []      | The paths for the command to exclude from formatting. See [Specifying Paths][] for more details. |
+| `extensions`             | []string       | []      | The extensions to use for standard mode path collection. See [Specifying Paths][] for more details. |
 | `formatter`              | map[string]any | default basic formatter | Formatter settings. See [Formatter](#formatter) for more details. |
 
-## Specifying paths
-
-### Standard
-
-In standard path mode, the you can specify a file or directory path directly. If specifying a file, it will simply include the file. If specifying a directory, it will include every file with the correct extension (as specified in `extensions`).
-
-### Doublestar
-
-In Doublestar mode, paths are specified using the format explained in the [doublestar](https://github.com/bmatcuk/doublestar) package. It is almost identical to bash and git's style of glob pattern specification.
-
 ## Formatter
 
 Formatter settings are specified by giving a formatter type in the `type` field, and specifying the rest of the formatter settings in the same block. For example, to get a default `basic` formatter, use the following configuration:
@@ -70,3 +60,5 @@ The basic formatter is a barebones formatter that simply takes the data provided
 ### Note on `max_line_length`
 
 It's not perfect; it uses the `best_width` setting from the yaml library. If there's a very long token that extends too far for the line width, it won't split it up properly. I will keep trying to make this work better, but decided to get a version of the feature in that works for a lot of scenarios even if not all of them.
+
+[Specifying Paths]: ./paths.md
diff --git a/docs/paths.md b/docs/paths.md
@@ -0,0 +1,11 @@
+# Paths
+
+`yamlfmt` can collect paths in two modes: Standard, or Doublestar.
+
+## Standard (default)
+
+In standard path mode, the you can specify a file or directory path directly. If specifying a file, it will simply include the file. If specifying a directory, it will include every file with the correct extension (as specified in `extensions`, default is `yml` and `yaml`).
+
+## Doublestar
+
+In Doublestar mode, paths are specified using the format explained in the [doublestar](https://github.com/bmatcuk/doublestar) package. It is almost identical to bash and git's style of glob pattern specification.