-
Notifications
You must be signed in to change notification settings - Fork 244
/
README.md.jinja
179 lines (106 loc) · 6.86 KB
/
README.md.jinja
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
{% from "macros.md.jinja" import items -%}
# Structured text tools
The following is a list of text-based file formats and command-line tools for manipulating each.
## Contents
- [awk-like](#awk-like)
- [awk](#awk)
- [POSIX commands](#posix-commands)
- [SQL-based tools](#sql-based-tools)
- [Other tools](#other-tools)
- [CSV](#csv)
- [HTML](#html)
- [JSON](#json)
- [TOML](#toml)
- [XML](#xml)
- [YAML](#yaml)
- [Configuration files](#configuration-files)
- [/etc/hosts](#etchosts)
- [INI](#ini)
- [Multiple formats](#multiple-formats)
- [Log files](#log-files)
- [Multiformat tools](#multiformat-tools)
- [Templating for structured text](#templating-for-structured-text)
- [Extra: interactive TUIs](#extra-interactive-tuis)
- [Extra: CLIs for single-file databases](#extra-clis-for-single-file-databases)
- [License](#license)
- [Disclosure](#disclosure)
## awk-like
Tools that work with lines of fields separated by delimiters but do not necessarily support [CSV field quoting](https://en.wikipedia.org/wiki/Comma-separated_values#Basic_rules).
### awk
AWK/awk is a programming language and a POSIX-standard command-line tool. (You will sometimes see "awk" used for the tool and "AWK" for the language. This document follows this convention. GNU Awk uses "Awk".) If you run Linux, macOS, or a BSD, you almost certainly have it installed. See below for Windows.
- If you already know how to program, the nawk [man page](https://www.freebsd.org/cgi/man.cgi?query=nawk&sektion=1) is a great way to learn AWK quickly. What you learn from it will apply to other implementations on different platforms. Read it first if you feel overwhelmed by the sheer size of the [GNU Awk manual](https://www.gnu.org/software/gawk/manual/gawk.html).
- [awk.info archive](https://web.archive.org/web/20160505033644/http://awk.info/) **—** an extensive resource on Awk.
- ["AWK Vs NAWK Vs GAWK"](https://www.thegeekstuff.com/2011/06/awk-nawk-gawk/) **—** a comparison of features present in different implementations.
- [busybox-w32](https://frippery.org/busybox/) includes a full implementation of POSIX awk and other tools like `sed` in a single Windows executable.
- [frawk](https://github.com/ezrosent/frawk) is a Rust implementation of a language partially compatible with AWK that supports [parallelism](https://github.com/ezrosent/frawk/blob/master/info/parallelism.md) and CSV input and output.
- [GNU Awk 5 binaries for Windows](https://sourceforge.net/projects/ezwinports/files/) by [EZWinPorts](https://www.gnu.org/software/emacs/manual/html_node/efaq-w32/EZWinPorts.html).
- [GoAWK](https://github.com/benhoyt/goawk) is a cross-platform implementation of awk with added support for CSV. The project provides binaries for many platforms, including Windows.
### POSIX commands
- `comm` **—** Select the lines common to two sorted files or the lines contained in only one of them. (Manual: `man 1 comm` on your system, [GNU](https://linux.die.net/man/1/comm), [FreeBSD](https://www.freebsd.org/cgi/man.cgi?query=comm&sektion=1).)
- `cut` **—** Select portions of each line in one or more files. (Manual: `man 1 cut`, [GNU](https://linux.die.net/man/1/cut), [FreeBSD](https://www.freebsd.org/cgi/man.cgi?query=cut&sektion=1).)
- `grep` **—** Select the lines that match or do not match a pattern from one or more files. (Manual: `man 1 grep`, [GNU](https://linux.die.net/man/1/grep), [FreeBSD](https://www.freebsd.org/cgi/man.cgi?query=grep&sektion=1).)
- `join` **—** Take two files sorted by a common field and join their lines on the value of that field. Lines with values that do not appear in the other file are discarded. (Manual: `man 1 join`, [GNU](https://linux.die.net/man/1/join), [FreeBSD](https://www.freebsd.org/cgi/man.cgi?query=join&sektion=1).)
- `paste` **—** Combine several consecutive lines in a text file into one. (Manual: `man 1 paste`, [GNU](https://linux.die.net/man/1/paste), [FreeBSD](https://www.freebsd.org/cgi/man.cgi?query=paste&sektion=1).)
- `sort` **—** Sort lines by key fields. (Manual: `man 1 sort`, [GNU](https://linux.die.net/man/1/sort), [FreeBSD](https://www.freebsd.org/cgi/man.cgi?query=sort&sektion=1).)
- `uniq` **—** Find or remove repeated lines. (Manual: `man 1 uniq`, [GNU](https://linux.die.net/man/1/uniq), [FreeBSD](https://www.freebsd.org/cgi/man.cgi?query=uniq&sektion=1).)
### Other tools
{{ items("awk-other") }}
## CSV
CSV, TSV, and other delimiter-separated value formats. Tools belong on this list if they support [field quoting](https://en.wikipedia.org/wiki/Comma-separated_values#Basic_rules).
{{ items("csv") }}
### SQL-based tools
See the [big comparison list](sql-based.md). It covers
- AlaSQL CLI
- csvq
- csvsql
- fsql
- Musoq
- q
- RBQL
- rows
- Sqawk (dbohdan)
- sqawk (tjunier)
- Squawk
- termsql
- trdsql
- textql
## HTML
{{ items("html") }}
## JSON
{{ items("json") }}
## TOML
With a format converter like Remarshal you can use [JSON](#json) tools to process TOML and YAML, but make sure you do not lose data in the conversion.
{{ items("toml") }}
## XML
{{ items("xml") }}
### See also
- ["Grep and Sed Equivalent for XML Command Line Processing"](http://stackoverflow.com/questions/91791/grep-and-sed-equivalent-for-xml-command-line-processing) on Stack Overflow.
## YAML
{{ items("yaml") }}
## Configuration files
### /etc/hosts
{{ items("hosts") }}
### INI
{{ items("ini") }}
### Multiple formats
{{ items("config") }}
## Log files
{{ items("log") }}
## Multiformat tools
Tools that support multiple input formats.
Programs that convert between only two formats in both directions are excluded.
We only count JSON support that is separate from YAML.
{{ items("multiformat") }}
## Templating for structured text
Listed below are restricted programming language interpreters and templating tools that produce structured text output. They are generally intended to remove repetition in configuration files. They are distinct from unstructed templating tools like the `jinja2` CLI program, which should not be added to this table.
{{ items("template") }}
### See also
- [A comparison table of Nickel and other configuration languages](https://github.com/tweag/nickel/blob/master/RATIONALE.md#comparison-with-other-configuration-languages).
## Extra: interactive TUIs
{{ items("tui") }}
## Extra: CLIs for single-file databases
{{ items("single-file-db") }}
## License
The contents of this document is licensed under the [Creative Commons Attribution 4.0 International License](http://creativecommons.org/licenses/by/4.0/). By contributing you agree to release your contribution under this license.
## Disclosure
[csv2html](https://github.com/dbohdan/csv2html), [hosts](https://gitlab.com/dbohdan/hosts), [Sqawk](https://github.com/dbohdan/sqawk), [jsonwatch](https://github.com/dbohdan/jsonwatch), [Remarshal](https://github.com/remarshal-project/remarshal), and [initool](https://github.com/dbohdan/initool) are developed by the curator of this document.