Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: bitfield/script
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v0.21.4
Choose a base ref
...
head repository: bitfield/script
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: v0.22.0
Choose a head ref
  • 8 commits
  • 10 files changed
  • 10 contributors

Commits on Apr 3, 2023

  1. Fix Bytes/AppendFile/WriteFile to return read errs 

    Calling any sink method (such as Bytes) should return a non-nil error if there is an error reading the pipe to completion. Due to a bug, Bytes, AppendFile, and WriteFile failed to do this. This commit fixes the bug, and also tests that the other sink methods behave correctly (which they do). Fixes #170.

Co-authored-by: Namho Kim <git@namho.kim>
    @bitfield @nk-87
    bitfield and nk-87 committed Apr 3, 2023
    Copy the full SHA
    5bcb73e View commit details

Commits on Apr 22, 2023

  1. Remove arbitrary 64KiB limit on line length 

    The default bufio.Scanner is limited to 64 * 1024 bytes, throwing a 'token too
long' error if a line exceeds this size. This commit sets the token limit
instead to math.MaxInt, which is the biggest possible slice capacity in Go, and
effectively limits the scan line length only by available memory (fixes #167).

Co-authored-by: Tim Hobbs <timothy.hobbs@servicelayers.com>
    @bitfield @hobti01
    bitfield and hobti01 committed Apr 22, 2023
    Copy the full SHA
    5d7a635 View commit details

  2. Disable longline test on Windows

    @bitfield
    bitfield committed Apr 22, 2023
    Copy the full SHA
    eb962d8 View commit details

  3. Use GitHub Actions for testing

    @bitfield
    bitfield committed Apr 22, 2023
    Copy the full SHA
    341ce0e View commit details

  4. Update CI badge

    @bitfield
    bitfield committed Apr 22, 2023
    Copy the full SHA
    10f067b View commit details

  5. Add Tee filter (fixes #145) 

    Co-authored-by: Hans Raaf <hara@oderwat.de>
Co-authored-by: "Masci, Richard (rx7322)" <rx7322@us.att.com>
Co-authored-by: Ryan <Emailryanmay@gmail.com>
Co-authored-by: Gerard Webb <53147028+gedw99@users.noreply.github.com>
    @bitfield @oderwat @ryan-n-may
    5 people committed Apr 22, 2023

    Unverified

    This commit is not signed, but one or more authors requires that any commit attributed to them is signed.
    Learn about vigilant mode
    Copy the full SHA
    bee1290 View commit details

  6. Add WithStderr (fixes #128, fixes #146) 

    Co-authored-by: "Masci, Richard (rx7322)" <rx7322@us.att.com>
Co-authored-by: Jesse Lang <jdlang@spscommerce.com>
Co-authored-by: Nithin Chintala <nithin.chintala@schrodinger.com>
Co-authored-by: floydwch <floydwch@gmail.com>
    @bitfield @jesselang
    @nchint @floydwch
    5 people committed Apr 22, 2023
    Copy the full SHA
    7acd35c View commit details

  7. Refactor

    @bitfield
    bitfield committed Apr 22, 2023
    Copy the full SHA
    95cd539 View commit details
Showing with 505 additions and 201 deletions.
  1. +0 −26 .circleci/config.yml
  2. +16 −0 .github/workflows/test.yml
  3. +13 −1 README.md
  4. +4 −4 go.mod
  5. +32 −16 go.sum
  6. +111 −86 script.go
  7. +310 −68 script_test.go
  8. +11 −0 script_unix_test.go
  9. +2 −0 testdata/script/args_sends_arguments_to_pipe.txtar
  10. +6 −0 testdata/script/stdin_sends_stdin_to_pipe.txtar
26 changes: 0 additions & 26 deletions .circleci/config.yml
View file

This file was deleted.

16 changes: 16 additions & 0 deletions .github/workflows/test.yml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Based on https://github.com/mvdan/github-actions-golang
on: [push, pull_request]
name: Tests
jobs:
test:
strategy:
matrix:
go-version: [1.19.x, 1.20.x]
os: [ubuntu-latest, macos-latest, windows-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/setup-go@v3
with:
go-version: ${{ matrix.go-version }}
- uses: actions/checkout@v3
- run: go test ./...
14 changes: 13 additions & 1 deletion README.md
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
[![Go Reference](https://pkg.go.dev/badge/github.com/bitfield/script.svg)](https://pkg.go.dev/github.com/bitfield/script)[![Go Report Card](https://goreportcard.com/badge/github.com/bitfield/script)](https://goreportcard.com/report/github.com/bitfield/script)[![Mentioned in Awesome Go](https://awesome.re/mentioned-badge-flat.svg)](https://github.com/avelino/awesome-go)[![CircleCI](https://circleci.com/gh/bitfield/script.svg?style=svg)](https://circleci.com/gh/bitfield/script)
[![Go Reference](https://pkg.go.dev/badge/github.com/bitfield/script.svg)](https://pkg.go.dev/github.com/bitfield/script)
[![Go Report Card](https://goreportcard.com/badge/github.com/bitfield/script)](https://goreportcard.com/report/github.com/bitfield/script)
[![Mentioned in Awesome Go](https://awesome.re/mentioned-badge-flat.svg)](https://github.com/avelino/awesome-go)
![Tests](https://github.com/bitfield/script/actions/workflows/test.yml/badge.svg)

```go
import "github.com/bitfield/script"
@@ -45,6 +48,7 @@ If you're already familiar with shell scripting and the Unix toolset, here is a
| `sed` | [`Replace`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Replace) / [`ReplaceRegexp`](https://pkg.go.dev/github.com/bitfield/script#Pipe.ReplaceRegexp) |
| `sha256sum` | [`SHA256Sum`](https://pkg.go.dev/github.com/bitfield/script#Pipe.SHA256Sum) / [`SHA256Sums`](https://pkg.go.dev/github.com/bitfield/script#Pipe.SHA256Sums) |
| `tail` | [`Last`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Last) |
| `tee` | [`Tee`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Tee) |
| `uniq -c` | [`Freq`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Freq) |
| `wc -l` | [`CountLines`](https://pkg.go.dev/github.com/bitfield/script#Pipe.CountLines) |
| `xargs` | [`ExecForEach`](https://pkg.go.dev/github.com/bitfield/script#Pipe.ExecForEach) |
@@ -99,6 +103,12 @@ What's that? You want to append that output to a file instead of printing it to
script.Args().Concat().Match("Error").First(10).AppendFile("/var/log/errors.txt")
```

And if we'd like to send the output to the terminal *as well as* to the file, we can do that:

```go
script.Echo("data").Tee().AppendFile("data.txt")
```

We're not limited to getting data only from files or standard input. We can get it from HTTP requests too:

```go
@@ -301,6 +311,7 @@ Filters are methods on an existing pipe that also return a pipe, allowing you to
| [`Replace`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Replace) | matching text replaced with given string |
| [`ReplaceRegexp`](https://pkg.go.dev/github.com/bitfield/script#Pipe.ReplaceRegexp) | matching text replaced with given string |
| [`SHA256Sums`](https://pkg.go.dev/github.com/bitfield/script#Pipe.SHA256Sums) | SHA-256 hashes of each listed file |
| [`Tee`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Tee) | input copied to supplied writers |

Note that filters run concurrently, rather than producing nothing until each stage has fully read its input. This is convenient for executing long-running comands, for example. If you do need to wait for the pipeline to complete, call [`Wait`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Wait).

@@ -325,6 +336,7 @@ Sinks are methods that return some data from a pipe, ending the pipeline and ext

| Version | New |
| ----------- | ------- |
| v0.22.0 | [`Tee`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Tee), [`WithStderr`](https://pkg.go.dev/github.com/bitfield/script#Pipe.WithStderr) |
| v0.21.0 | HTTP support: [`Do`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Do), [`Get`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Get), [`Post`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Post) |
| v0.20.0 | [`JQ`](https://pkg.go.dev/github.com/bitfield/script#Pipe.JQ) |

8 changes: 4 additions & 4 deletions go.mod
View file
Original file line number Diff line number Diff line change
@@ -3,8 +3,8 @@ module github.com/bitfield/script
go 1.12

require (
bitbucket.org/creachadair/shell v0.0.7
github.com/google/go-cmp v0.5.7
github.com/itchyny/gojq v0.12.7
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 // indirect
github.com/google/go-cmp v0.5.9
github.com/itchyny/gojq v0.12.12
github.com/rogpeppe/go-internal v1.10.0
mvdan.cc/sh/v3 v3.6.0
)
48 changes: 32 additions & 16 deletions go.sum
View file
Original file line number Diff line number Diff line change
@@ -1,19 +1,35 @@
bitbucket.org/creachadair/shell v0.0.7 h1:Z96pB6DkSb7F3Y3BBnJeOZH2gazyMTWlvecSD4vDqfk=
bitbucket.org/creachadair/shell v0.0.7/go.mod h1:oqtXSSvSYr4624lnnabXHaBsYW6RD80caLi2b3hJk0U=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/creack/pty v1.1.18/go.mod h1:MOBLtS5ELjhRRrroQr9kyvTxUAFNvYEK993ew/Vr4O4=
github.com/frankban/quicktest v1.14.4 h1:g2rn0vABPOOXmZUj+vbmUp0lPoXEMuhTpIluN0XL9UY=
github.com/frankban/quicktest v1.14.4/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.7 h1:81/ik6ipDQS2aGcBfIN5dHDB36BwrStyeAQquSYCV4o=
github.com/google/go-cmp v0.5.7/go.mod h1:n+brtR0CgQNWTVd5ZUFpTBC8YFBDLK/h/bpaJ8/DtOE=
github.com/itchyny/gojq v0.12.7 h1:hYPTpeWfrJ1OT+2j6cvBScbhl0TkdwGM4bc66onUSOQ=
github.com/itchyny/gojq v0.12.7/go.mod h1:ZdvNHVlzPgUf8pgjnuDTmGfHA/21KoutQUJ3An/xNuw=
github.com/itchyny/timefmt-go v0.1.3 h1:7M3LGVDsqcd0VZH2U+x393obrzZisp7C0uEe921iRkU=
github.com/itchyny/timefmt-go v0.1.3/go.mod h1:0osSSCQSASBJMsIZnhAaF1C2fCBTJZXrnj37mG8/c+A=
github.com/mattn/go-isatty v0.0.14/go.mod h1:7GGIvUiUoEMVVmxf/4nioHXj79iQHKdU27kJ6hsGG94=
github.com/mattn/go-runewidth v0.0.9/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI=
golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220227234510-4e6760a101f9/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38=
github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/renameio/v2 v2.0.0/go.mod h1:BtmJXm5YlszgC+TD4HOEEUFgkJP3nLxehU6hfe7jRt4=
github.com/itchyny/gojq v0.12.12 h1:x+xGI9BXqKoJQZkr95ibpe3cdrTbY8D9lonrK433rcA=
github.com/itchyny/gojq v0.12.12/go.mod h1:j+3sVkjxwd7A7Z5jrbKibgOLn0ZfLWkV+Awxr/pyzJE=
github.com/itchyny/timefmt-go v0.1.5 h1:G0INE2la8S6ru/ZI5JecgyzbbJNs5lG1RcBqa7Jm6GE=
github.com/itchyny/timefmt-go v0.1.5/go.mod h1:nEP7L+2YmAbT2kZ2HfSs1d8Xtw9LY8D2stDBckWakZ8=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/mattn/go-isatty v0.0.17/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
github.com/mattn/go-runewidth v0.0.14/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA=
github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
github.com/rivo/uniseg v0.4.4/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=
github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs=
github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ=
github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.3.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.3.0/go.mod h1:q750SLmJuPmVoN1blW3UFBPREJfb1KmY3vwxfr+nFDA=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 h1:go1bK/D/BFZV2I8cIQd1NKEZ+0owSTG1fDTci4IqFcE=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
mvdan.cc/editorconfig v0.2.0/go.mod h1:lvnnD3BNdBYkhq+B4uBuFFKatfp02eB6HixDvEz91C0=
mvdan.cc/sh/v3 v3.6.0 h1:gtva4EXJ0dFNvl5bHjcUEvws+KRcDslT8VKheTYkbGU=
mvdan.cc/sh/v3 v3.6.0/go.mod h1:U4mhtBLZ32iWhif5/lD+ygy1zrgaQhUu+XFy7C8+TTA=
Loading