Skip to content

Commit

Permalink
rebase (#1)
Browse files Browse the repository at this point in the history
* docs(Dockerfile): Fix example of usage on Dockerfile (gitleaks#545)

* Closes gitleaks#544

Co-authored-by: mercuriete <mercuriete@gmail.com>

* Pypi GitHub rules (gitleaks#546)

* adding pypi and basic azure rules

* Adding new github format

* splitting out github rules and retabbing

* Update README.md

* Update README.md

* Docker-based pre-commit configuration example (gitleaks#551)

* Docker-based pre-commit configuration example

* Docker-based pre-commit configuration example

* Bug 553 global allowlist (gitleaks#554)

* add global file check

* rm whitespace

* Add pre-commit support (gitleaks#552)

* Add pre-commit support

* Update README.md

* Removed unnecessary backslashes

* Include `offenderEntropy` in the JSON output (gitleaks#549)

* Pass the entropy data back to the Leak struct

Do this to make it easier to tune entropy checks and make decisions in
systems consuming the output.

~ B'ezrat Hashem ~

* Return negative number when entropy not checked

That way you can tell the difference between not checking or an
actual entropy level of 0

~ B'ezrat Hashem ~

* Make sure to handle range checks properly

Make sure to show when something had an entropy returned but was outside
range, or didn't have a hit at all, etc...

~ B'zrat Hashem ~

* Add a few doc strings

Follow the project's conventiona add a comment above the methods

~ B'ezrat Hashem ~

* Update tests and get them to pass

~ B'ezrat Hashem ~

* Remove checked in `.got` files

~ B'ezrat Hashem ~

* Add `*.got` to the `.gitignore`

Make sure the test output files aren't checked-in

~ B'ezrat Hashem ~

* Bump version for pre-commit

* Update README.md

* SSH auth: Add custom username support (gitleaks#536)

* Add support for custom username when using SSH auth
Previously, the user "git" was hard-coded
This commit also adds support for ssh:// URLs

* Add comment to explain username parsing

* Fix example in leaky-repo.toml (gitleaks#559)

Signed-off-by: Norman Ziegner <norman.ziegner@ufz.de>

* Fix issue gitleaks#523 (gitleaks#561)

* updating documentation regarding issue gitleaks#523

* improving README.md file

* Update README.md

* Update README.md

* Update type in config example (gitleaks#597)

* Remove --commit-from and --commit-to from docs (gitleaks#605)

* fix(git-symlink-commits): fix handling symbolic links in a git repository

* Fix default twitter rules (gitleaks#614)

Regexps for default Twitter rules ("Twitter Secret Key" and "Twitter Client
ID") have a small flaw that make the default configuration vulnerable to
some false-positives.

I believe these rules should detect the cases like (SOME_CLIENT_ID should
be longer):
```
"twitter_client_id": "SOME_CLIENT_ID"
```

However, currently the twitter rules also detect the false positives for the
cases like:
```
someObj := twitter.NewObjectWithALongName()
config.Twitter.DomainAccessToken
```

I'm trying to address this issue the similar way it's done for facebook client
ids and AWS secret keys, where the secret is expected to be quoted.

* Updated RuleId in Sarif (gitleaks#613)

* fix(git-symlink-commits): fix handling symbolic links in a git repository (gitleaks#612)

* Update alpine, use gitleaks user instead of root (gitleaks#615)

* Updating alpine, use gitleaks user instead of root

* remove comments

* use embed pkg for default config, update deps (gitleaks#616)

* bump golang docker version

* GitHub test (gitleaks#617)

* drop travis

* remove travis

* rename, use example

* typo

* different syntax

* rename to test

* split test and build into two jobs

* add gosec job

* drop gosec for now

* Create maintenance.md

* Fix premature exit for nogit scans, limit goroutines (gitleaks#619)

* fix premature exit on nogit scan, actually limit concurreny for nogit

* removing files scanned log

* fix(git-symlink-files): fix handling symbolic links in unstaged changes

* Update README.md

* No longer generate empty reports (gitleaks#577)

* return nil when no leaks found and lowercase report format option (gitleaks#3)

* lowercase format options for consistency

* lowercase report format description

Co-authored-by: eddie-northcutt-wfp0 <eddie.northcutt.wfp0@statefarm.com>

* Update FUNDING.yml

* update documentation

* Create README.md

* Fix README link

Fix up readme link

* Added a test for fixed Google and Square config

* Simplify tests and continue on object not found errors (gitleaks#633)

* adding files

* init better tests

* add basic .git

* more tests and test data

* nogit, unstaged tests

* removing unused testdata

* adding empty testdata

* rm gitmodules

* fixing tests

* remove with config

* adding with_config

* removing old test_data, updating config tests

* removing hooks in testdata repos

* Fixed typo in Readme.md

Fixed description of main example rule

* fix: fix the multiple scan executions from pre-commit hook (gitleaks#649)

Co-authored-by: dustin <dustinshimono@gmail.com>

* Add clarification in README.md that Go 1.16+ required. (gitleaks#651)

* fix: fix the multiple scan executions from pre-commit hook

* docs: add clarification note about Go version required\

Ref 646

Co-authored-by: dustin <dustinshimono@gmail.com>

* updating documentation on how to build the docker image

* Bump alpine to v3.14.2

This alpine release includes fixes for openssl CVE-2021-3711 and CVE-2021-3712

* update readme

* build: add apple m1 support

Signed-off-by: Rui Chen <rui@chenrui.dev>

* Fixed bug in extractLine function which was returning incorrect line # when the line contains multiple vuln types

* chore: lint code

* Added EOL to sample file

* Try to revert index file

* Made changes to add support for cross-platform (os) testing as tests would pass on MacOS but no Windows

* Had to re-add testdata/ folder to project in order to get scan tests passing on Windows

* Fix for Windows OS

* ADD_installation_from_bin

ADD_installation_from_bin

* fix: typo in readme

* Update README.md

* Update regex for AWS secret key

* Fix tests with existing AWS secret keys

* Adjust AWS secret key rule

* fix possible typo in README.md

* Introducing v8.0.0 changes (gitleaks#701)

* Introducing v8.0.0 changes

* hardcode username to avoid redaction in action

* actually use github.actor

* remove git fetch tag from docker build process, already available

* remove sorting tags

* actually we gonna use git describe for docker build

* override version in goreleaser

* trying again

* Update README.md

* Update README.md

* removing --simplify-merges and --show-pulls (gitleaks#707)

* remove --show-pulls from git log

* adding logic to ignore gitleaks config during scans (gitleaks#710)

* Update sarif.go (gitleaks#713)

* Update sarif.go

Provide correct version information for Sarif Reports

* Update test

Set version to correct Sarif version

* Commit debug log (gitleaks#716)

* adding debug log for number of commits scanned

* change debug text

* readd commitsha

* add global regex check (gitleaks#717)

* remove generic api key from default gitleaks config (gitleaks#719)

* Update README.md

* use exit code 126 on 'unknown-flag' errors (gitleaks#723)

* Update README.md

* lil hack to avoid scans 'finishing' when git errors are present (gitleaks#726)

* Update README.md

* Update README.md

* bump go-gitdiff (gitleaks#731)

* fix typo in config help

* fix typo in readme

* introducing secretGroup, the best group (gitleaks#734)

* working on deduping

* my eyes... oh god my poor eyes

* more readme

* more readme

* more readme

* more readme and formatting

* fix: format dates in log in a portable way (gitleaks#735)

The output of time.String() depends on the runtime environment and should only be used for debugging.

This commit ensures that a well-defined UTC time is written to the report.

* Update pre-commit step to run gitleaks checks (gitleaks#729)

* Fix pre-commit config.

* Debug output

* No debug and use redacted.

* Typo

* Add back staged.

* Update .pre-commit-hooks.yaml

* Update .pre-commit-hooks.yaml

* ignore gitleaks.toml by default

* fix deduplication issue caused by clobbered findings (gitleaks#742)

* fix deduplication issue caused by clobbered findings

* fix index

* remove indexing, slow is better than wrong

* fix off by one line number for --no-git

* remove writing default config, introduce GITLEAKS_CONFIG (gitleaks#746)

* remove writing default config, introduce GITLEAKS_CONFIG

* setting report format default to json, update readme

* add pre-commit instructions (gitleaks#749)

* do not fail on git rename warning (gitleaks#750)

* bump go-gitdiff

* Update README.md

* Update README.md

* better asciinema

* Update README.md

* fix regexp for aws_key and slack_webhook (gitleaks#754)

* Adding Tines sponsorship to readme

* fixing eof location bug (gitleaks#756)

* stricter ionic regex for less fps (gitleaks#757)

* write a report regardless if leaks are present (gitleaks#758)

* Adding Typeform to sponsorships

* limit goroutines on file scanning to avoid pegging them cores (gitleaks#759)

* always write sarif results

* limit number of goroutines for historic scanning as well (gitleaks#761)

* remove godoc text filtering (gitleaks#763)

* Fix typos in README.md (gitleaks#780)

comand ==> command
awsome ==> awesome
precendence ==> precedence

* Sarif results with empty rules now represents as [] instead of null/nil (gitleaks#786)

* Fix vendor name casing, Flutterwave typo (gitleaks#785)

* Fix: Typo in LinkedIn id (gitleaks#789)

* fixing segfault when using a rule with only a path (gitleaks#791)

* fixing segfault when using a rule with only a path

* DRYing

* allow non-last-element secret groups (gitleaks#792)

* build: updates for go1.17 (gitleaks#769)

* build: remove `GO111MODULE` as it turned on by default

Signed-off-by: Rui Chen <rui@chenrui.dev>

* build: update for go1.17

* run test

Signed-off-by: Rui Chen <rui@chenrui.dev>

* lint: remove unused code

* ignore k8s apiVersion in generic-api-key pattern (gitleaks#760)

* GitLab pats may contain underscores as well as dashes (gitleaks#794)

* gitlab pats may contain underscores as well as dashes

* include testdata index binary

Co-authored-by: Greg Johnson (codeEmitter) <gjohnson@gitlab.com>

* adding go mod/sum to ignore (gitleaks#797)

* Escape - character in regex character groups (gitleaks#802)

* fix char escape

* add test

* fix verbosity in make test

* Refactor `detect`, add `entropy` to all findings (gitleaks#804)

Refactor `detect`, add `entropy` to all findings

* Stop words (gitleaks#808)

* use regex for stopwords

* fix up regex

* rm stopwords

* Allow tag (gitleaks#809)

* gitleaks:allow signature

* readd all tests

* fixing tests

* Update README.md

* fixing a location off by one edge case for --no-git (gitleaks#812)

* detect: skip binary files with --no-git (gitleaks#810)

* remove stopwords from global allowlist

* use official docker image as pre-commit hook (gitleaks#818)

* use official docker image as pre-commit hook

* Update .pre-commit-hooks.yaml

* Update .pre-commit-hooks.yaml

* Update .pre-commit-hooks.yaml

* Update .pre-commit-hooks.yaml

* Update .pre-commit-hooks.yaml

* skip content checks for path only rules

* doc gitleaks-docker pre-commit hook (gitleaks#819)

relates to gitleaks#818

* Keyword (gitleaks#825)

* wip keywords optimization

* update readme

* limit concurrency to 4

* update readme

* normalize keyword check (gitleaks#830)

* fix ghcr.io typo in README.md (gitleaks#835)

* Standardize/alphabetize rules, add cmd/generate/config package (gitleaks#840)

* Update detect.go (gitleaks#839)

* optimize keywords (gitleaks#841)

* optimize keywords

* use defaults for concurrency again

* maybe fix out of bounds (gitleaks#843)

* Generate tps (gitleaks#845)

* WIP

* simplify tp generation

* unpin docker version in pre-commit hook (gitleaks#832)

eliminating inconsistency. e.g. for version 8.6.1, pre-commit hooks points to outdated version https://github.com/zricethezav/gitleaks/blob/c33ee3f25215635c0afbb210672779e7efb6f1d2/.pre-commit-hooks.yaml#L10

* fix EOL in secret suffix (gitleaks#847)

* fix EOL in secret suffix

* allow quoted key value syntax

* Update dockerfile (gitleaks#848)

* bump alpine and add default safe dir for git

* comment out safe dir fix

* update deps

* adding stopwords (gitleaks#849)

* adding stopwords

* format readme, update default config

* adding a ton of stopwords to the generic rule only as that is the loudest rule (gitleaks#851)

* nasty little bug (gitleaks#853)

* Removing private keyword from private key rule (gitleaks#858)

* fix no-git bug (gitleaks#859)

* Update README.md

* Adding JIT Security messages

* Update README.md

* Improve PlanetScale token detection (gitleaks#874)

This improves the PlanetScale token detection. It add some flexibility
in length. There is no guarantee that the length is always 43 characters
(in fact, it's very likely to change a bit soon).

Additionally, it adds support for detecting oauth tokens as well.

* feat: add algolia key support (gitleaks#866)

* feat: add algolia key support

* feat: add algolia key to generator

* chore: update algolia regex to match gen

* updating generic regex and algoia regex (gitleaks#875)

* ignore end line when comparing generic rules (gitleaks#879)

* Fix generic-api-key detected erroneously (zricethezav#877) (gitleaks#878)

* add combo to stopwords, update cmd/generate

* Limit newlines regex (gitleaks#881)

* improve regex to reduce fps with newlines

* remove version from regex and rely on stopwords instead

* add false positive for validation

* fix git unsafe directory (gitleaks#883)

* fix git unsafe directory

fixes gitleaks#846

* Update Dockerfile

Co-authored-by: x <>

* add link to gitleaks.io

* Create gitleaks.yml (gitleaks#884)

* Add gitleaks badge

* Update README.md

* user accounts don't need gitleaks license

* Update README.md

* Update gitleaks.yml

* Update README.md

* Fix duplicate TOML Rules and IDs  (gitleaks#889)

* Remove duplicate rule

* Fix duplicate rule IDs

Co-authored-by: Craig Smith <5344211-craigmsmith@users.noreply.gitlab.com>

* maintain parity with recent changes... need to create rule contributing guidelines (gitleaks#891)

* Update generate (gitleaks#892)

* maintain parity with recent changes... need to create rule contributing guidelines

* missed on, ensure uniqueness in rule-id

* Lint python commit script to satisfy PEP8 (gitleaks#893)

* contributing guidelines first draft (gitleaks#895)

* contributing guidelines first draft

* update links, add readme note

* Adding a bunch of new rules, update allowlist to include node_modules… (gitleaks#896)

* Adding a bunch of new rules, update allowlist to include node_modules and vendor folders, extend helper config functions

* use func instead of function in stopwords

* Remove ssn allowlist (gitleaks#898)

* Adding a bunch of new rules, update allowlist to include node_modules and vendor folders, extend helper config functions

* remove ssns from allowlist since default config does not detect ssn

* Fixes accidental type typos while translating rules from validation spreadsheet, adds bittrex rule

* adding airtable and adafruit (gitleaks#902)

* Fix Plaid, add Plaid access token (gitleaks#903)

* adding airtable and adafruit

* Fix plaid, add plaid-access-token

* Adding okta, codecov, zendesk, and updating Atlassian's rule to include `jira` keyword (gitleaks#904)

* okta

* adding codecov

* add jira to atlassians keywords

* adding zendesk

* Fix id and description for twitter tokens (gitleaks#905)

Co-authored-by: Craig Smith <5344211-craigmsmith@users.noreply.gitlab.com>

* adding travis ci

* capitilze twitter description

* update twitter rule generation description and id

* Add multi platform build (gitleaks#897)

Signed-off-by: Romain Barissat <romain-noreply@barissat.com>

* Fix proper names capitalization (gitleaks#907)

Co-authored-by: Craig Smith <5344211-craigmsmith@users.noreply.gitlab.com>

* adding access to generic rule keywords and identifiers

* Update README.md

* Update README.md

* Update README.md

* Add fix for issue gitleaks#915 (gitleaks#916)

Co-authored-by: André Breuer <andre.breuer@ndaal.eu>

* Feature: Adding the ability to extend configuration files (gitleaks#926)

* init

* working on default and path config extensions

* adding trace log level, consolidating some code

* cleaning things up, updating generate package

* fix tests

* formatting

* adding tests for extend

* extend not extends

* formatting

* only allow usedefault or path to be set

* update readme

* add note about allowlists

* more readme, expand env var for path

* actually dont support env var. ez attack

* add url for config

* update readme

* Feature/add sidekiq rules (gitleaks#933)

* Add sidekiq rules

* Added two new rules for sidekiq
* Other: Add keywords to square rules per Zach's instructions

* Validate now works, but test suite is failing

* Tests are now passing

* Add Sidekiq Rules: Ran go fmt

* * After resolving conflicts, had to rerun the rule generator to add back the semicolon char
* After running tests, had to fix one line in testdata/expected/report/sarif_simple.sarif

* * Added keywords to simple.toml for sidekiq-sensitive-url so that the rule matches what is in gitleaks.toml

Co-authored-by: Andrew Weiner <aweiner@frontrush.com>

* Add new rules for vault tokens (gitleaks#919)

* add new rules for vault tokens

* Configure max length for vault rules

* gitleaks allow docs (gitleaks#941)

* gitleaks allow docs

* reorder

* bump golang test version (gitleaks#942)

* add jwt rule (gitleaks#943)

* add jwt support

* ignore sample secrets

* Feat/ignore finding (gitleaks#938)

* add two test findings to gitleaksignore

* Explicit fingerprint (gitleaks#944)

* Update README.md

* safe file checking (gitleaks#946)

* Feat/add fingerprint no git (gitleaks#952)

* no-git support fingerprint support

* updating gitleaksignore w/ no-git false positives

* fix test

* draft: bump gitdiff, add git.Err state, better log messages (gitleaks#954)

* bump gitdiff, add git.Err state, better log messages

* remove cmd.Start

* forgot to start...

* add prefect and readme rules (gitleaks#961)

* Add grafana tokens rules (gitleaks#959)

* Add grafana tokens rules

* Adding upper bound limits to Grafana tokens

* ignore empty files (gitleaks#965)

* Update version in readme file (gitleaks#972)

* Pretty output (gitleaks#973)

pretty output

* add fingerprint to output

* update gitleaksignore

* refactor: more precise rule for private keys (gitleaks#930)

* refactor: more precise rule for private keys

The current regex didn't match PGP private keys anymore, since they start with `BEGIN PGP PRIVATE KEY BLOCK` and the `BLOCK` never matched for the existing regex. I've made that part optional so that all strings matching for the current regex will still match for the new regex.

* refactor: more precise rule for private keys

Co-authored-by: Fabian F Groß <fabian.f.gross@deutschebahn.com>

* Add pre-commit autoupdate command to README.md (gitleaks#978)

* Add baseline (gitleaks#975)

* Add baseline

* Update doc, add error, move baseline to detect namespace, ignore findings instead of reactively filter them out

* Update detect/detect.go

Co-authored-by: Zachary Rice <zricezrice@gmail.com>

* Update IsNew function (no check on tags - omit finger print check)

* Update README.md

Co-authored-by: Zachary Rice <zricezrice@gmail.com>

* Update examples in readme to make it ensure it's clear that a baseline is indeed a gitleaks report

* Fix test - updated tags doesn't make a finding new

* Add missing err assignment

* Allow scanner to continue without baseline if file is malformed

* Fix typo in comment

* Fix control flow err. (Real life testing)

* Fix wording

* Auto-ignore baseline path

* add rule for microsoft teams webhooks (gitleaks#970)

* Issue gitleaks#980: Add support for Telegram Bot API Token (gitleaks#981)

* Issue gitleaks#980: Add support for Telegram Bot API Token

* Replace test with random bot_id length by tests with fixed one.
Add tests for the corner cases.

Co-authored-by: Alex Goncharov <b4bay@users.noreply.github.com>

* Adding quiet mode to silence banner (gitleaks#852)

* Adding quiet mode to silence banner

* Changing flag description. Adding flag to README

* Updating argument name

* updating variable name to aline with argument

* fixing readme spacing

* Fixing variable name

* Update README.md

* Update .gitleaksignore

* Update README.md

* Minor cleanup to error handling and logging (gitleaks#985)

* silence warning about unchecked errors

* go-fmt change to add newline

* Zerolog requires you to always call .Msg()

When logging with zerolog, you need to always end with .Msg(), even if
you just pass an empty string.

If you read the README on https://github.com/rs/zerolog, they write:

> It is very important to note that when using the zerolog
> chaining API, as shown above (log.Info().Msg("hello world"), the
> chain must have either the Msg or Msgf method call. If you
> forget to add either of these, the log will not occur and there
> is no compile time error to alert you of this.

* Create empty slice without literal

* Fix variable / package name collision with literal

instead of having a variable named "config", which collides with the
package name "config", just pass a literal config.Config{} struct to the
function

* Replace call to deprecated ioutil.ReadAll()

Use io.ReadAll() instead

* Check error when closing jsonFile

Make it a warning and log error

* Upgrade go version to 1.19 (gitleaks#987)

* upgrade go version to 1.19

* upgrade go version to 1.19 in dockerfile and test.yml

* Detect Slack Workflow Webhook URLs (gitleaks#989)

* Output number of commits at info-level. (gitleaks#991)

* Exclude dacpac refactorlogs (gitleaks#990)

Dacpac refactorlogs contains Key's that are false positives. This commit excludes those files.

* Create USERS.md

* Update USERS.md (gitleaks#996)

* docs: added goreleaser to user list (gitleaks#997)

Thanks for gitleaks, its amazing to have it in the pipeline so I can have some extra peace of mind!

* docs: add Trendyol to users (gitleaks#998)

* Add detection rules for DigitalOcean tokens (gitleaks#1002)

* Add detection rules for DigitalOcean tokens

* go fmt correction

Signed-off-by: Norman Ziegner <norman.ziegner@ufz.de>
Signed-off-by: Rui Chen <rui@chenrui.dev>
Signed-off-by: Romain Barissat <romain-noreply@barissat.com>
Co-authored-by: mercuriete <mercuriete@yahoo.es>
Co-authored-by: mercuriete <mercuriete@gmail.com>
Co-authored-by: Zachary Rice <zricer@protonmail.com>
Co-authored-by: Pavel Shklovsky <pablo.shklovsky@gmail.com>
Co-authored-by: Pavel Shklovsky <pavel.shklovsky@dynamicyield.com>
Co-authored-by: AmitHofree <amit.hofree@gmail.com>
Co-authored-by: bplaxco <bplaxco@lupnix.org>
Co-authored-by: Tomy Guichard <tomy2e@live.fr>
Co-authored-by: Norman Ziegner <normo157@gmail.com>
Co-authored-by: Ramon <w0rmr1d3r@users.noreply.github.com>
Co-authored-by: Tomasz Wierzchowski <twierzchowski@gmx.com>
Co-authored-by: Emma Sax <emma.sax4@gmail.com>
Co-authored-by: Ido Markovitz <Ido.Markovitz@dynamicyield.com>
Co-authored-by: Ivan Kalita <72927991+ivankalitaonefootball@users.noreply.github.com>
Co-authored-by: rotem-cider <78903577+rotem-cider@users.noreply.github.com>
Co-authored-by: Ido-DY <85484711+Ido-DY@users.noreply.github.com>
Co-authored-by: Eddie Austin <84474478+eddie-austin@users.noreply.github.com>
Co-authored-by: eddie-northcutt-wfp0 <eddie.northcutt.wfp0@statefarm.com>
Co-authored-by: amith-legit <amit@legitsecurity.com>
Co-authored-by: Eli Schleifer <1265982+EliSchleifer@users.noreply.github.com>
Co-authored-by: b-abderrahmane <b-abderrahmane@outlook.com>
Co-authored-by: Dustin Shimono <5289+dustinsand@users.noreply.github.com>
Co-authored-by: dustin <dustinshimono@gmail.com>
Co-authored-by: Nick Russler <nick.russler@gmail.com>
Co-authored-by: Rui Chen <rui@chenrui.dev>
Co-authored-by: Matthew E. Grahlman <grahlmanmatthew@gmail.com>
Co-authored-by: dani <danielorihuelarodriguez@gmail.com>
Co-authored-by: Lucas Alcântara <lucasapd@gmail.com>
Co-authored-by: Fabian Kirschner <fabian.kirschner@gmail.com>
Co-authored-by: Dirk Pahl <dirk.pahl@deutschebahn.com>
Co-authored-by: M.Hassan Yousaf <olevanmughal@gmail.com>
Co-authored-by: Joost Voskuil <joost@foxhole.nl>
Co-authored-by: Florian Greinacher <florian@greinacher.de>
Co-authored-by: Ben Randall <veleek@gmail.com>
Co-authored-by: Andrzej Amghar <amandre@users.noreply.github.com>
Co-authored-by: Zachary Rice <zricezrice@gmail.com>
Co-authored-by: Greg Myers <myersg86@gmail.com>
Co-authored-by: Chris Wolf <chriswolfdesign@gmail.com>
Co-authored-by: Ian Mckay <contact@ianmckay.com.au>
Co-authored-by: Luca Regne <luca.rsr.luca@gmail.com>
Co-authored-by: Greg Johnson <gregjohnson2020@gmail.com>
Co-authored-by: Greg Johnson (codeEmitter) <gjohnson@gitlab.com>
Co-authored-by: jetexe <jetexe@users.noreply.github.com>
Co-authored-by: Isaac Dawson <60455448+idawson-gl@users.noreply.github.com>
Co-authored-by: Adam Shannon <adamkshannon@gmail.com>
Co-authored-by: foolioo <28758375+foolioo@users.noreply.github.com>
Co-authored-by: aeongdesu <osu@kakao.com>
Co-authored-by: 0xn3va <47100179+0xn3va@users.noreply.github.com>
Co-authored-by: Dirkjan Bussink <d.bussink@gmail.com>
Co-authored-by: SimonGurney <simongurney@outlook.com>
Co-authored-by: Don C. Bigler <94854301+dcb-imvaria@users.noreply.github.com>
Co-authored-by: Craig Smith <craigmarksmith@users.noreply.github.com>
Co-authored-by: Craig Smith <5344211-craigmsmith@users.noreply.gitlab.com>
Co-authored-by: Alex <52292902+alexrudd2@users.noreply.github.com>
Co-authored-by: Romain Barissat <romain-noreply@barissat.com>
Co-authored-by: anotherbridge <46713015+anotherbridge@users.noreply.github.com>
Co-authored-by: André Breuer <andre.breuer@ndaal.eu>
Co-authored-by: Andrew <weineran@users.noreply.github.com>
Co-authored-by: Andrew Weiner <aweiner@frontrush.com>
Co-authored-by: Malte Morgenstern <65773564+maltemorgenstern@users.noreply.github.com>
Co-authored-by: jmatosgrafana <100539023+jmatosgrafana@users.noreply.github.com>
Co-authored-by: Akash Chandwani <3483277+akashchandwani@users.noreply.github.com>
Co-authored-by: very-doge-wow <95224950+very-doge-wow@users.noreply.github.com>
Co-authored-by: Fabian F Groß <fabian.f.gross@deutschebahn.com>
Co-authored-by: Gawan Schroeder <34353307+gawansch@users.noreply.github.com>
Co-authored-by: Alex Goncharov <49787265+b4bay@users.noreply.github.com>
Co-authored-by: Alex Goncharov <b4bay@users.noreply.github.com>
Co-authored-by: Zane Durkin <zanedurkin@gmail.com>
Co-authored-by: Michael Jarvis <5694899+mojotx@users.noreply.github.com>
Co-authored-by: naoki kuroda <68233204+nnnkkk7@users.noreply.github.com>
Co-authored-by: Becojo <Becojo@users.noreply.github.com>
Co-authored-by: alexgit2k <alexgit2k@users.noreply.github.com>
Co-authored-by: Carlos Alexandro Becker <caarlos0@users.noreply.github.com>
Co-authored-by: Furkan Türkal <furkan.turkal@hotmail.com>
Co-authored-by: Michael Henriksen <mchnrksn@gmail.com>
  • Loading branch information
Show file tree
Hide file tree
Showing 640 changed files with 14,715 additions and 6,476 deletions.
1 change: 0 additions & 1 deletion .github/FUNDING.yml
@@ -1,4 +1,3 @@
# These are supported funding model platforms

github: [zricethezav]
custom: ["https://www.paypal.me/zricethezav"]
16 changes: 16 additions & 0 deletions .github/ISSUE_TEMPLATE/maintenance.md
@@ -0,0 +1,16 @@
---
name: Maintenance request
about: Suggest an idea for this project
title: ''
labels: enhancement
assignees: ''

---

**Describe the solution you'd like**
A clear and concise description of what you want to happen.

**Additional context**
Add any other context or screenshots about the feature request here.

cc @zricethezav
13 changes: 13 additions & 0 deletions .github/workflows/gitleaks.yml
@@ -0,0 +1,13 @@
name: gitleaks
on: [pull_request, push, workflow_dispatch]
jobs:
scan:
name: gitleaks
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- uses: gitleaks/gitleaks-action@v2
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
57 changes: 57 additions & 0 deletions .github/workflows/release.yml
@@ -0,0 +1,57 @@
name: Create and publish a Docker image

on:
release:
types: [published]

env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}

jobs:
build-and-push-image:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write

steps:
- name: Checkout repository
uses: actions/checkout@v2

- name: Set up QEMU
uses: docker/setup-qemu-action@8b122486cedac8393e77aa9734c3528886e4a1a8

- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@dc7b9719a96d48369863986a06765841d7ea23f6

- name: Log in to Docker Hub
uses: docker/login-action@49ed152c8eca782a232dede0303416e8f356c37b
with:
username: ${{ github.actor }}
password: ${{ secrets.DOCKER_PASSWORD }}

- name: Log in to the Container registry
uses: docker/login-action@49ed152c8eca782a232dede0303416e8f356c37b
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@98669ae865ea3cffbcbaa878cf57c20bbf1c6c38
with:
images: |
zricethezav/gitleaks
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
- name: Build and push Docker image
uses: docker/build-push-action@e551b19e49efd4e98792db7592c17c09b89db8d8
with:
platforms: linux/amd64,linux/arm64
context: .
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
26 changes: 26 additions & 0 deletions .github/workflows/test.yml
@@ -0,0 +1,26 @@
name: Test

on:
pull_request:
branches:
- "*"

jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2

- name: Set up Go
uses: actions/setup-go@v2
with:
go-version: 1.19

- name: Build
run: go build -v ./...

- name: Test
run: make test

- name: Validate Config
run: cd cmd/generate/config && go run main.go
7 changes: 7 additions & 0 deletions .gitignore
Expand Up @@ -6,8 +6,15 @@
*.dylib
*.DS_STORE
*.idea
*.got
gitleaks
build

# configs
.gitleaks.toml
cmd/generate/config/gitleaks.toml

# Test binary
*.out

dist/
744 changes: 744 additions & 0 deletions .gitleaksignore

Large diffs are not rendered by default.

29 changes: 29 additions & 0 deletions .goreleaser.yml
@@ -0,0 +1,29 @@
project_name: gitleaks

builds:
- main: main.go
binary: gitleaks
goos:
- darwin
- linux
- windows
goarch:
- amd64
- "386"
- arm
- arm64
goarm:
- "6"
- "7"
ldflags:
- -s -w -X=github.com/zricethezav/gitleaks/v8/cmd.Version={{.Version}}
archives:
- builds: [gitleaks]
format_overrides:
- goos: windows
format: zip
replacements:
amd64: x64
386: x32
release:
prerelease: true
11 changes: 11 additions & 0 deletions .pre-commit-hooks.yaml
@@ -0,0 +1,11 @@
- id: gitleaks
name: Detect hardcoded secrets
description: Detect hardcoded secrets using Gitleaks
entry: gitleaks protect --verbose --redact --staged
language: golang
pass_filenames: false
- id: gitleaks-docker
name: Detect hardcoded secrets
description: Detect hardcoded secrets using Gitleaks
entry: zricethezav/gitleaks protect --verbose --redact --staged
language: docker_image
8 changes: 0 additions & 8 deletions .travis.yml

This file was deleted.

103 changes: 103 additions & 0 deletions CONTRIBUTING.md
@@ -0,0 +1,103 @@
# Contribution guidelines

## General

### Issues

If you have a feature or bug fix you would like to contribute please check if
there are any open issues describing your proposed addition. If there are open
issues, make a comment stating you are working on fixing or implementing said
issue. If not, then please open an issue describing your addition. Make sure to
link your PR to an issue.

### Pull Requests

Fill out the template as best you can. Make sure your tests pass. If you see a
PR that isn't one you opened and want it introduced in the next release,
give it a :thumbsup: on the PR description.

## Adding new Gitleaks rules

If you want to add a new rule to the [default Gitleaks configuration](https://github.com/zricethezav/gitleaks/blob/master/config/gitleaks.toml) then follow these steps.

1. Create a `cmd/generate/config/rules/{provider}.go` file.
This file is used to generate a new Gitleaks rule.
Let's look at `beamer.go` for example. Comments have been added for context.

```golang
func Beamer() *config.Rule {
// Define Rule
r := config.Rule{
// Human redable description of the rule
Description: "Beamer API token",

// Unique ID for the rule
RuleID: "beamer-api-token",

// Regex capture group for the actual secret
SecretGroup: 1,


// Regex used for detecting secrets. See regex section below for more details
Regex: generateSemiGenericRegex([]string{"beamer"}, `b_[a-z0-9=_\-]{44}`),

// Keywords used for string matching on fragments (think of this as a prefilter)
Keywords: []string{"beamer"},
}

// validate
tps := []string{
generateSampleSecret("beamer", "b_"+secrets.NewSecret(alphaNumericExtended("44"))),
}
return validate(r, tps, nil)
}
```

Feel free to use this example as a template when writing new rules.
This file should be fairly self-explanatory except for a few items;
regex and secret generation. To help with maintence, _most_ rules should
be uniform. The functions,
[`generateSemiGenericRegex`](https://github.com/zricethezav/gitleaks/blob/master/cmd/generate/config/rules/rule.go#L31) and [`generateUniqueTokenRegex`](https://github.com/zricethezav/gitleaks/blob/master/cmd/generate/config/rules/rule.go#L44) will generate rules
that follow defined patterns.

The function signatures look like this:

```golang
func generateSemiGenericRegex(identifiers []string, secretRegex string) *regexp.Regexp

func generateUniqueTokenRegex(secretRegex string) *regexp.Regexp
```

`generateSemiGenericRegex` accepts a list of identifiers and a regex.
The list of identifiers _should_ match the list of `Keywords` in the rule
definition above. Both `identifiers` in the `generateSemiGenericRegex`
function _and_ `Keywords` act as filters for Gitleaks telling the program
"_at least one of these strings must be present to be considered a leak_"

`generateUniqueToken` just accepts a regex. If you are writing a rule for a
token that is unique enough not to require an identifier then you can use
this function. For example, Pulumi's API Token has the prefix `pul-` which is
unique enough to use `generateUniqueToken`. But something like Beamer's API
token that has a `b_` prefix is not unqiue enough to use `generateUniqueToken`,
so instead we use `generateSemiGenericRegex` and require a `beamer`
identifier is part of the rule.
If a token's prefix has more than `3` characters then you could
probably get away with using `generateUniqueToken`.

Last thing you'll want to hit before we move on from this file is the
validation part. You can use `generateSampleSecret` to create a secret for the
true positives (`tps` in the example above) used in `validate`.

1. Update `cmd/generate/config/main.go`. Add a line like
`configRules = append(configRules, rules.Beamer())` in `main()`. Try and keep
this alphabetically pretty please.

1. Change directories into `cmd/generate/config` and run `go run main.go`

```
cd cmd/generate/config && go run main.go
```

1. Check out your new rules in `config/gitleaks.toml` and see if everything looks good.

1. Open a PR
20 changes: 9 additions & 11 deletions Dockerfile
@@ -1,17 +1,15 @@
FROM golang:1.14.1 AS build
FROM golang:1.19 AS build
WORKDIR /go/src/github.com/zricethezav/gitleaks
ARG ldflags
COPY . .
RUN GO111MODULE=on CGO_ENABLED=0 go build -o bin/gitleaks -ldflags "-X="${ldflags} *.go
RUN VERSION=$(git describe --tags --abbrev=0) && \
CGO_ENABLED=0 go build -o bin/gitleaks -ldflags "-X="github.com/zricethezav/gitleaks/v8/cmd.Version=${VERSION}

FROM alpine:3.11
RUN apk add --no-cache bash git openssh
FROM alpine:3.16
RUN adduser -D gitleaks && \
apk add --no-cache bash git openssh-client
COPY --from=build /go/src/github.com/zricethezav/gitleaks/bin/* /usr/bin/
ENTRYPOINT ["gitleaks"]

# How to use me :
USER gitleaks

# docker build -t gitleaks .
# docker run --rm --name=gitleaks gitleaks --repo=https://github.com/zricethezav/gitleaks
RUN git config --global --add safe.directory '*'

# This will check for secrets in https://github.com/zricethezav/gitleaks
ENTRYPOINT ["gitleaks"]
50 changes: 12 additions & 38 deletions Makefile
@@ -1,52 +1,26 @@
.PHONY: test test-cover build release-builds
.PHONY: test test-cover

VERSION := `git fetch --tags && git tag | sort -V | tail -1`
PKG=github.com/zricethezav/gitleaks
LDFLAGS=-ldflags "-X=github.com/zricethezav/gitleaks/v6/version.Version=$(VERSION)"
_LDFLAGS="github.com/zricethezav/gitleaks/v6/version.Version=$(VERSION)"
VERSION := `git fetch --tags && git tag | sort -V | tail -1`
LDFLAGS=-ldflags "-X=github.com/zricethezav/gitleaks/v8/cmd.Version=$(VERSION)"
COVER=--cover --coverprofile=cover.out

test-cover:
go test ./... --race $(COVER) $(PKG) -v
go test -v ./... --race $(COVER) $(PKG)
go tool cover -html=cover.out

test:
go get golang.org/x/lint/golint
format:
go fmt ./...
go vet ./...
golint ./...
go test ./... --race $(PKG) -v

test-integration:
go test github.com/zricethezav/gitleaks/hosts -v -integration
test: format
go vet ./...
go test -v ./... --race $(PKG)

build:
go fmt ./...
golint ./...
build: format
go vet ./...
go mod tidy
go build $(LDFLAGS)

security-scan:
go get github.com/securego/gosec/cmd/gosec
gosec -no-fail ./...

release-builds:
rm -rf build
mkdir build
env GOOS="windows" GOARCH="amd64" go build -o "build/gitleaks-windows-amd64.exe" $(LDFLAGS)
env GOOS="windows" GOARCH="386" go build -o "build/gitleaks-windows-386.exe" $(LDFLAGS)
env GOOS="linux" GOARCH="amd64" go build -o "build/gitleaks-linux-amd64" $(LDFLAGS)
env GOOS="linux" GOARCH="arm" go build -o "build/gitleaks-linux-arm" $(LDFLAGS)
env GOOS="linux" GOARCH="mips" go build -o "build/gitleaks-linux-mips" $(LDFLAGS)
env GOOS="linux" GOARCH="mips" go build -o "build/gitleaks-linux-mips" $(LDFLAGS)
env GOOS="darwin" GOARCH="amd64" go build -o "build/gitleaks-darwin-amd64" $(LDFLAGS)

deploy:
@echo "$(DOCKER_PASSWORD)" | docker login -u "$(DOCKER_USERNAME)" --password-stdin
docker build --build-arg ldflags=$(_LDFLAGS) -f Dockerfile -t zricethezav/gitleaks:latest -t zricethezav/gitleaks:$(VERSION) .
echo "Pushing zricethezav/gitleaks:$(VERSION) and zricethezav/gitleaks:latest"
docker push zricethezav/gitleaks

dockerbuild:
docker build --build-arg ldflags=$(_LDFLAGS) -f Dockerfile -t zricethezav/gitleaks:latest -t zricethezav/gitleaks:$(VERSION) .
clean:
find . -type f -name '*.got.*' -delete
find . -type f -name '*.out' -delete

0 comments on commit 6ea64f5

Please sign in to comment.