FileNotFoundError in LineGetter.open_file

[jsoref]

We had a secret referencing a file in the secrets baseline, and then that file was deleted, the next time detect-secrets (v1.2.0) ran, it crashed with:

FileNotFoundError: [Errno 2] No such file or directory: ...

detect-secrets/detect_secrets/audit/common.py

Lines 146 to 148 in 70e6cf6

	def open_file(self) -> Iterator[NamedIO]:
	"""This is split up into a different function, so it can be overridden if necessary."""
	with open(self.filename) as f:

detect-secrets/detect_secrets/audit/common.py

Lines 156 to 158 in 70e6cf6

	with self.open_file() as f:
	lines = get_transformed_file(f, use_eager_transformers=self.use_eager_transformers)
	self._lines = self.raw_lines if not lines else lines

detect-secrets/detect_secrets/audit/common.py

Lines 82 to 90 in 70e6cf6

	while True:
	if secret.line_number:
	try:
	lines_to_scan = [line_getter.lines[secret.line_number - 1]]
	line_numbers = [secret.line_number - 1]
	except IndexError:
	raise SecretNotFoundOnSpecifiedLineError(secret.line_number)
	else:
	lines_to_scan = line_getter.lines

detect-secrets/detect_secrets/audit/report.py

Lines 27 to 43 in 70e6cf6

	def generate_report(
	baseline_file: str,
	class_to_print: SecretClassToPrint = None,
	line_getter_factory: Callable[[str], 'LineGetter'] = open_file,
	) -> List[Dict[str, Any]]:
	secrets: Dict[Tuple[str, str], Any] = {}
	for filename, secret in get_baseline_from_file(baseline_file):
	verified_result = VerifiedResult.from_secret(secret)
	if (
	class_to_print is not None and
	SecretClassToPrint.from_class(verified_result) != class_to_print
	):
	continue
	# Removal of the stored line number is required to force the complete file scanning to obtain all the secret occurrences. # noqa: E501
	secret.line_number = 0
	detections = get_raw_secrets_from_file(secret)

detect-secrets/detect_secrets/main.py

Lines 132 to 140 in 70e6cf6

	elif args.report:
	class_to_print = None
	if args.only_real:
	class_to_print = audit.report.SecretClassToPrint.REAL_SECRET
	elif args.only_false:
	class_to_print = audit.report.SecretClassToPrint.FALSE_POSITIVE
	print(
	json.dumps(
	audit.report.generate_report(args.filename[0], class_to_print),

[lorenzodb1]

Hi @jsoref, thanks for bringing up this issue to us. We will take a look at this issue in the next few days and keep you posted on it. Stay tuned!

[jpdakran]

@jsoref Hello. I just tested deleting a python file which had existing secrets in the baseline file. This ran fine and the secrets were deleted from the baseline file. Can you provide more information on the file you deleted?

[jpdakran]

@jsoref Can you please some more information regarding "the next time detect-secrets (v1.2.0) ran"? - Can you explain what command you are running when running detect-secrets? Is it pre-commit or CLI?

[jsoref]

We're using https://github.com/secret-scanner/action/ to wrap detect-secrets.

The code in question is roughly:
https://github.com/secret-scanner/action/blob/33d6b0b0b5f0cd113651222c196e07799d2c3d21/detect-new-secrets.sh#L30-L33

It's possible that the first detect-secret command (scan) failed and the second (audit) ran and failed (and that it perhaps shouldn't have been run if the first failed).

I'll see if I can create a sample repository that can reproduce the problem...

[jsoref]

@Anu48 just tripped on this, I might see if I can get her to create a reduced test case...