Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Recursive replace #2864

Merged
merged 4 commits into from Jan 16, 2023
Merged

Recursive replace #2864

merged 4 commits into from Jan 16, 2023

Conversation

masenf
Copy link
Collaborator

@masenf masenf commented Jan 16, 2023

Recursive Replace

Expand substitution expressions that result from a previous substitution expression replacement value (up to 100 times).

Add a new MatchRecursionError and stack depth tracking to avoid infinite recursion or python throwing a RecursionError and blowing up tox. Instead, log an error and take the last value as-is, which is what tox did prior to 4.3.0.

The behavior with this patch is more similar to pre-4.3.0 parser changes with the large caveat that replacement values are themselves expanded within their own context only, previously the value was replaced into the original string and the entire result was re-expanded, which led to some strange edge cases when expansions contained un-escaped special characters.

Fix #2863

Thanks for contribution

Please, make sure you address all the checklists (for details on how see
development documentation)!

  • ran the linter to address style issues (tox -e fix)
  • wrote descriptive pull request text
  • ensured there are test(s) validating the fix
  • added news fragment in docs/changelog folder
  • updated/extended the documentation

When a replacement references a replacement
in a non-testenv section it should also be expanded
Expand substitution expressions that result from a previous subsitution
expression replacement value (up to 100 times).

Fix tox-dev#2863
docs/changelog/2863.bugfix.rst Outdated Show resolved Hide resolved
# split alongside :, unless it's preceded by a single capital letter (Windows drive letter in paths)
ARG_DELIMITER = ":"
REPLACE_START = "{"
REPLACE_END = "}"
BACKSLASH_ESCAPE_CHARS = ["\\", ARG_DELIMITER, REPLACE_START, REPLACE_END, "[", "]"]
MAX_REPLACE_DEPTH = 100
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a test for this with a circular reference of length 5, and one with 101?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a test that checks different recursion levels and asserts that the log message is triggered and that expansion stops at the correct level.

Create a long chain of substitution values and assert that
they stop being processed after some time.
@gaborbernat gaborbernat merged commit 99b849b into tox-dev:main Jan 16, 2023
@masenf masenf deleted the recursive-replace branch January 16, 2023 04:46
descope bot added a commit to descope/django-descope that referenced this pull request Jan 30, 2023
This PR contains the following updates:

| Package | Type | Update | Change | Pending |
|---|---|---|---|---|
| [tox](https://togithub.com/tox-dev/tox)
([changelog](https://tox.wiki/en/latest/changelog.html)) | dev | patch |
`4.3.0` -> `4.3.1` | `4.4.2` (+6) |

---

### Release Notes

<details>
<summary>tox-dev/tox</summary>

### [`v4.3.1`](https://togithub.com/tox-dev/tox/releases/tag/4.3.1)

[Compare Source](https://togithub.com/tox-dev/tox/compare/4.3.0...4.3.1)

#### What's Changed

- Recursive replace by [@&#8203;masenf](https://togithub.com/masenf) in
[tox-dev/tox#2864

**Full Changelog**: tox-dev/tox@4.3.0...4.3.1

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Enabled.

♻ **Rebasing**: Whenever PR is behind base branch, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNC4xMDEuMCIsInVwZGF0ZWRJblZlciI6IjM0LjEwMS4wIn0=-->

Co-authored-by: descope[bot] <descope[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Tox 4.3.0 breaks substitution for values from other sections
2 participants