Back-porting Version Trimming #3681

shashank-elastic · 2024-05-15T18:28:26Z

Issues

Summary

Based on discussions in the issue #3563 following changes have been implemented

Preserve current and current -5 versions in stack schema and back port to those
- Currently we are release 8.14
- Current -3 will be release 8.14,8.13,8.12,8.11
- Current -5 is preserved 8.14,8.13,8.12,8.11,8.10,8.9

Test case test_previous_entries_gte_current_min_stack

We dont find value in modifying the version lock file when the backport versions are trimmed.
Also this can be a recurring process in every minor release prep as we will continue to shorten the backport versions consistently
To avoid failures of the test case like below

  FAILED tests/test_version_locking.py::TestVersionLock::test_previous_entries_gte_current_min_stack - 
  AssertionError: The following version.lock entries have previous locked versions which are lower than the 
  currently supported min_stack (8.9.0). To address this, run the `dev trim-version-l...```
 - This test has been removed.

To check with @approksiu do we need any user communications around this.

.github/workflows/lock-versions.yml

Mikaayenson · 2024-05-15T19:43:00Z

If we dont trim, then this file will grow indefinitely. It's not a problem, just something to think about. Also, we have EPR as the source of truth for rules.

If we followed the old approach, we would have to minstack rules and them trim which I'm not sure if that's necessary either.

approksiu · 2024-05-16T08:23:40Z

It would be good to document this in the repo docs, I don't think we need user communication outside of that. Thanks!

eric-forte-elastic

🟢 Manual review, looks good to me! 👍

terrancedejesus · 2024-05-16T20:35:06Z

I have a hunch we should export the diff, apply to some previous branches we still support, run unit tests and a package build just to confirm nothing is "funky".

brokensound77 · 2024-05-16T22:19:58Z

tests/test_version_locking.py

-class TestVersionLock(unittest.TestCase):
-    """Test version locking."""
-
-    def test_previous_entries_gte_current_min_stack(self):


I am not sure I fully understand the reasoning to remove this

Test case test_previous_entries_gte_current_min_stack
We dont find value in modifying the version lock file when the backport versions are trimmed.
Also this can be a recurring process in every minor release prep as we will continue to shorten the backport versions consistently
To avoid failures of the test case like below
FAILED tests/test_version_locking.py::TestVersionLock::test_previous_entries_gte_current_min_stack -
AssertionError: The following version.lock entries have previous locked versions which are lower than the
currently supported min_stack (8.9.0). To address this, run the `dev trim-version-l...```

This test has been removed.

Based on the original issue, this looks to still be needed

In #2001, we trimmed the backport matrix up from 7.13 to 7.16. This included the code to manage version lock changes. This created a situation where rules with previous entries which previously existed that were lower than the new min version (now 7.16) would fail this assertion in the lock code.

detection-rules/detection_rules/version_lock.py

Lines 268 to 271 in 79f575b

route = 'C'

# 4) on an old stack, after a breaking change has been made (updated fork)

assert stripped_version in lock_from_file.get("previous", {}), \

f"Expected {rule.id} @ v{stripped_version} in the rule lock"

remember, the scenarios across the version lock process include:

# scenarios to handle, assuming older stacks are always locked first: # 1) no breaking changes ever made or the first time a rule is created # 2) on the latest, after a breaking change has been locked # 3) on the latest stack, locking in a breaking change # 4) on an old stack, after a breaking change has been made

4 requires that there is contiguous continuity in order to know how many times to truly bump (these are forked rules and a rarer occurrence, making it harder to truly test). This means this should be restored and the trim-version-lock command should be run

I think the major concern is that since we're sliding the versions to the right, we will have to either update this test or lock versions for each rule to the oldest stack supported.

This created a situation where rules with previous entries which previously existed that were lower than the new min version (now 7.16) would fail this assertion in the lock code.

the version.lock needs to be trimmed. This test is written to prevent an erroneous state.

python -m detection_rules dev trim-version-lock --dry-run 8.9.0

█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄ █ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄ █▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█ The following versions would be collapsed to 8.9.0: 000047bb-b27a-47ec-8b62-ef1a5d2c9e19: 8.3 0136b315-b566-482f-866c-1d8e2477ba16: 8.3 015cca13-8832-49ac-a01b-a396114809f6: 8.3 02a4576a-7480-4284-9327-548a806b5e48: 8.3 03024bd9-d23f-4ec1-8674-3cf1a21e130b: 8.3 07b5f85a-240f-11ed-b3d9-f661ea17fbce: 8.3 0abf0c5b-62dd-48d2-ac4e-6b43fe3a6e83: 8.3 0c9a14d9-d65d-486f-9b5b-91e4e6b22bd0: 8.3 0ce6487d-8069-4888-9ddd-61b52490cebc: 8.3 0e52157a-8e96-4a95-a6e3-5faae5081a74: 8.3 0f4d35e4-925e-4959-ab24-911be207ee6f: 8.3 0f93cb9a-1931-48c2-8cd0-f173fd3e5283: 8.3 10a500bb-a28f-418e-ba29-ca4c8d1a9f2f: 8.3 119c8877-8613-416d-a98a-96b6664ee73a: 8.3 12051077-0124-4394-9522-8f4f4db1d674: 8.3 128468bf-cab1-4637-99ea-fdf3780a4609: 8.3 12a2f15d-597e-4334-88ff-38a02cb1330b: 8.3 12cbf709-69e8-4055-94f9-24314385c27e: 8.3 14de811c-d60f-11ec-9fd7-f661ea17fbce: 8.3 169f3a93-efc7-4df2-94d6-0d9438c310d1: 8.3 19de8096-e2b0-4bd8-80c9-34a820813fff: 8.3 1aa8fa52-44a7-4dae-b058-f3333b91c8d7: 8.3 1ba5160d-f5a2-4624-b0ff-6a1dc55d2516: 8.3 1c6a8c7a-5cb6-4a82-ba27-d5a5b8a40a38: 8.3 1e0a3f7c-21e7-4bb1-98c7-2036612fb1be: 8.3 2045567e-b0af-444a-8c0b-0b6e2dae9e13: 8.3 20457e4f-d1de-4b92-ae69-142e27a4342a: 8.3 2215b8bd-1759-4ffa-8ab8-55c8e6b32e7f: 8.3 227dc608-e558-43d9-b521-150772250bae: 8.3 26f68dba-ce29-497b-8e13-b4fde1db5a2d: 8.3 27071ea3-e806-4697-8abc-e22c92aa4293: 8.3 272a6484-2663-46db-a532-ef734bf9a796: 8.3 27f7c15a-91f8-4c3d-8b9e-1f99cc030a51: 8.3 29052c19-ff3e-42fd-8363-7be14d7c5469: 8.3 291a0de9-937a-4189-94c0-3e847c8b13e4: 8.3, 8.6 2abda169-416b-4bb3-9a6b-f8d239fd78ba: 8.3 2d8043ed-5bda-4caf-801c-c1feb7410504: 8.3 2dd480be-1263-4d9c-8672-172928f6789a: 8.3 2de10e77-c144-4e69-afb7-344e7127abd0: 8.3 333de828-8190-4cf5-8d7c-7575846f6fe0: 8.3 378f9024-8a0c-46a5-aa08-ce147ac73a4e: 8.3 37b211e8-4e2f-440f-86d8-06cc8f158cfa: 8.3 37f638ea-909d-4f94-9248-edd21e4a9906: 8.3 3805c3dc-f82c-4f8d-891e-63c24d3102b0: 8.3 38948d29-3d5d-42e3-8aec-be832aaaf8eb: 8.3 39144f38-5284-4f8e-a2ae-e3fd628d90b0: 8.3 3d3aa8f9-12af-441f-9344-9f31053e316d: 8.3 3e002465-876f-4f04-b016-84ef48ce7e5d: 8.3 3ed032b2-45d8-4406-bc79-7ad1eabb2c72: 8.3 3efee4f0-182a-40a8-a835-102c68a4175d: 8.3 40ddbcc8-6561-44d9-afc8-eefdbfe0cccd: 8.3 42bf698b-4738-445b-8231-c834ddefd8a0: 8.3 48819484-9826-4083-9eba-1da74cd0eaf2: 8.6 495e5f2e-2480-11ed-bea8-f661ea17fbce: 8.3 4d50a94f-2844-43fa-8395-6afbd5e1c5ef: 8.3 4edd3e1a-3aa0-499b-8147-4d2ea43b1613: 8.3 514121ce-c7b6-474a-8237-68ff71672379: 8.3 523116c0-d89d-4d7c-82c2-39e6845a78ef: 8.3 536997f7-ae73-447d-a12d-bff1e8f5f0a0: 8.3 54a81f68-5f2a-421e-8eed-f888278bb712: 8.3 565c2b44-7a21-4818-955f-8d4737967d2e: 8.3 56f2e9b5-4803-4e44-a0a4-a52dc79d57fe: 8.3 56fdfcf1-ca7c-4fd9-951d-e215ee26e404: 8.3 5930658c-2107-4afc-91af-e0e55b7f7184: 8.3 594e0cbf-86cc-45aa-9ff7-ff27db27d3ed: 8.3 5beaebc1-cc13-4bfc-9949-776f9e0dc318: 8.3 5e161522-2545-11ed-ac47-f661ea17fbce: 8.3 5e552599-ddec-4e14-bad1-28aa42404388: 8.3 60f3adec-1df9-4104-9c75-b97d9f078b25: 8.3 61ac3638-40a3-44b2-855a-985636ca985e: 8.3 65f9bccd-510b-40df-8263-334f03174fed: 8.3 66da12b1-ac83-40eb-814c-07ed1d82b7b9: 8.3 6731fbf2-8f28-49ed-9ab9-9a918ceb5a45: 8.3 675239ea-c1bc-4467-a6d3-b9e2cc7f676d: 8.3 676cff2b-450b-4cf1-8ed2-c0c58a4a2dd7: 8.3 684554fc-0777-47ce-8c9b-3d01f198d7f8: 8.3 6885d2ae-e008-4762-b98a-e8e1cd3a81e9: 8.3 68994a6c-c7ba-4e82-b476-26a26877adf6: 8.3 68a7a5a5-a2fc-4a76-ba9f-26849de881b4: 8.3 6951f15e-533c-4a60-8014-a3c3ab851a1b: 8.3 699e9fdb-b77c-4c01-995c-1c15019b9c43: 8.3 69c420e8-6c9e-4d28-86c0-8a2be2d1e78c: 8.3 6b84d470-9036-4cc0-a27c-6d90bbfe81ab: 8.3 6e9b351e-a531-4bdc-b73e-7034d6eed7ff: 8.3 6f435062-b7fc-4af9-acea-5b1ead65c5a5: 8.3 7024e2a0-315d-4334-bb1a-441c593e16ab: 8.3 7024e2a0-315d-4334-bb1a-552d604f27bc: 8.3 717f82c2-7741-4f9b-85b8-d06aeb853f4f: 8.3 721999d0-7ab2-44bf-b328-6e63367b9b29: 8.3 729aa18d-06a6-41c7-b175-b65b739b1181: 8.3 7453e19e-3dbf-4e4e-9ae0-33d6c6ed15e1: 8.3 7592c127-89fb-4209-a8f6-f9944dfd7e02: 8.3 764c8437-a581-4537-8060-1fdb0e92c92d: 8.3 785a404b-75aa-4ffd-8be5-3334a5a544dd: 8.3 78d3d8d9-b476-451d-a9e0-7a5addd70670: 8.3 7b3da11a-60a2-412e-8aa7-011e1eb9ed47: 8.3 7ba58110-ae13-439b-8192-357b0fcfa9d7: 8.3 7caa8e60-2df0-11ed-b814-f661ea17fbce: 8.3 7f89afef-9fc5-4e7b-bf16-75ffdf27f8db: 8.3 80084fa9-8677-4453-8680-b891d3c0c778: 8.3 809b70d3-e2c3-455e-af1b-2626a5a1a276: 8.3 81fe9dc6-a2d7-4192-a2d8-eed98afc766a: 8.3 827f8d8f-4117-4ae4-b551-f56d54b9da6b: 8.3 852c1f19-68e8-43a6-9dce-340771fe1be3: 8.3 8623535c-1e17-44e1-aa97-7a0699c3037d: 8.3 863cdf31-7fd3-41cf-a185-681237ea277b: 8.3 867616ec-41e5-4edc-ada2-ab13ab45de8a: 8.3 87594192-4539-4bc4-8543-23bc3d5bd2b4: 8.3 88671231-6626-4e1b-abb7-6e361a171fbb: 8.3 88fdcb8c-60e5-46ee-9206-2663adf1b1ce: 8.3 8a5c1e5f-ad63-481e-b53a-ef959230f7f1: 8.3 8acb7614-1d92-4359-bfcf-478b6d9de150: 8.3 9055ece6-2689-4224-a0e0-b04881e1f8ad: 8.3 91d04cd4-47a9-4334-ab14-084abe274d49: 8.3 92984446-aefb-4d5e-ad12-598042ca80ba: 8.3 93075852-b0f5-4b8b-89c3-a226efae5726: 8.3 931e25a5-0f5e-4ae0-ba0d-9e94eff7e3a4: 8.3 9395fd2c-9947-4472-86ef-4aceb2f7e872: 8.3 93e63c3e-4154-4fc6-9f86-b411e0987bbf: 8.3 93f47b6f-5728-4004-ba00-625083b3dcb0: 8.3 9510add4-3392-11ed-bd01-f661ea17fbce: 8.3 96b9f4ea-0e8c-435b-8d53-2096e75fcac5: 8.3 96e90768-c3b7-4df6-b5d9-6237f8bc36a8: 8.3 97314185-2568-4561-ae81-f3e480e5e695: 8.3 979729e7-0c52-4c4c-b71e-88103304a79f: 8.3 97a8e584-fd3b-421f-9b9d-9c9d9e57e9d7: 8.3 98995807-5b09-4e37-8a54-5cae5dc932d7: 8.3 98fd7407-0bd5-5817-cda0-3fcc33113a56: 8.3 9960432d-9b26-409f-972b-839a959e79e2: 8.3 9a3a3689-8ed1-4cdb-83fb-9506db54c61f: 8.3, 8.4 9cf7a0ae-2404-11ed-ae7d-f661ea17fbce: 8.3 9d110cb3-5f4b-4c9a-b9f5-53f0a1707ae2: 8.3 9d110cb3-5f4b-4c9a-b9f5-53f0a1707ae6: 8.3 9f9a2a82-93a8-4b1a-8778-1780895626d4: 8.3 a00681e3-9ed6-447c-ab2c-be648821c622: 8.3, 8.6 a2795334-2499-11ed-9e1a-f661ea17fbce: 8.3 a60326d7-dca7-4fb7-93eb-1ca03a1febbd: 8.3 a989fa1b-9a11-4dd8-a3e9-f0de9c6eb5f2: 8.3 a99f82f5-8e77-4f8b-b3ce-10c0f6afbc73: 8.3 ac706eae-d5ec-4b14-b4fd-e8ba8086f0e1: 8.3 acbc8bb9-2486-49a8-8779-45fb5f9a93ee: 8.3 ad3f2807-2b3e-47d7-b282-f84acbbe14be: 8.3 b2951150-658f-4a60-832f-a00d1e6c6745: 8.3 b45ab1d2-712f-4f01-a751-df3826969807: 8.3 b4bb1440-0fcb-4ed1-87e5-b06d58efc5e9: 8.3 b719a170-3bdb-4141-b0e3-13e3cf627bfe: 8.3 b8075894-0b62-46e5-977c-31275da34419: 8.3 bb9b13b2-1700-48a8-a750-b43b0a72ab69: 8.3 bba1b212-b85c-41c6-9b28-be0e5cdfc9b1: 8.3 bbd1a775-8267-41fa-9232-20e5582596ac: 8.3 bc0c6f0d-dab0-47a3-b135-0925f0a333bc: 8.3 bf1073bf-ce26-4607-b405-ba1ed8e9e204: 8.3 c1812764-0788-470f-8e74-eb4a14d47573: 8.3 c20cd758-07b1-46a1-b03f-fa66158258b8: 8.3 c749e367-a069-4a73-b1f2-43a3798153ad: 8.3 c74fd275-ab2c-4d49-8890-e2943fa65c09: 8.3 c7908cac-337a-4f38-b50d-5eeb78bdb531: 8.3 ca79768e-40e1-4e45-a097-0e5fbc876ac2: 8.3 cac91072-d165-11ec-a764-f661ea17fbce: 8.3 cad4500a-abd7-4ef3-b5d3-95524de7cfe1: 8.3 cc6a8a20-2df2-11ed-8378-f661ea17fbce: 8.3 cc92c835-da92-45c9-9f29-b4992ad621a0: 8.3 cd16fb10-0261-46e8-9932-a0336278cdbe: 8.3 cd89602e-9db0-48e3-9391-ae3bf241acd8: 8.3 cdbebdc1-dc97-43c6-a538-f26a20c0a911: 8.3 cde1bafa-9f01-4f43-a872-605b678968b0: 8.3 cf549724-c577-4fd6-8f9b-d1b8ec519ec0: 8.3 d48e1c13-4aca-4d1f-a7b1-a9161c0ad86f: 8.3 d5d86bf5-cf0c-4c06-b688-53fdc072fdfd: 8.3 d624f0ae-3dd1-4856-9aad-ccfe4d4bfa17: 8.3 d68eb1b5-5f1c-4b6d-9e63-5b6b145cd4aa: 8.3 d743ff2a-203e-4a46-a3e3-40512cfe8fbb: 8.3 d75991f2-b989-419d-b797-ac1e54ec2d61: 8.3 d8fc1cca-93ed-43c1-bbb6-c0dd3eff2958: 8.3 dca28dee-c999-400f-b640-50a081cc0fd1: 8.3 ded09d02-0137-4ccc-8005-c45e617e8d4c: 8.3 df7fda76-c92b-4943-bc68-04460a5ea5ba: 8.3 e08ccd49-0380-4b2b-8d71-8000377d6e49: 8.3 e12c0318-99b1-44f2-830c-3a38a43207ca: 8.3 e14c5fd7-fdd7-49c2-9e5b-ec49d817bc8d: 8.3 e26f042e-c590-4e82-8e05-41e81bd822ad: 8.3 e2a67480-3b79-403d-96e3-fdd2992c50ef: 8.3 e3c27562-709a-42bd-82f2-3ed926cced19: 8.3 e48236ca-b67a-4b4e-840c-fdc7782bc0c3: 8.3 e555105c-ba6d-481f-82bb-9b633e7b4827: 8.3 e6e3ecff-03dd-48ec-acbd-54a04de10c68: 8.3 e72f87d0-a70e-4f8d-8443-a6407bc34643: 8.3 e7cd5982-17c8-4959-874c-633acde7d426: 8.3 e9001ee6-2d00-4d2f-849e-b8b1fb05234c: 8.4 e90ee3af-45fc-432e-a850-4a58cf14a457: 8.3 e919611d-6b6f-493b-8314-7ed6ac2e413b: 8.3 ea248a02-bc47-4043-8e94-2885b19b2636: 8.3 ec8efb0c-604d-42fa-ac46-ed1cfbc38f78: 8.3 ecf2b32c-e221-4bd4-aa3b-c7d59b3bc01d: 8.3 edb91186-1c7e-4db8-b53e-bfa33a1a0a8a: 8.3 ee39a9f7-5a79-4b0a-9815-d36b3cf28d3e: 8.3 f06414a6-f2a4-466d-8eba-10f85e8abf71: 8.3 f30f3443-4fbb-4c27-ab89-c3ad49d62315: 8.3 f772ec8a-e182-483c-91d2-72058f76a44c: 8.3 f994964f-6fce-4d75-8e79-e16ccc412588: 8.3 fb01d790-9f74-4e76-97dd-b4b0f7bf6435: 8.3 fbd44836-0d69-4004-a0b4-03c20370c435: 8.3 fd7a6052-58fa-4397-93c3-4795249ccfa2: 8.3 ff4dd44a-0ac6-44c4-8609-3f81bc820f02: 8.3

once trimmed, the test passes as expected. Please

restore the test

run python -m detection_rules dev trim-version-lock 8.9.0

probably need to update docs/developing.md with these steps too (or wiki docs)

Thanks @brokensound77 from the above I have inferred

In the current state, I understand that assertion is going to fail without the version.lock file trimming.

The removed test case was introduced to catch such failures

but I have the following question,

I may be totally wrong in understanding, but if we leave the older min stack as is and not trim them to say 8.9.0 the impact for the customer clusters, is the below mentioned scenario?

remember, the scenarios across the version lock process include:

# scenarios to handle, assuming older stacks are always locked first: # 1) no breaking changes ever made or the first time a rule is created # 2) on the latest, after a breaking change has been locked # 3) on the latest stack, locking in a breaking change # 4) on an old stack, after a breaking change has been made

4 requires that there is contiguous continuity in order to know how many times to truly bump (these are forked rules and a rarer occurrence, making it harder to truly test). This means this should be restored and the trim-version-lock command should be run

And moving forward, in every release prep these are the steps that needs to be followed.

Trim Backport versions

Run trim-version-lock

Commit those changes before contiuing to release.

I have tried the dry run as for trimming

python -m detection_rules dev trim-version-lock --dry-run 8.9.0 █▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄ █ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄ █▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█ The following versions would be collapsed to 8.9.0: 000047bb-b27a-47ec-8b62-ef1a5d2c9e19: 8.3 0136b315-b566-482f-866c-1d8e2477ba16: 8.3 015cca13-8832-49ac-a01b-a396114809f6: 8.3 02a4576a-7480-4284-9327-548a806b5e48: 8.3 03024bd9-d23f-4ec1-8674-3cf1a21e130b: 8.3 07b5f85a-240f-11ed-b3d9-f661ea17fbce: 8.3 0abf0c5b-62dd-48d2-ac4e-6b43fe3a6e83: 8.3 0c9a14d9-d65d-486f-9b5b-91e4e6b22bd0: 8.3 0ce6487d-8069-4888-9ddd-61b52490cebc: 8.3 0e52157a-8e96-4a95-a6e3-5faae5081a74: 8.3 0f4d35e4-925e-4959-ab24-911be207ee6f: 8.3 0f93cb9a-1931-48c2-8cd0-f173fd3e5283: 8.3 10a500bb-a28f-418e-ba29-ca4c8d1a9f2f: 8.3 119c8877-8613-416d-a98a-96b6664ee73a: 8.3 12051077-0124-4394-9522-8f4f4db1d674: 8.3 128468bf-cab1-4637-99ea-fdf3780a4609: 8.3 12a2f15d-597e-4334-88ff-38a02cb1330b: 8.3 12cbf709-69e8-4055-94f9-24314385c27e: 8.3 14de811c-d60f-11ec-9fd7-f661ea17fbce: 8.3 169f3a93-efc7-4df2-94d6-0d9438c310d1: 8.3 19de8096-e2b0-4bd8-80c9-34a820813fff: 8.3 1aa8fa52-44a7-4dae-b058-f3333b91c8d7: 8.3 1ba5160d-f5a2-4624-b0ff-6a1dc55d2516: 8.3 1c6a8c7a-5cb6-4a82-ba27-d5a5b8a40a38: 8.3 1e0a3f7c-21e7-4bb1-98c7-2036612fb1be: 8.3 2045567e-b0af-444a-8c0b-0b6e2dae9e13: 8.3 20457e4f-d1de-4b92-ae69-142e27a4342a: 8.3 2215b8bd-1759-4ffa-8ab8-55c8e6b32e7f: 8.3 227dc608-e558-43d9-b521-150772250bae: 8.3 26f68dba-ce29-497b-8e13-b4fde1db5a2d: 8.3 27071ea3-e806-4697-8abc-e22c92aa4293: 8.3 272a6484-2663-46db-a532-ef734bf9a796: 8.3 27f7c15a-91f8-4c3d-8b9e-1f99cc030a51: 8.3 29052c19-ff3e-42fd-8363-7be14d7c5469: 8.3 291a0de9-937a-4189-94c0-3e847c8b13e4: 8.3, 8.6 2abda169-416b-4bb3-9a6b-f8d239fd78ba: 8.3 2d8043ed-5bda-4caf-801c-c1feb7410504: 8.3 2dd480be-1263-4d9c-8672-172928f6789a: 8.3 2de10e77-c144-4e69-afb7-344e7127abd0: 8.3 333de828-8190-4cf5-8d7c-7575846f6fe0: 8.3 378f9024-8a0c-46a5-aa08-ce147ac73a4e: 8.3 37b211e8-4e2f-440f-86d8-06cc8f158cfa: 8.3 37f638ea-909d-4f94-9248-edd21e4a9906: 8.3 3805c3dc-f82c-4f8d-891e-63c24d3102b0: 8.3 38948d29-3d5d-42e3-8aec-be832aaaf8eb: 8.3 39144f38-5284-4f8e-a2ae-e3fd628d90b0: 8.3 3d3aa8f9-12af-441f-9344-9f31053e316d: 8.3 3e002465-876f-4f04-b016-84ef48ce7e5d: 8.3 3ed032b2-45d8-4406-bc79-7ad1eabb2c72: 8.3 3efee4f0-182a-40a8-a835-102c68a4175d: 8.3 40ddbcc8-6561-44d9-afc8-eefdbfe0cccd: 8.3 42bf698b-4738-445b-8231-c834ddefd8a0: 8.3 48819484-9826-4083-9eba-1da74cd0eaf2: 8.6 495e5f2e-2480-11ed-bea8-f661ea17fbce: 8.3 4d50a94f-2844-43fa-8395-6afbd5e1c5ef: 8.3 4edd3e1a-3aa0-499b-8147-4d2ea43b1613: 8.3 514121ce-c7b6-474a-8237-68ff71672379: 8.3 523116c0-d89d-4d7c-82c2-39e6845a78ef: 8.3 536997f7-ae73-447d-a12d-bff1e8f5f0a0: 8.3 54a81f68-5f2a-421e-8eed-f888278bb712: 8.3 565c2b44-7a21-4818-955f-8d4737967d2e: 8.3 56f2e9b5-4803-4e44-a0a4-a52dc79d57fe: 8.3 56fdfcf1-ca7c-4fd9-951d-e215ee26e404: 8.3 5930658c-2107-4afc-91af-e0e55b7f7184: 8.3 594e0cbf-86cc-45aa-9ff7-ff27db27d3ed: 8.3 5beaebc1-cc13-4bfc-9949-776f9e0dc318: 8.3 5e161522-2545-11ed-ac47-f661ea17fbce: 8.3 5e552599-ddec-4e14-bad1-28aa42404388: 8.3 60f3adec-1df9-4104-9c75-b97d9f078b25: 8.3 61ac3638-40a3-44b2-855a-985636ca985e: 8.3 65f9bccd-510b-40df-8263-334f03174fed: 8.3 66da12b1-ac83-40eb-814c-07ed1d82b7b9: 8.3 6731fbf2-8f28-49ed-9ab9-9a918ceb5a45: 8.3 675239ea-c1bc-4467-a6d3-b9e2cc7f676d: 8.3 676cff2b-450b-4cf1-8ed2-c0c58a4a2dd7: 8.3 684554fc-0777-47ce-8c9b-3d01f198d7f8: 8.3 6885d2ae-e008-4762-b98a-e8e1cd3a81e9: 8.3 68994a6c-c7ba-4e82-b476-26a26877adf6: 8.3 68a7a5a5-a2fc-4a76-ba9f-26849de881b4: 8.3 6951f15e-533c-4a60-8014-a3c3ab851a1b: 8.3 699e9fdb-b77c-4c01-995c-1c15019b9c43: 8.3 69c420e8-6c9e-4d28-86c0-8a2be2d1e78c: 8.3 6b84d470-9036-4cc0-a27c-6d90bbfe81ab: 8.3 6e9b351e-a531-4bdc-b73e-7034d6eed7ff: 8.3 6f435062-b7fc-4af9-acea-5b1ead65c5a5: 8.3 7024e2a0-315d-4334-bb1a-441c593e16ab: 8.3 7024e2a0-315d-4334-bb1a-552d604f27bc: 8.3 717f82c2-7741-4f9b-85b8-d06aeb853f4f: 8.3 721999d0-7ab2-44bf-b328-6e63367b9b29: 8.3 729aa18d-06a6-41c7-b175-b65b739b1181: 8.3 7453e19e-3dbf-4e4e-9ae0-33d6c6ed15e1: 8.3 7592c127-89fb-4209-a8f6-f9944dfd7e02: 8.3 764c8437-a581-4537-8060-1fdb0e92c92d: 8.3 785a404b-75aa-4ffd-8be5-3334a5a544dd: 8.3 78d3d8d9-b476-451d-a9e0-7a5addd70670: 8.3 7b3da11a-60a2-412e-8aa7-011e1eb9ed47: 8.3 7ba58110-ae13-439b-8192-357b0fcfa9d7: 8.3 7caa8e60-2df0-11ed-b814-f661ea17fbce: 8.3 7f89afef-9fc5-4e7b-bf16-75ffdf27f8db: 8.3 80084fa9-8677-4453-8680-b891d3c0c778: 8.3 809b70d3-e2c3-455e-af1b-2626a5a1a276: 8.3 81fe9dc6-a2d7-4192-a2d8-eed98afc766a: 8.3 827f8d8f-4117-4ae4-b551-f56d54b9da6b: 8.3 852c1f19-68e8-43a6-9dce-340771fe1be3: 8.3 8623535c-1e17-44e1-aa97-7a0699c3037d: 8.3 863cdf31-7fd3-41cf-a185-681237ea277b: 8.3 867616ec-41e5-4edc-ada2-ab13ab45de8a: 8.3 87594192-4539-4bc4-8543-23bc3d5bd2b4: 8.3 88671231-6626-4e1b-abb7-6e361a171fbb: 8.3 88fdcb8c-60e5-46ee-9206-2663adf1b1ce: 8.3 8a5c1e5f-ad63-481e-b53a-ef959230f7f1: 8.3 8acb7614-1d92-4359-bfcf-478b6d9de150: 8.3 9055ece6-2689-4224-a0e0-b04881e1f8ad: 8.3 91d04cd4-47a9-4334-ab14-084abe274d49: 8.3 92984446-aefb-4d5e-ad12-598042ca80ba: 8.3 93075852-b0f5-4b8b-89c3-a226efae5726: 8.3 931e25a5-0f5e-4ae0-ba0d-9e94eff7e3a4: 8.3 9395fd2c-9947-4472-86ef-4aceb2f7e872: 8.3 93e63c3e-4154-4fc6-9f86-b411e0987bbf: 8.3 93f47b6f-5728-4004-ba00-625083b3dcb0: 8.3 9510add4-3392-11ed-bd01-f661ea17fbce: 8.3 96b9f4ea-0e8c-435b-8d53-2096e75fcac5: 8.3 96e90768-c3b7-4df6-b5d9-6237f8bc36a8: 8.3 97314185-2568-4561-ae81-f3e480e5e695: 8.3 979729e7-0c52-4c4c-b71e-88103304a79f: 8.3 97a8e584-fd3b-421f-9b9d-9c9d9e57e9d7: 8.3 98995807-5b09-4e37-8a54-5cae5dc932d7: 8.3 98fd7407-0bd5-5817-cda0-3fcc33113a56: 8.3 9960432d-9b26-409f-972b-839a959e79e2: 8.3 9a3a3689-8ed1-4cdb-83fb-9506db54c61f: 8.3, 8.4 9cf7a0ae-2404-11ed-ae7d-f661ea17fbce: 8.3 9d110cb3-5f4b-4c9a-b9f5-53f0a1707ae2: 8.3 9d110cb3-5f4b-4c9a-b9f5-53f0a1707ae6: 8.3 9f9a2a82-93a8-4b1a-8778-1780895626d4: 8.3 a00681e3-9ed6-447c-ab2c-be648821c622: 8.3, 8.6 a2795334-2499-11ed-9e1a-f661ea17fbce: 8.3 a60326d7-dca7-4fb7-93eb-1ca03a1febbd: 8.3 a989fa1b-9a11-4dd8-a3e9-f0de9c6eb5f2: 8.3 a99f82f5-8e77-4f8b-b3ce-10c0f6afbc73: 8.3 ac706eae-d5ec-4b14-b4fd-e8ba8086f0e1: 8.3 acbc8bb9-2486-49a8-8779-45fb5f9a93ee: 8.3 ad3f2807-2b3e-47d7-b282-f84acbbe14be: 8.3 b2951150-658f-4a60-832f-a00d1e6c6745: 8.3 b45ab1d2-712f-4f01-a751-df3826969807: 8.3 b4bb1440-0fcb-4ed1-87e5-b06d58efc5e9: 8.3 b719a170-3bdb-4141-b0e3-13e3cf627bfe: 8.3 b8075894-0b62-46e5-977c-31275da34419: 8.3 bb9b13b2-1700-48a8-a750-b43b0a72ab69: 8.3 bba1b212-b85c-41c6-9b28-be0e5cdfc9b1: 8.3 bbd1a775-8267-41fa-9232-20e5582596ac: 8.3 bc0c6f0d-dab0-47a3-b135-0925f0a333bc: 8.3 bf1073bf-ce26-4607-b405-ba1ed8e9e204: 8.3 c1812764-0788-470f-8e74-eb4a14d47573: 8.3 c20cd758-07b1-46a1-b03f-fa66158258b8: 8.3 c749e367-a069-4a73-b1f2-43a3798153ad: 8.3 c74fd275-ab2c-4d49-8890-e2943fa65c09: 8.3 c7908cac-337a-4f38-b50d-5eeb78bdb531: 8.3 ca79768e-40e1-4e45-a097-0e5fbc876ac2: 8.3 cac91072-d165-11ec-a764-f661ea17fbce: 8.3 cad4500a-abd7-4ef3-b5d3-95524de7cfe1: 8.3 cc6a8a20-2df2-11ed-8378-f661ea17fbce: 8.3 cc92c835-da92-45c9-9f29-b4992ad621a0: 8.3 cd16fb10-0261-46e8-9932-a0336278cdbe: 8.3 cd89602e-9db0-48e3-9391-ae3bf241acd8: 8.3 cdbebdc1-dc97-43c6-a538-f26a20c0a911: 8.3 cde1bafa-9f01-4f43-a872-605b678968b0: 8.3 cf549724-c577-4fd6-8f9b-d1b8ec519ec0: 8.3 d48e1c13-4aca-4d1f-a7b1-a9161c0ad86f: 8.3 d5d86bf5-cf0c-4c06-b688-53fdc072fdfd: 8.3 d624f0ae-3dd1-4856-9aad-ccfe4d4bfa17: 8.3 d68eb1b5-5f1c-4b6d-9e63-5b6b145cd4aa: 8.3 d743ff2a-203e-4a46-a3e3-40512cfe8fbb: 8.3 d75991f2-b989-419d-b797-ac1e54ec2d61: 8.3 d8fc1cca-93ed-43c1-bbb6-c0dd3eff2958: 8.3 dca28dee-c999-400f-b640-50a081cc0fd1: 8.3 ded09d02-0137-4ccc-8005-c45e617e8d4c: 8.3 df7fda76-c92b-4943-bc68-04460a5ea5ba: 8.3 e08ccd49-0380-4b2b-8d71-8000377d6e49: 8.3 e12c0318-99b1-44f2-830c-3a38a43207ca: 8.3 e14c5fd7-fdd7-49c2-9e5b-ec49d817bc8d: 8.3 e26f042e-c590-4e82-8e05-41e81bd822ad: 8.3 e2a67480-3b79-403d-96e3-fdd2992c50ef: 8.3 e3c27562-709a-42bd-82f2-3ed926cced19: 8.3 e48236ca-b67a-4b4e-840c-fdc7782bc0c3: 8.3 e555105c-ba6d-481f-82bb-9b633e7b4827: 8.3 e6e3ecff-03dd-48ec-acbd-54a04de10c68: 8.3 e72f87d0-a70e-4f8d-8443-a6407bc34643: 8.3 e7cd5982-17c8-4959-874c-633acde7d426: 8.3 e9001ee6-2d00-4d2f-849e-b8b1fb05234c: 8.4 e90ee3af-45fc-432e-a850-4a58cf14a457: 8.3 e919611d-6b6f-493b-8314-7ed6ac2e413b: 8.3 ea248a02-bc47-4043-8e94-2885b19b2636: 8.3 ec8efb0c-604d-42fa-ac46-ed1cfbc38f78: 8.3 ecf2b32c-e221-4bd4-aa3b-c7d59b3bc01d: 8.3 edb91186-1c7e-4db8-b53e-bfa33a1a0a8a: 8.3 ee39a9f7-5a79-4b0a-9815-d36b3cf28d3e: 8.3 f06414a6-f2a4-466d-8eba-10f85e8abf71: 8.3 f30f3443-4fbb-4c27-ab89-c3ad49d62315: 8.3 f772ec8a-e182-483c-91d2-72058f76a44c: 8.3 f994964f-6fce-4d75-8e79-e16ccc412588: 8.3 fb01d790-9f74-4e76-97dd-b4b0f7bf6435: 8.3 fbd44836-0d69-4004-a0b4-03c20370c435: 8.3 fd7a6052-58fa-4397-93c3-4795249ccfa2: 8.3 ff4dd44a-0ac6-44c4-8609-3f81bc820f02: 8.3

this looks very similar to your execution as well.

I think the key is this """Trim all previous entries within the version lock file which are lower than the min_version."""

That is what the trim command actually performed. and is updated in the PR

This created a situation where rules with previous entries which previously existed that were lower than the new min version (now 7.16) would fail this assertion in the lock code.

the version.lock needs to be trimmed. This test is written to prevent an erroneous state.

python -m detection_rules dev trim-version-lock --dry-run 8.9.0

█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄ █ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄ █▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█ The following versions would be collapsed to 8.9.0: 000047bb-b27a-47ec-8b62-ef1a5d2c9e19: 8.3

This may be a potential issue. When you run the time command, (in dry run) it says it will be collapsed to 8.9.0. When you actually run the command, it trims to:

"000047bb-b27a-47ec-8b62-ef1a5d2c9e19": { "min_stack_version": "8.10", "rule_name": "Attempt to Modify an Okta Policy Rule", "sha256": "8e250a9c8ff04c25044e7bd0932764e6d21ad669c07dcbd9589c825b771b13f2", "type": "query", "version": 207 },

shashank-elastic · 2024-05-20T14:24:40Z

To summarise the Changes for now and further

We have retained test_previous_entries_gte_current_min_stack test
We have run python -m detection_rules dev trim-version-lock 8.9.0 which has collapsed all version lock entries with previous:8.3 to 8.9 basically removing the previous entry.
Moving forward we drop a supported version and add a new version and run the trim-version-lock to the new min version.
This would not require any changes in rules min_stack_version @brokensound77 for confirmation

Reviews from @Mikaayenson / @brokensound77 / @terrancedejesus

shashank-elastic · 2024-05-20T14:42:01Z

Internal facing documentation for release process updated https://github.com/elastic/ia-trade-team/wiki/Detection-Rules-Releasing-v2#prepare-for-next-elastic-stack-minor-release

brokensound77

LGTM, thanks @shashank-elastic

A few notes

pay close attention to the backports to ensure no issues
there are a lot of rules with:

min_stack_comments = "New fields added: required_fields, related_integrations, setup"
min_stack_version = "8.3.0"

Which can likely be removed now in a separate PR

shashank-elastic · 2024-05-20T15:56:28Z

It would be good to document this in the repo docs, I don't think we need user communication outside of that. Thanks!
@approksiu Would adding a versions_supported.md at ~ detection-rules/tree/main/docs make it good enough for this request.

shashank-elastic · 2024-05-20T17:02:12Z

Version Doc Preview

https://github.com/elastic/detection-rules/blob/issue-3563/docs/versioning.md

shashank-elastic · 2024-05-20T17:26:35Z

Testing on Branch 8.10

Made the same changes to detection_rules/etc/version.lock.json detection_rules/etc/stack-schema-map.yaml
Ran python -m detection_rules dev trim-version-lock 8.9.0

detection-rules on  8.10 [$!?] is 📦 v0.1.0 via 🐍 v3.12.3 (.venv) on ☁️  shashank.suryanarayana@elastic.co 
❯ git status
On branch 8.10
Your branch is up to date with 'origin/8.10'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   .github/workflows/lock-versions.yml
        modified:   detection_rules/etc/stack-schema-map.yaml
        modified:   detection_rules/etc/version.lock.json

Ran Test , with some failures on 8.10
-- normalize_kql_keywords --> Unrelated to changes something is failing at local of mine can be ignored.

FAILED tests/kuery/test_lint.py::LintTests::test_upper_tokens - TypeError: parse() got an unexpected keyword argument 'normalize_kql_keywords'
FAILED tests/test_all_rules.py::TestValidRules::test_rule_type_changes - AssertionError: Expected 20457e4f-d1de-4b92-ae69-142e27a4342a @ v8.9 in the rule lock
FAILED tests/test_packages.py::TestPackages::test_package_loader_default_configs - AssertionError: Expected 20457e4f-d1de-4b92-ae69-142e27a4342a @ v8.9 in the rule lock
FAILED tests/test_packages.py::TestPackages::test_package_summary - AssertionError: Expected 20457e4f-d1de-4b92-ae69-142e27a4342a @ v8.9 in the rule lock
FAILED tests/test_packages.py::TestPackages::test_rule_versioning - AssertionError: Expected 20457e4f-d1de-4b92-ae69-142e27a4342a @ v8.9 in the rule lock
============================================================================================= 5 failed, 138 passed, 2 skipped, 1 warning in 31.44s =============================================================================================

Even after the trim we have the assertion failure


                elif min_stack < latest_locked_stack_version:
                    route = 'C'
                    # 4) on an old stack, after a breaking change has been made (updated fork)
>                   assert stripped_version in lock_from_file.get("previous", {}), \
                        f"Expected {rule.id} @ v{stripped_version} in the rule lock"
E                   AssertionError: Expected 20457e4f-d1de-4b92-ae69-142e27a4342a @ v8.9 in the rule lock

The version lock-diff on main and in my local is the same as here https://github.com/elastic/detection-rules/pull/3681/files#diff-d539b5f6ddb25f5e1ed43553241cee8ba802cd4417097d8fafcb2f90b557b935L1403

Something is really funky as we don't have unit test failures on Main @terrancedejesus

cc @brokensound77 @Mikaayenson

shashank-elastic · 2024-05-20T18:10:31Z

We are seeing some pattern here with failures in 8.9 as well for


FAILED tests/test_all_rules.py::TestValidRules::test_rule_type_changes - AssertionError: Expected 3805c3dc-f82c-4f8d-891e-63c24d3102b0 @ v8.9 in the rule lock
FAILED tests/test_packages.py::TestPackages::test_package_loader_default_configs - AssertionError: Expected 3805c3dc-f82c-4f8d-891e-63c24d3102b0 @ v8.9 in the rule lock
FAILED tests/test_packages.py::TestPackages::test_package_summary - AssertionError: Expected 3805c3dc-f82c-4f8d-891e-63c24d3102b0 @ v8.9 in the rule lock
FAILED tests/test_packages.py::TestPackages::test_rule_versioning - AssertionError: Expected 3805c3dc-f82c-4f8d-891e-63c24d3102b0 @ v8.9 in the rule lock

With the diff identical to this https://github.com/elastic/detection-rules/pull/3681/files#diff-d539b5f6ddb25f5e1ed43553241cee8ba802cd4417097d8fafcb2f90b557b935

shashank-elastic · 2024-05-21T14:59:15Z

Update

We have done Rule changes as well , where in the min_stack_version and min_stack_comments have been removed via python -m detection_rules dev trim-version-lock 8.9.0
- Please note the rule formatter must have rearranged tags and a=other fields, but those are just rearrangments and spacing and nothing has changed.
The 5 rules which had providers fields have been reverted back to original state with only min_stack_version and min_stack_comments. The below mentioned rules can be double checked by reviewers
-- rules/windows/command_and_control_common_webservices.toml
-- rules/windows/command_and_control_remote_file_copy_desktopimgdownldr.toml
-- rules/windows/command_and_control_remote_file_copy_mpcmdrun.toml
-- rules/windows/command_and_control_remote_file_copy_powershell.toml
-- rules_building_block/command_and_control_certutil_network_connection.toml

shashank-elastic · 2024-05-21T17:43:28Z

Test Failure test_build_fields_min_stack

FAILED tests/test_all_rules.py::TestBuildTimeFields::test_build_fields_min_stack - 
AssertionError: ['a87a4e42-1d82-44bd-b0bf-d9b7f91fb89e - Web Application Suspicious Activity: 
POST Request Declined -> uses a rule type with build fields requiring min_stack_versions to be 
set: related_integrations >= 8.3.0, required_fields >= 8.3.0, setup >= 8.3.0']

Version Diff.

There is no version forks, and safe to add minstack for the field requirement!

cc @Mikaayenson @brokensound77

shashank-elastic · 2024-05-21T17:57:10Z

We might be running into another issue here @brokensound77 with the recent changes of how the min_stack_version is dropped from the version lock file via trim and the rules updated.

We have a failure of test case test_build_fields_min_stack. Initially I though this was a one off issue for rule with id :a87a4e42-1d82-44bd-b0bf-d9b7f91fb89e But on digging further found the PR-2262 that has added this test case and has introduced "related_integrations": (Version('8.3'), None), as one of the build_filed_versions.

If I understand right then the trim-lock-versions have to be revisited and this is going to affect all the ( or most of the) rules that have min_stack_version stripped.

Since originally only version.lock was trimmed with out rule changes, the test cases have passed and the recent rule changes pushed has identified these

cc @Mikaayenson

detection_rules/devtools.py

…ver <= min supported

tests/test_all_rules.py

shashank-elastic · 2024-05-22T04:57:11Z

For the formatter bug we have the META #3558, which could be used to tackle the bug.

Mikaayenson

LGTM. This is a big change. I recommend deconflict time with the team to merge today. We also dont want to wait to long because you dont want to have to keep resolving merge conflicts.

Definitely check back ports when this goes in.

Mikaayenson · 2024-05-22T17:54:43Z

docs/versioning.md

+## Current Version
+
+The current version of prebuilt detection rules is `v8.14`.
+
+## Previous Versions Released
+
+The following version(s) are released along with the current version.
+
+- `v8.13`
+- `v8.12`
+- `v8.11`
+
+### Previous Versions Maintained
+
+The following version(s) are maintained along with the current version.
+
+- `v8.10`
+- `v8.9`


If we never want to update this again, I'd make this more conceptual.

To keep it simple to use and also use the Prep branch step to edit and reflect the right versions, this was kept non conceptual.

Mikaayenson · 2024-05-22T17:55:58Z

detection_rules/devtools.py

 @click.option('--dry-run', is_flag=True, help='Print the changes rather than saving the file')
-def trim_version_lock(stack_version: str, dry_run: bool):
+def trim_version_lock(stack_version: str, skip_rule_updates: bool, dry_run: bool):


This method is in dire need for a refactor, but to not reintroduce a bug, perhaps we can move this to a separate issue.

Mikaayenson · 2024-05-22T18:39:53Z

Updated protections.

This reverts commit 71d2c59.

Back-porting Version Trimming

ffc4f7b

shashank-elastic self-assigned this May 15, 2024

shashank-elastic requested review from brokensound77, Mikaayenson and eric-forte-elastic as code owners May 15, 2024 18:28

github-actions bot added the backport: auto label May 15, 2024

shashank-elastic requested review from traut and terrancedejesus May 15, 2024 18:28

shashank-elastic linked an issue May 15, 2024 that may be closed by this pull request

[FR] Back-porting Version Trimming #3563

Closed

shashank-elastic mentioned this pull request May 15, 2024

[FR] Back-porting Version Trimming #3563

Closed

Mikaayenson reviewed May 15, 2024

View reviewed changes

.github/workflows/lock-versions.yml Show resolved Hide resolved

Merge branch 'main' into issue-3563

227ee3b

shashank-elastic requested a review from Mikaayenson May 16, 2024 11:39

eric-forte-elastic approved these changes May 16, 2024

View reviewed changes

brokensound77 reviewed May 16, 2024

View reviewed changes

Run trim version-lock

2d85a9a

shashank-elastic requested a review from brokensound77 May 20, 2024 14:19

Merge branch 'main' into issue-3563

dd859c8

brokensound77 approved these changes May 20, 2024

View reviewed changes

shashank-elastic added 2 commits May 20, 2024 22:30

Add version Doc

cee8363

Add version Doc

3f5a999

shashank-elastic and others added 6 commits May 21, 2024 21:51

Merge branch 'main' into issue-3563

332779a

Fix rule formatting error

1193667

Fix rule formatting error

7f74669

Fix rule formatting error

49c3faf

Fix rule formatting error

68961c2

Fix rule formatting error

791c0a5

shashank-elastic commented May 21, 2024

View reviewed changes

detection_rules/devtools.py Show resolved Hide resolved

update test_build_fields_min_stack to ignore build time fields stack …

4db4397

…ver <= min supported

brokensound77 reviewed May 22, 2024

View reviewed changes

tests/test_all_rules.py Show resolved Hide resolved

Merge branch 'main' into issue-3563

eb7b021

shashank-elastic mentioned this pull request May 22, 2024

[Meta] Refactor Rule Formatter #3558

Open

Mikaayenson approved these changes May 22, 2024

View reviewed changes

shashank-elastic merged commit 71d2c59 into main May 22, 2024
9 checks passed

shashank-elastic deleted the issue-3563 branch May 22, 2024 18:41

Mikaayenson added a commit that referenced this pull request May 22, 2024

Revert "Back-porting Version Trimming (#3681)"

2c3dbfc

This reverts commit 71d2c59.

Mikaayenson restored the issue-3563 branch May 22, 2024 18:52

shashank-elastic mentioned this pull request May 22, 2024

Back-porting Version Trimming #3704

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Back-porting Version Trimming #3681

Back-porting Version Trimming #3681

shashank-elastic commented May 15, 2024 •

edited

Mikaayenson commented May 15, 2024

approksiu commented May 16, 2024

eric-forte-elastic left a comment

terrancedejesus commented May 16, 2024

brokensound77 May 16, 2024

brokensound77 May 16, 2024

Mikaayenson May 17, 2024

brokensound77 May 17, 2024 •

edited

brokensound77 May 17, 2024 •

edited

shashank-elastic May 20, 2024 •

edited by brokensound77

Mikaayenson May 20, 2024

shashank-elastic May 20, 2024

Mikaayenson May 21, 2024

shashank-elastic commented May 20, 2024

shashank-elastic commented May 20, 2024

brokensound77 left a comment

shashank-elastic commented May 20, 2024

shashank-elastic commented May 20, 2024

shashank-elastic commented May 20, 2024

shashank-elastic commented May 20, 2024

shashank-elastic commented May 21, 2024

shashank-elastic commented May 21, 2024

shashank-elastic commented May 21, 2024

shashank-elastic commented May 22, 2024

Mikaayenson left a comment

Mikaayenson May 22, 2024

shashank-elastic May 22, 2024

Mikaayenson May 22, 2024

Mikaayenson commented May 22, 2024

	route = 'C'
	# 4) on an old stack, after a breaking change has been made (updated fork)
	assert stripped_version in lock_from_file.get("previous", {}), \
	f"Expected {rule.id} @ v{stripped_version} in the rule lock"

Back-porting Version Trimming #3681

Back-porting Version Trimming #3681

Conversation

shashank-elastic commented May 15, 2024 • edited

Issues

Summary

Mikaayenson commented May 15, 2024

approksiu commented May 16, 2024

eric-forte-elastic left a comment

Choose a reason for hiding this comment

terrancedejesus commented May 16, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

brokensound77 May 17, 2024 • edited

Choose a reason for hiding this comment

brokensound77 May 17, 2024 • edited

Choose a reason for hiding this comment

shashank-elastic May 20, 2024 • edited by brokensound77

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shashank-elastic commented May 20, 2024

shashank-elastic commented May 20, 2024

brokensound77 left a comment

Choose a reason for hiding this comment

shashank-elastic commented May 20, 2024

shashank-elastic commented May 20, 2024

shashank-elastic commented May 20, 2024

shashank-elastic commented May 20, 2024

shashank-elastic commented May 21, 2024

shashank-elastic commented May 21, 2024

shashank-elastic commented May 21, 2024

shashank-elastic commented May 22, 2024

Mikaayenson left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Mikaayenson commented May 22, 2024

shashank-elastic commented May 15, 2024 •

edited

brokensound77 May 17, 2024 •

edited

brokensound77 May 17, 2024 •

edited

shashank-elastic May 20, 2024 •

edited by brokensound77