Changed behavior with httpd/mod_security due outbound anomaly score resetting in v4.2.0

[arminabf]

Affected Products:

• mod_security v2.9.7 with Apache httpd 2.4.58
• OWASP CRS v4.2.0

Description:

Upgrading OWASP CRS from v3.3.5 to v4.2.0 in conjunction with mod_security on Apache httpd causes issues
when using RedirectMatch on virtual-host scope.

Steps to Reproduce:

1. Run Apache httpd with mod_security enabled.
2. Configure OWASP CRS v3.3.5.
3. Set up a virtual host with the following RedirectMatch directive: RedirectMatch ^/$ https://<FQDN>/new/path
4. Access the root path (/) of the virtual host.
5. Observe successful redirection to https://<FQDN>/new/path.
6. Upgrade OWASP CRS to version 4.2.0.
7. Repeat steps 4 and 5.
8. Observe that the redirection no longer occurs, resulting in a 403 error.

Technical Details

The different behaviour appears to be related to newly introduced rules in v4.2.0 with IDs 959059 and 959159.
These rules (re)set specific anomaly scores (blocking_outbound_anomaly_score and _detection_outbound_anomaly_score)
to 0, leading to comparison of scores to yet uninitialized thresholds in the evaluation phase.

Analysis

Testing out pointed to those both rules as temporarily removing them leads to the same behavior as with CRS v3.3.5.

A further test showed that moving RedirectMatch into a <Location> also leads to a correct redirection as thresholds are then properly initialized.

Proposal

We propose to initialize threshold values in REQUEST-901-INITIALIZATION.conf and in crs-setup.conf.example on phase 3 instead of phase 1.

diff --git a/rules/REQUEST-901-INITIALIZATION.conf b/rules/REQUEST-901-INITIALIZATION.conf
index fe92572c..b8bfecb1 100644
--- a/rules/REQUEST-901-INITIALIZATION.conf
+++ b/rules/REQUEST-901-INITIALIZATION.conf
@@ -85,7 +85,7 @@ SecRule &TX:inbound_anomaly_score_threshold "@eq 0" \
 # Default Outbound Anomaly Threshold Level (rule 900110 in crs-setup.conf)
 SecRule &TX:outbound_anomaly_score_threshold "@eq 0" \
     "id:901110,\
-    phase:1,\
+    phase:3,\
     pass,\
     nolog,\
     tag:'OWASP_CRS',\

index 068884e3..25111c29 100644
--- a/crs-setup.conf.example
+++ b/crs-setup.conf.example
@@ -319,7 +319,7 @@ SecDefaultAction "phase:2,log,auditlog,pass"
 #
 #SecAction \
 #    "id:900110,\
-#    phase:1,\
+#    phase:3,\
 #    pass,\
 #    t:none,\
 #    nolog,\

[airween]

Hi @arminabf,

thanks for reporting.

First of all, could you explain why does it need to install CRS 3.3.5 and upgrade that to 4.2.0? Does it need it just to see the difference?

[airween]

To understand the issue, I think we need your logs - only lines which show the cause of 403. Could you share that?

[arminabf]

Hi @airween

First of all, could you explain why does it need to install CRS 3.3.5 and upgrade that to 4.2.0? Does it need it just to see the difference?

Correct, this is just to see the different behavior between both versions.

[fzipi]

Maybe @studersi wants to chime in here...

[dune73]

This is interesting. I have not recreated this setup and I do not really understand it yet.

@arminabf would you mind sharing your logs for the entire session (access to /, then redirect, then subsequent request.

If this is true, it's a very interesting effect.

I would prefer to keep initialization in phase 1 of course, but if it has to be, then shifting should not mean a problem on other platforms, I guess.

[airween]

@arminabf could you share your logs with us? You send it to security@coreruleset.org, if you don't want/can't share them here.

[arminabf]

Hi,

this is the audit_log entry with v3.3.5

--5842a83c-A--
[30/Apr/2024:06:26:05.770169 +0200] ZjBy3Sh_XeFfdh6zCHQFQQAAAAE ANON
--5842a83c-H--
Message: Warning. Unconditional match in SecAction. [file "/opt/usp/hsp/global/crs/rules/RESPONSE-959-BLOCKING-EVALUATION.conf"] [line "76"] [id "959099"] [msg "Outbound Anomaly Score Exceeded (Total Score: , Threshold: )"] [ver "OWASP_CRS/3.3.5"] [tag "anomaly-evaluation"]
Apache-Error: [file "apache2_util.c"] [line 277] [level 3] [client ANON] ModSecurity: Warning. Unconditional match in SecAction. [file "/crs/rules/RESPONSE-959-BLOCKING-EVALUATION.conf"] [line "76"] [id "959099"] [msg "Outbound Anomaly Score Exceeded (Total Score: , Threshold: )"] [ver "OWASP_CRS/3.3.5"] [tag "anomaly-evaluation"] [hostname "<FQDN>"] [uri "/"] [unique_id "ZjBy3Sh_XeFfdh6zCHQFQQAAAAE"]
Stopwatch: 1714451165765939 4245 (- - -)
Stopwatch2: 1714451165765939 4245; combined=3737, p1=0, p2=0, p3=563, p4=2271, p5=903, sr=0, sw=0, l=0, gc=0
Producer: ModSecurity for Apache/2.9.7 (http://www.modsecurity.org/); OWASP_CRS/3.3.5.
Server: Apache
Sanitised-Response-Headers: "Set-Cookie", "Authorization".
Engine-Mode: "ENABLED"

--5842a83c-Z--

and this is the audit_log of the same request with v4.2.0

--02163f3c-A--
[30/Apr/2024:06:30:06.063530 +0200] ZjBzzuKk5rKhtq_nbqqmzwAAACA ANON
--02163f3c-H--
Message: Access denied with code 403 (phase 4). Operator GE matched 0 at TX:blocking_outbound_anomaly_score. [file "/crs/rules/RESPONSE-959-BLOCKING-EVALUATION.conf"] [line "243"] [id "959100"] [msg "Outbound Anomaly Score Exceeded (Total Score: 0, Threshold: )"] [ver "OWASP_CRS/4.2.0"] [tag "anomaly-evaluation"] [tag "OWASP_CRS"]
Apache-Error: [file "apache2_util.c"] [line 277] [level 3] [client ANON] ModSecurity: Access denied with code 403 (phase 4). Operator GE matched 0 at TX:blocking_outbound_anomaly_score. [file "/crs/rules/RESPONSE-959-BLOCKING-EVALUATION.conf"] [line "243"] [id "959100"] [msg "Outbound Anomaly Score Exceeded (Total Score: 0, Threshold: )"] [ver "OWASP_CRS/4.2.0"] [tag "anomaly-evaluation"] [tag "OWASP_CRS"] [hostname "<FQDN>"] [uri "/"] [unique_id "ZjBzzuKk5rKhtq_nbqqmzwAAACA"]
Action: Intercepted (phase 4)
Stopwatch: 1714451406057018 6520 (- - -)
Stopwatch2: 1714451406057018 6520; combined=5746, p1=0, p2=0, p3=1129, p4=3863, p5=753, sr=0, sw=1, l=0, gc=0
Producer: ModSecurity for Apache/2.9.7 (http://www.modsecurity.org/); OWASP_CRS/4.2.0.
Server: Apache
Sanitised-Response-Headers: "Set-Cookie", "Authorization".
Engine-Mode: "ENABLED"

--02163f3c-Z--

Note that for debugging purposes following rule has been added to v3.3.5

+ SecAction \
+ "id:959099,\
+ phase:4,\
+ pass,\
+ t:none,\
+ msg:'Outbound Anomaly Score Exceeded (Total Score: %{TX.OUTBOUND_ANOMALY_SCORE}, Threshold: %{tx.outbound_anomaly_score_threshold})',\
+ tag:'anomaly-evaluation',\
+ ver:'OWASP_CRS/3.3.5'"

and for v4.2.0 the rule 959100 has been changed following:

- msg:'Outbound Anomaly Score Exceeded (Total Score: %{tx.blocking_outbound_anomaly_score})',\
+ msg:'Outbound Anomaly Score Exceeded (Total Score: %{tx.blocking_outbound_anomaly_score}, Threshold: %{tx.outbound_anomaly_score_threshold})',\

This way the audit_log's clearly show that "Total Score" and "Threshold" are not set for v3.3.5, whereas for v4.2.0 only "Threshold" is not set.

Attached is also detailed information for both requests with "SecDebugLogLevel 9" set.
debug_v3.txt
debug_v4.txt