Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to build rule exclusions for specific sites on a Multisite server? #3655

Open
Danrancan opened this issue Apr 9, 2024 · 18 comments
Open

Comments

@Danrancan
Copy link

Description

I am running an Ubuntu Based LEMP server. I have Modsecurity crs4.0 installed. The server is serving three websites: https://www.example1.com , https://example.com1, https://www.example2.com , https://example2.com , https://www.example3.com , https://example3.com .

Most of my rule exclusions that are set up apply only to example1.com, then I have one or two rules that apply to example2.com and example3.com. However, all of these rules are being applied across all websites on the server, so example1.com rule exclusions are also being applied to example2.com and example3.com.

How can I apply specific rule exclusions to specific websites/domains so that example1.com's RE's don't also apply to example2.com and example3.com?

I've been following a tutorial from Linuxbabe but I believe it only applies to crs3.3.4, and at that, I don't fully understand it. The following is an excerpt from the tutorial:

Note that if you have multiple applications such as (WordPress, Nextcloud, Drupal, etc) installed on the same server, then the above rule exclusions will be applied to all applications. To minimize the security risks, you should enable a rule exclusion for one application only. To do that, go to the /etc/nginx/modsec/coreruleset-3.3.0/rules/ directory.
cd /etc/nginx/modsec/coreruleset-3.3.4/rules
sudo nano REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf
Add the following line at the bottom of this file. If your WordPress is using the blog.yourdomain.com sub-domain and the request header send from visitor’s browser contains this sub-domain, then ModSecurity will apply the rule exclusions for WordPress.
SecRule REQUEST_HEADERS:Host "@streq blog.yourdomain.com" "id:1000,phase:1,setvar:tx.crs_exclusions_wordpress=1"
If you have installed Nextcloud on the same server, then you can also add the following line in this file, so if a visitor is accessing your Nextcloud sub-domain, ModSecurity will apply the Nextcloud rule exclusions.
SecRule REQUEST_HEADERS:Host "@streq nextcloud.yourdomain.com" "id:1001,phase:1,setvar:tx.crs_exclusions_nextcloud=1"

Can anyone explain how to properly apply RE's to specific sites in CRS4.0?

How to reproduce the misbehavior (-> curl call)

N/A

Logs

N/A

Your Environment

  • CRS version (e.g., v4.0): v4.0
  • Paranoia level setting (e.g. PL1) : PL2
  • ModSecurity version (e.g., 2.9.6): ngx_modsecurity Latest
  • Web Server and version or cloud provider / CDN (e.g., Apache httpd 2.4.54): Nginx 1.25.4 Mainline
  • Operating System and version: Ubuntu 22.04.4 Server for Raspberry Pi 4 (aarch64)

Confirmation

[X ] I have removed any personal data (email addresses, IP addresses,
passwords, domain names) from any logs posted.

@EsadCetiner
Copy link
Member

@Danrancan I'm personally hosting a few web application and I'm doing something similar to what linuxbabe.com is suggesting. First, you'll have to group all of your rule exclusions into rule ID ranges(Not the plugins but the rule exclusions you wrote, I'll cover plugins in a second), for example you may use the 1000 rule id range for WordPress rule exclusions, 2000 for drupal and so on. Just make sure to pick a rule id range that's reserved for internal use to avoid any potential rule id conflicts in the future, maybe you want to use the Comodo ModSecurity rules or Atomicorp rules alongside CRS in the future. Then, you want to create a rule that inspects the host header (The host header is what tells the web server/reverse-proxy what website to serve) and will disable a certain set of rule exclusions that doesn't have the correct host header.
For example:

SecRule REQUEST_HEADERS:Host "!@streq wordpress.example.com" \
    "id:1999,\
    phase:1,\
    pass,\
    t:none,\
    nolog,\
    ctl:ruleRemoveById=1000-1998"   

Note the ! before the streq operator, this means any traffic that's not being sent to wordpress.example.com won't have the wordpress rule exclusions applied.

For the plugins, you'll have to edit the example rule commented out in the plugin's config file (I'll use WordPress as an example here again). We want to do something similar to what we did above, but add a chained rule to the example provided in wordpress-rule-exclusions-before.conf that will check if the host header does not equal wordpress.example.com then disable the plugin.

SecRule &TX:wordpress-rule-exclusions-plugin_enabled "@eq 0" \
    "id:9507010,\
    phase:1,\
    pass,\
    nolog,\
    chain"
    SecRule REQUEST_HEADERS:Host "!@streq wordpress.example.com" \
        "t:none,\
        setvar:'tx.wordpress-rule-exclusions-plugin_enabled=0'"

This is a bit different to what linuxbabe.com is suggesting, but the general idea is still the same.

@Danrancan
Copy link
Author

Danrancan commented Apr 15, 2024

@EsadCetiner EsadCetiner

Thank you so much for your helpful answer. However, I have a few follow up questions regarding the plugins. Specifically, I have three wordpress sites running on the same server, then I have phpmyadmin, roundcube, and netdata also running on that server. In your example you disable the wordpress exclusions plugin for any site that is not equal to wordpress.example.com. You do this with a SecRule

    SecRule REQUEST_HEADERS:Host "!@streq wordpress.example.com" \
        "t:none,\
        setvar:'tx.wordpress-rule-exclusions-plugin_enabled=0'"

Since I have three wordpress sites, I am wondering If I can include all three wordpress sites in that rule as to keep the wordpress plugin exclusive to those threee sites.

Would something like this work?

SecRule &TX:wordpress-rule-exclusions-plugin_enabled "@eq 0" \
    "id:9507010,\
    phase:1,\
    pass,\
    nolog,\
    chain"
    SecRule REQUEST_HEADERS:Host "!@streq wordpress.example.com" \
    SecRule REQUEST_HEADERS:Host "!@streq wordpress.example2.com" \
    SecRule REQUEST_HEADERS:Host "!@streq wordpress.example3.com" \
        "t:none,\
        setvar:'tx.wordpress-rule-exclusions-plugin_enabled=0'"

Or...
would I have to write completely separate rules for each site such as:

SecRule &TX:wordpress-rule-exclusions-plugin_enabled "@eq 0" \
    "id:9507010,\
    phase:1,\
    pass,\
    nolog,\
    chain"
    SecRule REQUEST_HEADERS:Host "!@streq wordpress.example.com" \
        "t:none,\
        setvar:'tx.wordpress-rule-exclusions-plugin_enabled=0'"

and

SecRule &TX:wordpress-rule-exclusions-plugin_enabled "@eq 0" \
    "id:9507011,\
    phase:1,\
    pass,\
    nolog,\
    chain"
    SecRule REQUEST_HEADERS:Host "!@streq wordpress.example2.com" \
        "t:none,\
        setvar:'tx.wordpress-rule-exclusions-plugin_enabled=0'"

and

SecRule &TX:wordpress-rule-exclusions-plugin_enabled "@eq 0" \
    "id:9507012,\
    phase:1,\
    pass,\
    nolog,\
    chain"
    SecRule REQUEST_HEADERS:Host "!@streq wordpress.example3.com" \
        "t:none,\
        setvar:'tx.wordpress-rule-exclusions-plugin_enabled=0'"

Thanks for all of your help and tips!

@EsadCetiner
Copy link
Member

@Danrancan Both options would work fine, but for your first example you need to include the chain action (This tells ModSecurity that you are chaining another rule) and the t:none transformer, otherwise the config will be rejected and Nginx will refuse to start. End result should look like this:

SecRule &TX:wordpress-rule-exclusions-plugin_enabled "@eq 0" \
    "id:9507010,\
    phase:1,\
    pass,\
    nolog,\
    chain"
    SecRule REQUEST_HEADERS:Host "!@streq wordpress.example.com" \
        "t:none,\
        chain"
        SecRule REQUEST_HEADERS:Host "!@streq wordpress.example2.com" \
            "t:none,\
            chain"
            SecRule REQUEST_HEADERS:Host "!@streq wordpress.example3.com" \
                "t:none,\
                setvar:'tx.wordpress-rule-exclusions-plugin_enabled=0'"

Based on the examples you provided, I think it would be cleaner if you use a regular expression for the host header instead to match the 3 domains like so:

SecRule &TX:wordpress-rule-exclusions-plugin_enabled "@eq 0" \
    "id:9507010,\
    phase:1,\
    pass,\
    nolog,\
    chain"
    SecRule REQUEST_HEADERS:Host "!@rx ^wordpress\.example[23]?\.com$" \
        "t:none,\
        setvar:'tx.wordpress-rule-exclusions-plugin_enabled=0'"

If the regular expression ends up looking too messy for you (depending on your setup and familiarity with regular expressions) then maybe it'll make more sense to use multiple rules or chain multiple rules together.

@Danrancan
Copy link
Author

Danrancan commented Apr 23, 2024

@Danrancan Both options would work fine, but for your first example you need to include the chain action (This tells ModSecurity that you are chaining another rule) and the t:none transformer, otherwise the config will be rejected and Nginx will refuse to start. End result should look like this:

SecRule &TX:wordpress-rule-exclusions-plugin_enabled "@eq 0" \
    "id:9507010,\
    phase:1,\
    pass,\
    nolog,\
    chain"
    SecRule REQUEST_HEADERS:Host "!@streq wordpress.example.com" \
        "t:none,\
        chain"
        SecRule REQUEST_HEADERS:Host "!@streq wordpress.example2.com" \
            "t:none,\
            chain"
            SecRule REQUEST_HEADERS:Host "!@streq wordpress.example3.com" \
                "t:none,\
                setvar:'tx.wordpress-rule-exclusions-plugin_enabled=0'"

Based on the examples you provided, I think it would be cleaner if you use a regular expression for the host header instead to match the 3 domains like so:

SecRule &TX:wordpress-rule-exclusions-plugin_enabled "@eq 0" \
    "id:9507010,\
    phase:1,\
    pass,\
    nolog,\
    chain"
    SecRule REQUEST_HEADERS:Host "!@rx ^wordpress\.example[23]?\.com$" \
        "t:none,\
        setvar:'tx.wordpress-rule-exclusions-plugin_enabled=0'"

If the regular expression ends up looking too messy for you (depending on your setup and familiarity with regular expressions) then maybe it'll make more sense to use multiple rules or chain multiple rules together.

Thank you very much! This is of great help. One last thing I'm confused about. In your second example with the regular expressions you have:
SecRule REQUEST_HEADERS:Host "!@rx ^wordpress\.example[23]?\.com$" \
I am pretty bad at regular expressions and have difficulty understanding them. But in that example, I don't see anything telling the RE about the example2.com and example3.com. Is that what [23] means? Should I be inserting example2 and example3 in the "[]" like this?:

SecRule REQUEST_HEADERS:Host "!@rx ^wordpress\.example[example2example3]?\.com$"

If my assumption is correct, I do see a problem that I haven't discussed yet. that is, example2.com should actually be example2.net, and example3.com should be example3.xyz. All three of my domains have different .com's, with one being .xyz and the other being .net. Therefore, based on my former assumption, I think \.example[example2example3]?\.com$" wouldn't work since I actually need a .net and a .xyz. Is this correct?

If so, then I think using your first example in your most recent post would be the most suitable option, is that correct? Sorry, I'm making a lot of assumptions about things I'm not sure about here. But hopefully you can provide some clarification. Thanks again so much for your help and support.

@Danrancan
Copy link
Author

Danrancan commented Apr 23, 2024

SecRule &TX:wordpress-rule-exclusions-plugin_enabled "@eq 0"
"id:9507010,
phase:1,
pass,
nolog,
chain"
SecRule REQUEST_HEADERS:Host "!@Streq wordpress.example.com"
"t:none,
setvar:'tx.wordpress-rule-exclusions-plugin_enabled=0'"

Okay, so I think I've got the rules set up correctly, except I am confused on the rule ID ordering in Modsecurity. I am also using the phpmyadmin plugin, the roundcube plugin, and the wordpress hardening plugin. This is more or less my final draft for my rules, and I have listed all the sites and plugin's accordingly. However, I don't want my exclusive rules for example1.com or example2.net or example3.xyz, to block my plugin rules from activating somehow because of the ordering of the Rule ID's. I'm not positive how Modsec reads the rules and in what order, but I created a set of rules that currently seem to be working. I haven't tested any of the plugin's yet though, but the websites certainly seem to be working with the proper rules.

Do you think you could look over my work and verify that I am doing this correctly with rule ID's in the proper order?

To sumarize, I have rule ID's setup like this:

# All Sites:                    ID=1000-1099
(IP Whitelist Rules go here)

# example1.com: 	      ID=1100-1198
(example1.com Rules (1100-1198) go here)

SecRule REQUEST_HEADERS:Host "!@streq example1.com" \
    "id:1199,\
    phase:1,\
    pass,\
    t:none,\
    nolog,\
    ctl:ruleRemoveById=1100-1198"
# example2.net:           ID=1200-1298
(example2.net Rules (1200-1298) go here)

SecRule REQUEST_HEADERS:Host "!@streq example2.net" \
    "id:1299,\
    phase:1,\
    pass,\
    t:none,\
    nolog,\
    ctl:ruleRemoveById=1200-1298"
# example3.xyz:   ID=1300-1398
(example3.xyz Rules (1300-1398) go here)

SecRule REQUEST_HEADERS:Host "!@streq example3.xyz" \
    "id:1399,\
    phase:1,\
    pass,\
    t:none,\
    nolog,\
    ctl:ruleRemoveById=1300-1398"
# Plugins:          ID=1400-1499
SecRule &TX:wordpress-rule-exclusions-plugin_enabled "@eq 0" \
    "id:1400,\
    phase:1,\
    pass,\
    nolog,\
    chain"
    SecRule REQUEST_HEADERS:Host "!@streq example1.com" \
        "t:none,\
        chain"
        SecRule REQUEST_HEADERS:Host "!@streq example2.net" \
            "t:none,\
            chain"
            SecRule REQUEST_HEADERS:Host "!@streq example3.xyz" \
                "t:none,\
                setvar:'tx.wordpress-rule-exclusions-plugin_enabled=0'"

SecRule &TX:wordpress-hardening-plugin_enabled "@eq 0" \
    "id:1401,\
    phase:1,\
    pass,\
    nolog,\
    chain"
    SecRule REQUEST_HEADERS:Host "!@streq example1.com" \
        "t:none,\
        chain"
        SecRule REQUEST_HEADERS:Host "!@streq example2.net" \
            "t:none,\
            chain"
            SecRule REQUEST_HEADERS:Host "!@streq example3.xyz" \
                "t:none,\
                setvar:'tx.wordpress-hardening-plugin_enabled=0'"

SecRule &TX:roundcube-rule-exclusions-plugin_enabled "@eq 0" \
    "id:1402,\
    phase:1,\
    pass,\
    nolog,\
    chain"
    SecRule REQUEST_HEADERS:Host "!@streq mail.example1.com" \
        "t:none,\
        chain"
        SecRule REQUEST_HEADERS:Host "!@streq mail.example2.net" \
            "t:none,\
            chain"
            SecRule REQUEST_HEADERS:Host "!@streq mail.example3.xyz" \
                "t:none,\
                 setvar:'tx.roundcube-rule-exclusions-plugin_enabled=0'"

SecRule &TX:phpmyadmin-rule-exclusions-plugin_enabled "@eq 0" \
    "id:1403,\
    phase:1,\
    pass,\
    nolog,\
    chain"
    SecRule REQUEST_HEADERS:Host "!@streq pma.example4.cc" \
        "t:none,\
        setvar:'tx.phpmyadmin-rule-exclusions-plugin_enabled=0'"

That is the gist of all my rules. Any help with your eyes is greatly appreciated.

@dune73
Copy link
Member

dune73 commented Apr 23, 2024

I think you guys have this covered.

For the record, I recommend my users work with SecAppId and the corresponding variable this way.

That means you defined an AppID in the VH context and then you group your rule exclusions by the variable and skip accordingly. This effectively removes the need to work with rule ranges for grouping.

@EsadCetiner
Copy link
Member

@Danrancan

I am pretty bad at regular expressions and have difficulty understanding them.

I was there not too long ago so I get where your coming from, I strongly recommend playing with this website to generate regular expressions, it shows you the result of your regular expressions against whatever text you want to match. It will take some practise, but you'll get the hang of it eventually.

But in that example, I don't see anything telling the RE about the example2.com and example3.com. Is that what [23] means? Should I be inserting example2 and example3 in the "[]" like this?:

Your close, the [] is a character set, so it means match one of the characters in this list so it will match either example2.com or example3.com. The ? makes the character set optional, so the regex will match example.com example2.com and example3.com. There is something similar called a capturing group which is () and that does what you think [] does.

If my assumption is correct, I do see a problem that I haven't discussed yet. that is, example2.com should actually be example2.net, and example3.com should be example3.xyz. All three of my domains have different .com's, with one being .xyz and the other being .net. Therefore, based on my former assumption, I think .example[example2example3]?.com$" wouldn't work since I actually need a .net and a .xyz. Is this correct?
If so, then I think using your first example in your most recent post would be the most suitable option, is that correct? Sorry, I'm making a lot of assumptions about things I'm not sure about here. But hopefully you can provide some clarification. Thanks again so much for your help and support.

It could technically work, but the regular expression will get messy and hard to read so in this case, I'd use seperate rules. Your on the right track.

Okay, so I think I've got the rules set up correctly, except I am confused on the rule ID ordering in Modsecurity. I am also using the phpmyadmin plugin, the roundcube plugin, and the wordpress hardening plugin. This is more or less my final draft for my rules, and I have listed all the sites and plugin's accordingly. However, I don't want my exclusive rules for example1.com or example2.net or example3.xyz, to block my plugin rules from activating somehow because of the ordering of the Rule ID's. I'm not positive how Modsec reads the rules and in what order, but I created a set of rules that currently seem to be working. I haven't tested any of the plugin's yet though, but the websites certainly seem to be working with the proper rules.
Do you think you could look over my work and verify that I am doing this correctly with rule ID's in the proper order?

Absolutely! Just make sure that these rules you've created are loaded before the plugin itself(This would be exampleplugin-before.conf), If you look at the documentation for plugins it shows that you should load the configuration for plugins before the -before and -after.

I think you understand the general idea, but one last thing I want to mention is to make sure you assign your custom rule exclusions a large block of rule IDs (about 1000 for each application). This will give you plenty of room to grow and that you don't run out of rule IDs. Some Applications (Like Nextcloud) will trigger a lot of false positives with CRS and you'll have to write a lot of rule exclusions, and you could easily run out of rule IDs with just 100. I don't think it'll happen with the applications your running, but it's better to be prepared.

Please don't hesitate if you have any questions!

@EsadCetiner
Copy link
Member

@dune73

That means you defined an AppID in the VH context and then you group your rule exclusions by the variable and skip accordingly. This effectively removes the need to work with rule ranges for grouping.

Do you mean SecWebAppID?
That sounds interesting, I assume that would take place of the host header I'm using in this example?

@dune73
Copy link
Member

dune73 commented Apr 23, 2024

Yes exactly. Without relying on the client's information for rule routing. Purely server side config.

@dune73
Copy link
Member

dune73 commented Apr 23, 2024

The interesting thing is - IIRC - that WEBAPPID is ready in phase 1. So a perfect fit for this use case.

@EsadCetiner
Copy link
Member

@dune73 I just tested it and it works perfectly, I can confirm this works in phase 1. Wish I knew about this earlier, I think this will be a big help for @Danrancan and he won't have to chain together multiple rules checking the host header or using nasty regular expressions.

@dune73
Copy link
Member

dune73 commented Apr 23, 2024

Happy to help and thanks for the confirmation.

@azurit
Copy link
Member

azurit commented Apr 30, 2024

@Danrancan Ping.

@Danrancan
Copy link
Author

@Danrancan Ping.

Hwy, I'm here. Sometimes I have to take a break from this because I get busy with something else. But I'm here. I do not understannd at all @EsadCetiner's proposal to use SecWebAppID in my rules. I don't know what files he is referencing or how to make adjustments using his login. Is it better to use SecWebAppID or can I just stick with my already running RE's?

@EsadCetiner

Just make sure that these rules you've created are loaded before the plugin itself(This would be exampleplugin-before.conf), If you look at the documentation for plugins it shows that you should load the configuration for plugins before the -before and -after.

I'm not exactly sure what this means, but I'm thinking its something in my main.conf file located in /etc/nginx/modsec/main.conf. Here is my main.conf file below. Is this what you are talking about? Have I done it correctly?

cat /etc/nginx/modsec/main.conf

Include /etc/nginx/modsec/modsecurity.conf
Include /etc/nginx/modsec/crs4.0/crs-setup.conf

# === ModSecurity Plugins before rules
Include /etc/nginx/modsec/crs4.0/plugins/*-config.conf
Include /etc/nginx/modsec/crs4.0/plugins/*-before.conf

# === ModSecurity Core Rule Set Inclusion
Include /etc/nginx/modsec/crs4.0/rules/*.conf

# === Modsecurity Plugins After Rules
Include /etc/nginx/modsec/crs4.0/plugins/*-after.conf

@dune73
Copy link
Member

dune73 commented May 3, 2024

You can stick to what you are doing @Danrancan.

This proposal is just an alternative way to obtain the same functionality. I think it's more elegant, but yours is more or less equivalent.

@EsadCetiner
Copy link
Member

@Danrancan

I'm not exactly sure what this means, but I'm thinking its something in my main.conf file located in /etc/nginx/modsec/main.conf. Here is my main.conf file below. Is this what you are talking about? Have I done it correctly?

Yes that's what I meant, your all good here.

As for the SecWebAppID, it's essentially the same thing I was suggesting earlier with the host header. It works by setting the SecWebAppID directive in each server block context in Nginx and assigning it an value of say, WordPress or Roundcube.

server 
{
    modsecurity_rules 'SecWebAppID roundcube';

    # Whatever you have configured for Nginx
}

Then, instead of checking REQUEST_HEADERS:Host you check WebAppID like so:

SecRule &TX:roundcube-rule-exclusions-plugin_enabled "@eq 0" \
    "id:1402,\
    phase:1,\
    pass,\
    nolog,\
    chain"
    SecRule WebAppID "!@streq roundcube" \
        "t:none,\
        setvar:'tx.roundcube-rule-exclusions-plugin_enabled=0'"

This way, you don't have to use regular expression or chain many rules together. if you ever decide to add/remove/change domains the most you'll have to do is set SecWebAppID and you won't have to touch the ModSecurity rules.

If you find working with host headers easier then feel free to stick to it, but I think you might find SecWebAppID and WebAppID easier since you won't have to use regular expressions or chain multiple rules together.

@fzipi
Copy link
Member

fzipi commented May 29, 2024

Should this be added to the documentation if we don't have it?

@EsadCetiner
Copy link
Member

@fzipi It's a common enough use case, but I don't see it documented anywhere. I think we should also provide an example in the config for all plugins as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants