Support to ban subnets #927
Comments
|
I'm not sure where on that page it refers to subnets being blacklisted (ignoring /32 as a subnet). Generally only one IP fails to authenticate and subsequently gets blacklisted by fail2ban. Do you have a specific use case you are attempting to achieve? How would the subnet be determined from a single IP failing to log in? |
|
There is no specific use case, just a server in a sensitive area of Kind regards, Am 28.01.2015 um 01:27 schrieb Lee Clemens:
|
|
@K1LLUM1N471 I have a branch doing that, at the moment written relative #716 (as part of increment feature - ban-time-incr within factor plugin for observer). It can be easy configured in ...
# If geo feature enabled (dictionaries "geo.country" or "geo.region" are specified):
# - for each failure this factor appears like a simple "divider" for "maxretry" (inside of "findtime" interval);
# - for time of each ban it is a simple multiplier coefficient besides "bantime.factor" ("jail.Factor"),
# to change default behavior use "ban.Factor" in expression "bantime.formula".
# The larger value of factor leads to faster ban (few failures) and longest time of ban by increase.
# "geo.country" - dictionary to define factor by country of IP address (country:factor pair)
# (using of this method needs a Country, Region, or City database)
#geo.country =
geo.country = default:10
DE:1 RU:2
...Example above means - the factor for each country is 10, however for DE it is 1 and RU it is 2. This feature is configurable to use I will commit it this or next week and let know if as far as. |
|
@K1LLUM1N471 Can you help me understand "block iterated unauthorised access". Do you mean if an IP from RU gets banned, all IP space from RU gets banned? Or if one IP gets banned, for fail2ban to ban the entire subnet it is a member of (as allocated by the RIR)? |
|
@sebres Thanks for info. How reliable is the identification of a IP this way? @leeclemens I guess it is the second case. When one IP got banned, all the other IPs from this subnet got banned, too. For this reason I would prefer to set manually the array of IPs or subnets by the use of a blacklist, which is interpreted by Fail2Ban.
Kind regards, |
Good question, I would say how outdated the geoip database is. But this impacts failures/bans only, so if IP makes no failures - it country will not be taken into account. |
|
I +1 this request. I found that some China subnets work together for ssh-bruteforce : when one IP is banned an other one (very close to the 1st one) continues few minutes after. Regards,Hexasoft |
|
You can try replacing |
|
As I had the same concern as you seem to have, I tried myself at making a special subnet banning system using fail2ban, called fail2ban-subnets, that you can find here. |
|
I see reference by sebres using geoip to modify behaviour based on country of origin. Is this a concept that will be added to fail2ban soon? |
|
What i do every few months and that i'd like to be upstream is permaban full subnets after too many recidives. It goes like it takes like 4_5_20 = 400 attempts before getting a permaban (didnt checked my exacts rules) It works because these guys use whole ip ranges, are very persistant and switch to another ip of their subnet once they are hit by a recidive rule. tbh 99 of the ips are from china, it s just that i thought it was too discrimnatory to geoban china iptables-save | grep fail2ban |grep '/32' | wc -l I ban /16 if there s too much /24 banned as reported by a whois |
|
Using @justabaka Idea, this works for me: actionban = whois <ip> | grep route: | awk '{print $2}' | xargs -n1 -I{} iptables -I fail2ban-<name> 1 -s {} -j <blocktype> actionunban = whois <ip> | grep route: | awk '{print $2}' | xargs -n1 -I{} iptables -D fail2ban-<name> 1 -s {} -j <blocktype> |
|
@rodrigorojasmoraleda: this solution is really interesting, but unfortunately not generic enough. Indeed, in my experience, I've seen that each whois server replies with its own format. The string "route:" does not necessarily appear in the answer. Thus many IP will get through your fail2ban. For instance, using the 3 last IP addresses that attacked my companies' servers: I've seen whois answers with "CIDR" or "inetnum" or "route", and generally they do not use the / notation, but an ip range :{ I'm sure it's possible to write a rule to manage all possible cases, but it would require to know them all. Also since the rule would be complicated, it would probably be better to wrap it inside a module/plug-in. |
|
When I use : So removed 1 then it is ok. (Can anyone give some comment for this?) Also found that results of whois use different format & keyword by each hosting carrier such as route/CIDR/inetnum IPv4 etc. iptable-allports.conf & iptable-multiport.conf will be better if those are included in the module/plug-in. |
|
Right now (Fail2Ban v0.9.6) we have It could get into the database: And finally: So, should it be closed now? |
|
My point of view on the original issue and need for this, and maybe that can help decide if you can close it. Some hackers have controls of /24 and /16. The pattern is that if fail2ban bans an ip, their tools will understand this, and switch to another ip try to continue to connect. In these cases, fail2ban is of little uses because it ban individual ip. So what is needed is to ban the whole block to stop it. Personally one thing i would like to see before the issue is closed is more integration. Like what has been done for recidive (no need to fiddle yourself in the logs, you can just enable recidive in fail2ban conf). Maybe allow to enable actions/detection to manage things as subnet. |
|
Good idea. |
|
My process is manual. But what about toreit script to manage subnets? Also maybe with an automated process, it s not necessary to handle anything else than /24 range as x.x.x.0-255 which are 90% of these ban anyway. As fail2ban would handle the ban in realtime and not every few months like me. I permaban after about 400 attempts. |
|
FYI, During 10 days, I got results using my script above. fail2ban-client status sshd & recidive iptables -L -n |
|
As I see the conversation is still running here, just a following comment to my previous one #927 (comment) to say that I'm still using my fail2ban-subnets tool on a few servers and that it still work to protect them. I am also still actively supporting it to improve it as needed. |
|
I agree with XaF, I'm also still actively supporting an improvement of the subnet banning feature. What I would see as important features:
|
|
Please do support ban by network number. |
|
Reading over this I don't think I saw this suggested -- would it make more sense rather than banning a /24 when a particular /32 within the /24 hits a threshold to instead keep track of the entire /24 and when the aggregate /24 hits the threshold, ban the /24? In a sense you would be treating the /24 as a single IP. |
|
I support @rrauenza's suggestion as an option alongside single-IP banning -- what would be REALLY great is an option to track failures on a given /24 subnet (or, even better, the entire CIDR determined from So, for example, one could have: so that a given IP is banned for 3 failures in 10 minutes, but an entire subnet is banned for 30 failures (from that subnet) in 1 week even if no single IP within that subnet ever met the perIP ban criteria. This can help to combat these "slow burn" distributed botnets that use multiple IPs per subnet. I wouldn't recommend JUST tracking the subnet because you don't want to ban an entire subnet when only one IP has failed... but tracking both single-IP and subnet, with different findtime/maxretry for each, allows knocking out single-IP attacks on a short-term basis while simultaneously handling subnet attacks longer-term. |
|
thank you for your help, there is something what I do not understand No entry for 134.73.51 No entry for 134.73.51Chain INPUT (policy ACCEPT)) and maybe it is a good idea to add in recidive.conf but the subnets jail is definitvely working |
of course there is no entry, your chain (where fail2ban would add it) is called |
|
.... I will go walking for a while to cool down my brain ... So the dirty workaround is working fine .... |
:) In my own observer module (I have still to back-port several things to f2b to make it public, but no time ATM) I use our sqlite-database to determine a subnet, and use there a statement like this (basically I have not the bans only, but also failures, just current f2b-version does not have it): select jail, rtrim(ip, replace(ip, '.', '')) as 'cidrp', count(ip) as 'tickets'
from bans
-- where enofban > now:
where datetime(timeofban + bantime, 'unixepoch', 'localtime') > datetime('now', 'localtime')
group by jail, cidrp
having tickets >= 5
order by tickets desc, jailwhich retrieves the jails with
Of course you can use it from batch (shell) using something like that: Note: open it readonly to avoid blocking of fail2ban to long time. In scenario like this you would not need new jail and new action: foreach jail, cidrp [from_statement_result] {
fail2ban-client set $jail banip $cidrp.0/24
}Proper solution should still unban all IPs matching this cidr/mask, but we're speaking about a workaround here. |
|
Thank you very much, I will try this - better than re-parsing logs |
|
Just to remind that @XaF has a very similar script that will find problem subnets and ban the entire /24, see #927 (comment). @toreit also posted a script, see #927 (comment). Perhaps these can help. |
select jail, rtrim(ip, replace(ip, '.', '')) as 'cidrp', count(ip) as 'tickets'
from bans
-- where enofban > now:
where datetime(timeofban + bantime, 'unixepoch', 'localtime') > datetime('now', 'localtime')
group by jail, cidrp
having tickets >= 5
order by tickets desc, jailbut .. i do not need the bans, because they bypass all ban rules with the Class C net. |
But I told you already in the description: "basically I have not the bans only, but also failures, just current f2b-version does not have it.", so all my statements finding single failures would not help you with your stock fail2ban. Well, one possibility were v.0.11.1 with enabled Another possibility would be an enhancement like requested in #2304 (which is not yet ready too at the moment). If I get time, I'll try to fulfill a back-porting of my observer to provide regular solution for that. |
|
so many people talking about whois, why would any of you use whois?!? whois is completely useless! I have never used more than a single IP address (yes it changes every few months; irrelevant), yet whois always says that I am part of a '/24' subnet (or larger)! #927 (comment) here is the simplest solution that doesn't depend on whois you do not treat a subnet like a single ip address without adjusting the "maxretry"'s as well... so just monitor every subnet level! where The use of .95 as an upper bound would have to be increased based on the number of subnet sizes that are considered to avoid false positives. for example: As the size of the subnet increases Since the ratio decreases as the subnet size increases, you should be concerned that a large subnet would get banned based on the actions of only a few smaller subnets, but keep in mind that the upper bound of the binomial distribution can be approximated by We should also change maxretry into more of a statistical distribution, because the odds of all customers within a subnet entering wrong passwords in the same findtime period is exponentially small. maxretry should instead be based on two configuration parameters avgretry_perIP and maxretry_perIP(where maxretry_perIP is considered the upper 95/99/etc.% tolerance interval of the distribution given by the number of events in a single findtime period with avgretry_perIP being the average number of events in that findtime period), that way we can reduce the expected number of retries based on subnet size. This can be done with an inverse poisson/gamma distribution via. I hope this addresses everyone's concerns and that we can now focus on providing a flexible solution that requires only two new configuration parameters: density and avgretry_perIP; without requiring the tedious coding of whois parsing for every single possible whois format that can be provided, or the code upkeep required to accommodate new formats. I'm not familiar enough with IPv6, so treating '/64' in the same way as an individual IPv4 address may be invalid... and may require something like whois or a third configuration parameter. |
FWIW, just to clarify our intention, no one of us had really planned to use whois to identify a subnet (at most to determine and cache its country possibly, even to provide mechanism to increase ban factor by non target audience e. g. do a factorization depending on countries configuration). But current main problem is not a recognition of a net, rather it is a good implementation of conjoining several banned tickets after such determination to the single ticket with resulting adjustment of all properties and reban them all as a single entry.
This is also not an issue at all, because v.0.11 already uses a formula to manipulate
As already said this will be rather secondary. Primary it is internal handling inside ban-manager and actions. And of course the time to implement. Because it is always a scarce resource. |
For me it would be enough if the first IP that matches some whois or geoip criteria would start a ban of the whole /24 in which it resides and then the ban is hold as long as any IP in the range still is banned and when the last IP of the /24 is released then the whole range is released. I tried to do that with ugly ban and unban scripts but it is easy to get out of sync when much ban/unban action is going on in a subnet. Harald. |
Well conditionally it is possible right now without any modification of fail2ban core functionality.
If you ban a subnet you should have exactly one entry (single IP from the subnet) because other IPs would not be able to connect anymore. Single exception is if some IPs starting their attempts (and so generating failures) simultaneously, so one have to ignore the same banned subnet from following addresses. Also if the IP is not expanded to a subnet in fail2ban core (but in action only), ban-time-increment feature is impossible (because also applied for single IP only). So better would be to implement something like |
|
Since I setup my internet linux server, and checked the fail2ban logs, I deserve a solution for banning complete subnets. |
|
https://sven.rojek.de/posts/fail2ban-iprange-mit-blackliste-blocken looks like a good solution with plain fail2ban (article in german but english config snippets should make the approach clear). |
|
Just to add to some of the other solutions. I find it easiest to use below script that I hook into my actionban statements to capture CIDR from whois (yes, I prefer that over guessing some CIDR, but you could also skip most of it and stick /24 after the host) and sending it to syslog. That way I can setup a new jail to monitor those syslog entries with tweaked values for maxretry and findtime. Here's the python3 script I use (pure python): |
On the subject of determining when a group of offending IPs from a subnet have crossed a threshold for a /24 block.. |
Adding the "flags interval;" in addr-set definition enables the manual ban of ip addresses interval using a subnet mask (eg: 10.0.0.0/24). Relates to: - fail2ban#927
|
Distributed brute force attacks are really becoming an issue here. I need a simple solution to detect the following.
Unfortunately, this would clearly allow for denial of service, as there are some very large essentially public blocks and the customer is likely to be in one of them. So we need an exception for netblocks that have registered recent successful logins, to avoid locking out the customer. |
and others
Hello,
in which case does fail2ban support to ban subnets.
I used the instruction from http://cup.wpcoder.de/fail2ban-ip-blacklist/ and entered the subnet instead of the single IP, but fail2ban seems not to handle them at all.
Kind regards,
K1LLUM1N471
The text was updated successfully, but these errors were encountered: