Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Coturn under attack? #1470

Open
bab5470 opened this issue Apr 8, 2024 · 2 comments
Open

Coturn under attack? #1470

bab5470 opened this issue Apr 8, 2024 · 2 comments

Comments

@bab5470
Copy link

bab5470 commented Apr 8, 2024

We have the following coturn file:

tls-listening-port=5349
listening-ip=10.4.1.62
relay-ip=10.4.1.62
external-ip=186.160.83.74
verbose
use-auth-secret
static-auth-secret=redacted-for-security
userdb=/usr/local/var/db/turndb
realm=turn1.ourdomain.com
user-quota=50
max-bps=3000000
stale-nonce=600
cert=/etc/turnserver/certs/fullchain.pem
pkey=/etc/turnserver/certs/privkey.pem
no-stdout-log
log-file=/var/log/turnserver/turn.log
pidfile="/var/run/turnserver/turnserver.pid"
secure-stun
cli-password=redacted-for-security
no-tlsv1
no-tlsv1_1

And our server logs are filled with this:

56: : session 003000000000000011: realm <turn1.ourdomain.com> user <>: incoming packet message processed, error 401: Unauthorized
56: : session 002000000000000006: realm <turn1.ourdomain.com> user <>: incoming packet message processed, error 401: Unauthorized
56: : session 002000000000000002: realm <turn1.ourdomain.com> user <>: incoming packet message processed, error 401: Unauthorized
56: : session 002000000000000003: realm <turn1.ourdomain.com> user <>: incoming packet message processed, error 401: Unauthorized
56: : session 000000000000000002: realm <turn1.ourdomain.com> user <>: incoming packet message processed, error 401: Unauthorized
56: : session 000000000000000002: realm <turn1.ourdomain.com> user <>: incoming packet message processed, error 401: Unauthorized
56: : session 001000000000000001: realm <turn1.ourdomain.com> user <>: incoming packet message processed, error 401: Unauthorized
56: : session 000000000000000004: realm <turn1.ourdomain.com> user <>: incoming packet message processed, error 401: Unauthorized
56: : session 003000000000000010: realm <turn1.ourdomain.com> user <>: incoming packet message processed, error 401: Unauthorized
56: : session 000000000000000005: realm <turn1.ourdomain.com> user <>: incoming packet message processed, error 401: Unauthorized
56: : session 000000000000000005: realm <turn1.ourdomain.com> user <>: incoming packet message processed, error 401: Unauthorized
56: : session 001000000000000010: realm <turn1.ourdomain.com> user <>: incoming packet message processed, error 401: Unauthorized
56: : session 001000000000000001: realm <turn1.ourdomain.com> user <>: incoming packet message processed, error 401: Unauthorized
56: : session 001000000000000010: realm <turn1.ourdomain.com> user <>: incoming packet message processed, error 401: Unauthorized
56: : session 000000000000000002: realm <turn1.ourdomain.com> user <>: incoming packet message processed, error 401: Unauthorized
56: : session 001000000000000002: realm <turn1.ourdomain.com> user <>: incoming packet message processed, error 401: Unauthorized
56: : session 003000000000000004: realm <turn1.ourdomain.com> user <>: incoming packet message processed, error 401: Unauthorized
56: : session 000000000000000004: realm <turn1.ourdomain.com> user <>: incoming packet message processed, error 401: Unauthorized
56: : session 002000000000000004: realm <turn1.ourdomain.com> user <>: incoming packet message processed, error 401: Unauthorized
56: : session 001000000000000010: realm <turn1.ourdomain.com> user <>: incoming packet message processed, error 401: Unauthorized
56: : session 001000000000000010: realm <turn1.ourdomain.com> user <>: incoming packet message processed, error 401: Unauthorized
56: : session 001000000000000001: realm <turn1.ourdomain.com> user <>: incoming packet message processed, error 401: Unauthorized
56: : session 003000000000000010: realm <turn1.ourdomain.com> user <>: incoming packet message processed, error 401: Unauthorized
56: : session 001000000000000002: realm <turn1.ourdomain.com> user <>: incoming packet message processed, error 401: Unauthorized

It's logging so many entries so fast that the log file fills up the server's disk faster than we can clear it. Now I can turn off logging altogether—it's essentially compromised anyway—but rather than do that, I'd rather block the person chewing up compute/memory resources on my Coturn server.

I have two questions:

  1. How can we determine what IP this is coming from? We have verbose in the log file, but it isn't logging the IP.
  2. Is there some way, with fail2ban or some other approach, we can block IPs with a certain number of 401 unauthorized?

Thanks
Brad

@bab5470
Copy link
Author

bab5470 commented Apr 8, 2024

I see two log verbosity settings: verbose (lowercase) and Verbose (uppercase), which could be more intuitively named. A more typical naming convention for log levels is trace, debug, info, warning, error, fatal so that the verbosity can be increased or decreased.

I tried switching from verbose to Verbose, which gave me more log entries, but I still see no IP that could be used to block malicious traffic.

I also see suggestions in various GitHub issues to log to syslog and filter out the bad log entries. That's certainly better than logging everything to disk and running out of disk space. But logging bogus connections and filtering them out after the fact consumes CPU cycles that could be better used elsewhere and doesn't deter the attacker.

If the attacker can send connection entries fast enough, they can DoS the TURN server. If we could use fail2ban or some other similar technique to outright block the connections, it would make a denial-of-service attack harder.

At this point, I'm inclined to turn off logging entirely or route logs to /dev/null. This is a poor solution as it compromises logging and doesn't deter these malicious connections, but it's easier and less resource-intensive than filtering log data. :-\

@bab5470
Copy link
Author

bab5470 commented Apr 8, 2024

We've decided to comment out both verbose and Verbose. We no longer see the unauthorized connection entries, but we don't seem to get any logging data at all.

At least our servers aren't running out of disk space anymore, but this seems like a really poor solution to the issue.

There ought to be a way to leverage a tool like fail2ban to find unauthorized connection entries in the logs, block those IPs from making future connections, and shield our COTURN servers from attack.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant