Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support X-Debug header that sends sentry report #498

Merged
merged 4 commits into from Jan 14, 2020
Merged

Conversation

bloodearnest
Copy link
Contributor

@bloodearnest bloodearnest commented Dec 20, 2019

This generate a debug-level sentry report for the request.

As a precaution, the header is only respected from a trusted ip address. This provides a secure way to to targeted diagnostic debugging.

This change also reintroduced the X-Sentry-Id header which had been dropped as part of #496, as I figured out how to include that when possible. It's not possible to do so for soft timeouts, as we don't know until after the request is sent. Nor for errors that occur after headers have been sent. However, the sentry report is tagged with the request id, so it's still addressable.

As a precaution, the header is only respected from a trusted ip
address. This provides a secure way to to targeted diagnostic debugging.

This change also reintroduced the X-Sentry-Id header, as I figured out
how to include that when possible. It's not possible to do so for soft
timeouts, as we don't know until after the request is sent. Nor for
errors that occur after headers have been sent.
Copy link
Contributor

@tonysimpson tonysimpson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like X-Forwarded-For is being treated the wrong way round or my understanding of intention is wrong.

Other than that looks good.

Not relevant to the review but I assume we replace untrusted X-Forwarded-For at our entry point but it would be good to check if we aren't sure.

talisker/endpoints.py Outdated Show resolved Hide resolved
ip_str = request.environ.get('CLIENT_IP')
if ip_str is None:
# fallback to werkzeug's handling
ip_str = force_unicode(request.access_route[-1])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the IP of the last Proxy server - is that what was intended?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was, as a paranoid security measure. werkzeug includes a fix to trust up to N proxies, but by default, it only trusts the previous hop, which we emulate here.

So, we generally don't have a proxy that modifies X-Forwarded-For, it just has the client ip in it, so this works, as X-Forwarded-For just contains the clients ip.

If a user sends a forged Forwarded-For, its first in the list, and we still only only trust the one we append. We don't currently strip, but we could do.

If we had a proxy in the middle that did add an entry, yes, we'd need to support this. Werkzeug supports N trusted proxies config, we could do the same (defaults to 1).

Edit: original reply below, it's wrong.

It means we do not have actual client ips, but for our use that doesn't matter - the only thing we need them for is country code, which is resolved (where they do have client ips) at the FE's and added to a header.

In theory, we could/should strip X-Forwarded-For at the frontends, and thus trust the first ip in the chain, but didn't want that to be on by default. Maybe we could add a config option for saying client ip is trusted, off by default.

Note that we do log the full forwarded chain, we just don't trust it.

talisker/wsgi.py Show resolved Hide resolved
Copy link
Contributor

@tonysimpson tonysimpson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel the X-Forwarded-For stuff should probably have a comment in the code but I'm happy with it as is.
LGTM

@bloodearnest bloodearnest merged commit 93034dd into master Jan 14, 2020
@bloodearnest bloodearnest deleted the x-debug branch January 14, 2020 16:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants