Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(engine): add trustedproxies and remoteIP #2632

Merged
merged 24 commits into from Apr 6, 2021
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
de0f9eb
X-Forwarded-For handling is unsafe
sorenisanerd Aug 20, 2020
b343e7e
Merge branch 'master' into sorenh/issue2473
appleboy Jan 4, 2021
b7a35bf
Merge branch 'master' into sorenh/issue2473
thinkerou Jan 11, 2021
d0bf406
Merge branch 'master' into sorenh/issue2473
appleboy Jan 23, 2021
31a2afa
Merge branch 'sorenh/issue2473' of https://github.com/sorenh/gin into…
manucorporat Feb 7, 2021
c9ea8ec
refactor(): fix client Ip vulnerability
manucorporat Feb 7, 2021
39b372f
chore(engine): remove unused lookupHost
javierprovecho Feb 8, 2021
e14a43c
fix(engine): wrong syntax/behaviour at prepareCIDR
javierprovecho Feb 8, 2021
55ad88a
refactor move logic to remoteIP()
manucorporat Feb 8, 2021
6f562ea
add docs
manucorporat Feb 8, 2021
ffe7ac0
🐈
manucorporat Feb 8, 2021
2d426b6
add explanation 🦀
manucorporat Feb 8, 2021
fc99953
fix 🐨
manucorporat Feb 8, 2021
15f576c
docs(readme): remove old reference to hostnames
javierprovecho Feb 8, 2021
ba157c9
Merge branch 'master' into fix-client-ip-cve
manucorporat Feb 9, 2021
a598663
docs(engine): explain new gin.Engine properties
javierprovecho Feb 9, 2021
7e649c3
feat(engine): add ipv6 support to TrustedProxies
javierprovecho Feb 10, 2021
0876678
fix error
thinkerou Mar 27, 2021
56fbadc
Merge branch 'master' into fix-client-ip-cve
thinkerou Mar 27, 2021
feaee20
add unit test
thinkerou Mar 27, 2021
2d7fc06
add unit test
thinkerou Mar 27, 2021
3483d2c
remove nil statement
thinkerou Mar 27, 2021
65711ec
Merge branch 'master' into fix-client-ip-cve
appleboy Apr 4, 2021
9018d58
Merge branch 'master' into fix-client-ip-cve
appleboy Apr 6, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
37 changes: 37 additions & 0 deletions README.md
Expand Up @@ -2117,6 +2117,43 @@ func main() {
}
```

## Don't trust all proxies

Gin lets you specify which headers to hold the real client IP (if any),
as well as specifying which proxies (or direct clients) you trust to
specify one of these headers.

The `TrustedProxies` slice on your `gin.Engine` specifes the clients
truest to specify unspoofed client IP headers. Proxies can be specified
as IP's, CIDR's, or hostnames. Hostnames are resolved on each query,
such that changes in your proxy pool take effect immediately. The
hostname option is handy, but also costly, so only use if you have no
other option.

```go
import (
"fmt"

"github.com/gin-gonic/gin"
)

func main() {

router := gin.Default()
router.TrustedProxies = []string{"192.168.1.2"}

router.GET("/", func(c *gin.Context) {

// If the client is 192.168.1.2, use the X-Forwarded-For
// header to deduce the original client IP from the trust-
// worthy parts of that header.
// Otherwise, simply return the direct client IP
fmt.Printf("ClientIP: %s\n", c.ClientIP())
})

router.Run()
}
manucorporat marked this conversation as resolved.
Show resolved Hide resolved
```

## Testing

Expand Down
57 changes: 45 additions & 12 deletions context.go
Expand Up @@ -729,15 +729,13 @@ func (c *Context) ShouldBindBodyWith(obj interface{}, bb binding.BindingBody) (e
// X-Real-IP and X-Forwarded-For in order to work properly with reverse-proxies such us: nginx or haproxy.
// Use X-Forwarded-For before X-Real-Ip as nginx uses X-Real-Ip with the proxy's IP.
func (c *Context) ClientIP() string {
if c.engine.ForwardedByClientIP {
clientIP := c.requestHeader("X-Forwarded-For")
clientIP = strings.TrimSpace(strings.Split(clientIP, ",")[0])
if clientIP == "" {
clientIP = strings.TrimSpace(c.requestHeader("X-Real-Ip"))
}
if clientIP != "" {
return clientIP
}
ip, _, err := net.SplitHostPort(strings.TrimSpace(c.Request.RemoteAddr))
if err != nil {
return ""
}
remoteIP := net.ParseIP(ip)
if remoteIP == nil {
return ""
}

if c.engine.AppEngine {
Expand All @@ -746,11 +744,46 @@ func (c *Context) ClientIP() string {
}
}

if ip, _, err := net.SplitHostPort(strings.TrimSpace(c.Request.RemoteAddr)); err == nil {
return ip
if c.shouldCheckIPHeaders() {
for _, cidr := range c.engine.trustedCIDRs {
if cidr.Contains(remoteIP) {
for _, headerName := range c.engine.RemoteIPHeaders {
ip, valid := validateHeader(c.requestHeader(headerName))
if valid {
return ip
}
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

			if !cidr.Contains(remoteIP) {
                             continue
                         }
			for _, headerName := range c.engine.RemoteIPHeaders {
				ip, valid := validateHeader(c.requestHeader(headerName))
				if valid {
					return ip
				}
			}

suggestion, thanks!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

refactored it a little bit, review again... now thee logic is better separated between validating the trusted proxy and parsing the header, also a nw AP

ip, trusted = c.RemoteIP()

allows to even implement your own logic, or trust othe headers that might not be even related with IP!

Copy link
Contributor

@agmt agmt Aug 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you expect that HTTP proxy running on c.RemoteIP() resets X-Forwarded-For? Because if it appends, then we can't inherit trustiness of c.RemoteIP() to all other proxies.

}
}

return ""
return remoteIP.String()
}

func (c *Context) shouldCheckIPHeaders() bool {
return c.engine.ForwardedByClientIP &&
c.engine.RemoteIPHeaders != nil &&
len(c.engine.RemoteIPHeaders) > 0 &&
c.engine.trustedCIDRs != nil
}

func validateHeader(header string) (clientIP string, valid bool) {
if header == "" {
return
manucorporat marked this conversation as resolved.
Show resolved Hide resolved
}
items := strings.Split(header, ",")
for i, ipStr := range items {
ipStr = strings.TrimSpace(ipStr)
ip := net.ParseIP(ipStr)
if ip == nil {
return "", false
}
if i == 0 {
clientIP = ipStr
valid = true
}
}
return
}

// ContentType returns the Content-Type header of the request.
Expand Down
88 changes: 84 additions & 4 deletions context_test.go
Expand Up @@ -1392,11 +1392,23 @@ func TestContextClientIP(t *testing.T) {
c, _ := CreateTestContext(httptest.NewRecorder())
c.Request, _ = http.NewRequest("POST", "/", nil)

c.Request.Header.Set("X-Real-IP", " 10.10.10.10 ")
c.Request.Header.Set("X-Forwarded-For", " 20.20.20.20, 30.30.30.30")
c.Request.Header.Set("X-Appengine-Remote-Addr", "50.50.50.50")
c.Request.RemoteAddr = " 40.40.40.40:42123 "
c.engine.lookupHost = func(host string) ([]string, error) {
if host == "foo" {
return []string{"40.40.40.40", "30.30.30.30"}, nil
}
if host == "bar" {
return nil, errors.New("hostname lookup failed")
}
if host == "baz" {
return []string{"thisshouldneverhappen"}, nil
}
return []string{}, nil
}

resetContextForClientIPTests(c)

// Legacy tests (validating that the defaults don't break the
// (insecure!) old behaviour)
assert.Equal(t, "20.20.20.20", c.ClientIP())

c.Request.Header.Del("X-Forwarded-For")
Expand All @@ -1416,6 +1428,74 @@ func TestContextClientIP(t *testing.T) {
// no port
c.Request.RemoteAddr = "50.50.50.50"
assert.Empty(t, c.ClientIP())

// Tests exercising the TrustedProxies functionality
resetContextForClientIPTests(c)

// No trusted proxies
c.engine.TrustedProxies = []string{}
c.engine.RemoteIPHeaders = []string{"X-Forwarded-For"}
assert.Equal(t, "40.40.40.40", c.ClientIP())

// Last proxy is trusted, but the RemoteAddr is not
c.engine.TrustedProxies = []string{"30.30.30.30"}
assert.Equal(t, "40.40.40.40", c.ClientIP())

// Only trust RemoteAddr
c.engine.TrustedProxies = []string{"40.40.40.40"}
assert.Equal(t, "30.30.30.30", c.ClientIP())

// All steps are trusted
c.engine.TrustedProxies = []string{"40.40.40.40", "30.30.30.30", "20.20.20.20"}
assert.Equal(t, "20.20.20.20", c.ClientIP())

// Use CIDR
c.engine.TrustedProxies = []string{"40.40.25.25/16", "30.30.30.30"}
assert.Equal(t, "20.20.20.20", c.ClientIP())

// Use hostname that resolves to all the proxies
c.engine.TrustedProxies = []string{"foo"}
assert.Equal(t, "20.20.20.20", c.ClientIP())

// Use hostname that returns an error
c.engine.TrustedProxies = []string{"bar"}
assert.Equal(t, "40.40.40.40", c.ClientIP())

// X-Forwarded-For has a non-IP element
c.engine.TrustedProxies = []string{"40.40.40.40"}
c.Request.Header.Set("X-Forwarded-For", " blah ")
assert.Equal(t, "40.40.40.40", c.ClientIP())

// Result from LookupHost has non-IP element. This should never
// happen, but we should test it to make sure we handle it
// gracefully.
c.engine.TrustedProxies = []string{"baz"}
c.Request.Header.Set("X-Forwarded-For", " 30.30.30.30 ")
assert.Equal(t, "40.40.40.40", c.ClientIP())

c.engine.TrustedProxies = []string{"40.40.40.40"}
c.Request.Header.Del("X-Forwarded-For")
c.engine.RemoteIPHeaders = []string{"X-Forwarded-For", "X-Real-IP"}
assert.Equal(t, "10.10.10.10", c.ClientIP())

c.engine.RemoteIPHeaders = []string{}
c.engine.AppEngine = true
assert.Equal(t, "50.50.50.50", c.ClientIP())

c.Request.Header.Del("X-Appengine-Remote-Addr")
assert.Equal(t, "40.40.40.40", c.ClientIP())

// no port
c.Request.RemoteAddr = "50.50.50.50"
assert.Empty(t, c.ClientIP())
}

func resetContextForClientIPTests(c *Context) {
c.Request.Header.Set("X-Real-IP", " 10.10.10.10 ")
c.Request.Header.Set("X-Forwarded-For", " 20.20.20.20, 30.30.30.30")
c.Request.Header.Set("X-Appengine-Remote-Addr", "50.50.50.50")
c.Request.RemoteAddr = " 40.40.40.40:42123 "
c.engine.AppEngine = false
}

func TestContextContentType(t *testing.T) {
Expand Down
32 changes: 32 additions & 0 deletions gin.go
Expand Up @@ -11,6 +11,7 @@ import (
"net/http"
"os"
"path"
"strings"
"sync"

"github.com/gin-gonic/gin/internal/bytesconv"
Expand Down Expand Up @@ -82,6 +83,8 @@ type Engine struct {
// handler.
HandleMethodNotAllowed bool
ForwardedByClientIP bool
RemoteIPHeaders []string
TrustedProxies []string

// #726 #755 If enabled, it will thrust some headers starting with
// 'X-AppEngine...' for better integration with that PaaS.
Expand All @@ -103,6 +106,8 @@ type Engine struct {
// See the PR #1817 and issue #1644
RemoveExtraSlash bool

lookupHost func(string) ([]string, error)

delims render.Delims
secureJSONPrefix string
HTMLRender render.HTMLRender
Expand All @@ -114,6 +119,7 @@ type Engine struct {
pool sync.Pool
trees methodTrees
maxParams uint16
trustedCIDRs []*net.IPNet
}

var _ IRouter = &Engine{}
Expand All @@ -139,11 +145,14 @@ func New() *Engine {
RedirectFixedPath: false,
HandleMethodNotAllowed: false,
ForwardedByClientIP: true,
RemoteIPHeaders: []string{"X-Forwarded-For", "X-Real-IP"},
TrustedProxies: []string{"0.0.0.0/0"},
AppEngine: defaultAppEngine,
UseRawPath: false,
RemoveExtraSlash: false,
UnescapePathValues: true,
MaxMultipartMemory: defaultMultipartMemory,
lookupHost: net.LookupHost,
trees: make(methodTrees, 0, 9),
delims: render.Delims{Left: "{{", Right: "}}"},
secureJSONPrefix: "while(1);",
Expand Down Expand Up @@ -305,12 +314,35 @@ func iterate(path, method string, routes RoutesInfo, root *node) RoutesInfo {
func (engine *Engine) Run(addr ...string) (err error) {
defer func() { debugPrintError(err) }()

trustedCIDRs, err := engine.prepareCIDR()
if err != nil {
return err
}
engine.trustedCIDRs = trustedCIDRs
address := resolveAddress(addr)
debugPrint("Listening and serving HTTP on %s\n", address)
err = http.ListenAndServe(address, engine)
return
}

func (engine *Engine) prepareCIDR() ([]*net.IPNet, error) {
if engine.TrustedProxies != nil {
cidr := make([]*net.IPNet, len(engine.TrustedProxies), 0)
for _, trustedProxy := range engine.TrustedProxies {
if strings.Contains(trustedProxy, "/") {
javierprovecho marked this conversation as resolved.
Show resolved Hide resolved
trustedProxy += "/32"
}
_, cidrNet, err := net.ParseCIDR(trustedProxy)
if err != nil {
return cidr, err
}
cidr = append(cidr, cidrNet)
}
return cidr, nil
}
return nil, nil
}

// RunTLS attaches the router to a http.Server and starts listening and serving HTTPS (secure) requests.
// It is a shortcut for http.ListenAndServeTLS(addr, certFile, keyFile, router)
// Note: this method will block the calling goroutine indefinitely unless an error happens.
Expand Down