Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Out of memory error when handling malformed http.Request Body #38

Closed
EugZol opened this issue Feb 13, 2017 · 3 comments
Closed

Out of memory error when handling malformed http.Request Body #38

EugZol opened this issue Feb 13, 2017 · 3 comments

Comments

@EugZol
Copy link
Contributor

EugZol commented Feb 13, 2017

I have the following minimal example:

// main.go
package main

import (
	"net/http"
	"strings"

	"github.com/clbanning/mxj"
)

func main() {
	request, _ := http.NewRequest("POST", "/endpoint", strings.NewReader("not an XML"))
	_, _ = mxj.NewMapXmlReader(request.Body)
}

Running it (go run main.go) gives fatal error: runtime: out of memory after some seconds of freezing.

Full output:

fatal error: runtime: out of memory

runtime stack:
runtime.throw(0x6460b0, 0x16)
        /usr/local/go/src/runtime/panic.go:566 +0x95
runtime.sysMap(0xc4a2210000, 0x82000000, 0x0, 0x769fd8)
        /usr/local/go/src/runtime/mem_linux.go:219 +0x1d0
runtime.(*mheap).sysAlloc(0x750da0, 0x82000000, 0x40dc00)
        /usr/local/go/src/runtime/malloc.go:407 +0x37a
runtime.(*mheap).grow(0x750da0, 0x41000, 0x0)
        /usr/local/go/src/runtime/mheap.go:726 +0x62
runtime.(*mheap).allocSpanLocked(0x750da0, 0x41000, 0xc4200001a0)
        /usr/local/go/src/runtime/mheap.go:630 +0x4f2
runtime.(*mheap).alloc_m(0x750da0, 0x41000, 0x7f0100000000, 0x7fe848aece10)
        /usr/local/go/src/runtime/mheap.go:515 +0xe0
runtime.(*mheap).alloc.func1()
        /usr/local/go/src/runtime/mheap.go:579 +0x4b
runtime.systemstack(0x7fe848aece18)
        /usr/local/go/src/runtime/asm_amd64.s:314 +0xab
runtime.(*mheap).alloc(0x750da0, 0x41000, 0x10100000000, 0xc420010ff0)
        /usr/local/go/src/runtime/mheap.go:580 +0x73
runtime.largeAlloc(0x81ffffff, 0xc420010f01, 0xc42004f120)
        /usr/local/go/src/runtime/malloc.go:774 +0x93
runtime.mallocgc.func1()
        /usr/local/go/src/runtime/malloc.go:669 +0x3e
runtime.systemstack(0xc420020a00)
        /usr/local/go/src/runtime/asm_amd64.s:298 +0x79
runtime.mstart()
        /usr/local/go/src/runtime/proc.go:1079

goroutine 1 [running]:
runtime.systemstack_switch()
        /usr/local/go/src/runtime/asm_amd64.s:252 fp=0xc42004f028 sp=0xc42004f020
runtime.mallocgc(0x81ffffff, 0x5ef6e0, 0xc42004f101, 0x44fb40)
        /usr/local/go/src/runtime/malloc.go:670 +0x903 fp=0xc42004f0c8 sp=0xc42004f028
runtime.makeslice(0x5ef6e0, 0x81ffffff, 0x81ffffff, 0x72d880, 0xc42000c150, 0xc42004f190)
        /usr/local/go/src/runtime/slice.go:57 +0x7b fp=0xc42004f120 sp=0xc42004f0c8
bytes.makeSlice(0x81ffffff, 0x0, 0x0, 0x0)
        /usr/local/go/src/bytes/buffer.go:198 +0x77 fp=0xc42004f160 sp=0xc42004f120
bytes.(*Buffer).grow(0xc42008c730, 0x1, 0x4c00000040fffffe)
        /usr/local/go/src/bytes/buffer.go:106 +0x178 fp=0xc42004f1b0 sp=0xc42004f160
bytes.(*Buffer).WriteByte(0xc42008c730, 0x14c, 0x0, 0x0)
        /usr/local/go/src/bytes/buffer.go:235 +0x3c fp=0xc42004f1d8 sp=0xc42004f1b0
encoding/xml.(*Decoder).text(0xc42008c6e0, 0xffffffffffffffff, 0x0, 0x0, 0x0, 0x0)
        /usr/local/go/src/encoding/xml/xml.go:1076 +0x106 fp=0xc42004f388 sp=0xc42004f1d8
encoding/xml.(*Decoder).rawToken(0xc42008c6e0, 0xc42004fb10, 0x4649a4, 0x600780, 0xc4200109c0)
        /usr/local/go/src/encoding/xml/xml.go:525 +0x2b43 fp=0xc42004f778 sp=0xc42004f388
encoding/xml.(*Decoder).Token(0xc42008c6e0, 0xc420010a20, 0x7, 0xc42004fac0, 0x408334)
        /usr/local/go/src/encoding/xml/xml.go:249 +0x1484 fp=0xc42004f9c0 sp=0xc42004f778
github.com/clbanning/mxj.xmlToMapParser(0x0, 0x0, 0x0, 0x0, 0x0, 0xc42008c6e0, 0x0, 0x5caa11, 0x18, 0x28)
        /home/eugzol/Projects/go/src/github.com/clbanning/mxj/xml.go:311 +0xa5 fp=0xc42004fda0 sp=0xc42004f9c0
github.com/clbanning/mxj.xmlReaderToMap(0x72d8c0, 0xc420010f60, 0x0, 0xc42000cfdf, 0x1, 0x1)
        /home/eugzol/Projects/go/src/github.com/clbanning/mxj/xml.go:140 +0x240 fp=0xc42004fe80 sp=0xc42004fda0
github.com/clbanning/mxj.NewMapXmlReader(0x7fe84aa4e058, 0xc42000d090, 0x0, 0x0, 0x0, 0xc4200cc820, 0xc4200d00f0, 0x0)
        /home/eugzol/Projects/go/src/github.com/clbanning/mxj/xml.go:86 +0x96 fp=0xc42004fee8 sp=0xc42004fe80
main.main()
        /home/eugzol/Projects/go/src/github.com/MyceliumGear/test/main.go:12 +0x104 fp=0xc42004ff48 sp=0xc42004fee8
runtime.main()
        /usr/local/go/src/runtime/proc.go:183 +0x1f4 fp=0xc42004ffa0 sp=0xc42004ff48
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:2086 +0x1 fp=0xc42004ffa8 sp=0xc42004ffa0

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:2086 +0x1
exit status 2

Go version: go version go1.7.1 linux/amd64

@EugZol
Copy link
Contributor Author

EugZol commented Feb 13, 2017

The error doesn't occur in both of the following cases:

  • providing strings.NewReader("not an XML") to NewMapXmlReader
  • using 'valid' XML (like <test>this works fine</test>)

@clbanning
Copy link
Owner

clbanning commented Feb 13, 2017 via email

@EugZol EugZol mentioned this issue Feb 13, 2017
@EugZol
Copy link
Contributor Author

EugZol commented Feb 13, 2017

@clbanning What do you mean by "use-case"?

Just a single file will do with the provided content. Here's the file.

test.zip

I've traced the error to the custom byteReader implementation and was able to fix it — please look at the PR.

clbanning added a commit that referenced this issue Feb 13, 2017
buger pushed a commit to TykTechnologies/tyk that referenced this issue Sep 29, 2017
This was a crash in the mxj library concerning invalid input from a
reader, as it would not properly error and stop. We did not hit this
before in 2.3.x as we used a []byte, not an io.Reader.

The upstream issue is clbanning/mxj#38, fixed
back in February.

Add a regression test and bump the vendored version.

Fixes #1122.
clbanning added a commit that referenced this issue Aug 2, 2022
Return EOF no nil on end of file in ReadByte.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants