Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Ruby] Fix: (SIGSEGV) gRPC-Ruby issue on Windows. memory alloc infinite recursion/run out of memory #8195

Merged

Conversation

HannahShiSFB
Copy link
Contributor

On a 32-bit system, sizeof(mem_block) is 8, _upb_arena_alignup(sizeof(mem_block)) is 16.
In https://github.com/protocolbuffers/protobuf/blob/master/ruby/ext/google/protobuf_c/upb.c#L2361, alloc block size is 16384 + 8 bytes,
In https://github.com/protocolbuffers/protobuf/blob/master/ruby/ext/google/protobuf_c/upb.c#L2352, a->head.ptr moves forward 16 bytes.
So in https://github.com/protocolbuffers/protobuf/blob/master/ruby/ext/google/protobuf_c/upb.h#L380, if() is always false, and keeps calling malloc until infinite recursion(O0)/run out of memory(O3)

(gdb) f 16
#16 0x089fb8d0 in upb_arena_allocblock (a=0xa6bf14c, size=16384) at ../../../../ext/google/protobuf_c/upb.c:2361
2361 ../../../../ext/google/protobuf_c/upb.c: No such file or directory.
(gdb) p sizeof(mem_block)
$1 = 8
(gdb) p _upb_arena_alignup(sizeof(mem_block))
$2 = 16
(gdb) p UPB_MAX(size, a->next_block_size)
$3 = 16384
(gdb) f 18
#18 0x089f4d45 in upb_arena_malloc (a=0xa6bf14c, size=16384) at ../../../../ext/google/protobuf_c/upb.h:385
385 ../../../../ext/google/protobuf_c/upb.h: No such file or directory.
(gdb) p h->end - h->ptr
$4 = 16376
(gdb) p size
$5 = 16384

reference: gRPC-Ruby internal issue number b/171421508.
Issue description:
Segfault observed occasionally on Windows, from a cloud logging agent called google-fluentd, which captures logs from different processes on a VM and uses gRPC-Ruby to publish those logs message to a logging service.

How to reproduce:
0.Start a windows VM, e.x. windows 2016 datacenter. RDP into the VM then install the logging agent. Detailed Instruction here: https://cloud.google.com/logging/docs/agent/installation#joint-install

  1. stop Stackdriver Logging service from Services.
  2. clean folder LoggingAgent\Main\pos\winevtlog.pos\worker0\ and folder C:\var\log\google-fluentd\buffers.
  3. start Stackdriver Logging service from Services.
  4. stop Stackdriver Logging service from Services, when some large( over 300kb) log file appears in C:\var\log\google-fluentd\buffers\ folder
  5. run Main\bin\ruby.exe 'Main\bin\fluentd' -c 'fluent.conf' -x stackdriverlogging -vvv
    Then Worker 0 finished with signal SIGSEGV. For any large( over 300kb) log file, SIGSEGV shows by 20% chance.
  6. Once SIGSEGV has shown, as long as the log file remains, SIGSEGV shows every time when LoggingAgent runs.

@google-cla
Copy link

google-cla bot commented Jan 8, 2021

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.


What to do if you already signed the CLA

Individual signers
Corporate signers

ℹ️ Googlers: Go here for more info.

@google-cla google-cla bot added the cla: no label Jan 8, 2021
@HannahShiSFB
Copy link
Contributor Author

@googlebot I signed it!

@google-cla google-cla bot added cla: yes and removed cla: no labels Jan 8, 2021
@haberman haberman merged commit 7165c34 into protocolbuffers:master Jan 8, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants