Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SEGFAULTs consistently happening on CI #740

Open
lcobucci opened this issue Feb 28, 2023 · 8 comments · May be fixed by #741
Open

SEGFAULTs consistently happening on CI #740

lcobucci opened this issue Feb 28, 2023 · 8 comments · May be fixed by #741
Labels
bug help wanted

Comments

@lcobucci
Copy link

On #737 (review), @asgrim mentions that SEGFAULTs are constantly haunting this project.

I've done some digging around and managed to reproduce the issue!

Although I couldn't pinpoint exactly the problem (on the timebox I've allocated for this), it seems related to Psl\Shell\execute() and PHPUnit ANSI's formatter (when under a very specific environment).

The main trigger, though, is @Ocramius' good old friend: lack of Git's safe.directory configuration 😬

We can use Docker to reproduce the SEGFAULTs.
I've done with the following Dockerfile:

FROM ubuntu:latest

ENV DEBIAN_FRONTEND noninteractive

RUN apt-get update \
    && apt-get install -y --reinstall software-properties-common \
    && add-apt-repository -y ppa:ondrej/php \
    && apt-get install -y php8.1  php8.1-bz2 php8.1-cli php8.1-common php8.1-curl php8.1-dev php8.1-intl php8.1-mbstring php8.1-opcache php8.1-phpdbg php8.1-readline php8.1-xml  php8.1-zip php8.1-bcmath gdb libxml2-utils git

COPY --from=composer /usr/bin/composer /usr/bin/composer

CMD ["/bin/sh", "-c", "composer install && xmllint --schema vendor/phpunit/phpunit/phpunit.xsd phpunit.xml.dist && vendor/bin/phpunit"]

Then, we can see the problem after executing the following commands on the root dir of this project (against the latest commit of 8.4.x):

$ docker build -t bc-segfault-test . && docker run --rm -it -v $(pwd):/app -w /app bc-segfault-test

The "fun" begins when we start playing with the commands...
For example, no SEGFAULT happens when running any of these (after the previous commands):

$ docker run --rm -it -v $(pwd):/app -w /app bc-segfault-test vendor/bin/phpunit --testdox
$ docker run --rm -it -v $(pwd):/app -w /app bc-segfault-test vendor/bin/phpunit --colors=never
$ docker run --rm -v $(pwd):/app -w /app bc-segfault-test vendor/bin/phpunit
$ docker run --rm -it -v $(pwd):/app -w /app bc-segfault-test /bin/bash -c "git config --global --add safe.directory /app && vendor/bin/phpunit"
GBD

We can also use that image to play with GDB:

$ docker run --rm -it -v $(pwd):/app -w /app bc-segfault-test gdb php
GNU gdb (Ubuntu 12.1-0ubuntu1~22.04) 12.1
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from php...
(No debugging symbols found in php)
(gdb) run vendor/bin/phpunit
Starting program: /usr/bin/php vendor/bin/phpunit
warning: Error disabling address space randomization: Operation not permitted
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[Detaching after vfork from child process 23]
PHPUnit 9.5.27 by Sebastian Bergmann and contributors.

..................................[Detaching after fork from child process 25]
.[Detaching after fork from child process 26]
.[Detaching after fork from child process 27]
.[Detaching after fork from child process 28]
.[Detaching after fork from child process 29]
.[Detaching after fork from child process 30]
.[Detaching after fork from child process 31]
.[Detaching after fork from child process 32]
.[Detaching after fork from child process 33]
.[Detaching after fork from child process 34]
....................  63 / 758 (  8%)
............................................................... 126 / 758 ( 16%)
............................................................... 189 / 758 ( 24%)
............................................................... 252 / 758 ( 33%)
............................................................... 315 / 758 ( 41%)
............................................................... 378 / 758 ( 49%)
............................................................... 441 / 758 ( 58%)
............................................................... 504 / 758 ( 66%)
............................................................... 567 / 758 ( 74%)
............................................................... 630 / 758 ( 83%)
.........................................................[Detaching after fork from child process 35]
[Detaching after fork from child process 36]
[Detaching after fork from child process 37]
[Detaching after fork from child process 38]
[Detaching after fork from child process 39]
[Detaching after fork from child process 40]
[Detaching after fork from child process 41]
[Detaching after fork from child process 43]
[Detaching after fork from child process 44]
[Detaching after fork from child process 45]
.[Detaching after fork from child process 46]
[Detaching after fork from child process 47]
[Detaching after fork from child process 48]
[Detaching after fork from child process 49]
[Detaching after fork from child process 50]
[Detaching after fork from child process 51]
[Detaching after fork from child process 52]
[Detaching after fork from child process 54]
[Detaching after fork from child process 55]
[Detaching after fork from child process 56]
[Detaching after fork from child process 57]
[Detaching after fork from child process 58]
.[Detaching after fork from child process 59]
[Detaching after fork from child process 60]
[Detaching after fork from child process 61]
[Detaching after fork from child process 62]
[Detaching after fork from child process 63]
[Detaching after fork from child process 64]
[Detaching after fork from child process 65]
[Detaching after fork from child process 67]
[Detaching after fork from child process 68]
[Detaching after fork from child process 69]
.[Detaching after fork from child process 70]
[Detaching after fork from child process 73]
[Detaching after fork from child process 74]
.[Detaching after fork from child process 75]
[Detaching after fork from child process 78]
[Detaching after fork from child process 79]
[Detaching after fork from child process 82]
[Detaching after fork from child process 83]
[Detaching after fork from child process 84]
.[Detaching after fork from child process 85]
[Detaching after fork from child process 86]
[Detaching after fork from child process 87]
[Detaching after fork from child process 88]
[Detaching after fork from child process 89]
[Detaching after fork from child process 90]
[Detaching after fork from child process 92]
[Detaching after fork from child process 93]
[Detaching after fork from child process 94]
[Detaching after fork from child process 96]
[Detaching after fork from child process 97]
[Detaching after fork from child process 100]
[Detaching after fork from child process 101]
[Detaching after fork from child process 104]
[Detaching after fork from child process 105]
[Detaching after fork from child process 106]
[Detaching after fork from child process 107]
. 693 / 758 ( 91%)
[Detaching after fork from child process 108]
[Detaching after fork from child process 111]
[Detaching after fork from child process 112]
.[Detaching after fork from child process 113]
E.........................................................[Detaching after fork from child process 114]
[Detaching after fork from child process 115]
[Detaching after fork from child process 116]
[Detaching after fork from child process 117]
[Detaching after fork from child process 118]
[Detaching after fork from child process 119]
[Detaching after fork from child process 120]
[Detaching after fork from child process 122]
[Detaching after fork from child process 123]
[Detaching after fork from child process 125]
[Detaching after fork from child process 126]
[Detaching after fork from child process 127]
[Detaching after fork from child process 129]
[Detaching after fork from child process 130]
[Detaching after fork from child process 131]
[Detaching after fork from child process 133]
[Detaching after fork from child process 134]
[Detaching after fork from child process 135]
[Detaching after fork from child process 137]
[Detaching after fork from child process 138]
[Detaching after fork from child process 165]
.[Detaching after fork from child process 166]
[Detaching after fork from child process 167]
[Detaching after fork from child process 168]
[Detaching after fork from child process 169]
[Detaching after fork from child process 170]
[Detaching after fork from child process 171]
[Detaching after fork from child process 172]
[Detaching after fork from child process 174]
[Detaching after fork from child process 175]
[Detaching after fork from child process 177]
[Detaching after fork from child process 178]
[Detaching after fork from child process 179]
[Detaching after fork from child process 181]
[Detaching after fork from child process 182]
[Detaching after fork from child process 183]
[Detaching after fork from child process 185]
[Detaching after fork from child process 186]
[Detaching after fork from child process 187]
[Detaching after fork from child process 189]
[Detaching after fork from child process 190]
[Detaching after fork from child process 192]
.[Detaching after fork from child process 193]
[Detaching after fork from child process 194]
[Detaching after fork from child process 195]
[Detaching after fork from child process 196]
[Detaching after fork from child process 197]

Program received signal SIGSEGV, Segmentation fault.
0x00005593d581241b in zend_unfinished_execution_gc ()
(gdb) bt
#0  0x00005593d581241b in zend_unfinished_execution_gc ()
#1  0x00005593d5849f1b in ?? ()
#2  0x00005593d582e758 in zend_gc_collect_cycles ()
#3  0x00005593d582d517 in ?? ()
#4  0x00005593d58162ad in execute_ex ()
#5  0x00005593d57a14b4 in zend_call_function ()
#6  0x00005593d5849c12 in ?? ()
#7  0x00005593d584ac32 in ?? ()
#8  0x00005593d57480f7 in make_fcontext ()
#9  0x0000000000000000 in ?? ()

Changing the logic in Roave\BackwardCompatibility\Git\GitParseRevision to use exec() instead of Psl\Shell\execute() does remove the issue too, though I don't think that's the direction you want to go.

With all that said, I hope this helps @asgrim @Ocramius or @azjezz to investigate further and decide what you want to do to solve the SEGFAULTs.
@Ocramius
Copy link
Member

Likely an issue with how PSL executes commands here? It has support for async dispatch, so it may be hidden in its internals.

Anyway, safe.directory=* is safe in docker contexts, so we'd need to adjust that too :D

@lcobucci
Copy link
Author

lcobucci commented Mar 1, 2023

It's interesting to see that disabling the ANSI output seems to prevent the SEGFAULTs.

Perhaps that's a good first step until the root cause is addressed.

I also didn't test if upgrading to PHPUnit 10 has any impact.

@lcobucci
Copy link
Author

lcobucci commented Mar 1, 2023 •
edited

I just noticed that this project relies on ghcr.io/laminas/laminas-continuous-integration:1 to run CI, which already solves the safe.directory=*

Sooo, all my comments are kind of moot points, yay 🎉

Running the following command does give me a SEGFAULT:

docker run -it --rm \
  -w /github/workspace  -e GITHUB_ACTIONS=true -e CI=true \
  -v $(pwd):/github/workspace ghcr.io/laminas/laminas-continuous-integration:1 "{\"command\":\"./vendor/bin/phpunit\",\"php\":\"8.1\",\"extensions\":[\"dom\",\"json\",\"libxml\",\"simplexml\",\"bcmath\"],\"ini\":[\"memory_limit=-1\"],\"dependencies\":\"latest\",\"ignore_platform_reqs_8\":false,\"ignore_php_platform_requirement\":false,\"additional_composer_arguments\":[],\"before_script\":[\"xmllint --schema vendor/phpunit/phpunit/phpunit.xsd phpunit.xml.dist\"]}"

Let me check a bit more 👍

@lcobucci
Copy link
Author

lcobucci commented Mar 1, 2023

Okay, I investigated a bit more and found out that:

  1. Issue has nothing to do with the safe.directory crap
  2. Issue has 100% to do with PSL usage of proc_open in combination with PHPUnit's default printer (ANSI or not)
  3. PHPUnit 10.0 ships new stuff that makes the problem go away...

I´m sending a patch to upgrade it that should solve this 😁

@lcobucci lcobucci linked a pull request Mar 2, 2023 that will close this issue
Open
@azjezz
Copy link
Contributor

azjezz commented Mar 19, 2023 •
edited

Sorry for being late to the party :D

Issue has 100% to do with PSL usage of proc_open

is the issue just with PSL, or proc_open in general? 🤔 i need to do a bit more investigation to see what is going on here, but it seems really strange.

@lcobucci
Copy link
Author

Sorry for being late to the part :D

Issue has 100% to do with PSL usage of proc_open

is the issue just with PSL, or proc_open in general? thinking i need to do a bit more investigation to see what is going on here, but it seems really strange.

I think is proc_open + PHPUnit but I haven't tried to isolate things more.

@azjezz
Copy link
Contributor

azjezz commented Mar 27, 2023

it seems to be an issue in revolt event loop when using PHP 8.1.15-16 and 8.2.2-3, can you make sure that you are using at least 8.2.4 or 8.1.17?

@Ocramius
Copy link
Member

Yes, although I'd need to release the Laminas CI container first again

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug help wanted
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Upgrade to PHPUnit v10 lcobucci/BackwardCompatibilityCheck
3 participants
@Ocramius @lcobucci @azjezz