Memory consumption regression in 0.12.83

[schlndh]

Bug report

It seems that there is a memory consumption regression in version 0.12.83 caused by this. Phpstan throws the following error when analysing the project:

     mmap() failed: [12] Cannot allocate memory                                              
     PHP Fatal error:  Out of memory (allocated 4665122816) (tried to allocate 4294967304    
     bytes) in                                                                               
     phar:///___/vendor/phpstan/phpstan/phpstan.phar/src/Analyser/NodeScopeRes  
     olver.php on line 574

Since that line does $throwPoints = \array_merge($throwPoints, $branchScopeStatementResult->getThrowPoints()); it leads me to believe that it has something to do with the new exception funcionality. However, I do not understand phpstan internals so maybe I'm wrong. But it is definitely strange that it tries to allocate 4 more gigabytes.

Unfortunately, the project is not open source so I can't just link the source code here. But to give you a rough context it's about 5k PHP files with a total of about 600k LoC. I'm running phpstan with php 7.4 in a vagrant VM with 6 GB of RAM (barely enough to analyse the whole project with 0.12.82). (I also tried running it on host with a larger amount of RAM, but it was still not enough: Allowed memory size of 12884901888 bytes exhausted (tried to allocate 8589934600 bytes)). I'm not sure how much it matters but the project is not very well written with respect to @throws. Most of the methods don't have it and if it's there it's mostly just whatever phpstorm complained about. 😄

Perhaps using [$a, $b] instead of array_merge($a, $b) would be better for memory consumption? OFC it's not as nice of a data structure, but if it were to save gigabytes of memory usage it would seem worth it to me. Here is a small PoC with roughly what I suspect is going on:

<?php

ini_set('memory_limit', '1G');

$o = (object) [];
$node = [$o];
$intermediaryArrays = [];

for ($i = 0; $i < 10; $i++) {
    //$level = array_merge($node, $node, $node, $node, $node); // flat
    $level = [$node, $node, $node, $node]; // tree
    $intermediaryArrays[] = $level;
    $node = $level;
}

echo number_format(memory_get_usage()) . "\n";

With "flat" it outputs: 626,070,984
With "tree" it outputs: 396,320

Expected output

Memory consumption should be roughly in line with 0.12.82 + a reasonable amount for the new functionality.

[mergeable]

This bug report is missing a link to reproduction on phpstan.org.

It will most likely be closed after manual review.

[schlndh]

I have narrowed the problem to a single method. It is an insane legacy method that contains a large if. In particular it uses else if instead of elseif. It seems that changing else if to elseif halves the memory consumption reported with --debug --verbose.

So the problematic code looks something like this:

if (Utils::a($x)) {
   $i = 1;
} else if (Utils::b($x)) {
   $i = 2;
} ...

And there are tens of else if branches.

[schlndh]

Here is a generator for the problematic code:

<?php

echo "<?php


function ok(int \$i): bool {
	return rand(0, 1) === 1;
}

if (ok(0)) {
	echo 0;
}";

for ($i = 1; $i < $argv[1] ?? 0; $i++) {
	echo " else if (ok({$i})) {
	echo {$i};
}";
}

echo "\n";

When I generate code with 27 branches I get the following memory consumptions (both with -lmax --debug --verbose --memory-limit=12G):

0.12.82: 58 MB
0.12.83: 8.06 GB

[ondrejmirtes]

Good news! We're back to 58 MB on this example with dev-master :) I've got an idea for optimization: phpstan/phpstan-src@2911f68

Of course, if you'd have the huge if-else branch inside a try-catch, it'd be back to 8 GB, but I think this will help optimize how most impacted code looks like.

Feel free to test the latest revision with composer require --dev phpstan/phpstan:dev-master --prefer-dist.

[ondrejmirtes]

Alright, this was the actual root cause: phpstan/phpstan-src@7c0146c

I've reverted the micro-optimization as well, and we're now down to 58 MB each time :)

[schlndh]

Oh, I see. The throw points from the else branch were merged in twice and that's why it was exponential with else ifs. Nice catch!

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.