Memory is not being released in parallel execution

[zoilomora]

Q	A
Version	2.0.0
Bug?	yes
New feature?	no
Question?	yes
Documentation?	no
Related tickets	~

When executing tasks with parallel enabled: true, the memory is not being released and it is exceeding the limit established in PHP.

My configuration

grumphp:
  process_timeout: 120
  ascii:
    failed:
      - config/hooks/ko.txt
    succeeded:
      - config/hooks/ok.txt
  parallel:
    enabled: true
    max_workers: 32
  tasks:
    composer:
      strict: true
    jsonlint: ~
    phpcpd:
      exclude:
        - 'var'
        - 'vendor'
        - 'tests'
      min_lines: 60
    phpcs:
      standard:
        - 'phpcs.xml.dist'
      whitelist_patterns:
        - '/^src\/(.*)/'
        - '/^tests\/(.*)/'
      encoding: 'UTF-8'
    phplint: ~
    phpstan_shell:
      metadata:
        label: phpstan
        task: shell
      scripts:
        - ["-c", "phpstan analyse -l 9 src"]
    phpunit: ~
    behat:
      config: ~
      format: progress
      stop_on_failure: true
    phpversion:
      project: '8.2'
    securitychecker_local:
      lockfile: ./composer.lock
      format: ~

Steps to reproduce:
At the end of the vendor/bin/grumphp file add the following to check memory usage:

$memory = memory_get_usage() / 1024 / 1024;
print_r(round($memory, 3) . ' MB' . PHP_EOL);
exit();

Run ./vendor/bin/grumphp run once with each of this options:

• parallel: false
• parallel: true

Result:

parallel: false
Used Memory: 32.553 MB

parallel: true
Used Memory: 215.642 MB

When the different tasks are finished, shouldn't the memory be released?

Is this the desired behavior?

[veewee]

Running grumphp in parallel mode opens up a separate process for every task you start. There is communication between those 2 processes and that's probably what is taking up the additional MBs of space.
I'm not sure if that memory needs to get manually freed.

However, grumphp is just a tool that finishes at some point.
At that moment, the memory gets freed nevertheless.
Therefore I am not sure if this really is an issue.

So what do you think about this? Is this really a problem or is the problem rather that you need to increase PHP's memory limit in order to get grumphp running on your project?

[zoilomora]

I understand that if they are separate PHP processes, they should have the memory limit on each process.

I try to have the same memory limits in local as in production.

If the separate processes do not take up more than 32MB, it seems strange to me that all the tasks in the different processes take up more than 215MB.

The current limit is 256 MB and if I include more files the memory is exceeded. However, memory runs out when there is only 1 task left to finish.

I understand that the desired behavior would be: release that memory as tasks finish?