[BUG] dd-trace causes segfault when used with opcache + preload w/ autoload on PHP 8.1 (Laravel)

[phroggyy]

Bug description

When having ddtrace enabled, Laravel applications running on Alpine Linux 3.16, on the linux/amd64 platform in Docker, with PHP 8.1, will segfault upon executing a preload.php (as referenced in opcache config) which requires the vendor/autoload.php.

I have a complete reproduction including steps at my reproduction repo: https://github.com/phroggyy/dd-trace-preload-segfault-repro.

In particular, see the preload.php, opcache config and Dockerfile for the most relevant parts.

Relates to #1042.

I have also concluded that the extension works as expected in PHP 8.0.

Context (why autoload.php)

We rely on Composer's autoloader to be able to preload our entire application into memory. Since we require all application files (with a few exceptions that cause issues due to missing files in prod envs), we need the autoloader to be registered in order to use require_once properly, as that will also properly require chained dependencies.

Our entire preload script that we use in production looks as follows, but note that reproduction occurs with just the first line:

preload.php

<?php

require_once __DIR__ . '/vendor/autoload.php';

class Preloader
{
    private array $ignoredClasses = [];

    private static int $count = 0;

    private array $paths;

    private array $fileMap;

    private array $ignoredPaths = [];

    public function __construct(string ...$paths)
    {
        $this->paths = $paths;

        // We'll use composer's classmap
        // to easily find which classes to autoload,
        // based on their filename
        $classMap = require __DIR__ . '/vendor/composer/autoload_classmap.php';

        $this->fileMap = array_flip($classMap);
    }

    public function paths(string ...$paths): Preloader
    {
        $this->paths = array_merge(
            $this->paths,
            $paths
        );

        return $this;
    }

    public function ignoreClasses(string ...$names): Preloader
    {
        $this->ignoredClasses = array_merge(
            $this->ignoredClasses,
            $names
        );

        return $this;
    }

    public function ignorePaths(string ...$paths): Preloader
    {
        $this->ignoredPaths = array_merge(
            $this->ignoredPaths,
            $paths
        );

        return $this;
    }

    public function load(): void
    {
        // We'll loop over all registered paths
        // and load them one by one
        foreach ($this->paths as $path) {
            $this->loadPath(rtrim($path, '/'));
        }

        $count = self::$count;
    }

    private function loadPath(string $path): void
    {
        // If the current path is a directory,
        // we'll load all files in it
        if (is_dir($path)) {
            $this->loadDir($path);

            return;
        }

        // Otherwise we'll just load this one file
        $this->loadFile($path);
    }

    private function loadDir(string $path): void
    {
        $handle = opendir($path);

        // We'll loop over all files and directories
        // in the current path,
        // and load them one by one
        while ($file = readdir($handle)) {
            if (in_array($file, ['.', '..'])) {
                continue;
            }

            $this->loadPath("{$path}/{$file}");
        }

        closedir($handle);
    }

    private function loadFile(string $path): void
    {
        // We resolve the classname from composer's autoload mapping
        $class = $this->fileMap[$path] ?? null;

        // And use it to make sure the class, or the file pattern, shouldn't be ignored
        if ($this->shouldIgnoreClass($class) || $this->shouldIgnorePath($path)) {
            return;
        }

        // Finally we require the path,
        // causing all its dependencies to be loaded as well
        require_once($path);

        self::$count++;
    }

    private function shouldIgnoreClass(?string $name): bool
    {
        if ($name === null) {
            return true;
        }

        foreach ($this->ignoredClasses as $ignore) {
            if (strpos($name, $ignore) === 0) {
                return true;
            }
        }

        return false;
    }

    private function shouldIgnorePath(string $path): bool
    {
        foreach ($this->ignoredPaths as $ignoredPath) {
            $ignoredPath = preg_quote($ignoredPath, '/');
            if (preg_match("/$ignoredPath/", $path) === 1) {
                return true;
            }
        }

        return false;
    }
}

(new Preloader())
    ->paths(
        __DIR__ . '/vendor/laravel/framework',
        __DIR__ . '/app/Providers',
        __DIR__ . '/app/Http/Middleware',
        __DIR__ . '/app/Http/Controllers'
    )
    ->ignorePaths(
        'Illuminate/Foundation/Testing',
        'Illuminate/Database/PDO',
        'Illuminate/Database/DBAL',
        'Illuminate/Testing',
    )
    ->ignoreClasses(
        \Illuminate\Filesystem\Cache::class,
        \Illuminate\Log\LogManager::class,
        \Illuminate\Http\Testing\File::class,
        \Illuminate\Http\UploadedFile::class,
        \Illuminate\Support\Carbon::class
    )
    ->load();

### PHP version
8.1

### Tracer version
0.81.1

### Installed extensions

[PHP Modules]
Core
ctype
curl
date
ddtrace
dom
fileinfo
filter
ftp
hash
iconv
json
libxml
mbstring
mysqlnd
openssl
pcre
PDO
pdo_sqlite
Phar
posix
readline
Reflection
session
SimpleXML
sodium
SPL
sqlite3
standard
tokenizer
xml
xmlreader
xmlwriter
Zend OPcache
zlib

[Zend Modules]
Zend OPcache
ddtrace

### OS info

NAME="Alpine Linux"
VERSION_ID=3.16.3
PRETTY_NAME="Alpine Linux v3.16"

### Diagnostics and configuration

#### Output of phpinfo() (ddtrace >= 0.47.0)

ddtrace

Datadog PHP tracer extension
For help, check out the documentation at https://docs.datadoghq.com/tracing/languages/php/
(c) Datadog 2020

Datadog tracing support => disabled
Version => 0.81.1
DATADOG TRACER CONFIGURATION => {
"date": "2022-11-14T18:37:33Z",
"os_name": "Linux fccd33590a92 5.15.49-linuxkit #1 SMP PREEMPT Tue Sep 13 07:51:32 UTC 2022 aarch64",
"os_version": "5.15.49-linuxkit",
"version": "0.81.1",
"lang": "php",
"lang_version": "8.1.8",
"env": null,
"enabled": true,
"service": null,
"enabled_cli": false,
"agent_url": "http://host.docker.internal:8126",
"debug": true,
"analytics_enabled": false,
"sample_rate": 1,
"sampling_rules": [],
"tags": [],
"service_mapping": [],
"distributed_tracing_enabled": true,
"priority_sampling_enabled": true,
"dd_version": null,
"architecture": "aarch64",
"sapi": "cli",
"datadog.trace.request_init_hook": "/opt/datadog/dd-library/0.81.1/dd-trace-sources/bridge/dd_wrap_autoloader.php",
"open_basedir_configured": false,
"uri_fragment_regex": null,
"uri_mapping_incoming": null,
"uri_mapping_outgoing": null,
"auto_flush_enabled": false,
"generate_root_span": true,
"http_client_split_by_domain": false,
"measure_compile_time": true,
"report_hostname_on_root_span": false,
"traced_internal_functions": null,
"auto_prepend_file_configured": false,
"integrations_disabled": "default",
"enabled_from_env": false,
"opcache.file_cache": null
}

Directive => Local Value => Master Value
ddtrace.disable => Off => Off
ddtrace.cgroup_file => /proc/self/cgroup => /proc/self/cgroup
datadog.trace.request_init_hook => /opt/datadog/dd-library/0.81.1/dd-trace-sources/bridge/dd_wrap_autoloader.php => /opt/datadog/dd-library/0.81.1/dd-trace-sources/bridge/dd_wrap_autoloader.php
ddtrace.request_init_hook => /opt/datadog/dd-library/0.81.1/dd-trace-sources/bridge/dd_wrap_autoloader.php => /opt/datadog/dd-library/0.81.1/dd-trace-sources/bridge/dd_wrap_autoloader.php
datadog.trace.agent_url => no value => no value
datadog.agent_host => host.docker.internal => host.docker.internal
datadog.dogstatsd_url => no value => no value
datadog.distributed_tracing => On => On
datadog.dogstatsd_port => 8125 => 8125
datadog.env => no value => no value
datadog.autofinish_spans => Off => Off
datadog.trace.url_as_resource_names_enabled => On => On
datadog.integrations_disabled => default => default
datadog.priority_sampling => On => On
datadog.service => no value => no value
datadog.service_name => no value => no value
datadog.service_mapping => no value => no value
datadog.tags => no value => no value
datadog.trace.global_tags => no value => no value
datadog.trace.agent_port => 8126 => 8126
datadog.trace.analytics_enabled => Off => Off
datadog.trace.auto_flush_enabled => Off => Off
datadog.trace.cli_enabled => Off => Off
datadog.trace.measure_compile_time => On => On
datadog.trace.debug => On => On
datadog.trace.enabled => Off => On
datadog.trace.health_metrics_enabled => Off => Off
datadog.trace.health_metrics_heartbeat_sample_rate => 0.001 => 0.001
datadog.trace.db_client_split_by_instance => Off => Off
datadog.trace.http_client_split_by_domain => Off => Off
datadog.trace.redis_client_split_by_host => Off => Off
datadog.trace.memory_limit => no value => no value
datadog.trace.report_hostname => Off => Off
datadog.trace.resource_uri_fragment_regex => no value => no value
datadog.trace.resource_uri_mapping_incoming => no value => no value
datadog.trace.resource_uri_mapping_outgoing => no value => no value
datadog.trace.resource_uri_query_param_allowed => no value => no value
datadog.trace.http_url_query_param_allowed => * => *
datadog.trace.rate_limit => 0 => 0
datadog.trace.sample_rate => 1 => 1
datadog.sampling_rate => 1 => 1
datadog.trace.sampling_rules => [] => []
datadog.span_sampling_rules => [] => []
datadog.span_sampling_rules_file => no value => no value
datadog.trace.header_tags => no value => no value
datadog.trace.x_datadog_tags_max_length => 512 => 512
datadog.trace.propagate_service => Off => Off
datadog.propagation_style_extract => Datadog,B3,B3 single header => Datadog,B3,B3 single header
datadog.propagation_style_inject => Datadog => Datadog
datadog.trace.traced_internal_functions => no value => no value
datadog.trace.agent_timeout => 500 => 500
datadog.trace.agent_connect_timeout => 100 => 100
datadog.trace.debug_prng_seed => -1 => -1
datadog.log_backtrace => Off => Off
datadog.trace.generate_root_span => On => On
datadog.trace.spans_limit => 1000 => 1000
datadog.trace.agent_max_consecutive_failures => 3 => 3
datadog.trace.agent_attempt_retry_time_msec => 5000 => 5000
datadog.trace.bgs_connect_timeout => 2000 => 2000
datadog.trace.bgs_timeout => 5000 => 5000
datadog.trace.agent_flush_interval => 5000 => 5000
datadog.trace.agent_flush_after_n_requests => 10 => 10
datadog.trace.shutdown_timeout => 5000 => 5000
datadog.trace.startup_logs => On => On
datadog.trace.agent_debug_verbose_curl => Off => Off
datadog.trace.debug_curl_output => Off => Off
datadog.trace.beta_high_memory_pressure_percent => 80 => 80
datadog.trace.warn_legacy_dd_trace => On => On
datadog.trace.retain_thread_capabilities => Off => Off
datadog.version => no value => no value
datadog.trace.obfuscation_query_string_regexp => (?i)(?:p(?:ass)?w(?:or)?d|pass(?:?phrase)?|secret|(?:api?|private_?|public_?|access_?|secret_?)key(?:?id)?|token|consumer?(?:id|key|secret)|sign(?:ed|ature)?|auth(?:entication|orization)?)(?:(?:\s|%20)(?:=|%3D)[^&]+|(?:"|%22)(?:\s|%20)(?::|%3A)(?:\s|%20)(?:"|%22)(?:%2[^2]|%[^2]|[^"%])+(?:"|%22))|bearer(?:\s|%20)+[a-z0-9.-]|token(?::|%3A)[a-z0-9]{13}|gh[opsu][0-9a-zA-Z]{36}|eyI-L+.eyI-L+(?:.(?:[\w.+/=-]|%3D|%2F|%2B)+)?|[-]{5}BEGIN(?:[a-z\s]|%20)+PRIVATE(?:\s|%20)KEY[-]{5}[^\-]+[-]{5}END(?:[a-z\s]|%20)+PRIVATE(?:\s|%20)KEY|ssh-rsa(?:\s|%20)(?:[a-z0-9/.+]|%2F|%5C|%2B){100,} => (?i)(?:p(?:ass)?w(?:or)?d|pass(?:?phrase)?|secret|(?:api?|private_?|public_?|access_?|secret_?)key(?:?id)?|token|consumer?(?:id|key|secret)|sign(?:ed|ature)?|auth(?:entication|orization)?)(?:(?:\s|%20)(?:=|%3D)[^&]+|(?:"|%22)(?:\s|%20)(?::|%3A)(?:\s|%20)(?:"|%22)(?:%2[^2]|%[^2]|[^"%])+(?:"|%22))|bearer(?:\s|%20)+[a-z0-9.-]|token(?::|%3A)[a-z0-9]{13}|gh[opsu][0-9a-zA-Z]{36}|eyI-L+.eyI-L+(?:.(?:[\w.+/=-]|%3D|%2F|%2B)+)?|[-]{5}BEGIN(?:[a-z\s]|%20)+PRIVATE(?:\s|%20)KEY[-]{5}[^\-]+[-]{5}END(?:[a-z\s]|%20)+PRIVATE(?:\s|%20)KEY|ssh-rsa(?:\s|%20)(?:[a-z0-9/.+]|%2F|%5C|%2B){100,}
datadog.trace.forked_process => On => On
datadog.trace.agent_max_payload_size => 52428800 => 52428800
datadog.trace.agent_stack_initial_size => 131072 => 131072
datadog.trace.agent_stack_backlog => 12 => 12
datadog.trace.cakephp_enabled => On => On
datadog.trace.cakephp_analytics_enabled => Off => Off
datadog.cakephp_analytics_enabled => Off => Off
datadog.trace.cakephp_analytics_sample_rate => 1 => 1
datadog.cakephp_analytics_sample_rate => 1 => 1
datadog.trace.codeigniter_enabled => On => On
datadog.trace.codeigniter_analytics_enabled => Off => Off
datadog.codeigniter_analytics_enabled => Off => Off
datadog.trace.codeigniter_analytics_sample_rate => 1 => 1
datadog.codeigniter_analytics_sample_rate => 1 => 1
datadog.trace.curl_enabled => On => On
datadog.trace.curl_analytics_enabled => Off => Off
datadog.curl_analytics_enabled => Off => Off
datadog.trace.curl_analytics_sample_rate => 1 => 1
datadog.curl_analytics_sample_rate => 1 => 1
datadog.trace.elasticsearch_enabled => On => On
datadog.trace.elasticsearch_analytics_enabled => Off => Off
datadog.elasticsearch_analytics_enabled => Off => Off
datadog.trace.elasticsearch_analytics_sample_rate => 1 => 1
datadog.elasticsearch_analytics_sample_rate => 1 => 1
datadog.trace.eloquent_enabled => On => On
datadog.trace.eloquent_analytics_enabled => Off => Off
datadog.eloquent_analytics_enabled => Off => Off
datadog.trace.eloquent_analytics_sample_rate => 1 => 1
datadog.eloquent_analytics_sample_rate => 1 => 1
datadog.trace.guzzle_enabled => On => On
datadog.trace.guzzle_analytics_enabled => Off => Off
datadog.guzzle_analytics_enabled => Off => Off
datadog.trace.guzzle_analytics_sample_rate => 1 => 1
datadog.guzzle_analytics_sample_rate => 1 => 1
datadog.trace.laravel_enabled => On => On
datadog.trace.laravel_analytics_enabled => Off => Off
datadog.laravel_analytics_enabled => Off => Off
datadog.trace.laravel_analytics_sample_rate => 1 => 1
datadog.laravel_analytics_sample_rate => 1 => 1
datadog.trace.lumen_enabled => On => On
datadog.trace.lumen_analytics_enabled => Off => Off
datadog.lumen_analytics_enabled => Off => Off
datadog.trace.lumen_analytics_sample_rate => 1 => 1
datadog.lumen_analytics_sample_rate => 1 => 1
datadog.trace.memcached_enabled => On => On
datadog.trace.memcached_analytics_enabled => Off => Off
datadog.memcached_analytics_enabled => Off => Off
datadog.trace.memcached_analytics_sample_rate => 1 => 1
datadog.memcached_analytics_sample_rate => 1 => 1
datadog.trace.mongo_enabled => On => On
datadog.trace.mongo_analytics_enabled => Off => Off
datadog.mongo_analytics_enabled => Off => Off
datadog.trace.mongo_analytics_sample_rate => 1 => 1
datadog.mongo_analytics_sample_rate => 1 => 1
datadog.trace.mongodb_enabled => On => On
datadog.trace.mongodb_analytics_enabled => Off => Off
datadog.mongodb_analytics_enabled => Off => Off
datadog.trace.mongodb_analytics_sample_rate => 1 => 1
datadog.mongodb_analytics_sample_rate => 1 => 1
datadog.trace.mysqli_enabled => On => On
datadog.trace.mysqli_analytics_enabled => Off => Off
datadog.mysqli_analytics_enabled => Off => Off
datadog.trace.mysqli_analytics_sample_rate => 1 => 1
datadog.mysqli_analytics_sample_rate => 1 => 1
datadog.trace.nette_enabled => On => On
datadog.trace.nette_analytics_enabled => Off => Off
datadog.nette_analytics_enabled => Off => Off
datadog.trace.nette_analytics_sample_rate => 1 => 1
datadog.nette_analytics_sample_rate => 1 => 1
datadog.trace.pcntl_enabled => On => On
datadog.trace.pcntl_analytics_enabled => Off => Off
datadog.pcntl_analytics_enabled => Off => Off
datadog.trace.pcntl_analytics_sample_rate => 1 => 1
datadog.pcntl_analytics_sample_rate => 1 => 1
datadog.trace.pdo_enabled => On => On
datadog.trace.pdo_analytics_enabled => Off => Off
datadog.pdo_analytics_enabled => Off => Off
datadog.trace.pdo_analytics_sample_rate => 1 => 1
datadog.pdo_analytics_sample_rate => 1 => 1
datadog.trace.phpredis_enabled => On => On
datadog.trace.phpredis_analytics_enabled => Off => Off
datadog.phpredis_analytics_enabled => Off => Off
datadog.trace.phpredis_analytics_sample_rate => 1 => 1
datadog.phpredis_analytics_sample_rate => 1 => 1
datadog.trace.predis_enabled => On => On
datadog.trace.predis_analytics_enabled => Off => Off
datadog.predis_analytics_enabled => Off => Off
datadog.trace.predis_analytics_sample_rate => 1 => 1
datadog.predis_analytics_sample_rate => 1 => 1
datadog.trace.slim_enabled => On => On
datadog.trace.slim_analytics_enabled => Off => Off
datadog.slim_analytics_enabled => Off => Off
datadog.trace.slim_analytics_sample_rate => 1 => 1
datadog.slim_analytics_sample_rate => 1 => 1
datadog.trace.symfony_enabled => On => On
datadog.trace.symfony_analytics_enabled => Off => Off
datadog.symfony_analytics_enabled => Off => Off
datadog.trace.symfony_analytics_sample_rate => 1 => 1
datadog.symfony_analytics_sample_rate => 1 => 1
datadog.trace.web_enabled => On => On
datadog.trace.web_analytics_enabled => Off => Off
datadog.web_analytics_enabled => Off => Off
datadog.trace.web_analytics_sample_rate => 1 => 1
datadog.web_analytics_sample_rate => 1 => 1
datadog.trace.wordpress_enabled => On => On
datadog.trace.wordpress_analytics_enabled => Off => Off
datadog.wordpress_analytics_enabled => Off => Off
datadog.trace.wordpress_analytics_sample_rate => 1 => 1
datadog.wordpress_analytics_sample_rate => 1 => 1
datadog.trace.yii_enabled => On => On
datadog.trace.yii_analytics_enabled => Off => Off
datadog.yii_analytics_enabled => Off => Off
datadog.trace.yii_analytics_sample_rate => 1 => 1
datadog.yii_analytics_sample_rate => 1 => 1
datadog.trace.zendframework_enabled => On => On
datadog.trace.zendframework_analytics_enabled => Off => Off
datadog.zendframework_analytics_enabled => Off => Off
datadog.trace.zendframework_analytics_sample_rate => 1 => 1
datadog.zendframework_analytics_sample_rate => 1 => 1

</details>

[bwoebi]

Hey @phroggyy,

thanks a lot for the reproducer, I'm able to reproduce it and looking into it :-)

[bwoebi]

There was a change introduced in PHP 8.1, which causes differences in shutdown execution specifically for preloading. Reported at php/php-src#9957.

I'll check whether there's anything reasonable we can do to mitigate this bug on our side.

[phroggyy]

@bwoebi if it's not too much to ask, I'd love to understand how you debugged this - are there some good resources to understanding those internals (in this case, particularly the memory management of custom objects)? Would love to be able to contribute to both this extension and others when I find bugs (even though in this case it's in PHP), so any references would be much appreciated! (both on debugging, and just reference to the diff functions utilised)

[bwoebi]

In this particular instance it's pretty mundane, I just executed USE_ZEND_ALLOC=0 valgrind php-fpm, and from the resulting stacktrace (which is referenced in the php-src issue), I was able to conclude the issue immediately. I just had to infer a couple ??? by looking at the code because these docker containers are provided without debug symbols, sadly.