Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PreloadExplicitClassSymbolsFixer - introduction #4891

Closed
wants to merge 11 commits into from
5 changes: 5 additions & 0 deletions README.rst
Expand Up @@ -1798,6 +1798,11 @@ Choose from the list of available rules:
Pre incrementation/decrementation should be used if possible.
DEPRECATED: use ``increment_style`` instead.

* **preload_explicit_class_symbols**

Adds extra ``class_exists`` before a class to help opcache.preload to
discover always-needed symbols.

* **protected_to_private** [@Symfony, @PhpCsFixer]

Converts ``protected`` variables and methods to ``private`` where possible.
Expand Down
365 changes: 365 additions & 0 deletions src/Fixer/Preload/PreloadExplicitClassSymbolsFixer.php
@@ -0,0 +1,365 @@
<?php

/*
* This file is part of PHP CS Fixer.
*
* (c) Fabien Potencier <fabien@symfony.com>
* Dariusz Rumiński <dariusz.ruminski@gmail.com>
*
* This source file is subject to the MIT license that is bundled
* with this source code in the file LICENSE.
*/

namespace PhpCsFixer\Fixer\Preload;

use PhpCsFixer\AbstractFixer;
use PhpCsFixer\FixerDefinition\FixerDefinition;
use PhpCsFixer\FixerDefinition\VersionSpecification;
use PhpCsFixer\FixerDefinition\VersionSpecificCodeSample;
use PhpCsFixer\Tokenizer\Analyzer\ArgumentsAnalyzer;
use PhpCsFixer\Tokenizer\Analyzer\FunctionsAnalyzer;
use PhpCsFixer\Tokenizer\CT;
use PhpCsFixer\Tokenizer\Token;
use PhpCsFixer\Tokenizer\Tokens;
use PhpCsFixer\Tokenizer\TokensAnalyzer;

/**
* @author Tobias Nyholm <tobias.nyholm@gmail.com>
*/
final class PreloadExplicitClassSymbolsFixer extends AbstractFixer
{
private $functionsAnalyzer;
private $tokenAnalyzer;

public function __construct()
{
parent::__construct();
$this->functionsAnalyzer = new FunctionsAnalyzer();
}

/**
* {@inheritdoc}
*/
public function getDefinition()
{
return new FixerDefinition(
'Adds extra `class_exists` before a class to help opcache.preload to discover always-needed symbols.',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice idea to make this as a fixer!

Please provide a $description parameter with the reasoning, especially:

  • why the rule is worth to use?
  • what classes we don't need to preload, ie classes that is found by reflection and why

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a desciption

[
new VersionSpecificCodeSample(
'<?php
use App\Foo;

class MyClass
{
private $foo;
public function __construct()
{
$this->foo = new Foo();
}
}
',
new VersionSpecification(70300)
),
],
'A script for opcache.preload should classes that are used by the average '
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A script for opcache.preload should classes that are used by the average HTTP request.

should classes? sorry, i can't understand this sentence

.'HTTP request. A good preloading script uses reflection to load the '
.'hot class and all classes it depends on. Reflection cannot always find '
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is hot class?
there is only ONE hot class, or many of them (classes)?

.'all dependencies. Example: Classes used in the constructor or dependent '
Copy link
Member

@keradus keradus Apr 7, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reflection cannot always find all dependencies

why?

(sorry for dummy questions, I am unaware about the idea and description here is not helping me to get what exactly should be added as outcome of this rule and why, and what not as it's automatically picked up by some preloading script)

.'classes of the constructor. This fixer will make sure we load classes '
.'that are always used and that cannot be found by reflection. This will '
.'improve the preload performance.'
);
}

/**
* {@inheritdoc}
*/
public function isCandidate(Tokens $tokens)
{
return \PHP_VERSION_ID >= 70300 && ($tokens->isTokenKindFound(T_CLASS) || $tokens->isTokenKindFound(T_TRAIT));
}

/**
* {@inheritdoc}
*/
protected function applyFix(\SplFileInfo $file, Tokens $tokens)
{
$this->tokenAnalyzer = new TokensAnalyzer($tokens);
$candidates = $this->parse($tokens, '__construct');

// Remove false positives
$candidates = array_filter($candidates, function ($candidate) {
return !\in_array($candidate, ['parent', 'self'], true);
});

if (empty($candidates)) {
return;
}

$candidates = array_unique($candidates);
$classesNotToLoad = $this->getPreloadedClasses($tokens);

$classesToLoad = array_diff($candidates, $classesNotToLoad);
$this->injectClasses($tokens, $classesToLoad);
}

/**
* Parse functions for used classes. Make a recursive call to other function.
*
* @param string $functionName
*
* @return string[] classes
*/
private function parse(Tokens $tokens, $functionName)
{
if (null === $functionIndex = $this->findFunction($tokens, $functionName)) {
return [];
}

$classes = [];
$methodAttributes = $this->tokenAnalyzer->getMethodAttributes($functionIndex);

// If not public
if (T_PRIVATE === $methodAttributes['visibility'] || T_PROTECTED === $methodAttributes['visibility']) {
// Get argument types
$arguments = $this->functionsAnalyzer->getFunctionArguments($tokens, $functionIndex);
foreach ($arguments as $argument) {
if ($argument->hasTypeAnalysis() && !$argument->getTypeAnalysis()->isReservedType()) {
$classes[] = $argument->getTypeAnalysis()->getName();
}
}

// Get return type
$returnType = $this->functionsAnalyzer->getFunctionReturnType($tokens, $functionIndex);
if (null !== $returnType && !$returnType->isReservedType()) {
$classes[] = $returnType->getName();
}
}

// Parse the body of the method
$blockStart = $tokens->getNextTokenOfKind($functionIndex, ['{']);
$blockEnd = $tokens->findBlockEnd(Tokens::BLOCK_TYPE_CURLY_BRACE, $blockStart);

for ($i = $blockStart; $i < $blockEnd; ++$i) {
$token = $tokens[$i];
// Find Foo::class, new Foo() and function calls.
if ($token->isGivenKind(T_NEW)) {
$class = $this->getFullClassReference($tokens, $tokens->getNextMeaningfulToken($i));
$classes[] = $class;
} elseif ($token->isGivenKind(T_DOUBLE_COLON)) {
$class = $this->getFullClassReference($tokens, $tokens->getPrevMeaningfulToken($i));
$classes[] = $class;
} elseif ($token->isGivenKind(T_OBJECT_OPERATOR)) {
// FIXME Better check if function call to avoid false positive like "$this->bar = 2;"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Nyholm , are those 3 FIXME cases covered by tests already? if not, maybe we can add a test case for each, so we can easier understand what is example code snippet you struggle with and expected outcome ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are. They are the failing tests.
I'll work with them now to see if I can make sure they are green

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests are green. Locally at least. =)

$classes = array_merge($classes, $this->parse($tokens, $tokens[$tokens->getNextMeaningfulToken($i)]->getContent()));
}
}

return $classes;
}

/**
* Get a class name by searching for tokens near the index given. Ie, the line might be:
* $this->bar = new \Foo\Bar().
*
* If $index is representing `Bar`, then this method returns string "\Foo\Bar".
*
* @param null|int $index
*
* @return null|string
*/
private function getFullClassReference(Tokens $tokens, $index)
{
if (null === $index) {
return null;
}

// Find tokens that can be included in classes
$allowed = [[T_STRING], [T_NS_SEPARATOR]];

$first = $index - 1;
while (isset($tokens[$first])) {
if ($tokens[$first]->equalsAny($allowed)) {
--$first;
} else {
++$first;

break;
}
}

$last = $index + 1;
while (isset($tokens[$last])) {
if ($tokens[$last]->equalsAny($allowed)) {
++$last;
} else {
--$last;

break;
}
}

// Print the result
$class = '';
for ($i = $first; $i <= $last; ++$i) {
$class .= $tokens[$i]->getContent();
}

if (empty($class)) {
return null;
}

return $class;
}

/**
* Get classes that are found by the preloader. Ie classes we shouldn't include in `class_exists`.
*
* @return string[]
*/
private function getPreloadedClasses(Tokens $tokens)
{
$classes = $this->getExistingClassExists($tokens);

// Parse public methods
foreach ($tokens as $functionIndex => $token) {
if (!$token->isGivenKind(T_FUNCTION)) {
continue;
}
$methodAttributes = $this->tokenAnalyzer->getMethodAttributes($functionIndex);
if (T_PUBLIC !== $methodAttributes['visibility']) {
continue;
}

// Get argument types
$arguments = $this->functionsAnalyzer->getFunctionArguments($tokens, $functionIndex);
foreach ($arguments as $argument) {
if ($argument->hasTypeAnalysis() && !$argument->getTypeAnalysis()->isReservedType()) {
$classes[] = $argument->getTypeAnalysis()->getName();
}
}

// Get return type
$returnType = $this->functionsAnalyzer->getFunctionReturnType($tokens, $functionIndex);
if (null !== $returnType && !$returnType->isReservedType()) {
$classes[] = $returnType->getName();
}
}

// TODO parse public properties
// TODO Get the class' interfaces and parent
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

marker
(so we don't merge before it's solved)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Nyholm , would you like to finalize this PR?


return array_unique($classes);
}

/**
* Find a function in the tokens.
*
* @param string $name
*
* @return null|int the index or null. The index is to the "function" token.
*/
private function findFunction(Tokens $tokens, $name)
{
foreach ($tokens as $index => $token) {
if (!$token->isGivenKind(T_FUNCTION)) {
continue;
}

$nextTokenIndex = $tokens->getNextMeaningfulToken($index);
$nextToken = $tokens[$nextTokenIndex];

if ($nextToken->getContent() !== $name) {
continue;
}

return $index;
}

return null;
}

/**
* Inject "class_exists" at the top of the file.
*/
private function injectClasses(Tokens $tokens, array $classes)
{
$insertAfter = null;
foreach ($tokens as $index => $token) {
if (!$token->isGivenKind(T_CLASS)) {
continue;
}

$insertAfter = $tokens->getPrevMeaningfulToken($index);

break;
}

if (null === $insertAfter) {
return;
}

$newTokens = [];
foreach ($classes as $class) {
$newTokens[] = new Token([T_STRING, 'class_exists']);
$newTokens[] = new Token('(');

$parts = explode('\\', $class);
$lastPart = array_pop($parts);
foreach ($parts as $part) {
if (!empty($part)) {
$newTokens[] = new Token([T_STRING, $part]);
}
$newTokens[] = new Token([T_NS_SEPARATOR, '\\']);
}
$newTokens[] = new Token([T_STRING, $lastPart]);

$newTokens[] = new Token([T_DOUBLE_COLON, '::']);
$newTokens[] = new Token([CT::T_CLASS_CONSTANT, 'class']);
$newTokens[] = new Token(')');
$newTokens[] = new Token(';');
$newTokens[] = new Token([T_WHITESPACE, "\n"]);
Copy link
Member

@keradus keradus Apr 7, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
$newTokens[] = new Token([T_WHITESPACE, "\n"]);
$newTokens[] = new Token([T_WHITESPACE, $this->whitespacesConfig->getLineEnding()]);

(you need to implement WhitespacesAwareFixerInterface also)

}

$tokens->insertAt($insertAfter + 2, $newTokens);
}

/**
* Get all class_exists in the beginning of the file.
*
* @return array
*/
private function getExistingClassExists(Tokens $tokens)
{
$classes = [];
foreach ($tokens as $index => $token) {
if ($token->isGivenKind(T_CLASS)) {
// Stop when a class is found
break;
}

if (!$this->functionsAnalyzer->isGlobalFunctionCall($tokens, $index)) {
continue;
}

if ('class_exists' === $token->getContent()) {
$argumentsStart = $tokens->getNextTokenOfKind($index, ['(']);
$argumentsEnd = $tokens->getNextTokenOfKind($index, [')']);
$argumentAnalyzer = new ArgumentsAnalyzer();

foreach ($argumentAnalyzer->getArguments($tokens, $argumentsStart, $argumentsEnd) as $start => $end) {
$argumentInfo = $argumentAnalyzer->getArgumentInfo($tokens, $start, $end);
$class = $argumentInfo->getTypeAnalysis()->getName();
if ('::class' === substr($class, -7)) {
$classes[] = substr($class, 0, -7);
}

// We are only interested in first argument
break;
}
}
}

return $classes;
}
}