Skip to content

Commit

Permalink
feature #30994 [Form] Added support for caching choice lists based on…
Browse files Browse the repository at this point in the history
… options (HeahDude)

This PR was merged into the 5.1-dev branch.

Discussion
----------

[Form] Added support for caching choice lists based on options

| Q             | A
| ------------- | ---
| Branch?       | master
| Bug fix?      | no
| New feature?  | yes
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | ~
| License       | MIT
| Doc PR        | symfony/symfony-docs#13182

Currently, the `CachingFactoryDecorator` is responsible for unnecessary memory usage, anytime a choice option is set with a callback option defined as an anonymous function or a loader, then a new hash is generated for the choice list, while we may expect the list to be reused once "finally" configured in a form type or choice type extension.

A simple case is when using one of the core intl choice types in a collection:
```php
// ...
class SomeFormType extends AbstractType
{
    public function buildForm(FormBuilderInterface $builder, array $options)
    {
        $builder
            ->add('some_choices', ChoiceType::class, [
                // before: cached choice list (unnecessary overhead)
                // after: no cache (better perf)
                'choices' => $someObjects,
                'choice_value' => function (?object $choice) { /* return some string */ },
            ])

            // see below the nested effects
            ->add('nested_fields', CollectionType::class, [
                'entry_type' => NestedFormType::class,
            ])
    // ...
}

// ...
class NestedFormType extends AbstractType
{
    public function buildForm(FormBuilderInterface $builder, array $options)
    {
        $builder
            // ...
            ->add('some_other_choices', ChoiceType::class, [
                // before: cached choice list for every entry because we define a new closure instance for each field
                // after: no cache, a bit better for the same result, but much better if it were not nested in a collection
                'choices' => $someOtherObjects,
                'choice_value' => function (?object $otherChoice) { /* return some string */ },
            ])

            ->add('some_loaded_choices', ChoiceType::class, [
                // before: cached but for every entry since every field will have its
                //         own instance of loader, generating a new hash
                // after: no cache, same pro/cons as above
                'choice_loader' => new CallbackChoiceLoader(function() { /* return some choices */}),
                // or
                'choice_loader' => new SomeLoader(),
            ])

            ->add('person', EntityType::class, [
                // before: cached but for every entry, because we define extra `choice_*` option
                // after: no cache, same pro/cons as above
                'class' => SomeEntity::class,
                'choice_label' => function (?SomeEntity $choice) { /* return some label */},
            ])

            // before: cached for every entry, because the type define some "choice_*" option
            // after: cached only once, better perf since the same loader is used for every entry
            ->add('country', CountryType::class)

            // before: cached for every entry, because the type define some "choice_*" option
            // after: no cache, same pro/cons as above
            ->add('locale', LocaleType::class, [
                'preferred_choices' => [ /* some preferred locales */ ],
                'group_by' => function (?string $locale, $label) { /* return some group */ },
            ])
// ...
```

In such cases, we would expect every entries to use the same cached intl choice list, but not, as many list and views as entries will be kept in cache. This is even worse if some callback options like `choice_label`, `choice_value`, `choice_attr`, `choice_name`, `preferred_choices` or `group_by` are used.
This PR helps making cache explicit when needed and ~deprecate~ drop unexpected implicit caching of choice list for most simple cases responsible of unnecessary overhead.

The result is better performance just by upgrading to 5.1 \o/.
But to solve the cases above when cache is needed per options, one should now use the new `ChoiceList` static methods to wrap option values, which is already done internally in this PR.

```php
use Symfony\Component\Form\ChoiceList\ChoiceList;
// ...
class NestedFormType extends AbstractType
{
    public function buildForm(FormBuilderInterface $builder, array $options)
    {
        $builder
            // explicitly shared cached choice lists between entries

            ->add('some_other_choices', ChoiceType::class, [
                'choices' => $someOtherObjects,
                'choice_value' => ChoiceList::value($this, function (?object $otherChoice) {
                    /* return some string */
                }),
            ]),

            ->add('some_loaded_choices', ChoiceType::class, [
                'choice_loader' => ChoiceList::lazy($this, function() {
                    /* return some choices */
                })),
                // or
                'choice_loader' => ChoiceList::loader($this, new SomeLoader()),
            ]),

            ->add('person', EntityType::class, [
                'class' => SomeEntity::class,
                'choice_label' => ChoiceList::label($this, function (?SomeEntity $choice) {
                    /* return some label */
                },
            ])

            // nothing to do :)
            ->add('country', CountryType::class)

            ->add('locale', LocaleType::class, [
                'preferred_choices' => ChoiceList::preferred($this, [ /* some preferred locales */ ]),
                'group_by' => ChoiceList::groupBy($this, function (?string $locale, $label) {
                    /* return some group */
                }),
            ])
// ...
```

I've done some nice profiling with Blackfire and the simple example above in a fresh website skeleton and only two empty entries as initial data, then submitting an empty form. That gives the following results:

 * Rendering the form - Before vs After

  <img width="714" alt="Screenshot 2020-02-16 at 9 24 58 PM" src="https://user-images.githubusercontent.com/10107633/74612132-de533180-5102-11ea-9cc4-296a16949d90.png">

 * Rendering the form - Before vs After with `ChoiceList` helpers

  <img width="670" alt="Screenshot 2020-02-16 at 9 26 51 PM" src="https://user-images.githubusercontent.com/10107633/74612155-122e5700-5103-11ea-9c16-5d80a7541f4b.png">

 * Submitting the form - Before vs After

  <img width="670" alt="Screenshot 2020-02-16 at 9 28 01 PM" src="https://user-images.githubusercontent.com/10107633/74612172-3be77e00-5103-11ea-9a18-4294e05402d2.png">

 * Submitting the form - Before vs After with `ChoiceList` helpers

  <img width="670" alt="Screenshot 2020-02-16 at 9 29 10 PM" src="https://user-images.githubusercontent.com/10107633/74612193-689b9580-5103-11ea-86b9-5b4906200021.png">

_________

TODO:
- [x] Docs
- [x] More profiling
- [x] Add some tests

#EUFOSSA

Commits
-------

b25973c [Form] Added support for caching choice lists based on options
  • Loading branch information
fabpot committed Feb 21, 2020
2 parents f01bbc7 + b25973c commit 269c4a2
Show file tree
Hide file tree
Showing 21 changed files with 835 additions and 119 deletions.
93 changes: 42 additions & 51 deletions src/Symfony/Bridge/Doctrine/Form/Type/DoctrineType.php
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
use Symfony\Bridge\Doctrine\Form\DataTransformer\CollectionToArrayTransformer;
use Symfony\Bridge\Doctrine\Form\EventListener\MergeDoctrineCollectionListener;
use Symfony\Component\Form\AbstractType;
use Symfony\Component\Form\ChoiceList\ChoiceList;
use Symfony\Component\Form\ChoiceList\Factory\CachingFactoryDecorator;
use Symfony\Component\Form\Exception\RuntimeException;
use Symfony\Component\Form\FormBuilderInterface;
Expand All @@ -40,9 +41,9 @@ abstract class DoctrineType extends AbstractType implements ResetInterface
private $idReaders = [];

/**
* @var DoctrineChoiceLoader[]
* @var EntityLoaderInterface[]
*/
private $choiceLoaders = [];
private $entityLoaders = [];

/**
* Creates the label for a choice.
Expand Down Expand Up @@ -115,43 +116,26 @@ public function configureOptions(OptionsResolver $resolver)
$choiceLoader = function (Options $options) {
// Unless the choices are given explicitly, load them on demand
if (null === $options['choices']) {
$hash = null;
$qbParts = null;
// If there is no QueryBuilder we can safely cache
$vary = [$options['em'], $options['class']];

// If there is no QueryBuilder we can safely cache DoctrineChoiceLoader,
// also if concrete Type can return important QueryBuilder parts to generate
// hash key we go for it as well
if (!$options['query_builder'] || null !== $qbParts = $this->getQueryBuilderPartsForCachingHash($options['query_builder'])) {
$hash = CachingFactoryDecorator::generateHash([
$options['em'],
$options['class'],
$qbParts,
]);

if (isset($this->choiceLoaders[$hash])) {
return $this->choiceLoaders[$hash];
}
// hash key we go for it as well, otherwise fallback on the instance
if ($options['query_builder']) {
$vary[] = $this->getQueryBuilderPartsForCachingHash($options['query_builder']) ?? $options['query_builder'];
}

if (null !== $options['query_builder']) {
$entityLoader = $this->getLoader($options['em'], $options['query_builder'], $options['class']);
} else {
$queryBuilder = $options['em']->getRepository($options['class'])->createQueryBuilder('e');
$entityLoader = $this->getLoader($options['em'], $queryBuilder, $options['class']);
}

$doctrineChoiceLoader = new DoctrineChoiceLoader(
return ChoiceList::loader($this, new DoctrineChoiceLoader(
$options['em'],
$options['class'],
$options['id_reader'],
$entityLoader
);

if (null !== $hash) {
$this->choiceLoaders[$hash] = $doctrineChoiceLoader;
}

return $doctrineChoiceLoader;
$this->getCachedEntityLoader(
$options['em'],
$options['query_builder'] ?? $options['em']->getRepository($options['class'])->createQueryBuilder('e'),
$options['class'],
$vary
)
), $vary);
}

return null;
Expand All @@ -162,7 +146,7 @@ public function configureOptions(OptionsResolver $resolver)
// field name. We can only use numeric IDs as names, as we cannot
// guarantee that a non-numeric ID contains a valid form name
if ($options['id_reader'] instanceof IdReader && $options['id_reader']->isIntId()) {
return [__CLASS__, 'createChoiceName'];
return ChoiceList::fieldName($this, [__CLASS__, 'createChoiceName']);
}

// Otherwise, an incrementing integer is used as name automatically
Expand All @@ -176,7 +160,7 @@ public function configureOptions(OptionsResolver $resolver)
$choiceValue = function (Options $options) {
// If the entity has a single-column ID, use that ID as value
if ($options['id_reader'] instanceof IdReader && $options['id_reader']->isSingleId()) {
return [$options['id_reader'], 'getIdValue'];
return ChoiceList::value($this, [$options['id_reader'], 'getIdValue'], $options['id_reader']);
}

// Otherwise, an incrementing integer is used as value automatically
Expand Down Expand Up @@ -214,35 +198,21 @@ public function configureOptions(OptionsResolver $resolver)
// Set the "id_reader" option via the normalizer. This option is not
// supposed to be set by the user.
$idReaderNormalizer = function (Options $options) {
$hash = CachingFactoryDecorator::generateHash([
$options['em'],
$options['class'],
]);

// The ID reader is a utility that is needed to read the object IDs
// when generating the field values. The callback generating the
// field values has no access to the object manager or the class
// of the field, so we store that information in the reader.
// The reader is cached so that two choice lists for the same class
// (and hence with the same reader) can successfully be cached.
if (!isset($this->idReaders[$hash])) {
$classMetadata = $options['em']->getClassMetadata($options['class']);
$this->idReaders[$hash] = new IdReader($options['em'], $classMetadata);
}

if ($this->idReaders[$hash]->isSingleId()) {
return $this->idReaders[$hash];
}

return null;
return $this->getCachedIdReader($options['em'], $options['class']);
};

$resolver->setDefaults([
'em' => null,
'query_builder' => null,
'choices' => null,
'choice_loader' => $choiceLoader,
'choice_label' => [__CLASS__, 'createChoiceLabel'],
'choice_label' => ChoiceList::label($this, [__CLASS__, 'createChoiceLabel']),
'choice_name' => $choiceName,
'choice_value' => $choiceValue,
'id_reader' => null, // internal
Expand Down Expand Up @@ -274,6 +244,27 @@ public function getParent()

public function reset()
{
$this->choiceLoaders = [];
$this->entityLoaders = [];
}

private function getCachedIdReader(ObjectManager $manager, string $class): ?IdReader
{
$hash = CachingFactoryDecorator::generateHash([$manager, $class]);

if (isset($this->idReaders[$hash])) {
return $this->idReaders[$hash];
}

$idReader = new IdReader($manager, $manager->getClassMetadata($class));

// don't cache the instance for composite ids that cannot be optimized
return $this->idReaders[$hash] = $idReader->isSingleId() ? $idReader : null;
}

private function getCachedEntityLoader(ObjectManager $manager, $queryBuilder, string $class, array $vary): EntityLoaderInterface
{
$hash = CachingFactoryDecorator::generateHash($vary);

return $this->entityLoaders[$hash] ?? ($this->entityLoaders[$hash] = $this->getLoader($manager, $queryBuilder, $class));
}
}
24 changes: 12 additions & 12 deletions src/Symfony/Bridge/Doctrine/Tests/Form/Type/EntityTypeTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -1205,13 +1205,13 @@ public function testLoaderCaching()
'property3' => 2,
]);

$choiceLoader1 = $form->get('property1')->getConfig()->getOption('choice_loader');
$choiceLoader2 = $form->get('property2')->getConfig()->getOption('choice_loader');
$choiceLoader3 = $form->get('property3')->getConfig()->getOption('choice_loader');
$choiceList1 = $form->get('property1')->getConfig()->getAttribute('choice_list');
$choiceList2 = $form->get('property2')->getConfig()->getAttribute('choice_list');
$choiceList3 = $form->get('property3')->getConfig()->getAttribute('choice_list');

$this->assertInstanceOf('Symfony\Component\Form\ChoiceList\Loader\ChoiceLoaderInterface', $choiceLoader1);
$this->assertSame($choiceLoader1, $choiceLoader2);
$this->assertSame($choiceLoader1, $choiceLoader3);
$this->assertInstanceOf('Symfony\Component\Form\ChoiceList\LazyChoiceList', $choiceList1);
$this->assertSame($choiceList1, $choiceList2);
$this->assertSame($choiceList1, $choiceList3);
}

public function testLoaderCachingWithParameters()
Expand Down Expand Up @@ -1265,13 +1265,13 @@ public function testLoaderCachingWithParameters()
'property3' => 2,
]);

$choiceLoader1 = $form->get('property1')->getConfig()->getOption('choice_loader');
$choiceLoader2 = $form->get('property2')->getConfig()->getOption('choice_loader');
$choiceLoader3 = $form->get('property3')->getConfig()->getOption('choice_loader');
$choiceList1 = $form->get('property1')->getConfig()->getAttribute('choice_list');
$choiceList2 = $form->get('property2')->getConfig()->getAttribute('choice_list');
$choiceList3 = $form->get('property3')->getConfig()->getAttribute('choice_list');

$this->assertInstanceOf('Symfony\Component\Form\ChoiceList\Loader\ChoiceLoaderInterface', $choiceLoader1);
$this->assertSame($choiceLoader1, $choiceLoader2);
$this->assertSame($choiceLoader1, $choiceLoader3);
$this->assertInstanceOf('Symfony\Component\Form\ChoiceList\LazyChoiceList', $choiceList1);
$this->assertSame($choiceList1, $choiceList2);
$this->assertSame($choiceList1, $choiceList3);
}

protected function createRegistryMock($name, $em)
Expand Down
1 change: 1 addition & 0 deletions src/Symfony/Component/Form/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ CHANGELOG
5.1.0
-----

* Added a `ChoiceList` facade to leverage explicit choice list caching based on options
* Added an `AbstractChoiceLoader` to simplify implementations and handle global optimizations
* The `view_timezone` option defaults to the `model_timezone` if no `reference_date` is configured.
* Added default `inputmode` attribute to Search, Email and Tel form types.
Expand Down
135 changes: 135 additions & 0 deletions src/Symfony/Component/Form/ChoiceList/ChoiceList.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
<?php

/*
* This file is part of the Symfony package.
*
* (c) Fabien Potencier <fabien@symfony.com>
*
* For the full copyright and license information, please view the LICENSE
* file that was distributed with this source code.
*/

namespace Symfony\Component\Form\ChoiceList;

use Symfony\Component\Form\ChoiceList\Factory\Cache\ChoiceAttr;
use Symfony\Component\Form\ChoiceList\Factory\Cache\ChoiceFieldName;
use Symfony\Component\Form\ChoiceList\Factory\Cache\ChoiceLabel;
use Symfony\Component\Form\ChoiceList\Factory\Cache\ChoiceLoader;
use Symfony\Component\Form\ChoiceList\Factory\Cache\ChoiceValue;
use Symfony\Component\Form\ChoiceList\Factory\Cache\GroupBy;
use Symfony\Component\Form\ChoiceList\Factory\Cache\PreferredChoice;
use Symfony\Component\Form\ChoiceList\Loader\CallbackChoiceLoader;
use Symfony\Component\Form\ChoiceList\Loader\ChoiceLoaderInterface;
use Symfony\Component\Form\FormTypeExtensionInterface;
use Symfony\Component\Form\FormTypeInterface;

/**
* A set of convenient static methods to create cacheable choice list options.
*
* @author Jules Pietri <jules@heahprod.com>
*/
final class ChoiceList
{
/**
* Creates a cacheable loader from any callable providing iterable choices.
*
* @param FormTypeInterface|FormTypeExtensionInterface $formType A form type or type extension configuring a cacheable choice list
* @param callable $choices A callable that must return iterable choices or grouped choices
* @param mixed|null $vary Dynamic data used to compute a unique hash when caching the loader
*/
public static function lazy($formType, callable $choices, $vary = null): ChoiceLoader
{
return self::loader($formType, new CallbackChoiceLoader($choices), $vary);
}

/**
* Decorates a loader to make it cacheable.
*
* @param FormTypeInterface|FormTypeExtensionInterface $formType A form type or type extension configuring a cacheable choice list
* @param ChoiceLoaderInterface $loader A loader responsible for creating loading choices or grouped choices
* @param mixed|null $vary Dynamic data used to compute a unique hash when caching the loader
*/
public static function loader($formType, ChoiceLoaderInterface $loader, $vary = null): ChoiceLoader
{
return new ChoiceLoader($formType, $loader, $vary);
}

/**
* Decorates a "choice_value" callback to make it cacheable.
*
* @param FormTypeInterface|FormTypeExtensionInterface $formType A form type or type extension configuring a cacheable choice list
* @param callable $value Any pseudo callable to create a unique string value from a choice
* @param mixed|null $vary Dynamic data used to compute a unique hash when caching the callback
*/
public static function value($formType, $value, $vary = null): ChoiceValue
{
return new ChoiceValue($formType, $value, $vary);
}

/**
* Decorates a "choice_label" option to make it cacheable.
*
* @param FormTypeInterface|FormTypeExtensionInterface $formType A form type or type extension configuring a cacheable choice list
* @param callable|false $label Any pseudo callable to create a label from a choice or false to discard it
* @param mixed|null $vary Dynamic data used to compute a unique hash when caching the option
*/
public static function label($formType, $label, $vary = null): ChoiceLabel
{
return new ChoiceLabel($formType, $label, $vary);
}

/**
* Decorates a "choice_name" callback to make it cacheable.
*
* @param FormTypeInterface|FormTypeExtensionInterface $formType A form type or type extension configuring a cacheable choice list
* @param callable $fieldName Any pseudo callable to create a field name from a choice
* @param mixed|null $vary Dynamic data used to compute a unique hash when caching the callback
*/
public static function fieldName($formType, $fieldName, $vary = null): ChoiceFieldName
{
return new ChoiceFieldName($formType, $fieldName, $vary);
}

/**
* Decorates a "choice_attr" option to make it cacheable.
*
* @param FormTypeInterface|FormTypeExtensionInterface $formType A form type or type extension configuring a cacheable choice list
* @param callable|array $attr Any pseudo callable or array to create html attributes from a choice
* @param mixed|null $vary Dynamic data used to compute a unique hash when caching the option
*/
public static function attr($formType, $attr, $vary = null): ChoiceAttr
{
return new ChoiceAttr($formType, $attr, $vary);
}

/**
* Decorates a "group_by" callback to make it cacheable.
*
* @param FormTypeInterface|FormTypeExtensionInterface $formType A form type or type extension configuring a cacheable choice list
* @param callable $groupBy Any pseudo callable to return a group name from a choice
* @param mixed|null $vary Dynamic data used to compute a unique hash when caching the callback
*/
public static function groupBy($formType, $groupBy, $vary = null): GroupBy
{
return new GroupBy($formType, $groupBy, $vary);
}

/**
* Decorates a "preferred_choices" option to make it cacheable.
*
* @param FormTypeInterface|FormTypeExtensionInterface $formType A form type or type extension configuring a cacheable choice list
* @param callable|array $preferred Any pseudo callable or array to return a group name from a choice
* @param mixed|null $vary Dynamic data used to compute a unique hash when caching the option
*/
public static function preferred($formType, $preferred, $vary = null): PreferredChoice
{
return new PreferredChoice($formType, $preferred, $vary);
}

/**
* Should not be instantiated.
*/
private function __construct()
{
}
}

0 comments on commit 269c4a2

Please sign in to comment.