Jabxjab archive:dump and archive:restore commands (#5148)

* Add ArchiveCommands.php * Add SQL dump operation * Add "description", "tags", "destination" and "overwrite" options to annotation * Refactor getDrupalFilesDir(); Create database.tar sub-archive; Create files.tar sub-archive; Create master archive.tar.gz * Refactor creating database and file archives * Add createMasterArchive() * Add createManifestFile() * Rename "db_dump" option into "db" * Update default generator value * Add "--destination" and "--overwrite" options * Update info messages * Log info when creating manifest file * Remove "master" archive mentions in logs * Refactor archive structure ("database" and "files" are now directories) * Add "code" component * Add getExcludesByPaths() * Refactor code excludes * Exclude css, js, php and styles from file component * Exclude from code archive everything from web/ except modules, themes, profiles * Refactor Drupal contrib projects excludes * Exclude sites/*/settings.local.php from code archive * Add validateSensitiveData() * Remove todo * Fix method name * Fix getCodeComponentPath() * Rename getCodeComponentPath() into getProjectPath() * Add site validation * Update command description * Fix validateSite() * Add ArchiveTest * Fix code style issues * Add support for non-Composer managed, non-web docroot sites * Fix command for non-web docroot sites * Exclude settings.*.php * Fix testArchiveDumpCommand() test * Add cleanUp() shutdown function Use Symfony Filesystem component * Copy code archive component to archive dir * Use constants for archive component names * Refactor excludes * Simplify cleanUp() * Rename constants * Rename getExcludesByPaths() into getRegexpsForPaths() * Update phpdocs * Rename var * Use Path::join() * Use array_map in getRegexpsForPaths() * Rename getIterator() * Fix missing manifest file * Return absolute path to the archive * Delete temp archive.tar.gz if exists * Add scenarios to testArchiveDumpCommand() * Fix code style issues * Fix testArchiveDumpCommand() * Rename ArchiveDumpCommands and refactor getDrupalFilesComponentPath() * Remove setSimulated() * Set php memory_limit to 256M in appveyor.yml * Fix file iterator local file path handling * Remove unused constants * Create temp dir on command execution only * Add ArchiveRestoreCommands backbone * Add confirmation prompts * Implement extractArchive() * Add check for file existence * Add check for filename extension * Add todo * Import code file for a local target only * Import code files for a remote targets * Add 'docroot' property to the manifest file * Fix $aliasConfigContext * Fix nested php includes in validateSensitiveData() * Bootstrap Drupal core to get a path to Drupal public files * Disable type3/phar-stream-wrapper which loads with Drupal core and prevents creating *.tar archives * Fix getDrupalFilesComponentPath() * Remove Site Alias manager * Refactor validateSensitiveData() to include site/*/settings.php file without include/require directives * Fix scenario when backup dir is inside code dir (and on Windows OS) * Fix scenario when backup dir is inside code dir (and on Windows OS) using \Webmozart\PathUtil\Path util * Fix getCodeComponentPath() * Improve verbosity in validateSensitiveData() * Add dt() to Exception messages * Refactor Drupal-related excludes - exclude paths defined in composer.json in "extra"/"installer-paths" section * Detect Drupal files exclude for code component * Use 'composer info --path --format=json' to exclude packages installation paths * Add "composer" as alias to "php composer.phar" in appveyor.yml * Rename $installedPackagesBaseDirs into $installedPackagesRelativePaths * Use native Symfony\Process for "composer info" command execution * Add '%composer-root' to core:status command * Add 'composer-root' to @field-labels/@default-fields of core:status command * Detect composer root for a remote site * Add rsyncFiles() * Add importFiles() * Add getSiteAlias(), getSiteStatus() * Fix $evaluatedPath instantiation for a remote site * Detect composer root for a remote site * Add rsyncFiles() * Add importFiles() * Add getSiteAlias(), getSiteStatus() * Fix $evaluatedPath instantiation for a remote site * Add blueprint of importDatabase() * Update importDatabase() * Fix todos * Fix code style issues * Add testArchiveRestoreCommand() to ArchiveTest * Add test scenario for "--overwrite" option * Add test scenario for "--code", "--code_path", "--db", "--db_path", "--files", "--files_path" options * PROPOSAL: Replace direct use of register_shutdown_function with Robo _tmpDir() facility. * Proposal 2: Provide our own tmpDir service in FsUtils * Code style * Simplify tests: do one less archive:dump, and instead use file_put_contents to test with and without --overwrite. * Update download url of cacert to new location * Try curl instead of DownloadFile * Skip tests of unsupported configurations * Remove eval() from validateSensitiveData() * Remove support for a remote site for archive:restore command * Add an option to restore code into current dir if empty * Remove ssh-related option * Add --destination_path option * Rename archive:restore options * Do not check $destination dir (creates automatically by rsync) * Throw exception on Drupal bootstrap failure when getting Drupal files path * Add archive:restore options: db-name, db-port, db-host, db-user, db-password, db-prefix, db-driver * Replace underscores with dashes in option names * Update tests for archive:restore * Remove $autodetectDestination * Fix indentations * Disable SecurityUpdatesTest.php temporarily * Require database connection option(s) when --destination-path options is provided * Add support for relative paths in --destination-path option * Add test scenarios for testArchiveRestoreCommand() * Set sut db password to empty string * Use self::getDbUrl() to construct --db-url * Add tearDown() to delete a test file * Update testing scenario to use invalid db host instead of password * Add --db-driver option in tests * Add testing scenarios for database component involving --destination-path option * Revert "Disable SecurityUpdatesTest.php temporarily" This reverts commit f7e66e9. * Allow archive:restore to extract archive and files to cwd. Improve FsUtils::tmpDir, and use it in archive:restore. Add default options to sut to avoid including Drush sources in archive:dumps of the sut, for more convenient ad-hoc testing. * Code style * WIP: Avoid overwriting sut in archive restore tests * Fix var name typo * Remove unused namespace, @throws tag and $root var * Remove unused namespaces and ARCHIVE_SUBDIR_NAME const * Refactor extractArchive() into getExtractDir() * Create database if not exists database import * Remove duplicated testing scenarios * Update archive:restore tests * Remove tearDown() * Use setDbSpec() before calling query() in ArchiveRestoreCommands::importDatabase() * Fix drop/create db in importDatabase() * Remove escaping "\" in --exclude-code-paths * Add composer.json and composer.lock to a SUT's archive * Run 'composer install' after archive:restore in testArchiveRestoreCommand() * Manage SUT's settings.php file * Refactor settings.php file setup procedures * Remove unused arg * Delete destination directory in case when code component is involved and add a confirmation dialog * Add confirmation dialog for Drupal files restore if destination path already exists and --overwrite is not set * Fix tests * Show prompt on file import if Drupal files dir exists and one of --code or --overwrite option is not set * Refactor setupSettingsPhp() * Refactor backupSettingsPhp() * Add installComposerDependencies() * Add assertRestoredSiteStatus() * Remove "composer install" code in favor of the method * Remove excessive assertion * Add "composer install" to testArchiveRestoreCommand() * Change $filesRelativePath to a real value * Assert the restored site is OK * Add more assertions that the restored site is OK * Update dependencies to fix testNoInsecureProductionPhpPackage failure. * Remove "composer install" from tests * Rename "destination path" property/method name * Add --code-no-composer-install and --code-composer-install-timeout options * Add "composer install" to importCode() * Remove setupSettingsPhp() and backupSettingsPhp() from tests * Add --site-subdir and --setup-database-connection options * Add getDrupalRootPath() * Update $options default value * Add setupLocalSettingsPhp() * Update testArchiveRestoreCommand() * Fix code style issue * Add saveSettingsLocalPhp() * Add database connection settings to settings.local.php file * Update dependencies to fix testNoInsecureProductionPhpPackage failure. * Fix null command references (in service commands list) during Drupal bootstrap phase (which causes "PHP Fatal error: Cannot redeclare drupal_error_levels() (previously declared in..." in test_81_drupal10_highest CI pipeline) * Jabxjab archive dump command updates (#5165) * field-delete: Fix field being deleted from all bundles instead of only the requested bundle (#5158) * Updates input default options and provides destination validation. * Removes automatic composer install and adds user feedback. * Resolves phpcs feedback. * Adds composer install to test to allow Drupal installation. * Resolves code sniff. * Adds needed class. * Runs composer update to resolve security warnings. * Updates guzzle version. Co-authored-by: Dieter Holvoet <dieter.holvoet@gmail.com> Co-authored-by: Ryan Wagner <ryan.wagner@pantheon.io> * Validate that the Phar extension is available * Remove class phpdoc descriptions * Remove unused "use" statement * Remove --code-no-composer-install and --code-composer-install-timeout options * Remove @var phpdocs on explicitly declared class properties * Add @Usage examples for archive:restore * Add @Usage examples for archive:dump * Validate "rsync" exists for archive:restore command Co-authored-by: Sergei Churilo <sergei.churilo@pantheon.io> Co-authored-by: Sergei Churilo <jabxjab@gmail.com> Co-authored-by: J Ryan Wagner <jryanwagner00@gmail.com> Co-authored-by: Dieter Holvoet <dieter.holvoet@gmail.com> Co-authored-by: Ryan Wagner <ryan.wagner@pantheon.io>
drush-ops · Jul 7, 2022 · 7f24f24 · 7f24f24
1 parent 36924fa
commit 7f24f24
Show file tree

Hide file tree

Showing 10 changed files with 1,765 additions and 13 deletions.
diff --git a/appveyor.yml b/appveyor.yml
@@ -30,7 +30,8 @@ install:
   - copy php.ini-production php.ini
 
   # https://github.com/php-coveralls/php-coveralls/pull/223/files
-  - appveyor DownloadFile http://curl.haxx.se/ca/cacert.pem -FileName C:\cacert.pem
+  # - appveyor DownloadFile https://curl.se/ca/cacert.pem -FileName C:\cacert.pem
+  - curl -fsS -o C:\cacert.pem https://curl.se/ca/cacert.pem
   - echo curl.cainfo=C:\cacert.pem >> php.ini
 
   - echo extension_dir=ext >> php.ini
@@ -48,12 +49,14 @@ install:
   - echo extension=php_pgsql.dll >> php.ini
   - echo extension=php_gd2.dll >> php.ini
   - echo extension=php_fileinfo.dll >> php.ini
+  - echo memory_limit=256M >> php.ini
   - SET PATH=C:\tools\php74;%PATH%
   #Install Composer
   - cd %APPVEYOR_BUILD_FOLDER%
   #- appveyor DownloadFile https://getcomposer.org/composer.phar
   - php -r "readfile('http://getcomposer.org/installer');" | php
-  #Install dependencies via Composer
+  - echo @php %cd%\composer.phar %%* > composer.bat
+  # Install dependencies via Composer.
   - php composer.phar install --prefer-dist -n
   - SET PATH=%APPVEYOR_BUILD_FOLDER%;%APPVEYOR_BUILD_FOLDER%/vendor/bin;%PATH%
   # Uncomment this and on_finish line below to enable RDP into build machine https://www.appveyor.com/docs/how-to/rdp-to-build-worker/

diff --git a/composer.json b/composer.json
@@ -41,7 +41,7 @@
     "consolidation/site-alias": "^3.1.3",
     "consolidation/site-process": "^4.1.3 || ^5",
     "enlightn/security-checker": "^1",
-    "guzzlehttp/guzzle": "^6.3 || ^7.0",
+    "guzzlehttp/guzzle": "^6.5 || ^7.0",
     "league/container": "^3.4 || ^4",
     "psy/psysh": "~0.11",
     "symfony/event-dispatcher": "^4.0 || ^5.0 || ^6.0",

diff --git a/composer.lock b/composer.lock
diff --git a/src/Boot/DrupalBoot8.php b/src/Boot/DrupalBoot8.php
@@ -263,26 +263,26 @@ public function addDrupalModuleDrushCommands($manager): void
 
         // Set the command info alterers.
         if ($container->has(DrushServiceModifier::DRUSH_COMMAND_INFO_ALTERER_SERVICES)) {
-            $serviceCommandInfoAltererlist = $container->get(DrushServiceModifier::DRUSH_COMMAND_INFO_ALTERER_SERVICES);
+            $serviceCommandInfoAltererList = $container->get(DrushServiceModifier::DRUSH_COMMAND_INFO_ALTERER_SERVICES);
             $commandFactory = Drush::commandFactory();
-            foreach ($serviceCommandInfoAltererlist->getCommandList() as $altererHandler) {
+            foreach ($serviceCommandInfoAltererList->getCommandList() as $altererHandler) {
                 $commandFactory->addCommandInfoAlterer($altererHandler);
                 $this->logger->debug(dt('Commands are potentially altered in !class.', ['!class' => get_class($altererHandler)]));
             }
         }
 
-        $serviceCommandlist = $container->get(DrushServiceModifier::DRUSH_CONSOLE_SERVICES);
         if ($container->has(DrushServiceModifier::DRUSH_CONSOLE_SERVICES)) {
-            foreach ($serviceCommandlist->getCommandList() as $command) {
+            $serviceCommandList = $container->get(DrushServiceModifier::DRUSH_CONSOLE_SERVICES);
+            foreach ($serviceCommandList->getCommandList() as $command) {
                 $manager->inflect($command);
                 $this->logger->debug(dt('Add a command: !name', ['!name' => $command->getName()]));
                 $application->add($command);
             }
         }
         // Do the same thing with the annotation commands.
         if ($container->has(DrushServiceModifier::DRUSH_COMMAND_SERVICES)) {
-            $serviceCommandlist = $container->get(DrushServiceModifier::DRUSH_COMMAND_SERVICES);
-            foreach ($serviceCommandlist->getCommandList() as $commandHandler) {
+            $serviceCommandList = $container->get(DrushServiceModifier::DRUSH_COMMAND_SERVICES);
+            foreach ($serviceCommandList->getCommandList() as $commandHandler) {
                 $manager->inflect($commandHandler);
                 $this->logger->debug(dt('Add a commandfile class: !name', ['!name' => get_class($commandHandler)]));
                 $runner->registerCommandClass($application, $commandHandler);

diff --git a/src/Command/ServiceCommandlist.php b/src/Command/ServiceCommandlist.php
@@ -17,6 +17,6 @@ public function addCommandReference($command): void
 
     public function getCommandList(): array
     {
-        return $this->commandList;
+        return array_filter($this->commandList);
     }
 }