Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jabxjab archive:dump and archive:restore commands #5148

Merged
merged 214 commits into from Jul 7, 2022
Merged
Show file tree
Hide file tree
Changes from 213 commits
Commits
Show all changes
214 commits
Select commit Hold shift + click to select a range
4cf7316
Add ArchiveCommands.php
Jan 24, 2022
81ba6c9
Add SQL dump operation
Jan 24, 2022
90a8681
Add "description", "tags", "destination" and "overwrite" options to a…
Jan 24, 2022
6e8911d
Refactor getDrupalFilesDir();
Jan 25, 2022
e3755c9
Refactor creating database and file archives
Jan 25, 2022
114071d
Add createMasterArchive()
Jan 25, 2022
421651d
Add createManifestFile()
Jan 25, 2022
099c8a7
Rename "db_dump" option into "db"
Jan 25, 2022
81ba0dd
Update default generator value
Jan 25, 2022
d6266dd
Add "--destination" and "--overwrite" options
Jan 25, 2022
4a25245
Update info messages
Jan 25, 2022
a225279
Log info when creating manifest file
Jan 25, 2022
06dfc08
Remove "master" archive mentions in logs
Jan 25, 2022
787493d
Refactor archive structure ("database" and "files" are now directories)
Jan 26, 2022
1b12911
Add "code" component
Jan 26, 2022
c4e62f4
Add getExcludesByPaths()
Jan 26, 2022
4e7828b
Refactor code excludes
Jan 26, 2022
13bc858
Exclude css, js, php and styles from file component
Jan 26, 2022
d70cbe0
Exclude from code archive everything from web/ except modules, themes…
Jan 27, 2022
409d5f0
Refactor Drupal contrib projects excludes
Jan 27, 2022
e55fce7
Exclude sites/*/settings.local.php from code archive
Jan 27, 2022
d3346ae
Add validateSensitiveData()
Jan 27, 2022
5fabfc5
Remove todo
Jan 27, 2022
17e2086
Fix method name
Jan 28, 2022
d97e02d
Fix getCodeComponentPath()
Jan 28, 2022
fe14016
Rename getCodeComponentPath() into getProjectPath()
Jan 28, 2022
4dec98b
Add site validation
Jan 28, 2022
12cd1fc
Update command description
Jan 28, 2022
d2df4ed
Fix validateSite()
Jan 28, 2022
e52ee2e
Add ArchiveTest
Jan 28, 2022
329bbca
Fix code style issues
Jan 28, 2022
dd40537
Add support for non-Composer managed, non-web docroot sites
Jan 31, 2022
9b5a6c8
Fix command for non-web docroot sites
Jan 31, 2022
8ac02b6
Exclude settings.*.php
Jan 31, 2022
a850997
Fix testArchiveDumpCommand() test
Jan 31, 2022
a300d59
Add cleanUp() shutdown function
Feb 1, 2022
2bfc9eb
Copy code archive component to archive dir
Feb 1, 2022
2199a09
Use constants for archive component names
Feb 1, 2022
9e4b75f
Refactor excludes
Feb 1, 2022
977900c
Simplify cleanUp()
Feb 1, 2022
d67faf6
Rename constants
Feb 1, 2022
95b74b4
Rename getExcludesByPaths() into getRegexpsForPaths()
Feb 1, 2022
7b7476e
Update phpdocs
Feb 1, 2022
95c2f1e
Merge branch '11.x' into archive-dump-command
jabxjab Feb 1, 2022
90dc700
Rename var
Feb 1, 2022
5f81996
Use Path::join()
Feb 1, 2022
8a5ffd0
Use array_map in getRegexpsForPaths()
Feb 1, 2022
f86121e
Rename getIterator()
Feb 1, 2022
9bd56d7
Merge branch '11.x' into archive-dump-command
jabxjab Feb 1, 2022
6eb1086
Fix missing manifest file
Feb 2, 2022
8a9d851
Return absolute path to the archive
Feb 2, 2022
7f05ffe
Delete temp archive.tar.gz if exists
Feb 2, 2022
948c7fe
Add scenarios to testArchiveDumpCommand()
Feb 2, 2022
910780b
Fix code style issues
Feb 2, 2022
a73e6f8
Merge remote-tracking branch 'origin/archive-dump-command' into archi…
Feb 2, 2022
d97616c
Fix testArchiveDumpCommand()
Feb 2, 2022
8231582
Rename ArchiveDumpCommands and refactor getDrupalFilesComponentPath()
Feb 2, 2022
c02c576
Remove setSimulated()
Feb 2, 2022
08a502c
Set php memory_limit to 256M in appveyor.yml
Feb 2, 2022
198821d
Fix file iterator local file path handling
Feb 2, 2022
ac05787
Remove unused constants
Feb 3, 2022
866e304
Merge branch '11.x' into archive-dump-command
jabxjab Mar 3, 2022
83c448d
Create temp dir on command execution only
Mar 3, 2022
b0919a7
Merge remote-tracking branch 'origin/archive-dump-command' into archi…
Mar 3, 2022
646e1f6
Add ArchiveRestoreCommands backbone
Mar 3, 2022
046f089
Add confirmation prompts
Mar 4, 2022
7114237
Implement extractArchive()
Mar 4, 2022
cf638cb
Add check for file existence
Mar 9, 2022
53b8fc4
Add check for filename extension
Mar 9, 2022
05e8c81
Merge branch '11.x' into archive-dump-command
jabxjab Mar 9, 2022
ad0cbfc
Add todo
Mar 9, 2022
7bb8445
Import code file for a local target only
Mar 9, 2022
d9a3eaf
Import code files for a remote targets
Mar 9, 2022
accaa1c
Add 'docroot' property to the manifest file
Mar 9, 2022
2238007
Fix $aliasConfigContext
Mar 10, 2022
87faad9
Merge branch '11.x' into archive-dump-command
jabxjab Mar 10, 2022
95fdf66
Fix nested php includes in validateSensitiveData()
Mar 11, 2022
ebdd7e1
Bootstrap Drupal core to get a path to Drupal public files
Mar 13, 2022
d6f76b9
Disable type3/phar-stream-wrapper which loads with Drupal core and pr…
Mar 13, 2022
57c5b83
Fix getDrupalFilesComponentPath()
Mar 13, 2022
a0978e7
Remove Site Alias manager
Mar 13, 2022
f9988db
Refactor validateSensitiveData() to include site/*/settings.php file …
Mar 13, 2022
8ef4743
Merge branch '11.x' into archive-dump-command
jabxjab Mar 13, 2022
b8e0682
Fix scenario when backup dir is inside code dir (and on Windows OS)
Mar 13, 2022
36ed35b
Merge remote-tracking branch 'origin/archive-dump-command' into archi…
Mar 13, 2022
ba3c0be
Fix scenario when backup dir is inside code dir (and on Windows OS) u…
Mar 13, 2022
3cbe088
Merge remote-tracking branch 'origin/archive-dump-command' into archi…
Mar 13, 2022
0f760ff
Fix getCodeComponentPath()
Mar 13, 2022
e0d03ba
Improve verbosity in validateSensitiveData()
Mar 13, 2022
4c939df
Add dt() to Exception messages
Mar 14, 2022
650a372
Refactor Drupal-related excludes - exclude paths defined in composer.…
Mar 14, 2022
0d7d837
Detect Drupal files exclude for code component
Mar 14, 2022
2fc7c92
Use 'composer info --path --format=json' to exclude packages installa…
Mar 14, 2022
a486209
Add "composer" as alias to "php composer.phar" in appveyor.yml
Mar 14, 2022
25cd93a
Rename $installedPackagesBaseDirs into $installedPackagesRelativePaths
Mar 14, 2022
da8a687
Use native Symfony\Process for "composer info" command execution
Mar 14, 2022
a3d1022
Add '%composer-root' to core:status command
Mar 10, 2022
ad44967
Add 'composer-root' to @field-labels/@default-fields of core:status c…
Mar 10, 2022
310162b
Detect composer root for a remote site
Mar 10, 2022
cbc4811
Add rsyncFiles()
Mar 11, 2022
b7a8e3a
Add importFiles()
Mar 11, 2022
454a390
Add getSiteAlias(), getSiteStatus()
Mar 11, 2022
cd827dc
Fix $evaluatedPath instantiation for a remote site
Mar 11, 2022
aa47e32
Detect composer root for a remote site
Mar 10, 2022
a97535c
Add rsyncFiles()
Mar 11, 2022
e6d28f9
Add importFiles()
Mar 11, 2022
6c7f2cb
Add getSiteAlias(), getSiteStatus()
Mar 11, 2022
ffaf9d0
Fix $evaluatedPath instantiation for a remote site
Mar 11, 2022
c508306
Add blueprint of importDatabase()
Mar 14, 2022
4ad10c2
Update importDatabase()
Mar 15, 2022
d235854
Fix todos
Mar 15, 2022
1a47d72
Fix code style issues
Mar 15, 2022
994c681
Merge remote-tracking branch 'upstream/11.x' into archive-restore-com…
Mar 15, 2022
54535ca
Add testArchiveRestoreCommand() to ArchiveTest
Mar 15, 2022
b724bf4
Merge branch '11.x' into archive-dump-command
jabxjab Mar 16, 2022
dcb9164
Merge branch '11.x' into archive-restore-command
jabxjab Mar 16, 2022
ed49878
Add test scenario for "--overwrite" option
Mar 28, 2022
dc0f3bb
Add test scenario for "--code", "--code_path", "--db", "--db_path", "…
Mar 28, 2022
4a8f260
Merge branch 'archive-dump-command' of https://github.com/jabxjab/dru…
greg-1-anderson May 18, 2022
48a4765
PROPOSAL: Replace direct use of register_shutdown_function with Robo …
greg-1-anderson May 19, 2022
4854e03
Proposal 2: Provide our own tmpDir service in FsUtils
greg-1-anderson May 19, 2022
f4bb527
Code style
greg-1-anderson May 19, 2022
52ef17b
Simplify tests: do one less archive:dump, and instead use file_put_co…
greg-1-anderson May 20, 2022
27bed6a
Update download url of cacert to new location
greg-1-anderson May 20, 2022
cfc8dce
Try curl instead of DownloadFile
greg-1-anderson May 20, 2022
2058727
Merge branch 'archive-restore-command' of https://github.com/jabxjab/…
greg-1-anderson May 20, 2022
9fb02c3
Merge branch '11.x' into jabxjab-archive-dump-command
greg-1-anderson May 20, 2022
3f1b8ce
Skip tests of unsupported configurations
greg-1-anderson May 20, 2022
16e51df
Remove eval() from validateSensitiveData()
May 23, 2022
b28124e
Remove support for a remote site for archive:restore command
May 23, 2022
1516fd5
Add an option to restore code into current dir if empty
May 24, 2022
e826896
Remove ssh-related option
May 25, 2022
3f5da75
Add --destination_path option
May 25, 2022
cecf35f
Rename archive:restore options
May 25, 2022
7ca3fbd
Do not check $destination dir (creates automatically by rsync)
May 25, 2022
af6d29f
Throw exception on Drupal bootstrap failure when getting Drupal files…
May 25, 2022
443c008
Add archive:restore options: db-name, db-port, db-host, db-user, db-p…
May 25, 2022
635b332
Replace underscores with dashes in option names
May 25, 2022
4c86392
Update tests for archive:restore
May 25, 2022
a016676
Remove $autodetectDestination
May 26, 2022
233e746
Fix indentations
May 26, 2022
f7e66e9
Disable SecurityUpdatesTest.php temporarily
May 26, 2022
85b87a9
Require database connection option(s) when --destination-path options…
May 26, 2022
5c3f925
Add support for relative paths in --destination-path option
May 26, 2022
e681e9b
Merge branch '11.x' into jabxjab-archive-dump-command
jabxjab May 27, 2022
2989b10
Add test scenarios for testArchiveRestoreCommand()
May 27, 2022
a6bb6bb
Set sut db password to empty string
May 27, 2022
8c5ea64
Use self::getDbUrl() to construct --db-url
May 27, 2022
951c1a7
Add tearDown() to delete a test file
May 27, 2022
de2fbdc
Update testing scenario to use invalid db host instead of password
May 27, 2022
4f869fa
Add --db-driver option in tests
May 27, 2022
17e3184
Add testing scenarios for database component involving --destination-…
May 27, 2022
237c8f5
Revert "Disable SecurityUpdatesTest.php temporarily"
May 27, 2022
7394da5
Merge branch '11.x' into jabxjab-archive-dump-command
jabxjab May 30, 2022
d6ad8f7
Allow archive:restore to extract archive and files to cwd. Improve Fs…
greg-1-anderson Jun 3, 2022
6c83ac8
Code style
greg-1-anderson Jun 3, 2022
443c63f
WIP: Avoid overwriting sut in archive restore tests
greg-1-anderson Jun 3, 2022
37bbae7
Fix var name typo
Jun 6, 2022
e309b62
Remove unused namespace, @throws tag and $root var
Jun 6, 2022
a73205d
Remove unused namespaces and ARCHIVE_SUBDIR_NAME const
Jun 6, 2022
e3a33f5
Refactor extractArchive() into getExtractDir()
Jun 6, 2022
50b21a9
Create database if not exists database import
Jun 6, 2022
6d99d63
Remove duplicated testing scenarios
Jun 6, 2022
79f132a
Update archive:restore tests
Jun 6, 2022
0aaf2f7
Remove tearDown()
Jun 6, 2022
0af00db
Use setDbSpec() before calling query() in ArchiveRestoreCommands::imp…
Jun 6, 2022
91bbff0
Fix drop/create db in importDatabase()
Jun 6, 2022
2cfc814
Remove escaping "\" in --exclude-code-paths
Jun 7, 2022
76453b5
Add composer.json and composer.lock to a SUT's archive
Jun 7, 2022
085bf85
Run 'composer install' after archive:restore in testArchiveRestoreCom…
Jun 7, 2022
977048a
Manage SUT's settings.php file
Jun 7, 2022
a1546dc
Refactor settings.php file setup procedures
Jun 7, 2022
d87be11
Remove unused arg
Jun 7, 2022
f17df0e
Delete destination directory in case when code component is involved …
Jun 7, 2022
bc3576d
Add confirmation dialog for Drupal files restore if destination path …
Jun 7, 2022
05b0014
Fix tests
Jun 8, 2022
663940a
Show prompt on file import if Drupal files dir exists and one of --co…
Jun 8, 2022
c0d7857
Refactor setupSettingsPhp()
Jun 8, 2022
f88d5e6
Refactor backupSettingsPhp()
Jun 8, 2022
bb57463
Add installComposerDependencies()
Jun 8, 2022
5409dcc
Add assertRestoredSiteStatus()
Jun 8, 2022
27a9c8a
Remove "composer install" code in favor of the method
Jun 8, 2022
21fd98c
Remove excessive assertion
Jun 8, 2022
a537380
Add "composer install" to testArchiveRestoreCommand()
Jun 8, 2022
8c4a526
Change $filesRelativePath to a real value
Jun 8, 2022
8470eee
Assert the restored site is OK
Jun 8, 2022
ca2ba94
Add more assertions that the restored site is OK
Jun 8, 2022
1d87ff0
Update dependencies to fix testNoInsecureProductionPhpPackage failure.
greg-1-anderson Jun 8, 2022
6c1affb
Remove "composer install" from tests
Jun 9, 2022
713f7ff
Rename "destination path" property/method name
Jun 9, 2022
67545c7
Add --code-no-composer-install and --code-composer-install-timeout op…
Jun 9, 2022
4b5ad2c
Add "composer install" to importCode()
Jun 9, 2022
7a16377
Remove setupSettingsPhp() and backupSettingsPhp() from tests
Jun 9, 2022
9345138
Add --site-subdir and --setup-database-connection options
Jun 9, 2022
0e20a6b
Add getDrupalRootPath()
Jun 9, 2022
bd0bcbb
Update $options default value
Jun 9, 2022
79528c5
Add setupLocalSettingsPhp()
Jun 10, 2022
3ee5142
Update testArchiveRestoreCommand()
Jun 10, 2022
48fed8d
Fix code style issue
Jun 10, 2022
fa4668d
Add saveSettingsLocalPhp()
Jun 10, 2022
81e8d01
Add database connection settings to settings.local.php file
Jun 10, 2022
950c41b
Update dependencies to fix testNoInsecureProductionPhpPackage failure.
Jun 13, 2022
841fa04
Fix null command references (in service commands list) during Drupal …
Jun 14, 2022
2a0b7c9
Jabxjab archive dump command updates (#5165)
rwagner00 Jun 23, 2022
4639811
Validate that the Phar extension is available
greg-1-anderson Jun 23, 2022
f5cf904
Merge branch '11.x' into jabxjab-archive-dump-command
jabxjab Jun 27, 2022
44fc67e
Remove class phpdoc descriptions
Jun 27, 2022
14b2a81
Remove unused "use" statement
Jun 27, 2022
b49910a
Remove --code-no-composer-install and --code-composer-install-timeout…
Jun 27, 2022
47b0cf7
Remove @var phpdocs on explicitly declared class properties
Jun 27, 2022
1224b03
Add @usage examples for archive:restore
Jun 28, 2022
9585fbc
Add @usage examples for archive:dump
Jun 28, 2022
2ec211c
Validate "rsync" exists for archive:restore command
Jun 28, 2022
5cdba2a
Merge branch '11.x' into jabxjab-archive-dump-command
greg-1-anderson Jul 7, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
7 changes: 5 additions & 2 deletions appveyor.yml
Expand Up @@ -30,7 +30,8 @@ install:
- copy php.ini-production php.ini

# https://github.com/php-coveralls/php-coveralls/pull/223/files
- appveyor DownloadFile http://curl.haxx.se/ca/cacert.pem -FileName C:\cacert.pem
# - appveyor DownloadFile https://curl.se/ca/cacert.pem -FileName C:\cacert.pem
- curl -fsS -o C:\cacert.pem https://curl.se/ca/cacert.pem
- echo curl.cainfo=C:\cacert.pem >> php.ini

- echo extension_dir=ext >> php.ini
Expand All @@ -48,12 +49,14 @@ install:
- echo extension=php_pgsql.dll >> php.ini
- echo extension=php_gd2.dll >> php.ini
- echo extension=php_fileinfo.dll >> php.ini
- echo memory_limit=256M >> php.ini
- SET PATH=C:\tools\php74;%PATH%
#Install Composer
- cd %APPVEYOR_BUILD_FOLDER%
#- appveyor DownloadFile https://getcomposer.org/composer.phar
- php -r "readfile('http://getcomposer.org/installer');" | php
#Install dependencies via Composer
- echo @php %cd%\composer.phar %%* > composer.bat
# Install dependencies via Composer.
- php composer.phar install --prefer-dist -n
- SET PATH=%APPVEYOR_BUILD_FOLDER%;%APPVEYOR_BUILD_FOLDER%/vendor/bin;%PATH%
# Uncomment this and on_finish line below to enable RDP into build machine https://www.appveyor.com/docs/how-to/rdp-to-build-worker/
Expand Down
2 changes: 1 addition & 1 deletion composer.json
Expand Up @@ -41,7 +41,7 @@
"consolidation/site-alias": "^3.1.3",
"consolidation/site-process": "^4.1.3 || ^5",
"enlightn/security-checker": "^1",
"guzzlehttp/guzzle": "^6.3 || ^7.0",
"guzzlehttp/guzzle": "^6.5 || ^7.0",
"league/container": "^3.4 || ^4",
"psy/psysh": "~0.11",
"symfony/event-dispatcher": "^4.0 || ^5.0 || ^6.0",
Expand Down
2 changes: 1 addition & 1 deletion composer.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 6 additions & 6 deletions src/Boot/DrupalBoot8.php
Expand Up @@ -263,26 +263,26 @@ public function addDrupalModuleDrushCommands($manager): void

// Set the command info alterers.
if ($container->has(DrushServiceModifier::DRUSH_COMMAND_INFO_ALTERER_SERVICES)) {
$serviceCommandInfoAltererlist = $container->get(DrushServiceModifier::DRUSH_COMMAND_INFO_ALTERER_SERVICES);
$serviceCommandInfoAltererList = $container->get(DrushServiceModifier::DRUSH_COMMAND_INFO_ALTERER_SERVICES);
$commandFactory = Drush::commandFactory();
foreach ($serviceCommandInfoAltererlist->getCommandList() as $altererHandler) {
foreach ($serviceCommandInfoAltererList->getCommandList() as $altererHandler) {
$commandFactory->addCommandInfoAlterer($altererHandler);
$this->logger->debug(dt('Commands are potentially altered in !class.', ['!class' => get_class($altererHandler)]));
}
}

$serviceCommandlist = $container->get(DrushServiceModifier::DRUSH_CONSOLE_SERVICES);
if ($container->has(DrushServiceModifier::DRUSH_CONSOLE_SERVICES)) {
foreach ($serviceCommandlist->getCommandList() as $command) {
$serviceCommandList = $container->get(DrushServiceModifier::DRUSH_CONSOLE_SERVICES);
foreach ($serviceCommandList->getCommandList() as $command) {
$manager->inflect($command);
$this->logger->debug(dt('Add a command: !name', ['!name' => $command->getName()]));
$application->add($command);
}
}
// Do the same thing with the annotation commands.
if ($container->has(DrushServiceModifier::DRUSH_COMMAND_SERVICES)) {
$serviceCommandlist = $container->get(DrushServiceModifier::DRUSH_COMMAND_SERVICES);
foreach ($serviceCommandlist->getCommandList() as $commandHandler) {
$serviceCommandList = $container->get(DrushServiceModifier::DRUSH_COMMAND_SERVICES);
foreach ($serviceCommandList->getCommandList() as $commandHandler) {
$manager->inflect($commandHandler);
$this->logger->debug(dt('Add a commandfile class: !name', ['!name' => get_class($commandHandler)]));
$runner->registerCommandClass($application, $commandHandler);
Expand Down
2 changes: 1 addition & 1 deletion src/Command/ServiceCommandlist.php
Expand Up @@ -17,6 +17,6 @@ public function addCommandReference($command): void

public function getCommandList(): array
{
return $this->commandList;
return array_filter($this->commandList);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you know why it became necessary to add this? Would it be possible to prevent empty items from being added to the list, rather than remove them every time the accessor is called?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@greg-1-anderson Spent some time investigating it and did not found a real cause of this. What I found is that the dependency references for config.export.commands and config.export.commands turining into NULL once added which causes fatal error on Drupal10 CI pipeline but not on Drupal9-related ones.

}
}