Import remote tasks with Composer

CHANGELOG.md

@@ -2,6 +2,7 @@
 
 ## Not released yet
 
+* Add support for importing remote functions and tasks
 * Add a option `ignoreValidationErrors` on `AsTask` attribute to ignore
   parameters & options validation errors
 * Add a way to merge an application `box.json` config file used by `castor:repack`command

bin/generate-tests.php

@@ -28,7 +28,13 @@
 $application
     ->run(new ArrayInput(['command' => 'list', '--format' => 'json']), $o = new BufferedOutput())
 ;
-$applicationDescription = json_decode($o->fetch(), true);
+$json = $o->fetch();
+
+try {
+    $applicationDescription = json_decode($json, true, flags: \JSON_THROW_ON_ERROR);
+} catch (JsonException $e) {
+    throw new RuntimeException('Could not get the list of commands: ' . $json, previous: $e);
+}
 
 $taskFilterList = [
     '_complete',
@@ -69,10 +75,14 @@
     'log:info',
     'log:with-context',
     'parallel:sleep',
+    'remote-import:remote-tasks',
     'run:ls',
     'run:run-parallel',
+    // Imported tasks
+    'pyrech:hello-example',
+    'pyrech:foobar',
 ];
-$optionFilterList = array_flip(['help', 'quiet', 'verbose', 'version', 'ansi', 'no-ansi', 'no-interaction', 'context']);
+$optionFilterList = array_flip(['help', 'quiet', 'verbose', 'version', 'ansi', 'no-ansi', 'no-interaction', 'context', 'no-remote', 'update-remotes']);
 foreach ($applicationDescription['commands'] as $task) {
     if (in_array($task['name'], $taskFilterList, true)) {
         continue;
@@ -144,6 +154,7 @@ function add_test(array $args, string $class, ?string $cwd = null)
         env: [
             'COLUMNS' => 120,
             'ENDPOINT' => $_SERVER['ENDPOINT'],
+            'CASTOR_NO_REMOTE' => 1,
         ],
         timeout: null,
     );

doc/getting-started/basic-usage.md

@@ -73,6 +73,10 @@ import(__DIR__ . '/my-app/castor');
 > You cannot dynamically import tasks. The `import()` function must be called
 > at the top level of the file.
 
+> [!NOTE]
+> You can also import functions from a remote resource. See the
+> [related documentation](../going-further/extending-castor/remote-imports.md).
+
 ## Overriding task name, namespace or description
 
 The `Castor\Attribute\AsTask` attribute takes three optional

doc/going-further/extending-castor/events.md

@@ -30,6 +30,10 @@ function my_event_listener(AfterApplicationInitializationEvent|AfterExecuteTaskE
 
 Here is the built-in events triggered by Castor:
 
+* `Castor\Event\BeforeApplicationInitializationEvent`: This event is triggered
+  before the application has been initialized and before the task and context
+  has been looked at. It provides access to the `Application` instance;
+
 * `Castor\Event\AfterApplicationInitializationEvent`: This event is triggered
   after the application has been initialized. It provides access to the
   `Application` instance and an array of `TaskDescriptor` objects;

doc/going-further/extending-castor/remote-imports.md

@@ -0,0 +1,142 @@
+# Import remote functions
+
+Castor can import functions from your filesystem but also from a remote resource.
+
+## Importing functions
+
+When importing functions from a remote resource, Castor will use Composer to
+download the packages and store them in `.castor/vendor/`.
+
+To import functions, you need to use the same `import()` function used to import
+your tasks, but this time with a different syntax for the `path` argument.
+
+The import syntax depends on the source of the packages.
+
+### From a Composer package (scheme `composer://`)
+
+This is the most common use case when the functions to import are defined in a
+Composer package. You can directly import them by using the package name
+prefixed with the `composer://` scheme:
+
+```php
+use function Castor\import;
+
+import('composer://vendor-name/project-name');
+```
+
+This will import all the tasks defined in the package.
+
+#### Specify the version
+
+You can define the version of the package to import by using the `version`
+argument:
+
+```php
+use function Castor\import;
+
+import('composer://vendor-name/project-name', version: '^1.0');
+```
+
+You can use any version constraint supported by Composer (like `*`, `dev-main`,
+`^1.0`, etc.). See the [Composer documentation](https://getcomposer.org/doc/articles/versions.md#writing-version-constraints)
+for more information.
+
+> [!TIP]
+> The `version` argument is optional and will default to `*`.
+
+#### Import from a package not pushed to packagist.org
+
+In some cases, you may have a Composer package that is not pushed to
+packagist.org (like a private package hosted on packagist.com or another package
+registry). In such cases, you can import it by using the `vcs` argument to
+specify the repository URL where the package is hosted:
+
+```php
+
+use function Castor\import;
+
+import('composer://vendor-name/project-name', vcs: 'https://github.com/organization/repository.git');
+```
+
+### From a repository (scheme `package://`)
+
+If the functions you want to import are not available as a Composer package, you
+can still import them by using a special configuration that Composer will
+understand. This will now use the `package://` scheme.
+
+```php
+use function Castor\import;
+
+import('composer://vendor-name/project-name', source: [
+    'url' => 'https://github.com/organization/repository.git',
+    'type' => 'git', // 'Any source type supported by Composer (git, svn, etc)'
+    'reference' => 'main', //  A commit id, a branch or a tag name
+]);
+```
+
+> [!NOTE]
+> The "vendor-name/project-name" name can be whatever you want and we only be
+> used internally by Castor and Composer to make the repository behave like a
+> normal Composer package.
+
+> [!TIP]
+> Rather than using the `package://` scheme, it may be simpler to create a
+> standard `composer.json` to your repository and import your newly created
+> package by using the `composer://` scheme and the `vcs` argument.
+
+## Import only a specific file
+
+No matter where does the package come from (Composer package, git repository,
+etc.), you can restrict the file (or directory) to be imported. This is
+configured by using the `file` argument specifying the path inside the package
+or repository.
+
+```php
+use function Castor\import;
+
+import('composer://vendor-name/project-name', file: 'castor/my-tasks.php');
+```
+
+> [!NOTE]
+> The `file` argument is optional and will empty by default, causing Castor to
+> import and parse all the PHP files in the package. While handy, it's probably
+> not what you want if your package contains PHP code that are not related to
+> Castor.
+
+## Preventing remote imports
+
+In case you have trouble with the imported functions (or if you don't trust
+them), you can prevent Castor from importing and running any of them. Add the
+`--no-remote` option when calling any Castor tasks:
+
+```bash
+$ castor --no-remote my-task
+```
+
+This will trigger a warning to remind you that the remote imports are disabled.
+Also, any task or configuration using an imported function will trigger an error
+with Castor complaining about undefined functions.
+
+If you want to disable remote imports every time, you can define the
+`CASTOR_NO_REMOTE` environment variable to 1:
+
+```bash
+$ export CASTOR_NO_REMOTE=1
+$ castor my-task # will not import any remote functions
+```
+
+## Update imported packages
+
+When you import a package in a given version, Castor will not update
+automatically update the packages once a new version of your dependency is
+available.
+
+To update your dependencies, you will either need to:
+- change the required version yourself (thus every one using your Castor project
+will profit of the update once they run your project);
+- force the update on your side only by either using the `--update-remotes`
+option or by removing the `.castor/vendor/` folder.
+
+```bash
+$ castor --update-remotes
+```

doc/going-further/index.md

@@ -7,5 +7,5 @@ help you write your tasks.
 * [Interacting with Castor](interacting-with-castor/advanced-context.md): how to
 interact with Castor's output, context, logs, etc.
 * [Extending Castor](extending-castor/events.md): how to wire some logic
-inside Castor, how to redistribute your project as your own phar or static
-binary.
+inside Castor, import remote tasks, redistribute your project as your own phar
+or static binary, etc.

examples/remote-import.php

@@ -0,0 +1,25 @@
+<?php
+
+namespace remote_import;
+
+use Castor\Attribute\AsTask;
+
+use function Castor\import;
+
+// Importing tasks from a Composer package
+import('composer://pyrech/castor-example', version: '^1.0');
+// Importing tasks from a Composer package not published on packagist (but still having a composer.json)
+import('composer://pyrech/castor-example-package-not-published', version: '*', vcs: 'https://github.com/pyrech/castor-example-package-not-published.git');
+// Importing tasks from a repository not using Composer
+import('package://pyrech/foobar', source: [
+    'url' => 'https://github.com/pyrech/castor-example-misc.git',
+    'type' => 'git',
+    'reference' => 'main', //  commit id, branch or tag name
+]);
+
+#[AsTask(description: 'Use functions imported from remote packages')]
+function remote_tasks(): void
+{
+    \pyrech\helloExample(); // from composer://pyrech/castor-example
+    \pyrech\foobar(); // from package://pyrech/foobar
+}

phpstan-baseline.neon

@@ -5,6 +5,11 @@ parameters:
 			count: 1
 			path: examples/args.php
 
+		-
+			message: "#^Function pyrech\\\\.* not found\\.$#"
+			count: 2
+			path: examples/
+
 		-
 			message: "#^Default value of the parameter \\#1 \\$data \\(array\\{\\}\\) of method Castor\\\\Context\\:\\:__construct\\(\\) is incompatible with type array\\{name\\: string, production\\: bool, foo\\?\\: string\\}\\.$#"
 			count: 1

src/Console/Application.php

@@ -15,12 +15,14 @@
 use Castor\Descriptor\TaskDescriptor;
 use Castor\Descriptor\TaskDescriptorCollection;
 use Castor\Event\AfterApplicationInitializationEvent;
+use Castor\Event\BeforeApplicationInitializationEvent;
 use Castor\EventDispatcher;
 use Castor\ExpressionLanguage;
 use Castor\Fingerprint\FingerprintHelper;
 use Castor\FunctionFinder;
 use Castor\GlobalHelper;
 use Castor\PlatformHelper;
+use Castor\Remote\Importer;
 use Castor\WaitForHelper;
 use Monolog\Logger;
 use Psr\Cache\CacheItemPoolInterface;
@@ -59,10 +61,11 @@ public function __construct(
         public readonly ExpressionLanguage $expressionLanguage,
         public readonly Logger $logger,
         public readonly Filesystem $fs,
+        public readonly WaitForHelper $waitForHelper,
+        public readonly FingerprintHelper $fingerprintHelper,
+        public readonly Importer $importer,
         public HttpClientInterface $httpClient,
         public CacheItemPoolInterface&CacheInterface $cache,
-        public WaitForHelper $waitForHelper,
-        public FingerprintHelper $fingerprintHelper,
     ) {
         $handler = ErrorHandler::register();
         $handler->setDefaultLogger($logger, [
@@ -131,6 +134,9 @@ public function doRun(InputInterface $input, OutputInterface $output): int
         $this->symfonyStyle = new SymfonyStyle($input, $output);
         $this->logger->pushHandler(new ConsoleHandler($output));
 
+        $event = new BeforeApplicationInitializationEvent($this);
+        $this->eventDispatcher->dispatch($event);
+
         $descriptors = $this->initializeApplication($input);
 
         // Must be done after the initializeApplication() call, to ensure all
@@ -215,6 +221,24 @@ private function initializeApplication(InputInterface $input): TaskDescriptorCol
             ));
         }
 
+        $this->getDefinition()->addOption(
+            new InputOption(
+                'no-remote',
+                null,
+                InputOption::VALUE_NONE,
+                'Skip the import of all remote remote packages',
+            )
+        );
+
+        $this->getDefinition()->addOption(
+            new InputOption(
+                'update-remotes',
+                null,
+                InputOption::VALUE_NONE,
+                'Force the update of remote packages',
+            )
+        );
+
         return new TaskDescriptorCollection($tasks, $symfonyTasks);
     }
 

src/Console/ApplicationFactory.php

@@ -10,11 +10,15 @@
 use Castor\ExpressionLanguage;
 use Castor\Fingerprint\FingerprintHelper;
 use Castor\FunctionFinder;
+use Castor\HasherHelper;
 use Castor\Listener\GenerateStubsListener;
 use Castor\Listener\UpdateCastorListener;
 use Castor\Monolog\Processor\ProcessProcessor;
 use Castor\PathHelper;
 use Castor\PlatformHelper;
+use Castor\Remote\Composer;
+use Castor\Remote\Importer;
+use Castor\Remote\Listener\RemoteImportListener;
 use Castor\Stub\StubsGenerator;
 use Castor\WaitForHelper;
 use Monolog\Logger;
@@ -49,6 +53,8 @@ public static function create(): SymfonyApplication
         $cache = new FilesystemAdapter(directory: $cacheDir);
         $logger = new Logger('castor', [], [new ProcessProcessor()]);
         $fs = new Filesystem();
+        $fingerprintHelper = new FingerprintHelper($cache);
+        $importer = new Importer($logger, new Composer($fs, $logger, $fingerprintHelper));
         $eventDispatcher = new EventDispatcher(logger: $logger);
         $eventDispatcher->addSubscriber(new UpdateCastorListener(
             $cache,
@@ -59,6 +65,7 @@ public static function create(): SymfonyApplication
             new StubsGenerator($logger),
             $rootDir,
         ));
+        $eventDispatcher->addSubscriber(new RemoteImportListener($importer));
 
         /** @var SymfonyApplication */
         // @phpstan-ignore-next-line
@@ -70,12 +77,16 @@ public static function create(): SymfonyApplication
             new ExpressionLanguage($contextRegistry),
             $logger,
             $fs,
+            new WaitForHelper($httpClient, $logger),
+            $fingerprintHelper,
+            $importer,
             $httpClient,
             $cache,
-            new WaitForHelper($httpClient, $logger),
-            new FingerprintHelper($cache),
         );
 
+        // Avoid dependency cycle
+        $importer->setApplication($application);
+
         $application->setDispatcher($eventDispatcher);
         $application->add(new DebugCommand($rootDir, $cacheDir, $contextRegistry));
 

src/Event/BeforeApplicationInitializationEvent.php

@@ -0,0 +1,13 @@
+<?php
+
+namespace Castor\Event;
+
+use Castor\Console\Application;
+
+class BeforeApplicationInitializationEvent
+{
+    public function __construct(
+        public readonly Application $application,
+    ) {
+    }
+}