Provide an ability to depend on a local project module from a non-project module

[netvl]

One of the best features of Pkl is that Pkl modules can be completely standalone, i.e. when you have a some_module.pkl file, in many cases you can just run pkl eval some_module.pkl, and get the evaluation results right away — and the CLI will take care for resolving any external dependencies, including package:// modules and their transitives, caching them and preparing them for evaluation. Or, alternatively, you can pass this Pkl module directly to an evaluator within your program, and it will be processed without any extra setup.

Unfortunately, this model breaks down when you need to depend on a local, rather than remote, project. Specifically, the package:// URLs in import, extend and amend statements must always point to some remote repository. This is usually fine — for example, when you want to just use some existing published template, it works perfectly well.

In some cases, however, there is a need to depend on a local project as if it was already packaged and deployed to a repository. One of such cases is when you develop a library (e.g. a Java library) and a Pkl package which are supposed to be used together, for example, when a library expects a Pkl module as its configuration input, and this Pkl module must use the template provided by the Pkl package.

Then, if you want to actually test how your library can load a Pkl module, and you want this Pkl module to use your in-project Pkl package, then you are actually out of luck: it is simply not possible to specify such a dependency, because package:// URLs do not work for local paths. At the same time, you cannot use relative file paths to package modules either, because the package may use external dependencies, which won't be resolved in such case.

There are several workarounds with various degrees of convenience:

1. You can require that your library/application/etc always consumes a Pkl project, not a single module.

   Unfortunately, in many cases this is not viable, because it increases complexity for the user. The fact that non-project modules are self-contained can be very valuable for certain uses.

2. You can publish your package to a repository and make your tests depend on this package.

   This, unfortunately, makes tests dependent on external resources, which may not be desirable or even possible. It also decouples tests from the source code, in a way, because whatever these tests are evaluating is not guaranteed to match whatever is currently present in the repository.

3. Build your Pkl project locally and restructure the built files into the cache structure in a temporary directory, and pass this directory as the module cache directory to the evaluator.

   This will result in Pkl evaluator to always use local files instead of fetching them from the remote repository, where they might not even exist. This does work, but it requires extra logic within your build system (e.g. some Gradle tasks), and extra degree of integration between the build logic and test logic because you'd need to pass the cache directory into your tests somehow. Another issue with this model is that tooling like IDEA plugin will not recognize modules depending on such locally installed packages, resulting in completely red Pkl files.

   Additionally, this makes your build logic depend on the Pkl implementation detail, i.e. the structure of the cache directory. This is less than ideal, because these details can change; see e.g. SPICE-0003.

4. Put your testing Pkl modules into the same Pkl project — or a different, but also local, project — and evaluate them in the tests as a part of their Pkl project.

   This requires adding support for evaluating Pkl projects to your code, in addition to the "regular" path of consuming a single module. Unfortunately, this means that you won't be able to validate your single-module loading code path, and you will have to maintain project-based config loading logic as well, even if it is not used in your real world scenarios, or does not even make sense for them.

Another reason to add such an ability is to increase consistency. In non-project modules, it is currently possible to depend on single modules or packages defined remotely, and it will all work transparently. It is also possible to depend on remote packages, but it is not possible to depend on local packages, which breaks the symmetry.

	...remote single module	...remote package	...local single module	...local package
Can local single module depend on...	Yes	Yes	Yes	No

Given that local projects can depend on local projects, it does not feel like this limitation is something fundamental.

Therefore, it would be great if Pkl provided an ability to depend on a local package from a local standalone module, for example, by using a different URI schema:

amends "package-local://path/to/directory/with/pklproject#/ModuleWithinPackage.pkl"

This would evaluate the referenced project, resolve its dependencies, etc in a way similar to both how depending on a remote package would work:

amends "package://repo.com/some_package#/ModuleWithinPackage.pkl"

and how depending on a local package from another local package would work.