add coursier 2.0.16

[bollwyvl]

Checklist

• Title of this PR is meaningful: e.g. "Adding my_nifty_package", not "updated meta.yaml"
• License file is packaged (see here for an example)
• Source is from official source
• Package does not vendor other packages. (If a package uses the source of another package, they should be separate packages or the licenses of all packages need to be packaged)
  • this will take some investigating...
• If static libraries are linked in, the license of the static library is packaged.
• Build number is 0
• A tarball (url) rather than a repo (e.g. git_url) is used in your recipe (see here for more details)
  • building from source seems... interesting
• GitHub users listed in the maintainer section have posted a comment confirming they are willing to be listed there
• When in trouble, please check our knowledge base documentation before pinging a team.

On the general, @conda-forge/help-java:

• is it required to do a conda_build_config.yaml for each JDK major version? If so, i probably need to add in a few places
• should we point the COURSIER_CACHE somewhere in the installed env?

related:

• https://github.com/krassowski/jupyterlab-lsp/pull/415

[conda-forge-linter]

Hi! This is the friendly automated conda-forge-linting service.

I wanted to let you know that I linted all conda-recipes in your PR (recipes/coursier) and found some lint.

Here's what I've got...

For recipes/coursier:

• Selectors are suggested to take a <two spaces>#<one space>[<expression>] form. See lines [12]

[conda-forge-linter]

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipes/coursier) and found it was in an excellent condition.

[bollwyvl]

Welp: I still don't know what to do about the licenses... looking around for something, but doesn't appear there's anything built into coursier that does, as yarn might...

[bollwyvl]

Hm, maven appears to be able to, with an extension... http://www.mojohaus.org/license-maven-plugin/add-third-party-mojo.html

[bollwyvl]

tried the JSON report of fetch: https://get-coursier.io/docs/cli-fetch

Does not include the licenses, potentially related to:
coursier/coursier#1790

[jan-janssen]

In case the license does not change, you can simply upload it directly to the recipe repository.

[bollwyvl]

But in case it does? Granted, something that is Apache-2.0 is unlikely to change, but I guess I'd rather have it fail with a predictable sums do not match than just... not know, and keep shipping the wrong one.

[dbast]

@bollwyvl Thanks for bringing coursier. Super handy tool and bridge into the java world with lots of artefacts on maven central (I use it to e.g. install the almond scala kernel for Jupyter, get+run the the bfg repo cleaner jar or plantuml.jar).

Why do you want to pin it for Java 8-11? The used command coursier bootstrap ... --standalone is afaik not compiling anything, it gets the required dependencies and wraps them as binary blob into a shell script then called coursier. The resulting file contains code build by upstream of coursier and runs with all the supported java versions.

There are anyway two options to package coursier here:

• (as currently done) The result of coursier bootstrap ... --standalone -> Package size is about 35 MB.
• The result of coursier bootstrap ... (no standalone) that is the same as cp ./coursier $PREFIX/bin/coursier -> Package size is about 32 kb and coursier gets what it needs during runtime, which is maybe not bad as it anyway is used to get artifacts from maven central etc.

The second option is maybe with less license issues. (Similar tools like sbt or gradle are anyway also packaged as binary blob from upstream without rebuild because of chicken and egg issues otherwise).

I would do the following changes:

• Remove conda_build_config.yaml from PR (we are building nothing here)
• Remove openjdk from run requirements, as many systems have java on system level. The user can still decide to install openjdk via conda if there is no system java (Instead add openjdk to tests as requirement).
• (Maybe change the build script to just only cp ./coursier "$PREFIX/bin/coursier" (similar on windows))

[bollwyvl]

As usual with stuff outside of my python wheelhouse (ha!), I just _need_ this and don't actually _want_ it, much less want to _maintain_ it, but will if it lets me do jupyter demos that will work tomorrow. I'm definitely open to the above suggestions, and will peruse them when I get a chance. I find the lack of license tooling disturbing, and would _definitely_ appreciate ways to automate that. The rust stuff I've had to build is feasible... it can say least when I'm out of compliance. Alternately, and frankly my preference, open another, better PR that does it right!

[dbast]

I would be fine with co-maintaining the recipe and also do the suggested changes, if you are ok with it.

[bollwyvl]

install the almond scala kernel for Jupyter

You've nailed it here for my present use case. The specific thing I have in mind is getting to where I can package almond + metals for a demo on jupyterlab-lsp, with this just being a stepping stone. The most terse instructions we've gotten to (starting with curlbashing... some version of coursier) still has a raft of fiddlying things.

If we can get this working properly where installing a conda.lock, solved offline, Just Works On CI On Windows, and nothing gets magically added at runtime, then we can actually test against and support it instead of hand-waving docs, and Oh Yeah, Have Java.

Why do you want to pin it for Java 8-11?

Other feedstocks that do Binary Java Things seem to, and it's what coursier tests against. I need to look at some tarballs with diffoscope and see if i can see anything illuminating, as of course, it would be preferable to have i builds instead of i * j builds.

coursier gets what it needs during runtime

Anywhere I can, I try to not to package this kind of behavior. Having a part of a toolchain that has to hot bootstrap itself before getting to it's actually doing seems... tenuous, no matter how magical the underlying npm/rust/prolog/go/jvm/elixir is.

Further, if we end up having to scrape poms in the cache to get licenses (which is kinda how it's looking), all the more reason to have each build not be polluted with the coursier dependencies.

Remove openjdk from run requirements

I find supporting "system" JDK almost as frustrating as system x11. If would not wish another "getting started with pyspark" talk, with the first 30 minutes being getting x environment variables Just Right for wherever the jdk happens to be, on anyone.

To suit both, we could have a multi-output recipe with a coursier-no-jdk, that didn't carry the dependency, but i think if I conda install -c conda-forge coursier, and it can work out of the box, it should.

[dbast]

Well, other feedstocks build = compile against openjdk, which this recipe does not. coursier boostrap ... gets the pre-built coursier binaries + dependencies which were built by upstream and tested against various different java versions. Thus pinning is not required and afaik even does nothing except of multiplying the number of builds to be done, which should be all the same.

coursier is a special thing, as it is an artifact fetcher which has a tiny version (used in that PR) that can download the full standalone version of itself. As coursier anyway is used to fetch other artifacts, why not let it fetch itself during usage? Less MB to package and less license trouble (32k vs 35MB).

The pyspark and almond case you refer to (and also in case of r-sparklyr) are very interesting regarding some detail: Lots of work with Spark is often done on Hadoop systems, where the system java + installed dependencies (patched to e.g. read faster from AWS S3) should be used. If now by installing coursier (to boostrap almond) another java is installed via conda, this is then counter-productive/intuitive. And as coursier is not compiled via this PR it is also in no way linked against the conda java and works very well with the system java on such systems. The pyspark and r-sparklyr recipes also for that reason specifically don't list list openjdk as run requirement as this is expected from the underlying cluster system.

This means, I would still vote for the same changes as written before.

[dbast]

@bollwyvl How to unblock this PR? What is your idea? Keep the PR as it is and solve the license situation and get it merged?

Should I try my ideas via separate PR?

[conda-forge-linter]

Hi! This is the friendly automated conda-forge-linting service.

I wanted to let you know that I linted all conda-recipes in your PR (recipes/coursier) and found some lint.

Here's what I've got...

For recipes/coursier:

• The top level meta keys are in an unexpected order. Expecting ['package', 'source', 'build', 'requirements', 'test', 'outputs', 'about', 'extra'].

[bollwyvl]

Sorry, had been meaning to get around to this. As of cff75a0 ( a linting push to come), it's building:

• coursier-base which actually contains the (still) heavyweight cached coursier
• coursier which depends on coursier-base and openjdk

So for most purposes wanting to use coursier during a build, a downstream can specify

requirements: 
  build: 
  - coursier

and not worry about the jdk version (unless they actually care).

For your enterprise setting, conda install -c conda-forge coursier-base would not grab openjdk.

I'm still not sold on shipping the un-bootstrapped thing with the sole purpose of immediately downloading something else as soon as its used, which means I still need to try to find the licenses...

[conda-forge-linter]

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipes/coursier) and found it was in an excellent condition.

[bollwyvl]

*@mariusvniekerk thanks for the review... What do you think about the license situation?*

…

On Thu, Dec 24, 2020, 17:27 Marius van Niekerk ***@***.***> wrote: ***@***.**** approved this pull request. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#13411 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAALCRAEPKNNDP7J43ACHN3SWO55RANCNFSM4UPL46BA> .

[mariusvniekerk]

I'd recommend pulling them from a canonical place like maven central if needed

[mariusvniekerk]

Probably best to use a tool like license-gradle-plugin to automate it

[dbast]

@bollwyvl Thanks for updating the PR.

Something similar to the mentioned gradle plugin for sbt would be https://github.com/sbt/sbt-license-report

Either we run that during build or upstreams runs it and publishes the output with each tagged release.

Though the output is just a list of licenses with links which still would require collecting and aggregating them.

[stale]

Hi friend!

We really, really, really appreciate that you have taken the time to make a PR on conda-forge/staged-recipes! conda-forge only exists because people like you donate their time to build and maintain conda recipes for use by the community.

In an effort to maintain this repository and increase the signal-to-noise for open PRs, the maintainers of staged-recipes close excessively old PRs after six months. This PR will remain open for another month, and then will be closed.

If you'd like to keep it open, please comment/push and we will be happy to oblige! Note that very old PRs will likely need to be rebased on master so that they can be rebuilt with the most recent CI scripts. If you have any trouble, or we missed reviewing this PR in the first place (sorry!), feel free to ping the team using a special command in a comment on the PR to get the attention of the staged-recipes team.

Cheers and thank you for contributing to this community effort!

[stale]

Hi again! About a month ago, we commented on this PR saying it would be closed in another month if it was still inactive. It has been a month and so now it is being closed. Thank you so much for making it in the first place and contributing to the community project that is conda-forge. If you'd like to reopen this PR, please feel free to do so at any time!

Cheers and have a great day!

[bollwyvl]

once again, I don't really care about this package, and if someone else would prefer to do an alternate PR that allows building against a modern (2.x) coursier with known-good, conda-forge delivered openjdk, i'd happily delete this branch.

That being said, my "need" for this has warmed up again, and will probably bump to 2.0.16 and do some looking into the licensing situation...

[bollwyvl]

@conda-forge-admin please restart CI

[bollwyvl]

Well, the jar/wrapper approach appears to work. Maybe we should see if we can get the binary launchers to work:

https://get-coursier.io/docs/cli-installation#native-launcher

This probably has a smaller surface of "can go entirely wrong," and may be better able to extract the licenses if it's doing a build...

[stale]

Hi friend!

We really, really, really appreciate that you have taken the time to make a PR on conda-forge/staged-recipes! conda-forge only exists because people like you donate their time to build and maintain conda recipes for use by the community.

In an effort to maintain this repository and increase the signal-to-noise for open PRs, the maintainers of staged-recipes close excessively old PRs after six months. This PR will remain open for another month, and then will be closed.

If you'd like to keep it open, please comment/push and we will be happy to oblige! Note that very old PRs will likely need to be rebased on main so that they can be rebuilt with the most recent CI scripts. If you have any trouble, or we missed reviewing this PR in the first place (sorry!), feel free to ping the team using a special command in a comment on the PR to get the attention of the staged-recipes team.

Cheers and thank you for contributing to this community effort!

[stale]

Hi again! About a month ago, we commented on this PR saying it would be closed in another month if it was still inactive. It has been a month and so now it is being closed. Thank you so much for making it in the first place and contributing to the community project that is conda-forge. If you'd like to reopen this PR, please feel free to do so at any time!

Cheers and have a great day!