osgi and aspectj annotations(?) are not recognized by ijar and logs are flooded with the noise

[dkashyn-sfdc]

Description of the bug:

ijar produces extra logging that just floods the output without any real value.

That message is originating from

bazel/third_party/ijar/classfile.cc

Line 1595 in 472b96d

fprintf(stderr, "ijar: skipping unknown attribute: \"%s\".\n",

and it seems like different treatment required for different attributes.

aspectj output alone is huge

Which category does this issue belong to?

Java Rules

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

No response

Which operating system are you running Bazel on?

OSX, RHEL

What is the output of bazel info release?

7.1

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse HEAD ?

No response

Is this a regression? If yes, please try to identify the Bazel commit where the bug was introduced.

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

INFO: From Extracting interface for jar external/org_osgi_org_osgi_core/org.osgi.core-4.3.0.jar [for tool]:
ijar: skipping unknown attribute: "Bridge".
ijar: skipping unknown attribute: "Bridge".
ijar: skipping unknown attribute: "Varargs".
ijar: skipping unknown attribute: "Varargs".
ijar: skipping unknown attribute: "Bridge".

aspectj is much more noisy, thousands of lines like (other annotations from org.aspectj.weaver are there as well see https://github.com/codehaus/backport175/blob/master/org.aspectj/modules/weaver/src/org/aspectj/weaver/AjAttribute.java for more of those)

ijar: skipping unknown attribute: "org.aspectj.weaver.EffectiveSignature".
ijar: skipping unknown attribute: "org.aspectj.weaver.AjSynthetic".
ijar: skipping unknown attribute: "org.aspectj.weaver.AjSynthetic".
ijar: skipping unknown attribute: "org.aspectj.weaver.AjSynthetic".
ijar: skipping unknown attribute: "org.aspectj.weaver.TypeMunger".
ijar: skipping unknown attribute: "org.aspectj.weaver.TypeMunger".
ijar: skipping unknown attribute: "org.aspectj.weaver.TypeMunger".
ijar: skipping unknown attribute: "org.aspectj.weaver.WeaverVersion".
ijar: skipping unknown attribute: "org.aspectj.weaver.Aspect".
ijar: skipping unknown attribute: "org.aspectj.weaver.SourceContext".
ijar: skipping unknown attribute: "org.aspectj.weaver.WeaverState".
ijar: skipping unknown attribute: "org.aspectj.weaver.MethodDeclarationLine

[hvadehra]

@dkashyn-sfdc do you have a minimal repro?

[dkashyn-sfdc]

Not yet, but if you add org.osgi.core-4.3.0.jar as a dep to ANY project you'll see at least 4 of the warnings related to OSGI bit. I can try to locate exact aspectj dep to do the same if it helps.

[dkashyn-sfdc]

Another one is INFO: From Extracting interface for jar external/org_apache_ws_commons_axiom_axiom_dom/axiom-dom-1.2.20.jar:

With endless output like

INFO: From Extracting interface for jar external/org_apache_ws_commons_axiom_axiom_dom/axiom-dom-1.2.20.jar:
ijar: skipping unknown attribute: "org.aspectj.weaver.MethodDeclarationLineNumber".
ijar: skipping unknown attribute: "org.aspectj.weaver.MethodDeclarationLineNumber".
ijar: skipping unknown attribute: "org.aspectj.weaver.MethodDeclarationLineNumber".
ijar: skipping unknown attribute: "org.aspectj.weaver.MethodDeclarationLineNumber".
ijar: skipping unknown attribute: "org.aspectj.weaver.MethodDeclarationLineNumber".
ijar: skipping unknown attribute: "org.aspectj.weaver.WeaverVersion".
ijar: skipping unknown attribute: "org.aspectj.weaver.WeaverState".
ijar: skipping unknown attribute: "org.aspectj.weaver.WeaverVersion".
ijar: skipping unknown attribute: "org.aspectj.weaver.WeaverState".
ijar: skipping unknown attribute: "org.aspectj.weaver.WeaverVersion".
ijar: skipping unknown attribute: "org.aspectj.weaver.WeaverState".
ijar: skipping unknown attribute: "org.aspectj.weaver.WeaverVersion".
ijar: skipping unknown attribute: "org.aspectj.weaver.WeaverState".

and

❯ cat run_baseline.log | grep MethodDeclarationLineNumber |wc -l
3400

❯ cat run_baseline.log | grep '"org.aspectj.weaver.' |wc -l
8735

[dkashyn-sfdc]

./ijar axiom-dom-1.2.20.jar axiom-dom-1.2.20-ijar.jar --target_label @@org_apache_ws_commons_axiom_axiom_dom//:org_apache_ws_commons_axiom_axiom_dom
ijar: skipping unknown attribute: "org.aspectj.weaver.MethodDeclarationLineNumber".
ijar: skipping unknown attribute: "org.aspectj.weaver.MethodDeclarationLineNumber".
ijar: skipping unknown attribute: "org.aspectj.weaver.MethodDeclarationLineNumber".
ijar: skipping unknown attribute: "org.aspectj.weaver.MethodDeclarationLineNumber".

https://repo1.maven.org/maven2/org/apache/ws/commons/axiom/axiom-dom/1.2.20/

[hvadehra]

Thanks. It looks like this is WAI. There is some precedent for skipping this warning for well-known cases, but I'm not sure how well that scales for all such cases. Another option could be to cache these and only report an unknown attribute once per ijar invocation which could go a long way towards reducing spam?

That aside, I would have expected most builds today to use the header compiler (Turbine), and not ijar. what Bazel version are you using? Do you have header compilation explicitly disabled?

In the meantime, you might be able to workaround this with --output_filter and/or --ui_event_filters

[fmeum]

@hvadehra Based on the logs, I think that these warnings are emitted for java_import targets, which always use ijar.

[hvadehra]

I think that these warnings are emitted for java_import targets, which always use ijar.

Ah, right. Is it possible to maybe wrap the corresponding source jars in a java_library(..., srcs = ['org.osgi.core-4.3.0.srcjar']) instead (will likely need renaming the .jar to .srcjar)?

[dkashyn-sfdc]

In our cases this is some external dep. Is it even beneficial to use ijar in this case? While it might help with the code that we own for external libs that are changing in ways that will erase all of the benefits from ijar anyway.

Turbine will be even more overkill in this case since artifacts are pre-built...

I suspect that in this case ijar and jar will be identical with probably even more overhead for ijar creation.

[hvadehra]

Then maybe just build with --nouse_ijars? AFAICT it only applies to java_import. (well, also javac compilation iff header compilation is disabled).

[dkashyn-sfdc]

I'll try --output_filter since java header compilation is something we are only experimenting with for now.

It is not speeding things up for us unless we use RBE and slowing them down for local env :(

Back to ijar: shouldn't those messages be deduplicated at least? There is no big value of reporting EVERY annotation and get 5k+ messages for a single jar file...

[fmeum]

@dkashyn-sfdc Are you using a recent version of rules_java and Bazel, which use a Graal Native Image of Turbine? Together with --experimental_java_classpath=bazel, I've found it to be faster on incremental changes even locally (although I haven't tested it on truly large individual files, e.g. generated code).

[fmeum]

I submitted #22072 to deduplicate the warnings.

[dkashyn-sfdc]

still seeing this with 7.2.0rc1