Generated Java/C# code is missing docstrings of verbose enum entries

[zhanghaocars]

Are Verbose enums actually working?
I have used Kaitai various versions(0.8 0.9 0.10), different language output(java, C#), different ksy file from official repo, and different platform(windows, linux), none of them have generate the enum comment string.

[GreyCat]

I can say for sure that docstring generation was never implemented for Java or C#. I'm not sure if we even have standards to specify these so it will be picked up properly by IDEs and javadoc / xmldoc.

For the record, it was never advertised as "it will generate docstrings". KS docs merely says:

allows other regular keys (like doc and doc-ref) to specify documentation.

[generalmimon]

@GreyCat:

For the record, it was never advertised as "it will generate docstrings".

Well, the user guide says that docstrings added in doc keys to attributes will exported into the target language - see https://doc.kaitai.io/user_guide.html#docstrings:

A very simple example is that we can add docstrings to every attribute, using syntax like that:

  - id: rating
    type: s4
    doc: Rating, can be negative

These docstrings are not just the comments in the .ksy file, they'll actually get exported into the target language as well (for example, in Java they’ll become JavaDoc, in Ruby they’ll become RDoc/YARD, etc).

Furthermore, the https://doc.kaitai.io/user_guide.html#ksy-documentation section clarifies that not only sequence attributes but also other symbols can be documented. Verbose enums are missing there, but I don't think it's intentional.

@GreyCat Anyway, can we agree that it would be good to propagate enum entries' docstrings to the generated code whenever possible so that they can be picked up by IDEs?

---

I'm not sure if we even have standards to specify these so it will be picked up properly by IDEs and javadoc / xmldoc.

I've been thinking for quite some time that we need special tests in KST repo for the presence and readability of generated docstrings. I know that these tests might not be that easy to make in all languages, because I guess in many languages, comments are not accessible at runtime (Python, where you can access the docstring via the __doc__ attribute, is an exception). But if IDEs can read the docstrings, so can we. We may need additional libraries in each language to be able to extract the docstrings properly, but I think it shouldn't be that complicated with the right tool, and incorporating this into our automated test suite is the best way to ensure that the docstring generation works, so to me it's worth it (even if it's clearly not a top priority).

[Mingun]

Another way to check it is just generate unique docstrings and check presence of them in the generated files. Of course, this will not protect against wrong placed comments, but at least it will ensure that documentation was generated.

[GreyCat]

@generalmimon

Anyway, can we agree that it would be good to propagate enum entries' docstrings to the generated code whenever possible so that they can be picked up by IDEs?

Yeah, absolutely no concern about the intent — I'm a bit careful as I'm not sure if this is possible to get these corner cases like verbose enums for all target languages to export docstrings.

@Mingun Like your idea — it's really simple and easy to implement. Longer term, we can actually extrapolate to what @generalmimon suggests, that is not just grepping through generated sources, but really trying to find these generated strings in generated documentation or some kind of docstrings database that IDE might use.

[GreyCat]

KST-wise, what do you folks think of something like that?

id: foo_bar
data: expr_array.bin
asserts:
  - actual: aint_size
    expected: 4
docstrings:
  - contains: Sample documentation for foo
  - contains: Some other random stuff we put in

For now, it will generate a separate test that will check existence of these substrings in generated sources. Later on, based on language, we can run actual documentation generator (javadoc, yard, Sphinx, etc), and either grep it blindly there, or even be more specific on where this should be found.