first head_comment is removed from output

[endersonmaia]

Describe the bug

When converting from TOML to YAML, and using an element value as the content for a head_comment, the output YAML has the first comment removed.

Version of yq: v4.35.2
Operating system: Ubuntu 23.10
Installed via: binary release

Input TOML

sample.toml:

[logging.LOG_LEVEL]
default = "info"
go-type = "LogLevel"
description = """
One of "debug", "info", "warning", "error"."""

[features.FEATURE_FLAG]
default = "false"
go-type = "bool"
description = """
When enabled this will active the feauture

You should take care of the dependent services"""

Command
The command you ran:

yq -ptoml -oy '
  .[]
  | to_entries
  | map
    (
      {
        .key: .value.default // "",
        "description": .value.description
      }
    )
  | .[] headComment |= "-- " + .description
  | del(.[].description)
  | .[]
' sample.toml \
| yq '
  {
    "software":
      {
        "config": .
      }
  }
  '

Actual behavior

software:
  config:
    LOG_LEVEL: info
    # -- When enabled this will active the feature
    #
    # You should take care of the dependent services
    FEATURE_FLAG: "false"

Expected behavior

software:
  config:
    # -- One of "debug", "info", "warning", "error".
    LOG_LEVEL: info
    # -- When enabled the feature will be activated
    # You should take care of configure the dependent services
    FEATURE_FLAG: "false"

Additional context

The LOG_LEVEL head comment was lost. This is the main issue.

I can live with the empty line at the comment, but if you have any tips on how to add an extra #

Maybe out of scope

Another weird thing happened when I removed the last | .[], the comment was split and put into different places.

software:
  config:
    - LOG_LEVEL: info
    # You should take care of the dependent services
    - FEATURE_FLAG: "false"
      # -- When enabled this will active the feauture

But that's maybe out os scope for the main issue. I could solve this by removing the empty line from the original TOML, which is not always under my control. So, tell me if I need to open another issue for that.

[mikefarah]

The reason the head_comment is removed is because yq has some workarounds on pre-processing headers, you can turn this off by using the "--header-preprocess". Alternatively, you can do this in a single expression like so:

yq '
{"software": { "config":  (
.[] as $entry ireduce   ({}; 
    . * $entry
)
| to_entries 
| with(.[]; 
    .key headComment = "-- " + .value.description
    | .value = .value.default // ""
)
| from_entries 
) }}'

Explanation:

• Use reduce to create a new map, based on the children of top level LOG_LEVEL/FEATURE_FLAG map
• Go through each of the entries, add the headComment to the key and set the value to be the default

This has actually revealed a bug in the with_entries operator, so I've manually done to_entries / with / from_entries to work around it. I'll fix the with_entries bug for the next release.

[mikefarah]

In 4.41.1 you can now do:

yq '
{"software": { "config":  (
    # merge top level object values into a new map
    .[] as $entry ireduce   ({}; 
        . * $entry
    )
    # process comments, set value to default
    | with_entries(
        .key headComment = "-- " + .value.description
        | .value = .value.default // ""
    )
) }}'