"Unable to fetch data" error on Visualize tab, all graphs ok on Home tab

[kornix]

Hi All.

Have an standalone (no docker) Akvorado setup on a server running Rocky 9.3 / Go 1.22.2 / Clickhouse 24.3.2.23 / Kafka 3.7.0.

After upgrading Akvorado from 1.9.3 to 1.10.0 (applying all required migrations and schemas), i can see all summary graphs on Home tab, but no detailed graph on Visualize tab - just a red error message "Unable to fetch data! Server returned an error: Error" instead (see attached screenshots).

Clickhouse shows error containing current Dimension used to build the graph like that:

Row 5:
──────
last_error_time:    2024-04-09 12:35:30
last_error_message: Not found column __table5.ExporterAddress
                                              ^^^^^^^^^^^^^^^^^

All flows tables seems to be created after orchestrator restart, but the error looks like they are not:

┌─name──────────────────────────────────────────┐

1. │ asns │
2. │ exporters │
3. │ exporters_consumer │
4. │ flows │
5. │ flows_1h0m0s │
6. │ flows_1h0m0s_consumer │
7. │ flows_1m0s │
8. │ flows_1m0s_consumer │
9. │ flows_5m0s │
10. │ flows_5m0s_consumer │
11. │ flows_LAABIGYMRYZPTGOYIIFZNYDEQM_raw │
12. │ flows_LAABIGYMRYZPTGOYIIFZNYDEQM_raw_consumer │
13. │ flows_raw_errors │
14. │ flows_raw_errors_consumer │
15. │ icmp │
16. │ networks │
17. │ protocols │
    └───────────────────────────────────────────────┘

Any ideas where i should take a look?

Thank you in advance.

[kornix]

A little update: Sankey graph type works ok with the same (problematic) dimensions (e. g. when taking SrcAS vs. DstAS etc.), other graph types throws error.

[kornix]

No. Akvorado 1.9.3 was used with Clickhouse 22.6. I've updated environment required by Akvorado (golang, clickhouse, kafka and zookeeper) to the last stable/recommended versions before updating Akvorado from 1.9.3 to 1.10.0. All system updates are applied on a regular basis by dnf-automatic, so the system runs up-to-date software since it's installation.

[vincentbernat]

I get the same problem with ClickHouse 24.3. Keep 23.8 for now.

[kornix]

Downgraded Clickhouse to 23.12. This made the trick.

Very appreciate for your help.

[vincentbernat]

See ClickHouse/ClickHouse#62464.

[FestplattenSchnitzel]

Works for me after upgrading Clickhouse to 24.5.1.1763, which seems to contain the linked fix.

[kornix]

Problem with Akvorado v1.10.0 Visualize tab graphs was solved when downgraded Clickhouse to 23.12.