Compile Error: #1

cmurrayis · 2022-07-13T23:35:05Z

cmurrayis
Jul 13, 2022

Hi,

Came across your project after seeing a Twitter post. I'm trying to compile per the directions on a fresh Deb 11 install however receiving errors during the make command process:

Begin:

▶ fetching node modules…
➤ YN0050: The --frozen-lockfile option is deprecated; use --immutable and/or --immutable-cache instead
➤ YN0000: ┌ Resolution step
➤ YN0061: │ lru-cache@npm:7.8.0 is deprecated: Please update to latest patch version to fix memory leak https://github.com/isaacs/node-lru-cache/issues/227
➤ YN0032: │ fsevents@npm:2.3.2: Implicit dependencies on node-gyp are discouraged
➤ YN0002: │ @vueuse/router@npm:8.6.0 [d54e3] doesn't provide vue (p9ce65), requested by vue-demi
➤ YN0000: │ Some peer dependencies are incorrectly met; run yarn explain peer-requirements <hash> for details, where <hash> is the six-letter p-prefixed code
➤ YN0000: └ Completed in 4s 144ms

End:

 YN0028: │ The lockfile would have been modified by this install, which is explicitly forbidden.
➤ YN0000: └ Completed in 12s 266ms
➤ YN0000: Failed with errors in 16s 375ms
make: *** [Makefile:85: console/frontend/node_modules] Error 1

I've followed through and confirmed that Go and Yarn are installed and reporting their versions correctly:

root@flow:/akvorado# yarn -v
3.2.1

root@flow:/akvorado# go version
go version go1.18.4 linux/amd64

I am sure something simple has been missed?

vincentbernat · 2022-07-14T05:52:12Z

vincentbernat
Jul 14, 2022
Maintainer

As a workaround, modify the Makefile to remove the --frozen-lockfile argument. I'll investigate a bit on how to make this work with both Yarn and Yarn 2.

0 replies

vincentbernat · 2022-07-14T20:27:44Z

vincentbernat
Jul 14, 2022
Maintainer

This should be fixed now (see #4).

0 replies

cmurrayis · 2022-07-14T21:33:36Z

cmurrayis
Jul 14, 2022
Author

This appeared to clear this error however the follow is now present:

root@flow:/docker/akvorado# make
▶ formatting code…
▶ running lint…
▶ compiling protocol buffers definition…
make: protoc: No such file or directory
make: *** [Makefile:73: inlet/flow/decoder/flow-1.pb.go] Error 127
root@flow:/docker/akvorado# make clean all
▶ cleaning…
▶ building golang.org/x/tools/cmd/goimports@latest…
▶ fetching node modules…

> vue-demi@0.13.4 postinstall /docker/akvorado/console/frontend/node_modules/vue-echarts/node_modules/vue-demi
> node ./scripts/postinstall.js


> vue-echarts@6.2.3 postinstall /docker/akvorado/console/frontend/node_modules/vue-echarts
> node ./scripts/postinstall.js

[vue-echarts] Switched to Vue 3 environment.

> vue-demi@0.13.4 postinstall /docker/akvorado/console/frontend/node_modules/@vueuse/router/node_modules/vue-demi
> node ./scripts/postinstall.js


> vue-demi@0.13.4 postinstall /docker/akvorado/console/frontend/node_modules/@vueuse/core/node_modules/vue-demi
> node ./scripts/postinstall.js


> esbuild@0.14.49 postinstall /docker/akvorado/console/frontend/node_modules/esbuild
> node install.js

added 283 packages in 5.697s
▶ formatting code…
▶ building github.com/mgechev/revive@latest…
▶ running lint…
▶ building google.golang.org/protobuf/cmd/protoc-gen-go@v1.28.0…
▶ compiling protocol buffers definition…
make: protoc: No such file or directory
make: *** [Makefile:73: inlet/flow/decoder/flow-1.pb.go] Error 127

6 replies

vincentbernat Jul 14, 2022
Maintainer

protoc can also be installed from protobuf-compiler package. And yarn init is not needed (corepack enable either I think).

cmurrayis Jul 14, 2022
Author

Hi Vincent,
Once compiled it seems many additional steps are required to execute the application. I am trying to generate configuration files as that appears to be what I need to do however I'm just randomly testing things without knowing what is required exactly in each file

root@flow:/docker/akvorado/bin# ./akvorado inlet inlet.conf
8:07AM INF akvorado/common/http/root.go:160 > starting HTTP server listen=0.0.0.0:8080 module=akvorado/common/http
8:07AM INF akvorado/inlet/snmp/root.go:119 > starting SNMP component module=akvorado/inlet/snmp
8:07AM WRN akvorado/inlet/geoip/root.go:100 > skipping GeoIP component: no database specified module=akvorado/inlet/geoip
8:07AM INF akvorado/inlet/kafka/root.go:76 > starting Kafka component module=akvorado/inlet/kafka
8:07AM ERR akvorado/inlet/kafka/root.go:84 > unable to create async producer error="kafka: client has run out of available brokers to talk to: dial tcp 127.0.0.1:9092: connect: connection refused" brokers=127.0.0.1:9092 module=akvorado/inlet/kafka
8:07AM INF akvorado/inlet/snmp/root.go:211 > stopping SNMP component module=akvorado/inlet/snmp
8:07AM INF akvorado/inlet/snmp/root.go:209 > SNMP component stopped module=akvorado/inlet/snmp
8:07AM INF akvorado/common/http/root.go:197 > stopping HTTP component module=akvorado/common/http
8:07AM INF akvorado/common/http/root.go:200 > HTTP component stopped module=akvorado/common/http
Error: unable to start component: unable to create Kafka async producer: kafka: client has run out of available brokers to talk to: dial tcp 127.0.0.1:9092: connect: connection refused

root@flow:/docker/akvorado/bin# ./akvorado orchestrator orchestrator.conf
8:08AM INF akvorado/common/http/root.go:160 > starting HTTP server listen=0.0.0.0:8080 module=akvorado/common/http
8:08AM INF akvorado/common/clickhousedb/root.go:67 > starting ClickHouse component module=akvorado/common/clickhousedb
8:08AM INF akvorado/orchestrator/clickhouse/root.go:92 > starting ClickHouse component module=akvorado/orchestrator/clickhouse
8:08AM INF akvorado/orchestrator/kafka/root.go:45 > starting Kafka component module=akvorado/orchestrator/kafka
8:08AM ERR akvorado/orchestrator/clickhouse/migrations.go:83 > cannot execute check error="dial tcp 127.0.0.1:9000: connect: connection refused" module=akvorado/orchestrator/clickhouse step="create protocols dictionary"
8:08AM ERR akvorado/orchestrator/kafka/root.go:57 > unable to get admin client for topic creation error="kafka: client has run out of available brokers to talk to: dial tcp 127.0.0.1:9092: connect: connection refused" brokers=127.0.0.1:9092 module=akvorado/orchestrator/kafka
8:08AM INF akvorado/orchestrator/kafka/root.go:49 > Kafka component stopped module=akvorado/orchestrator/kafka
8:08AM INF akvorado/orchestrator/clickhouse/root.go:115 > stopping ClickHouse component module=akvorado/orchestrator/clickhouse
8:08AM INF akvorado/orchestrator/clickhouse/root.go:118 > ClickHouse component stopped module=akvorado/orchestrator/clickhouse
8:08AM INF akvorado/common/clickhousedb/root.go:94 > stopping ClickHouse component module=akvorado/common/clickhousedb
8:08AM INF akvorado/common/clickhousedb/root.go:97 > ClickHouse component stopped module=akvorado/common/clickhousedb
8:08AM INF akvorado/common/http/root.go:197 > stopping HTTP component module=akvorado/common/http
8:08AM INF akvorado/common/http/root.go:200 > HTTP component stopped module=akvorado/common/http
Error: unable to start component: unable to get admin client for topic creation: kafka: client has run out of available brokers to talk to: dial tcp 127.0.0.1:9092: connect: connection refused

vincentbernat Jul 14, 2022
Maintainer

The configuration is described in the documentation.

cmurrayis Jul 14, 2022
Author

Hi Vincent,

Yes sorry I forgot to add that I had run per the documentation already

./akvorado orchestrator --dump --check /dev/null > akvorado.yaml

root@flow:/docker/akvorado/bin# ./akvorado orchestrator akvorado.yaml
8:17AM INF akvorado/common/http/root.go:160 > starting HTTP server listen=0.0.0.0:8080 module=akvorado/common/http
8:17AM INF akvorado/common/clickhousedb/root.go:67 > starting ClickHouse component module=akvorado/common/clickhousedb
8:17AM INF akvorado/orchestrator/clickhouse/root.go:92 > starting ClickHouse component module=akvorado/orchestrator/clickhouse
8:17AM INF akvorado/orchestrator/kafka/root.go:45 > starting Kafka component module=akvorado/orchestrator/kafka
8:17AM ERR akvorado/orchestrator/clickhouse/migrations.go:83 > cannot execute check error="dial tcp 127.0.0.1:9000: connect: connection refused" module=akvorado/orchestrator/clickhouse step="create protocols dictionary"
8:17AM ERR akvorado/orchestrator/kafka/root.go:57 > unable to get admin client for topic creation error="kafka: client has run out of available brokers to talk to: dial tcp 127.0.0.1:9092: connect: connection refused" brokers=127.0.0.1:9092 module=akvorado/orchestrator/kafka
8:17AM INF akvorado/orchestrator/kafka/root.go:49 > Kafka component stopped module=akvorado/orchestrator/kafka
8:17AM INF akvorado/orchestrator/clickhouse/root.go:115 > stopping ClickHouse component module=akvorado/orchestrator/clickhouse
8:17AM INF akvorado/orchestrator/clickhouse/root.go:118 > ClickHouse component stopped module=akvorado/orchestrator/clickhouse
8:17AM INF akvorado/common/clickhousedb/root.go:94 > stopping ClickHouse component module=akvorado/common/clickhousedb
8:17AM INF akvorado/common/clickhousedb/root.go:97 > ClickHouse component stopped module=akvorado/common/clickhousedb
8:17AM INF akvorado/common/http/root.go:197 > stopping HTTP component module=akvorado/common/http
8:17AM INF akvorado/common/http/root.go:200 > HTTP component stopped module=akvorado/common/http
Error: unable to start component: unable to get admin client for topic creation: kafka: client has run out of available brokers to talk to: dial tcp 127.0.0.1:9092: connect: connection refused

root@flow:/docker/akvorado/bin# ./akvorado orchestrator akvorado.yaml --check
root@flow:/docker/akvorado/bin#

vincentbernat Jul 15, 2022
Maintainer

You need to run a Kafka broker and a ClickHouse server.

cmurrayis · 2022-07-15T07:42:17Z

cmurrayis
Jul 15, 2022
Author

Hi Vincent, Are there any other specifics required for this? The documentation says that the action of compiling our own install was that everything was in a single executable. The reason for compiling rather than using docker is due to being unable to get it to disable the included example exporter's and no exporter data being captured from legitimate sources with all troubleshooting documentation followed.

…

On Fri, 15 July 2022, 3:58 pm Vincent Bernat, ***@***.***> wrote: You need to run a Kafka broker and a ClickHouse server. — Reply to this email directly, view it on GitHub <#1 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AVKJAOQ2MRLE2JX2T4E6CMDVUD4ZVANCNFSM53QMISEA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

1 reply

vincentbernat Jul 15, 2022
Maintainer

I have updated the documentation to be less misleading. The intro contains a picture explaining the different components. You should try again the docker-compose path. I have updated a bit the troubleshooting section to help getting started.

cmurrayis · 2022-07-18T23:01:30Z

cmurrayis
Jul 18, 2022
Author

@vincentbernat

Seems like it is still gathering data correctly however the graphs don't appear to work on the main page or visualise page:

6 replies

vincentbernat Jul 19, 2022
Maintainer

The queries for the graphs are using InIfBoundary = external.

cmurrayis Jul 19, 2022
Author

Ok so maybe the JunOS reports it differently compared to the Mikrotik?

Also the scale is totally incorrect:

These should be around the 400 - 500mbps for the Mikrotik data. Do you know why the Juniper data isn't being displayed?

cmurrayis Jul 19, 2022
Author

vincentbernat Jul 19, 2022
Maintainer

The boundary is derived from interface names using rules in the configuration file.

cmurrayis Jul 19, 2022
Author

Ok so the configuration file needs to be manually configured gotcha.

I assume the same is said about client Transit customers not showing under our ASN in the chart and not showing us in the middle?

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Compile Error: #1

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 5 comments 13 replies

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

Compile Error: #1

cmurrayis Jul 13, 2022

Replies: 5 comments · 13 replies

vincentbernat Jul 14, 2022 Maintainer

vincentbernat Jul 14, 2022 Maintainer

cmurrayis Jul 14, 2022 Author

vincentbernat Jul 14, 2022 Maintainer

cmurrayis Jul 14, 2022 Author

vincentbernat Jul 14, 2022 Maintainer

cmurrayis Jul 14, 2022 Author

vincentbernat Jul 15, 2022 Maintainer

cmurrayis Jul 15, 2022 Author

vincentbernat Jul 15, 2022 Maintainer

cmurrayis Jul 18, 2022 Author

vincentbernat Jul 19, 2022 Maintainer

cmurrayis Jul 19, 2022 Author

cmurrayis Jul 19, 2022 Author

vincentbernat Jul 19, 2022 Maintainer

cmurrayis Jul 19, 2022 Author

cmurrayis
Jul 13, 2022

Replies: 5 comments 13 replies

vincentbernat
Jul 14, 2022
Maintainer

vincentbernat
Jul 14, 2022
Maintainer

cmurrayis
Jul 14, 2022
Author

vincentbernat Jul 14, 2022
Maintainer

cmurrayis Jul 14, 2022
Author

vincentbernat Jul 14, 2022
Maintainer

cmurrayis Jul 14, 2022
Author

vincentbernat Jul 15, 2022
Maintainer

cmurrayis
Jul 15, 2022
Author

vincentbernat Jul 15, 2022
Maintainer

cmurrayis
Jul 18, 2022
Author

vincentbernat Jul 19, 2022
Maintainer

cmurrayis Jul 19, 2022
Author

cmurrayis Jul 19, 2022
Author

vincentbernat Jul 19, 2022
Maintainer

cmurrayis Jul 19, 2022
Author