Add kine support

[vitorsavian]

Proposed Changes

• Add kine to rke2

Types of Changes

• New Feature

Verification

• You can set kine using the minimal docs

• after setting the database you will need to create a config.yaml

datastore-endpoint: "mysql://root:$PASSWORD@tcp(localhost:3306)/kine"
token: "test"

• then run rke2

rke2 server

Testing

Linked Issues

• Kine TLS support for k3s k3s-io/k3s#9563
• Make Kine work in RKE2 #5213

User-Facing Change

Kine is now supported

Further Comments

Waiting for

• Add tls for kine k3s-io/k3s#9572

[brandond]

This looks good!

What happens if you try to start a server with with datastore-endpoint set but etcd disabled, or try to start an etcd-only node with the datastore-endpoint set? Does it handle that properly?

[vitorsavian]

so @brandond if I start a server with datastore-endpointand after that I init a server with --disable-etcd and --datastore-endpoint it panics on the second server.

INFO[0000] Applying Pod Security Admission Configuration
INFO[0000] Starting rke2 v1.29.3+dev.e9946e62-dirty (e9946e62d20c875f320f03eaef51fd2643e12441.dirty)
INFO[0000] Starting temporary kine to reconcile with datastore
INFO[0000] Configuring mysql database connection pooling: maxIdleConns=2, maxOpenConns=0, connMaxLifetime=0s
INFO[0000] Configuring database table schema and indexes, this may take a moment...
INFO[0000] Database tables and indexes are up to date
INFO[0000] Kine available at unix://kine.sock
INFO[0000] Reconciling bootstrap data between datastore and disk
INFO[0001] TTL events watch channel closed
INFO[0001] TTL events work queue has shut down
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x1eb1262]

goroutine 1 [running]:
github.com/k3s-io/k3s/pkg/clientaccess.(*Info).Get(0x0, {0x3e8f2a5, 0x8}, {0x0, 0x0, 0x0})
        /go/pkg/mod/github.com/vitorsavian/k3s@v0.0.0-20240313175543-f3e6fe801132/pkg/clientaccess/token.go:279 +0x42
github.com/k3s-io/k3s/pkg/etcd.ClientURLs({0xc000dcffb0?, 0x0?}, 0xc000da9b30?, {0x0, 0x0})
        /go/pkg/mod/github.com/vitorsavian/k3s@v0.0.0-20240313175543-f3e6fe801132/pkg/etcd/etcd.go:1367 +0x6d
github.com/k3s-io/k3s/pkg/cluster.(*Cluster).Start(0xc000dcffb0, {0x4707b38?, 0xc000da9b30})
        /go/pkg/mod/github.com/vitorsavian/k3s@v0.0.0-20240313175543-f3e6fe801132/pkg/cluster/cluster.go:47 +0x105
github.com/k3s-io/k3s/pkg/daemons/control.prepare({0x4707b38, 0xc000da9b30}, 0xc000dbd408)
        /go/pkg/mod/github.com/vitorsavian/k3s@v0.0.0-20240313175543-f3e6fe801132/pkg/daemons/control/server.go:297 +0x2ff
github.com/k3s-io/k3s/pkg/daemons/control.Server({0x4707b38?, 0xc000da9b30}, 0xc000dbd408)
        /go/pkg/mod/github.com/vitorsavian/k3s@v0.0.0-20240313175543-f3e6fe801132/pkg/daemons/control/server.go:36 +0xf7
github.com/k3s-io/k3s/pkg/server.StartServer({0x4707b38?, 0xc000da9b30}, 0xc000dbd400, 0x1?)
        /go/pkg/mod/github.com/vitorsavian/k3s@v0.0.0-20240313175543-f3e6fe801132/pkg/server/server.go:56 +0x85
github.com/k3s-io/k3s/pkg/cli/server.run(0xc000d9d340, 0x6c545e0, {0x0, 0x0, 0x38271e0?}, {0xc000afe908, 0x0, 0x0?})
        /go/pkg/mod/github.com/vitorsavian/k3s@v0.0.0-20240313175543-f3e6fe801132/pkg/cli/server/server.go:486 +0x339a
github.com/k3s-io/k3s/pkg/cli/server.RunWithControllers(...)
        /go/pkg/mod/github.com/vitorsavian/k3s@v0.0.0-20240313175543-f3e6fe801132/pkg/cli/server/server.go:48
github.com/rancher/rke2/pkg/rke2.Server(_, {{0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, {{0x0, ...}, ...}, ...})
        /source/pkg/rke2/rke2.go:123 +0x656
github.com/rancher/rke2/pkg/cli/cmds.ServerRun(0xc000d9d340?)
        /source/pkg/cli/cmds/server.go:167 +0x89
github.com/urfave/cli.HandleAction({0x3669560?, 0x4245b20?}, 0x6?)
        /go/pkg/mod/github.com/urfave/cli@v1.22.14/app.go:524 +0x50
github.com/urfave/cli.Command.Run({{0x3e6e13a, 0x6}, {0x0, 0x0}, {0x0, 0x0, 0x0}, {0x3ffb46d, 0x15}, {0xc000db4108, ...}, ...}, ...)
        /go/pkg/mod/github.com/urfave/cli@v1.22.14/command.go:175 +0x63e
github.com/urfave/cli.(*App).Run(0xc0007d3340, {0xc0000740f0, 0xf, 0xf})
        /go/pkg/mod/github.com/urfave/cli@v1.22.14/app.go:277 +0xb27
main.main()
        /source/main.go:23 +0x97b

But if I sent without --datastore-endpoint, the etcd only server returns this

WARN[0202] Received HTTP bootstrap request from {IP}:45458, but embedded etcd is not enabled.   

in the use case with etcd-only it stays on this part, but this was resolved in my new change in the k3s kine tls PR.

INFO[0134] Waiting for apiserver addresses
INFO[0134] Waiting to retrieve kube-proxy configuration; server is not ready: https://127.0.0.1:6444/v1-rke2/readyz: 500 Internal Server Error
{"level":"warn","ts":"2024-03-18T21:25:04.636534-0300","logger":"etcd-client","caller":"v3@v3.5.9-k3s1/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc000887180/kine.sock","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = latest balancer error: last connection error: connection error: desc = \"transport: authentication handshake failed: tls: failed to verify certificate: x509: certificate signed by unknown authority\""}
WARN[0136] Failed to get apiserver address from etcd: context deadline exceeded

[brandond]

--disable-etcd should have no impact since we're not using etcd... unless we have somehow misused some of the etcd code to do some of the kine setup either here, or in k3s?