Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Proposal] config feature roadmap #3254

Open
1 task
shenqidebaozi opened this issue Mar 16, 2024 · 22 comments
Open
1 task

[Proposal] config feature roadmap #3254

shenqidebaozi opened this issue Mar 16, 2024 · 22 comments
Labels
proposal proposal

Comments

@shenqidebaozi
Copy link
Sponsor Member

shenqidebaozi commented Mar 16, 2024

Dear Kratos friends, the config package has been widely used since its inception.
Received some feedback on issues and feature requirements during this period.
The current plan is to expand the config package functionality. Please feel free to comment below.

@shenqidebaozi shenqidebaozi added the proposal proposal label Mar 16, 2024
Copy link

dosubot bot commented Mar 16, 2024

It seems like you've got this under control, if you want help or have specific questions, let me know what I can do for you!

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

@shenqidebaozi shenqidebaozi changed the title [Proposal] Config feature roadmap [Proposal] config feature roadmap Mar 16, 2024
@xiongpan828
Copy link
Contributor

基于proto生成结构体 并发安全的配置热更新,支持配置validate option选项

@kratos-ci-bot
Copy link
Collaborator

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


Generate structure based on proto, concurrent and safe configuration hot update, support configuration of validate option option

@shenqidebaozi
Copy link
Sponsor Member Author

基于proto生成结构体 并发安全的配置热更新,支持配置validate option选项

@xiongpan828
These features are already supported.

  1. In kratos-layout, proto is used as the configuration definition by default
  2. It is inherently concurrency secure
  3. Use PGV directly to validate.

@xiongpan828
Copy link
Contributor

基于proto生成结构体配置安全的配置热更新,支持配置validate选项选项

@xiongpan828 这些功能已经得到支持。

  1. kratos-layout,默认使用proto作为配置定义
  2. 它本质上是并发安全的
  3. 直接使用PGV来验证。
  1. 目前Watch需要选择一个key,如果想监听配置任意位置更新呢
  2. 当前配置结构体一直读取过程中,热更新进行了set ,底层配置上并发安全的,但是结构体不是
  3. PGV生成校验代码目前是在外部显示调用,如果放框架内部是不是要优雅一点

@kratos-ci-bot
Copy link
Collaborator

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


Secure configuration hot update based on proto generated structure configuration, supports configuration of validate option options

@xiongpan828 These features are already supported.

  1. In kratos-layout, proto is used as the configuration definition by default
  2. It is inherently concurrency safe
  3. Use PGV directly to verify.
  1. Currently, Watch needs to select a key. What if you want to monitor the configuration for updates at any location?
  2. While the current configuration structure is being read, the hot update is set. The underlying configuration is concurrency-safe, but the structure is not.
  3. The verification code generated by PGV is currently called externally. If placed inside the framework, should it be more elegant?

@shenqidebaozi
Copy link
Sponsor Member Author

shenqidebaozi commented Mar 16, 2024

基于proto生成结构体配置安全的配置热更新,支持配置validate选项选项

@xiongpan828 这些功能已经得到支持。

  1. kratos-layout,默认使用proto作为配置定义
  2. 它本质上是并发安全的
  3. 直接使用PGV来验证。
  1. 目前Watch需要选择一个key,如果想监听配置任意位置更新呢
  2. 当前配置结构体一直读取过程中,热更新进行了set ,底层配置上并发安全的,但是结构体不是
  3. PGV生成校验代码目前是在外部显示调用,如果放框架内部是不是要优雅一点
    @xiongpan828
  1. Watch any key, all you need to do is set a root key。Can consider supporting the entire watch configuration, but I haven't figured out if it's necessary yet.

The config package focuses more on how to parse, merge, and finally serialize configurations into user structures

  1. The security of the structure should not be guaranteed by the framework, as this part itself is user-defined. Config only ensures that the dynamic update process is concurrent and secure within the config package, and provides a hook externally. This part should be implemented by users by locking the structure.
  2. The config package should not rely on the PGV library, as it is not a mandatory component to rely on.

@flc1125
Copy link
Contributor

flc1125 commented Mar 17, 2024

注入到 context.Context

我现在的做法(参考):

package main

import (
	"context"
	"fmt"

	"github.com/go-kratos/kratos/v2"

	"github.com/go-kratos-ecosystem/components/v2/config"
	"github.com/go-kratos-ecosystem/components/v2/env"
	"github.com/go-kratos-ecosystem/components/v2/provider"
	"github.com/go-kratos-ecosystem/components/v2/timezone"
)

type Config struct {
	Host string
}

func main() {
	ctx, err := provider.Pipe(context.Background(),
		// 注入到 context 中
		config.Provider(&Config{
			Host: "localhost",
		}),
		// 注入其他的 Provider
		timezone.Provider(timezone.Local("Asia/Shanghai")),
		env.Provider(env.Prod),
	)
	if err != nil {
		panic(err)
	}

	app := kratos.New(
		kratos.Context(ctx),
		kratos.Server(),
	)

	if err := app.Run(); err != nil {
		panic(err)
	}
}

// DemoService is a demo service.
func DemoService(ctx context.Context) error {
	cfg, ok := config.FromContext[*Config](ctx)
	if !ok {
		return fmt.Errorf("config not found")
	}
	_ = cfg

	return nil
}

参考资料:

@xiongpan828
Copy link
Contributor

注入到 context.Context

我现在的做法(参考):

package main

import (
	"context"
	"fmt"

	"github.com/go-kratos/kratos/v2"

	"github.com/go-kratos-ecosystem/components/v2/config"
	"github.com/go-kratos-ecosystem/components/v2/env"
	"github.com/go-kratos-ecosystem/components/v2/provider"
	"github.com/go-kratos-ecosystem/components/v2/timezone"
)

type Config struct {
	Host string
}

func main() {
	ctx, err := provider.Pipe(context.Background(),
		// 注入到 context 中
		config.Provider(&Config{
			Host: "localhost",
		}),
		// 注入其他的 Provider
		timezone.Provider(timezone.Local("Asia/Shanghai")),
		env.Provider(env.Prod),
	)
	if err != nil {
		panic(err)
	}

	app := kratos.New(
		kratos.Context(ctx),
		kratos.Server(),
	)

	if err := app.Run(); err != nil {
		panic(err)
	}
}

// DemoService is a demo service.
func DemoService(ctx context.Context) error {
	cfg, ok := config.FromContext[*Config](ctx)
	if !ok {
		return fmt.Errorf("config not found")
	}
	_ = cfg

	return nil
}

参考资料:

👍👍👍谢谢 get了

@shcw
Copy link

shcw commented Mar 18, 2024

菜鸟咨询:能否进行配置文件没配置的部分给默认值 (比较推荐的方式) 😁

@kratos-ci-bot
Copy link
Collaborator

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


Rookie consultation: Is it possible to give default values ​​to the unconfigured parts of the configuration file (a more recommended method) 😁

@flc1125
Copy link
Contributor

flc1125 commented Mar 18, 2024

菜鸟咨询:能否进行配置文件没配置的部分给默认值 (比较推荐的方式) 😁

我的做法:(不过用了泛型,可能会有点性能损耗,我自己测了下,损耗不大。结合实际并发情况使用吧。)

image

相关包代码:

package values

// If returns trueVal if condition is true, otherwise falseVal
//
// Example:
//
//	If(true, "foo", "bar") // "foo"
//	If(false, "foo", "bar") // "bar"
func If[T any](condition bool, trueVal T, falseVal T) T {
	if condition {
		return trueVal
	}

	return falseVal
}

// With calls the given callbacks with the given value then return the value.
//
// Example:
//
//	With("foo", func(s string) string {
//		return s + "bar"
//	}, func(s string) string {
//		return s + "baz"
//	}) // "foobarbaz"
func With[T any](value T, callbacks ...func(T) T) T {
	for _, callback := range callbacks {
		if callback != nil {
			value = callback(value)
		}
	}

	return value
}

type Proxy[T any] struct {
	value T
}

func NewProxy[T any](value T) *Proxy[T] {
	return &Proxy[T]{
		value: value,
	}
}

func (p *Proxy[T]) When(condition bool, callbacks ...func(T) T) *Proxy[T] {
	if condition {
		return p.With(callbacks...)
	}
	return p
}

func (p *Proxy[T]) Value() T {
	return p.value
}

@shenqidebaozi
Copy link
Sponsor Member Author

注入到 context.Context

我现在的做法(参考):

package main

import (
	"context"
	"fmt"

	"github.com/go-kratos/kratos/v2"

	"github.com/go-kratos-ecosystem/components/v2/config"
	"github.com/go-kratos-ecosystem/components/v2/env"
	"github.com/go-kratos-ecosystem/components/v2/provider"
	"github.com/go-kratos-ecosystem/components/v2/timezone"
)

type Config struct {
	Host string
}

func main() {
	ctx, err := provider.Pipe(context.Background(),
		// 注入到 context 中
		config.Provider(&Config{
			Host: "localhost",
		}),
		// 注入其他的 Provider
		timezone.Provider(timezone.Local("Asia/Shanghai")),
		env.Provider(env.Prod),
	)
	if err != nil {
		panic(err)
	}

	app := kratos.New(
		kratos.Context(ctx),
		kratos.Server(),
	)

	if err := app.Run(); err != nil {
		panic(err)
	}
}

// DemoService is a demo service.
func DemoService(ctx context.Context) error {
	cfg, ok := config.FromContext[*Config](ctx)
	if !ok {
		return fmt.Errorf("config not found")
	}
	_ = cfg

	return nil
}

参考资料:

This is not a good proposal and should not be provided by the config package

@flc1125
Copy link
Contributor

flc1125 commented Mar 18, 2024

注入到 context.Context
我现在的做法(参考):

package main

import (
	"context"
	"fmt"

	"github.com/go-kratos/kratos/v2"

	"github.com/go-kratos-ecosystem/components/v2/config"
	"github.com/go-kratos-ecosystem/components/v2/env"
	"github.com/go-kratos-ecosystem/components/v2/provider"
	"github.com/go-kratos-ecosystem/components/v2/timezone"
)

type Config struct {
	Host string
}

func main() {
	ctx, err := provider.Pipe(context.Background(),
		// 注入到 context 中
		config.Provider(&Config{
			Host: "localhost",
		}),
		// 注入其他的 Provider
		timezone.Provider(timezone.Local("Asia/Shanghai")),
		env.Provider(env.Prod),
	)
	if err != nil {
		panic(err)
	}

	app := kratos.New(
		kratos.Context(ctx),
		kratos.Server(),
	)

	if err := app.Run(); err != nil {
		panic(err)
	}
}

// DemoService is a demo service.
func DemoService(ctx context.Context) error {
	cfg, ok := config.FromContext[*Config](ctx)
	if !ok {
		return fmt.Errorf("config not found")
	}
	_ = cfg

	return nil
}

参考资料:

This is not a good proposal and should not be provided by the config package这不是一个好的建议,不应由配置包提供

这个确实,适合用工具或者有个类似 bootstrap 统一进行初始化的包实现好点。

@CodeFish-xiao
Copy link

can't reading bool value from environment config

@XiaoK29
Copy link

XiaoK29 commented Mar 18, 2024

希望支持byte序列化,因为配置有从配置中心读取到程序中,目前的Scan方法只能读取文件path

@kratos-ci-bot
Copy link
Collaborator

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


We hope to support byte serialization because the configuration is read from the configuration center into the program. The current Scan method can only read the file path.

@JellyTony
Copy link
Contributor

Load 支持多次调用,因为有时候比如用配置中心,先拿到本地的配置然后才会连上配置中心接着 load

@kratos-ci-bot
Copy link
Collaborator

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


Load supports multiple calls, because sometimes, for example, when using the configuration center, you first get the local configuration and then connect to the configuration center and then load

@JellyTony
Copy link
Contributor

希望支持先 path 然后 在 scan , 类似 https://github.com/go-micro/go-micro/blob/master/config/config.go#L82

config.Get("cos", "user", "domain").Scan(&domain)

@kratos-ci-bot
Copy link
Collaborator

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


Hope to support path first and then scan, similar to https://github.com/go-micro/go-micro/blob/master/config/config.go#L82

config.Get("cos", "user", "domain").Scan(&domain)

@go-slark
Copy link

go-slark commented Apr 5, 2024

1.Need to load the configuration from the config center and write it to a local configuration file, such as configmap, in case the config center goes down and the service doesn't start?
2. Multi-configuration source loading priority/order

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
proposal proposal
Projects
None yet
Development

No branches or pull requests

9 participants