igo (go evaluator in go)

This library is compatible with Go 1.17+

Please refer to CHANGELOG.md if you encounter breaking changes.

• Motivation
• Introduction
• Usage
• Performance
• Bugs
• Contribution
• License

Motivation

The goal of this library is to be able dynamically execute go code directly from Go/WebAssembly within reasonable time. Some existing alternative providing go evaluation on the fly are prohibitively slow:

• GoEval
• GoVal
• Yaegi .

See performance section for details.

Introduction

In order to reduce execution time, this project first produces execution plan alongside with state needed to execute it. One execution plan can be shared alongside many instances state needed by executor. State holds both variables and execution state used in the evaluation code.

package mypkg

import "github.com/viant/igo"

func usage() {
	scope := igo.NewScope()
	code := "go code here"
	executor, stateNew, err := scope.Compile(code)
	if err != nil {
		panic(err)
	}
	state := stateNew() //creates memory instance needed by executor 
	executor.Exec(state)
}
	

Usage

Expression

package mypkg

import (
	"log"
	"reflect"
	"github.com/viant/igo"
)

func ExampleScope_BoolExpression() {
	type Performance struct {
		Id      int
		Price   float64
		MetricX float64
	}
	scope := igo.NewScope()
	_, err := scope.DefineVariable("perf", reflect.TypeOf(Performance{}))
	_, err = scope.DefineVariable("threshold", reflect.TypeOf(0.0))
	if err != nil {
		log.Fatalln(err)
	}
	//Compile bool expression
	expr, err := scope.BoolExpression("perf.MetricX > threshold && perf.Price > 1.0")
	if err != nil {
		log.Fatalln(err)
	}

	perfs := []Performance{
		{MetricX: 1.5, Price: 3.2},
		{MetricX: 1.2, Price: 1.2},
		{MetricX: 1.7, Price: 0.4},
	}
	var eval = make([]bool, len(perfs))
	for i := range perfs {
		_ = expr.Vars.SetValue("perf", perfs[i])
		_ = expr.Vars.SetFloat64("threshold", 0.7)
		eval[i] = expr.Compute()
	}
}

Go evaluation

package mypkg

import (
	"log"
	"fmt"
	"github.com/viant/igo"
)

func ExampleScope_Compile() {
	code := `type Foo struct {
        ID int
        Name string
    }
    var foos = make([]*Foo, 0)
    for i:=0;i<10;i++ {
        foos = append(foos, &Foo{ID:i, Name:"nxc"})
    }
    s := 0
    for i, foo := range foos {
        if i %2  == 0 {
        s += foo.ID
    }
}`

	scope := igo.NewScope()
	executor, stateNew, err := scope.Compile(code)
	if err != nil {
		log.Fatalln(err)
	}

	state := stateNew() //variables constructor, one per each concurent execution, execution can be shared
	executor.Exec(state)
	result, _ := state.Int("s")
	fmt.Printf("result: %v\n", result)
}

Setting code variables

package mypkg

import (
	"log"
	"fmt"
	"reflect"
	"github.com/viant/igo"
)

func ExampleScope_DefineVariable() {
	code := `
	x := 0.0
	for _, account := range accounts {
		x += account.Total
	}
	`
	type Account struct {
		Total float64
	}

	scope := igo.NewScope()
	err := scope.RegisterType(reflect.TypeOf(Account{})) //Register all non-primitive types used in code 
	if err != nil {
		log.Fatalln(err)
	}
	executor, stateNew, err := scope.Compile(code)
	if err != nil {
		log.Fatalln(err)
	}
	state := stateNew()
	err = state.SetValue("accounts", []Account{
		{Total: 1.3},
		{Total: 3.7},
	})
	if err != nil {
		log.Fatalln(err)
	}
	executor.Exec(state)
	result, _ := state.Float64("x")
	fmt.Printf("result: %v\n", result)
}

Go function

package mypkg

import (
	"log"
	"fmt"
	"reflect"
	"github.com/viant/igo"
)

func ExampleScope_Function() {
	type Foo struct {
		Z int
	}
	scope := igo.NewScope()
	_ = scope.RegisterType(reflect.TypeOf(Foo{}))
	fn, err := scope.Function(`func(x, y int, foo Foo) int {
            return (x+y)/foo.Z
        }`)
	if err != nil {
		log.Fatalln(err)
	}
	typeFn, ok := fn.(func(int, int, Foo) int)
	if !ok {
		log.Fatalf("expected: %T, but had: %T", typeFn, fn)
	}
	r := typeFn(1, 2, Foo{3})
	fmt.Printf("%v\n", r)
}

Registering types

To use data types defined outside the code, register type with (Scope).RegisterType(type) function or (Scope).RegisterNamedType(name, type)

    scope := igo.NewScope()
    _ = scope.RegisterType(reflect.TypeOf(Foo{}))

DefineVariable

Registering function

To use function defined outside the code, register type with (Scope).RegisterFunc(name, function) function

    scope := igo.NewScope()
    scope.RegisterFunc(testCase.fnName, testCase.fn)

Performance

Expression evaluation

See benchmark for the following expression evaluation:

10 + (5 * x / y * (z - 7))

BenchmarkScope_IntExpression_Native-16          134789700                8.627 ns/op           0 B/op          0 allocs/op
BenchmarkScope_IntExpression-16                 19722770                57.06 ns/op            8 B/op          1 allocs/op
BenchmarkScope_IntExpression_GoVal-16             625620                2040 ns/op          2328 B/op        11 allocs/op

GoVal is ~255 slower for the presented expression comparing to natively compiled code, while Igo is only ~7 times slower

Code execution

See benchmark for the following code:

count :=0
for i :=0;i<100;i++ {
    count += i
}
print(count)

GoEval evaluation takes almost ~24K time longer than natively compiled code, whereas this project is only around ~35 slower. As point of reference using native go reflection adds on average around 100x time execution overhead.

Benchmark_Loop_Native-16                        35385890               30.36 ns/op             0 B/op          0 allocs/op
Benchmark_Loop_Igo-16                            1000000              1081  ns/op               0 B/op          0 allocs/op
Benchmark_Loop_GoEval-16                            1429            739672  ns/op          788350 B/op       3180 allocs/op

See the following benchmark that runs 100 000 000 loop iteration:

	z := 0
	a := 100000000
	r := 1
	for i := 1; i <= a; i++ {
		r += i
	}
	z = r

BenchmarkLoop_Yaegi-16    	                      1	       3581461319 ns/op	          47560 B/op	    681 allocs/op
BenchmarkLongLoop_Igo-16                          2         661982642 ns/op               8 B/op          0 allocs/op
BenchmarkLongLoop_Native-16                      48          24813792 ns/op               0 B/op          0 allocs/op

Igo is ~26x times slower than natively compile code, whereas Yaegi is ~144x times slower than natively compile code

Bugs

This project does not implement full golang spec, but just a subset. At least following expression/types/construct are not supported

• map type
• named interface types (since pointers are used to access/mutate data)
• go routines
• select expression
• switch expression
• closures

Contributing to igo

Igo is an open source project and contributors are welcome!

See TODO list

Credits and Acknowledgements

Library Author: Adrian Witas