Skip to content

Snowflake is a network service for generating unique ID numbers at high scale with some simple guarantees (golang).

License

Notifications You must be signed in to change notification settings

Terry-Mao/gosnowflake

Repository files navigation

Terry-Mao/gosnowflake

Terry-Mao/gosnowflake is a network service for generating unique ID numbers at high scale with some simple guarantees (golang).

Requeriments

golang 1.2 is required.

zookeeper is required.

Installation

Just pull Terry-Mao/gosnowflake from github using go get:

# install golang from https://code.google.com/p/go/downloads/list
# here golang 1.2(linux-amd64)
$ wget https://go.googlecode.com/files/go1.2.linux-amd64.tar.gz
$ tar -xvf go1.2.linux-amd64.tar.gz
$ cp -R go /usr/local/
$ vim /etc/profile.d/golang.sh
# add below to golang.sh
# export GOROOT=/usr/local/go
# export PATH=$PATH:$GOROOT/bin
# export GOPATH=/data/apps/go
$ source /etc/profile.d/golang.sh
# download the code
$ go get -u github.com/Terry-Mao/gosnowflake
# find the dir
$ cd $GOPATH/src/github.com/Terry-Mao/gosnowflake
# compile
$ go install
$ cp ./gosnowflake-example.conf $GOPATH/bin/gosnowflake.conf
$ cp log.xml $GOPATH/bin/log.xml
$ cd $GOPATH/bin
# run
$ ./gosnowflake -conf=./gosnowflake.conf
# for help
$ ./gosnowflake -h
# test
$ cd $GOPATH/src/github.com/Terry-Mao/gosnowflake/client
$ go test -conf=./test-examples.conf

Document

# gosnowflake configuration file example

# Note on units: when memory size is needed, it is possible to specify
# it in the usual form of 1k 5GB 4M and so forth:
#
# 1kb => 1024 bytes
# 1mb => 1024*1024 bytes
# 1gb => 1024*1024*1024 bytes
#
# units are case insensitive so 1GB 1Gb 1gB are all the same.

# Note on units: when time duration is needed, it is possible to specify
# it in the usual form of 1s 5M 4h and so forth:
#
# 1s => 1000 * 1000 * 1000 nanoseconds
# 1m => 60 seconds
# 1h => 60 minutes
#
# units are case insensitive so 1h 1H are all the same.

[base]
# When running daemonized, gosnowflake writes a pid file in 
# /tmp/gosnowflake.pid by default. You can specify a custom pid file 
# location here.
pid /tmp/gosnowflake.pid

# Sets the maximum number of CPUs that can be executing simultaneously.
# This call will go away when the scheduler improves. By default the number of 
# logical CPUs is set.
# 
# maxproc 4

# By default gosnowflake listens for connections from all the network interfaces
# available on the server on 8080 port. It is possible to listen to just one or 
# multiple interfaces using the "rpc.bind" configuration directive, 
# followed by one or more IP addresses and port.
#
# Examples:
#
# Note this directive is only support "golang rpc" protocol
# rpc.bind 192.168.1.100:8080,10.0.0.1:8080
# rpc.bind 127.0.0.1:8080
# rpc.bind :8080

# By default gosnowflake thrift listens for connections from all the network interfaces
# available on the server on 8080 port. It is possible to listen to just one or 
# multiple interfaces using the "rpc.bind" configuration directive, 
# followed by one or more IP addresses and port.
#
# Examples:
#
# Note this directive is only support "thrift" protocol
# thrift.bind 192.168.1.100:8080,10.0.0.1:8080
# thrift.bind 127.0.0.1:8080
# thrift.bind :8080

# This is used by gosnowflake service profiling (pprof).
# By default gosnowflake pprof listens for connections from local interfaces on 6971
# port. It's not safty for listening internet IP addresses.
#
# Examples:
#
# pprof.bind 192.168.1.100:6971,10.0.0.1:6971
# pprof.bind 127.0.0.1:6971
# pprof.bind :6971

# This is used by gosnowflake service get stat info by http.
# By default gosnowflake pprof listens for connections from local interfaces on 6972
# port. It's not safty for listening internet IP addresses.
#
# Examples:
#
# stat.bind 192.168.1.100:6971,10.0.0.1:6971
# stat.bind 127.0.0.1:6971
# stat.bind :6971

# The working directory.
#
# The log will be written inside this directory, with the filename specified
# above using the 'logfile' configuration directive.
#  
# Note that you must specify a directory here, not a file name.
dir ./

# Log4go configuration path
log ./log.xml

################################## ZOOKEEPER ##################################
[zookeeper]
# The zookeeper cluster section. When gosnowflake start, it will register data 
# in the zookeeper cluster and create a ephemeral node. When gosnowflake died, 
# the node will drop by zookeeper cluster. 

# Zookeeper cluster addresses. Mutiple address split by a ",".
# Examples:
#
# addr 192.168.1.100:2181,10.0.0.1:2181
# addr 127.0.0.1:2181
# addr 10.20.216.122:2181

# Zookeeper cluster session idle timeout seconds. Zookeeper will close the 
# connection after a client is idle for N seconds.
# Examples:
#
# timeout 30s
# timeout 15s
timeout 30s

# gosnowflake zookeeper root path.
path /gosnowflake-servers

################################## GOSNOWFLAKE ################################
[snowflake]
# snowflake must set a datacenter [0, 31], must be unique in all datacenter.
# Examples:
#
# datacenter 0
# datacenter 1
datacenter 0

# register which worker, must be unique in one datacenter.
# multiple worker id register can split by a ",".
# Examples:
#
# worker 0
# worker 0,1,2
worker 0,1,2

RPC API

SnowflakeRPC.NextId: generate a snowflake id.

SnowflakeRPC.DatacenterId: get gosnowflake service's datacenterId.

SnowflakeRPC.Timestamp: get gosnowflake service's current timestamp.

SnowflakeRPC.Ping: get gosnowflake service's status.

Usage

if err := Init(MyConf.ZKServers, MyConf.ZKPath, MyConf.ZKTimeout); err != nil {
    panic(err)
}
c := NewClient(MyConf.WorkerId)                                             
defer c.Close()
defer c.Destroy()
time.Sleep(1 * time.Second)                                             
id, err := c.Id()                                                       
if err != nil {                                                         
    panic(err)
}                                                                       
fmt.Printf("gosnwoflake id: %d\n", id)                                  

Highly Available

use heartbeat or keepalived apply a VIP for the client.

a easy keepalived configration: here

(1.0.1)

use zookeeper for client failover

golang client sdk: here

(1.1)

LICENSE

gosnowflake is is distributed under the terms of the GNU General Public License, version 3.0 GPLv3

About

Snowflake is a network service for generating unique ID numbers at high scale with some simple guarantees (golang).

Resources

License

Stars

Watchers

Forks

Packages

No packages published