Goutube

Streaming utility to build largely-scalable, highly available, fast, and secure distributed streaming APIs!

Simple reason is that distributed systems are cool because of many reasons and some of these reasons are largely-scalability, high availability, secure, and fast.
With this project, I attempt to go in-depth on how to build a system that grows in functionality as well as users and the team developing it.
It's my attempt to broaden my knowledge and make it strong by developing this real-world end-to-end product.

Configuration parameters

Parameters	Type	Usage
DataDir	string	Path to the directory to store data locally.
BindAddr	string	The address that Goutube will bind to for communication with other members on the ring.
RPCPort	int	The port that Goutube will bind to for the member's RPC server.
ReplicationPort	string	If this server allows replication (by setting the ParticipationRule). This is the port where leader and its follower will communicate to sync with each other. (eventually consistent)
NodeName	string	Unique node name to identify this member.
SeedAddresses	[]string	Addresses of other members to join the ring upon start up.
VirtualNodeCount	int	Number of virtual nodes to create on the ring for this member.
ACLModelFile	string	Path to the model file for authorization using casbin
ACLPolicyFile	string	Path to the policy file for authorization using casbin
ServerTLSConfig	*tls.Config	To configure the server for authentication.
PeerTLSConfig	int	To configure the client for authentication (to connect to other servers for replication, forwarding, etc.)
LeaderAddresses	[]string	Hash function to calculate position of the server on the ring.
Rule	ParticipationRule	Participation rule to define the boundary and extension of the server.
MaxChunkSize	uint64	Size of the chunk that can be processed by this server.
MultiStreamPercent	int	Percents of the number of followers to return upon GetMetadata request?

Feel free to create a PR, I’m more than happy to review and merge it.

Feel free to create an issue, if you have any problem running this distributed system or any suggestions.

Name		Name	Last commit message	Last commit date
Latest commit History 42 Commits
.github/workflows		.github/workflows
api/streaming/v1		api/streaming/v1
cert		cert
client		client
log		log
pointcron		pointcron
.gitignore		.gitignore
LICENSE		LICENSE
Makefile		Makefile
README.md		README.md
agent.go		agent.go
agent_test.go		agent_test.go
arc.go		arc.go
arc_test.go		arc_test.go
architecture.png		architecture.png
authorizer.go		authorizer.go
cmd.go		cmd.go
config.go		config.go
const.go		const.go
distributed.go		distributed.go
distributed_test.go		distributed_test.go
files.go		files.go
fsm.go		fsm.go
go.mod		go.mod
go.sum		go.sum
in_memory_store.go		in_memory_store.go
loadbalancer.go		loadbalancer.go
loadbalancer_test.go		loadbalancer_test.go
locus.go		locus.go
locus_test.go		locus_test.go
picker.go		picker.go
picker_test.go		picker_test.go
point.go		point.go
point_test.go		point_test.go
promise.go		promise.go
redis_store.go		redis_store.go
replicate.go		replicate.go
replication_cluster.go		replication_cluster.go
replication_cluster_test.go		replication_cluster_test.go
resolver.go		resolver.go
resolver_test.go		resolver_test.go
server.go		server.go
server_test.go		server_test.go
streaming.go		streaming.go
svc_util.go		svc_util.go
tcp_transport.go		tcp_transport.go
tls.go		tls.go
transport.go		transport.go
transport_test.go		transport_test.go
types.go		types.go
util.go		util.go