[Feature] Provide convenient scripts and package for Venice services (controller, router, server)

[eolivelli]

Willingness to contribute

I can collaborate

Feature Request Proposal

Users need a way to bootstrap conveniently the services, configure them.

Currently it is pretty hard to bootstrap the services:

• no default configuration files
• no scripts to start/stop the services
• no way to configure logging/JVM properties
• no released tar.gz package
• no easy way to run the admin tool

Motivation

What is the use case for this feature?

Bootstrap the services in a VM.
Provide support for a future standard docker image

Details

Some examples:

• ZooKeeper: https://github.com/apache/zookeeper/tree/master/bin
• Pulsar: https://github.com/apache/pulsar/tree/master/bin

What component(s) does this bug affect?

• Controller: This is the control-plane for Venice. Used to create/update/query stores and their metadata.
• Router: This is the stateless query-routing layer for serving read requests.
• Server: This is the component that persists all the store data.
• VenicePushJob: This is the component that pushes derived data from Hadoop to Venice backend.
• Thin Client: This is a stateless client users use to query Venice Router for reading store data.
• Fast Client: This is a stateful client users use to query Venice Server for reading store data.
• Da Vinci Client: This is an embedded, stateful client that materializes store data locally.
• Samza: This is the library users use to make nearline updates to store data.
• Admin Tool: This is the stand-alone client used for ad-hoc operations on Venice.

[eolivelli]

I have found that if you build venice with

./gradlew shadowDistTar

Then you can find some tarballs for the services in the "distribution" directories.
This is a good starting point but it is not enough:

• it should be only 1 tarball
• it should provide default configuration files
• it should provide the scripts to run the admin tool
• it would be great to have a script like this https://github.com/apache/pulsar/blob/master/docker/pulsar/scripts/apply-config-from-env.py to help building docker images