Get Started with Data Streaming with gRPC

A demo to show how to get started with building Data Streaming API using gRPC. We will use Redpanda as our streaming platform.

Prerequisites

To run the demo you need the following tools on your local machine,

• Docker Desktop
• Redpanda CLI
• gRPC cURL

Data Streaming Platform Setup

The entire demo is containerized and all the applications could be started using the Docker compose,

docker compose up -d

The docker compose starts the following services,

• redpanda-0 - A single node Redpanda server
• console - The Redpanda console
• todo-app-server - The Todo Application gRPC server
• todo-list - The Todo Application client that receives the streaming messages from the gRPC todo-app-server

The Todo application runs with the following environment variables defaults, please change them as needed if you deploy without these defaults.

Todo gRPC Server

# gRPC service port
PORT=9090
# Redpanda Brokers
BROKERS=redpanda-0:9092
# Topic to store the Todo
TOPICS=todo-list
# Running environment, typically used for grpcurl
ENV=dev
# The consumer group used while consuming messages
CONSUMER_GROUP_ID=grpc-todo-app

Todo gRPC Client (Todo List)

SERVICE_ADDRESS=todo-app-server:9090

NOTE:

The individual application binaries for Todo App gRPC Server and Todo App Client are available on the application repo. You can download them and run the application individually.

Interact With Todo Service

List Available Services

grpcurl -plaintext "localhost:$PORT" list

Should return an output like,

grpc.reflection.v1.ServerReflection
grpc.reflection.v1alpha.ServerReflection
todo.Todo

List Service Methods

grpcurl -plaintext "localhost:$PORT" list todo.Todo

It should return the following methods,

todo.Todo.AddTodo
todo.Todo.TodoList

View List of Todo

On a new terminal run the following command to view the list of Todos added by earlier steps,

docker compose logs -f todo-list

The output should be something like,

todo-list  | 2023-12-05T05:20:19.235Z   INFO    client/main.go:44       Task    {"Title": "Finish gRPC Demo README", "Description": "Complete the README update of the gRPC Data Streaming Demo App.", "Completed": false, "Last Updated": "Thursday, 01-Jan-70 00:00:00 UTC", "Partition": 0, "Offset": 0}
todo-list  | 2023-12-05T05:20:19.236Z   INFO    client/main.go:44       Task    {"Title": "Finish gRPC Demo README", "Description": "Complete the README update of the gRPC Data Streaming Demo App.", "Completed": false, "Last Updated": "Thursday, 01-Jan-70 00:00:00 UTC", "Partition": 0, "Offset": 1}

TIP:

You can also use

grpcurl -plaintext "localhost:$PORT" todo.Todo/TodoList

Add a TODO

grpcurl -plaintext -d @ "localhost:$PORT" todo.Todo/AddTodo <<EOM
{
  "task": {
    "title": "Finish gRPC Demo README",
    "description": "Complete the README update of the gRPC Data Streaming Demo App.",
    "completed": false
  }
}
EOM

Once the task is added the terminal running the client should show an output similar to,

{
  "Title": "Finish gRPC Demo README",
  "Description": "Complete the README update of the gRPC Data Streaming Demo App.",
  "Completed": false,
  "Last Updated": "Thursday, 01-Jan-70 00:00:00 UTC",
  "Partition": 0,
  "Offset": 1
}

References

The demo uses the protobuf definitions from https://github.com/kameshsampath/demo-protos

Cleanup

docker compose down