Snap-Telemetry Plugin for Amazon Web Services (AWS) Simple Queue Service (SQS)

Snap-Telemetry Plugin for AWS SQS sends metric values to AWS SQS.

1. Getting Started

• System Requirements
• Installation

2. Documentation

• Configuration and Usage
• Publisher Output

2. Issues and Roadmap
3. Acknowledgements

Getting Started

Read the system requirements, supported platforms, and installation guide for obtaining and using this Snap plugin.

System Requirements

• golang 1.7+ (needed only for building)

Operating systems

All OSs currently supported by snap:

• Linux/amd64
• Darwin/amd64

Installation

The following sections provide a guide for obtaining the plugin. The plugin is written in Go. Make sure you follow the guide for setting up your Go workspace.

Download

The simplest approach is to use go get to fetch and build the plugin. The following command will place the binary in your $GOPATH/bin folder where you can load it into snap.

$ go get github.com/opsvision/snap-plugin-publisher-awssqs

Building

The following provides instructions for building the plugin yourself if you decided to downlaod the source. We assume you already have a $GOPATH setup for golang development. The plugin utilizes glide for library management.

$ mkdir -p $GOPATH/src/github.com/opsvision
$ cd $GOPATH/src/github.com/opsvision
$ git clone http://github.com/opsvision/snap-plugin-publisher-awssqs
$ glide up
[INFO]	Downloading dependencies. Please wait...
[INFO]	--> Fetching updates for ...
[INFO]	Resolving imports
[INFO]	--> Fetching updates for ...
[INFO]	Downloading dependencies. Please wait...
[INFO]	Setting references for remaining imports
[INFO]	Exporting resolved dependencies...
[INFO]	--> Exporting ...
[INFO]	Replacing existing vendor dependencies
[INFO]	Project relies on ... dependencies.
$ go install

Source structure

The following file structure provides an overview of where the files exist in the source tree.

snap-plugin-publisher-awssqs
├── awssqs
│   └── awssqs.go
├── glide.yaml
├── LICENSE
├── main.go
├── metadata.yml
├── README.md
├── scripts
│   ├── load.sh
│   └── unload.sh
└── tasks
    └── awssqs.yaml

Documentation

Configuration and Usage

Set up the Snap framework

Load the Plugin

Once the framework is up and running, you can load the plugin.

$ snaptel plugin load snap-plugin-publisher-awssqs
Plugin loaded
Name: awssqs
Version: 1
Type: publisher
Signed: false
Loaded Time: Tue, 24 Jan 2017 20:45:48 UTC

Task File

You need to create or update a task file to use the AWS SQS publisher plugin. We have provided an example, tasks/awssqs.yaml shown below. In our example, we utilize the psutil collector so we have some data to work with. There are four (4) configuration settings you can use.

Setting	Description	Required?
debug_file	An absolute path to a log file - this makes debugging easier.	No
akid	The Amazon API Key ID	Yes
secret	The Amazon Secret	Yes
queue	The Amazon SQS URL; you can follow this tutorial for setting up SQS.	Yes

Note: The Region, required by AWS, is extrapolated from the queue URL.

---
  version: 1
  schedule:
    type: "simple"
    interval: "5s"
  max-failures: 10
  workflow:
    collect:
      config:
      metrics:
        /intel/psutil/load/load1: {} 
        /intel/psutil/load/load15: {}
        /intel/psutil/load/load5: {}
        /intel/psutil/vm/available: {}
        /intel/psutil/vm/free: {}
        /intel/psutil/vm/used: {}
      publish:
        - plugin_name: "awssqs"
          config:
            debug_file: "/tmp/awssqs-debug.log"
            akid: "1234ABCD"
            secret: "1234ABCD"
            queue: "https://sqs.us-east-1.amazonaws.com/208379614050/sqs_demo"

Once the task file has been created, you can create and watch the task.

$ snaptel task create -t awssqs.yaml
Using task manifest to create task
Task created
ID: ad6d5e24-a280-4fa4-85d5-6b7795739f90
Name: Task-ad6d5e24-a280-4fa4-85d5-6b7795739f90
State: Running

$ snaptel task list
ID                                       NAME                                         STATE     ...
ad6d5e24-a280-4fa4-85d5-6b7795739f90     Task-ad6d5e24-a280-4fa4-85d5-6b7795739f90    Running   ...

Note: Truncated results for brevity.

Publisher Output

The AWS SQS publisher plugin sends a JSON string to the queue with six (6) attributes shown below.

{
  "hostname":"localhost",
  "metric":"intel.psutil.load.load15",
  "plugin":"awssqs",
  "timestamp":"2017-01-25T13:17:39Z",
  "type":"float64",
  "value":"0.05"
}

Issues and Roadmap

• Testing: The testing being done is rudimentary at best. Need to improve the testing.

Note: Please let me know if you find a bug or have feedbck on how to improve the collector.

Acknowledgements

• Author: @dishmael
• Company: OpsVision Solutions