Skip to content

Commit

Permalink
Added %timefilter% min/max,autointerval,bump
Browse files Browse the repository at this point in the history
  • Loading branch information
nyurik committed Dec 1, 2017
1 parent 1436e92 commit 417b06c
Show file tree
Hide file tree
Showing 5 changed files with 212 additions and 59 deletions.
30 changes: 18 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ This plugin will automatically inject a projection called `"projection"`. Use it
# Querying ElasticSearch
By default, Vega's [data](https://vega.github.io/vega/docs/data/) element can use embedded and external data with a `"url"` parameter. Kibana plugin adds support for the direct ElasticSearch queries by overloading the "url"` value.

Here is an example of an ES query that gets data from `logstash-*` index, filtering by
Here is an example of an ES query that gets data from `logstash-*` index.

```yaml
{
Expand All @@ -75,9 +75,17 @@ Here is an example of an ES query that gets data from `logstash-*` index, filter
"hist": {
"date_histogram": {
"field": "@timestamp",
"interval": "2h",
"time_zone": "America/New_York",
"min_doc_count": 1
// interval value will depend on the daterange picker
// Use an integer to set approximate bucket count
"interval": {"%autointerval%": true},
// Make sure we get an entire range, even if it has no data
"extended_bounds": {
"min": {"%timefilter%": "min"},
"max": {"%timefilter%": "max"}
},
// Use this for linear (e.g. line, area) graphs
// Without it, empty buckets will not show up
"min_doc_count": 0
}
}
}
Expand All @@ -86,11 +94,7 @@ Here is an example of an ES query that gets data from `logstash-*` index, filter

// This is a useful trick to access just the list of aggregation results named "hist"
//
"format": {
"type": "json",
"property": "aggregations.hist.buckets"
},

"format": { "property": "aggregations.hist.buckets" },
}
],
...
Expand Down Expand Up @@ -138,11 +142,11 @@ Query may be specified with individual range and dashboard context as well. This
// "%timefilter%" will be replaced with the current
// values of the time filter (from the upper right corner)
"%timefilter%": true

// Only work with %timefilter%
// Shift the current timefilter by 10 units back
"shift": 10,

// supports week, day (default), hour, minute, second.
"unit": "minute"
}
Expand All @@ -157,6 +161,8 @@ Query may be specified with individual range and dashboard context as well. This
...
```

The `"%timefilter%"` can also be used to specify a single min or max value. As shown above, the date_histogram's `extended_bounds` can be set with two values - min and max. Instead of hardcoding a value, you may use `"min": {"%timefilter%": "min"}`, which will be replaced with the begining of the current time range. The `shift` and `unit` values are also supported. The `"interval"` can also be set dynamically, depending on the currently picked range: `"interval": {"%autointerval%": 10}` will try to get about 10-15 datapoints (buckets).


# Vega vs VegaLite
VegaLite is a simplified version of Vega, useful to quickly get started, but has a number of limitations. VegaLite is automatically converted into Vega before rendering. Compare [logstash-simple_line-vega](public/examples/logstash/logstash-simple_line-vega.json) and [logstash-simple_line-vegalite](public/examples/logstash/logstash-simple_line-vegalite.json) (both use the same ElasticSearch logstash data). You may use [this editor](https://vega.github.io/editor/) to convert VegaLite into Vega.
Expand All @@ -166,7 +172,7 @@ VegaLite is a simplified version of Vega, useful to quickly get started, but has
## Browser Debugging console
Use browser debugging tools (e.g. F12 or Ctrl+Shift+J in Chrome) to inspect the `VEGA_DEBUG` variable:
* `view` - access to the Vega View object. See [Vega Debugging Guide](https://vega.github.io/vega/docs/api/debugging/) on how to inspect data and signals at runtime. For VegaLite, `VEGA_DEBUG.view.data('source_0')` would get the main dataset. For Vega, it uses the data name as defined in your Vega spec.
* `spec` - Vega JSON specification after some modifications by this plugin. In case of VegaLite, this is the output of the VegaLite compiler.
* `spec` - Vega JSON specification after some modifications by this plugin. In case of VegaLite, this is the output of the VegaLite compiler.
* `vlspec` - If this is a VegaLite graph, JSON specification of the graph before VegaLite compilation.

## Data
Expand Down
6 changes: 3 additions & 3 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "vega_vis",
"version": "0.4.2",
"version": "0.5.0",
"description": "Build vega visualizations into Kibana",
"main": "index.js",
"kibana": {
Expand Down Expand Up @@ -49,8 +49,8 @@
"leaflet-vega": "^0.5.3",
"semver": "^5.4.1",
"split.js": "^1.3.5",
"vega": "^3.0.7",
"vega-lite": "^2.0.0",
"vega": "^3.0.8",
"vega-lite": "^2.0.2",
"vega-schema-url-parser": "^1.0.0-beta.3"
}
}
73 changes: 65 additions & 8 deletions public/default.spec.json
Original file line number Diff line number Diff line change
@@ -1,22 +1,79 @@
{
// Use the "reformat" buttons above to remove comments

"$schema": "https://vega.github.io/schema/vega-lite/v2.json",
"title": "Tesla Motors Daily Stock Price",
"title": "Event counts from all indexes",

// Define the data source
"data": {
// A CSV file with the Tesla stock prices, from
// https://www.quandl.com/api/v3/datasets/WIKI/TSLA.csv
"url": "https://gist.githubusercontent.com/nyurik/2419ec5f0edf34852ab62b1bc0e9ad70/raw/8b09ff7084a2f930e0130c1391858d62ae703d16/TSLA.csv"
// Specifying object as url takes the data from the ES
// This query counts the number of documents per time interval,
// assuming you have a @timestamp field in your data
"url": {
// Which indexes to search
"index": "_all",
// Connect dashboard context (the date range and filters)
"%context_query%": "@timestamp",
// The body element may contain "aggs" and "query" subfields
"body": {
"aggs": {
"time_buckets": {
"date_histogram": {
// Use date histogram aggregation on @timestamp field
"field": "@timestamp",
// interval value will depend on the daterange picker
// Use an integer to set approximate bucket count
"interval": {"%autointerval%": true},
// Make sure we get an entire range, even if it has no data
"extended_bounds": {
"min": {"%timefilter%": "min"},
"max": {"%timefilter%": "max"}
},
// Use this for linear (e.g. line, area) graphs
// Without it, empty buckets will not show up
"min_doc_count": 0
}
}
}
},
// Speed up the response by only including aggregation results
"size": 0
},
// The full ES result has this structure, but for this graph we only need
// the list of bucket values, so use format.property:
// {
// "aggregations": {
// "time_buckets": {
// "buckets": [{
// "key_as_string": "2015-11-30T22:00:00.000Z",
// "key": 1448920800000,
// "doc_count": 0
// }, {
// "key_as_string": "2015-11-30T23:00:00.000Z",
// "key": 1448924400000,
// "doc_count": 0
// }, ...
"format": {"property": "aggregations.time_buckets.buckets"}
},

// Define how data should be shown
// Other mark values: area, bar, circle, line, point, rect, rule, square, text, and tick
// See https://vega.github.io/vega-lite/docs/mark.html
"mark": "line",

// See https://vega.github.io/vega-lite/docs/encoding.html
"encoding": {
"x": {
"field": "Date",
// The "key" value is the timestamp in milliseconds. Use it for X axis.
"field": "key",
"type": "temporal",
"axis": {"title": false}
"axis": {"title": false} // Customize X axis format
},
"y": {
"field": "Close",
// The "doc_count" is the count per bucket. Use it for Y axis.
"field": "doc_count",
"type": "quantitative",
"axis": {"title": "Closing Price"}
"axis": {"title": "Document count"}
}
}
}

0 comments on commit 417b06c

Please sign in to comment.