Unusably slow with many data points

[Evertvdw]

Hello,

I have a question regarding zooming with large amounts of data. I currently have a graph with >20k points in it and the zooming functionality in that is pretty much useless, as it is to slow.

A possible solution I thought of is to update the points of the graph only when the dragging or zooming ends, to keep it usable. Is there a way to seperate the update of the datasets and of the axis in the charts? That way you can zoom and see what your doing on the axis and after you finish dragging call the update method of the chart.

Any suggestions here? I'm trying to make charts work with a lot of data, any help is appreciated!

Thanks!

[ericnkatz]

Probably worth having a debounce period configurable.

edit: nevermind, tried this and it was more 'performant' but perception of its performance is just as bad.

[Hadatko]

For this reason we switched to use: http://dygraphs.com/
Here it was extremly slow for around 8000points on mobile.
Mobile could handle around 20000 points now.

[Evertvdw]

I figured out some method to increase the performance for chart-js. All of the points are always rendered, even if they are not visible (e.a. outside of the current view). I made a plugin that only puts the data that is currently visible into the datasets and keep a seperate object with all the data.

Here is the plugin I made (for time series only). How to use it: create a 'allData' array in your chart data object in which you nest one array for each dataset. Put all the data in this array (best to use an array of objects like {x: .., y: ..}) and only the data will be put in the actual dataset that is visible.

Feel free to adapt it to other use cases. I used this for updating graphs to be able to keep some sort of history to which you can pan, but not keep this extra data lagging my graph.

// Keep the real data in a seperate object called allData
// Put only that part of allData in the dataset to optimize zoom/pan performance
// Author: Evert van der Weit - 2018

function filterData(chartInstance) {
    var datasets = chartInstance.data.datasets;
    var originalDatasets = chartInstance.data.allData;
    var chartOptions = chartInstance.options.scales.xAxes[0];

    var startX = chartOptions.time.min
    var endX = chartOptions.time.max

    for(var i = 0; i < originalDatasets.length; i++) {
        var dataset = datasets[i];
        var originalData = originalDatasets[i];
        
        if (!originalData.length) break

        var s = startX;
        var e = endX;
        var sI = null;
        var eI = null;

        for (var j = 0; j < originalData.length; j++) {
            if ((sI==null) && originalData[j].x > s) {
                sI = j
            }
            if ((eI==null) && originalData[j].x > e) {
                eI = j
            }
        }
        if (sI==null) sI = 0
        if (originalData[originalData.length - 1].x < s) eI = 0
            else if (eI==null) eI = originalData.length

        dataset.data = originalData.slice(sI, eI)
    }
}

var datafilterPlugin = {
    beforeUpdate: function(chartInstance) {
        filterData(chartInstance)
    }
}

module.exports = datafilterPlugin;

[benmccann]

@Evertvdw do you think there's a performance bug in the main Chart.js repo related to the handling of points not being rendered? If so, it would be great to file a bug there with details

[veggiesaurus]

@benmccann There is definitely a bug in the main Charts.js repo. Current approach is to fade out clipped points by applying an alpha, rather than not rendering them:

if ((chartArea !== undefined) && ((model.x < chartArea.left) || (chartArea.right * errMargin < model.x) || (model.y < chartArea.top) || (chartArea.bottom * errMargin < model.y))) {
	// Point fade out
	if (model.x < chartArea.left) {
		ratio = (x - model.x) / (chartArea.left - model.x);
	} else if (chartArea.right * errMargin < model.x) {
		ratio = (model.x - x) / (model.x - chartArea.right);
	} else if (model.y < chartArea.top) {
		ratio = (y - model.y) / (chartArea.top - model.y);
	} else if (chartArea.bottom * errMargin < model.y) {
		ratio = (model.y - y) / (model.y - chartArea.bottom);
	}
	ratio = Math.round(ratio * 100) / 100;
	ctx.strokeStyle = color(ctx.strokeStyle).alpha(ratio).rgbString();
	ctx.fillStyle = color(ctx.fillStyle).alpha(ratio).rgbString();
}

helpers.canvas.drawPoint(ctx, pointStyle, radius, x, y);

The last three lines are an unnecessary expense for points with ratio==0 (all points outside the chart area). I have added a PR here: chartjs/Chart.js#5363

[etimberg]

This looks like an issue mostly in the main chart.js code. Now that Chart.js is at v3, there are more solutions to this:

1. Data decimation
2. V3 has a rewritten animation system that should be faster
3. Points outside the chart area are not drawn

[Pecacheu]

@Evertvdw Just to let you know I made a Chart.js plugin inspired by your code which solves the performance problems, which might help in case anyone else is having this issue (which still happens on the latest Chart.js and plugin version). https://github.com/Pecacheu/chartjs-filter-plugin