Tests take FOREVER to run

[mattwynne]

🤔 What's the problem you've observed?

The tests take an eternity to run on a pull request. We have a huge matrix of OS and Ruby versions, and GitHub seems to take a long time to run them all.

✨ Do you have a proposal for making it better?

Not sure. Do we really have to check all Ruby versions on all platforms? Could we just check the latest Ruby on all platforms and then run older Rubies just on, say, ubuntu?

📚 Any additional context?

Well, I'm trying to do #1677 but it's taken me all evening to merge four PRs to get the release ready.

---

This text was originally generated from a template, then edited by hand. You can modify the template here.

[luke-hill]

Popping in some random stats from the last few cron days. All references are to the rake tasks, taken as an average

type	bundle	rake	cucumber	cck	examples/sinatra	examples/i18n (Estimate)
Ubuntu 2.7-3.2	5.075	1.463	1.967	11.650	0.030	31.565
Mac OS 2.7-3.2	8.163	2.910	4.666	19.848	0.052	55.863
Windows 2.7-3.2	7.335	20.83	7.178	73.343	0.070	205.245
Ubuntu JRuby	21.300	8.520	10.255	149.100	0.491	355.334
Ubuntu TruffleRuby	16.170	16.560	24.575	140.700	0.825	296.170

I think if we re-architect the examples folder to run as one homogeneous group somehow it would remove a lot of the latent cd'ing e.t.c.

Also as you can see it's basically only JRuby, Windows and Truffle that's slowing us down.