Project motivations
You must have heard about mono-repos by now. It’s all the rage.
A Monorepo is when you put ALL of your enterprise code, regardless of the language, in a single meticulously organized single repository, and power it with a military-grade build system, and in some cases — military grade virtual filesystem.
Google is doing it, AirBnB is doing it, Twitter, Facebook, — the list goes on. But, does it really make sense to put all of your codebase in "one basket" so to speak? And even so, wouldn't that mean building this giant repo is a gargantuan task?
Yes and no. You see, over the last decade several highly successful internet companies adopted Ruby as the de-facto language for building enterprise components and services. Companies in this list include Github, AirBnB, Coinbase, SlideShare, Zendesk, SoundCloud, Groupon, Kickstarter, Scribd — the list goes on. As these companies became more and more successful, their technology stack became more and more diversified, complex, decoupled, and distributed.
It was during this hyper growth phase that many of the brightest minds in the industry (admittedly, starting with Google) came to the same conclusion that the architectural decoupling of software components, while highly desired as a software design concept, is not at all synonymous with a concept of a poly-repo — a practice of keeping small units of code, — think Ruby gems, — in separate repos. The way you manage your source control can either stand in a way of developer productivity, or serve as a tremendous accelerator.
It was then that Google had publicly pioneered the concept of a mono-repo, and slowly but surely it spread to some of the companies in our Ruby Giants list above. It turns out that keeping all source code together in what is now known as a mono-repo offers several unique benefits.
Without spending too much time on this, I highly recommend you read both of those pieces:
-
Monorepos: Please don’t! by Matt Klein from Lyft Engineering, and
-
Monorepo: please do! by Adam Jacobs, former CTO of Chef.
For one — with service meshes consisting of hundreds or even thousands of polyglot services, it becomes increasingly challenging to release new features that require changes across wide range of co-dependent poly-repos.
To properly review such a feature, an engineer must review several pull requests across multiple repositories, understand the cross-dependencies, catch any possible bugs that may arise from the new service boundary interactions.
Finally, the original engineer, having received all of the approvals on their pull request, must carefully orchestrate the deployment sequence as to not end up with the new code in production that’s relying on a yet-to-be deployed downstream service!
You can probably imagine that in this world no amount of safeguards, reviews, and canary deploy systems can help avoid issues altogether. And the folks that I spoke to, from those companies, pretty much said so.
Here is what Bazel team themselves say about Bazel:
Bazel only rebuilds what is necessary. With advanced local and distributed caching, optimized dependency analysis and parallel execution, you get fast and incremental builds.
Build and test Java, C++, Android, iOS, Go and a wide variety of other language platforms. Bazel runs on Windows, macOS, and Linux.
Bazel helps you scale your organization, codebase and Continuous Integration system. It handles codebases of any size, in multiple repositories or a huge monorepo.
Easily add support for new languages and platforms with Bazel's familiar extension language. Share and re-use language rules written by the growing Bazel community.
Enter Bazel: one exceedingly powerful build system — Bazel, itself written in Java, Go and offering a scripting language called Starlark (a subset of Python) whose interpreter is written in Go. But why not? 🥇
Bazel comes with native support for C++, Go, Python, with support for NodeJS and Angular added later.
And if you dig around you'll find support for some really obscure languages, such as D, Haskell, and Closure.
Ruby support has been notably lacking. If you take a look at the Bazel github organization, you might notice that Ruby is barely mentioned, with the only one link to the Ruby Rules by Yugui who created them back a couple years ago as a personal hobby project.
Luckily for the ruby community she agreed to take part in this project, and so now we are collaborating and hoping to make Ruby Rules Production Ready by February 2020.
Written by: Konstantin Gredeskoul