Create eLabFTW Terraform provider

[din14970]

In the context of my previous post I wrote a Python library that wraps elabapi-python for declaratively defining templates, items and other eLabFTW resources. The idea behind it was that I could define resources using YAML files, which I could then "apply" to the instance. Instead of modifying things on the fly either with the API or through the UI, a declarative approach to managing resources in eLabFTW would allow an administrator or user to manage system state through git and CI/CD. The main reason I wanted this is because the team I was working with could not decide on their data model which made making changes very cumbersome.

As I was developing the library, the more I discovered I was reinventing terraform in Python (though using YAML as config language instead of HCL). This precipitated in me the idea that perhaps it would be a better idea to shift my efforts to writing a terraform provider for eLabFTW. These days many popular services with an API have terraform providers, which allows platform engineers to not only manage cloud resources declaritively, but also the configuration/state of the services inside the same code base. I'm also sure that with a terraform provider, eLabFTW would be considered much more seriously by engineering/ops teams.

Terraform providers are stand-alone executables written in golang that act as an intermediary between terraform itself on the one hand and the API client on the other hand. Terraform talks to it through grpc, and the provider makes the API calls through the client library. Since eLabFTW has an OpenAPI specification, it is trivial to generate a Go client SDK (I tried it locally following the template in the elabapi-python repo). Writing a provider would involve wrapping this SDK according to the guide. It is something I would like to take a stab at, but I must admit I have 0 experience with golang so it will also be a learning process. I have heard that golang is an easy language which makes me hopeful I can figure it out; however it will take some time.

I wanted to post this here both to get feedback on the idea and see if anyone wants to contribute to the project. I was thinking to first create the repos (provider and go client) under my personal github account and if you see value in supporting it "officially" to move them under the elabftw organization.

[NicolasCARPi]

I think it's a good idea. We evoked already internally the idea of an ansible plugin to declare things for instance. Wouldn't an ansible plugin make more sense than a terraform one? Given that terraform is more for resources in cloud provider and ansible is more about service configuration provisioning.

[din14970]

There is some overlap in what the tooling can do but in my opinion Ansible on the spectrum from declarative to imperative is more on the imperative side. You make playbooks, consisting of a sequence of steps, which are an abstraction on top of scripts. Ideally steps are idempotent so that in effect you create a reproducible state even if you re-run the same playbook. Also, if implemented, there is some functionality for undoing a playbook. However, Ansible also makes it easy to run non-idempotent steps. The order of steps in a playbook are usually important.

By contrast, Terraform providers take care of the order of operations. All you have to do in terraform code is declare the resources you would like provisioned. This is not only for cloud resources but can be anything within an application. In the case of eLabFTW this could be item types, templates, items, experiments, users, ... .

Personally I use Ansible to manage servers where all I have is SSH access and I need to interact with files. For services with an API I would always prefer Terraform.

[jat255]

@din14970 I would certainly be interested in this and perhaps contributing (although I know little about terraform and ansible). We have been working with a research group here on expressing their templates in a simplified JSONSchema representation, which allows for validation of the metadata entered by users. It also allows them to version their template structures with more detail. I was hoping eventually to write a converter that could automatically transform from our JSONSchema to eLabFTW template definitions automatically, but that would be in the future.