Python Dependencies

requirements.txt

This file is needed for a Data Job that uses external Python libraries. It only applies to Data Job runs over the Cloud (Kubernetes) infrastructure. The effect is the same as executing 'pip install -r requirements.txt'. For local runs of the job, you will need to install the requirements manually in your execution virtualenv.

requirements.txt file follows a specific format that is described here: https://pip.pypa.io/en/stable/reference/pip_install/#example-requirements-file.

We recommend explicitly specifying the versions of the dependencies and using fixed versions. This makes the Data Job builds much more stable and reproducible. Do not override VDK version, all jobs are automatically running the same (maybe latest) version of VDK thus ensure better quality and reproducibility.

➡️ Next Section: Resiliency