Coding Standard

• REST API and Services
• Java
• Python
  • Clarification on access modifiers in python
  • Public Python interfaces
• Helm Charts
• Development Status
• CLI (Command Line interfaces)
• Error handling
• README files

We aim for:

• Code maintainability
• Low coupling and high cohesion
• Simplicity (don't overengineer)

VDK coding styles aim for code style and formatting consistent across its components. The value of consistency is enabling automated formatting, avoiding back and forth reformatting, and making the code easy to read for all contributors.

REST API and Services

We follow 12 factor app recommendation for building web/api services. Make sure you have read and are familiarized with the document.

The rules outlined in https://opensource.zalando.com/restful-api-guidelines/ are recommended read and nice to follow but not a must.

Java

We follow Google Java Coding style and it is enforced by a pre-commit hook.

Python

The coding standard is the Python regular PEP 8. It's enforced by pre-commit hooks like black.

Clarification on access modifiers in python

Python uses _ (underscore) symbol to determine the access control for a specific data member or a member function of a class

• Public methods have not underscore prefix
• Protected methods or attributes have single underscore as prefix - for example _execute_protected()
• Private methods or attributes have double underscore as prefix - for example __execute_private()

Public Python interfaces

Any backwards compatibility guarantees apply only to public interfaces. Public interfaces are modules and packages defined or imported in vdk.api.*. unless the documentation explicitly declares them to be provisional or internal interfaces. Anything else is considered internal. All public interfaces (classes or methods) must have documentation. The documentation must specify clearly:

• What is the purpose of the method/class
• What are the possible effects and side effects
• For each argument what are the preconditions
• Example usages

Anything not in vdk.api.* is considered internal, with exceptions made fo modules under vdk.internal.core which may be used in plugin development only.

Deprecations

This policy applies to all public-facing APIs, modules, classes, methods, and any other public functionalities.

• Versioning: Packages (and plugins) follow semantic versioning (MAJOR.MINOR.PATCH)
• Announcement: Deprecation notices must be included in both the application's changelog and documentation.
• Alternative Features: Always provide information about alternative methods, classes, or modules if available.
• Time Frame: The old version and new version must exist in the same time so users can migrate (feature flags can be helpful here).
• Warnings: Use Python's warnings standard library to issue a DeprecationWarning during runtime.

import warnings

def deprecated_function():
    warnings.warn("deprecated_function is deprecated and will be removed in version 2.0.0. Use new_function instead.", DeprecationWarning)

• Deprecation Review: Before final removal, review the usage statistics if available. Delay removal if a large portion of the user base still relies on the deprecated feature.

Helm Charts

Versatile Data Kit Helm Charts aim to be part of Bitnami Application Catalog. We try to follow (where feasible) best practices established by Bitnami and reuse best practices, charts, and ideas from bitnami charts repository

Development Status

Each public python distribution (for example a vdk plugin) should be classified based on its development status.

• In setup.py (or setup.cfg) define classifier "Development Status" as defined in https://pypi.org/classifiers.
• The semantics of the "Development Status" classifier are same as one defined in https://martin-thoma.com/software-development-stages

We use the same semantics for the development status of nonpython releases as well.

CLI (Command Line interfaces)

CLI is built following 12 Factor CLI Apps. Make sure you have read and are familiarized with the document.

Summarized those are:

1. Great help is essential. Every command and option should have detailed help and examples.
2. Prefer flags (key/value input) to arguments (positional input)
3. Version option/comand
4. Stdout is for data and output and stderr is for logs and messages!
5. Make errors informative. Basically follow VDK error guidelines
6. Be fancy if you can
7. Prompt if you can
8. Use tables but allow output in csv or json
9. Be speedy (< 1 second respond time!)
10. Encourage contributions
11. Be clear about subcommands
12. Follow XDG-spec for config or data files.

Error handling

Errors should not explain our (the developer) problem. Explain their (the user, and sometimes the caller) problem, and provide info valuable for THEM to understand what actions to take.

To ensure efficient communication and effective troubleshooting, all user-facing errors, warnings, and alerts must include clear information on:

• What the system was trying to do when the error occurred (avoiding overly technical language)
• Why the error might have happened
• The potential impact (consequences) on the user or system
• And any recommended countermeasures that users can take to recover from the error or to prevent the error from recurring.

See also Error handling format which explains in more detail and proposes a format that aims to help write clearer and more consistent error messages.

README files

Each VDK project or plugin has a README file that resides in its root folder and is named README.md. Each independentally releasable sub-component (e.g plugin, job-builder, job-base image) must have a README file as well.

A README file is written in Markdown.

A README file of a project or a plugin should contain (whichever is applicable):

• what the project is about (e.g. its purpose);
• how to install the project (e.g. how to install and configure its dependencies);
• how to configure the project (e.g. where the configuration file resides, and what each configuration property means);
• how to use it (e.g. what command to execute and examples).
• how to build (and test) the project (e.g. how to compile its source code); (note: needed only if there's no dedicated contributing file for the project or plugin)