Improve modules quality by examples

[marcosnils]

Problem #

In many cases it's not clear for module consumers how to use them once they find them in the Daggerverse. Even though https://daggerverse.dev has detailed docs about the module functions, their arguments and return types, sometimes modules are designed to be consumed in a particular way which is currently hard to express given the current module documentation features.

For example, looking at this module: https://daggerverse.dev/mod/github.com/jcsirot/daggerverse/java@c591e9e0b99def2fc8a67bb090fca5cd06cf6a1d

Daggerverse.dev doesn't show anything about how this module could be used but if you navigate to the module README file here https://github.com/jcsirot/daggerverse/tree/c591e9e0b99def2fc8a67bb090fca5cd06cf6a1d/java#usage, you can find more approachable example(s) on how this module can be utilized.

The purpose of this proposal is to elevate our modules quality by enabling module authors to define examples of their existing modules and surface them in the Daggerverse.

Solution #

Similar to other package managers and indexers like npm, pkg.go.dev, docs.rs, pulumi registry, etc. the goal is to provide Dagger module authors a way to define examples for their modules and later render them in Daggerverse.dev so consumers have a better UX while using them.

The challenge is that we would like Dagger module examples to follow the same UX that other aspects of the Dagger platform have. This translates to these examples ideally being defined through working code and the ability to be consumed in any of the officially supported SDKs.

After brainstorming with Dagger team members @vito @helderco and @jedevc, we came to the conclusion that we'd like to move forward with a simple initial approach:

1. Create an examples submodule inside your module that installs your main module and write examples functions there.

Pros:

• Leverage existing tooling and straightforward implementation
• Follows a pattern that some community members are already adopting

Cons:

• Doesn't generate examples for all SDK's automatically

example:

We want to write an example for the java module mentioned above:

1 . Run dagger init --sdk go --source examples examples.
2. Run dagger -m examples install .

1 & 2 create an examples module that installs the java module.

Once that's done, process to create a test function as needed:

func (e *JavaExample) Java() *Container {
    c:= dag.Java().WithJdk("17").
        WithMaven("3.9.5").
        Container().
        WithExec([]string{"mvn", "--version"})
        if "foo" == "bar" { fmt.Println("test") } // this line won't get translated to the examples.
        return c
}

Once this module gets published in the Daggerverse, the Java function defined in the examples submodule will be rendered appropriately in the module page.

UX proposal #

Some ideas came up internally at Dagger about how we could approach this. The one that's getting the most traction is following a similar pattern how Go handles examples in their docs (https://go.dev/blog/examples) by establishing a convention on how to define them to then render them accordingly in the package docs.

The proposed UX is as follows:

Given a module M, with a type T and function F, any files under a ./examples folder in the module source directory with the conventions described below will be counted as module examples.

func M() { ... } //defines example for module M
func TF() { ... } //defines example for type T and function F 

Multiple example functions for a module/type/function may be provided by appending a distinct suffix to the name:

func M_suffix() { ... }
func TF_suffix() { ... }

Warning

• The suffix method requires some changes in the engine since we're not currently parsing underscore (_) characters in functions which are supported by the GraphQL api.
• Using Go as an example here. This applies for all the other SDK's with their standard code conventions

Next steps

We're planning to spend a few cycles on the proposed solution to get a feel about how approachable it is and hopefully come up with a working PoC to share with the community. Any other ideas or suggestions are always welcome.

[shykes]

In "solution 1", what steps does the user follow, to add an example to a module?

[shykes]

In "solution 1", can you generate a CLI example too?

[marcosnils]

In "solution 1", what steps does the user follow, to add an example to a module?

Both solution 1 and 2 steps to create examples has been added here #7217 (comment)

In "solution 1", can you generate a CLI example too?

Yes, both solutions would allow it 🙏

[marcosnils]

updated the original issue and left the current solution that's being prototyped.