Documentation for limayaml

[Hefeweizen]

This comment is about SSH port forwards, specifically, but all of lima.yaml generally.

I started up two k3s instances simultaneously. They both wanted to port-forward 6443. However, with this conflict, only one was able to do so. I resolved this by changing the port-forwarding options on the second VM:

portForwards:
- guestPort: 6443
  hostPort: 16443

In order to accomplish this, I had to read through limayaml's default.go and limayaml.go. It'd be nice if this were documented in a better form.

I'm happy to help with this. However, I wanted to discuss the form this should take.

godoc is the communities preference for technical, api-focused documentation. I'm not sure if there's something better to create end-user documentation. A static markdown doc could get this work started, but I'm not sure of the appetite for keeping a separate file up-to-date.

[afbjorklund]

Maybe json schema could be used (for documenting yaml), both for validation and visualization?

https://json-schema-everywhere.github.io/yaml

It has some nice tools, like: https://github.com/coveooss/json-schema-for-humans

[afbjorklund]

We could provide a basic jsonschema for validation, similar to what is available for cloud-config...

• Generated user-data is failing cloud-config schema validation #2265

But it will never have the same level of documentation and features, that is in the custom solution.

At least not without "extending" the official https://json-schema.org/ with additional features ?

---

Typically you want to generate the json schema, since it is much to tedious to maintain by hand.

Something like https://github.com/perenecabuto/json_schema_generator, but in the go code

[afbjorklund]

Here was the code, for the test above:

lima.json

{
   "definitions" : {},
   "properties" : {
      "cpus" : {
         "description": "CPUs",
	 "default" : null,
         "type" : [
            "integer",
            "null"
         ]
      },
      "disk" : {
         "description": "Disk size",
	 "default" : null,
         "type" : [
            "string",
            "null"
         ]
      },
      "memory" : {
         "description": "Memory size",
         "default" : null,
         "type" : [
            "string",
            "null"
         ]
      },
      "images" : {
         "type" : "array"
      }
   },
   "required" : [],
   "type" : "object"
}

I also experimented with translating to XML, just so that I could use XSL to present something nice...

Apparently that feature is not available now, so nothing like https://apps.0install.net/gui/vs-code.xml

[jandubois]

builtin defaults should be in the documentation schema, instead of "null"

The defaults (in the templates) need to be null for the override.yaml and default.yaml mechanisms to work. I would veto any changes that break these mechanisms (unless we have a superior replacement, and a transition path).

[afbjorklund]

I found some code that would generate a jsonschema. It works OK, but you have to tell it explicitly to not dereference pointers.
It actually found some bugs in both the limayaml and default.yaml, but it is all about yaml validation and not documentation...

[afbjorklund]

• Add command to generate jsonschema for limayaml #2306

[jandubois]

In order to accomplish this, I had to read through limayaml's default.go and limayaml.go.

I'm not sure why you had to read the source code for this. Isn't the documentation in default.yaml not enough?

lima/examples/default.yaml

Lines 303 to 348 in b03835d

	# Port forwarding rules. Forwarding between ports 22 and ssh.localPort cannot be overridden.
	# Rules are checked sequentially until the first one matches.
	# portForwards:
	# - guestPort: 443
	# hostIP: "0.0.0.0" # overrides the default value "127.0.0.1"; allows privileged port forwarding
	# # default: hostPort: 443 (same as guestPort)
	# # default: guestIP: "127.0.0.1" (also matches bind addresses "0.0.0.0", "::", and "::1")
	# # default: proto: "tcp" (only valid value right now)
	#
	# - guestPortRange: [4000, 4999]
	# hostIP: "0.0.0.0" # overrides the default value "127.0.0.1"
	# # default: hostPortRange: [4000, 4999] (must specify same number of ports as guestPortRange)
	#
	# - guestPort: 80
	# hostPort: 8080 # overrides the default value 80
	#
	# - guestIP: "127.0.0.2" # overrides the default value "127.0.0.1"
	# hostIP: "127.0.0.2" # overrides the default value "127.0.0.1"
	# # default: guestPortRange: [1, 65535]
	# # default: hostPortRange: [1, 65535]
	#
	# - guestPort: 8888
	# ignore: true (don't forward this port)
	#
	# - guestPort: 7443
	# guestIP: "0.0.0.0" # Will match *any* interface
	# guestIPMustBeZero: true # Restrict matching to 0.0.0.0 binds only
	# hostIP: "0.0.0.0" # Forwards to 0.0.0.0, exposing it externally
	#
	# - guestSocket: "/run/user/{{.UID}}/my.sock"
	# hostSocket: mysocket
	# # default: reverse: false
	# # "guestSocket" can include these template variables: {{.Home}}, {{.UID}}, and {{.User}}.
	# # "hostSocket" can include {{.Home}}, {{.Dir}}, {{.Name}}, {{.UID}}, and {{.User}}.
	# # "reverse" can only be used for unix sockets right now, not for tcp sockets.
	# # Put sockets into "{{.Dir}}/sock" to avoid collision with Lima internal sockets!
	# # Sockets can also be forwarded to ports and vice versa, but not to/from a range of ports.
	# # Forwarding requires the lima user to have rw access to the "guestsocket",
	# # and the local user rwx access to the directory of the "hostsocket".
	#
	# # Lima internally appends this fallback rule at the end:
	# - guestIP: "127.0.0.1"
	# guestPortRange: [1, 65535]
	# hostIP: "127.0.0.1"
	# hostPortRange: [1, 65535]
	# # Any port still not matched by a rule will not be forwarded (ignored)

This can certainly be improved, but I don't think having a separate file documenting this would really help much, and would be harder to keep up-to-date.

Or were you not aware that the settings are documented in default.yaml? In that case we should maybe put a comment into all the other templates that full documentation for the schema can be found in the default template?

[Hefeweizen]

if that's the canonical source, great! Then a pointer or two would be helpful.

The beginning of my search was on the command line, seeing that port 6443 was owned by ssh: lsof -Pi4 | grep 6443.

I tried google from there: "lima ssh config" and some variations.
Mostly that pointed me `~/.lima//ssh.config and other generic 'use ssh to connect' style help.

Github search on this repo was next. "forward" as a search term points to the perl test script first. Then, libyaml.go and defaults.go appear. examples/default.yaml is only on the second page of results, and it starts with X11 and agent forward keywords; but then, it summarizes to only show top matches to limit space.

I like the suggestion of commenting in all the other templates. I'd also suggest a line item in the main README.md.

Also: thanks for the responsiveness!

[Hefeweizen]

Thinking out loud, a troubleshooting.md doc would be nice. My use case would seem common: start up two identical VMs to compare apps. identical VMs will have port conflicts so a quick q&a on how to set ports would be helpful.

[jandubois]

Maybe start a PR and wait for feedback?

Also feel free to create a PR to add the reference from all other templates back to default.yaml, but maybe get agreement on wording first.