From 0c51dea7773afcaa7dbe4ba9b35acd87e6e3463d Mon Sep 17 00:00:00 2001 From: Derek Nola Date: Thu, 16 Nov 2023 11:56:47 -0800 Subject: [PATCH 1/3] Move Advanced OS preop into requirements Signed-off-by: Derek Nola --- docs/advanced/advanced.md | 69 ------------------------------- docs/faq/faq.md | 2 +- docs/installation/requirements.md | 69 +++++++++++++++++++++++++++++-- docs/known-issues/known-issues.md | 10 ++++- docs/quick-start/quick-start.md | 2 + 5 files changed, 76 insertions(+), 76 deletions(-) diff --git a/docs/advanced/advanced.md b/docs/advanced/advanced.md index bbe4bb811..2a88b8618 100644 --- a/docs/advanced/advanced.md +++ b/docs/advanced/advanced.md @@ -280,75 +280,6 @@ An example of disabling auto-starting and service enablement with the install sc curl -sfL https://get.k3s.io | INSTALL_K3S_SKIP_START=true INSTALL_K3S_SKIP_ENABLE=true sh - ``` -## Additional OS Preparations - -### Old iptables versions - -Several popular Linux distributions ship a version of iptables that contain a bug which causes the accumulation of duplicate rules, which negatively affects the performance and stability of the node. See [Issue #3117](https://github.com/k3s-io/k3s/issues/3117) for information on how to determine if you are affected by this problem. - -K3s includes a working version of iptables (v1.8.8) which functions properly. You can tell K3s to use its bundled version of iptables by starting K3s with the `--prefer-bundled-bin` option, or by uninstalling the iptables/nftables packages from your operating system. - -:::info Version Gate - -The `--prefer-bundled-bin` flag is available starting with the 2022-12 releases (v1.26.0+k3s1, v1.25.5+k3s1, v1.24.9+k3s1, v1.23.15+k3s1). - -::: - -### Red Hat Enterprise Linux / CentOS / Fedora - -It is recommended to turn off firewalld: -```bash -systemctl disable firewalld --now -``` - -If you wish to keep firewalld enabled, by default, the following rules are required: -```bash -firewall-cmd --permanent --add-port=6443/tcp #apiserver -firewall-cmd --permanent --zone=trusted --add-source=10.42.0.0/16 #pods -firewall-cmd --permanent --zone=trusted --add-source=10.43.0.0/16 #services -firewall-cmd --reload -``` - -Additional ports may need to be opened depending on your setup. See [Inbound Rules](../installation/requirements.md#inbound-rules-for-k3s-server-nodes) for more information. If you change the default CIDR for pods or services, you will need to update the firewall rules accordingly. - -If enabled, it is required to disable nm-cloud-setup and reboot the node: -```bash -systemctl disable nm-cloud-setup.service nm-cloud-setup.timer -reboot -``` - -### Ubuntu / Debian - -It is recommended to turn off ufw (uncomplicated firewall): -```bash -ufw disable -``` - -If you wish to keep ufw enabled, by default, the following rules are required: -```bash -ufw allow 6443/tcp #apiserver -ufw allow from 10.42.0.0/16 to any #pods -ufw allow from 10.43.0.0/16 to any #services -``` - -Additional ports may need to be opened depending on your setup. See [Inbound Rules](../installation/requirements.md#inbound-rules-for-k3s-server-nodes) for more information. If you change the default CIDR for pods or services, you will need to update the firewall rules accordingly. - -### Raspberry Pi - -Raspberry Pi OS is Debian based, and may suffer from an old iptables version. See [workarounds](#old-iptables-versions). - -Standard Raspberry Pi OS installations do not start with `cgroups` enabled. **K3S** needs `cgroups` to start the systemd service. `cgroups`can be enabled by appending `cgroup_memory=1 cgroup_enable=memory` to `/boot/cmdline.txt`. - -Example cmdline.txt: -``` -console=serial0,115200 console=tty1 root=PARTUUID=58b06195-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait cgroup_memory=1 cgroup_enable=memory -``` - -Starting with Ubuntu 21.10, vxlan support on Raspberry Pi has been moved into a separate kernel module. -```bash -sudo apt install linux-modules-extra-raspi -``` - ## Running K3s in Docker There are several ways to run K3s in Docker: diff --git a/docs/faq/faq.md b/docs/faq/faq.md index 61702b4ed..96eb86c42 100644 --- a/docs/faq/faq.md +++ b/docs/faq/faq.md @@ -63,7 +63,7 @@ If you are having an issue with deploying K3s, you should: 1) Check the [Known Issues](../known-issues/known-issues.md) page. -2) Check that you have resolved any [Additional OS Preparation](../advanced/advanced.md#additional-os-preparations). Run `k3s check-config` and ensure that it passes. +2) Check that you have resolved any [Additional OS Preparation](../installation/requirements.md#operating-systems). Run `k3s check-config` and ensure that it passes. 3) Search the K3s [Issues](https://github.com/k3s-io/k3s/issues) and [Discussions](https://github.com/k3s-io/k3s/discussions) for one that matches your problem. diff --git a/docs/installation/requirements.md b/docs/installation/requirements.md index e37f1d579..cb3b33daf 100644 --- a/docs/installation/requirements.md +++ b/docs/installation/requirements.md @@ -33,9 +33,70 @@ K3s is expected to work on most modern Linux systems. Some OSs have additional setup requirements: -- If you are using **RHEL/CentOS/Fedora**, follow [these steps](../advanced/advanced.md#red-hat-enterprise-linux--centos--fedora). -- If you are using **Ubuntu/Debian**, follow [these steps](../advanced/advanced.md#ubuntu--debian). -- If you are using **Raspberry Pi OS**, follow [these steps](../advanced/advanced.md#raspberry-pi). +#### Red Hat Enterprise Linux / CentOS / Fedora +
+ +It is recommended to turn off firewalld: +```bash +systemctl disable firewalld --now +``` + +If you wish to keep firewalld enabled, by default, the following rules are required: +```bash +firewall-cmd --permanent --add-port=6443/tcp #apiserver +firewall-cmd --permanent --zone=trusted --add-source=10.42.0.0/16 #pods +firewall-cmd --permanent --zone=trusted --add-source=10.43.0.0/16 #services +firewall-cmd --reload +``` + +Additional ports may need to be opened depending on your setup. See [Inbound Rules](#inbound-rules-for-k3s-server-nodes) for more information. If you change the default CIDR for pods or services, you will need to update the firewall rules accordingly. + +If enabled, it is required to disable nm-cloud-setup and reboot the node: +```bash +systemctl disable nm-cloud-setup.service nm-cloud-setup.timer +reboot +``` +
+ +#### Ubuntu / Debian + +
+ +Older Debian release may suffer from a known iptables bug. See [Known Issues](../known-issues/known-issues.md#iptables). + +It is recommended to turn off ufw (uncomplicated firewall): +```bash +ufw disable +``` + +If you wish to keep ufw enabled, by default, the following rules are required: +```bash +ufw allow 6443/tcp #apiserver +ufw allow from 10.42.0.0/16 to any #pods +ufw allow from 10.43.0.0/16 to any #services +``` + +Additional ports may need to be opened depending on your setup. See [Inbound Rules](#inbound-rules-for-k3s-server-nodes) for more information. If you change the default CIDR for pods or services, you will need to update the firewall rules accordingly. +
+ +#### Raspberry Pi + +
+ +Raspberry Pi OS is Debian based, and may suffer from a known iptables bug. See [Known Issues](../known-issues/known-issues.md#iptables). + +Standard Raspberry Pi OS installations do not start with `cgroups` enabled. **K3S** needs `cgroups` to start the systemd service. `cgroups`can be enabled by appending `cgroup_memory=1 cgroup_enable=memory` to `/boot/cmdline.txt`. + +Example cmdline.txt: +``` +console=serial0,115200 console=tty1 root=PARTUUID=58b06195-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait cgroup_memory=1 cgroup_enable=memory +``` + +Starting with Ubuntu 21.10, vxlan support on Raspberry Pi has been moved into a separate kernel module. +```bash +sudo apt install linux-modules-extra-raspi +``` +
For more information on which OSs were tested with Rancher managed K3s clusters, refer to the [Rancher support and maintenance terms.](https://rancher.com/support-maintenance-terms/) @@ -89,7 +150,7 @@ Flannel relies on the [Bridge CNI plugin](https://www.cni.dev/plugins/current/ma Typically, all outbound traffic is allowed. -Additional changes to the firewall may be required depending on the OS used. See [Additional OS Preparations](../advanced/advanced.md#additional-os-preparations). +Additional changes to the firewall may be required depending on the OS used. ## Large Clusters diff --git a/docs/known-issues/known-issues.md b/docs/known-issues/known-issues.md index 02fcca138..7af0c22fc 100644 --- a/docs/known-issues/known-issues.md +++ b/docs/known-issues/known-issues.md @@ -10,9 +10,15 @@ If you plan to use K3s with docker, Docker installed via a snap package is not r ### Iptables -If you are running iptables in nftables mode instead of legacy you might encounter issues. We recommend utilizing newer iptables (such as 1.6.1+) to avoid issues. +Iptables versions 1.8.0-1.8.4 have known issues that can cause K3s to fail. Several popular Linux distributions ship with these versions by default. One bug causes the accumulation of duplicate rules, which negatively affects the performance and stability of the node. See [Issue #3117](https://github.com/k3s-io/k3s/issues/3117) for information on how to determine if you are affected by this problem. -Additionally, versions 1.8.0-1.8.4 have known issues that can cause K3s to fail. See [Additional OS Preparations](../advanced/advanced.md#old-iptables-versions) for workarounds. +K3s includes a working version of iptables (v1.8.8) which functions properly. You can tell K3s to use its bundled version of iptables by starting K3s with the `--prefer-bundled-bin` option, or by uninstalling the iptables/nftables packages from your operating system. + +:::info Version Gate + +The `--prefer-bundled-bin` flag is available starting with the 2022-12 releases (v1.26.0+k3s1, v1.25.5+k3s1, v1.24.9+k3s1, v1.23.15+k3s1). + +::: ### Rootless Mode diff --git a/docs/quick-start/quick-start.md b/docs/quick-start/quick-start.md index 213b76c81..cd8b8bdb5 100644 --- a/docs/quick-start/quick-start.md +++ b/docs/quick-start/quick-start.md @@ -5,6 +5,8 @@ weight: 10 This guide will help you quickly launch a cluster with default options. The [installation section](../installation/installation.md) covers in greater detail how K3s can be set up. +Make sure your nodes meet the the [requirements](../installation/requirements.md) before proceeding. + For information on how K3s components work together, refer to the [architecture section.](../architecture/architecture.md) :::info From d9c556ec5ed58e7a4a3ae9a2e41f3a49cd12fe4a Mon Sep 17 00:00:00 2001 From: Derek Nola Date: Thu, 16 Nov 2023 12:13:35 -0800 Subject: [PATCH 2/3] Readd blurb about iptables 1.6.1 Signed-off-by: Derek Nola --- docs/known-issues/known-issues.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/docs/known-issues/known-issues.md b/docs/known-issues/known-issues.md index 7af0c22fc..c9bebc46d 100644 --- a/docs/known-issues/known-issues.md +++ b/docs/known-issues/known-issues.md @@ -10,6 +10,13 @@ If you plan to use K3s with docker, Docker installed via a snap package is not r ### Iptables +If you are running iptables v1.6.1 and older in nftables mode you might encounter issues. We recommend utilizing newer iptables (such as 1.6.1+) to avoid issues or running iptables legacy mode. + +``` +update-alternatives --set iptables /usr/sbin/iptables-legacy +update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy +``` + Iptables versions 1.8.0-1.8.4 have known issues that can cause K3s to fail. Several popular Linux distributions ship with these versions by default. One bug causes the accumulation of duplicate rules, which negatively affects the performance and stability of the node. See [Issue #3117](https://github.com/k3s-io/k3s/issues/3117) for information on how to determine if you are affected by this problem. K3s includes a working version of iptables (v1.8.8) which functions properly. You can tell K3s to use its bundled version of iptables by starting K3s with the `--prefer-bundled-bin` option, or by uninstalling the iptables/nftables packages from your operating system. From 1861bf0fb7ee793149d52f9abeb7c85bbf173fad Mon Sep 17 00:00:00 2001 From: Derek Nola Date: Fri, 17 Nov 2023 11:04:07 -0800 Subject: [PATCH 3/3] Convert to tabs using querystring for linking Signed-off-by: Derek Nola --- docs/installation/network-options.md | 2 +- docs/installation/requirements.md | 22 ++++++++-------------- 2 files changed, 9 insertions(+), 15 deletions(-) diff --git a/docs/installation/network-options.md b/docs/installation/network-options.md index 92a024e29..f45d94951 100644 --- a/docs/installation/network-options.md +++ b/docs/installation/network-options.md @@ -13,7 +13,7 @@ This page describes K3s network configuration options, including configuration o * Flannel options can only be set on server nodes, and must be identical on all servers in the cluster. * The default backend for Flannel is `vxlan`. To enable encryption, use the `wireguard-native` backend. -* Using `vxlan` on Rasperry Pi with recent versions of Ubuntu requires [additional preparation](../advanced/advanced.md#raspberry-pi). +* Using `vxlan` on Rasperry Pi with recent versions of Ubuntu requires [additional preparation](./requirements.md?os=pi#operating-systems). * Using `wireguard-native` as the Flannel backend may require additional modules on some Linux distributions. Please see the [WireGuard Install Guide](https://www.wireguard.com/install/) for details. The WireGuard install steps will ensure the appropriate kernel modules are installed for your operating system. You must ensure that WireGuard kernel modules are available on every node, both servers and agents, before attempting to use the WireGuard Flannel backend. diff --git a/docs/installation/requirements.md b/docs/installation/requirements.md index cb3b33daf..1c77cd4c8 100644 --- a/docs/installation/requirements.md +++ b/docs/installation/requirements.md @@ -32,9 +32,8 @@ Prior to May 2023 releases (v1.24.14+k3s1, v1.25.10+k3s1, v1.26.5+k3s1, v1.27.2+ K3s is expected to work on most modern Linux systems. Some OSs have additional setup requirements: - -#### Red Hat Enterprise Linux / CentOS / Fedora -
+ + It is recommended to turn off firewalld: ```bash @@ -56,11 +55,8 @@ If enabled, it is required to disable nm-cloud-setup and reboot the node: systemctl disable nm-cloud-setup.service nm-cloud-setup.timer reboot ``` -
- -#### Ubuntu / Debian - -
+ + Older Debian release may suffer from a known iptables bug. See [Known Issues](../known-issues/known-issues.md#iptables). @@ -77,11 +73,8 @@ ufw allow from 10.43.0.0/16 to any #services ``` Additional ports may need to be opened depending on your setup. See [Inbound Rules](#inbound-rules-for-k3s-server-nodes) for more information. If you change the default CIDR for pods or services, you will need to update the firewall rules accordingly. -
- -#### Raspberry Pi - -
+ + Raspberry Pi OS is Debian based, and may suffer from a known iptables bug. See [Known Issues](../known-issues/known-issues.md#iptables). @@ -96,7 +89,8 @@ Starting with Ubuntu 21.10, vxlan support on Raspberry Pi has been moved into a ```bash sudo apt install linux-modules-extra-raspi ``` -
+ + For more information on which OSs were tested with Rancher managed K3s clusters, refer to the [Rancher support and maintenance terms.](https://rancher.com/support-maintenance-terms/)