v1.1.3

Note: in the Bottlerocket v1.0.8 release, for the aws-k8s-1.20 and aws-k8s-1.21 variants, we set the default Kubernetes CPU manager policy to "static". We heard from several users that this breaks usage of the Fluent Bit log processor. In Bottlerocket v1.1.3, we've changed the default back to "none", but have added a setting so you can use the "static" policy if desired. To do so, set settings.kubernetes.cpu-manager-policy to "static". To do this in user data, for example, pass the following:

[settings.kubernetes]
cpu-manager-policy = "static"

OS Changes

• Fix parsing of lists of values in domain name search field of DHCP option sets (#1646, thanks @hypnoce!)
• Add setting for configuring Kubernetes CPU manager policy and reconcile policy (#1638)

Build Changes

• Update SDK to 0.22.0 (#1640)
• Store build artifacts per architecture (#1630)

Documentation Changes

• Update references to the ECS variant for GA release (#1637)

v1.1.2

With this release, the aws-ecs-1 variant has graduated from preview status and is now generally available. It's been updated to include Docker 20.10. 🎉

OS Changes

• Add aws-k8s-1.21 variant with Kubernetes 1.21 support (#1612)
• Add settings for configuring kubelet containerLogMaxFiles and containerLogMaxSize (#1589) (Thanks, @samjo-nyang!)
• Add settings for configuring kubelet systemReserved (#1606)
• Add kdump support, enabled by default in VMware variants (#1596)
• In host containers, allow mount propagations from privileged containers (#1601)
• Mark ipv6 lease as optional for eth0 (#1602)
• Add recommended device filters to open-vm-tools (#1603)
• In host container definitions, default "enabled" and "superpowered" to false (#1580)
• Allow pubsys refresh-repo to use default key path (#1575)
• Update default host containers (#1609)

Build Changes

• Add grep package to all variants (#1562)
• Update Rust dependencies (#1623, #1574)
• Update third-party packages (#1619, #1616, #1625)
• In GitHub Actions, pin rust toolchain to match version in SDK (#1621)
• Add imdsclient library for querying IMDS (#1372, #1598, #1610)
• Remove reqwest proxy workaround in metricdog and updog (#1592)
• Simplify conditional compilation in early-boot-config (#1576)
• Only build shibaken for aws variants (#1591)
• Silence tokio mut warning in thar-be-settings (#1593)
• Refactor package and variant dependencies (#1549)
• Add derive attributes at start of list in model-derive (#1572)
• Limit threads during pubsys validate-repo (#1564)

Documentation Changes

• Document the deprecation of the aws-k8s-1.16 variant (#1600)
• Update README for VMware and add a QUICKSTART-VMWARE (#1559)
• Add ap-northeast-3 to supported region list (#1566)
• Add details about the two default Bottlerocket volumes to README (#1588)
• Document webpki-roots version in webpki-roots-shim (#1565)

v1.1.1

Security fixes

• Patch runc for CVE-2021-30465 (232c5741ecec)

v1.1.0

Deprecation Notice

The Kubernetes 1.16 variant, aws-k8s-1.16, will lose support in July, 2021. Kubernetes 1.16 is no longer receiving support upstream. We recommend replacing aws-k8s-1.16 nodes with a later variant, preferably aws-k8s-1.19 if your cluster supports it. See this issue for more details.

Important Notes

New variants with new defaults

This release introduces two new variants, aws-k8s-1.20 and vmware-k8s-1.20. We plan for all new variants, including these, to contain the following changes:

• The kernel is Linux 5.10 rather than 5.4.
• The kernel lockdown mode is set to "integrity" rather than "none".

The ECS preview variant, aws-ecs-1, has also been updated with these changes.

Existing aws-k8s variants will not receive these changes as they could affect existing workloads.

ECS task networking

The aws-ecs-1 variant now supports the awsvpc mode of ECS task networking. This allocates an elastic network interface and private IP address to each task.

OS Changes

• Add Linux kernel 5.10 for use in new variants (#1526)
• Add aws-k8s-1.20 variant with Kubernetes 1.20 support (#1437, #1533)
• Add vmware-k8s-1.20 variant with Kubernetes 1.20 for VMware (#1511, #1529, #1523, #1502, #1554)
• Remove aws-k8s-1.15 variant (#1487, #1492)
• Constrain ephemeral port range (#1560)
• Support awsvpc networking mode in ECS (#1246)
• Add settings for QPS and burst limits of Kubernetes registry pulls, event records, and API (#1527, #1532, #1541)
• Add setting to allow configuration of Kubernetes TLS bootstrap (#1485)
• Add setting for configuring Kubernetes cloudProvider to allow usage outside AWS (#1494)
• Make Kubernetes cluster-dns-ip optional to support usage outside of AWS (#1482)
• Change parameters to support healthy CIS scan (#1295) (Thanks, @felipeac!)
• Generate stable machine IDs for VMware and ARM KVM guests (#1506, #1537)
• Enable "integrity" kernel lockdown mode for aws-ecs-1 preview variant (#1530)
• Remove override for default service start timeout (#1483)
• Restrict access to bootstrap container user data with SELinux (#1496)
• Split SELinux policy rules for trusted subjects (#1558)
• Add symlink to allow usage of secrets store CSI drivers (#1544)
• Prevent bootstrap containers from restarting (#1508)
• Add udev rules to mount CD-ROM only when media is present (#1516)
• Add resize2fs binary to sbin (#1519) (Thanks, @samjo-nyang!)
• Only restart a host container if affected by settings change (#1480)
• Support file patterns when specifying log files in logdog (#1509)
• Daemonize thar-be-settings to avoid zombie processes (#1507)
• Add support for AWS region ap-northeast-3: Osaka (#1504)
• Generate pause container URI with standard template variables (#1551)
• Get cluster DNS IP from cluster when available (#1547)

Build Changes

• Use kernel 5.10 in aws-ecs-1 variant (#1555)
• Build only the packages needed for the current variant (#1408, #1520)
• Use a friendly name for VMware OVA files in build outputs (#1535)
• Update SDK to 0.21.0 (#1497, #1529)
• Allow variants to specify extra kernel parameters (#1491)
• Move kernel console settings to variant definitions (#1513)
• Update vmw_backdoor dependency (#1498) (Thanks, @lucab!)
• Archive old migrations (#1540)
• Refactor default settings and containerd configs to shared files (#1538, #1542)
• Check cargo version at start of build so we have a clear error when it's too low (#1503)
• Fix concurrency issue in validate-repo that led to hangs (#1521)
• Update third-party package dependencies (#1543, #1556)
• Update Rust dependencies in the tools/ workspace (#1548)
• Update tokio-related Rust dependencies in the sources/ workspace (#1479)
• Add upstream runc patches addressing container scheduling failure (#1546)
• Retry builds on known BuildKit internal errors (#1557, #1561)

Documentation Changes

• Document the deprecation of the aws-k8s-1.15 variant (#1476)
• Document the need to quote most Kubernetes labels/taints (#1550) (Thanks, @ellistarn!)
• Fix VMware spelling and document user data sources (#1534)

v1.0.8

Deprecation Notice

Bottlerocket 1.0.8 is the last release where we plan to support the Kubernetes 1.15 variant, aws-k8s-1.15. Kubernetes 1.15 is no longer receiving support upstream. We recommend replacing aws-k8s-1.15 nodes with a later variant, preferably aws-k8s-1.19 if your cluster supports it. See this issue for more details.

OS Changes

• Support additional kubelet arguments: kube-reserved, eviction-hard, cpu-manager-policy, and allow-unsafe-sysctls (#1388, #1472, #1465)
• Expand file and process restrictions in the SELinux policy (#1464)
• Add support for bootstrap containers (#1387, #1423)
• Make host containers inherit proxy env vars (#1432)
• Allow gzip compression of user data (#1366)
• Add 'apply' mode to apiclient for applying settings from URIs (#1391)
• Add compat symlink for kubelet volume plugins (#1417)
• Remove bottlerocket.version attribute from ECS agent settings (#1395)
• Make Kubernetes taint values optional (#1406)
• Add guestinfo to available VMWare user data retrieval methods (#1393)
• Include source of invalid base64 data in error messages (#1469)
• Update eni-max-pods data file (#1468)
• Update default host container versions (#1443, #1441, #1466)
• Fix avc denial for dbus-broker (#1434)
• Fix case of outputted JSON keys in host container user data (#1439)
• Set mode of host container persistent storage directory after creation (#1463)
• Add "current" persistent storage location for host containers (#1416)
• Write static-pods manifest to tempfile before persisting it (#1409)

Build Changes

• Update default variant to aws-k8s-1.19 (#1394)
• Update third-party packages (#1460)
• Update Rust dependencies (#1461, #1462)
• Update dependencies of host-ctr (#1371)
• Add support for specifying a variant's supported architectures (#1431)
• Build OVA packages and include them in repos (#1428)
• Add support for qcow2 as an image format (#1425) (Thanks, @mikalstill!)
• Prevent unneeded artifacts from being copied through build process (#1426)
• Change image format for vmware-dev variant to vmdk (#1397)
• Remove tough dependency from update_metadata (#1390)
• Remove generate_constants logic from build.rs of parse-datetime (#1376)
• In the tools workspace, update to tokio v1, reqwest v0.11, and tough v0.11 (#1370)
• Run static and non-static Rust builds in parallel (#1368)
• Disable CMDLINE_EXTEND kernel configuration (#1473)

Documentation Changes

• Document metrics settings in README (#1449)
• Fix broken links for symlinked files in models README (#1444)
• Document apiclient update as primary CLI update method (#1421)
• Use apiclient set in introductory documentation, explain raw mode separately (#1418)
• Prefer resolve:ssm: parameters for simplicity in QUICKSTART (#1363)
• Update quickstart guides to have arm64 examples (#1360)
• Document the deprecation of the aws-k8s-1.15 variant (#1476)

v1.0.7

Security fixes

• containerd: update to 1.4.4 (#1401)

OS Changes

• systemd: update to 247.4 to fix segfault in some cases (#1400)
• apiserver: reap exited child processes (#1384)
• host-ctr: specify non-colliding runc root (#1359)
• updog: update signal-hook dependency (#1328)

v1.0.6

OS Changes

• Add metricdog to support sending anonymous metrics (#1006, #1322)
• Add a vmware-dev variant (#1292, #1288, #1290)
• Add Kubernetes static pods support (#1317)
• Add high-level 'set' subcommand for changing settings using apiclient (#1278)
• Allow admin container to use SSH public keys from user data (#1331, #1358, #19)
• Add support for kubelet in standalone mode and TLS auth (#1338)
• Add https-proxy and no-proxy settings to updog (#1324)
• Add support for pulling host-containers from ECR Public (#1296)
• Add network proxy support to aws-k8s-1.19 (#1337)
• Modify default SELinux label for containers to align with upstream (#1318)
• Add aliases for container-selinux types to align with community (#1316)
• Update default versions of admin and control containers (#1347, #1344)
• Update ecs-agent to 1.50.2 (#1353)
• logdog: Add eni logs for Kubernetes (#1327)

Build Changes

• Add the ability to output vmdk via qemu-img (#1289)
• Add support for kmod kits to ease building of third-party kernel modules (#1287, #1286, #1285,#1357)
• storewolf: Declare dependencies on model and defaults files (#1319)
• storewolf: Refactor default settings files to allow sharing (#1303, #1329)
• Switch from TermLogger to SimpleLogger (#1282, thanks @hencrice!)
• Allow overriding the "pretty" name of the OS inside the image (#1330)
• Specify bash in link-variant task for use of bash features (#1323)
• Fix invalid symlinks when the BUILDSYS_NAME variable is set (#1312)
• Track and clean output files for builds (#1291)
• Update third-party software packages (#1340, #1336, #1334, #1333, #1335, #1190, #1265, #1315, #1352, #1356)

Documentation Changes

• Add lockdown notes to SECURITY_GUIDANCE.md (#1281)
• Clarify use case for update repos (#1339)
• Fix broken link from API docs to top-level docs (#1306)

v1.0.5

Note for aws-ecs-1 variant: due to a change in the ECS agent's data store schema, the aws-ecs-1 variant cannot be downgraded after updating to v1.0.5. Attempts to downgrade may result in inconsistencies between ECS and the Bottlerocket container instance.

OS Changes

• Add aws-k8s-1.19 variant with Kubernetes 1.19 (#1256)
• Update ecs-agent to 1.48.1 (#1201)
• Add high-level update subcommands to apiclient (#1219, #1232)
• Add kernel lockdown settings (#1223, #1279)
• Add restart-commands for docker, kubelet, containerd (#1231, #1262, #1258)
• Add proper restarts for host-containers (#1230, #1235, #1242, #1258)
• Fix SELinux policy (#1236)
• Set version and revision strings for containerd (#1248)
• Add host-container user-data setting (#1244, #1247)
• Add network proxy settings (#1204, #1262, #1258)
• Update kernel to 5.4.80-40.140 (#1257)
• Update third-party software packages (#1264)
• Update Rust dependencies (#1267)

Build Changes

• Improve support for out-of-tree kernel modules (#1220)
• Fix message in partition size check condition (#1233, thanks @pranavek!)
• Split the datastore module into its own crate (#1249)
• Update SDK to v0.15.0 (#1263)
• Update Github Actions to ignore changes that only include .md files (#1274)

Documentation Changes

• Add documentation comments to Dockerfile (#1254)
• Add a note about CPU usage during builds (#1266)
• Update README to point to discussions (#1273)

v1.0.4

Security fixes

• Patch containerd for CVE-2020-15257 (f3677c1406)

v1.0.3

OS Changes

• Support setting Linux kernel parameters (sysctl) via settings (see README) (#1158, #1171)
• Create links under /dev/disk/ephemeral for ephemeral storage devices (#1173)
• Set default RLIMIT_NOFILE in CRI to 65536 soft limit and a 1048576 hard limit (#1180)
• Add rtcsync directive to chrony config file (#1184, thanks @errm!)
• Add /etc/ssl/certs symlink to the CA certificate bundle for compatibility with the cluster autoscaler (#1207)
• Add procps dependency to docker-engine so that docker top works (#1210)

Build Changes

• Align optimization level for crate and dependency builds (#1155)
• pubsys no longer requires an Infra.toml file for basic usage (#1166)
• Makefile: Check that $BUILDSYS_ARCH has a supported value (#1167)
• Build migrations in parallel (#1192)
• Allow file URLs for role in pubsys-setup (#1194)
• Update Rust dependencies (#1196)
• Update SDK to v0.14.0 (#1198)
• Fix an occasional issue with KMS signing in pubsys (#1205)
• Backport selected fixes from containerd 1.4 (#1216)
• Update third-party package dependencies (#1176, #1195)
• Switch to SDK v0.14.0 (#1198)

Documentation Changes

• Nits and fixes (#1170, #1179)
• Add missing prerequisites for building Bottlerocket (#1191)