Use Docker containerd snapshotter for new and wiped installs (#4360)

Prefer the containerd snapshotter by using it by default for new installs and
when no Docker data is present (e.g. after datadisk wipe). The snapshotter is
enabled by a dockerd flag which is set when a flag file is present in the data
partition. This flag file can be used also to opt-in for this snapshotter on
legacy installs (high level API through OS Agent and Supervisor TBD), to
migrate to the containerd snapshotter this file can be simply created manually.

Testing shown no major problems when migrating, the old overlay2 folder can be
(and should be - to avoid situations where the data disk might run out of
space) deleted before the docker.service is started in the docker-prepare
script.

Note that there's no offline migration path, OS needs to be connected to the
internet to re-download the images when migrating. This could be theoretically
possible through docker image save/load functions but guarding for enough of
space and other edge cases would be probably too complex to justify it.

Refs #4252
Refs #4253 - easier opt-in method is still needed
Closes #4254 - migration is handled seamlessly by Docker

buildroot-external/package/hassio/create-data-partition.sh

@@ -23,11 +23,14 @@ container=$(docker run --privileged -e DOCKER_TLS_CERTDIR="" \
 	-v "${build_dir}/data/":/data \
 	-v "${build_dir}/data/docker/":/var/lib/docker \
 	-v "${build_dir}":/build \
-	-d "docker:${docker_version}-dind" --storage-driver overlay2)
+	-d "docker:${docker_version}-dind" --feature containerd-snapshotter)
 
 docker exec "${container}" sh /build/dind-import-containers.sh "${channel}"
 
 docker stop "${container}"
 
+# Indicator for docker-prepare.service to use the containerd snapshotter
+sudo touch "${build_dir}/data/.docker-use-containerd-snapshotter"
+
 # Unmount data image
 sudo umount "${build_dir}/data/"

buildroot-external/rootfs-overlay/etc/docker/daemon.json

@@ -1,5 +1,4 @@
 {
-    "storage-driver": "overlay2",
     "log-driver": "journald",
     "log-opts": {
         "tag": "{{.Name}}"

buildroot-external/rootfs-overlay/usr/lib/systemd/system/docker-prepare.service

@@ -0,0 +1,11 @@
+[Unit]
+Description=Docker environment and config initialization
+Before=docker.service
+RequiresMountsFor=/mnt/data
+
+[Service]
+Type=oneshot
+ExecStart=/usr/libexec/docker-prepare
+
+[Install]
+WantedBy=multi-user.target

buildroot-external/rootfs-overlay/usr/lib/systemd/system/docker.service.d/hassos.conf

@@ -2,4 +2,8 @@
 RequiresMountsFor=/etc/docker /mnt/data /var/lib/docker
 
 [Service]
+# env file must be created by docker-prepare.service
+EnvironmentFile=/run/dockerd.env
+ExecStart=
+ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock $DOCKERD_FLAGS
 OOMScoreAdjust=-400

buildroot-external/rootfs-overlay/usr/libexec/docker-prepare

@@ -0,0 +1,23 @@
+#!/bin/sh
+
+set -e
+
+if [ ! -d /mnt/data/docker ] || [ -z "$(ls -A /mnt/data/docker)" ]; then
+    echo "[INFO] Docker data is wiped, creating containerd snapshotter flag"
+    touch /mnt/data/.docker-use-containerd-snapshotter
+fi
+
+DOCKERD_FLAGS=""
+
+if [ -f /mnt/data/.docker-use-containerd-snapshotter ]; then
+    echo "[INFO] Using Docker containerd snapshotter"
+    DOCKERD_FLAGS="${DOCKERD_FLAGS} --feature containerd-snapshotter"
+
+    if [ -d /mnt/data/docker/overlay2 ]; then
+        echo "[INFO] Removing no longer used overlay2 directory"
+        # Allow the removal to fail without failing the service
+        rm -rf /mnt/data/docker/overlay2 || true
+    fi
+fi
+
+echo "DOCKERD_FLAGS=\"${DOCKERD_FLAGS}\"" > /run/dockerd.env