Afer builder changes, ARM images are now correctly published with their
platform, and when skopeo is used to inspect/pull the image on x86 without any
other flags, it fails with:
Error parsing manifest for image: Error choosing image instance: no image found in image index for architecture amd64, variant "", OS linux
Pass the correct arch in skopeo operations to fix that.
The retry when fetching containers from the registry didn't work because the
script was executed with `set -e`. Capture the error code also for non-zero
exit status.
Also use while loop instead of recursion and back off exponentially - start
with 5s and multiply by 3 (i.e. 5s, 15s, 45s - waiting in total up to 1 minute
for the registry to recover).
Importing docker-archive format leads to some layers missing in the content
storage which results in some image metadata missing. This appears to be the
same regression as moby/moby#49473. Importing OCI archives when bootstrapping
the data partition seems to work this bug around.
Fixes#4385
We still face occasional build errors when fetching from the Docker registry
fails and is not retried with the Skopeo's built-in retry mechanism that was
enabled in #1866. This happens on some network failures, or when premature EOF
is returned when fetching the HTTP data. Seems we're not the only ones having
such issues [1].
To workaround this, add a generic retry shell function that simply retries when
the command ends with a non-zero status, no matter what was the actual cause of
the error.
[1] https://www.github.com/containers/common/issues/654
It seems that the GitHub container registry sometimes returns 503
service unavailable temporarily ("Error fetching tags list: invalid status
code from registry 503"). Use skopeo's retry mechanism to try up to 5
times before failing.
* Avoid race condition when fetching containers during build
So far only a single builder was active for each architecture. This
toghether with the naming scheme to include architecture/machine name
made sure that an image could only be fetched or used by a single
builder.
However, since most systems are now aarch64, multiple runners are now
active for a single architecture. This makes it necessary to lock
fetching/coping of container images to avoid race conditions.
* Use skopeo to download container images
Separate container download from image build. This will allow to share
the downloaded images between multiple builds.
We won't store the Supervisor container with the version tag, just with
the latest tag. This allows to simplify the procedure a bit. It seems
there is no downside to this approach.
* Use official Docker in Docker images to build data partition
Instead of building our own Debian based image let's use the official
Docker in Docker image. This avoids building an image for the hassio
data partition and speeds up build as well.
This calls mount commands using sudo to mount the data partition as part
of the buildroot build now. This is not much different from before as
mount has been called as root inside the container, essentially equates
to the same "isolation" level.
* Use image digest as part of the file name
The landing page has no version information in the tag. To avoid
potentially source caching issues, use the digest as part of the file
name.
