Allow configuration of the swap size via /etc/default/haos-swapfile file. By
setting the SWAPSIZE variable in this file, swapfile get recreated on the next
reboot to the defined size. Size can be either in bytes or with optional units
(B/K/M/G, accepting some variations but always interpreted as power of 10). The
size is then rounded to 4k block size. If no override is defined or the value
can't be parsed, it falls back to previously used 33% of system RAM.
Fixes#968
* Move rauc.db to boot partition
The RAUC metadata file contains information that is tightly related to the
system and kernel partitions. With the possibility to migrate data disk, the
rauc.db can contain bogus information when moved to a different system. Removal
of the file on "device wipe" is also not desirable, because the information
about slot status is lost.
Relocate the rauc.db to the boot partition after a system upgrade (as this
can't be handled by RAUC hooks, because it needs to be executed after all slots
and metadata is written) and adjust the script for recreating it. The downside
is that its content in /mnt/data would be recreated if the boot slot is changed
or system downgraded but this should be handled quite gracefully.
Also remove the raucdb-first-boot service which is no longer necessary
with the file not present in the data partition.
* Fix shellcheck and mount path
The /etc/usb_modeswitch.d is present and empty but it can't be written to allow
user modification. Bind-mount it like other /etc folders to make it possible to
adjust usb_modeswitch config.
Fixes#3785
If data disk is adopted on Yellow using the mechanism added in #3686, it
contains RAUC version information that is very likely invalid. In such case,
remove the file on first boot and have it recreated by the raucdb-update
service.
The timeout of 90s was introduced before it was ensured that the timesync
systemd unit starts after network is online. Now with that, it makes less sense
to wait that long - if network is unreachable at the point the time
synchronization starts, and the server fails to reply on the first sync, the
polling interval is exponentially increased and the benefit of waiting for more
attempts is doubtful.
Since another synchronization attempt is done after network changes its state,
we should rely on that instead of having the 90 seconds interval as a waiting
period for plugging the network cable. Worst case, there are other mechanisms
that should set the time to a reasonably accurate value, making the NTP sync
less importart for most of the cases.
* Relocate HAOS Systemd drop-ins to /usr/lib/systemd
With some exceptions, Systemd drop-ins overriding default unit configuration
have been placed to `/etc/systemd/system`. This is meant for user overrides of
those, or per `man 5 systemd.unit` for "system unites created by the
administrator". Relocate all of these to `/usr/lib/systemd` which should be
used as path for units "installed by the distribution package manager" which is
closer to what we're trying to achieve.
This will make it easier to detect changes to unit files once we enable the
possibility to edit the content of /etc.
* Patch systemd-timesyncd.service instead of replacing it fully
RAUC currently doesn't know the version of the booted slot when booted for the
first time or after wiping the data partition. As a result `ha os info` is
missing this information too.
As there's no built-in mechanism for generating these data by RAUC itself, add
a oneshot service that checks if the boot slot information is contained in the
rauc.db and if not, then generate it.
RAUC seems to cope quite well even with bogus data contained in rauc.db but in
any case, a test has been added to check that everything works as expected.
* Add fsfreeze support for QEMU/KVM/Proxmox installations
Add fsfreeze scripts which calls the new Supervisor API to freeze Home
Assistant Core and add-ons which support the backup freeze scripts
(`backup_pre` and `backup_post`).
This allows to create safe snapshots with databases running.
* Fix lint issues
Pull in the swapfile creation service haos-swapfile.service when
swap.target is reached. This makes sure the service is started even when
other targets are used (e.g. rescue.target).
* Delete Bluetooth device cache regularly
Delete stale Bluetooth devices from the BlueZ device cache every week.
This makes sure that the overlay partition doesn't run out of inodes
which has happened in real world scenarios where many new Bluetooth
devices are discovered.
BlueZ maintains these files on a best effort base. So removing them
while BlueZ is running should be safe.
An alternative considered was to lower BlueZ GATT caching (e.g. by
using Cache=yes instead of always, to cache only paired devices).
However, this would hurt performance and battery lifetime of Bluetooth
devices due to additional unnecessary GATT attributes reads. This is in
particular true for Bluetooth 5.1 devices which support the Database
Hash charactristic. Caching has also helped reliability with
intermittent connections (see
https://github.com/bluez/bluez/issues/191).
More importantly, besides the GATT attribute cache the same files are
also used to cache the device names as well. This is independent of the
above mentioned GATT cache configuration (see device_store_cached_name
in BlueZ). So disabling the GATT caching alone wouldn't solve the
particular problem we are facing.
See also: https://github.com/home-assistant/supervisor/issues/4490
* Use access timestamp instead of modification timestamp
The modification timestamp gets updated regularly (on each connect) it
seems. However, using access timestamp might be more accurate, as it
seems to preserves slightly more cache files. This additional devices
might be devices we don't regularly connect but are still around (and
therefor we shouldn't reread the GATT attributes regularly).
So deleting cache entries with access time older than 7 days. Which
essentially deletes all the entries of devices which haven't been seen
the last 7 days.
To read the current LED configuration correctly /mnt/boot is required.
This change makes sure that the boot partition is mounted when the OS
Agent starts.
* Enable Multi-Gen LRU
Multi-Gen LRU should improve performance under memory pressure. This is
especially useful for embedded platforms where memory is scarce.
* Add service to configure Multi-Gen LRU
Use min_ttl_ms of 1 which is the least aggressive in terms of lag. Since
we are a server application, we can tune trashing prevention with a
higher acceptable lag.
* Use zswap instead of swap in zram
This requires a swap file which will get generated automatically on
startup.
* Fix file size and free disk space comparison
* Set zswap factor to 33%
* Set vm.swappiness to 1
Decrease swapping to a minimum. This is also recommended for database
work loads by the MariaDB documentation. In practice it causes the least
amount of writes to disk when under memory pressure, while still making
swap available when needed.
* Deactivate any external data disk device on first boot (#2390)
* Use lsblk to determine the underlying device file
Comparing major number is not reliable, e.g. virtio disks have the same
major number despite being different devices. Use lsblk to find the
underlying device, and compare the device name instead.
This is more readable than passing arguments to the daemon directly. It
also shortens the ExecStart command significantly, which is stored in
every log entry in systemd-journald.
A higher file system commit interval can help to decrease the amount of
writes. In tests, a commit interval of higher than 30s seems not to help
much in practice. Settle with 30s for now.
* Bump buildroot
* buildroot 99b62b8bd3...97287bbebf (3):
> package/dbus-broker: bump to release 32
> package/dbus-broker: new package
> Merge pull request #3 from home-assistant/2022.02.x-haos-cgroup-v2
* Use dbus-broker as default D-Bus broker
The dbus-broker (Linux D-Bus Message Broker) aims to be a high
performance and reliable D-Bus broker which can be used as a drop in
replacement to the reference implementation D-Bus broker. In tests it
showed significantly better performance especially when routing BLE
messages.
* Allow dbus-broker to start early
For HAOS device wipe feature we need haos-agent.service and
udisk2.service early. Both require a working D-Bus broker.
The options PrivateTmp and PrivateDevices add additional After=
orderings which doesn't allow dbus-broker to be started early.
* Fix D-Bus dependency
D-Bus services should just depend on dbus.socket.
A faster restart policy is unlikely to help. Increasing the limit makes
it less likely to run into cloud service rate limits (e.g. container
registry).
* Add systemd-journal-remote to the image
This allows to access journald's log from within Supervisor and expose
more system logs to users.
* Allow to access systemd-journal-gatewayd from Supervisor
Create a systemd-journal-gatewayd.socket service using a Unix socket and
bind mount it into the Supervisor container. This allows to query
systemd-journald from Supervisor directly.
Currently the hassos-apparmor.service wants the
hassos-supervisor.service and vice-versa. This is unnecessary and leads
to activation of hassos-supervisor.service when reload/restart
hassos-apparmor.service (Supervisor is doing that on startup).
Make hassos-apparmor.service independent and add dependency as well as
ordering from hassos-supervisor.service side.
Since we start the HomeAssistant shell directly on tty the service
responsible for starting did not restart the shell on exit. Remove the
RemainAfterExit flag to make sure that the shell restarts on exit.
* Start ha-cli on tty1 instead of a getty
Instead of starting a getty start the ha-cli directly. This will show
the banner right on startup with the important information such as IP
address of the instance or the URL to reach it.
* Use default shell as root shell instead of HA CLI
Instead of using the ha-cli.sh script as login shell use the regular
shell. Amongst other things, this allows to run VS Code devcontainers
remotely via SSH or using scp. The HA CLI is still available using the
`ha` command.
* Enable systemd-time-wait-sync.service by default
Enable the systemd-time-wait-sync.service by default. This allows to use
the time-sync.target which allows to make sure services only get started
once the time is synchronized.
* Make sure time is synchronized when starting hassos-supervisor.service
Use the time-sync.target to make sure that the Supervisor gets stsarted
after the time has been synchronized.
* Set timeout for systemd-time-wait-sync.service
Don't delay startup forever in case time synchronization doesn't work.
This allows to boot the system even without Internet connection.
The latest version of OS Agent sets haos.wipe=1 as kernel argument to
trigger a device wipe. Let systemd pickup this kernel command line
argument and start haos-wipe.service.
This rather complex architecture allows to add other triggers in the
future, e.g. a button read in the boot loader.
There are incident reports on the internet where poeple report that
fsck.(v)fat actually leads to problems rather file system fixes. Around
the time when Home Assistant OS added fsck.fat for the boot partition,
reports of empty boot partitions or file with weired filenames started
to appear. This could be caused by fsck.fat.
Disable fsck on the boot partition.
The supervisor container requires the "hassio-supervisor" AppArmor
profile. Make sure our AppArmor service hassos-apparmor is a dependency
of the hassos-supervisor.service.
* Use systemd-growfs instead of resize2fs (#1106)
Since systemd 236 systemd has a built-in file system growing mechanism.
The mechanism relies on the kernels online file system resize
capabilities instead of the external resize2fs utility. Online resizing
is supposedly much faster since the kernel takes care of things.
This also makes sure that external file systems get resized which
previously have not been taken care of.
* Drop HA OS specific file system resizing
Since we have systemd-growfs in place now we can drop our file system
resizing code.
* Make sure /dev/disk/by-label/hassos-data is present after resizing
Note: systemd will retry mnt-data.mount later, so at least in theory
this shouldn't really matter. However, the journal has a lot of churn
due to that reordering.
When we write the update to the boot partiton, there is nothing which
makes sure that data is written to disk. This leaves a rather large
window (probably around 30s) where a machine reset/poweroff can lead
to a corrupted boot partition. Use the sync mount option to minimize the
corruption window.
Note that sync is not ideal for flash drives normally. But since we
write very little and typically only on OS update to the boot partition,
this shouldn't be a problem.
* Avoid waiting for external drive unnecessarily
Even though the condition to start hassos-data.service is not met (the
file /mnt/overlay/data-move is not there by default), it seems that
systemd waits for the dependencies for hassos-data.service. Don't
Require or Wants any dependencies which might not be present by
default.
* Use systemd to wait for partition using partlabel device
* Use sfdisk which allows to wipe filesystem signatures
Even though we zap the partition table using sgdisk, the file system
superblock (which contains the file system label) does survive. This
can cause problems when trying to reuse a disk previously already
labeled using hassos-data: It might take precendence on next boot
over the existing data partition on the eMMC.
Make sure to clean all file system signatures using sfdisk.
* Make the datactl command more robust
Validate target disk (partition) size to avoid a copy attempt which will
fail. If e2image operation fails, make sure the leftover copy is not
regonized as data partition.
* Fix hassos-data service device unit dependencies
* Rewrite datactl command
Prepare the target partition as part of the datactl command. Rely on
partlabel for the target disk since we are always using GPT on the
target disk. Use systemd and partlabel mechanism to wait and find
the target data disk. Keep using the file system label to identify
the source disk.
Also use e2image instead of raw dd to move data. This should
speed up the processes significantly.
* Fix corner case when reusing same disk again
* Update buildroot-patches for 2020.11-rc1 buildroot
* Update buildroot to 2020.11-rc1
Signed-off-by: Stefan Agner <stefan@agner.ch>
* Don't rely on sfdisk --list-free output
The --list-free (-F) argument does not allow machine readable mode. And
it seems that the output format changes over time (different spacing,
using size postfixes instead of raw blocks).
Use sfdisk json output and calculate free partition space ourselfs. This
works for 2.35 and 2.36 and is more robust since we rely on output which
is meant for scripts to parse.
* Migrate defconfigs for Buildroot 2020.11-rc1
In particular, rename BR2_TARGET_UBOOT_BOOT_SCRIPT(_SOURCE) to
BR2_PACKAGE_HOST_UBOOT_TOOLS_BOOT_SCRIPT(_SOURCE).
* Rebase/remove systemd patches for systemd 246
* Drop apparmor/libapparmor from buildroot-external
* hassos-persists: use /run as directory for lockfiles
The U-Boot tools use /var/lock by default which is not created any more
by systemd by default (it is under tmpfiles legacy.conf, which we no
longer install).
* Disable systemd-update-done.service
The service is not suited for pure read-only systems. In particular the
service needs to be able to write a file in /etc and /var. Remove the
service. Note: This is a static service and cannot be removed using
systemd-preset.
* Disable apparmor.service for now
The service loads all default profiles. Some might actually cause
problems. E.g. the profile for ping seems not to match our setup for
/etc/resolv.conf:
[85503.634653] audit: type=1400 audit(1605286002.684:236): apparmor="DENIED" operation="open" profile="ping" name="/run/resolv.conf" pid=27585 comm="ping" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
Drop AVAHI and use systemd-resolved to announce hostname via mDNS
and LLMNR. Also continue to offer the _workstation._tcp.local service
since it is used by the CoreDNS mDNS plug-in.
In case a container image is corrupted `docker inspect` might fail:
# docker inspect --format='{{.Id}}' "${SUPERVISOR_IMAGE}"
Error response from daemon: readlink /mnt/data/docker/overlay2: invalid argument
In that same state the `docker images` command still shows the images.
Since `docker inspect` returns an error SUPERVISOR_IMAGE_ID will be empty
and a simple `docker pull` will be attempted. That does not suffice to
recover from a corrupted container image.
Use `docker images` to get the image ids and make sure to delete all
image ids found by that command.
Also don't use RuntimeDirectory since it deletes the runtime directory
between the service start attempts which defeats the purpose.
* Simplify self healing capabilities of Supervisor service
Instead of relying on time based information on how long the container
has been running use a startup marker file to infer if the last startup
has been successful.
* Update buildroot-external/rootfs-overlay/usr/sbin/hassos-supervisor
Co-authored-by: Pascal Vizeli <pascal.vizeli@syshack.ch>
Co-authored-by: Pascal Vizeli <pascal.vizeli@syshack.ch>
* automatically fsck to repair partitions
* add fsck.fat so rpi boot partition can be repaired
* Use Wants= instead of Requires=
Co-authored-by: Pascal Vizeli <pascal.vizeli@syshack.ch>
* add dosfstools to all images
* run hassos-data and hassos-expand after fsck
Co-authored-by: Pascal Vizeli <pascal.vizeli@syshack.ch>
The Docker socket path is /run/docker.sock. Also only one path can be
used per property. This fixes the supervisor service, which currently
refuses to start due to missing Docker socket.
