Files
supervisor/tests/resolution/test_resolution_manager.py
T
ccdd8c1ef4 Add repair to move local data blocking a mount target (#7089)
* Add repair to move local data blocking a mount target

When an add-on writes into a media/share directory while its network
mount is not in place (#7037), the local data blocks re-creating the
mount: mounting over a non-empty directory is refused. Until now this
failed silently at Supervisor startup — the bind mounts were created as
fire-and-forget tasks — and the only way out was to remove the data
manually over SSH/Samba and re-create the mount via the API.

Surface the condition as a new mount_target_not_empty issue and offer a
move_local_data suggestion. The fixup moves the blocking data to a
<name>_local_recovery folder in a user-accessible location — media or
share for bind mount targets, local backup storage for backup mounts
(their data mount directory is not reachable for users) — then reloads
the mount. Nothing is deleted; users can inspect and clean up the
recovered data via the media browser or the share and backup folders.

Bind mount failures during load are now awaited and routed into
resolution issues instead of being swallowed as fire-and-forget tasks;
bind failures other than blocking local data create the existing
mount_failed issue. A successful mount reload dismisses a stale local
data issue.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Attach move local data suggestion to the mount failed issue

Review feedback on the repair: rather than introducing a separate
mount_target_not_empty issue type, keep a single mount failed issue
per mount and offer moving the blocking data as an additional
suggestion alongside reload and remove. Reload stays available for
users who prefer to clear the data themselves, and at most one repair
exists per mount. Adding is idempotent, so an already-raised mount
failed issue just gains the extra suggestion.

When re-creating the bind mount after a successful reload fails on
blocking local data, the mount failed issue is re-added together with
the move suggestion, since the reload already dismissed it.

This also resolves the reviewer note about not-a-directory conflicts
being reported under a not-empty issue type: the issue type no longer
encodes the filesystem detail, while the error messages keep the
distinction.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Keep an empty directory in place after relocating local data

If remounting fails after the local data was moved aside (e.g. the
server is unreachable at that moment), the renamed directory left
nothing behind: media/share consumers saw the folder disappear
entirely. Recreate an empty directory right after the rename so the
path stays present regardless of whether the remount succeeds.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Drop move local data suggestion once the data was moved

When the remount after relocating local data fails (e.g. the server is
unreachable at that moment), the mount failed issue stays — but the
move suggestion stayed with it, offering to move data that is no
longer in the way. Dismiss the suggestion after the relocation step so
only reload and remove remain for the leftover failure. Detection
re-adds the move suggestion if local data blocks the target again.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Filter Core-facing suggestions by minimum Core version

The fix flow translations for a new suggestion ship with a Core
release. Older Core frontends render an unknown suggestion as an
empty, unlabeled menu entry in the repair fix flow. Filter such
suggestions from Core-facing output — the issue events sent over the
websocket and the resolution API responses when the caller is Home
Assistant — until the connected Core is new enough. Other API
consumers like the CLI always see the full suggestion list.

The move_local_data suggestion requires Core 2026.9.0b0, the release
its fix flow translations are targeted at.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Make fixup failure tests independent of error propagation behavior

Suppress a potential ResolutionFixupError from the failing fixup calls
so the tests pass both while fixup errors are swallowed and once they
propagate to the caller (#7150).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Address review: one recovery folder, check OSError for known issues

Move all local data blocking a mount into a single recovery folder so
the user finds it as one fix: when more than one directory holds data,
later ones become subfolders named after their parent directory
instead of numbered sibling folders.

Also run OSError from the relocation through check_oserror to pick up
known filesystem issues like corruption (bad message).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Only filter Core-facing suggestions on v1 surfaces

Per review: no Core version predating the repair suggestion filtering
in its own fix flow (home-assistant/core#179540) supports the v2 API,
so the Supervisor-side compatibility filter is only needed where old
Core versions actually look. Filter the v1 resolution endpoints and
the legacy websocket issue payloads; the v2 endpoints and v2 event
payloads always carry the full suggestion list. The suggestions for
issue endpoint gets a v1 handler for this.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-19 23:40:47 +02:00

536 lines
20 KiB
Python

"""Tests for resolution manager."""
import asyncio
from typing import Any
from unittest.mock import AsyncMock, PropertyMock, patch
from awesomeversion import AwesomeVersion
import pytest
from supervisor.const import FeatureFlag
from supervisor.coresys import CoreSys
from supervisor.exceptions import ResolutionError
from supervisor.resolution.const import (
ContextType,
IssueType,
SuggestionType,
UnhealthyReason,
UnsupportedReason,
)
from supervisor.resolution.data import Issue, Suggestion
from supervisor.resolution.validate import _migrate_checks_config
def test_properies_unsupported(coresys: CoreSys):
"""Test resolution manager properties unsupported."""
assert coresys.core.supported
coresys.resolution.add_unsupported_reason(UnsupportedReason.OS)
assert not coresys.core.supported
def test_properies_unhealthy(coresys: CoreSys):
"""Test resolution manager properties unhealthy."""
assert coresys.core.healthy
coresys.resolution.add_unhealthy_reason(UnhealthyReason.SUPERVISOR)
assert not coresys.core.healthy
async def test_resolution_dismiss_suggestion(coresys: CoreSys):
"""Test resolution manager suggestion apply api."""
coresys.resolution.add_suggestion(
clear_backup := Suggestion(SuggestionType.CLEAR_FULL_BACKUP, ContextType.SYSTEM)
)
assert coresys.resolution.suggestions[-1].type == SuggestionType.CLEAR_FULL_BACKUP
coresys.resolution.dismiss_suggestion(clear_backup)
assert clear_backup not in coresys.resolution.suggestions
with pytest.raises(ResolutionError):
coresys.resolution.dismiss_suggestion(clear_backup)
async def test_resolution_apply_suggestion(coresys: CoreSys):
"""Test resolution manager suggestion apply api."""
coresys.resolution.add_suggestion(
clear_backup := Suggestion(SuggestionType.CLEAR_FULL_BACKUP, ContextType.SYSTEM)
)
coresys.resolution.add_suggestion(
create_backup := Suggestion(
SuggestionType.CREATE_FULL_BACKUP, ContextType.SYSTEM
)
)
mock_backups = AsyncMock()
mock_health = AsyncMock()
coresys.backups.do_backup_full = mock_backups
coresys.resolution.healthcheck = mock_health
await coresys.resolution.apply_suggestion(clear_backup)
await coresys.resolution.apply_suggestion(create_backup)
assert mock_backups.called
assert mock_health.called
assert clear_backup not in coresys.resolution.suggestions
assert create_backup not in coresys.resolution.suggestions
with pytest.raises(ResolutionError):
await coresys.resolution.apply_suggestion(clear_backup)
async def test_resolution_dismiss_issue(coresys: CoreSys):
"""Test resolution manager issue apply api."""
coresys.resolution.add_issue(
updated_failed := Issue(IssueType.UPDATE_FAILED, ContextType.SYSTEM)
)
assert coresys.resolution.issues[-1].type == IssueType.UPDATE_FAILED
coresys.resolution.dismiss_issue(updated_failed)
assert updated_failed not in coresys.resolution.issues
with pytest.raises(ResolutionError):
coresys.resolution.dismiss_issue(updated_failed)
async def test_resolution_create_issue_suggestion(coresys: CoreSys):
"""Test resolution manager issue and suggestion."""
coresys.resolution.create_issue(
IssueType.UPDATE_ROLLBACK,
ContextType.CORE,
"slug",
[SuggestionType.EXECUTE_REPAIR],
)
assert coresys.resolution.issues[-1].type == IssueType.UPDATE_ROLLBACK
assert coresys.resolution.issues[-1].context == ContextType.CORE
assert coresys.resolution.issues[-1].reference == "slug"
assert coresys.resolution.suggestions[-1].type == SuggestionType.EXECUTE_REPAIR
assert coresys.resolution.suggestions[-1].context == ContextType.CORE
async def test_resolution_dismiss_unsupported(coresys: CoreSys):
"""Test resolution manager dismiss unsupported reason."""
coresys.resolution.add_unsupported_reason(UnsupportedReason.SOFTWARE)
coresys.resolution.dismiss_unsupported(UnsupportedReason.SOFTWARE)
assert UnsupportedReason.SOFTWARE not in coresys.resolution.unsupported
with pytest.raises(ResolutionError):
coresys.resolution.dismiss_unsupported(UnsupportedReason.SOFTWARE)
async def test_suggestions_for_issue(coresys: CoreSys):
"""Test getting suggestions that fix an issue."""
coresys.resolution.add_issue(
corrupt_repo := Issue(
IssueType.CORRUPT_REPOSITORY, ContextType.STORE, "test_repo"
)
)
# Unrelated suggestions don't appear
coresys.resolution.add_suggestion(
Suggestion(SuggestionType.EXECUTE_RESET, ContextType.SUPERVISOR)
)
coresys.resolution.add_suggestion(
Suggestion(SuggestionType.EXECUTE_REMOVE, ContextType.STORE, "other_repo")
)
assert coresys.resolution.suggestions_for_issue(corrupt_repo) == set()
# Related suggestions do
coresys.resolution.add_suggestion(
execute_remove := Suggestion(
SuggestionType.EXECUTE_REMOVE, ContextType.STORE, "test_repo"
)
)
coresys.resolution.add_suggestion(
execute_reset := Suggestion(
SuggestionType.EXECUTE_RESET, ContextType.STORE, "test_repo"
)
)
assert coresys.resolution.suggestions_for_issue(corrupt_repo) == {
execute_reset,
execute_remove,
}
async def test_issues_for_suggestion(coresys: CoreSys):
"""Test getting issues fixed by a suggestion."""
coresys.resolution.add_suggestion(
execute_reset := Suggestion(
SuggestionType.EXECUTE_RESET, ContextType.STORE, "test_repo"
)
)
# Unrelated issues don't appear
coresys.resolution.add_issue(Issue(IssueType.FATAL_ERROR, ContextType.CORE))
coresys.resolution.add_issue(
Issue(IssueType.CORRUPT_REPOSITORY, ContextType.STORE, "other_repo")
)
assert coresys.resolution.issues_for_suggestion(execute_reset) == set()
# Related issues do
coresys.resolution.add_issue(
fatal_error := Issue(IssueType.FATAL_ERROR, ContextType.STORE, "test_repo")
)
coresys.resolution.add_issue(
corrupt_repo := Issue(
IssueType.CORRUPT_REPOSITORY, ContextType.STORE, "test_repo"
)
)
assert coresys.resolution.issues_for_suggestion(execute_reset) == {
fatal_error,
corrupt_repo,
}
def _supervisor_event_message(event: str, data: dict[str, Any]) -> dict[str, Any]:
"""Make mock supervisor event message for ha websocket."""
return {
"type": "supervisor/event",
"data": {
"event": event,
"data": data,
},
}
async def test_events_on_issue_changes(
coresys: CoreSys, supervisor_internet, ha_ws_client: AsyncMock
):
"""Test events fired when an issue changes."""
# Creating an issue with a suggestion should fire exactly one issue changed event
assert coresys.resolution.issues == []
assert coresys.resolution.suggestions == []
coresys.resolution.create_issue(
IssueType.CORRUPT_REPOSITORY,
ContextType.STORE,
"test_repo",
[SuggestionType.EXECUTE_RESET],
)
await asyncio.sleep(0)
assert len(coresys.resolution.issues) == 1
assert len(coresys.resolution.suggestions) == 1
issue = coresys.resolution.issues[0]
suggestion = coresys.resolution.suggestions[0]
issue_expected = {
"type": "corrupt_repository",
"context": "store",
"reference": "test_repo",
"reference_extra": None,
"uuid": issue.uuid,
}
suggestion_expected = {
"type": "execute_reset",
"context": "store",
"reference": "test_repo",
"reference_extra": None,
"uuid": suggestion.uuid,
}
assert _supervisor_event_message(
"issue_changed", issue_expected | {"suggestions": [suggestion_expected]}
) in [call.args[0] for call in ha_ws_client.async_send_command.call_args_list]
# Adding a suggestion that fixes the issue changes it
ha_ws_client.async_send_command.reset_mock()
coresys.resolution.add_suggestion(
execute_remove := Suggestion(
SuggestionType.EXECUTE_REMOVE, ContextType.STORE, "test_repo"
)
)
await asyncio.sleep(0)
messages = [
call.args[0]
for call in ha_ws_client.async_send_command.call_args_list
if call.args[0].get("data", {}).get("event") == "issue_changed"
]
assert len(messages) == 1
sent_data = messages[0]
assert sent_data["type"] == "supervisor/event"
assert sent_data["data"]["event"] == "issue_changed"
assert sent_data["data"]["data"].items() >= issue_expected.items()
assert len(sent_data["data"]["data"]["suggestions"]) == 2
assert suggestion_expected in sent_data["data"]["data"]["suggestions"]
assert {
"type": "execute_remove",
"context": "store",
"reference": "test_repo",
"reference_extra": None,
"uuid": execute_remove.uuid,
} in sent_data["data"]["data"]["suggestions"]
# Removing a suggestion that fixes the issue changes it again
ha_ws_client.async_send_command.reset_mock()
coresys.resolution.dismiss_suggestion(execute_remove)
await asyncio.sleep(0)
assert _supervisor_event_message(
"issue_changed", issue_expected | {"suggestions": [suggestion_expected]}
) in [call.args[0] for call in ha_ws_client.async_send_command.call_args_list]
# Applying a suggestion should only fire an issue removed event.
# Mock healthcheck to avoid running the system-checks fan-out, which is
# not relevant to this assertion (ISSUE_REMOVED is fired by dismiss_issue
# inside the fixup, before apply_suggestion calls healthcheck).
ha_ws_client.async_send_command.reset_mock()
with (
patch("shutil.disk_usage", return_value=(42, 42, 2 * (1024.0**3))),
patch.object(coresys.resolution, "healthcheck", new_callable=AsyncMock),
):
await coresys.resolution.apply_suggestion(suggestion)
await asyncio.sleep(0)
assert _supervisor_event_message("issue_removed", issue_expected) in [
call.args[0] for call in ha_ws_client.async_send_command.call_args_list
]
async def test_resolution_apply_suggestion_multiple_copies(coresys: CoreSys):
"""Test resolution manager applies correct suggestion when has multiple that differ by reference."""
coresys.resolution.add_suggestion(
remove_store_1 := Suggestion(
SuggestionType.EXECUTE_REMOVE, ContextType.STORE, "repo_1"
)
)
coresys.resolution.add_suggestion(
remove_store_2 := Suggestion(
SuggestionType.EXECUTE_REMOVE, ContextType.STORE, "repo_2"
)
)
coresys.resolution.add_suggestion(
remove_store_3 := Suggestion(
SuggestionType.EXECUTE_REMOVE, ContextType.STORE, "repo_3"
)
)
await coresys.resolution.apply_suggestion(remove_store_2)
assert remove_store_1 in coresys.resolution.suggestions
assert remove_store_2 not in coresys.resolution.suggestions
assert remove_store_3 in coresys.resolution.suggestions
async def test_events_on_unsupported_changed(coresys: CoreSys):
"""Test events fired when unsupported changes."""
with patch.object(
type(coresys.homeassistant.websocket), "_async_send_command"
) as send_message:
# Marking system as unsupported tells HA
assert coresys.resolution.unsupported == set()
coresys.resolution.add_unsupported_reason(UnsupportedReason.CONNECTIVITY_CHECK)
await asyncio.sleep(0)
assert coresys.resolution.unsupported == {UnsupportedReason.CONNECTIVITY_CHECK}
send_message.assert_called_once_with(
_supervisor_event_message(
"supported_changed",
{"supported": False, "unsupported_reasons": ["connectivity_check"]},
)
)
# Adding the same reason again does nothing
send_message.reset_mock()
coresys.resolution.add_unsupported_reason(UnsupportedReason.CONNECTIVITY_CHECK)
await asyncio.sleep(0)
assert coresys.resolution.unsupported == {UnsupportedReason.CONNECTIVITY_CHECK}
send_message.assert_not_called()
# Adding and removing additional reasons tells HA unsupported reasons changed
coresys.resolution.add_unsupported_reason(UnsupportedReason.JOB_CONDITIONS)
await asyncio.sleep(0)
assert coresys.resolution.unsupported == {
UnsupportedReason.CONNECTIVITY_CHECK,
UnsupportedReason.JOB_CONDITIONS,
}
send_message.assert_called_once_with(
_supervisor_event_message(
"supported_changed",
{
"supported": False,
"unsupported_reasons": ["connectivity_check", "job_conditions"],
},
)
)
send_message.reset_mock()
coresys.resolution.dismiss_unsupported(UnsupportedReason.CONNECTIVITY_CHECK)
await asyncio.sleep(0)
assert coresys.resolution.unsupported == {UnsupportedReason.JOB_CONDITIONS}
send_message.assert_called_once_with(
_supervisor_event_message(
"supported_changed",
{"supported": False, "unsupported_reasons": ["job_conditions"]},
)
)
# Dismissing all unsupported reasons tells HA its supported again
send_message.reset_mock()
coresys.resolution.dismiss_unsupported(UnsupportedReason.JOB_CONDITIONS)
await asyncio.sleep(0)
assert coresys.resolution.unsupported == set()
send_message.assert_called_once_with(
_supervisor_event_message(
"supported_changed", {"supported": True, "unsupported_reasons": None}
)
)
async def test_events_on_unhealthy_changed(coresys: CoreSys):
"""Test events fired when unhealthy changes."""
with patch.object(
type(coresys.homeassistant.websocket), "_async_send_command"
) as send_message:
# Marking system as unhealthy tells HA
assert coresys.resolution.unhealthy == set()
coresys.resolution.add_unhealthy_reason(UnhealthyReason.DOCKER)
await asyncio.sleep(0)
assert coresys.resolution.unhealthy == {UnhealthyReason.DOCKER}
send_message.assert_called_once_with(
_supervisor_event_message(
"health_changed",
{"healthy": False, "unhealthy_reasons": ["docker"]},
)
)
# Adding the same reason again does nothing
send_message.reset_mock()
coresys.resolution.add_unhealthy_reason(UnhealthyReason.DOCKER)
await asyncio.sleep(0)
assert coresys.resolution.unhealthy == {UnhealthyReason.DOCKER}
send_message.assert_not_called()
# Adding an additional reason tells HA unhealthy reasons changed
coresys.resolution.add_unhealthy_reason(UnhealthyReason.UNTRUSTED)
await asyncio.sleep(0)
assert coresys.resolution.unhealthy == {
UnhealthyReason.DOCKER,
UnhealthyReason.UNTRUSTED,
}
send_message.assert_called_once_with(
_supervisor_event_message(
"health_changed",
{"healthy": False, "unhealthy_reasons": ["docker", "untrusted"]},
)
)
async def test_dismiss_issue_removes_orphaned_suggestions(coresys: CoreSys):
"""Test dismissing an issue also removes any suggestions which have been orphaned."""
with patch.object(
type(coresys.homeassistant.websocket), "_async_send_command"
) as send_message:
coresys.resolution.create_issue(
IssueType.MOUNT_FAILED,
ContextType.MOUNT,
"test",
[SuggestionType.EXECUTE_RELOAD, SuggestionType.EXECUTE_REMOVE],
)
await asyncio.sleep(0)
assert len(coresys.resolution.issues) == 1
assert len(coresys.resolution.suggestions) == 2
send_message.assert_called_once()
send_message.reset_mock()
issue = coresys.resolution.issues[0]
coresys.resolution.dismiss_issue(issue)
await asyncio.sleep(0)
# The issue and both suggestions should be dismissed as they are now orphaned
assert coresys.resolution.issues == []
assert coresys.resolution.suggestions == []
# Only one message should fire to tell HA the issue was removed
send_message.assert_called_once_with(
_supervisor_event_message(
"issue_removed",
{
"type": "mount_failed",
"context": "mount",
"reference": "test",
"reference_extra": None,
"uuid": issue.uuid,
},
)
)
@pytest.mark.parametrize(
("legacy_slug", "new_slug"),
[
("addon_pwned", "app_pwned"),
("deprecated_addon", "deprecated_app"),
("deprecated_arch_addon", "deprecated_arch_app"),
("detached_addon_missing", "detached_app_missing"),
("detached_addon_removed", "detached_app_removed"),
],
)
def test_resolution_file_migration_legacy_check_slugs(legacy_slug: str, new_slug: str):
"""Test that resolution.json with legacy check slugs is migrated to new names."""
# Create a checks config with legacy slug
legacy_config = {legacy_slug: {"enabled": False}}
# Migrate it using the same function used in schema validation
migrated_config = _migrate_checks_config(legacy_config)
# Verify the legacy slug was migrated to the new slug
assert new_slug in migrated_config
assert legacy_slug not in migrated_config
assert migrated_config[new_slug]["enabled"] is False
async def test_core_compatible_suggestions(coresys: CoreSys):
"""Test suggestions gated on a minimum Core version are filtered."""
coresys.resolution.add_issue(
issue := Issue(IssueType.MOUNT_FAILED, ContextType.MOUNT, reference="test"),
suggestions=[SuggestionType.MOVE_LOCAL_DATA, SuggestionType.EXECUTE_RELOAD],
)
for version, expected_types in [
(None, {SuggestionType.EXECUTE_RELOAD}),
(AwesomeVersion("landingpage"), {SuggestionType.EXECUTE_RELOAD}),
(AwesomeVersion("2026.8.3"), {SuggestionType.EXECUTE_RELOAD}),
(
AwesomeVersion("2026.9.0b0"),
{SuggestionType.EXECUTE_RELOAD, SuggestionType.MOVE_LOCAL_DATA},
),
(
AwesomeVersion("2026.10.1"),
{SuggestionType.EXECUTE_RELOAD, SuggestionType.MOVE_LOCAL_DATA},
),
]:
with patch.object(
type(coresys.homeassistant),
"version",
new=PropertyMock(return_value=version),
):
assert {
suggestion.type
for suggestion in coresys.resolution.core_compatible_suggestions(
coresys.resolution.suggestions_for_issue(issue)
)
} == expected_types, f"unexpected filtering for Core {version}"
# The legacy issue event payload applies the same filter
message = coresys.resolution._make_issue_message(issue) # pylint: disable=protected-access
assert {
suggestion["type"] for suggestion in message["suggestions"]
} == expected_types
# With the v2 API enabled the Core filters itself — no filtering
coresys.config.set_feature_flag(
FeatureFlag.SUPERVISOR_WEBSOCKET_V2_API, True
)
message = coresys.resolution._make_issue_message(issue) # pylint: disable=protected-access
assert {suggestion["type"] for suggestion in message["suggestions"]} == {
SuggestionType.EXECUTE_RELOAD,
SuggestionType.MOVE_LOCAL_DATA,
}
coresys.config.set_feature_flag(
FeatureFlag.SUPERVISOR_WEBSOCKET_V2_API, False
)