The API query-count assertions depend on resolving mask.icloud.com, whose
mask.icloud.com -> mask.apple-dns.net CNAME chain was recursed to the public
internet. dnsmasq fires extra DNSKEY validation queries depending on whether
Apple currently DNSSEC-signs icloud.com / apple-dns.net, and Apple toggles this
over time. The runtime DS-probing workaround in conftest.py could not reliably
model dnsmasq's behaviour (e.g. when Apple returns SERVFAIL on DS), so the suite
went flaky again.
Serve the icloud.com and apple-dns.net zones from the local authoritative
PowerDNS server instead, so the chain resolves hermetically and the query counts
are deterministic regardless of Apple's upstream DNSSEC posture. The DS-probing
fixture is dropped and the expected counters become fixed constants again.
Signed-off-by: DL6ER <dl6er@dl6er.de>
The pytest API tests assert exact query counts that depend on whether icloud.com and apple-dns.net are DNSSEC-signed. When Apple removed DNSSEC from those zones (April 2026), dnsmasq stopped firing two DNSKEY validation queries during the mask.icloud.com CNAME chain walk, breaking 7 tests on every CI run - including re-runs of previously green commits.
Instead of hardcoding either set of numbers, detect the current DNSSEC state at test startup by querying the local pdns_recursor (port 5555, bypassing FTL to avoid counter pollution) for DS records on both domains. Four module-level constants (TOTAL, FORWARDED, DNSKEY, TOP_DOMAIN) are set accordingly, and the 11 affected assertions now reference these constants.
The bats "Special domain: Record is returned when explicitly allowed" test is preserved unchanged - the hybrid detection makes it safe regardless of upstream DNSSEC posture.
Signed-off-by: Dominik <dl6er@dl6er.de>
Add missing NULL check after strdup() when making a writable copy of
cJSON reference strings in api_list_write(). Without this, a failed
allocation under memory pressure would dereference NULL in the
lowercasing loop. Also fix minor docstring typo in punycode test.
Signed-off-by: Dominik <dl6er@dl6er.de>
When adding or searching for exact domains, the API unconditionally
passes the input through idn2_to_ascii_lz() for IDN normalization.
This round-trips punycode domains: decode to Unicode, validate against
IDNA2008, re-encode to ASCII. Characters like emoji are disallowed by
IDNA2008 (RFC 5892), so valid punycode domains such as
xn--4ca0bs45142c.com (äöü😀.com) are rejected with "string contains
a disallowed character" even though they are perfectly valid DNS names.
Fix by checking whether the input is already pure ASCII before calling
idn2_to_ascii_lz(). If every byte is <= 0x7F, skip IDN conversion
entirely — the domain is already in a DNS-compatible form and only
needs lowercasing and valid_domain() validation. Non-ASCII input
(actual Unicode domains) still goes through the IDN conversion path.
Applied to both the list API (src/api/list.c) and the search API
(src/api/search.c).
Fixes: https://github.com/pi-hole/FTL/issues/2837
Signed-off-by: Dominik <dl6er@dl6er.de>
Replaces the generic parallel auth tests with targeted tests that
exercise the specific race conditions fixed by PR #2835:
- Set-Cookie SID mismatch from pi_hole_extra_headers global buffer race
- Corrupted session listings from unsynchronised get_all_sessions() with
JSON_REF_STR_IN_OBJECT reading auth_data[] during concurrent deletes
- HTTP 200 with valid=false from api->session pointer going stale when
delete_session() memsets the slot between check_client_auth() and
get_session_object()
Uses barrier-synchronised bursts and correctness checks (not just
crash detection) so the tests fail without the fix and pass with it.
Signed-off-by: Dominik <dl6er@dl6er.de>
Add test/api/test_s_auth_stress.py exercising the auth subsystem under
concurrent load (12 threads, kept below max_sessions=16). Sessions are
created sequentially (respecting FTL's 3-attempts/s rate limit with
retry-on-429 backoff), then concurrency targets the thread-safety
surfaces:
- Parallel session validation (concurrent GET /api/auth with SIDs)
- Concurrent logout with cross-session isolation checks (logout half,
verify other half survives)
- Mixed session check/logout operations from a shared session pool
- Concurrent wrong-password rejection (verifies no SIGSEGV; runs last
since it intentionally triggers rate-limiting)
Every test cleans up its sessions to avoid exhausting max_sessions
across test boundaries. Bump expected config rotation count in
test_final.bats from 14 to 16 for the stress test's password
set/remove cycle.
Also add test/libs/ to .gitignore per review feedback — the directory is
populated at test time by test/run.sh cloning bats-core.
See alse: https://github.com/pi-hole/FTL/pull/2835
Signed-off-by: Dominik <dl6er@dl6er.de>
test_openapi.py: Store auth_method per endpoint alongside errors so
the assertion message shows the correct auth method for each failing
endpoint instead of the last loop iteration's value.
run.sh: Update comment to reflect that BATS no longer terminates FTL
(termination was moved to test_final.bats).
Signed-off-by: Dominik <dl6er@dl6er.de>
Add pytest tests for all previously untested GET API endpoints:
dns/blocking, domains (all type/kind combinations and single lookup),
groups, stats/summary, stats/top_domains, stats/top_clients,
stats/upstreams, stats/query_types, stats/recent_blocked,
stats/database (error handling), dhcp/leases, endpoints, info/ftl,
info/login, info/version, info/messages, info/client, info/database,
info/system, network/devices, network/interfaces, logs (dnsmasq, ftl,
webserver), and padd.
All assertions use exact expected values derived from the deterministic
BATS DNS query seeding (137 total queries, 49 blocked, 47 forwarded,
41 cached, 11 active clients, 8 gravity domains). On failure, the
full JSON response is dumped to /tmp/ftl_test_*.json for easy
inspection.
Fix double-free bug in printFTLenv() (src/config/env.c): when
printFTLenv() was called more than once (e.g. after config reload
triggered by the CLI password test), it would free item->error a
second time because neither the pointer nor the error_allocated flag
were reset after the first free. This produced "Trying to free NULL
pointer in printFTLenv()" warnings. Fix: set item->error = NULL and
item->error_allocated = false after freeing.
Files modified:
src/config/env.c — reset error/error_allocated after free
test/api/test_api.py — add 34 new endpoint tests (22 -> 56 total)
Signed-off-by: Dominik <dl6er@dl6er.de>
Move all API-related tests (HTTP endpoints, config validation, auth,
search, history, lists, Lua pages, OpenAPI spec validation) from BATS
shell tests to pytest. BATS retains DNS, regex, CLI, and system-level
tests. The auth test suite now removes the password at the end, leaving
no net state change.
Fix ResponseVerifyer to respect the OpenAPI "required" field: optional
properties absent from FTL's response are silently skipped instead of
flagged as errors. The required list is propagated through all recursive
calls (top-level objects, allOf, nested objects, array items).
Add a set_config() helper that changes FTL configuration via the API
instead of shelling out to the CLI. The API PATCH is synchronous, so no
log polling or sleeps are needed.
Add build.sh test-api target to run only the pytest API tests (skips
BATS and perf tests).
Test migration mapping (25 BATS tests removed, all covered in pytest):
| # | Removed BATS test | Pytest counterpart |
|---|---|---|
| 1 | HTTP server responds with JSON error 404 to unknown API path | TestHTTPErrors::test_api_404_returns_json |
| 2 | HTTP server responds with error 404 to path outside /admin | TestHTTPErrors::test_non_admin_path_returns_404 |
| 3 | Config validation working on the API (type-based checking) | TestConfigValidationAPIType (2 tests) |
| 4 | Config validation working on the API (validator-based checking) | TestConfigValidationAPIValidator (4 tests) |
| 5 | Changing a config option set forced by ENVVAR is not possible via the API | TestEnvvarProtectedConfig::test_api_rejects_envvar_override |
| 6 | API domain search: Non-existing domain | TestDomainSearch::test_nonexistent_domain |
| 7 | API domain search: antigravity.ftl | TestDomainSearch::test_antigravity_domain |
| 8 | API domain search: Internationalized/partially capital domain | TestDomainSearch::test_punycode_normalization |
| 9 | API history: Returns full 24 hours | TestHistory::test_history_returns_24h |
| 10 | API history/clients: Returns full 24 hours | TestHistory::test_history_clients_returns_24h |
| 11 | Check /api/lists?type=block | TestLists::test_block_lists_only |
| 12 | Check /api/lists?type=allow | TestLists::test_allow_lists_only |
| 13 | Check /api/lists without type parameter | TestLists::test_all_lists_includes_both_types |
| 14 | API: No UNKNOWN reply in API | TestQueries::test_no_unknown_reply |
| 15 | API: No UNKNOWN status in API | TestQueries::test_no_unknown_status |
| 16 | Lua server page outside /admin is not served by default | TestLuaServerPages::test_lua_page_outside_admin_not_served_by_default |
| 17 | Lua server page is generating proper backtrace | TestLuaServerPages::test_lua_page_generates_proper_backtrace |
| 18 | Lua server page outside of webhome is served without login | TestLuaServerPages::test_lua_page_outside_webhome_served_without_login |
| 19 | API validation (checkAPI.py) | TestEndpointCoverage (3 tests) + TestEndpointResponses + TestTeleporter |
| 20 | API authorization (without password): No login required | TestAuthWorkflow::test_01_no_password_means_session_valid |
| 21 | Create, set, and use application password | TestAuthWorkflow::test_02 + test_03 + test_04 |
| 22 | CLI password file is as expected | TestAuthWorkflow::test_04b_cli_password_file |
| 23 | API authorization: Setting password | TestAuthWorkflow::test_05_set_password |
| 24 | API authorization (with password): Incorrect password is rejected | TestAuthWorkflow::test_06_incorrect_password_rejected |
| 25 | API authorization (with password): Correct password is accepted | TestAuthWorkflow::test_07_correct_password_accepted |
New tests without a BATS predecessor:
- TestAuthWorkflow::test_08_rate_limiting_enforced
- TestAuthWorkflow::test_09_remove_password
- TestAuthWorkflow::test_10_no_password_after_removal
Final count: 192 BATS + 38 pytest = 230 total (was 216 BATS). No tests
lost, 14 net new.
Signed-off-by: Dominik <dl6er@dl6er.de>
Add new config option webserver.api.client_history_global_max controling if the activities chart should sort and show the *global* (integrated over 24 hours) or the `local` (measured individually in each time slot) most active clients
Allow setting webserver.api.maxClients to 0 to always return all clients in /api/history/clients
Signed-off-by: DL6ER <dl6er@dl6er.de>
DL6ER
copilot-swe-agent[bot]
Mohammed Alzahrani
Dominik
Christian König