diff --git a/README.md b/README.md index 160f3b5..8531803 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,7 @@ git clone git@github.com:pi-hole/docs.git cd docs sudo pip install mkdocs sudo pip install mkdocs-material -mkdocs serve +mkdocs serve --dev-addr 0.0.0.0:8000 ``` Deploying to GitHub pages: diff --git a/docs/extra.css b/docs/extra.css index ebe9442..8b0999f 100644 --- a/docs/extra.css +++ b/docs/extra.css @@ -11,7 +11,8 @@ body, input { font-family: "Source Sans Pro", "Roboto","Helvetica Neue",Helvetica,Arial,sans-serif; color: black; } -code { +.md-typeset code, .md-typeset pre { display: inline-block; white-space: pre-wrap; -} \ No newline at end of file + color: rgb(83, 43, 168); +} diff --git a/docs/ftldns/blockingmode.md b/docs/ftldns/blockingmode.md new file mode 100644 index 0000000..244f5ac --- /dev/null +++ b/docs/ftldns/blockingmode.md @@ -0,0 +1,44 @@ +Pi-hole *FTL*DNS supports two different methods for blocking queries. Both have their advantages and drawbacks. They are summarized on this page. The blocking mode can be configured in `/etc/pihole/pihole-FTL.conf`. + +## Pi-hole's IP based blocking +`/etc/pihole/pihole-FTL.conf` setting: +``` +BLOCKINGMODE=IP +``` + +Queries will be answered with the local IP addresses of your Pi-hole (as configured in your `setupVars.conf` file) +``` +;; QUESTION SECTION: +;doubleclick.net. IN ANY + +;; ANSWER SECTION: +doubleclick.net. 2 IN A 192.168.2.11 +doubleclick.net. 2 IN AAAA fda2:2001:4756:0:ab27:beff:ef37:4242 +``` + +##### Advantage +- Shows blocking page from which blocked webpages can be whitelisted + +##### Disadvantages +- Requires a webserver to run on your Pi-hole +- May cause time-outs for HTTPS content even with properly configured firewall rules + +## Pi-hole's NXDOMAIN based blocking +`/etc/pihole/pihole-FTL.conf` setting: +``` +BLOCKINGMODE=NXDOMAIN +``` +Queries DNS queries will be answered with an empty response (no answer section) and status `NXDOMAIN` (*no such domain*) +``` +;; QUESTION SECTION: +;doubleclick.net. IN ANY +``` + +##### Advantages +- The client does not even try to establish a connection for the requested website +- Speedup and less traffic +- Solves potential HTTPS timeouts as requests are never performed +- No need to run a webserver on your Pi-hole (reduces complexity when running other web services on the same machine) + +##### Disadvantage +- Blocking page cannot be shown and whitelisting has to be performed from the dashboard or CLI diff --git a/docs/ftldns/in-depth.md b/docs/ftldns/in-depth.md new file mode 100644 index 0000000..a54e894 --- /dev/null +++ b/docs/ftldns/in-depth.md @@ -0,0 +1,19 @@ +## Domain lists format +Since Pi-hole v4.0, we use a simpler domain list format for the two important block list files `gravity.list` and `black.list`. In contrast to the traditional HOSTS format (which caused a lot of overhead), the domain list format is the minimal possible solution for saving memory while still using plain text lists for your convenience. When *FTL*DNS imports these two files, they are walked by our improved list parser speeding up the loading of block lists significantly. Regardless which blocking mode (`IP` or `NXDOMAIN`) is selected, *FTL*DNS will always load the lists into it's internal hashed cache to be able to determine the blocking status within a few milliseconds, even when you're using huge blocking lists on low-end devices. With everything we do, we design *FTL*DNS for maximum efficiency also on low-performance devices. + +## Linux capabilities +Capabilities (POSIX 1003.1e, [capabilities(7)](http://man7.org/linux/man-pages/man7/capabilities.7.html)) provide fine-grained control over superuser permissions, allowing use of the `root` user to be avoided. +For the purpose of performing permission checks, traditional UNIX implementations distinguish two categories of processes: *privileged processes* (superuser or `root`), and *unprivileged processes*. Privileged processes bypass all kernel permission checks, while unprivileged processes are subject to full permission checking based on the process's credentials (user and group permissions and supplementary process capabilities). Capabilities are implemented on Linux using extended attributes ([xattr(7)](http://man7.org/linux/man-pages/man5/attr.5.html)) in the `security` namespace. Extended attributes are supported by all major Linux file systems, including Ext2, Ext3, Ext4, Btrfs, JFS, XFS, and ReiserFS. + +For your safety and comfort, `pihole-FTL` is run by the entirely unprivileged user `pihole`. +Whereas `dnsmasq` is running as `root` process, we designed `pihole-FTL` to be run by the entirely unprivileged user `pihole`. As a consequence, `pihole-FTL` will not be able to access the files of any other user on this system or mess around with your system's configuration. + +However, this also implies that *FTL*DNS cannot bind to ports 53 (DNS) among some other necessary capabilities related to DHCP services. To establish a strong security model, we explicitly grant the `pihole-FTL` process additional capabilities so that `pihole-FTL` (but no other processes which may be started by `pihole`) can bind to port 53, etc., without giving any additional permissions to the `pihole` user. + +We specifically add the following capabilities to `pihole-FTL`: + +- `CAP_NET_BIND_SERVICE`: Allows *FTL*DNS binding to TCP/UDP sockets below 1024 (specifically DNS service on port 53) +- `CAP_NET_RAW`: use RAW and PACKET sockets (we need a RAW socket for handling DHCPv6 requests) +- `CAP_NET_ADMIN`: modify routing tables and other network-related operations (to allow for handling DHCP requests) + +Users that cannot use Linux capabilites for various reasons (lacking kernel or file system support) can modify the startup scripts of `pihole-FTL` to ensure the daemon is started as `root`. However, be aware of that you do so on your own risk (although we don't expect problems to arise). diff --git a/docs/ftldns/index.md b/docs/ftldns/index.md new file mode 100644 index 0000000..7af0648 --- /dev/null +++ b/docs/ftldns/index.md @@ -0,0 +1,5 @@ +

FTLDNS
powered by Pi-hole®

+ +*FTL*DNS[™](https://pi-hole.net/trademark-rules-and-brand-guidelines/) (`pihole-FTL`) offers DNS services within the Pi-hole[®](https://pi-hole.net/trademark-rules-and-brand-guidelines/) project. +It provides blazing fast DNS and DHCP services. It can also provide TFTP and more as the resolver part based on the popular `dnsmasq`. Furthermore, FTL offers an interactive API where extensive network analysis data and statistics may be queried. + diff --git a/docs/ftldns/regex.md b/docs/ftldns/regex.md new file mode 100644 index 0000000..8b7ac78 --- /dev/null +++ b/docs/ftldns/regex.md @@ -0,0 +1,134 @@ +A regular expression, or RegEx for short, is a pattern that can be used for building arbitrarily complex blocking rules in *FTL*DNS. +We implement the Extended Regular Expressions (ERE) flavor similar to the one used by the UNIX `egrep` (or `grep -E`) command. + +Our implementation is computationally inexpensive as each domain is only checked once for a match (if you query `google.com`, it will be checked against your RegEx. Any subsequent query to the same domain will not be checked again until you restart `pihole-FTL`). + +## How to use regular expressions for blocking +Add a line +``` +BLOCKINGREGEX=^abc$ +``` +in your `/etc/pihole/pihole-FTL.conf` and restart `pihole-FTL`. **Note: this is subject to change (we plan to support a list of regular expressions for blocking in the near future).** + +## Pi-hole regular expressions tutorial +We provide a short but thorough introduction to our regular expressions implementation. This may come in handy if you are designing blocking rules (see also our cheat sheet below!). In our implementation, all characters match themselves except for the following special characters: `.[{}()\*+?|^$`. If you want to match those, you need to escape them like `\.` for a literal period, but no rule without exception (see character groups below for further details). + +### Anchors (`^` and `$`) +First of all, we look at anchors which can be used to indicate the start or the end of a domain, respectively. If you don't specify anchors, the match may be partial (see examples below). + +Example | Interpretation +--- | --- +`domain` | **partial match**. Without anchors, a text may appear anywhere in the domain. This matches `some.domain.com`, `domain.com` and `verylongdomain.com` and more +`^localhost$` | **exact match** matching *only* `localhost` but neither `a.localhost` nor `localhost.com` +`^abc` | matches any domain **starting** (`^`) in "abc" like `abcdomain.com`, `abc.domain.com` but not `def.abc.com` +`com$` | matches any domain **ending** (`$`) in "com" such as `domain.com` but not `domain.com.co.uk` + +### Wildcard (`.`) +An unescaped period stands for any *single* character. + +Example | Interpretation +--- | --- +`^domain.$` | matches `domaina`, `domainb`, `domainc`, but not `domain` + +### Bounds and multipliers (`{}`, `*`, `+`, and `?`) +With bounds, one can denote the number of times something has to occur: + +Bound | Meaning +--- | --- +`ab{4}` | matches a domain that contains a single `a` followed by four `b` (matching only `abbbb`) +`ab{4,}` | matches a domain that contains a single `a` followed by *at least* four `b` (matching also `abbbbbbbb`) +`ab{3,5}` | matches a domain that contains a single `a` followed by three to five `b` (matching only `abbb`, `abbbb`, and `abbbbb`) + +Multipliers are shortcuts for some of the bounds that are needed most often: + +Multipliers | Bounds equivalent | Meaning +--- | --- | --- +`?` | `{0,1}` | never or once (optional) +`*` | `{0,}` | never or more (optional) +`+` | `{1,}` | once or more (mandatory) + +To illustrate the usefulness of multipliers (and bounds), we provide a few examples: + +Example | Interpretation +--- | --- +`^r-*\.movie` | matches a domain like `r------movie.com` where the number of dashes can be arbitrary (also none) +`^r-?\.movie` | matches only the domains `rmovie.com` and `r-movie.com` but not those with more than one dash +`^r-+\.movie` | matches only the domains with at least one dash, i.e., not `rmovie.com` +`^a?b+` | matches domains like `abbbb.com` (zero or one `a` at the beginning followed by one or more `b`) + +### Character groups (`[]`) +With character groups, a set of characters can be matched: + +Character group | Interpretation +--- | --- +`[abc]` | matches `a`, `b`, or `c` (using explicitly specified characters) +`[a-c]` | matches `a`, `b`, or `c` (using a *range*) +`[a-c]+` | matches any non-zero number of `a`, `b`, `c` +`[a-z]` | matches any single lowercase letter +`[a-zA-Z]` | matches any single letter +`[a-z0-9]` | matches any single lowercase letter or any single digit +`[^a-z]` | **Negation** matching any single character *except* lowercase letters +`abc[0-9]+` | matches the string `abc` followed by a number of arbitrary length + +Bracket expressions are an exception to the character escape rule. Inside them, all special characters, including the backslash (`\`), lose their special powers, i.e. they match themselves exactly. Furthermore, to include a literal `]` in the list, make it the first character (like `[]]` or `[^]]` if negated). To include a literal `-`, make it the first or last character, or the second endpoint of a range (e.g. `[a-z-]` to match `a` to `z` and `-`). + +### Groups (`()`) +Using groups, we can enclose regular expressions, they are most powerful when combined with bounds or multipliers (see also alternations below). + +Example | Interpretation +--- | --- +`(abc)` | matches `abc` (trivial example) +`(abc)*` | matches zero or more copies of `abc` like `abcabc` but not `abcdefabc` +`(abc){1,3}` | matches one, two or three copies of `abc`: `abc`, `abcabc`, `abcabcabc` but nothing else + +### Alternations (`|`) +Alternations can be used as an "or" operator in regular expressions. + +Example | Interpretation +--- | --- +`(abc)|(def)` | matches `abc` *and* `def` +`domain(a|b)\.com` | matches `domaina.com` and `domainb.com` but not `domain.com` or `domainx.com` +`domain(a|b)*\.com` | matches `domain.com`, `domainaaaa.com` `domainbbb.com` but not `domainab.com` (any number of `a` or `b` in between `domain` and `.com`) + +### Character classes (`[:class:]`) +In addition to character groups, there are also some special character classes available, such as + +Character class | Group equivalent | Interpretation +--- | --- | --- +`[:digit:]` | `[0-9]` | matches digits +`[:lower:]` | `[a-z]` | matched lowercase letters +`[:upper:]` | `[A-Z]` | matched uppercase letters +`[:alpha:]` | `[A-Za-z]` | matches alphabetic characters +`[:alnum:]` | `[A-Za-z0-9]` | matches alphabetic characters and digits + +## Advanced examples +After going through our quick tutorial, we provide some more advances examples so you can test your knowledge. + +### Block domain with only numbers +``` +^[0-9][^a-z]+\.((com)|(edu))$ +``` +Blocks domains containing only numbers (no letters) and ending in `.com` or `.edu`. Blocks `555661.com`, and `456.edu`, but not `555g555.com` + +### Block domains without subdomains +``` +^[a-z0-9]+([\-]{1}[a-z0-9]+)*\.[a-z]{2,7}$ +``` +A domain name shall not start or end with a dash but can contain any number of them. It must be followed by a TLD (we assume a valid TLD length of two to seven characters) + +## Cheatsheet + +Expression | Meaning | Example +------------ | ------------- | ----------- +`^` | Beginning of string | `^client` matches strings that begin with `client`, such as `client.server.com` but not `more.client.server.com` (exception: within a character range (`[]`) `^` means negation) +`$` | End of string | `ing$` matches `exciting` but not `ingenious` +`*` | Match zero or more of the previous | `ah*` matches `ahhhhh` or `a` +`?` | Match zero or one of the previous | `ah?` matches `a` or `ah` +`+` | Match one or more of the previous | `ah+` matches `ah` or `ahhh` but not `a` +`.` | Wildcard character, matches any character | `do.*` matches `do`, `dog`, `door`, `dot`, etc.;
`do.+` matches `dog`, `door`, `dot`, etc. but not `do` (wildcard with `+` requires at least one extra character for matching) +`( )` | Group | Enclose regular expressions, see the example for `|` +`|` | Alternation | `(mon|tues)day` matches `monday` or `tuesday` but not `friday` or `mondiag` +`[ ]` | Matches a range of characters | `[cbf]ar` matches `car`, `bar`, or `far`; +`[^]`| Negation | `[^0-9]` matches any character *except* `0` to `9` +`{ }` | Matches a specified number of occurrences of the previous | `[0-9]{3}` matches any three-digit number like `315` but not `31`;
`[0-9]{2,4}` matches two- to four-digit numbers like `12`, `123`, and `1234` but not `1` or `12345`;
`[0-9]{2,}` matches any number with two or more digits like `1234567`, `123456789`, but not `1` +`\` | Used to escape a special character not inside `[]` | `google\.com` matches `google.com` diff --git a/docs/images/FTLDNS.png b/docs/images/FTLDNS.png new file mode 100644 index 0000000..2e31463 Binary files /dev/null and b/docs/images/FTLDNS.png differ diff --git a/mkdocs.yml b/mkdocs.yml index 65f7992..dbabb98 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -24,7 +24,16 @@ pages: - 'Installation': 'main/basic-install.md' - 'Post-Install': 'main/post-install.md' - 'Updating': 'main/update.md' +- 'FTLDNS': + - 'Overview': "ftldns/index.md" + - 'Blocking mode': "ftldns/blockingmode.md" + - 'RegEx blocking': "ftldns/regex.md" +# - 'Privacy levels': "ftldns/privacy.md" +# - 'Long-term database': "ftldns/database.md" +# - 'API documentation': "ftldns/api.md" + - 'In-depth manual': "ftldns/in-depth.md" - 'Guides': +# - 'Pi-hole as All-Around DNS Solution': guides/unbound.md - 'Pi-hole and OpenVPN Server': - 'Overview': 'guides/vpn/overview.md' - 'Installation': 'guides/vpn/installation.md'