Merge branch 'master' into ftldns/debug

Signed-off-by: DL6ER <dl6er@dl6er.de>

Conflicts:
	mkdocs.yml
This commit is contained in:
DL6ER
2018-05-05 23:31:22 +02:00
7 changed files with 357 additions and 3 deletions

View File

@@ -0,0 +1,22 @@
# *FTL*DNS compatibility list
We tested *FTL*DNS on the following devices:
| Board | Tested OS | CPU architecture | Suitable binaries
|---|---|---|---
| VirtualBox | Ubuntu 16.10 | amd64 | `linux-x86_64`
| Raspberry Pi Zero | Raspbian Jessie, Stretch | armv6l | `arm-linux-gnueabi`
| Raspberry Pi 1 | Raspbian Jessie, Stretch | armv6 | `arm-linux-gnueabi`
| Raspberry Pi 2 | Raspbian Jessie, Stretch | armv7l | `arm-linux-gnueabihf` and `arm-linux-gnuabi`
| Raspberry Pi 3 | Raspbian Jessie, Stretch | armv7l | `arm-linux-gnuabi` and `arm-linux-gnueabihf`
| Raspberry Pi 3 B+ | Raspbian Jessie, Stretch | armv7l | `arm-linux-gnuabi` and `arm-linux-gnueabihf`
| Raspberry Pi 3 | openSUSE | aarch64 | `aarch64-linux-gnu`
| NanoPi NEO | armbian Ubuntu 16.04 | armv7l | `arm-linux-gnueabihf`
| Odroid-C2 | Ubuntu 16.04 | aarch64 | `aarch64-linux-gnu`
| C.H.I.P | Debian | armv7l | `arm-linux-gnueabihf`
| OrangePi Zero | armbian Ubuntu 16.04 | armv7l | `arm-linux-gnueabihf`
| BeagleBone Black| Debian Jessie, Stretch | armv7l | `arm-linux-gnueabihf`
<!-- | | | | | | -->
Devices we do not officially support include MIPS and `armv5` (or lower) devices. You may, however, be successful with building binaries yourself from the source code, but we do not provide pre-built binaries for these targets.

38
docs/ftldns/compile.md Normal file
View File

@@ -0,0 +1,38 @@
We pre-compile *FTL*DNS for you to save you the trouble of compiling anything yourself. However, sometimes you may want to make your own modifications. To test them, you have to compile *FTL*DNS from source. Luckily, you don't have to be a programmer to build *FTL*DNS from source and install it on your system; you only have to know the basics we provide in here. With just a few commands, you can build *FTL*DNS from source like a pro.
#### Installing the Required Software
First, we'll install the basic software you'll need to compile from source, like the GCC compiler and other utilities.
Install them by running the following command in a terminal:
```
sudo apt install build-essential libgmp-dev m4
```
You'll also need to compile a recent version of `nettle` as *FTL*DNS uses `libnettle` for handling DNSSEC. Compile and install a recent version of `nettle` (we tested 3.4):
```
wget https://ftp.gnu.org/gnu/nettle/nettle-3.4.tar.gz
tar -xvzf nettle-3.4.tar.gz
cd nettle-3.4
./configure
make
sudo make install
```
#### Get the *FTL*DNS source
Now, clone the *FTL*DNS repo (or your own fork) to get the source code of *FTL*DNS:
```
git clone https://github.com/pi-hole/FTL.git
cd FTL
```
If you want to build another branch and not `master`, use checkout to get to this branch (e.g. `git checkout development`).
*FTL*DNS can now be compiled and installed:
```
make -j 4
sudo make install
```
Finally, restart *FTL*DNS to use the new binary:
```
sudo service pihole-FTL restart
```

86
docs/ftldns/configfile.md Normal file
View File

@@ -0,0 +1,86 @@
You can create a file `/etc/pihole/pihole-FTL.conf` that will be read by *FTL*DNS on startup.
Possible settings (**the option shown first is the default**):
### SOCKET_LISTENING
`SOCKET_LISTENING=localonly|all`
Listen only for local socket connections or permit all connections
### QUERY_DISPLAY
`QUERY_DISPLAY=yes|no`
Display all queries? Set to `no` to hide query display
### AAAA_QUERY_ANALYSIS
`AAAA_QUERY_ANALYSIS=yes|no`
Allow `FTL` to analyze AAAA queries from pihole.log?
### RESOLVE_IPV6
`RESOLVE_IPV6=yes|no`
Should `FTL` try to resolve IPv6 addresses to host names?
### RESOLVE_IPV4
`RESOLVE_IPV4=yes|no`
Should `FTL` try to resolve IPv4 addresses to host names?
### MAXDBDAYS
`MAXDBDAYS=365`
How long should queries be stored in the database?
Setting this to `0` disables the database
**[More details](database.md)**
### DBINTERVAL
`DBINTERVAL=1.0`
How often do we store queries in FTL's database [minutes]?
**[More details](database.md)**
### DBFILE
`DBFILE=/etc/pihole/pihole-FTL.db`
Specify path and filename of FTL's SQLite long-term database. Setting this to `DBFILE=` disables the database altogether
**[More details](database.md)**
### MAXLOGAGE
`MAXLOGAGE=24.0`
Up to how many hours of queries should be imported from the database and logs? Maximum is 744 (31 days)
### FTLPORT
`FTLPORT=4711`
On which port should FTL be listening?
### PRIVACYLEVEL
`PRIVACYLEVEL=0|1|2|3`
Which privacy level is used?
**[More details](privacylevels.md)**
### IGNORE_LOCALHOST
`IGNORE_LOCALHOST=no|yes`
Should `FTL` ignore queries coming from the local machine?
### BLOCKINGMODE
`BLOCKINGMODE=IP|NXDOMAIN`
Should `FTL` reply queries to blocked domains with IPs or `NXDOMAIN`?
**[More details](blockingmode.md)**
### BLOCKINGREGEX
`BLOCKINGREGEX=...`
If defined, `FTL` will block all incoming queries which match this RegEx.
**[More details](regex.md)**

View File

@@ -1,3 +1,28 @@
## Available interfaces
Pi-hole stats can be accessed via a standard Unix socket (`var/run/pihole/FTL.sock`), a telnet-like connection (TCP socket on port `4711`) as well as indirectly via the Web API (`admin/api.php`), and command line (`pihole -c -j`). You can out find more details below.
## Command line arguments
- `debug` - Don't go into daemon mode (stay in foreground) + more verbose logging
- `test` - Start `FTL` and process everything, but shut down immediately afterwards
- `version` - Don't start `FTL`, show only version
- `tag` - Don't start `FTL`, show only git tag
- `branch` - Don't start `FTL`, show only git branch `FTL` was compiled from
- `no-daemon` or `-f` - Don't go into background (daemon mode)
- `help` or `-h` - Don't start `FTL`, show help
- `dnsmasq-test` - Test resolver config file syntax
- `--` everything behind `--` will be passed as options to the internal resolver
Command line arguments can be arbitrarily combined, e.g. `pihole-FTL debug test`
## File locations
- `/var/log/pihole-FTL.log` log file
- `/var/run/pihole-FTL.pid` PID file
- `/var/run/pihole-FTL.port` file containing port on which `FTL` is listening
- `/var/run/pihole/FTL.sock` Unix socket
## Domain lists format
Since Pi-hole v4.0, we use a simpler domain list format for the two important block list files `gravity.list` and `black.list`. In contrast to the traditional HOSTS format (which caused a lot of overhead), the domain list format is the minimal possible solution for saving memory while still using plain text lists for your convenience. When *FTL*DNS imports these two files, they are walked by our improved list parser speeding up the loading of block lists significantly. Regardless which blocking mode (`IP` or `NXDOMAIN`) is selected, *FTL*DNS will always load the lists into it's internal hashed cache to be able to determine the blocking status within a few milliseconds, even when you're using huge blocking lists on low-end devices. With everything we do, we design *FTL*DNS for maximum efficiency also on low-performance devices.

View File

@@ -0,0 +1,36 @@
Using privacy levels you can specify which level of detail you want to see in your Pi-hole statistics. The privacy level may be changed at any time without having to restart the DNS resolver. Note that queries with (partially) hidden details cannot be disclosed with a subsequent reduction of the privacy level. They can be changed either from the Settings page on the dashboard or in [FTL's config file](configfile.md).
The available options are
### Level 0 - show everything
Doesn't hide anything, all statistics are available
### Level 1 - hide domains
Show and store all domains as `hidden`
This setting disables
- Top Domains
- Top Ads
### Level 2 - hide domains and clients
Show and store all domains as `hidden` and clients as `0.0.0.0`
This setting disables
- Top Domains
- Top Ads
- Top Clients
- Clients over time
### Level 3 - paranoia mode (hide everything)
Disabled basically everything except the most anonymous statistics
This setting disables
- Top Domains
- Top Ads
- Top Clients
- Clients over time
- Query Log
- Long-term database logging

144
docs/ftldns/telnet-api.md Normal file
View File

@@ -0,0 +1,144 @@
Connect via e.g. `telnet 127.0.0.1 4711` or use `echo ">command" | nc 127.0.0.1 4711`
- `>quit`: Closes connection to client
- `>kill`: Terminates FTL
- `>stats` : Get current statistics
```
domains_being_blocked 116007
dns_queries_today 30163
ads_blocked_today 5650
ads_percentage_today 18.731558
unique_domains 1056
queries_forwarded 4275
queries_cached 20238
clients_ever_seen 11
unique_clients 9
status enabled
```
- `>overTime` : over time data (10 min intervals)
```
1525546500 163 0
1525547100 154 1
1525547700 164 0
1525548300 167 0
1525548900 151 0
1525549500 143 0
[...]
```
- `>top-domains` : get top domains
```
0 8462 x.y.z.de
1 236 safebrowsing-cache.google.com
2 116 pi.hole
3 109 z.y.x.de
4 93 safebrowsing.google.com
5 96 plus.google.com
[...]
```
Variant: `>top-domains (15)` to show (up to) 15 entries
- `>top-ads` : get top ad domains
```
0 8 googleads.g.doubleclick.net
1 6 www.googleadservices.com
2 1 cdn.mxpnl.com
3 1 collector.githubapp.com
4 1 www.googletagmanager.com
5 1 s.zkcdn.net
[...]
```
Variant: `>top-ads (14)` to show (up to) 14 entries
- `top-clients` : get recently active top clients (IP addresses + host names (if available))
```
0 9373 192.168.2.1 router
1 484 192.168.2.2 work-machine
2 8 127.0.0.1 localhost
```
Variant: `>top-clients (9)` to show (up to) 9 client entries or `>top-clients withzero (15)` to show (up to) 15 clients even if they have not been active recently (see PR #124 for further details)
- `>forward-dest` : get forward destinations (IP addresses + host names (if available)) along with the percentage. The first result (ID -2) will always be the percentage of domains answered from blocklists, whereas the second result (ID -1) will be the queries answered from cache
```
-2 18.70 blocklist blocklist
-1 67.10 cache cache
0 14.20 127.0.0.1 localhost
```
Variant: `>forward-dest unsorted` to show forward destinations in unsorted order (equivalent to using `>forward-names`)
- `>querytypes` : get collected query types percentage
```
A (IPv4): 53.45
AAAA (IPv6): 45.32
ANY: 0.00
SRV: 0.64
SOA: 0.05
PTR: 0.54
TXT: 0.00
```
- `>getallqueries` : get all queries that FTL has in memory
```
1525554586 A fonts.googleapis.com 192.168.2.100 3 0 4 6
1525554586 AAAA fonts.googleapis.com 192.168.2.100 3 0 4 5
1525554586 A www.mkdocs.org 192.168.2.100 3 0 4 7
1525554586 AAAA www.mkdocs.org 192.168.2.100 2 0 3 21
1525554586 A squidfunk.github.io 192.168.2.100 2 0 3 20
1525554586 A pi-hole.net 192.168.2.100 3 0 4 5
1525554586 AAAA squidfunk.github.io 192.168.2.100 3 0 1 6
1525554586 AAAA pi-hole.net 192.168.2.100 2 0 1 18
1525554586 A github.com 192.168.2.100 3 0 4 5
1525554586 AAAA github.com 192.168.2.100 2 0 1 18
```
Variants: `>getallqueries (37)` show (up to) 37 latest entries, `>getallqueries-time 1483964295 1483964312` gets all queries that FTL has in its database in a limited time interval, `>getallqueries-time 1483964295 1483964312 (17)` show matches in the (up to) 17 latest entries, `>getallqueries-domain www.google.com` gets all queries that FTL has in its database for a specific domain name, `>getallqueries-client 2.3.4.5` : gets all queries that FTL has in its database for a specific client name *or* IP
- `>recentBlocked` : get most recently pi-holed domain name
```
www.googleadservices.com
```
Variant: `>recentBlocked (4)` show the four most recent blocked domains
- `>memory` : get information about `FTL`'s memory usage due to its internal data structure
```
memory allocated for internal data structure: 2944708 bytes (2.94 MB)
dynamically allocated allocated memory used for strings: 23963 bytes (23.96 KB)
Sum: 2968671 bytes (2.97 MB)
```
- `>clientID` : Get ID of currently connected client
```
6
```
- `>version` : Get version information of the currently running FTL instance
```
version v1.6-3-g106498d-dirty
tag v1.6
branch master
date 2017-03-26 13:10:43 +0200
```
- `>dbstats` : Get some statistics about the FTL long-term storage database (this request may take some time for processing in case of a large database file)
```
queries in database: 2700304
database filesize: 199.20 MB
SQLite version: 3.23.1
```
- `>domain pi-hole.net`: Get detailed information about domain (if available)
```
Domain "pi-hole.net", ID: 254
Total: 179
Blocked: 0
Wildcard blocked: false
```
- `>cacheinfo`: Get DNS server cache size and usage information
```
cache-size: 500000
cache-live-freed: 0
cache-inserted: 15529
```

View File

@@ -26,12 +26,15 @@ pages:
- 'Updating': 'main/update.md'
- 'FTLDNS':
- 'Overview': "ftldns/index.md"
- 'Debugging FTLDNS': "ftldns/debugging.md"
- 'Configuration': "ftldns/configfile.md"
- 'Blocking mode': "ftldns/blockingmode.md"
- 'RegEx blocking': "ftldns/regex.md"
# - 'Privacy levels': "ftldns/privacy.md"
- 'Privacy levels': "ftldns/privacylevels.md"
# - 'Long-term database': "ftldns/database.md"
# - 'API documentation': "ftldns/api.md"
- 'Debugging FTLDNS': "ftldns/debugging.md"
- 'Compatibility': "ftldns/compatibility.md"
- 'Install from source': "ftldns/compile.md"
- 'Telnet API': "ftldns/telnet-api.md"
- 'In-depth manual': "ftldns/in-depth.md"
- 'Guides':
# - 'Pi-hole as All-Around DNS Solution': guides/unbound.md