Dnstap streams receiver

This Python module acts as a DNS tap streams receiver for DNS servers. Input streams can be a unix socket or multiple remote dns servers. The output is printed directly to stdout or send to remote tcp address in JSON, YAML or one line text format and more.

Installation

PyPI

Deploy the dnstap receiver in your DNS server with the pip command.

pip install dnstap_receiver

After installation, you can execute the dnstap_receiver to start-it.

Usage:

usage: dnstap_receiver [-h] [-l L] [-p P] [-u U] [-v] [-c C]

optional arguments:
  -h, --help  show this help message and exit
  -l L        IP of the dnsptap server to receive dnstap payloads (default: '0.0.0.0')
  -p P        Port the dnstap receiver is listening on (default: 6000)
  -u U        read dnstap payloads from unix socket
  -v          verbose mode
  -c C        external config file

Docker Hub

Pull the dnstap receiver image from Docker Hub.

docker pull dmachard/dnstap-receiver:latest

Deploy the container

docker run -d -p 6000:6000 --name=dnstap01 dmachard/dnstap-receiver

Follow containers logs

docker logs dnstap01 -f

Inputs handler

Severals inputs handler are supported to read incoming dnstap messages:

TCP socket
Unix socket

TCP socket

The TCP socket input enable to receive dnstap messages from multiple dns servers. This is the default input if you execute the binary without arguments. The receiver is listening on localhost interface and the tcp port 6000. You can change binding options with -l and -p arguments.

./dnstap_receiver -l 0.0.0.0 -p 6000

You can also activate TLS on the socket, add the following config as external config file to activate the tls support, configure the path of the certificate and key to use.

input:
  tcp-socket:
    # enable tls support
    tls-support: true
    # provide certificate server path
    tls-server-cert: /etc/dnstap_receiver/server.crt
    # provide certificate key path
    tls-server-key: /etc/dnstap_receiver/server.key

Then execute the dnstap receiver with the configuration file:

./dnstap_receiver -c /etc/dnstap-receiver/dnstap.conf

Unix socket

The unix socket input enables read dnstap message from a unix socket. Configure the path of the socket with the -u argument.

./dnstap_receiver -u /var/run/dnstap.sock

Outputs handler

Outputs handler can be configured to forward messages in several modes.

Stdout
Stdout metrics
TCP socket
Syslog

Stdout

This output enables to forward dnstap messages directly to Stdout. Add the following configuration as external config to activate this output:

output:
  stdout:
    # enable or disable
    enable: true
    # format available text|json|yaml
    format: text

Output can be formatted in different way:

text (default one)
json
yaml

Text format:

2020-09-16T18:51:53.547352+00:00 centos RESOLVER_QUERY NOERROR - - IP4 UDP 43b ns2.google.com. A
2020-09-16T18:51:53.591736+00:00 centos RESOLVER_RESPONSE NOERROR - - IP4 UDP 59b ns2.google.com. A

JSON format:

{
    "identity": "dev-centos8",
    "query-name": "www.google.com.",
    "query-type": "A",
    "source-ip": "192.168.1.114",
    "message": "CLIENT_QUERY",
    "protocol": "IP4",
    "transport": "UDP",
    "source-port": 42222,
    "length": 43,
    "timestamp": "2020-09-16T18:51:53.591736+00:00",
    "code": "NOERROR"
}

YAML format:

code: NOERROR
length: 49
message: RESOLVER_QUERY
protocol: IP4
query-name: dns4.comlaude-dns.eu.
query-type: AAAA
source-ip: '-'
source-port: '-'
timestamp: '2020-09-16T18:51:53.591736+00:00'
transport: UDP

TCP socket

This output enables to forward dnstap message to a remote tcp collector. Add the following configuration as external config to activate this output:

output:
  # forward to remote tcp destination
  tcp-socket:
    # enable or disable
    enable: true
    # format available text|json|yaml
    format: text
    # delimiter
    delimiter: "\n"
    # retry interval in seconds to connect
    retry: 5
    # remote ipv4 or ipv6 address
    remote-address: 10.0.0.2
    # remote tcp port
    remote-port: 8192

Syslog

This output enables to forward dnstap message to a syslog server. Add the following configuration as external config to activate this output:

output:
  syslog:
    # enable or disable
    enable: false
    # syslog over tcp or udp
    transport: udp
    # format available text|json
    format: text
    # retry interval in seconds to connect
    retry: 5
    # remote ipv4 or ipv6 address of the syslog server
    remote-address: 10.0.0.2
    # remote port of the syslog server
    remote-port: 514

Example of output on syslog server

Sep 22 12:43:01 bind CLIENT_RESPONSE NOERROR 192.168.1.100 51717 IP4 UDP 173b www.netflix.fr. A
Sep 22 12:43:01 bind CLIENT_RESPONSE NOERROR 192.168.1.100 51718 IP4 UDP 203b www.netflix.fr. AAAA

Stdout metrics

This output enables to generate metrics in one line and print-it to stdout. Add the following configuration as external config to activate this output:

output:
  metrics:
    # enable or disable
    enable: true
    # print every N seconds.
    interval: 300
    # cumulative statistics, without clearing them after printing
    cumulative: false

Example of output

2020-10-13 05:19:35,522 18 QUERIES, 3.6 QPS, 1 CLIENTS, 18 IP4, 0 IP6, 
18 UDP, 0 TCP, 17 NOERROR, 1 NXDOMAIN, 18 A, 0 AAAA

More options

External config file

The dnstap_receiver binary can takes an external config file with the -c argument See config file example.

./dnstap_receiver -c /etc/dnstap-receiver/dnstap.conf

Verbose mode

You can execute the binary in verbose mode with the -v argument:

./dnstap_receiver -v
2020-09-12 23:47:35,833 Start dnstap receiver...
2020-09-12 23:47:35,833 Using selector: EpollSelector
2020-09-12 23:47:35,834 Listening on 0.0.0.0:6000

Filtering feature

This feature can be useful if you want to ignore some messages and keep just what you want. Several filter are available:

by qname field
by dnstap identity field.

By dnstap identity

You can filtering incoming dnstap messages according to the dnstap identity field. A regex can be configured in the external configuration file to do that

filter:
  # dnstap identify filtering feature with regex support
  dnstap-identities: dnsdist01|unbound01

By qname

You can filtering incoming dnstap messages according to the query name. A regex can be configured in the external configuration file to do that

filter: 
  # qname filtering feature with regex support
  qname-regex: ".*.com"

Tested DNS servers

This dnstap receiver has been tested with success with the following dns servers:

ISC - bind
PowerDNS - dnsdist, pdns-recursor
NLnet Labs - nsd, unbound

bind

Dnstap messages supported:

RESOLVER_QUERY
RESOLVER_RESPONSE
CLIENT_QUERY
CLIENT_RESPONSE
AUTH_QUERY
AUTH_RESPONSE

Build with dnstap support

Download latest source and build-it with dnstap support:

./configure --enable-dnstap
make && make install

Unix socket

Update the configuration file /etc/named.conf to activate the dnstap feature:

options {
    dnstap { client; auth; resolver; forwarder; };
    dnstap-output unix "/var/run/named/dnstap.sock";
    dnstap-identity "dns-bind";
    dnstap-version "bind";
}

Execute the dnstap receiver with named user:

su - named -s /bin/bash -c "dnstap_receiver -u "/var/run/named/dnstap.sock""

pdns-recursor

Dnstap messages supported:

RESOLVER_QUERY
RESOLVER_RESPONSE

Unix socket

Update the configuration file to activate the dnstap feature:

vim /etc/pdns-recursor/recursor.conf
lua-config-file=/etc/pdns-recursor/recursor.lua

vim /etc/pdns-recursor/recursor.lua
dnstapFrameStreamServer("/var/run/pdns-recursor/dnstap.sock")

Execute the dnstap receiver with pdns-recursor user:

su - pdns-recursor -s /bin/bash -c "dnstap_receiver -u "/var/run/pdns-recursor/dnstap.sock""

TCP stream

Update the configuration file to activate the dnstap feature with tcp mode and execute the dnstap receiver in listening tcp socket mode:

vim /etc/pdns-recursor/recursor.conf
lua-config-file=/etc/pdns-recursor/recursor.lua

vim /etc/pdns-recursor/recursor.lua
dnstapFrameStreamServer("10.0.0.100:6000")

dnsdist

Dnstap messages supported:

CLIENT_QUERY
CLIENT_RESPONSE

Unix socket

Create the dnsdist folder where the unix socket will be created:

mkdir -p /var/run/dnsdist/
chown dnsdist.dnsdist /var/run/dnsdist/

Update the configuration file /etc/dnsdist/dnsdist.conf to activate the dnstap feature:

fsul = newFrameStreamUnixLogger("/var/run/dnsdist/dnstap.sock")
addAction(AllRule(), DnstapLogAction("dnsdist", fsul))
addResponseAction(AllRule(), DnstapLogResponseAction("dnsdist", fsul))

Execute the dnstap receiver with dnsdist user:

su - dnsdist -s /bin/bash -c "dnstap_receiver -u "/var/run/dnsdist/dnstap.sock""

TCP stream

Update the configuration file /etc/dnsdist/dnsdist.conf to activate the dnstap feature with tcp stream and execute the dnstap receiver in listening tcp socket mode:

fsul = newFrameStreamTcpLogger("127.0.0.1:8888")
addAction(AllRule(), DnstapLogAction("dnsdist", fsul))
addResponseAction(AllRule(), DnstapLogResponseAction("dnsdist", fsul))

nsd

Dnstap messages supported:

AUTH_QUERY
AUTH_RESPONSE

Build with dnstap support

Download latest source and build-it with dnstap support:

./configure --enable-dnstap
make && make install

Unix socket

Update the configuration file /etc/nsd/nsd.conf to activate the dnstap feature:

dnstap:
    dnstap-enable: yes
    dnstap-socket-path: "/var/run/nsd/dnstap.sock"
    dnstap-send-identity: yes
    dnstap-send-version: yes
    dnstap-log-auth-query-messages: yes
    dnstap-log-auth-response-messages: yes

Execute the dnstap receiver with nsd user:

su - nsd -s /bin/bash -c "dnstap_receiver -u "/var/run/nsd/dnstap.sock""

unbound

Dnstap messages supported:

CLIENT_QUERY
CLIENT_RESPONSE
RESOLVER_QUERY
RESOLVER_RESPONSE
CLIENT_QUERY
CLIENT_RESPONSE

Build with dnstap support

Download latest source and build-it with dnstap support:

./configure --enable-dnstap
make && make install

Unix socket

Update the configuration file /etc/unbound/unbound.conf to activate the dnstap feature:

dnstap:
    dnstap-enable: yes
    dnstap-socket-path: "dnstap.sock"
    dnstap-send-identity: yes
    dnstap-send-version: yes
    dnstap-log-resolver-query-messages: yes
    dnstap-log-resolver-response-messages: yes
    dnstap-log-client-query-messages: yes
    dnstap-log-client-response-messages: yes
    dnstap-log-forwarder-query-messages: yes
    dnstap-log-forwarder-response-messages: yes

Execute the dnstap receiver with unbound user:

su - unbound -s /bin/bash -c "dnstap_receiver -u "/usr/local/etc/unbound/dnstap.sock""

TCP stream

Update the configuration file /etc/unbound/unbound.conf to activate the dnstap feature with tcp mode and execute the dnstap receiver in listening tcp socket mode:

dnstap:
    dnstap-enable: yes
    dnstap-socket-path: ""
    dnstap-ip: "10.0.0.100@6000"
    dnstap-tls: no
    dnstap-send-identity: yes
    dnstap-send-version: yes
    dnstap-log-client-query-messages: yes
    dnstap-log-client-response-messages: yes

TLS stream

Update the configuration file /etc/unbound/unbound.conf to activate the dnstap feature with tls mode and execute the dnstap receiver in listening tcp/tls socket mode:

dnstap:
    dnstap-enable: yes
    dnstap-socket-path: ""
    dnstap-ip: "10.0.0.100@6000"
    dnstap-tls: yes
    dnstap-send-identity: yes
    dnstap-send-version: yes
    dnstap-log-client-query-messages: yes
    dnstap-log-client-response-messages: yes

About


Author	Denis Machard d.machard@gmail.com
PyPI	https://pypi.org/project/dnstap_receiver/
Github	https://github.com/dmachard/dnstap-receiver
DockerHub	https://hub.docker.com/r/dmachard/dnstap-receiver

Name		Name	Last commit message	Last commit date
Latest commit History 100 Commits
.github/workflows		.github/workflows
dnstap_receiver		dnstap_receiver
tests		tests
Dockerfile		Dockerfile
LICENSE		LICENSE
MANIFEST.in		MANIFEST.in
README.md		README.md
setup.py		setup.py

License

jinmeiib/dnstap-receiver

Folders and files

Latest commit

History

Repository files navigation

Dnstap streams receiver

Table of contents

Installation

PyPI

Docker Hub

Inputs handler

TCP socket

Unix socket

Outputs handler

Stdout

TCP socket

Syslog

Stdout metrics

More options

External config file

Verbose mode

Filtering feature

By dnstap identity

By qname

Tested DNS servers

bind

Build with dnstap support

Unix socket

pdns-recursor

Unix socket

TCP stream

dnsdist

Unix socket

TCP stream

nsd

Build with dnstap support

Unix socket

unbound

Build with dnstap support

Unix socket

TCP stream

TLS stream

About

About

Resources

License

Stars

Watchers

Forks

Languages