Blocking undesirable domains using BIND

On Dec 28, 2007 8:49 AM, Kevin Kinsey <kdkdaleco.biz> wrote:
> Olivier Nicole wrote:
> >> Again, I'm not trying to convince you
otherwise or say that using
> >> BIND is a bad idea.  It's just that I'm
curious because we use
> >> Squid for this sort of thing, and I was
wondering why BIND instead?
> >
> > I think another issue is that Squid will only
filter HTTP/FTP
> > connections, while DNS would allow to filter any
type of traffic that
> > would try to go to places with a bad name.
> >
> > Olivier
>
> In the absence of egress filtering on the firewall,
that
> would definitely be an advantage.  Does anyone use
BIND
> for filtering in a small to medium business
environment
> then?  How does it perform?

Performs fine.

# rndc status
number of zones: 17210
...

My 17000+ zones are loaded from the DNS-BH project and
increase the
startup time of named to about 10 seconds and bump the
resident memory
size up to about 55M. (AMD Duron 750MHz).

There's no real performance hit per se by DNS blackholing,
other than
the resource utilization increase needed for handling
additional
zones; your name server would normally be handling these DNS
lookups
anyway.You're just overriding the response locally rather
than
recursing for it. The zones themselves typically end up
being very
small, like a single wildcard record pointing to 127.0.0.1
or a
honeypot or whatever.

DS
_______________________________________________
freebsd-questionsfreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-que
stions
To unsubscribe, send any mail to
"freebsd-questions-unsubscribefreebsd.org"

Kevin Kinsey wrote:
> Just a question, and I'm not trying to cast doubt on
your plan; I'm 
> curious why using BIND for this purpose instead of a
proxy, which is
> a more typical application as I understand it?

I was trying to do something similar.  I didn't research too
hard, but figured the only way to use Bind would be to make
my server authoritative for all those domains, which meant a
huge config file and potential overhead, as well as 
possibly breaking access to desirable servers in the
domains.

So hosts seemed easier, but apparently Bind never looks at
hosts.  I did find that Squid (which I already had installed
and in limited use) has its own DNS resolver, and it does
look at hosts first before going to the nameserver.

Then I found this site:  http://everythin
gisnt.com/hosts.html and put their list in hosts, and
now client PCs get a squid error in place of ad junk.  Works
ok for me ;)

  -Rob
_______________________________________________
freebsd-questionsfreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-que
stions
To unsubscribe, send any mail to
"freebsd-questions-unsubscribefreebsd.org"

On Dec 28, 2007 11:28 AM, Rob <bitabyssgmail.com> wrote:
> Kevin Kinsey wrote:
> > Just a question, and I'm not trying to cast doubt
on your plan; I'm
> > curious why using BIND for this purpose instead of
a proxy, which is
> > a more typical application as I understand it?
>
> I was trying to do something similar.  I didn't
research too hard, but figured the only way to use Bind
would be to make my server authoritative for all those
domains, which meant a huge config file and potential
overhead, as well as
> possibly breaking access to desirable servers in the
domains.
>
> So hosts seemed easier, but apparently Bind never looks
at hosts.  I did find that Squid (which I already had
installed and in limited use) has its own DNS resolver, and
it does look at hosts first before going to the nameserver.
>
> Then I found this site:  http://everythin
gisnt.com/hosts.html and put their list in hosts, and
now client PCs get a squid error in place of ad junk.  Works
ok for me ;)
>
>   -Rob
>

Well... you were right about overhead. In the last two days
I wrote a
script that would fetch a list of domains from several
different
sites, and output a valid BIND configuration file that could
be
included in the main config. I just ran the second test and
the
results are extremely poor. With only 27,885 blocked domains
the
server is now consuming 208 MB of ram. The first time I
tried
reloading the full list of domains (91,137 of them) and that
nearly
crashed my server. Had to kill bind, remove two of the
largest
sources, and try a second time.

Honestly, I can't figure out what BIND could possibly be
using so much
memory for. It's taking up about 7 KB for each zone. The
zone file
itself is not even 1 KB, and given that all the records are
pointing
to the exact same thing it seems to be needlessly wasting
memory. In
addition to that, if I comment out the blacklist config file
and run
rndc reload, it only frees up about 16 MB. So it doesn't
even release
memory when it is no longer needed.

It looks like my plan of using BIND for filtering purposes
will not
work. Given how poorly it performed on this test I'm
actually inclined
to try another name server to see if something else would be
more
memory-efficient. If I can't find anything then I'll need to
put some
other piece of software to intercept BIND's recursive
queries and
block the domains that way.

- Max
_______________________________________________
freebsd-questionsfreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-que
stions
To unsubscribe, send any mail to
"freebsd-questions-unsubscribefreebsd.org"

On Dec 30, 2007 9:52 AM, Maxim Khitrov <mkhitrovgmail.com> wrote:
> > I was trying to do something similar.  I didn't
research too hard, but figured the only way to use Bind
would be to make my server authoritative for all those
domains, which meant a huge config file and potential
overhead, as well as
> > possibly breaking access to desirable servers in
the domains.
> >
> > So hosts seemed easier, but apparently Bind never
looks at hosts.  I did find that Squid (which I already had
installed and in limited use) has its own DNS resolver, and
it does look at hosts first before going to the nameserver.
> >
> > Then I found this site:  http://everythin
gisnt.com/hosts.html and put their list in hosts, and
now client PCs get a squid error in place of ad junk.  Works
ok for me ;)
> Well... you were right about overhead. In the last two
days I wrote a
> script that would fetch a list of domains from several
different
> sites, and output a valid BIND configuration file that
could be
> included in the main config. I just ran the second test
and the
> results are extremely poor. With only 27,885 blocked
domains the
> server is now consuming 208 MB of ram. The first time I
tried
> reloading the full list of domains (91,137 of them) and
that nearly
> crashed my server. Had to kill bind, remove two of the
largest
> sources, and try a second time.

Nearly 100,000 zones on that server is a fairly impressive
amount.
Give it credit for what you're trying to do. 
Nonetheless, crashing
is unacceptable.

> Honestly, I can't figure out what BIND could possibly
be using so much
> memory for. It's taking up about 7 KB for each zone.
The zone file
> itself is not even 1 KB, and given that all the records
are pointing
> to the exact same thing it seems to be needlessly
wasting memory. In
> addition to that, if I comment out the blacklist config
file and run
> rndc reload, it only frees up about 16 MB. So it
doesn't even release
> memory when it is no longer needed.

My experience, albeit with a smaller number of zones, is a
bit different.

First  you need to account for main program memory and
memory utilized
by the nameserver's cache, if any. You may also be running
your own
authoritative zones which will add memory utilization
outside of that.
You can't account for all of the utilized memory in your
additional
blocking zones.

Without my blocking zones loaded, I have 6 native zones on
my
nameserver and the resident memory size of named is 2.2 MB.
After a
fresh server startup, I expect minimum memory for cached
records, so
that comes out to be about 375 KB/zone, unscientifically. If
I restart
named (kill and start server fresh) with my blocking zones
in the
config, I come out with 17239 zones and a resident process
memory size
of 59 MB. (Unscientifically again,) this breaks down to
about 3.5
KB/zone.

In my configuration, each of these blocking zones points to
a simple
zone file 244B in size on disk:

$TTL 86400
    
          IN      SOA     ns.local. admin.local. (
                                1       ; serial
                                1h      ; refresh
                                30m     ; retry
                                7d      ; expiration
                                1h )    ; minimum

                IN      NS      ns.local.

                IN      A       127.0.0.1
*               IN      A       127.0.0.1

So all told, I seem to notice somewhat slimmer utilization
than you
(roughly half the memory utilization per zone, and though I
have 61%
as many zones loaded my named takes only 28% of the memory
yours
does.)

> It looks like my plan of using BIND for filtering
purposes will not
> work. Given how poorly it performed on this test I'm
actually inclined
> to try another name server to see if something else
would be more
> memory-efficient.

You will almost certainly find most of the popular
alternatives to be
much more resource efficient. djbdns in particular would be
my next
choice if memory efficiency and stability are concerns.

DS
_______________________________________________
freebsd-questionsfreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-que
stions
To unsubscribe, send any mail to
"freebsd-questions-unsubscribefreebsd.org"

On Dec 30, 2007 12:31 PM, Darren Spruell <phatbuckettgmail.com> wrote:
> On Dec 30, 2007 9:52 AM, Maxim Khitrov <mkhitrovgmail.com> wrote:
> > > I was trying to do something similar.  I
didn't research too hard, but figured the only way to use
Bind would be to make my server authoritative for all those
domains, which meant a huge config file and potential
overhead, as well as
> > > possibly breaking access to desirable servers
in the domains.
> > >
> > > So hosts seemed easier, but apparently Bind
never looks at hosts.  I did find that Squid (which I
already had installed and in limited use) has its own DNS
resolver, and it does look at hosts first before going to
the nameserver.
> > >
> > > Then I found this site:  http://everythin
gisnt.com/hosts.html and put their list in hosts, and
now client PCs get a squid error in place of ad junk.  Works
ok for me ;)
> > Well... you were right about overhead. In the last
two days I wrote a
> > script that would fetch a list of domains from
several different
> > sites, and output a valid BIND configuration file
that could be
> > included in the main config. I just ran the second
test and the
> > results are extremely poor. With only 27,885
blocked domains the
> > server is now consuming 208 MB of ram. The first
time I tried
> > reloading the full list of domains (91,137 of
them) and that nearly
> > crashed my server. Had to kill bind, remove two of
the largest
> > sources, and try a second time.
>
> Nearly 100,000 zones on that server is a fairly
impressive amount.
> Give it credit for what you're trying to do. 
Nonetheless, crashing
> is unacceptable.
>
> > Honestly, I can't figure out what BIND could
possibly be using so much
> > memory for. It's taking up about 7 KB for each
zone. The zone file
> > itself is not even 1 KB, and given that all the
records are pointing
> > to the exact same thing it seems to be needlessly
wasting memory. In
> > addition to that, if I comment out the blacklist
config file and run
> > rndc reload, it only frees up about 16 MB. So it
doesn't even release
> > memory when it is no longer needed.
>
> My experience, albeit with a smaller number of zones,
is a bit different.
>
> First  you need to account for main program memory and
memory utilized
> by the nameserver's cache, if any. You may also be
running your own
> authoritative zones which will add memory utilization
outside of that.
> You can't account for all of the utilized memory in
your additional
> blocking zones.
>
> Without my blocking zones loaded, I have 6 native zones
on my
> nameserver and the resident memory size of named is 2.2
MB. After a
> fresh server startup, I expect minimum memory for
cached records, so
> that comes out to be about 375 KB/zone,
unscientifically. If I restart
> named (kill and start server fresh) with my blocking
zones in the
> config, I come out with 17239 zones and a resident
process memory size
> of 59 MB. (Unscientifically again,) this breaks down to
about 3.5
> KB/zone.
>
> In my configuration, each of these blocking zones
points to a simple
> zone file 244B in size on disk:
>
> $TTL 86400
>                IN      SOA     ns.local. admin.local.
(
>                                 1       ; serial
>                                 1h      ; refresh
>                                 30m     ; retry
>                                 7d      ; expiration
>                                 1h )    ; minimum
>
>                 IN      NS      ns.local.
>
>                 IN      A       127.0.0.1
> *               IN      A       127.0.0.1
>
> So all told, I seem to notice somewhat slimmer
utilization than you
> (roughly half the memory utilization per zone, and
though I have 61%
> as many zones loaded my named takes only 28% of the
memory yours
> does.)
>
> > It looks like my plan of using BIND for filtering
purposes will not
> > work. Given how poorly it performed on this test
I'm actually inclined
> > to try another name server to see if something
else would be more
> > memory-efficient.
>
> You will almost certainly find most of the popular
alternatives to be
> much more resource efficient. djbdns in particular
would be my next
> choice if memory efficiency and stability are
concerns.
>
> DS
>

I was using the exact same zone file as you, one real master
zone, and
the three slave root zones from the default config. Not sure
why it
reacted as it did to the blacklist config, but I think I now
found a
perfect solution. This morning I played around with MaraDNS,
which is
actually a pretty good DNS server. One problem with it was
that it
didn't allow includes in the main config. That means that
everything
has to be in a single file and that's a bit messy. It did a
lot better
with memory usage, taking up about 70MB for 27 or 28
thousand domains,
but still not great.

I then installed dnsmasq, which is able to read domain info
from the
hosts file. Just for the fun of it, I loaded domains from
all the
sources I've gathered into a separate hosts file - a total
of 155,150
entries. Dnsmasq loaded that file and has been running for
several
minutes now. It's currently taking up a total of 17MB! Now
granted, it
doesn't need to deal with whole zone files, but this still
goes to
show the level of efficiency that can be achieved in theory
even with
this many entries.

Dnsmasq also provides a DHCP server, which was the next item
on my
to-configure list. Unfortunately, it can only forward DNS
requests to
an upstream server (like the one provided by your ISP). So
here's what
I'll do... BIND will stay, but only for the purposes of
serving the
root zones and my local zone file. I'll bring its caching to
a
minimum, and have it listen only on 127.0.0.1:54. Dnsmasq
will then
listen on *:53 and use BIND as its upstream server. It will
be
responsible for filtering domains and caching query results.
BIND will
serve the real zone files and resolve any queries received
from
dnsmasq. The memory usage for all DNS-related processes
should be no
more than 30MB and I have my filtering solution in place


- Max
_______________________________________________
freebsd-questionsfreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-que
stions
To unsubscribe, send any mail to
"freebsd-questions-unsubscribefreebsd.org"

Darren Spruell wrote:
> On Dec 28, 2007 8:49 AM, Kevin Kinsey <kdkdaleco.biz> wrote:

>> In the absence of egress filtering on the firewall,
that
>> would definitely be an advantage.  Does anyone use
BIND
>> for filtering in a small to medium business
environment
>> then?  How does it perform?
> 
> Performs fine.
> 
> # rndc status
> number of zones: 17210
> ...

<snip>

Thanks, Darren.

-- 
Pity the meek, for they shall inherit the earth.
		-- Don Marquis
_______________________________________________
freebsd-questionsfreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-que
stions
To unsubscribe, send any mail to
"freebsd-questions-unsubscribefreebsd.org"