|
| News | Profile | Code | Photography | Looking Glass | Projects | System Statistics | Uncategorized |
Blog |
So, Gentoo decided to add the x86 and amd64 keywords to x11-base/xorg-x11-7.0-r1. The upgrade was smooth, once I defined VIDEO_CARDS and INPUT_DEVICES in /etc/make.conf. However, stuff like this just annoys me:
!!! All ebuilds that could satisfy "xcalc" have been masked. !!! One of the following masked packages is required to complete your request: - x11-apps/xcalc-1.0.1 (masked by: ~x86 keyword)
Seriously guys, what gives? For xorg-x11 to be stable, all of the previously bundled components should be marked as stable. I had to unmask x11-apps/xload, too. Pathetic.
I'm not this disgruntled, though.
Finally got around to creating a HOWTO for it. Here goes.
If you want to create pretty graphs with FlowScan and Cisco's NetFlow protocol, keep reading.
There are a couple software components required:
Collector on a Debian Host
If you want to run the collector on the Debian GNU/Linux platform, start off by installing these packages (I was using Debian testing, 02/2006): rrdtool, flowscan-cugrapher, flowscan-cuflow, flowscan, fprobe, flow-tools, apache2-mpm-prefork (or whatever). Debconf might prompt you for some config for fprobe, just keep hitting enter, we'll deal with it later.
The configuration files for some of the packages (flowscan, flow-tools) haven't been debianized - meaning they have paths that don't make sense on a Debian system. I suggest making the following directories to start off:
# mkdir -p /var/lib/netflow/ft
# mkdir /var/lib/netflow/rrds
# mkdir /var/lib/netflow/scoreboard
That's what I decided to use, but some other HOWTOs suggest /var/netflow, /usr/local/var/netflow, /cf/netflow, and other odd pathnames that don't seem to fall in line with what the FHS outlines. You can put stuff wherever you want, just be consistant, and make sure you have some free space on the selected FS :)
To setup the collector, edit /etc/flow-tools/flow-capture.conf and put in the following:
-w /var/lib/netflow/ft -E 1G -N 0 -n 287 -S 60 -V 5 -z 9 0/0/555
Please only put this ONE line in the configuration file, even though it says you can add multiple listen statements. It was.. unreliable when doing it that way. Basically each line in that file is treated as a string of arguments to flow-capture(1). You can look them all up at your leisure, but I suggest only changing the argument to -w, which is where to save the flows, and the last argument. The 0/0/555 defines the bind IP, IP to receive flows from, and UDP port to listen to. So, in our example, it binds to 0 (expanded to 0.0.0.0) which is "any" address, listens to flows from 0 (0.0.0.0) which means accept flows from "any" source, and listens on UDP port 555. After you've got that set, bounce it:
# /etc/init.d/flow-capture restart
Stopping flow-capture: flow-capture.
Starting flow-capture: flow-capture.
#
Next step is to configure flowscan to use CUFlow instead of the (default) CampusIO module, and tell it where to find the NetFlow data.
Start editing /etc/flowscan/flowscan.cf and change the FlowFileGlob and ReportClasses lines to the following:
FlowFileGlob /var/lib/netflow/ft/ft-v05.*
ReportClasses CUFlow
The first line tells flowscan to look in your /var/lib/netflow/ft directory for NetFlow files starting with ft-v05. The v05 is version 5. The ReportClasses directive tells it to use CUFlow.pm, which we will be editing a configuration file for shortly. The rest of the file you can probably keep intact.
Now, check out /etc/flowscan/CUFlow.cf. This is where it starts getting interesting, because you can really make this file complex if you have a number of hosts reporting, and you want to classify traffic based on a number of discriminators.
The first couple lines will have the directive Subnet followed by a network number and prefix length (CIDR). You should only list networks (one per Subnet line) that will send traffic through the hosts/interfaces you have reporting. For example, if you have just one host reporting, put its address with a 32-bit prefix. If you configure NetFlow on the perimiter router for your AS, I'd suggest enumerating all the networks you advertise via BGP as Subnet statements in this configuration file. Maybe add the loopback address of your router, too. For reference, I use the following:
# dax
Subnet 69.9.189.180/30
Subnet 69.9.186.232/29
# nat
Subnet 67.85.206.67/32
# nonce
Subnet 207.150.167.80/29
Subnet 207.157.167.88/31
Next section is actual Network statements. These are just filters for you to further classify traffic when using CUGrapher.pl. Syntax is "Network $network_number/$mask $name. I have the following:
# dax
Network 69.9.189.180/30 dax-transit
Network 69.9.186.232/29 dax-loopback
# nat
Network 67.85.206.67/32 nat-transit
# nonce
Network 207.150.167.80/29 nonce-transit
Network 207.150.167.88/31 nonce-transit0
In my case, the Subnet statements actually match the Network statements. This probably won't be what you do. If you have a Subnet statement of 10.0.0.0/8 (your enterprise) you might only have two Network statements; 10.2.10.0/24 (Engineering) or 10.255.254.0/23 (Management). Totally up to you...
Next section is pretty self-explanatory, OutputDir. Just specify the dir where you want the RRD files written. Mine's the following:
OutputDir /var/lib/netflow/rrds
If you want to track multicast traffic, just add a a line saying "Multicast" and I think multicast traffic will be analyzed. I didn't try this, since I'd probably only see OSPF hellos and LSA's. Pretty boring.
The next two directives are for the scoreboards. You can probably safely just copy what I have here, or change the pathnames to something else. Note, you don't need these readable by your web browser. As we'll see later on, the grapher application that links to these files actually reads them in, instead of just redirecting to them.
# Keep top N lists
# Show the top ten talkers, storing reports in /cflow/flows/reports
# and keeping the current report in /etc/httpd/data/reports/topten.html
Scoreboard 10 /var/lib/netflow/scoreboard /var/lib/netflow/topten.html
# Same, but build an over-time average top N list
AggregateScore 10 /var/lib/netflow/scoreboard/agg.dat /var/lib/netflow/scoreboard/overall.html
The next couple sections are all optional components, but will allow you to differentiate traffic based on some predefined criteria.
For all of the options above, you can specify multiple filter directives, assuming they have a unique $name. Ports can be grouped together using commas, and range statements, like: "Service udp/55-60,tcp/45 RandomApp." Here's a sample:
# Our two netflow exporters. Produce service and protocol reports for the
# total, and each of these.
Router 10.3.4.6 dax
Router 10.3.4.3 nat
Router 10.3.4.2 nonce
# Services we are interested in
Service 20-21/tcp ftp
Service 22/tcp ssh
Service 23/tcp telnet
Service 25/tcp smtp
Service 53/udp,53/tcp dns
Service 80/tcp http
Service 110/tcp pop3
Service 119/tcp nntp
Service 143/tcp imap
Service 412/tcp,412/udp dc
Service 443/tcp https
Service 1214/tcp kazaa
Service 4661-4662/tcp,4665/udp edonkey
Service 5190/tcp,9898/tcp aim
Service 6346-6347/tcp gnutella
Service 6665-6669/tcp irc
# protocols we are interested in
Protocol 1 icmp
Protocol 6 tcp
Protocol 17 udp
Protocol 41 ipv6
Protocol 47 gre
Protocol 50 esp
Protocol 51 ah
Protocol 57 skip
Protocol 88 eigrp
# ToS bit percentages to graph
#TOS 0 normal
#TOS 1-255 other
# Interested in traffic to/from AS 1
ASNumber 91 RPI-AS
ASNumber 3456 TW-CABLE
ASNumber 6128 CABLE-NET-1
ASNumber 11426 SCRR-11426
ASNumber 21840 SAGONET-TPA
ASNumber 22909 DNEO-OSP1
You can now go ahead and start flowscan. I suggest running it from the shell prompt for the first time (as root). You might see the following:
# flowscan
2006/02/28 18:55:26 working on file /var/lib/netflow/ft/ft-v05.2006-02-28.185001 -0500...
2006/02/28 18:55:26 flowscan-1.020 CUFlow: Cflow::find took 0 wallclock secs ( 0.03 usr + 0.00 sys = 0.03 CPU) for 4296 flow file bytes, flow hit ratio: 187/ 203
2006/02/28 18:55:26 flowscan-1.020 CUFlow: report took 0 wallclock secs ( 0.01 usr 0.00 sys + 0.00 cusr 0.02 csys = 0.02 CPU)
sleep 30...
sleep 30...
sleep 30...
If it looks good, then I suggest setting up flowscan to start at boot, and log to a file. There's a flowscan.gz in the flowscan documentation examples directory, which is probably a good start.
The last piece required is to configure CUGrapher. The configuration file is in /etc/flowscan/CUGrapher.cf. Set the OutputDir to the directory where the RRD files are stored (just like above) and set your Organization as whatever you'd like. The organization will appear at the top of the CUGrapher page. The only other things I ended up setting were the AggregateScore and Scoreboard directives. I ended up pointing them to the same files as were specified in CUFlow.cf. Feel free to change other options in that file, if you don't like the [pretty plain] detaults.
If everything is working correctly, wait a few minutes and then head to http://site/cgi-bin/CUGrapher.cgi.
Reporter on a Debian Host
First, install the fprobe package, and just hit enter through most of the debconf menus, since we'll hack the configuration files afterwards.
Edit /etc/default/fprobe and fill in the INTERFACE and FLOW_COLLECTOR variables at the top. Interface should be obvious, and the FLOW_COLLECTOR will be in the format of host:port. You can use DNS.
The OTHER_ARGS can be anything specified in the fprobe(8) manpage. I just use "-p -a $local_ip" where my $local_ip is a loopback IP.
# /etc/init.d/fprobe stop && /etc/init.d/fprobe start
Stopping fprobe: fprobe.
Starting fprobe: fprobe.
(Right, "restart" doesn't work for some reason) That should be it!
Collector on a FreeBSD Host
This assumes you're running a modern version of FreeBSD, 5.x or 6.x, and have a synchronized ports tree. I suggest using portpgrade and portinstall for all ports-related activity. For starters, portinstall the following:
Eh, I should warn you that flowscan has cflowd as a B-dep, which in turn needs gcc 2.95 to build correctly. So, you might have to temporarily remove any CFLAGS in /etc/make.conf that won't work with gcc 2.95. I had to replace --march=pentium4 with --march=pentiumpro.
Probably not a bad idea to pick the same data directory as detailed in the Debian directions. If you forgot:
# mkdir -p /var/lib/netflow/ft
# mkdir /var/lib/netflow/rrds
# mkdir /var/lib/netflow/scoreboard
The flow-tools port doesn't come with any startup scripts for flow-capture, so I suggest rolling your own or sticking it in /etc/rc.local. Just as before, something like:
# Start flow-capture
/usr/local/bin/flow-capture -E 1G -N 0 -n 287 -S 60 -V 5 -w /var/lib/netflow/ft -z 9 0/0/555
Next, you need to update FlowScan.pm, since the version in ports is ancient (see here, too). Gra FlowScan.pm from here and overwrite the existing file in /usr/local/var/db/flows/bin.
Configure flowscan.cf similarly to the Debian instructions above. Copy flowscan.cf.sample to flowscan.cf, and edit.
Unfortunately, there's no port for CUFlow, so you're going to have to fetch it yourself. Blah
There's one more component you need, Cflow.pm. It's tricky, so listen carefully. Grab the flow-tools source from here and extract. If you're like me, you might have searched ports by now and found a net-mgmt/p5-Cflow port. Do NOT install it. It won't link against the OSU library (-DOSU w/flow-tools) and will leave you troubleshooting for hours trying to figure out what the heck is wrong.
Change your working directory to flow-tools-0.6x and do the typical ./configure && make. You'll get errors, I bet, most likely stuff like:
flow-cat.c: In function `main':
flow-cat.c:555: error: label at end of compound statement
*** Error code 1
You'll get tons of these. In each case, you're going to need to add a semicolon at the beginning of the line referenced. So, for the error above, change the following in src/flow-cat.c:
} /* FOREACH filename in dir */
to...
; } /* FOREACH filename in dir */
Yes, you'll have to do it a couple of times before it finishes compiling. Then, you might get another error:
flow-header.c: In function `main':
flow-header.c:51: warning: implicit declaration of function `getopt'
flow-header.c:55: error: `optarg' undeclared (first use in this function)
flow-header.c:55: error: (Each undeclared identifier is reported only once
flow-header.c:55: error: for each function it appears in.)
flow-header.c:75: error: `optind' undeclared (first use in this function)
*** Error code 1
If you look up getopt(3), it needs <unistd.h> included. Add the following to the beginning of the include statements in flow-header.c:
#include <unistd.h>
There might be a couple more compound statement errors, but it should compile after that. Now, don't do a make install or anything, just cd into contrib, and extract Cflow-1.051.tar.gz. Do the following after cd'ing into Cflow-1.051:
% perl Makefile.PL
You should see some output. If there is a -DOSU in the Found flow-tools line, you're golden. Run make, and then do a make install as root. This will, strangely enough, actually install the Perl module and update the FreeBSD package system as well. You'll find a bsdpan-Cflow-1.051 package installed, if you run pkg_info.
Now, read the Debian instructions for configuring /usr/local/var/db/flows/bin/CUFlow.cf. It's practically the same. Lastly, copy /usr/local/etc/rc.d/cflowd-flowscan.sh.sample to /usr/local/etc/rc.d/cflowd-flowscan.sh and start it up! Watch the log in /usr/local/var/db/flows/flowscan.log, it should look ok.
The last part is to get CUGrapher.pl installed. I cheated, and just ripped the two files from the Debian package. Get the original and Debian patch. Apply the patch, copy CUGrapher.cf to /usr/local/var/db/flows/bin, and then follow the Debian instructions to configure it. Copy CUGrapher.pl to /usr/local/www/cgi-bin (or whatever path executes CGI scripts) and change the my $conffile path to /usr/local/var/db/flows/bin/CUGrapher.cf.
Done!
Reporter on a FreeBSD Host
There's not much to this. Install net-mgmt/fprobe from ports, and start it up like so:
/usr/local/sbin/fprobe -p -i fxp0 -a 10.3.4.6 dax:555
Again, there's no startup script for fprobe, so either make one or abuse /etc/rc.local again.
Conclusion
Other than a couple of the scripts and apps needing some upkeep, configuration isn't that terrible. Beats convincing management to spend thousands of dollars on some similar commercial junk :)
I think I've finally found a good IPv6 tunnel provider, SixXS:
Connecting to 2001:4f8:0:2::1e:80... connected. HTTP request sent, awaiting response... 200 OK Length: 275,666,325 (263M) [application/x-gzip] 100%[========================>] 275,666,325 835.46K/s ETA 00:00
That's dax pulling a file from mirrors.isc.org. Also, quick RTT:
[dax:14:57]% ping6 -c4 mirrors.isc.org [p8] PING6(56=40+8+8 bytes) 2001:4830:1500:21::2 --> 2001:4f8:0:2::1e 16 bytes from 2001:4f8:0:2::1e, icmp_seq=0 hlim=59 time=73.616 ms 16 bytes from 2001:4f8:0:2::1e, icmp_seq=1 hlim=59 time=73.894 ms 16 bytes from 2001:4f8:0:2::1e, icmp_seq=2 hlim=59 time=73.475 ms 16 bytes from 2001:4f8:0:2::1e, icmp_seq=3 hlim=59 time=73.810 ms --- mirrors.isc.org ping6 statistics --- 4 packets transmitted, 4 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 73.475/73.699/73.894/0.164 ms
Awesome.
Displaying page 3 of 3 of 19 results
     |
This HTML for this page was generated in 0.033 seconds. |