"The advantage of this approach is that it's dumb, it's cheap -- and it catches stuff you don't know about already." --mjr (who wasn't really talking about this code.. but..)
The other way is to make a list of all the "normal" system behaviour -- connections from "trusted" systems, ordinary mail delivery, cron logs, named stats messages, and so on-- and ignore it. Everything else is considered "out of the ordinary" and hence should be paid close attention to. This program attempts to do this. I didn't come up with this idea on my own; in late 1996 I'd been reading about it in the IDS list (Intrusion Detection Systems) for some time, and on 4 December 1996, decided to implement it.
This program is designed to analyze Unix syslog files, the concept applies to logfiles with largely predictable content-- web server error logs, for instance.
So you spend some time looking through your syslog, and you come up with entries such as:
Apr 15 12:34:45 mybox in.telnetd: connect from shell.isp.com Apr 22 01:25:26 mybox in.telnetd: connect from linux.friendly.org Apr 23 14:27:39 mybox in.telnetd: connect from linux.friendly.org Apr 27 20:13:02 mybox in.telnetd: connect from pc2.foo.edu Apr 29 03:31:29 mybox in.telnetd: REFUSED CONNECT FROM 0wned.broken.comYou would then construct a rule such as:
in.telnetd\[\d+\]: connect from (friendly\.org|pc2\.foo\.edu)
This essentially says
"match the string '
]: connect from either
Note that regexp metacharacters such as [, ], and . are escaped by putting a backslash in front of them. Also note the use of the perl regexp \d+, which matches 1 or more digits.
Now, when you run checksyslog against this syslog file, only two entries will stand out:
Apr 15 12:34:45 mybox in.telnetd: connect from shell.isp.com Apr 29 03:31:29 mybox in.telnetd: REFUSED CONNECT FROM 0wned.broken.com
The rule file also supports definitions, so instead of spelling out the friendly.org and pc2.foo.edu hostnames multiple times in the rule file (which can get tedious, particularly if you want to add or remove things from the list), you can do something like:
%SECURE_HOSTS=(friendly\.org|pc2\.foo\.edu) . . . # ignore telnet or ftp sessions from %SECURE_HOSTS in.telnetd\[\d+\]: connect from %SECURE_HOSTSThe construction of rules is the most painful part of configuring checksyslog. There is a sample rule file which hopefully makes things clearer.
*.debug @loghost.my.domainso "Everything" gets logged to a single file; this is how I like to do things. If you're one of these people with 700 different lines in your syslog.conf pointing to different files and log levels, then you'll have to run a separate instance of checksyslog to analyze those files. (My philosophy: "Log 'em all and let Grep sort 'em out." :-)
In some cases (observed when logging from Solaris 2.5 to Linux), the end of line will be padded with extra (invisible to the nekkid eye) characters. This will affect your usage of '$' to indicate the end-of-line in the rules file.
Edit the example.rules file to suit your particular setup -- you may need to run checksyslog several times in order to separate the wheat from the chaff. Copy the rulefile to an appropriate location, and chmod it to 400 (why give out more information when you don't need to?)
--rulesrulefilename -- Specifies the location of the rule file. This is a required argument.
--loglogfilename -- Specifies the location of the logfile (i.e., /var/log/syslog). If not set, input is taken from stdin.
--today-- Only show messages from today
--filterfilterstring -- Only extract message matching this particular filter (make sure you quote "things that might look like arguments to the shell")
In an effort to make the resulting output more palatable, another program named 'resort' breaks up the output of checksyslog on process name, instead of by date:
== == nimue -- in.ftpd (2 entries) == May 3 01:23:02 nimue in.ftpd: connect from 172.16.195.3 May 3 01:23:04 nimue in.ftpd: connect from 22.214.171.124 == == nimue -- ftpd (4 entries) == May 3 01:23:03 nimue ftpd: FTP LOGIN FAILED (cannot set guest privileges) for nimue.int.jammed.com [172.16.195.3], ftp May 3 01:23:03 nimue ftpd: FTP session closed May 3 01:23:04 nimue ftpd: FTP LOGIN FAILED (cannot set guest privileges) for nimue.int.jammed.com [126.96.36.199], ftp May 3 01:23:04 nimue ftpd: FTP session closed == == nimue -- PAM_pwdb (19 entries) == May 1 01:13:08 nimue PAM_pwdb: (su) session closed for user root May 1 13:22:23 nimue PAM_pwdb: (su) session opened for user root by jwa(uid=100) May 1 13:23:02 nimue PAM_pwdb: (su) session closed for user root May 1 14:59:11 nimue PAM_pwdb: (su) session opened for user root by jwa(uid=100) [ .. snipped for brevity .. ]
checksyslog --today --rules /usr/lib/checksyslog.rules --log /var/log/syslog | resortI run a variant of this out of cron at 23:59 (so
--todayworks. I really otta add a
If your rules are sufficently vague or if an intruder can peek at your rulefile, in theory he or she could launch an attack on your system without triggering checksyslog -- as long as the intruder imbeds a rule entry in each system log file message he creates. In practice, this is difficult to accomplish, but it is something that should be taken into account.
You can't grep dead trees. But hopefully, this program will reduce the amount of paper :)
Please contact me (firstname.lastname@example.org) if you have any comments, suggestions, questions, bugs, or useful rules.