Passive OS Fingerprinting

Problem

Can I use some tool(s) to possibly perform some OS fingerprinting?

Solution

OS fingerprinting is the idea that every platform has a unique TCP/IP stack. There are several tools and methods that use Snort to determine an OS platform of a given system crossing your network(s). If you are interested in modifying the Snort source code to detect/determine a hosts' OSes, the snortfp project would be best suited to your needs. If you are going to use your own methods to combine the data, using p0f is your best tool in determining the host OS. Finally, if you are going to use commercial products, the Sourcefire RNA product is the way to go.

snortfp

This was a preprocessor that modified the Snort 2.1.2 code and the web frontend ACID to display OS fingerprints of a host. However, this project hasn't been worked on since that version of Snort. Also, this patch only seems to patch Snort correctly on Linux platforms; if you are using BSD, Solaris, or Windows platforms, you are out of luck, unless you care to recreate the code. Download the patch files from http://mysite.verizon.net/sdreed.

The project uses Snort rule extensions to detect an OS fingerprint. Then it changes the ACID web code to display and search through these OS fingerprints, as can be seen in Figure 7-5, from the site mentioned earlier.

Figure 7-5. Example screenshot from snortfp

As this code runs on an older version of Snort, use it with caution and monitor it for faults.

p0f

p0f (http://lcamtuf.coredump.cx/p0f.shtml) is probably the most widely used OS-detection tool that doesn't need to actively probe the target to determine the OS. While letting the p0f tool keep its own log and then maintaining that log along with your IDS logs may be enough for some organizations, most will want to integrate the data into a useable, searchable format; that would be a database. Specifically pushing the p0f data into your ESM/SIM would probably provide analysts with some helpful information when an event is detected. The current p0f tool 2.x doesn't natively support database output, yet. There is a project called p0f_db (available at: http://nk.puslapiai.lt/projects_en.shtml) that will parse p0f data into both a MySQL and PostgreSQL databases. As of Version 0.3, this code seems to run only on Linux and not BSD systems.

Sourcefire RNA

Finally, since this is a Snort book, it's only fair that we show you a commercial product that helps solve this problem as well. This quote from the sourcefire web site (http://www.sourcefire.com) that gives a brief overview of the application(s) should help.

If buying Sourcefire is an option to you, contact them for a demo and sales.

Discussion

Because operating system vendors add their own "tweaks" to the TCP/IP stack implementation, each operating system has a unique "signature." For a detailed discussion of this topic, see Fydor's paper on the nmap site:

http://www.insecure.org/nmap/nmap-fingerprinting-article.html

The use of passive fingerprinting can be helpful in determining the attacker and/or the victim's risk. For example, if you get a Snort alarm for a Microsoft IIS command execution attempt ../../cmd.exe?/+dir on an Apache server, the p0f logs can be used to show that the "victim" machine is not vulnerable to the attack.

While this type of information can be a virtual gold mine for government organizations and certain other businesses, not every organization is going to be able use or keep this type of information. There are several projects and code modifications available to those who would like to use this capability. However, this also is dipping into the realm of the Enterprise Security Manager/Security Incident Manager (ESM/SIM) vendors such as Arcsight, Netforensics, and Guardent. These vendors specialize in taking data from multiple types and normalizing them into a single event, such as taking in logs from your firewalls, IDS, routers, vulnerability scanners, and even patch management logs in some cases. But as you can probably guess, that topic alone could be enough for another book!

snortfp

This tool seems to compile correctly only on Linux platforms and not on BSD platforms. This patch is actually two parts combined into a single file that you download from the author's site. The first part is the actual Snort source code patch, while the second part is a modification to the ACID web frontend (Chapter 5). The Snort patch adds a couple of changes to the Snort code; the major modification is to the rule structure itself. This change enables the ruleset that you downloaded to trigger an event on packets that can be used to determine an OS. These packets are then logged to the modified ACID database where they will be displayed. As mentioned earlier, the ACID frontend has to be modified as well as the MySQL database (no other databases are supported at this time). This modification takes the form of some structure changes, as well as the addition of a single new table to the ACID schema.

First you need to get the patch file (snort-2.1.2fp.patch.gz) from the author's site (http://mysite.verizon.net/sdreed) as well as the Snort source code (snort-2.1.2.tar.gz) from snort.org. Then, follow these steps to get it installed:

# put the snort and the patch in the same directory then patch # the snort source code snortuser# pwd /tmp/SNORT_FP snortuser# ls snort-2.1.2.tar.gz snort-2.1.2fp.patch.gz snortuser# tar xvfz snort-2.1.2.tar.gz snortuser# gunzip snort-2.1.2fp.patch.gz # Now we patch snort snortuser# patch -p0 < snort-2.1.2fp.patch # Now we build snort with mysql support and whatever else we want snortuser# ./configure --with-mysql -other-options # only make the application don't install yet unless you are sure # that this build and patch worked. snortuser# make # the second part of this patch is to patch ACID database schema # and the ACID web interface. # First let's go patch the database. Create a database and the schema # using the scripts found in the snort config directory. snortuser# mysqladmin create snortfpdb snortuser# cd snort-2.1.2/contrib snortuser# mysql -D snortfpdb < create_mysql snortuser# mysql -D snortfpdb < snortdb-extra # Now we apply the snortfp schema changes # COMMENT OUT the line "use snort;" from create_mysql_fp unless your # database is named snort. snortuser# mysql -D snortfpdb < create_mysql_fp # Now we install the ACID web interface like in recipe x.x # Then we patch ACID using the new file in the contrib directory of # snort # If ACID was installed in /usr/local/apache2/htdocs/ACID/ directory. # NOTE: Please run ACID web CREATE AG group schema change first # otherwise the changes for snortfp aren't made. snortuser# cp ACID-0.9.6b21fp.patch /usr/local/apache2/htdocs/ACID/ snortuser# cd /usr/local/apache2/htdocs/ACID snortuser# patch -p1 < ACID-0.9.6b21fp.patch # reload your main ACID page and notice the "Network Fingerprints" # button! # TEST SNORT TO PUSH IN SOME EXAMPLE DATA snortuser# cd /tmp/SNORT_FP/snort-2.1.2/ # change the "snort.conf" file to handle our new data # "output database: log, mysql, user=snrtfp password=password # dbname=snortfp host=localhost" # NOW let's test it out # Create a directory called "log" in the snort directory if you don't # want to change anything on your system snortuser# ./src/snort -c etc/snort.conf -l log -i eth0 -T # if you don't get errors or a core dump then the patch was applied # correctly and you should see some data in your ACID frontend.

p0f

p0f has been kept up to date since the new author started with Version 2.0 a little over a year ago. This tool uses the libpcap library just like Snort to passively sniff packets off the network. The best description of how p0f detects the operating system comes from the author himself at http://lcamtuf.coredump.cx/p0f/:

The passive OS fingerprinting technique is based on analyzing the information sent by a remote host while performing usual communication taskssuch as whenever a remote party visits your webpage, connects to your MTAor whenever you connect to a remote system while browsing the web or performing other routine tasks. In contrast to active fingerprinting (with tools such as NMAP or Queso), the process of passive fingerprinting does not generate any additional or unusual traffic, and thus cannot be detected. Captured packets contain enough information to identify the remote OS, thanks to subtle differences between TCP/IP stacks, and sometimes certain implementation flaws that, although harmless, make certain systems quite unique. Some additional metrics can be used to gather information about the configuration of a remote system or even its ISP and network setup.

This tool can be used in conjunction with Snort to help accomplish the same goals as those of the snortfp project, though with a little less work. One possible use if you are not planning on pushing this data into an ESM and don't want to modify your frontend for IDS data would be to create a new frontend for this data to be searched in addition to the IDS data. While creating an additional step in your analyst's investigation process, it might prove useful in determining network threat. The following example creates a p0f logfile that is rotated and marked for the date.

#!/bin/sh # mydate=`date +"%d%m%Y" ` echo "STOPPING p0f detection" killall p0f echo "Rotating LOGS" mv /logs/p0f.log /logs/p0f.log.$mydate echo "STARTING P0f detection" p0f -i -o /logs/p0f.log -l & p0fPID=`ps -aux | grep p0f | grep -v "grep" | awk '{ print $2 }' ` echo "Started p0f with PID: $p0fPID " # All Done

For simplicity, we'll set this script to rotate daily. The following example should be added to your crontab on your sensor(s) to rotate the previous script every day at 11:55 PM.

# edit crontab (crontab -e) and add 55 11 * * * /bin/sh /path/to/the/above/script

Another solution would be to log the p0f logs into a database to be searched from a web frontend, ESM/SIM, or other custom tool. The advantages of placing logs in a database are:

Scalability

Databases are easier to manage and maintain than flat files, especially in high traffic networks.

Searchable

Once the logs are in a database, you can create charts, graphs, and even trends from the data for reports.

Limited access

If you work with law enforcement at some point, you'll find it handy if the data has as few modifications as possible and comes from as few sources as possible. For example, set up one account with write access and no read access, using this account for the portion of any tool that actually collects the data, while creating another account for searching through the data that has read and search permissions but not change access permissions.

As of p0f Version 1, there was a project from Bill Stearns called p0f-mysql that gave some support to logging p0f data to a database. However, in p0f Version 2.x there is a new project called p0f-db that supports both MySQL and PostgreSQL database formats for the data. Again, however, this seems to work only on Linux platforms and not BSD platforms. This application is actually a second application to run as an output portion of p0f. First download and install the source code, available at the following site: http://nk.puslapiai.lt/projects/p0f_db/:

bash$ bunzip2 p0f_db-0.3.tar.bz2 | tar xvf - bash$ ./configure --with-mysql & make # OR if you prefer PostgreSQL #bash$ ./configure --with-postgres & make

Once this compiles correctly, create the database and account for p0f_db to use (using the same method as for the ACID setup). The following code uses the direct MySQL command line; if you are not comfortable editing a MySQL database through this method, a web-based tool called phpmyadmin is available at http://www.phpmyadmin.net or http://sourceforge.net.

bash$ mysql -u root -p # Create the database mysql> CREATE database p0f_db; # create the user for the database mysql> GRANT UPDATE, INTSERT, SELECT on p0f_db.* TO p0fdata@localhost IDENTIFIED BY "password"; mysql> flush privileges; mysql> exit

Once the database and its permissions have been set, remove the snort-inline options from the p0f-db code, unless you are deploying inline on a sensor. Edit the file in p0f_db/doc/create_mysql.sql removing or commenting out all lines that start with DROP. Once complete, create the schema for the database to use to handle data.

bash$ mysql -D p0f_db < doc/create_mysql.sql # Next edit the file p0f_db.conf with the database information from the previous example. # mysql host=localhost user=p0fdata password=password dbname=p0f_db

Finally, now that the database is set up to handle the p0f-db data, simply start p0f-db and watch for errors to correct. If no errors are found, simply run either via script/cron or directly as needed, as in the following:

bash$ ./src/p0f_db ---config=doc/p0f_db.conf ---log=/logs/p0f.log

Hopefully these examples should give you some ideas on how to use and integrate this information into the processes and procedures your analysts follow when investigating an event.

See Also

Snort!(fp) (http://mysite.verizon.net/sdreed)

p0f (http://lcamtuf.coredump.cx/p0f.shtmlhttp://lcamtuf.coredump.cx/p0f.shtml)

p0f_db (http://nk.puslapiai.lt/projects/p0f_db/)

p0f-mysql (http://www.stearns.org/p0f-mysql)

Sourcefire (http://www.sourcefire.com)