NFR Security

NFR Security was founded in 1996 by Marcus Ranum, a pioneer in IDS technology called Network Flight Recorder or NFR. As of 2003 NFR has been renamed to NFR Security, Inc., and the procuct line has been renamed to Sentivist Intelligent Intrusion Manager. The NFR software differs from most other products, as it provides analysis of data starting at the packet level as a sniffer would, and then it provides stateful packet inspection (misuse detection) and protocol anomaly detection using a scriptable open source language called N-Code. NFR can operate as a true IDS hybrid solution, inspecting everything in the OSI reference model from layer 2 protocols (Ethernet 802.1-based packets) up to layer 7 applications.

NFR Detection Methodology

The meaning of stateful packet inspection and protocol anomaly detection is highly misunderstood. Most of the confusion arises from the notion that examining TCP header information will provide sufficient information to successfully detect an attack. In fact, it is necessary to buffer the entire connection, including the headers and bodies of the messages transmitted, to truly identify what is happening in a connection, and most IDS products fail in that respect. Solutions that do not guarantee that data is reassembled (defragmented) correctly have an almost impossible time ensuring the correctness of the originating data.

NFR’s methodology is focused on achieving data correctness during the capture and decoding of packets. NFR provides a two-layer detection mechanism, shown in Figure 11-1. The lower layer is a high-speed engine with advanced buffering techniques that ensures proper state is maintained in packet transactions, data is correctly defragmented, and entire message bodies are recorded and reassembled in sequence for accurate detection and analysis. The upper layer provides a scriptable detection language. N-Code is a unique detection language that provides a rapid signature-development platform for intrusion detection.

Figure 11-1: Two-layer detection mechanism

NFR Architecture

The Sentivist product operates in a three-tier environment, consisting of the sensors, a Central Management System (CMS), and an Administrative Interface (AI), as shown in Figure 11-2.

Figure 11-2: AI GUI Interfaces

Sentivist Sensor

The Sentivist sensors are passive devices that collect data on the wire. The sensors provide detection capabilities from <10 MB/s connection rates through Gig speeds, using multiple network connections per single sensor. NFR’s sensors are 1-unit high rack-mountable appliances. The sensor software boots off their CD-ROM, which contains a custom written OS boot code for the sensors’ operation.

Sentivist Central Management System CMS

The collection point or Central Management System provides a single collection/aggregation point for data collected by the sensors. The data spooled on the sensors are pushed into a proprietary data file store located on the CMS. The Many to one relationship provides for 20 to 30 sensors push of alerts and forensics data to the CMS.

CMS also provides for a multifaceted Output mechanism. The CMS can provide data via any scriptable data format (such as shell scripts, or perl programs) or via the AI.

Administrative Interface (AI)

The AI is a Windows 32 program that provides a GUI interface to the alerts, forensics, and controls of the Sentivist environment. The AI allows you to select and set up the sensors’ detection capabilities, to monitor alerts and events collected by the sensors in real time, and to query supporting data collected by the sensors. All management of the IDS environment can be done via the AI.

Sentivist Signatures

One of the most unique characteristics of the Sentivist system is its attack signature model. Unlike other systems that rely on either Snort-based signatures, or closed proprietary signature models, NFR’s Sentivist provides an open signature format. The signature library permits true hybrid detection capabilities within the N-Code language. Much like Perl, N-Code provides the flexibility of a true lexical language for exploring traffic streams in real time.

The following function illustrates N-Code’s lexicon. In the example, we are looking at an excerpt that will record the time, IP source, source port, IP destination, destination port, and the source’s operating system if the real-time datastream being monitored is a returning device (that is, if it has been seen before and is stored in the $same_host[$hostarray] array).

func detect_os { $os_type = (tcp.connSrcOS); # will return a OS id string of the source #operating system seen in the current connection. If ($same_host[$hostarray] = $current_host){ #Let's check and see if we #have seen this host before? Same host is an array element and current host is #the host detected this session, the values are defined elsewhere in the code. If ("$same_host[$osarray]" eq "$os_type" ){ #Same host, is it the #same operating system? If it is then let's save the information. record attack:CURRENT_TIME, tcp.connsrc, tcp.connsport, tcp.conndst, tcp.conndport, $os_type, "Returning system"; #we record the information including source and destination IP and port #information, along with the selected operating system. $osarray++; } $hostarray++; }

N-Code provides Sentivist with the capability to analyze data from any OSI layer including the application layer. Here are some samples of data types that N-Code can analyze.

N-Code can also delve into the payload of most protocols, such as HTTP, FTP, POP, SMTP, and so on, and then can look at the payload of the actual application within the session. We will look at some of the signatures offered by NFR’s Rapid Response Team (RRT), a group dedicated to and responsible for the creation of the detection signatures in the “Sentivist Deployment Strategy” section of this chapter.

Organization of Signatures

Out of the box, Sentivist’s sensors provide a clean detection canvas because no signatures are loaded. Once a sensor is plugged in and configured, a sensor examines all traffic but does not start to analyze the data until the first time N-Code packages or backends are uploaded from the CMS. (NFR refers to signatures as backends.)

Sentivist provides signatures that are ordered in a hierarchy. Figure 11-3 shows a screen shot of the file hierarchy from the AI. Packages are parent groups that contain similar signatures or backends. A single package can contain multiple backends, such as the web package called “WWW,” which contains signatures related to web-based activity. “IIS Attacks” is a backend belonging to the WWW package, since IIS is a web-based application. Frequently the parent package will do some form of logical operation and will pass parameters off to the signature. For the WWW package, all state activity is maintained in the root package, so if a Code Red attack is detected, HTTP state will be established, and maintained by the WWW root package. The WWW backend will recognize that a web session was established, and that the content of the web session meets the criteria IIS based traffic passing parameters to the IIS signature, so that the actual IIS signature can detect the Code Red worm.

Figure 11-3: Sentivist AI package selection

A State Model in N Code

Sentivist can maintain state and execute anomaly-detection N-Code packages. The RRT (Rapid Response Team) has successfully crafted packages that maintain state and anomaly detection for several protocols, such as Web (based on RFCs such as 1945 and 2616 defining HTTP 1.x), POP, SMTP, and so on. This means that all ISO-based activity that requires state and anomaly intelligence is done by the package.

Once data has been reassembled and sorted, it is typically passed off to an application-based backend, such as the IIS Attacks backend in the WWW package. The IIS Attacks backend will then parse and alert on attacks that are relevant to the particular server type. Providing a granular means of analyzing data also makes it easier to deal with false positives when they occur. For instance, in a network environment that has no Apache servers serving as web servers, the package can be configured to alert only on IIS-based attacks.

Alerts and Forensics

N-Code provides two primary forms of data collection: alerts and record statements. It is important to understand their purpose.

Alerts

As we see in the following, both examples refer to ALERT_ID 27-64, which is the Code Red Worm event. The alert ID is assigned by the RRT as a unique event identifier. Let’s look at the alert’s details in the AI Figure 11-4 and examine the associated N-Code that generated the event in Figure 11-5.

Figure 11-4: Alert Detail from the AI

Figure 11-5: The alert statement in N-Code

We can see the components of the N-Code displayed in the AI with all the appropriate fields completed. We can the particulars of the Alert ID 27-64 with a Alert Confidence of 90, Alert Severity High, and Alert Impact as a code execution. Let us look at the components that make up the alert data type in more detail.

Record Statements (Forensics)

In addition to the alert statements Sentivist provides a means to collect forensics data associated with an alert. This data is called a query and is accessible in the AI interface in packages. The data is collected by an N-Code statement called a record. The record statement permits Sentivist to extend its ability to analyze data that is provided in an alert. The query data is available via the AI. In fact, the content of a query consists of supplemental forensics data that is representative of the particular alert being examined. The supplemental information is available for all of the signatures provided by NFR.

Following the information in the CodeRed alert shown in Figure 11-4 we see the supporting forensics data that was recorded for the Code Red alert event in Figure 11-6. This was a result of running a query in the AI for CodeRed, which executed the following N-Code to generate the results.

record www:CURRENT_TIME, tcp.connsrc, tcp.connsport, tcp.conndst, tcp.conndport,…

Figure 11-6: Query result

This record statement is not the entire signature for detecting the CodeRed Worm, nor does an operator of the AI require to see or modify the signature during operation. This information is provided to illustrate how N-Code is executed to achieve results.

This record statement is not the entire signature for detecting the Code Red worm. In fact, the signature examines the whole HTTP state, both client and server side session is checked for correct state and ensures no hidden content exists in the traffic. The www signature identifies the Code red worm as it assembles the client request to the server during the initial connection and GET request. This is only a short excerpt from a complex signature (state engine) written to maintain web transactions of all kinds.

  Note

You can read more about N-Code in the resources provided by NFR: https://support.nfr.com/nid-v3/nde/docs/Customizing_NFR_NID_Sensor_Using_N-Code.pdf.

Cool Things You Can Do with N Code

N-Code is a very flexible language. It allows you to explore the realm that traditionally only network sniffers used to address, and it offers you the flexibility of writing your own detection conditions.

For example, NFR could be used to identify all devices that are performing port address translation (PAT) that are present on your network. This could be accomplished by collecting MAC addresses and matching them to IP addresses—when multiple MAC addresses are associated to a single IP, you could then trigger an alert.

NFR could also be used to detect certain protocol behavior. For instance, if you want to see whether a particular port is used during certain times of the day, such as port 31,337 after 6 P.M., you could use N-Code to generate an alert. The possibilities are truly endless, and it only requires short development efforts to explore N-Code’s capabilities.

Central Management Server

The Central Management Server provides a central repository for all data collected, and it is a central point for managing the sensors, but it also provides for direct manipulation of events within the CMS environment.

The CMS runs as a client application on either Sun Solaris or Linux. By default, the CMS is installed into the /usr/local/nfr directory, as shown in Figure 11-7, and it is recommended that /usr/local/nfr receive a dedicated partition, or potentially a dedicated drive.

Figure 11-7: /usr/local/nfr directory

The amount of space required will depend on how much date needs to be stored. Space usage is dependent on the number of sensors, the number of backends loaded, and the length of time the data is to be saved. A 30GB hard drive is a good starting point. Further information on installation can be found at the NFR web site.

Starting Sentivist

Sentivist runs a collection service on the CMS that will pool information from sensors and distribute backend signatures, variables, and updates to sensors. Once installed, start the Sentivist services by executing ./nfrstart.sh start. This will start Sentivist on the CMS.

This startup process can be automated by setting up a self-demonizing program:

Command -> Chkconfig –add nfr Result --- nfr 0:off 1:off 2:on 3:on 4:on 5:on 6:off

Sentivist Services

The best way to explore Sentivist CMS is to see it operating, so we will examine some of the key services that Sentivist runs on the CMS. Other processes do exist, but will not be examined in depth.

These CMS services can be observed by running ps –fu nfr. On doing so, the first thing we see is that all nfr daemons run as the user that was selected during installation, which in our case is nfr (as listed in the UID column). This is the default.

UID PID PPID C STIME TTY TIME CMD nfr 21849 1 0 11:45 ? 00:00:00 bin/nfrwatch nfr 21852 21849 0 11:45 ? 00:00:00 bin/alertd nfr 21854 21849 0 11:45 ? 00:00:00 bin/getserver nfr 21856 21849 0 11:45 ? 00:00:00 bin/guisrv nfr 21857 21849 0 11:45 ? 00:00:00 bin/remotestatd nfr 21858 21849 0 11:45 ? 00:00:00 bin/spaceman

Let’s look at the running services:

There are other important services that are not listed in the ps excerpt that can be invoked by NFR when needed or invoked manually: bin/run_mirror and bin/get.

bin/run_mirror

This binary provides for the sensors and CMS to be synchronized. The mirror function will synchronize or mirror signatures and variables to all or selected sensors. This can be executed manually by running the binary from a command line.

For example, the following script will allow a user to monitor the status of the mirror: Users can execute the following command to see the process of a mirror occur, the purpose of the mirror will give insight into the services running on the Sentivist CMS.

do while true; ps –fu nfr; sleep 3; clear; done;

While this script is running, you will see the packages and backends copy out to the sensors in the process list. Once the mirror finishes copying files to a sensor, a sensor synchronization stage occurs. During this time, the sensor is compiling the signatures to byte code, and it will start using the newly loaded signatures immediately.

bin/get -n [nid name] -s [filename]

This binary provides the CMS with the means to collect information and system files from the sensors. The binary can provide an extensive capability to collect information from sensors directly. The following components can be accessed via bin/get from the sensor. From sy/ : sample usage would be sy/ipaddr0 or sy/ntpkey.

For example, bin/get –n [NID name] –s sy/hostname will return the name of the sensor. Here is an example:

>bin/get -n nid300 -s sy/hostname nid300

Additionally bin/get provides a means to execute diagnostics on a sensor by executing bin/get –n [NID name] –x exec_sys_info.

Here’s an example:

>bin/get -n nid300 -x exec_sysinfo /release -------- NID-315 3.2 CD Copyright (c) 1996-2003, NFR Security, Inc. NFR Security, Inc. http://www.nfr.com /hostname ----------- nid300 etc/patch_info -------------- 756 0 Network interfaces (ifconfig) ----------------------------- fxp0: flags=8843 mtu 1500 inet 10.0.0.8 netmask 0xffffff00 broadcast 10.0.0.255 ether 00:02:b3:87:f9:b4 media: Ethernet autoselect (100baseTX) status: active fxp1: flags=9943 mtu 1500 ether 00:20:ed:11:01:06 media: Ethernet autoselect (100baseTX) status: active fxp2: flags=9943 mtu 1500 ether 00:20:ed:11:01:07 media: Ethernet autoselect (10baseT/UTP) status: active lo0: flags=8049 mtu 16384 inet 127.0.0.1 netmask 0xff000000

In this example, exec_sysinfo returned information from the sensor, including version info and an ifconfig equivalent showing the state of the interfaces on the sensor.

Sentivist Deployment Strategy

Deploying Sentivist IDS is relatively simple and designed to be done quickly. However, the second step is tuning and configuring, and this is by far the most important step for successful upkeep of any IDS product.

Sensor deployment for Sentivist follows the same general principal as for all IDSs— covering network elements from the inside out will provide for more thorough protection. Sentivist signature elements can be selected for each sensor based on the sensor’s placement.

Sentivist provides several system-wide variables and backends. These enable the sensors to be aware of their surroundings:

The following discussions will look at basic placement principals and consider what signatures would be best suited for each sensor’s respective environment. The discussion of where to place backends and signatures will also provide some insight into their use. Note that the placement of backends in these particular cases are not to be considered universal solutions—they are simply the best solution for the case.

Case Study: Web Farm

A web farm is a high-speed network segment that is populated by web-based servers. In this case, the firewalls are considered secured as per the manufacturer of the solution, up and provide no access outside of the web-based applications required to run a web server. These servers provide web transactions, such as HTTP (on port 80) and HTTPS (on port 443) services. Additionally, management services such as Simple Network Management Protocol (SNMP) and ICMP are permitted in order to manage the web farm devices.

For this case, see the Web Farm sample illustration, the following core backends would be necessary:

Here are some additional signatures that could be used for this case:

To shore up the IDS detection capability, the Policy backend is used. The policy backend provides for an ipfilter style or access control list (ACL) style of control. The variables allow for TCP, UDP, and ICMP filtering. The rules are made up of the following elements:

command source_ip destination_ip source_port destination_port

For our web farm, the Policy backend variables would reflect the following rules.

RULES_TCP "alert any any any any" "ok any any any 80" "ok any any high any" "ok any any any 443" "ok inside any 22 any"

UDP and ICMP rules would also be used RULES_UDP and RULES_ICMP would be configured in the following manner.

Case Study 2:  Corporate Network Services

In this case we will look at a corporate network segment that much like the web farm segment, but smaller and more diverse. In this case, we will include services that would commonly be used by a corporate network in daily operation, such as mail, DNS services, web services, and potentially an Internet Relay Chat (IRC) type of messaging service. This is a protected network segment, and it includes critically important network elements.

We can use the following backends that were used in the previous case:

The following backends should also be considered:

With this environment, shoring up the configuration with the policy package may be a bit more difficult, but it is still a viable way to provide a “catch everything else” solution. Let's look at the three policy backends and how they complement the sensor deployment.

RULES_TCP "alert any any any any" "ok any any any 80" "ok any any high any" "ok any any any 443" "ok inside any 22 any"

  Note

These variables will require a separate entry for all incoming and outgoing protocols, the

"ok" entries will prevent alerting on protocols, and port that are expected, and will cause false alert.

RULES_UDP "alert outside any any any" RULES_ICMP "alert outside any any any"

Case Study 3: Bare Internet Connection

When using an IDS on a bare Internet segment, it can become quite difficult to separate false positives from true positives. More often than not, the purpose of the IDS in this case is to alert people that “a-ha it is an attack.” Tuning of this sensor is a bit more complicated, since we are trying to watch everything. As a result, the alerts will provide information about both benign and malicious activity. Even if the events are malicious, attacks may not affect the network the sensor is watching, since the event may have been stopped by the firewall, it may not involve any of the protected hosts, or it may simply have been an automated scan that just happened to sweep the protected network.

For this scenario, we will take a stock system and tune down some elements that will create extensive false positives or that would load the system down so significantly that it might be difficult to read through the alerts.

The following backends have been covered in the last section.

  • Authentication
  • POP
  • DNS
  • SMB
  • FTP
  • SMTP
  • ICMP
  • SSH
  • IMAP
  • Telnet
  • IRC
  • TFTP
  • MISC
  • >WWW

Here are some other packages that may provide additional value:

Using the Policy package is not advisable outside the default installed state, since a large number of false positives will be seen as a result of the backend being enabled incorrectly. This backend would have to be used with caution and careful observation. By default, the backend is set up to monitor IP ranges inside TCP, ICMP, and UDP that are part of the reserved IP ranges.

Case Study 4: LAN Segment

A LAN segment can be the most interesting case to work with, and NFR can provide useful information for managers when the cost of ownership for security equipment becomes an issue.

Traffic on an internal network segment will consist of data created or altered by staff, people on associated networks (partners or affiliates), and guests, invited or not! The organization's assets are located on the internal segments, and this includes traffic that may or may not follow corporate policy. For this type of case, tuning requires hands-on exposure to the data. Since no two network environments are the same, no two IDS deployment tunings will be the same.

Initially it is best to disable all the backends, and then start adding in backends such as Trojans, DDoS, WWW. Once these are in place, they should be used to watch the network traffic for a period of time to identify trends. These results will dictate what packages and backends will cause a flood of false positives. Normalization will occur once false positives are reduced to almost zero, this can be called a normal traffic pattern. Once the data starts to normalize, additional backends can be loaded. However, after each backend is loaded, you should wait for a while to see the results of the change. If well executed, this can take from several hours to several days. The result of this process is guaranteed to provide a clear definition of the types of data on the network.

NFR Reporting

Within the AI several reports can be generated for trending and general graphing of alert traffic. All these reports are available via the AI. The reports are simple and functional to produce. Additionally the report data can be exported to .CSV format for use in a spreadsheet application such as MS Excel.

NFR provides several report mechanisms, for detailed usage of the reporting mechanisms refer to the nfr AI users manual https://support.nfr.com/nid-v3/console/ai/docs/NFR_AI_v3_Users_Guide.pdf.

Extending NFR

The extensibility of NFR is a true strength of the product. We have seen how we can extend the product’s detection ability with N-Code, and how interacting with the CMS can extend visibility to the sensors. NFR additionally provides generic alerting and command line access. All of these provide a means to extend the alerting and management in an automated fashion.

One such extension uses Perl. The NFR’s Perl tools provide access to the data and alerts that NFR stores on the CMS via command line (CLI). There are several Perl tools provided by NFR: popups.pl, isummary.pl, query.pl, all provide a quick means to access the data without starting the AI.

Summary

In this chapter we looked at the NFR Sentivist Intrusion Detection system and examined how it operates and handles data in a unique fashion. We also examined the product three tier architecture and several key services that operate the product and can be used to diagnose the services. The chapter examined four deployment scenarios examining the signatures used in context of each case deployment. We examined Sentivist signatures, as it would relate to Web Farm, Corporate Network Services, Bare Internet connection, and Lan Segment deployments, looking at the best way to approach each of the solutions when deploying Sentivist Intelligent Intrusion Manager.

Related

Категории