Logging
Introduction
Logging events that occur on the router is an important tool available to router and network administrators. Logging provides real-time and historical information about router operations, which you can use to help trace and analyze the sequences of events leading to a problem on the router or network, or both. The JUNOS software provides two mechanisms for logging events: system logging (sometimes called syslog) and tracing. With system logging, the JUNOS software generates system log messages (also called syslog messages) that record events that occur systemwide on the router, such as a user logging in to the router or an interface starting up; failure and error conditions, such as a login failure or the unexpected closure of a peer process; and emergency or critical conditions, such as a router shutting down due to excessive heat. JUNOS system logging is very similar to the Unix syslog function. Tracing (sometimes also called trace logging) is specific to routing protocols and records information about protocol operation, such as the exchange of protocol packets when a protocol is starting or sending regularly scheduled updates.
Both system logging and tracing save log messages to files. These files are stored in the /var/log directory on the router's hard disk for M-series and T-series routers and in the /cf/var/log directory on J-series routers. You can redirect system log messages to a remote server that is running a standard syslogd utility, to the terminal of a user who is logged in to the router, or to the console.
The JUNOS software can generate thousands of different system log messages, from all parts of the system, including hardware, routing software processes, and forwarding software. The messages are categorized by source and severity. Because you are almost never interested in saving and reviewing all system log messages generated by the router, use the source and severity as log message filters.
Each system logging message is identified with a priority, consisting of a facility and a severity level. The facility is the source of the message, which is the router process or event that generated the message. Table 5-1 lists all the JUNOS system log facilities.
Some are the same as those used by the Unix syslog utility, and some are specific to the JUNOS software.
|
Facility name |
Facility code |
Message source |
|---|---|---|
|
any |
Any facility |
|
|
authorization |
AUTH, AUTHPRIV |
Authentication and authorization attempts |
|
change-log |
CHANGE |
Router configuration changes |
|
conflict-log |
CONFLICT |
Router configuration changes that are inconsistent with the router hardware |
|
CONSOLE |
Kernel messages to the console (/dev/console) |
|
|
CRON |
Scheduled processes |
|
|
daemon |
DAEMON |
JUNOS software processes |
|
firewall |
FIREWALL |
Packet filtering done by firewall filters |
|
ftp |
FTP |
FTP |
|
interactive-commands |
INTERACT |
Commands issued at the JUNOS CLI or by a JUNOScript client application |
|
kernel |
KERNEL |
JUNOS kernel |
|
NTP |
NTP |
|
|
pfe |
PFE |
Packet forwarding software |
|
SYSLOG |
System logging |
|
|
user |
USER |
User processes |
Each system log message has a severity level (see Table 5-2) that reflects the seriousness of the event that generates the message. Each severity level has a name and number, which are the same as those used by the Unix syslog utility. The lower the number, the more critical the event.
|
Severity name |
Severity number |
Description |
|---|---|---|
|
any |
All severity levels |
|
|
none |
All severity levels |
|
|
debug |
7 |
Information normally used in debugging |
|
info |
6 |
Informational events about normal router operations |
|
notice |
5 |
Conditions that are not errors but are of more interest than normal router events |
|
warning |
4 |
General warnings for events you might want to keep an eye on |
|
error |
3 |
General error conditions |
|
critical |
2 |
Critical errors, such as hard drive failures |
|
alert |
1 |
Errors that require immediate correction, such as corrupted system files |
|
emergency |
0 |
Conditions that cause the router to stop functioning |
Depending on how you configure system logging, the JUNOS system log messages have one of the following formats. The first format is the default.
Mar 17 11:12:29 router1 mib2d[2885]: SNMP_TRAP_LINK_DOWN: ifIndex 2, ifAdminStatus up(1), ifOperStatus down(2), ifName t1-0/0/0:1 Mar 17 11:12:29 router1 mib2d[2885]: % DAEMON-4-SNMP_TRAP_LINK_DOWN: ifIndex 2, ifAdminStatus up(1), ifOperStatus down(2), ifName t1-0/0/0:1
The system log message includes a timestamp, the router's name, and the message itself. The timestamp indicates the date and time when the message was logged. Missing from the timestamp is any indication of the time zone. If all your routers are located in a small geographic area, this is not much of a problem. However, if your operations are more global, you should make sure that you either configure all routers to use the same time zone ( UTC is a good choice) or, less optimally, that you know which routers are using which time zone. Knowing the time accurately on your network's routers is critical when you are searching through logfiles to debug a problem between two routers and are trying determine what happened when. Setting time zones is discussed in Recipe 6.2.
The second part of the log message is the actual log message itself, which shows the source of the message and the message code and description. The message in the previous example was generated by the MIB-II process, and the specific process number is 2885. (If the process is still running, you can see it with the show system processes command.) The message code consists of a prefix, in this case SNMP_, which is the process that generated the message, and a unique message identifier (trAP_LINK_DOWN). The text string at the end describes the message. The second message format above shows two other pieces of information, the facility code name (here, DAEMON, indicating that the message source is a JUNOS software process) and the numeric severity level (4, which is a warning message).
A quick way to find out what a system log message means is to use the help syslog command. You can just cut and paste the message code into the command:
aviva@router1> help syslog SNMP_TRAP_LINK_DOWN Name: SNMP_TRAP_LINK_DOWN Message: ifIndex , ifAdminStatus , ifOperStatus , ifName Help: linkDown trap was sent Description: The SNMP agent process (snmpd) generated a linkDown trap because the indicated interface changed state to 'down'. Type: Event: This message reports an event, not an error Severity: warning