Internet Information Services (IIS) and the Internet Protocols

IIS 6 is an integrated set of Internet services available in Microsoft Windows Server 2003 that provides Web publishing, file transfer, network news, and mail services. IIS 6 includes servers for Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP), Network News Transfer Protocol (NNTP), and Simple Mail Transfer Protocol (SMTP). In this chapter, we look at each protocol and the functionality it provides.

HTTP

HTTP is an application-level protocol used for communications and data transmission between client computers and HTTP servers. Client requests are frequently submitted from browser software such as Microsoft Internet Explorer and Netscape Navigator to HTTP servers such as IIS. HTTP is the protocol underlying today's World Wide Web.

Early versions of HTTP, referred to as HTTP/0.9, have existed since 1990. Between 1990 and 1996, HTTP continued to evolve in an uncontrolled manner. It was common for different vendors to modify the protocol to suit their own needs. RFC 1945 was the first formal HTTP specification. To some degree, a major objective of RFC 1945 was to summarize and unify the implementation details. As a result, the industry generally expected it to be replaced rather quickly. For example, HTTP/1 lacked sufficient definition for caching, hierarchical proxies, virtual hosts, and persistent connections. A more stringent version, HTTP/1.1, was defined in RFC 2068, which has been updated in RFC 2616. IIS 6 contains an HTTP/1.1-compliant server.

Before analyzing HTTP in depth, you must understand certain basic terms as they relate to the protocol:

• MessageThe basic unit of communication between a client and a server. Messages are usually sent as part of a TCP connection between a short-lived TCP port on the client to TCP port 80 on the server, although administrators can configure the server to use other TCP ports.
• ResourceAn object or service available on the server that is identified in a message. Typically, this will be a Hypertext Markup Language (HTML) orExtensible Markup Language (XML) page, but it can be any sort of object,including a text document, graph, or sound file.
• RequestA message from the client to the server that requests a resource. Most HTTP interactions involve a client sending a GET request message to the server.
• ResponseA message from the server to the client that returns information initiated by a request message. This is the contents of the page or filerequested by the client.
• MethodThe action to be performed on the requested resource. Also known as a request method, this is the specific action that the HTTP server is being requested to perform. Usually, this is a GET, but there are other methods a client can request.
• ClientAny program that establishes a connection to an HTTP server to issue requests. Typically this is a Web browser such as Internet Explorer, Opera, or Mozilla. There are other applications that can act as HTTP clients.
• ServerA process that accepts HTTP requests for connections from client programs, and provides response data. A single program can utilize both client and server components. In Windows Server 2003, the HTTP server is a component of IIS 6.
• CacheA client proxy or server's store of response messages, used to retain resources that are designated as cacheable. By caching responses, retrieval of data that has been recently requested does not require additional network traffic, as the request can be fulfilled locally. Internet Explorer holds a cache of recently accessed resources.
• CacheableA response message that is permitted to be stored by the requestor, as determined by a set of rules governing caching. By default, resources are cacheable.
• TunnelA Transport-Layer intermediary between client and server programs that does not take part in the request/response process, except to relay information between the programs. A tunnel is closed when both endpoint parties to the communication have closed their connections.
• GatewayAn HTTP server that receives requests on behalf of another server, often appearing to the client to be the server that was queried.
• ProxyA program that acts as both client and server in HTTP communication, receiving request messages from a client program, repackaging the requests as if the proxy were the client, and returning the responses to the original requestor. Microsoft Internet Security and Acceleration (ISA) Server provides a secure, high-performance proxy facility.
• Uniform Resource Identifier (URI)A standard format used to describe a resource requested from a server.
• Uniform Resource Locator (URL)A standard naming method used to define the location of any resource on the Internet. URLs take the format protocol://hostname:port/filename, for example, http://kapoho.com:80/tcp/default.html.
• RangeHTTP messages are represented in sequences, or ranges, of bytes. When a client wishes to retrieve a resource from an HTTP server, it might need to specify a range of the total bytes that make up the resource, as the entire resource might be too large to transfer in a single transaction. Servers might also specify the range measurement that they are capable of accepting, such as bytes.
• EntityA resource, or portion of a resource, that is being referred to in arequest/response exchange between HTTP clients, proxies, and servers.
• VariantResources can be represented by more than one name; additional names for a single resource are called variants.

HTTP in Operation

HTTP is a request/response protocol. A client wishing to retrieve a resource from an HTTP server issues a request message containing a request method, URI, protocol version ID, and resource-specific information. The following is a trace of a simple request andresponse between an HTTP client and server. The full content of Capture 21-01 is provided on the companion CD-ROM.

12 1.838983 0050564050E1 0050564050EA HTTP GET Request (from client using port 4283) 10.10.1.68 10.10.1.74 + Frame: Base frame properties + ETHERNET: ETYPE = 0x0800 : Protocol = IP: DOD Internet Protocol + IP: ID = 0x9BAB; Proto = TCP; Len: 314 + TCP: .AP..., len: 274, seq:1182807736-1182808010, ack:3591543841, win:17520, src: 4283 dst: 80 HTTP: GET Request (from client using port 4283) HTTP: Request Method = GET HTTP: Uniform Resource Identifier = /rebecca.htm HTTP: Protocol Version = HTTP/1.1 HTTP: Accept = image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */* HTTP: Accept-Language = en-us HTTP: Accept-Encoding = gzip, deflate HTTP: User-Agent = Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.0.3 HTTP: Host = kapoho10 HTTP: Connection = Keep-Alive 13 1.848998 0050564050EA 0050564050E1 HTTP Response (to client using port 4283) 10.10.1.74 10.10.1.68 + Frame: Base frame properties + ETHERNET: ETYPE = 0x0800 : Protocol = IP: DOD Internet Protocol + IP: ID = 0xA119; Proto = TCP; Len: 441 + TCP: .AP..., len: 401, seq:3591543841-3591544242, ack:1182808010, win:17246, src: 80 dst: 4283 HTTP: Response (to client using port 4283) HTTP: Protocol Version = HTTP/1.1 HTTP: Status Code = OK HTTP: Reason = OK HTTP: Content-Length = 176 HTTP: Content-Type = text/html HTTP: Last-Modified = Tue, 27 Aug 2002 22:05:16 GMT + HTTP: Undocumented Header = Accept-Ranges: bytes + HTTP: Undocumented Header = ETag: "0b23ed3154ec21:68a" HTTP: Server = Microsoft-IIS/6.0 HTTP: Date = Tue, 27 Aug 2002 22:06:49 GMT HTTP: Data: Number of data bytes remaining = 176 (0x00B0)

In this trace, a client, kapoho10, issues a GET method request to the server with the intention of getting the page ebecca.htm. The server then responds with the page (contained in the data field). The preceding summary omits the necessary name resolution and TCP connection overhead, although Capture 21-01 (found in the Captures folder on the companion CD-ROM) includes these exchanges.

An HTTP request can be issued to the server, a tunneling agent, a proxy, or a gateway. Each participant in the transaction can simultaneously process multiple HTTP connections, which are typically issued over TCP port 80, although the participant applications can specify other ports.

One restriction of most implementations of HTTP/1 was that a separate connection had to be established for each request. With HTTP/1.1, multiple transactions can be processed over a single connection, which is kept open. This is called a keep-alive, which can improve protocol performance and reduce server overhead. You can see the keep-alive request in the preceding trace. HTTP provides no mechanisms for guaranteed delivery of messages. HTTP relies on TCP to provide this functionality.

After receiving a request message, an HTTP server responds with a message indicating success or error, as well as protocol-version information and possible resource-specific data, including the data itself. In the preceding example, the server returns the page ebecca.htm to the client.

The protocol version is sent in the format "HTTP/x.y," where x is the "major" version identifier and y is the "minor" version identifier (in the preceding example, this is HTTP/1.1). By sending version numbers as part of the HTTP message, clients and servers negotiate communication format, with both client and server sending the highest protocol version they both understand. With the exception of tunnels, which do not maintain any awareness of the contents of HTTP data, each party in the process can potentially cache message content to facilitate faster retrieval for future requests.

A proxy is not permitted to send a request message in a protocol version higher than the proxy itself supports. In addition, it is required to do one of three things: downgrade any incoming request message with a higher version than its own, respond to the client with an error message, or switch to tunnel mode. Proxies are also required to upgrade any lower protocol version requests to match their own, but return the response to the client in the same major version as the original request.

URIs

A URI is simply a standard format for defining a retrievable resource. A familiar term, the URL, is actually a subset of the URI, as is the less familiar Uniform Resource Name (URN). A retrievable item can be requested by referencing the item's location, as in the case of a URL, or by referencing the item's distinguished name, as in the case of a URN. The HTTP protocol does not limit the length of URIs, and servers are required to be capable of receiving a URI with a length that is at least as long as that of any resource they serve.

URI Syntax

URIs can be absolute or relative to a base URI. Although RFC 2396 defines generic syntax, several RFCs define the syntax of a URI. An absolute URI lists the entire scheme (this is usually the name of the protocol used in this request) and path that will be used to request the resource, whereas relative URIs build on a previously established base location and scheme. A typical URI construction is as follows:

://:/?

It might appear as:

http://search.msn.com/results.asp?q=Thomas+Lee+Joseph+Davies+TCP%2FIP

In this example, the scheme is http:, indicating that the request will be passed over HTTP. The slashes are reserved characters that serve as separators between scheme and scheme-specific details. The host name given here is search.msn.com, so the request is issued over TCP port 80 to the server at search.msn.com. If a port were specified, the TCP connection would be opened over that port. If for some reason the target server is not listening for requests over this port number, an error results. The absolute path given hereis to the search facility, with the rest of the URI being parameters to send to this search engine. The presence of a question mark following this path indicates that a query is being performed, and the remainder of the URI is query parameter information (in this case, the keywords that should be searched for).

HTTP Messages

As noted earlier, HTTP clients and servers use HTTP messages as the basic form of communication. These messages are designed to support a heterogeneous collection of servers and follow a well-defined syntax.

Message Types

RFC 822 defines HTTP request and response messages as being used to transfer data and resources, called entities, from client to server, and vice versa. These messages are comprised of a start line that might or might not be followed by header fields, which are often simply referred to as headers. An end-of-line marker (carriage return line feed [CRLF]) indicates the end of the headers, and message body data might follow. HTTP/1.1dictates that no more than one CRLF be used sequentially in any HTTP message to eliminate unnecessary parsing on the part of either the server or client, although flawed implementations of HTTP often issue multiple CRLFs in succession. If a CRLF is received where none is expected, it is ignored. Syntax of a typical HTTP message is the following:

Request-Line|Response-Line *((general-header|request-header|response- header|entity-header)CRLF)CRLF [message-body]

Message Headers

HTTP header fields are used to define parameters regarding the message being transmitted, whether the message is general, request, response, or entity information. Header fields can be preceded by any amount of linear white space (LWS), and consist simply of the header name followed by a colon (:) and the header value. General header fields apply to both request and response messages but do not relate to the entity being transferred. Request and response header fields are specific to their respective message types, and entity header fields provide additional information about the resource being passed in the message.

Request Messages

Client programs issue request messages both to establish communication parameters and to initiate resource transfer and manipulation. The following example shows the headers of the message generated by a client requesting a resource from a server (as demonstrated in Capture 21-01):

HTTP: GET Request (from client using port 4283) HTTP: Request Method = GET HTTP: Uniform Resource Identifier = /rebecca.htm HTTP: Protocol Version = HTTP/1.1 HTTP: Accept = image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */* HTTP: Accept-Language = en-us HTTP: Accept-Encoding = gzip, deflate HTTP: User-Agent = Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.0.3 HTTP: Host = kapoho10 HTTP: Connection = Keep-Alive

This output is a single line in the HTTP message, with spaces separating the individual elements, and terminated with a CRLF sequence. This line identifies several criteria: the general type of request being made (a GET, in this case); the method token, or action to perform on the requested resource (again a GET); the URI of the requested resource(/Rebecca.htm); and the protocol version number (HTTP/1.1). There are extra header fields to inform the server what type of object the client will accept, the client's language, the user agent in use, and whether to keep connections alive between client and server. The client HTTP port number listed in this request, 4283, is unrelated to the serverTCP port. Although the client made its HTTP request over TCP/port 4283, the requestwas issued to TCP port 80. This can be seen in the TCP header of the HTTP GET message, as follows:

TCP: .AP..., len: 274, seq:1182807736-1182808010, ack:3591543841, win:17520, src: 4283 dst: 80 TCP: Source Port = 0x10BB TCP: Destination Port = Hypertext Transfer Protocol TCP: Sequence Number = 1182807736 (0x468036B8) TCP: Acknowledgement Number = 3591543841 (0xD6129C21) TCP: Data Offset = 20 (0x14) TCP: Reserved = 0 (0x0000) + TCP: Flags = 0x18 : .AP... TCP: Window = 17520 (0x4470) TCP: Checksum = 0xDB66 TCP: Urgent Pointer = 0 (0x0) TCP: Data: Number of data bytes remaining = 274 (0x0112)

Request Message Methods

Methods are actions that a request message can ask to be performed at the server or applied to a resource entity. Table 21-1 lists common HTTP/1.1 methods and the actions that they request.

Safe Methods

Methods can be said to be safe, meaning that if they are properly implemented, they should cause no ill effects to the server. For example, the GET and HEAD methods are considered safe methods because they retrieve only data from the server and do not actually manipulate that data while it resides on the server.

Response Messages

After a server receives a client request message, it returns an HTTP response message, similar in construction to the client request, in the following format:

Status line *(( general header | response header | entity header ) CRLF) CRLF [message body]

Each of these components is analyzed in further detail in this section.

Response Message Status Lines

When a server receives a request message from a client, it evaluates the request method and might or might not perform an action on the requested resource as a result. Regardless of whether or not the action is performed, the server must respond to the requesting client. The response is sent in the form of a status line with the following syntax:

Status Line HTTP version Status code Reason phrase

Status codes are organized into classes that identify the generic response type. Table 21-2 lists the classes and meanings of status codes defined in HTTP/1.1.

Each class of status response has codes that the server returns to the client indicating specific information. Table 21-3 summarizes the individual codes that a server can return to a requesting client, as defined in RFC 2616. The messages listed are merely recommendations and can be customized without affecting the protocol.

A typical server response line might appear as follows:

HTTP: Response (to client using port 4283) HTTP: Protocol Version = HTTP/1.1 HTTP: Status Code = OK HTTP: Reason = OK HTTP: Content-Length = 176 HTTP: Content-Type = text/html HTTP: Last-Modified = Tue, 27 Aug 2002 22:05:16 GMT + HTTP: Undocumented Header = Accept-Ranges: bytes + HTTP: Undocumented Header = ETag: "0b23ed3154ec21:68a" HTTP: Server = Microsoft-IIS/6.0 HTTP: Date = Tue, 27 Aug 2002 22:06:49 GMT HTTP: Data: Number of data bytes remaining = 176 (0x00B0)

Note that this data does not appear to list the actual code number returned by the server to the client. However, analysis of the raw data sent shows that the following information was actually sent:

HTTP/1.1 200 OK

Status codes are extensible, meaning that server implementations can issue codes not listed in the preceding tables. Because the client might or might not understand the meaning of a particular extended status code, the code must be issued as an extension of an existing class. If the specific code number is unrecognized, the client responds as if it received a generic code from that class. For example, if the server issues error code 509 and the client does not recognize this code, it treats the error as a 500 error.

Header Fields

A client can generate request headers to query parameters regarding a resource or to negotiate content with the server. A server can issue response headers to provide information that cannot be included in the status line. Both request and response headers can indicate the presence of an entity to be transferred by including entity header fields and possible entity data. Although these field names cannot be universally extended without an accompanying change in the protocol version, extension headers can be used, provided both the client and the server understand them. Any header that is not recognized by the recipient is ignored. Internet Explorer sends nonstandard header fields to IIS 6, which is capable of utilizing them.

Headers can be either end-to-end headers or hop-by-hop headers. End-to-end headers are transmitted to the final message recipient, and must be stored as part of a cached entry. Proxies are forbidden to modify many end-to-end headers, cautioned against modifying others, and might or might not be permitted to add headers to a message. Hop-by-hop headers are useful only to the next recipient on a path, and proxies neither cache nor forward these headers.

Headers defined in RFC 2616 are summarized in Tables 21-4, 21-5, 21-6, and 21-7. Sender and recipient can refer to either the client or the server in these tables, as both can send messages containing header fields during the transaction.

HTTP Codings

HTTP uses content codings to specify data transformation mechanisms, such as compression, that have been applied to an entity. By specifying the coding method in header fields, client and server applications can determine how to decode the entity to make it legible. Content codings are registered with the Internet Assigned Numbers Authority (IANA), and include the following:

• GNU's Not Unix (GNU) Zip (Gzip)A file compression format defined in RFC 1952.
• CompressA UNIX compression-encoding format.
• DeflateA combination of the zlib and deflate encoding and compression mechanisms defined in RFCs 1950 and 1951.
• IdentityThe default encoding mechanism, which indicates that there is no encoding performed on the entity.

Transfer Codings

Transfer codings are used to ensure safe passage of an entity through the request/response path. Transfer coding is not an entity property, as is content coding. Rather, transfer coding is applied to the entire entity message body. Transfer coding values indicate whether encoding has been or can be applied to the message. If transfer coding is applied to a message, the values must indicate whether the data has been chunked, or broken into more manageable pieces. This coding must not be applied more than once to the message body so that a client can accurately determine message transfer length. Any server that receives transfer coding values it does not understand should return a 501 (Not Implemented) response so that the client can request or apply a different encoding mechanism. The encoding formats used for transfer encoding are the same as those listed for content coding, with the addition of the chunked value.

Chunked Transfer Coding

Chunked transfer coding is used to modify the body of a message so that it can be sent as a series of smaller pieces. Each chunk is sent with its own size information and, possibly, entity headers, so that the recipient can accurately determine whether it has received all the chunks that comprise the message. HTTP/1.1-compliant applications are required to be capable of receipt and decoding of chunked transfer coding, and must ignore any extensions to transfer coding values that they do not understand.

HTTP Content Negotiation

Content negotiation is the process by which the HTTP client and server determine the preferred representation for a response. Client software might be able to interpret only specific entity media types, as might also be the case at the server. Additionally, user preferences for parameters, such as language and file format, affect the negotiation process. Content negotiation can be said to be agent-driven, wherein the client chooses the best media representation after receiving a response from a server; server-driven, meaning that the server specifies preferred representations; or transparent, which is a combination of both agent-driven and server-driven negotiation.

Agent-Driven Content Negotiation

In agent-driven negotiation, the client software, called a user agent, issues a request message to a server that indicates its own capabilities by means of header fields, such as Accept, Accept-Charset, and Accept-Language. When the server receives this request, it responds with a message indicating its own capabilities, and the user agent then responds with its own choice of representation from the list provided by the server.

Agent-driven negotiation is advantageous when the user wants to dictate certain content parameters, such as the language used to display a Web page, or when the server cannot ascertain the client's capabilities by analyzing its request message. Additionally, load balancing for heavily trafficked servers can be provided using public caches (additional servers or proxies that maintain cached copies of the information on the origin server), and agent-driven negotiation can be used to determine message parameters with the computers providing these caches. However, agent-driven negotiation has the disadvantage of requiring additional message transfer between client and server, because the client must first request a listing of the server's capabilities before choosing a message format.

Most Web client interactions, as illustrated in Capture 21-01, do not use agent-driven content negotiation.

Server-Driven Content Negotiation

In server-driven content negotiation, the server uses an algorithm to select what it considers to be the best format for messages between itself and the client. The server can base its determination on parameters it receives in the client's request message, as well as on its own capabilities, and even parameters such as the requesting client's network address. Server-driven negotiation allows the server to send the requested entity in its initial response package, as determined by its best guess as to which media type the client prefers (based on header fields such as Accept, Accept-Charset, and Accept-Language), rather than waiting for another request after the client has received a response and chosen from the list of available formats.

Server-driven negotiation reduces the number of messages that need to be transferred to determine acceptable media types, but also has several disadvantages. First, this type of negotiation requires the server to guess what the preferred format on the client's end is, which might or might not match what the user prefers. Second, because this negotiation type requires the user agent to describe its capabilities in each request it makes to the server, it might be inefficient and might also violate the user's privacy. Third, the server must perform additional processing for each response to determine the optimal format for that response. Last, servers that are providing public caches to facilitate load balancing for another server might not be able to service requests for different users from their caches, because each user agent might provide different parameters.

Transparent Content Negotiation

Transparent content negotiation provides a combination of both agent-driven and server-driven negotiation mechanisms. When a cache receives a client request containing parameters that it can fulfill, it can negotiate the content format itself, rather than forwarding the request parameters to the origin server. In this case, the cache is acting as the server would in server-driven negotiation, thus saving work at the server that originally provides a resource. HTTP/1.1 does not provide any guidelines for transparent negotiation, although many implementations provide their own mechanisms as extensions of HTTP/1.1.

HTTP Caching

To make HTTP as efficient as possible, clients, servers, proxies, and gateways can cache content retrieved as part of the request/response process. The server specifies caching in HTTP/1.1. The originating server for any resource decides whether or not othermachines along the path should cache a message. Servers might specify that a message cannot be cached by any computer, must be cached by these computers, or can be cached based on variables, such as the age of the message. Originating servers do not, however, always assign dictates as to whether or not, or for how long a message can be cached, so caches also use a mechanism called heuristic expiration. In heuristic expiration, a cache uses information contained in message headers to estimate the point at which the message can be considered stale, and thus would need to be re-retrieved, should it berequested. Microsoft ISA Server, for example, enables the administrator to set time to live (TTL) values on cache content based on headers or static settings.

Because this is inherently unreliable in providing accurate gauges for message freshness, HTTP/1.1 strongly encourages servers to provide explicit expiration on any responses they send. RFC 2616 outlines implementation details for caching and content expiration.

FTP

FTP, defined in RFC 959, is one of the older application protocols. Although HTTP is perhaps better known, FTP has been a mainstay of file transfer on the Internet for a considerable time. Microsoft IIS 6 implements a full-featured FTP server service and Windows .NET Server 2003 includes a simple command-line FTP client and FTP functionality in Internet Explorer. Although additions to FTP have been proposed in several RFCs and drafts, its core functionality remains largely unchanged. This chapter does not provide instruction in how to use the FTP client, but instead defines the protocol itself.

FTP is used to share and transfer files between computers, as well as use other computers for remote storage purposes. As is the case with HTTP, FTP is an Application Layer protocol that relies on TCP to ensure guaranteed delivery of data. Although the RFC definition of FTP does not provide any true method for recovering a lost connection and picking up file transfer where it left off, IIS 6 implements a process called FTP restart to add this functionality.

FTP is an inherently nonsecure protocol because it transmits user passwords as clear text. To help provide secure mechanisms to be used in FTP transmissions, security extensions to the FTP protocol are defined in RFC 2228, although IIS 6 does not implement these.

As with HTTP, FTP has its own unique terminology, some of which is defined in the following list:

• FTP commandsCommands issued between two computers in an FTP session to control the flow of information from one computer to the other.
• Control connectionA connection established between client and server components for the exchange of FTP commands and replies.
• ReplyAn acknowledgment sent by the server over the control connection in response to client commands.
• Data connectionA full-duplex connection established for the purpose of transferring data between two computers, whether they are made up of a client and a server or two servers. This connection is separate and distinct from the control connection.
• Data Transfer Process (DTP)The entity that establishes data connections, as well as managing connections once they have been opened. A DTP can be active, meaning that it is listening for connections between client and server, or passive, meaning that data is currently being passed and the DTP is idle.
• Protocol Interpreter (PI)On the client side of an FTP session, the user PI initiates a control connection from the client port to the server FTP process, as well as issuing commands from client to server. On the server side of an FTP session, the server PI listens for a user PI connection and commands, and governs the issuing of responses and the server DTP. The server PI and server DTP comprise the server FTP process.

These components are shown in Figure 21-1.

Figure 21-1: FTP components.

FTP Operation

FTP is a client/server protocol that uses two connections between client and server, unlike SMTP, HTTP, and NNTP, which use a single connection. With FTP, a control connection is established to communicate between client and server, and data connections areestablished to transfer files between an FTP client and server.

FTP Connections

FTP connections are established between an FTP client and an FTP server. An FTP session between client and server can be initiated either by a user, through an FTP client interface, or programmatically on Windows Server 2003 using the Win32 application programming interface (API). In any case, the user PI initiates the actual connection. The user PI is responsible for opening a TCP connection to the FTP server and for sending a command to the server PI requesting that an FTP connection be opened between them. The server PI listens for connection requests on TCP port 21 by default, and after receiving a connection request from a user PI, begins the process of establishing a control connection.

Every FTP session actually consists of two separate connections—a control connection and a data connection. The control connection follows Telnet specifications, and it is used to negotiate communication parameters, issue commands and responses, and monitor the status of any data connection that is opened between the two computers. The task of opening and monitoring the data connection is handled by the DTP components on both the FTP client and server. The data connection is the actual mechanism over which data transfer occurs. Whereas a data connection can be dynamically opened and closed in a single session between the two computers, the control connection always remains open for the duration of a single session.

A user can initiate an FTP session between a client and server by using FTP client software. Windows Server 2003 includes a simple command-line FTP client, and Internet Explorer provides GUI-based FTP features. The client FTP program includes both the user PI and user DTP components shown in Figure 21-1. This enables the client to initiate a control connection between the client and the FTP server (comprised of the server PI and server DTP).

In Windows Server 2003, you can use a Web browser, such as Internet Explorer, to make and utilize an FTP connection to an FTP server. You can also use the built-in FTP tool simply by typing FTP <hostname or IP address of an FTP server> from the command prompt.

After the control connection has been established between FTP client and server, the user can issue commands to the server that open a data connection between the two computers. Data is then passed bidirectionally (full-duplexed) over this connection. When the data transfer is complete, the data connection can be closed, although the control connection remains open until the user initiates its disconnection and the server performs the actual process of closing the connection. Figure 21-2 diagrams this process.

Figure 21-2: A client/server FTP session.

The following trace shows a simple FTP session between an FTP client and server, in which the client connects, downloads a single file, and then disconnects:

C:>ftp kapoho10.kapoho.com Connected to kapoho10.kapoho.com. 220-Microsoft FTP Service 220 KAPOHO10 FTP User (kapoho10.kapoho.com:(none)): ftp 331 Anonymous access allowed, send identity (e-mail name) as password. Password: 230-Welcome to FTP service at Kapoho10 230 Anonymous user logged in. ftp> get rebecca.txt 200 PORT command successful. 150 Opening ASCII mode data connection for rebecca.txt(16 bytes). 226 Transfer complete. ftp: 16 bytes received in 0.00Seconds 16000.00Kbytes/sec. ftp> quit 221 C:>

The trace for this short FTP session is in Capture 21-02 on the companion CD-ROM. In theory, an FTP client can request an FTP server to create a data connection to another FTP server. This process is allowed in the FTP RFC, but it is rarely implemented. Most popular FTP clients create data connections only between the FTP client and server systems.

FTP Data

FTP data transfer occurs over the data connection between the FTP server and FTP client. The control connection is reserved for passing and receiving FTP commands that control the session, as well as exchanging data transfer parameters. The sender and receiver in any FTP session negotiate the data transfer format to ensure that the receiving end can correctly reconstruct the data that is sent.

Because each computer can store its data in its own specific logical byte sizes (the number of bits that comprise a data byte on the disk), mechanisms must be in place to ensure that the data sent to the receiving computer is transmitted in an agreed-on format. The FTP specification provides for specific data structures and data type representation, although most FTP servers and clients only use ASCII or binary data transfer.

Data Structures

FTP classifies three different data structures, or characteristics, of a file stored on the computer. Some systems store data as a series of fixed-length records, whereas other computers store data as a series of characters and separators. Because of the disparity in storage formats, transfers of data must be made in a format that the sending and receiving computers can both reconstruct and write appropriately to their own disks. In FTP,these structures are defined as file structure, record structure, and page structure, asdescribed here:

• File structureThis data is stored as a continuous series of bytes that have no internal structure, such as record markers. File structure is the default data structure used in FTP transfers and is assumed by both parties unless a command is issued specifying one of the other structures.
• Record structureThis data is stored on disk as a series of sequential records. All FTP implementations are required to accept record structures for text files, whether the text is in ASCII or Extended Binary Coded Decimal Interchange Code (EBCDIC) format.
• Page structureThese files can be discontinuous data, containing sections of file data interspersed with descriptors of that data. FTP refers to these file sections as pages.

When page structure data is sent between two hosts using FTP, each page must be sent with a page header comprised of 1-byte fields that provide informational parameters. Each header begins with a Header Length field that defines the number of bytes comprising the header. Following this is a page index, or number, that identifies this page's place in the overall file, and a Data Length field that specifies the length of the page data itself. The next field in the header identifies the page type, which can be a normal data page; a descriptor page that defines properties for the file; an access-controlled page, which provides information about access control to the file; or the last page in the file. There might also be optional header fields that define properties, such as access control to the individual page.

Data Types

Within each of the data structures defined in FTP, data can be stored as different data types. These data types define the on-disk byte size of the file data. Some data types provide explicit definition of data byte size, whereas others provide implicit definition by using logical byte size. Acceptable data types in IIS FTP transfers are defined in the following list:

• ASCII typeDefault data type used in FTP transfers, which must be accepted by all compliant implementations of FTP. ASCII is most often used for the transmission of text files. In an ASCII transfer, the sending computer retrieves the requested file from its own disk in whatever format was used to store it. The file is then converted to ASCII, which dictates 8-bit character bytes, and transmitted in 8-bit increments (transfer byte size is always 8 bits, regardless of data type). The receiving computer reassembles the 8-bit ASCII text and stores it to disk in its own native format.
• Image typeSent as a continuous stream of bits that the receiving computer must store as a series of contiguous bits. The recipient might need to pad the end of each file or record (depending on its own internal storage mechanisms) with 0s that are stripped off when the file is retrieved. This data type is most efficient for files that need to be stored in binary format, as opposed to ASCII text (executable files versus text documents, for example).

Connections and Transmission Modes

To transmit data between a client and server using FTP, both a control connection and a data connection must be established. The control connection is used both to setparameters for the data connection and to monitor the passage of data over the data connection. Although the control connection remains open during the entire client/server FTP session, multiple data connections can be dynamically opened and closed.

Data Connection Establishment and Management

FTP connections between client and server are initiated by the user PI. The FTP server listens on TCP port 21 (by default) for connection requests. When the server PI hears a connection request from a user PI, it opens a control connection. The use of nondefault ports for connections can only be initiated by the user PI, and not the server PI. When the client issues a command that initiates data transfer, the server DTP opens a data connection and data transfer begins. Both user DTP and server DTP monitor the connection to determine which computer might be sending or receiving at any given time. The initial connection, logon, and data transfer process, based on the earlier FTP session and shown in Capture 21-02, is as follows:

1 1.791683 FTP Client DNS Server DNS 0xA2A:Std Qry for kapoho10.kapoho.com. of type Ho 10.10.1.68 10.10.1.200 2 1.791683 DNS Server FTP Client DNS 0xA2A:Std Qry Resp. for kapoho10.kapoho.com. of t 10.10.1.200 10.10.1.68 3 1.801697 FTP Client FTP Server TCP ....S., len: 0, seq: 476820006- 476820006, ack: 10.10.1.68 10.10.1.74 4 1.811712 FTP Server FTP Client TCP .A..S., len: 0, seq:2161805281- 2161805281, ack 10.10.1.74 10.10.1.68 5 1.811712 FTP Client FTP Server TCP .A...., len: 0, seq: 476820007- 476820007, ack: 10.10.1.68 10.10.1.74 6 1.811712 FTP Server FTP Client FTP Resp. to Port 2413, '220-Microsoft FTP Service' 10.10.1.74 10.10.1.68 7 2.012000 FTP Client FTP Server TCP .A...., len: 0, seq: 476820007- 476820007, ack: 10.10.1.68 10.10.1.74 8 2.032029 FTP Server FTP Client FTP Resp. to Port 2413, '220 KAPOHO10 FTP' 10.10.1.74 10.10.1.68 9 2.212288 FTP Client FTP Server TCP .A...., len: 0, seq: 476820007- 476820007, ack: 10.10.1.68 10.10.1.74 10 3.053497 FTP Client FTP Server FTP Req. from Port 2413, 'USER ftp' 10.10.1.68 10.10.1.74 11 3.063512 FTP Server FTP Client FTP Resp. to Port 2413, '331 Anonymous access allowed 10.10.1.74 10.10.1.68 12 3.213728 FTP Client FTP Server TCP .A...., len: 0, seq: 476820017- 476820017, ack: 10.10.1.68 10.10.1.74 13 9.953419 FTP Client FTP Server FTP Req. from Port 2413, 'PASS tfl@kapoho.com' 10.10.1.68 10.10.1.74 14 9.963433 FTP Server FTP Client FTP Resp. to Port 2413, '230-Welcome to FTP service a 10.10.1.74 10.10.1.68 15 10.123664 FTP Client FTP Server TCP .A...., len: 0, seq: 476820038- 476820038, ack: 10.10.1.68 10.10.1.74 16 10.123664 FTP Server FTP Client FTP Resp. to Port 2413, '230 Anonymous user logged in 10.10.1.74 10.10.1.68 17 10.323952 FTP Client FTP Server TCP .A...., len: 0, seq: 476820038- 476820038, ack: 10.10.1.68 10.10.1.74 18 15.180936 FTP Client FTP Server FTP Req. from Port 2413, 'PORT 10,10,1,68,9,111' 10.10.1.68 10.10.1.74 19 15.190950 FTP Server FTP Client FTP Resp. to Port 2413, '200 PORT command successful. 10.10.1.74 10.10.1.68 20 15.190950 FTP Client FTP Server FTP Req. from Port 2413, 'RETR rebecca.txt' 10.10.1.68 10.10.1.74 21 15.190950 FTP Server FTP Client FTP Resp. to Port 2413, '150 Opening ASCII mode data 10.10.1.74 10.10.1.68 22 15.200965 FTP Server FTP Client TCP ....S., len: 0, seq:1569488684- 1569488684, ack 10.10.1.74 10.10.1.68 23 15.200965 FTP Client FTP Server TCP .A..S., len: 0, seq:3313623390- 3313623390, ack 10.10.1.68 10.10.1.74 24 15.200965 FTP Server FTP Client TCP .A...., len: 0, seq:1569488685- 1569488685, ack 10.10.1.74 10.10.1.68 25 15.200965 FTP Server FTP Client FTP Data Transfer To Client, Port = 2415, size 16 10.10.1.74 10.10.1.68 26 15.200965 FTP Server FTP Client TCP .A...F, len: 0, seq:1569488701- 1569488701, ack 10.10.1.74 10.10.1.68 27 15.200965 FTP Client FTP Server TCP .A...., len: 0, seq:3313623391- 3313623391, ack 10.10.1.68 10.10.1.74 28 15.220993 FTP Client FTP Server TCP .A...F, len: 0, seq:3313623391- 3313623391, ack 10.10.1.68 10.10.1.74 29 15.231008 FTP Server FTP Client TCP .A...., len: 0, seq:1569488702- 1569488702, ack 10.10.1.74 10.10.1.68 30 15.331152 FTP Client FTP Server TCP .A...., len: 0, seq: 476820079- 476820079, ack: 10.10.1.68 10.10.1.74 31 15.331152 FTP Server FTP Client FTP Resp. to Port 2413, '226 Transfer complete.' 10.10.1.74 10.10.1.68 32 15.531440 FTP Client FTP Server TCP .A...., len: 0, seq: 476820079- 476820079, ack: 10.10.1.68 10.10.1.74 33 17.654493 FTP Client FTP Server FTP Req. from Port 2413, 'QUIT' 10.10.1.68 10.10.1.74 34 17.654493 FTP Server FTP Client FTP Resp. to Port 2413, '221 ' 10.10.1.74 10.10.1.68 35 17.664507 FTP Client FTP Server TCP .A...F, len: 0, seq: 476820085- 476820085, ack: 10.10.1.68 10.10.1.74 36 17.674521 FTP Server FTP Client TCP .A...., len: 0, seq:2161805598- 2161805598, ack 10.10.1.74 10.10.1.68 37 17.674521 FTP Server FTP Client TCP .A...F, len: 0, seq:2161805598- 2161805598, ack 10.10.1.74 10.10.1.68 38 17.674521 FTP Client FTP Server TCP .A...., len: 0, seq: 476820086- 476820086, ack: 10.10.1.68 10.10.1.74

When an active transfer is occurring, the DTP on the receiving computer is passive and the DTP on the sending computer is actively controlling data transfer. Because the data connection is automatically closed after completion of a data transfer, a data connection can be kept open either by negotiating a nondefault port before beginning transfer, or by switching to a different transfer mode for the file(s) in question.

FTP defines the three transfer modes covered in the following list.

• Stream modeIn stream mode data transfer, the data is sent as a series of bytes, with little or no processing performed on the data before it is sent. Data that is sent in stream mode can be of any representation type (file, record, or image). If file structure data is sent, the end of the file is indicated when the sending computer closes the data connection. If the data being sent is record structure, each record is followed by a two-character control code that indicates the end of the record; the end of the file itself is indicated by a similar control code.
• Block modeBlock mode data is sent as a series of data blocks with no filler bits, preceded by header bytes that contain a Count field and possible descriptor code. The Count field indicates each block's length in bytes. The descriptor code indicates whether this is the last block in either the record or the file, whether the data is suspected to contain errors (possibly because of bad media), or to identify a restart marker, which is described later in this section. Any data representation type can be used in block mode, and record structures are permitted.
• Compressed modeCompressed mode data is sent as three types of information: regular data, compressed data, and control information. Regular data is sent as a simple byte string. Compressed data uses an algorithm to compress filler bytes and represent them with a single filler byte, thus decreasing theactual number of bytes that need to be sent. Control information is sent as a2-byte escape sequence and descriptor code, which are the same codes used in block mode data transfer, including a possible restart marker.

FTP Restart

FTP restart provides a mechanism for resuming file transfers that were interruptedbefore completion. FTP restart is a feature of the FTP service in Windows Server 2003. FTP itself provides little in terms of recovery mechanisms, but it does provide for the insertion of restart markers in both block and compressed data modes. When an FTP server implements FTP restart, it periodically sends a restart marker (essentially a place marker) in the data being transferred to the receiving computer.

The receiver collects these restart markers, and if the data connection is lost, it can use them to resume the file transfer where it was interrupted. After resuming the connection between itself and the server, the client first queries the server to determine if the file has changed since the interruption of the transfer. If the file has not changed, the client issues the last restart marker value to the server and requests that the transfer be resumed at that point. If the file has changed since the data transfer interruption, the client requests that the entire file be transmitted from the beginning.

Although the server inserts restart markers only in transmission of block or compressed data, stream data can also be recovered if the connection is lost. Because stream data is sent simply as a series of bytes, the client need only calculate the byte offset of the last data it received and then request the server to resume the file transfer at that offset point.

FTP restart commands are not used only when a connection has been broken. Some FTP clients send the command REST 0 before downloading a new file from a server. This command is used to ensure that the data transfer begins with the first byte of the requested file.

FTP Commands and Responses

FTP communication between an FTP client and server is transmitted as a series of commands that the client needs fulfilled, and responses from the server. This process is similar to the HTTP request/response functionality in HTTP communication.

FTP Commands

The user PI issues commands over the control connection to initiate file transfer. FTP commands can be commands that verify the user's identity with the FTP server, such as USER, ACCT, and PASS; commands that navigate the file system on the remote host, such as CDUP, XCUP, and CWD; session origination and termination commands, such as QUIT, BYE, and REIN; or commands that control the parameters of file transfer as well as the transfer itself, such as PORT, TYPE, MODE, GET, PUT, RETR, and STOR.

FTP Replies

Similar to other Internet type services, FTP servers issue reply codes in response to client commands. These extensible reply codes are sent in the form of a three-digit number, with the value of the first and second digits indicating the type of response.

• The first digit in an FTP response code indicates the general type of response the server wants to pass to the client.
• The second digit is used to provide a more specific indication of the meaning of the response.
• Values for the third digit in a response code are extensible, meaning that they can be customized for specific implementations.

Tables 21-8 and 21-9 describe the representations of the first and second digits in response codes. As an example of response-code meanings, a server issuing code 250 wouldindicate that the requested file system action was successfully completed; the 2 indicates success, and the 5 indicates a file system operation.

NNTP

NNTP provides a user with the ability to submit news articles to a news server. NNTP also enables these articles to be propagated to other news servers, thus providing peer-to-peer discussion groups that are global in nature.

The network news facility was a feature of ARPANET, which preceded today's Internet. Almost all Internet service providers (ISPs) today allow access to some form of news, and there are third-party companies that run news servers for individual subscribers as well as for smaller ISPs that outsource their news provision.

In the early days, "news" included bulletins about the ARPANET, details supportinginformation, and actual data. Today, there are tens of thousands of newsgroups available that cover just about any human activity. Some of the news servers are private andrequire specific permission to access. Some news servers are provided by individual com panies (for example, news.microsoft.com) as part of the support of their products. Most ISPs provide news access to their customers.

The basic operation of NNTP is described in RFC 977. Some common extensions to NNTP are described in RFC 2980. The format of NNTP news articles is described in RFC 1036. Although never issued as a formal RFC, Henry Spencer's "Son of 1036" is consideredbest practice for newsreaders and servers in terms of message formats. You can obtain this document from ftp://ftp.zoo.toronto.edu/pub/news.txt.Z (for the text version) orftp://ftp.zoo.toronto.edu/pub/news.ps.Z (for the PostScript version).

NNTP is a client/server protocol that runs over TCP. It has many similarities to FTP and SMTP. NNTP is used in two different situations. The first enables an NNTP client program to post and retrieve news messages from a news server's news base. The second enables two NNTP servers to transfer articles, thus propagating them across a network of NNTP servers.

As with other Internet protocols, NNTP has its own unique terminology, some of which is defined in the following list:

• News clientA news client is a program run on a client computer that enables a user to read and compose news articles. Some news clients require the user to be actively connected to a news server. Other news clients allow the user to read and compose offline, which is popular for dial-up users because it can reduce their online time. Microsoft provides Outlook Express as a newsreader client in most recent versions of Windows.
• News serverA news server is a server process that manages a set of news articles and enables news clients to read existing articles and compose new ones. IIS 6 includes a basic news server product. Microsoft Exchange 2000 provides a fuller news service.
• News articleA news article is a single item of news not dissimilar to an e-mail message. It is composed on a news client and sent to the news server using NNTP. The news server on which the post was first made can be configured to replicate this message to other news servers that do not already have it. A news article consists of several header lines, followed by a blank line, followed by the body of the news article.
• Article headersEach news article has a series of header lines that consist of a keyword, a colon, a blank, and some additional information. The FROM: header, for example indicates who the article is from. The details of the news message format are contained in RFC 1036 (and the "Son of 1036" document mentioned earlier).
• Article bodyThe article body is the content of the article, the information the sender wishes to propagate to the other users of a particular newsgroup.
• Article numberNNTP servers are expected to include an article number. This is a monotonically increasing number starting with 1. A given server for a given group might have some of the articles that have been posted to the group, the first of which might be greater than 1. It is also possible that there could be gaps.
• Internal current article pointerWhen a news client is accessing a news server and retrieving news articles, the server maintains a current article pointer. Set to the start of a group when the group is first selected, this pointer is automatically advanced as articles are read. This feature is used in online newsreaders to simplify their communication with the server.
• NewsgroupA newsgroup is a collection of articles, generally on a single subject. Newsgroups are stored in a folder on the news server, typically with each article being a separate file. The server can then provide articles in this group to a news client when requested.
• News hierarchyThe news hierarchy is a set of groups with a common root. For example, the hierarchy uk.* consists of a set of groups about or relating to the United Kingdom, and comp.* is a large hierarchy that relates to computing matters. Hierarchies vary in how well groups propagate, with some hierarchies being truly global and others being more regional. Hierarchies can have specific rules and guidelines about how new groups are created. For uk.*, the rules are set out at www.usenet.org.uk. There are hundreds of news hierarchies available. See http://www.magmacom.com/~leisen/mlnh/mlnhtables.html for a reasonably up-to-date list of existing global and regional news hierarchies.
• News baseThe news base is the set of all the articles in all the hierarchies held by a single news server. This is sometimes referred to as the news spool.
• UsenetUsenet is a set of eight news hierarchies. Often referred to as the Big 8, Usenet formally consists of the hierarchies biz.*, comp.*,humanities.*, misc.*, news.*, rec.*, sci.*, and soc.*. Usenet is often thought of as incorporatingthe alt.* hierarchy as well. Most ISPs provide unrestricted access to all ofthe groups in the main Usenet hierarchy as well as some or all of thealt.* hierarchies.
• PropagationFor hierarchies or groups that are held on more than one news server , you can configure your server to propagate the articles to peer servers. Through propagation, articles are transferred to all news servers configured to carry the specific group.
• ExpiryExpiry is the process a news server goes through to purge older articles.
• Peer serverA peer server is a news server that another news server can send articles to or obtain articles from. ISPs offering a news service could have a large number of peers.
• Cross postIn some cases, a poster might want to send an article to two or more newsgroups. To avoid posting the same article multiple times, you can send one copy with multiple newsgroups contained in the newsgroup header. Any news client that reads any of the groups can retrieve the article.
• Flood fillFlood fill refers to the mechanism that NNTP servers use to propagate articles. When an article is posted on a server, that server attempts to offer it to all news peers. These peers in turn offer it to all their peers, and so on. This offers fast propagation but can be wasteful on bandwidth when serversare offering articles to another server that already includes that article. The flood fill algorithm used today in Usenet means that a message can be posted and propagated within minutes.
• Control messagesControl messages are standard news messages with a special header. Control messages are sent to servers to create or delete newsgroups or to request that an article be cancelled (removed from a server's news base).

NNTP Operation

The basic operation of NNTP is similar to SMTP. Connections are made between a news client and server or between two servers. Once this connection is established, the client (or the initiating server) sends commands to the server, which might include data, such as a news message the client is posting to the server. The server attempts to perform the command and then sends back a response, which might include data, such as a news article that is retrieved and sent to the client.

NNTP Connections

As noted earlier, NNTP connections are initiated over TCP either by a news client (to post or retrieve news) or between two servers (to propagate articles and control messages). NNTP clients use an ephemeral TCP port, whereas the NNTP server port is TCP port 119. When a server contacts a peer, it uses a local ephemeral TCP port as well.

In Windows Server 2003, you can use the built-in program Outlook Express as a news client. Outlook Express is installed automatically, but before you can use it as an NNTP client, you need to configure it using the built-in configuration wizard. There are a large range of Windows-based news clients, including both free and commercial products.

After the client has created the TCP connection with the server, it can issue commands to the news server. Once the client has completed sending commands and receivingresponses, it can issue a final command to the server to terminate the news session.

The following trace shows a simple NNTP session between a client and a news server:

Collecting Usenet News from news2.kapoho.net <- 200 NNTP Service 6.0.3621.0 Version: 6.0.3621.0 Posting Allowed -> MODE READER <- 200 NNTP Service 6.0.3621.0 Version: 6.0.3621.0 Posting Allowed -> XHDR Message-ID -> XOVER -> GROUP mct.mentor.announce -> GROUP mct.mentor.discussion -> STAT 80 <- 211 0 5 4 mct.mentor.announce <- 211 38 44 81 mct.mentor.discussion <- 223 80 USTw+JYDu7b9EAXe@mail.psp.co.uks -> GROUP mct.mentor.discussion -> XHDR Message-ID 81 -> NEWGROUPS 020911 230405 GMT <- 211 38 44 81 mct.mentor.discussion <- 221 Xhdr information follows <- 81 <- . -> ARTICLE <- 231 New newsgroups follow. <- . -> QUIT <- 220 0 article <- 205 closing connection - goodbye! News connection to news2.kapoho.net closed - filing continues News from news2.kapoho.net completed, 1 articles fetched, 0 posted

This session is from the log of a news client. The corresponding NNTP traffic can be seen in Capture 21-03 on the companion CD-ROM.

NNTP Commands and Responses

NNTP communication occurs between a news client or news server and another news server. The communication comprises one or more commands sent from the initiating system to the news server plus responses received back from the server. The communication can also include news articles being sent from client to server, from server to client, and from server to server.

NNTP Commands

The NNTP client issues commands to the server across the active connection. NNTP commands are a single word followed by, in some cases, a parameter. In command lines that have parameters, the command is separated from the parameter by one or more space or tab characters. The command line must be complete with all parameters and must not contain more than one command. Each command line is terminated by a CRLF pair. A command line can be, at most, 512 characters long (510 characters plus the CRLF pair), and there is no provision for the continuation of command lines.

Table 21-10 sets out the NNTP commands defined in RFC 850, RFC 977, and RFC 2980 and describes their functions.

NNTP Responses

A news server issues two types of responses to client commands: textual messages and numeric status codes. The intention of command/status responses was that the news client program would interpret them before any possible display is done. The text messages, on the other hand, are meant to be displayed by the news client at the user's computer.

The textual responses are sent only after a numeric status response line has been sent to the client to indicate that text will follow. Text is sent as a series of successive lines of textual matter, each terminated with a CRLF pair. Usually just one line is sent, but RFC 977 allows for longer textual responses. A single line containing only a period (.) is sent to indicate the end of the text (that is, the server sends a CRLF pair at the end of the last line of text, a period, and another CRLF pair). You can see the text messages in the command examples previously shown in Table 21-10.

Like other Internet-type services, NNTP servers issue numeric codes in response to client commands. These extensible reply codes are sent in the form of a three-digit number, with the value of the first and second digits indicating the type of response.

• The first digit in an NNTP response code indicates the general type of response the server wants to pass to the client. See Table 21-11 for more details on the meaning of the first digit.
• The second digit is used to provide a more specific indication of the meaning of the response. See Table 21-12 for more details on the meaning of the second digit.
• Values for the third digit in a response code are extensible and can be customized for specific implementations.

Tables 21-11 and 21-12 list and describe the first- and second-digit values.

SMTP

For many users, e-mail has replaced traditional mail as a form of communication. The contents of this book were sent among the authors, editors, and publisher using e-mail. E-mail is seen by some to be the killer application for the Internet. SMTP is used to transmit mail between mail servers and from mail clients to mail servers. Most mail clients use a different protocol, either Post Office Protocol 3 (POP3) or Internet Message Access Protocol (IMAP), to retrieve mail.

SMTP is designed to do exactly what its name implies—provide reliable, efficient mechanisms for the transfer of electronic mail. SMTP transfers messages from a mail client to a mail server and between mail servers. SMTP is not, however, responsible for managing mailboxes or for allowing a client to download incoming mail. RFC 821 defines SMTP, although features and refinements have been added in numerous subsequent RFCs. Although SMTP uses familiar terminology, a few terms might be unknown and are defined here:

• SMTP clientThe program that initiates an SMTP connection to a receiver SMTP process to send mail. The SMTP client process controls the transfer of mail between itself and the SMTP server, issues commands, and receives replies from the SMTP server.
• SMTP server processThe process that waits for an SMTP client process to establish a connection, then receives commands from the client and carries out the operations specified in those commands.
• Transmission channelThe full-duplex channel opened between SMTP client and server processes for the purposes of command/reply sequences and mail transfer.
• Reverse-pathSpecifies the mail's sender. The reverse-path can be simply the name of the user who is sending the mail, or it can include a list of hosts that relayed this mail from its original sender. The first host listed is the most recent relay, and the last host listed is the first relay.
• Forward-pathSpecifies the mail's recipients. Optionally, the forward-path can specify a list of relays to be used to route the mail to its intended recipients, with the current SMTP server being the first relay in the list and the final destination being the last.
• Reverse-path bufferUsed to hold the list of reverse-path parameters for a transaction, possibly including a list of hosts who relayed the mail. This can be cleared as a result of various commands being issued.
• Forward-path bufferUsed to hold the list of forward-path parameters for a transaction, possibly including a list of relay hosts to whom the mail is to be routed. This can also be cleared as a result of various commands.

  More Info

  SMTP is defined in RFC 821, which can be found in the Rfc folder on the companion CD-ROM.

SMTP Operation

Although numerous enhancements have been added to SMTP since RFC 821, it remains a fairly simple client/server protocol. SMTP, like HTTP and FTP, is an Application Layer protocol that relies on underlying protocols to ensure data delivery. Although SMTP can in theory utilize other protocols, TCP is the most common and can be assumed to be the underlying protocol throughout this section.

SMTP communication is initiated by a user's mail system, the SMTP client. The SMTP client establishes a mail session with an SMTP server by opening a TCP connection to the server, then issuing either a HELO or EHLO SMTP command to begin a session. Extended implementations of SMTP, such as that included with IIS 6, can be configured to require the client to provide authentication credentials that verify the client is permitted to use the SMTP server. Most often, these are simply a user name and password that are recognized by the receiving system.

After the transmission channel has been established, the SMTP client issues a MAIL command that informs the SMTP server that it wants to send mail. If the server is capable of receiving mail at that time, it responds with an OK reply. The SMTP client then issues one or more RCPT commands that identify the recipients of the messages it wants to send; each RCPT command represents a single mail recipient. The recipients can be other users in the same mail system or users in external domains.

If the SMTP server is capable of receiving mail addressed to the recipient named in the RCPT command, it issues an OK reply to the client and the client is free to issue another RCPT command. If the SMTP server is not capable of delivering mail to the designated recipient, it returns an error reply to the client and the client can then move on to the next command. The command/reply sequence is strictly ordered; the client must receive a single reply before the server can issue another command, and a server is not permitted to issue more than one reply to any command.

Because not all recipients can be using the same SMTP system, the client must provide the name of the ultimate destination host as well as the mailbox name in that mail system. The syntax of SMTP mail addresses is the familiar username@domain format, where information to the right of the "@" symbol identifies the destination host, and the user name identifies the name of the mailbox to which the mail should be delivered. SMTP differentiates between sending and mailing: if mail is sent, the client is designating that the mail should be delivered immediately to the recipient's mail interface, provided the recipient is online and using a mail system that uses this functionality. More often, however, mail is mailed, which designates that it should be delivered to the recipient's mailbox on a receiving server. In addition, the send functionality is not a required SMTP implementation, and it can be assumed that this chapter refers to the mail functionality unless specified otherwise.

SMTP mail has both a forward-path and a reverse-path. The forward-path is the path that the mail must take to reach its final destination, whether it uses a direct path or a series of relays. It is important not to confuse SMTP relays with routers; SMTP relays are SMTP servers that can receive mail from one SMTP host and forward that mail to another SMTP host, independent of underlying routing mechanisms. The reverse-path in an SMTP mail message is the name of the sender, which can be as simple as username@domain.

It could otherwise consist of a list of relay hosts between the original sender and the current receiver SMTP. The MAIL command uses the reverse-path as its argument, and the RCPT command uses the forward-path. If multiple recipients' mailboxes reside on the same SMTP host, SMTP encourages the sending of a single copy of the mail to the destination SMTP host.

Once the receiver SMTP has accepted the recipient addresses and provided the appropriate reply, the client is free to begin issuing the DATA command, which informs the server of its intent to begin transferring the mail message. The server replies with a code accepting the sender's intent, and the client then issues the data. The mail data includes not only the body of the mail, but also the memo header information, such as the To:, cc:, bcc:, and Subject lines. If the transfer of the mail data is successful, the server replies with a message indicating receipt and processing, and the client can then issue commands to terminate the transmission connection.

If the sender specifies invalid destination information in the forward-path of the mail, but the server knows the correct destination, the server can reply to the sender with a message allowing the client to correct the error. When the client wishes to terminate the SMTP session, it indicates this by issuing the QUIT command, and the server then closes the transmission connection.

A typical SMTP session might look similar to this:

220-mail.kapoho.com; Sun, 2115 Sep 2001 15:36:29 -0100 (BST) HELO kapoho10.kapoho.com 250 Pleased to meet you MAIL FROM: 250 ... Sender ok RCPT TO: 250 ... Recipient ok RCPT TO: 250 ... Recipient ok DATA 354 Enter mail, end with "." on a line by itself This is a test post. . Message accepted for delivery QUIT 221 mail.kapoho.com closing connection

SMTP Commands

The SMTP client issues the SMTP commands to the SMTP server. The SMTP command follows a straightforward syntax, as shown here (brackets indicate optional command parameters):

[ ]

The SMTP client process issues commands to perform functions, such as opening a transmission channel or initiating a mail transfer. In turn, the SMTP server process attempts to execute the command and then returns the responses. Commands can be issuedindividually or as part of a series of commands, but each command must be followed by a reply from the SMTP server. Table 21-13 lists common SMTP commands, their descriptions, and their syntax.

SMTP Replies

SMTP replies are issued by the SMTP server in response to commands sent by the SMTP client. Every command must generate one and only one reply. Similar to the response codes issued by FTP servers, SMTP receivers issue a three-digit code number followed by descriptive text. As in FTP, the first digit of the response code indicates the general type of response, and the second digit provides additional information within thatresponse category. Tables 21-14 and 21-15 list the values and meanings of both first- and second-digit values.

Since RFC 821 was published, extensions to the protocol have been introduced, including the following:

• Authentication
• Message size declaration, which allows a server to limit the size of mail that it will accept
• Turning of the transmission connection, so that sender and receiver switch roles
• Delivery status notifications

  More Info

  The RFCs that describe these features have been included in the Rfc folder on the companion CD-ROM.

Summary

IIS 6 provides enhanced support for Web-based services through its implementations of the HTTP/1.1, NNTP, SMTP, and FTP protocols. HTTP/1.1 improves on earlier implementations of HTTP in its support for multiple requests over a single connection, header compression, authentication mechanisms, and enhanced caching and proxy definitions. NNTP allows the transmission, storage, and replication of news messages. SMTP allows an IIS server to send and receive electronic messages on behalf of the clients it serves, facilitating e-mail communication for companies that might not require a full-fledged messaging system.

In IIS 6, SMTP is implemented as a secure protocol, allowing for authentication and verification mechanisms. The implementation of FTP in IIS 6 has also been improved, providing FTP restart, which allows a lost download connection to be resumed at the point at which it left off. By complying with the most recent standards for each of these protocols, IIS 6 ensures that Web services can be provided in the most quick and efficient manner possible.