MFT Concepts

The Master File Table (MFT) is the heart of NTFS because it contains the information about all files and directories. Every file and directory has at least one entry in the table, and the entries by themselves are very simple. They are 1 KB in size, but only the first 42 bytes have a defined purpose. The remaining bytes store attributes, which are small data structures that have a very specific purpose. For example, one attribute is used to store the file's name, and another is used to store the file's content. Figure 11.1 shows the basic layout of an MFT entry where there is some header information and three attributes.

Figure 11.1. An MFT entry has a small header, and the rest of it is used to store different types of attributes. This entry has three attributes.

Microsoft calls each entry in the table a file record, but I think calling each entry an MFT entry is simpler and results in fewer terms to remember. Each entry is given an address based on its location in the table, starting with 0. To date, all entries have been 1,024 bytes in size, but the exact size is defined in the boot sector.

Like everything in NTFS, the MFT is a file. What makes this confusing is that the MFT has an entry for itself. The first entry in the table is named $MFT, and it describes the on-disk location of the MFT. In fact, it is the only place where the location of the MFT is described; therefore, you need to process it to determine the layout and size of the MFT. The starting location of the MFT is given in the boot sector, which is always located in the first sector of the file system. We can see this in Figure 11.2 where the boot sector is used to find the first MFT entry, which shows that the MFT is fragmented and goes from clusters 32 to 34 and 56 to 58. Like FAT, NTFS uses clusters, which are groups of consecutive sectors.

Figure 11.2. The relationship between the boot sector and $MFT with respect to determining the layout of the MFT.

In Microsoft's implementation of NTFS, the MFT starts as small as possible and expands when more entries are needed. In theory, an OS could create a fixed number of entries when the file system is created, but the dynamic nature of Microsoft's implementation allows it to easily make the file system larger when more space is added from volume spanning. Microsoft does not delete MFT entries after they have been created.

MFT Entry Contents

The size of each MFT entry is defined in the boot sector, but all versions from Microsoft have used a size of 1,024 bytes. The first 42 bytes of the data structure contain 12 fields, and the remaining 982 bytes are unstructured and can be filled with attributes. You can think of an MFT entry as a large box that is used to store your possessions. On the outside of the box is basic information, such as your name and address. The basic information is equivalent to an MFT entry's fixed fields. The inside of the box is initially empty, but it can be used to store anything as long as it is in a container that is smaller than the box. This is similar to how an MFT entry has no internal structure and it contains several attributes that contain specific information.

The first field in each MFT entry is the signature, and a standard entry will have the ASCII string "FILE." If an error is found in the entry, it may have the string "BAAD." There is also a flag field that identifies if the entry is being used and if the entry is for a directory. The allocation status of an MFT entry also can be determined from the $BITMAP attribute in the $MFT file, which is shown in Chapter 13.

If a file cannot fit its attributes into one entry, it can use multiple entries. When this occurs, the first entry is called the base file record, or base MFT entry, and each of the subsequent entries contains the address of the base entry in one of its fixed fields.

Chapter 13 shows the data structure for an MFT entry and dissects our example file system image.

MFT Entry Addresses

Each MFT entry is sequentially addressed using a 48-bit value, and the first entry has an address of 0. The maximum MFT address changes as the MFT grows and is determined by dividing the size of $MFT by the size of each entry. Microsoft calls this sequential address the file number.

Every MFT entry also has a 16-bit sequence number that is incremented when the entry is allocated. For example, consider MFT entry 313 with a sequence number of 1. The file that allocated entry 313 is deleted, and the entry is reallocated to a new file. When the entry is reallocated, it has a new sequence number of 2. The MFT entry and sequence number are combined, with the sequence number in the upper 16-bits, to form a 64-bit file reference address, as is shown in Figure 11.3.

Figure 11.3. Example of the MFT entry address and sequence number combining to form a file reference address.

NTFS uses the file reference address to refer to MFT entries because the sequence number makes it easier to determine when the file system is in a corrupt state. For example, if the system crashes at some point while the various data structures for a file are being allocated, the sequence number can determine whether a data structure contains an MFT entry address because the previous file used it or because it is part of the new file. We also can use it when recovering deleted content. For example, if we have an unallocated data structure with a file reference number in it, we can determine if the MFT entry has been reallocated since this data structure used it. The sequence number can be useful during an investigation, but in this chapter I will primarily refer to the file number, or MFT entry address, for simplicity.

File System Metadata Files

Because every byte in the volume is allocated to a file, there must exist files that store the file system's administrative data. Microsoft calls these files metadata files, but this may cause confusion because we also refer to file metadata. I will refer to these special files as file system metadata files.

Microsoft reserves the first 16 MFT entries for file system metadata files.[2] The reserved entries that are not used are in an allocated state and have only basic and generic information. Every file system metadata file is listed in the root directory, although they are typically hidden from most users. The name of each file system metadata file begins with a "$," and the first letter is capitalized. We will cover each of the file system metadata files in Chapter 12, but they are listed in Table 11.1 as an easy reference.

[2] Microsoft documentation says it reserves only the first 16 entries, but in practice the first entry that is allocated to a user file or directory is entry 24. Entries 17 to 23 are sometimes used as overflow when the reserved entries are not enough.