Regular Files versus Directories in Unix

In Unix-based systems, including Linux and MacOS, all data is treated as files, following the "everything is a file" philosophy. This design principle is one of the key elements distinguishing Unix from other operating systems. Here, devices, directories, and even processes are represented as files.
Regular Files: Regular files in Unix can contain text, data, or program instructions. They are essentially containers for information, accessed via their file path. Regular files can be binary or text files. A text file is human-readable because it contains printable characters, while a binary file contains data that's not typically human-readable as it may be encoded for machine reading.
Directories: In Unix systems, directories are special types of files that serve as a catalog or a container that can hold other files and directories. Essentially, a directory file provides mappings from human-readable names to inode numbers (the low-level data structure that Unix uses to store files). Unix-based operating systems distinguish directories from regular files in their handling and functionalities:

1. File System Hierarchy: Unix employs a hierarchical file system structure. Every file and directory starts from the root directory, represented as a forward slash (/). This differs from some other systems such as Windows, which segregates file systems by drive letter (e.g., C:, D:).
2. Permissions: Unix handles permissions differently for files and directories. For files, read (r) allows a user to view the file, write (w) lets a user modify the file, and execute (x) allows a user to run the file as a program. For directories, read (r) lets a user view the names of files in the directory, write (w) allows a user to create or delete files in the directory, and execute (x) allows a user to access files within the directory and use the cd command to change to the directory.
3. Hard Links: In Unix, you can create hard links to files, essentially different pointers to the same physical file on disk. However, Unix does not permit the creation of hard links to directories to prevent potential issues such as infinite loops in the directory structure. The only exception to this rule is the "." and ".." directories in each directory, which link to the directory itself and its parent directory, respectively.
4. Soft or Symbolic Links: While hard links aren't allowed for directories, symbolic (or soft) links are. A symbolic link is a special file that serves as a reference to another file or directory.
5. Special Files: Unix also includes special files not typically seen in other operating systems. This includes character and block device files, sockets, and named pipes (FIFOs). These special files reside in the /dev directory and represent various hardware and software devices.

The manner in which Unix treats regular files and directories differently from other operating systems lies in its unique design philosophy that 'everything is a file,' its distinct permissions scheme, the structure of its file system, and its handling of links.

A regular file is anything that is not one of the special file types we will look at below. In UNIX, all regular files are stored as sequences of bytes. Unlike some operating systems, UNIX makes no distinctions between binary files and text files. In addition, file "extensions" have no significance in UNIX. All filename extensions are managed by convention. For example, most C compilers expect C source code programs to be named file.c, but the operating system does not treat .c files in any special way. Files may have multiple extensions, such as file.tar.gz (a commonly used naming convention for archive files created with the tar program and compressed with the gzip program).

A directory file contains a list of the files in the directory. As we have seen, directory files are marked in ls -l listings by a d in the initial position of the permission string (for example, drwxr-xr-x).
So far in this course, we have concentrated on regular files and directories, which are the two files you will deal with most often. However, thanks to the UNIX "everything is a file" philosophy, you will find a number of other things that look like files, but have some special properties.

When a user logs in, the UNIX system puts the user in the home directory. The directory in which the user is working is called the working directory (also known as the current directory). For the C and Korn shells, the tilde (~) character can be used as the shell metacharacter for the home directory. The working directory can be substituted with . (dot). The parent of the working directory is substituted with .. (double dots).

To gain a full picture of the internal operation of filesystems, it is necessary to understand what the user sees, why things are presented they way they are, and what the main concepts are. Users experienced in UNIX may wish to skip this module. On the other hand, users new to UNIX and those starting to program in the UNIX environment will find these concepts useful. A basic implementation of the ls program helps to reinforce the material presented and provides an introduction to file-related libraries and system calls, a topic that will be expanded upon in the next module. One peculiarity that UNIX introduced was the notion that everything in the UNIX namespace (file tree) is visible as a file and that the same operations can be applied to all file types. Thus one can open and read a directory in the same way in which a file can be opened and read. Of course, this does not always have the desired effect. For example, running the UNIX command cat on a directory will likely produce a screen full of unreadable characters. However, these and other simple concepts are one of the great strengths of UNIX. The following sections provide introductory material which describe file-based concepts and start to paint a picture of how these components fit together.