Lesson 10

In this module you learned commands commonly used in shell scripts

1. to find a file,
2. determine file size, sort, and
3. search file contents.

You also learned how to pause your script to display a message for several seconds, tee the output of a command to the screen and a file, and how to clear the screen. You will be using many of these commands to complete the course project.

Unix shell scripts, typically written for shells like Bash, use a variety of commands to perform tasks such as file manipulation, text processing, and system control. Below is a concise description of the most commonly used commands in Unix shell scripts, focusing on their typical use cases:

• echo
  • Purpose: Outputs text or variable values to the terminal or a file.
  • Example: echo "Hello, $USER" displays a greeting with the current user's name.
  • Common Use: Displaying messages, debugging, or writing to files (echo "data" > file.txt).
• ls
  • Purpose: Lists directory contents.
  • Example: ls -l shows files in long format with details like permissions and sizes.
  • Common Use: Checking available files or directories in scripts before processing.
• cd
  • Purpose: Changes the current working directory.
  • Example: cd /path/to/dir navigates to the specified directory.
  • Common Use: Navigating the filesystem within a script to access files or directories.
• pwd
  • Purpose: Prints the current working directory.
  • Example: pwd outputs the full path of the current directory.
  • Common Use: Confirming the script’s working directory for debugging or logging.
• cp
  • Purpose: Copies files or directories.
  • Example: cp file1.txt file2.txt creates a copy of file1.txt.
  • Common Use: Backing up files or duplicating data in scripts.
• mv
  • Purpose: Moves or renames files or directories.
  • Example: mv oldname.txt newname.txt renames a file.
  • Common Use: Organizing files or relocating them during script execution.
• rm
  • Purpose: Removes files or directories.
  • Example: rm -r dir/ deletes a directory and its contents recursively.
  • Common Use: Cleaning up temporary files or old data.
• mkdir
  • Purpose: Creates new directories.
  • Example: mkdir new_folder creates a directory named new_folder.
  • Common Use: Setting up directories for storing output or temporary files.
• cat
  • Purpose: Concatenates and displays file contents.
  • Example: cat file.txt prints the contents of file.txt.
  • Common Use: Reading file contents or combining multiple files (cat file1.txt file2.txt > combined.txt).
• grep
  • Purpose: Searches text for patterns using regular expressions.
  • Example: grep "error" log.txt finds lines containing “error” in log.txt.
  • Common Use: Filtering output or searching logs for specific information.
• find
  • Purpose: Searches for files or directories matching criteria.
  • Example: find / -name "*.txt" locates all .txt files in the filesystem.
  • Common Use: Locating files for processing or cleanup.
• sed
  • Purpose: Stream editor for text manipulation (search, replace, delete).
  • Example: sed 's/old/new/g' file.txt replaces all instances of “old” with “new”.
  • Common Use: Modifying file contents or transforming text in pipelines.
• awk
  • Purpose: Processes and analyzes text, especially structured data.
  • Example: awk '{print $1}' file.txt prints the first column of a file.
  • Common Use: Extracting fields from logs or CSVs in scripts.
• cut
  • Purpose: Extracts sections from lines of text.
  • Example: cut -d',' -f1 data.csv extracts the first column from a comma-separated file.
  • Common Use: Parsing delimited files like CSVs.
• sort
  • Purpose: Sorts lines of text.
  • Example: sort file.txt sorts the lines in file.txt alphabetically.
  • Common Use: Organizing output or preparing data for further processing.
• uniq
  • Purpose: Removes or reports duplicate lines from sorted input.
  • Example: sort file.txt | uniq removes duplicate lines.
  • Common Use: Cleaning up lists or logs.
• wc
  • Purpose: Counts lines, words, or characters in files or input.
  • Example: wc -l file.txt counts the number of lines.
  • Common Use: Checking file sizes or validating data.
• chmod
  • Purpose: Changes file permissions.
  • Example: chmod +x script.sh makes a script executable.
  • Common Use: Setting up scripts or securing files.
• chown
  • Purpose: Changes file ownership.
  • Example: chown user:group file.txt assigns ownership to a user and group.
  • Common Use: Managing access in multi-user environments.
• ps
  • Purpose: Displays information about running processes.
  • Example: ps aux lists all running processes with details.
  • Common Use: Monitoring or managing processes in scripts.
• kill
  • Purpose: Terminates processes by ID.
  • Example: kill 1234 stops the process with PID 1234.
  • Common Use: Stopping runaway or specific processes.
• ifconfig or ip
  • Purpose: Configures or displays network interfaces.
  • Example: ip addr shows network interface details.
  • Common Use: Checking network status or configuring interfaces.
• ping
  • Purpose: Tests network connectivity to a host.
  • Example: ping google.com checks connectivity to Google.
  • Common Use: Verifying network availability in scripts.
• curl
  • Purpose: Transfers data from or to a server (e.g., HTTP requests).
  • Example: curl -O http://example.com/file.txt downloads a file.
  • Common Use: Fetching data from APIs or downloading files.
• tar
  • Purpose: Archives or extracts files.
  • Example: tar -czf archive.tar.gz dir/ creates a compressed archive.
  • Common Use: Backing up or distributing files.

These commands are frequently combined in scripts using pipes (|), redirection (>, >>, <), and control structures like if, for, or while to automate tasks. For example, a script might use grep and awk to parse logs, sort and uniq to summarize data, and echo to log results. Mastery of these commands enables efficient automation and system management in Unix environments.

This module covered the following commands:

Command	Purpose	Description
find	File Search	Searches for files and directories based on criteria such as name, type, size, modification time, and permissions. Can execute actions on matching files.
grep	Search File Contents	Searches for patterns (regular expressions) inside files. Often used to find lines matching a string or pattern. Can be case-sensitive or case-insensitive.
sort	Sort Data	Reads input (lines from a file or standard input) and outputs them in sorted order. Can sort alphabetically, numerically, or based on custom fields.
tee	Split Output	Reads from standard input and writes to both standard output and one or more files simultaneously. Useful for logging output while still displaying it.
wc	Count Lines, Words, Bytes	Stands for "word count"; used to count the number of lines (-l), words (-w), characters (-m), or bytes (-c) in files or input streams.
clear	Clear Terminal Screen	Clears the visible terminal screen, making it easier to display fresh output without clutter from previous commands.
sleep	Pause Execution	Pauses script execution for a specified number of seconds (or fractions of a second with sleep 0.5). Useful for delays, rate-limiting, or timing tasks.

🛠️ Quick Example of How They Might Work Together in a Script

#!/bin/bash

# Find all .log files
find /var/log -name "*.log" > logfiles.txt

# Search for 'error' entries
grep "error" $(cat logfiles.txt) > errors_found.txt

# Sort the error entries
sort errors_found.txt > errors_sorted.txt

# Display sorted errors while saving them
cat errors_sorted.txt | tee error_report.txt

# Show line, word, and byte counts
wc error_report.txt

# Clear the screen
clear

# Sleep for 5 seconds before script exit
sleep 5

📌 Summary

• find ➔ Locate files
• grep ➔ Search inside files
• sort ➔ Order results
• tee ➔ Save and display output
• wc ➔ Summarize file size/length
• clear ➔ Clean terminal view
• sleep ➔ Introduce controlled pauses

This module covered the following key terms and concepts:

1. Options
2. Regular expressions
3. redirection

The next module shows how to spot some frequently found shell script errors and demonstrates techniques for ging your shell scripts.

Click the Exercise link below to begin working on the course project.
Search Player Script - Exercise