Unix / Linux - File Management

Unix / Linux - File Management

Effective file management is crucial for working efficiently in a Unix-like environment. This chapter covers the essential commands for creating, viewing, modifying, organizing, and deleting files.

In Unix, almost everything is treated as a file. The main types you’ll encounter are:

• Ordinary Files: Contain data such as text, images, or program code/executables.
• Directories: Special files that hold other files and directories.
• Special Files: Represent devices (like hard drives /dev/sda, terminals /dev/tty), or communication channels (sockets, pipes).
• Symbolic Links: Pointers to other files or directories.

Listing Files: ls

The ls command lists directory contents. See the ‘Directory Management’ chapter for more details on its options (-l, -a, -h, etc.).

$ ls
file1.txt  notes.md  project_dir

$ ls -lh project_dir
total 4.0K
-rw-r--r-- 1 user group 1.2K Jul 25 15:00 main.c

Determining File Type: file

The file command examines a file’s content and attempts to determine its type.

$ file file1.txt
file1.txt: ASCII text

$ file /bin/bash
/bin/bash: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=..., stripped

$ file project_dir
project_dir: directory

$ file image.jpg
image.jpg: JPEG image data, JFIF standard 1.01, resolution (DPI), density 72x72, segment length 16, baseline, precision 8, 1920x1080, frames 3

Using Wildcards (Globbing)

Shells interpret special characters (metacharacters or wildcards) to match multiple filenames. This is called globbing.

• *: Matches zero or more characters. *.txt matches file1.txt, report.txt, .txt.
• ?: Matches exactly one character. file?.txt matches file1.txt, fileA.txt, but not file10.txt.
• [...]: Matches any one character within the brackets. [abc]*.log matches files starting with a, b, or c and ending in .log. Ranges can be used: [a-z]*.dat matches files starting with any lowercase letter.

Examples with ls:

# List all files ending in .log
$ ls *.log
app.log  kernel.log  sys.log

# List files starting with 'data' followed by a single digit
$ ls data?.csv
data1.csv  data3.csv

# List files starting with 'img' followed by '[abc]'
$ ls img[abc].jpg
imga.jpg imgc.jpg

Hidden Files

Files whose names begin with a dot (.) are considered hidden. ls does not show them by default; use ls -a or ls -A (-A omits . and ..).

Hidden files are commonly used for configuration files (e.g., ~/.bashrc, ~/.vimrc, ~/.ssh/config) or directories (~/.config, ~/.local).

$ ls -a ~
. .. .bash_history .bashrc .profile .ssh .vimrc Documents Downloads Music

Creating Files: touch, Editors

The touch command is the standard way to create empty files or update the modification timestamp of existing files.

# Create an empty file named new_file.txt
$ touch new_file.txt

# Create multiple empty files
$ touch data.log config.yml

# Update timestamp of existing_file.md
$ touch existing_file.md

To create files with content, you use a text editor. Common command-line editors include:

• nano: A simple, beginner-friendly editor. Often recommended for new users.
• vim (or vi): A powerful, ubiquitous, but modal editor with a steeper learning curve. Covered in detail in its own chapter.
• emacs: Another powerful, extensible editor.

# Create or edit a file using nano
$ nano my_notes.txt

# Create or edit a file using vim
$ vim my_script.py

Viewing File Content

Several commands display file content:

• cat (concatenate): Prints the entire file content to standard output. Best for short files.
• less: A pager that displays content screen by screen. Allows scrolling forward/backward (using arrow keys, PageUp/Down) and searching (/). Press q to quit. This is the recommended tool for viewing larger files.
• more: An older pager, similar to less but with fewer features (e.g., limited backward scrolling). less is generally preferred.
• head: Displays the first few lines of a file (10 by default). Use -n count to specify the number of lines.
• tail: Displays the last few lines of a file (10 by default). Use -n count. The -f option (‘follow’) is very useful for monitoring log files as they grow.

# Display entire short file
$ cat config.yml

# View long file interactively
$ less /var/log/syslog

# Show first 5 lines
$ head -n 5 data.csv

# Show last 20 lines
$ tail -n 20 results.log

# Monitor additions to a log file in real-time
$ tail -f application.log

Counting Words, Lines, Characters: wc

The wc (word count) command counts lines, words, and bytes/characters in files.

$ wc report.txt
  42   210  1536 report.txt

The output shows: lines (42), words (210), bytes (1536), filename.

Common options:

• -l: Count lines only.
• -w: Count words only.
• -c: Count bytes only.
• -m: Count characters (can differ from bytes for multi-byte encodings).

$ wc -l *.py
  150 script1.py
   75 script2.py
  225 total

Copying Files: cp

The cp (copy) command copies files or directories.

Syntax: cp [options] source destination

# Copy file.txt to file.bak in the same directory
$ cp file.txt file.bak

# Copy file.txt into the backup/ directory (keeps original name)
$ cp file.txt backup/

# Copy file.txt to backup/ directory and rename it report_final.txt
$ cp file.txt backup/report_final.txt

# Copy multiple files into the documents/ directory
$ cp notes.md report.pdf documents/

# Copy a directory recursively (MUST use -r or -R)
$ cp -r project/ project_backup/

Useful options:

• -r or -R: Recursive copy (for directories).
• -i: Interactive (prompt before overwriting).
• -v: Verbose (show files being copied).
• -p: Preserve attributes like modification time and permissions.

Renaming or Moving Files: mv

The mv (move) command renames files or moves them to different directories.

Syntax: mv [options] source destination

# Rename oldname.txt to newname.txt
$ mv oldname.txt newname.txt

# Move file.log into the logs/ directory
$ mv file.log logs/

# Move multiple files into the archive/ directory
$ mv data*.csv results.dat archive/

# Move file.tmp to processed/ and rename it final.dat
$ mv file.tmp processed/final.dat

Common options:

• -i: Interactive (prompt before overwriting).
• -v: Verbose (show actions).

Deleting Files: rm

The rm (remove) command deletes files. Use with extreme caution, as deleted files are generally not recoverable from the command line!

Syntax: rm [options] filename(s)

# Delete temp_file.txt
$ rm temp_file.txt

# Delete multiple files
$ rm *.tmp data.old

# Delete recursively (directories and contents) - VERY DANGEROUS!
$ rm -r old_project/

# Interactive delete (prompts for each file)
$ rm -i *.log

Key options:

• -r or -R: Recursive delete (for directories). Required to delete non-empty directories.
• -i: Interactive (prompt before deleting). Recommended for safety.
• -f: Force delete (suppress prompts, override permissions if possible). Use with extreme care!

Double-check commands involving wildcards (*) with rm before executing. Running ls *.tmp first to see what would be deleted is a good safety habit.

Standard Streams: stdin, stdout, stderr

Unix programs typically interact via three standard data streams:

• Standard Input (stdin, file descriptor 0): The default source of input data, usually the keyboard.
• Standard Output (stdout, file descriptor 1): The default destination for normal program output, usually the terminal screen.
• Standard Error (stderr, file descriptor 2): The default destination for error messages, usually the terminal screen.

These streams can be redirected. For example, > redirects stdout to a file, < redirects stdin from a file, and | (pipe) redirects stdout of one command to stdin of another. Redirection and pipes are powerful concepts covered in more detail elsewhere.