shell_commands

Shell Commands

IPython provides convenient ways to interact with the underlying operating system’s shell directly from your interactive session or notebook. This allows you to run system commands like listing files, creating directories, or using command-line tools without leaving your Python environment.

Running Shell Commands with !

Any line starting with an exclamation mark (!) is treated as a shell command and executed by the system’s default shell (like bash on Linux/macOS or cmd.exe on Windows).

# List files in the current directory (Linux/macOS)
!ls -l

# List files in the current directory (Windows)
!dir

# Print the current working directory (cross-platform using Python)
!python -c "import os; print(os.getcwd())"

# Create a new directory
!mkdir my_new_directory

# Show network configuration (Linux/macOS)
!ifconfig
# (or !ip addr on newer Linux)

# Show network configuration (Windows)
!ipconfig

The output of the shell command is printed directly below the cell.

Capturing Shell Command Output

You can assign the output of a shell command to a Python variable. IPython captures the standard output (stdout) and returns it as a list of strings (split by newline characters).

files = !ls
print(f"Type of 'files': {type(files)}")
print(f"Number of files/dirs: {len(files)}")
print("First few entries:")
for i, f in enumerate(files[:5]):
    print(f"  {i}: {f}")

# Example on Windows:
dir_output = !dir *.txt
print(dir_output)

Note: This captures stdout. Errors printed to stderr are usually still displayed directly but not captured in the variable unless you use shell redirection (e.g., !command 2>&1). The output format might sometimes include terminal escape codes for colors, which might need cleaning if processing programmatically.

Using Python Variables in Shell Commands

You can embed Python variables directly into your shell commands. There are two main ways:

1. Using Curly Braces {}

Place the Python variable name inside curly braces {} within the shell command string.

filename = "my_data.txt"
message = "Hello from Python!"

# Create a file with content from a Python variable
!echo {message} > {filename}

# Display the file content
!cat {filename}
# (use !type {filename} on Windows)

# List the specific file
!ls -l {filename}

2. Using Dollar Sign $

Alternatively, prefix the Python variable name with a dollar sign $. This syntax is similar to shell variable expansion but works with Python variables in IPython.

dirname = "new_folder"

# Create the directory using the variable
!mkdir $dirname

# Check if it exists
!ls -d $dirname

# Pass a list of arguments (IPython handles expansion)
files_to_list = ["my_data.txt", dirname]
!ls -l {files_to_list} # Works with braces too
# !ls -l $files_to_list # $ syntax might be less robust for lists/complex objects

The {} syntax is generally preferred as it’s more explicit and often handles different types (like lists) more predictably.

Use Cases

Integrating shell commands is useful for:

• File and directory management (ls, dir, mkdir, rm, cp, mv).
• Running command-line utilities (e.g., grep, awk, curl, wget).
• Version control operations (!git status, !git log).
• Compiling code or running external processes.
• Quick system checks (!pwd, !uname -a, !hostname).

While Python’s os and subprocess modules offer more control for complex interactions, the ! syntax provides a quick and easy way for simple shell operations within an interactive workflow.