Requests - File Upload

Requests: Seamless File Uploads

The requests library makes uploading files via HTTP POST requests straightforward. This is essential for interacting with web applications that require file submissions, such as uploading images, documents, or data files. This chapter covers how to upload single and multiple files, and how to customize file uploads.

We’ll use https://httpbin.org/post as our test endpoint, which echoes back the details of the POST request, including uploaded files.

Uploading a Single File

To upload a file, you pass a dictionary to the files parameter of requests.post(). The dictionary keys are the field names expected by the server, and the values are file-like objects.

It’s crucial to open files in binary mode ('rb') for uploads. Using a with statement (context manager) ensures the file is automatically closed after the request, which is a best practice.

Example: Uploading test.txt

First, create a sample file named test.txt in your project directory with some content:

Content of test.txt:

This is a test file for demonstrating file uploads with Python Requests.

Now, let’s write the Python code to upload it:

import requests

url = 'https://httpbin.org/post'
file_path = 'test.txt'

# Best practice: use a context manager to handle the file
try:
    with open(file_path, 'rb') as f:
        files_to_upload = {'file': f}  # 'file' is the field name
        response = requests.post(url, files=files_to_upload)

    print(f"Status Code: {response.status_code}")
    # httpbin.org/post returns a JSON response detailing the upload
    response_data = response.json()
    print("Uploaded file content (from server response):")
    print(response_data['files']['file'])

except FileNotFoundError:
    print(f"Error: The file {file_path} was not found.")
except requests.exceptions.RequestException as e:
    print(f"An error occurred: {e}")

Output (example)

Status Code: 200
Uploaded file content (from server response):
This is a test file for demonstrating file uploads with Python Requests.

In the response from httpbin.org, the files field will contain the content of the uploaded file, confirming a successful upload.

Customizing File Uploads

You can customize the filename, content type, and add custom headers for the uploaded file by providing a tuple instead of just the file object.

Example: Specifying Filename, Content-Type, and Custom Headers

import requests

url = 'https://httpbin.org/post'
file_path = 'test.txt' # Assuming test.txt exists

# Tuple format: (filename, file_object, content_type, custom_headers)
# Content_type and custom_headers are optional.

try:
    with open(file_path, 'rb') as f:
        files_spec = {
            'document': ('report.txt', f, 'text/plain', {'X-Custom-Info': 'MyReportV1'})
        }
        response = requests.post(url, files=files_spec)

    print(f"Status Code: {response.status_code}")
    response_data = response.json()
    print("Server Response (files part):")
    print(response_data['files'])
    print("Server Response (headers part - to see Content-Disposition):")
    # The actual multipart headers are complex, httpbin shows the parsed file part.
    # To see Content-Disposition header sent by requests, you'd inspect the raw request or use a tool like Wireshark.
    # httpbin shows the file content under the key 'document'.

except FileNotFoundError:
    print(f"Error: The file {file_path} was not found.")
except requests.exceptions.RequestException as e:
    print(f"An error occurred: {e}")

Output (example, focusing on the ‘files’ part of the response)

Status Code: 200
Server Response (files part):
{'document': 'This is a test file for demonstrating file uploads with Python Requests.'}
Server Response (headers part - to see Content-Disposition):
...

In this case, the server will see the file uploaded with the field name document, the filename report.txt, content type text/plain, and the custom header X-Custom-Info associated with this part of the multipart request.

Uploading Multiple Files

To upload multiple files in a single request, simply add more entries to the files dictionary. Each key-value pair represents a file to be uploaded.

Example: Uploading Two Files

Let’s assume you have file1.txt and image.png.

import requests

url = 'https://httpbin.org/post'

# Create dummy files for the example if they don't exist
with open('file1.txt', 'w') as f1: f1.write('Content of file1')
with open('image.png', 'wb') as f2: f2.write(b'\x89PNG\r\n\x1a\n\0\0\0\rIHDR...') # Dummy PNG data

files_to_upload = []
try:
    # Prepare file list for opening with context managers
    # (filename_in_form, actual_filename_on_disk, content_type_if_needed)
    file_specs = [
        ('text_file', 'file1.txt', 'text/plain'),
        ('image_file', 'image.png', 'image/png')
    ]

    # Build the files dictionary for requests
    # Need to keep file objects open until request is made
    opened_files = [] # To hold file objects that need closing if not using with for each
    files_dict_for_requests = {}

    for form_name, disk_filename, content_type in file_specs:
        f_obj = open(disk_filename, 'rb')
        opened_files.append(f_obj)
        files_dict_for_requests[form_name] = (disk_filename, f_obj, content_type)

    response = requests.post(url, files=files_dict_for_requests)

    print(f"Status Code: {response.status_code}")
    response_data = response.json()
    print("Uploaded files (from server response):")
    print(response_data['files'])

finally:
    for f_obj in opened_files:
        f_obj.close()

Output (example, focusing on the ‘files’ part)

Status Code: 200
Uploaded files (from server response):
{
  "image_file": "data:image/png;base64,...",
  "text_file": "Content of file1"
}

Note: httpbin.org might return binary file content as a base64 encoded data URI. The important part is that both image_file and text_file keys are present in the files section of the response, indicating they were received by the server.

Sending In-Memory Content as a File

You don’t always need to read from a disk file. You can also send string or byte content directly as if it were a file:

import requests

url = 'https://httpbin.org/post'

files = {'file': ('custom_filename.txt', 'Hello, this is in-memory content!')}
response = requests.post(url, files=files)

print(f"Status Code: {response.status_code}")
response_data = response.json()
print(response_data['files']['file'])

Output (example)

Status Code: 200
Hello, this is in-memory content!

Real-World Applications

File uploads are common in web applications for features like: user profile picture updates, submitting reports or documents, attaching files to messages, or uploading datasets for processing.

Always refer to the API documentation of the service you are interacting with to understand the expected field names, file types, and size limits for uploads.