Python requests module: How to upload files (form-data)

Introduction

The Python requests module vastly simplifies HTTP requests in Python, including the capability to upload files. This tutorial covers the basic to advanced use cases for file uploading.

Basic File Upload

Uploading a file using the requests module is straightforward. At its simplest, you can upload a file by specifying the file path and the target URL:

import requests

url = 'http://example.com/upload'
files = {'file': open('report.xls', 'rb')}

response = requests.post(url, files=files)
print(response.text)

In the above example, we open the file in binary read mode (rb), which is crucial when dealing with non-text files, like images or PDFs. The ‘files’ dictionary is the key aspect here, telling `requests` which files to upload.

Uploading Multiple Files

If you need to upload more than one file, simply expand the files dictionary:

files = {
    'file1': open('report1.xls', 'rb'),
    'file2': open('report2.xls', 'rb'),
}
response = requests.post(url, files=files)

It is generally a good idea to close files after you’re done with them. This can be handled automatically using a with statement:

with open('report1.xls', 'rb') as f1, open('report2.xls', 'rb') as f2:
    files = {'file1': f1, 'file2': f2}
    response = requests.post(url, files=files)

Setting Custom File Names and Content Types

Sometimes, you want to set a different file name or define a specific content-type. This can be done by providing a tuple instead of just the file object:

files = {
    'file': ('report.pdf', open('report.pdf', 'rb'), 'application/pdf')
}
response = requests.post(url, files=files)

In this tuple, the first element is the custom file name, the second is the file object, and the third is the MIME type of the file.

Adding Additional Data

You might also need to send additional data along with your files. This is simply another dictionary in the same POST request:

data = {'key': 'value'}
files = {'file': open('report.pdf', 'rb')}
response = requests.post(url, data=data, files=files)

File Upload with Request Headers

Occasionally, you’ll need to include custom headers in your request:

headers = {'Authorization': 'Bearer YOUR_TOKEN'}
response = requests.post(url, files=files, headers=headers)

Advanced Usage

What if your server expects the files in a JSON structure, or you need to perform a more advanced operation like streaming file uploads? For JSON-encoded files, you may need to serialize your data before sending:

import json

file_content = open('report.pdf', 'rb').read()
file_json = json.dumps({'file_content': file_content.decode('utf-8')})
response = requests.post(url, data=file_json, headers={'Content-Type': 'application/json'})

For streamed uploads, you’ll need to use the stream=True parameter and handle the file read:

with open('report.pdf', 'rb') as f:
    response = requests.post(url, data=f, stream=True)

Note: When streaming, the entire file is not read into memory at once, which is quite useful for large files.

Error Handling

Handling exceptions is crucial to understand what went wrong if something fails during the request:

try:
    response = requests.post(url, files=files)
    response.raise_for_status()
except requests.exceptions.HTTPError as e:
    print(f'HTTP Error: {e}')
except requests.exceptions.RequestException as e:
    print(f'Request Exception: {e}')

This exception handling ensures you get a clear understanding of whether it’s an HTTP error or some other request-related issue.

Summary

In conclusion, the Python requests module provides an incredibly user-friendly approach to uploading files. By mastering the examples given, from basic to advanced, you will be able to handle a wide variety of file upload scenarios in your applications.