Python: How to Upload Files with ‘httpx’ (form-data)

Overview

File uploads are a common task for many web applications. In Python, the ‘httpx’ library offers a straightforward way to handle file uploads using multipart/form-data. This tutorial explores how you can perform file uploads using httpx with stepwise examples.

Setting up the Environment

Before diving into file uploads, ensure that you have the httpx module installed in your Python environment. You can install it using pip:

pip install httpx

Basic File Upload

Let’s start with a basic example of uploading a single file to a server using the POST method and multipart/form-data.

import httpx

url = 'https://example.com/upload'
files = {'file': open('example.txt', 'rb')}
response = httpx.post(url, files=files)
print(response.status_code)

Here, we open the file in binary reading mode and pass it to the files parameter of the httpx post method. After sending the request, we print out the response status code.

Uploading Multiple Files

To upload multiple files at once, you can modify the files dictionary to include multiple file tuples.

files = [
    ('image', ('foo.png', open('foo.png', 'rb'), 'image/png')),
    ('document', ('bar.pdf', open('bar.pdf', 'rb'), 'application/pdf'))
]
response = httpx.post(url, files=files)
print(response.status_code)

Each file tuple consists of the field name, a tuple with the filename, file object, and content type.

Adding Additional Form Fields

Besides file data, you can send additional form fields in the same POST request.

files = {'file': ('example.txt', open('example.txt', 'rb'), 'text/plain')}
data = {'username': 'user234', 'token': 'secret'}
response = httpx.post(url, data=data, files=files)
print(response.status_code)

Here, we provide a dictionary of non-file form fields to the data parameter, which gets sent along with the files.

Streaming File Uploads

For large file uploads, you can stream the file to conserve memory. Use a with block to ensure the file is properly closed after the upload.

with open('largefile.zip', 'rb') as f:
    response = httpx.post(url, files={'file': ('largefile.zip', f)})
print(response.status_code)

This ensures that the file is kept open only for the duration of the upload.

Advanced Usage: Asynchronous Uploads

Httpx also supports asynchronous requests, which is ideal for I/O bound tasks such as file uploads. Use the ‘async with’ statement along with ‘await’ to perform the upload.

import httpx

async def upload_file(url, filepath):
    async with httpx.AsyncClient() as client:
        with open(filepath, 'rb') as f:
            files = {'file': (filepath, f)}
            response = await client.post(url, files=files)
            return response.status_code

# Usage example
# import asyncio
# asyncio.run(upload_file('https://example.com/upload', 'example.txt'))

Ensure to run this in an asynchronous context, like within an async function and using asyncio.run when calling from synchronous code.

Handling Errors and Timeouts

Handle potential exceptions and timeouts by using httpx’s built-in functionalities.

try:
    response = httpx.post(url, files=files, timeout=30)
    response.raise_for_status()
except httpx.HTTPError as err:
    print(f'HTTP error occurred: {err}')
except Exception as err:
    print(f'An error occurred: {err}')

This snippet will catch any HTTP errors that arise from the request and any other errors, all while respecting a 30-second timeout.

Handling Response Data

Once the file is uploaded, you might need to handle JSON responses from the server.

response = httpx.post(url, files=files)
if response.status_code == 200:
    print('Upload successful')
    print(response.json())  # Handle JSON response

Here, we check for a successful status code and print out the JSON response with the .json() method.

Conclusion

In this tutorial, we covered how to upload files using the ‘httpx’ library in Python from basic uploads to more advanced asynchronous methods. With the examples provided, you can apply file upload functionality in your projects with confidence and efficiency.