Sling Academy
Home/Python/Python: How to Upload Files with ‘httpx’ (form-data)

Python: How to Upload Files with ‘httpx’ (form-data)

Last updated: January 06, 2024

Overview

File uploads are a common task for many web applications. In Python, the ‘httpx’ library offers a straightforward way to handle file uploads using multipart/form-data. This tutorial explores how you can perform file uploads using httpx with stepwise examples.

Setting up the Environment

Before diving into file uploads, ensure that you have the httpx module installed in your Python environment. You can install it using pip:

pip install httpx

Basic File Upload

Let’s start with a basic example of uploading a single file to a server using the POST method and multipart/form-data.

import httpx

url = 'https://example.com/upload'
files = {'file': open('example.txt', 'rb')}
response = httpx.post(url, files=files)
print(response.status_code)

Here, we open the file in binary reading mode and pass it to the files parameter of the httpx post method. After sending the request, we print out the response status code.

Uploading Multiple Files

To upload multiple files at once, you can modify the files dictionary to include multiple file tuples.

files = [
    ('image', ('foo.png', open('foo.png', 'rb'), 'image/png')),
    ('document', ('bar.pdf', open('bar.pdf', 'rb'), 'application/pdf'))
]
response = httpx.post(url, files=files)
print(response.status_code)

Each file tuple consists of the field name, a tuple with the filename, file object, and content type.

Adding Additional Form Fields

Besides file data, you can send additional form fields in the same POST request.

files = {'file': ('example.txt', open('example.txt', 'rb'), 'text/plain')}
data = {'username': 'user234', 'token': 'secret'}
response = httpx.post(url, data=data, files=files)
print(response.status_code)

Here, we provide a dictionary of non-file form fields to the data parameter, which gets sent along with the files.

Streaming File Uploads

For large file uploads, you can stream the file to conserve memory. Use a with block to ensure the file is properly closed after the upload.

with open('largefile.zip', 'rb') as f:
    response = httpx.post(url, files={'file': ('largefile.zip', f)})
print(response.status_code)

This ensures that the file is kept open only for the duration of the upload.

Advanced Usage: Asynchronous Uploads

Httpx also supports asynchronous requests, which is ideal for I/O bound tasks such as file uploads. Use the ‘async with’ statement along with ‘await’ to perform the upload.

import httpx

async def upload_file(url, filepath):
    async with httpx.AsyncClient() as client:
        with open(filepath, 'rb') as f:
            files = {'file': (filepath, f)}
            response = await client.post(url, files=files)
            return response.status_code

# Usage example
# import asyncio
# asyncio.run(upload_file('https://example.com/upload', 'example.txt'))

Ensure to run this in an asynchronous context, like within an async function and using asyncio.run when calling from synchronous code.

Handling Errors and Timeouts

Handle potential exceptions and timeouts by using httpx’s built-in functionalities.

try:
    response = httpx.post(url, files=files, timeout=30)
    response.raise_for_status()
except httpx.HTTPError as err:
    print(f'HTTP error occurred: {err}')
except Exception as err:
    print(f'An error occurred: {err}')

This snippet will catch any HTTP errors that arise from the request and any other errors, all while respecting a 30-second timeout.

Handling Response Data

Once the file is uploaded, you might need to handle JSON responses from the server.

response = httpx.post(url, files=files)
if response.status_code == 200:
    print('Upload successful')
    print(response.json())  # Handle JSON response

Here, we check for a successful status code and print out the JSON response with the .json() method.

Conclusion

In this tutorial, we covered how to upload files using the ‘httpx’ library in Python from basic uploads to more advanced asynchronous methods. With the examples provided, you can apply file upload functionality in your projects with confidence and efficiency.

Next Article: Python httpx: How to make GET and POST requests

Previous Article: Python ‘requests’ module: How to force use of IPv4 or IPv6

Series: Python: Network & JSON tutorials

Python

You May Also Like

  • Python Warning: Secure coding is not enabled for restorable state
  • Python TypeError: write() argument must be str, not bytes
  • 4 ways to install Python modules on Windows without admin rights
  • Python TypeError: object of type ‘NoneType’ has no len()
  • Python: How to access command-line arguments (3 approaches)
  • Understanding ‘Never’ type in Python 3.11+ (5 examples)
  • Python: 3 Ways to Retrieve City/Country from IP Address
  • Using Type Aliases in Python: A Practical Guide (with Examples)
  • Python: Defining distinct types using NewType class
  • Using Optional Type in Python (explained with examples)
  • Python: How to Override Methods in Classes
  • Python: Define Generic Types for Lists of Nested Dictionaries
  • Python: Defining type for a list that can contain both numbers and strings
  • Using TypeGuard in Python (Python 3.10+)
  • Python: Using ‘NoReturn’ type with functions
  • Type Casting in Python: The Ultimate Guide (with Examples)
  • Python: Using type hints with class methods and properties
  • Python: Typing a function with default parameters
  • Python: Typing a function that can return multiple types