Python httpx: How to set request timeout

Introduction

Understanding how to manage request timeouts is crucial when making HTTP calls in Python. Here, we explore how to set request timeouts using the httpx library for both synchronous and asynchronous requests.

Setting a Basic Timeout

To start with, installing the httpx library is a prerequisite:

pip install httpx

Then, you can specify a timeout duration by passing the timeout parameter:

import httpx

timeout = 5.0  # Timeout of 5 seconds
response = httpx.get('https://example.com', timeout=timeout)

This sets a time limit on how long the client will wait for the server to send a response.

When setting timeouts, consider the nature of the request and the server’s expected response time. A balance between too short and too long of a timeout is essential.

Understanding Timeout Exceptions

When a timeout is exceeded, an httpx.TimeoutException is raised:

try:
    response = httpx.get('https://example.com', timeout=5.0)
except httpx.TimeoutException:
    print('The request timed out.')

This exception handling is useful to manage requests that might hang longer than expected.

Advanced Timeout Configurations

For fine-grained control, use a httpx.Timeout object:

from httpx import Timeout

custom_timeout = Timeout(
    connect=10.0,
    read=10.0,
    write=10.0,
    pool=10.0
)
response = httpx.get('https://example.com', timeout=custom_timeout)

The above code sets individual timeout limits for the connect, read, write, and pool operations.

Asynchronous Timeouts

When working with asynchronous requests, the timeout parameter works similarly:

import httpx

async def fetch_url():
    async with httpx.AsyncClient() as client:
        try:
            response = await client.get('https://example.com', timeout=5.0)
        except httpx.TimeoutException:
            print('Async request timed out.')

The same timeout rules apply to asynchronous requests, making it a versatile option.

Timeouts with HTTP Proxies

If you are using HTTP proxies, timeouts are still applicable:

proxies = {
    'http://': 'http://localhost:8080',
    'https://': 'https://localhost:8080',
}
response = httpx.get('https://example.com', proxies=proxies, timeout=5.0)

Whether using a proxy or not, timeout behavior remains consistent.

Adjusting Timeouts Mid-Request

Modifying timeout duration mid-request can be tricky but possible:

timeout = httpx.Timeout(10.0, pool=None)
with httpx.Client(timeout=timeout) as client:
    client.timeout.read = 20.0
    response = client.get('https://example.com')

This example shows how to alter the read timeout for a client instance, demonstrating advanced customization of httpx.

Conclusion

In summary, the httpx library offers both simple and advanced methods to manage request timeouts. This guide has covered how to set timeouts properly to ensure that your HTTP requests in Python are efficient and fail-safe. Always aim to optimize timeout settings based on the specific needs of your application.