Python Requests module: Tracking redirection and history

Introduction

Understanding HTTP redirection is key when making web requests. The Python Requests module simplifies these tasks, allowing you to track redirections and the request history with ease.

Getting Started with the Requests Module

To begin, you need the Requests module installed in your Python environment:

pip install requests

Once installed, you can start making HTTP requests. Here’s a basic example:

import requests

response = requests.get('http://example.com')
print(response.status_code)
print(response.url)

Understanding Redirection

Some HTTP requests lead to redirections. The server responds with a header telling the client to look somewhere else.

Here’s how to check if a response was redirected:

response = requests.get('http://example.com/some-redirect-url')
if response.history:
    print("Request was redirected")
for resp in response.history:
    print(resp.status_code, resp.url)

And to get the final URL:

print("Final destination:", response.url)

Handling Redirection Manually

If you require control over redirection, disable it as follows:

response = requests.get('http://example.com/some-redirect', allow_redirects=False)
print(response.status_code)

To follow the redirect manually:

if response.status_code == 302:
    new_url = response.headers['Location']
    follow_up_resp = requests.get(new_url)
    print(follow_up_resp.url)

Advanced Handling of Redirection Chains

For more complex redirection scenarios, such as capturing cookies or detailed information throughout the redirection chain, use session objects:

with requests.Session() as s:
    response = s.get('http://example.com/some-redirect')
    for resp in response.history:
        print(resp.status_code, resp.url, resp.cookies)

Special types of redirection

HTTP status codes like 301, 302, and 307 represent different types of redirection. You can adapt to each type:

if response.status_code == 301:
    # Permanent redirection logic
elif response.status_code == 302:
    # Temporary redirection logic
elif response.status_code == 307:
    # Redirect but keep the method as POST

Error Handling

When tracking redirection, errors might happen. Ensure robust code by handling them:

try:
    response = requests.get('http://example.com/bad-redirect')
    response.raise_for_status()
except requests.exceptions.HTTPError as errh:
    print ("An Http Error occurred:\n", errh)
except requests.exceptions.ConnectionError as errc:
    print ("An Error Connecting to the API occurred:\n", errc)
except requests.exceptions.Timeout as errt:
    print ("A Timeout Error occurred:\n", errt)
except requests.exceptions.RequestException as err:
    print ("An Unknown Error occurred:\n", err)

Advanced Redirections with Parameters

By using session objects, you can pass along custom headers, authentication, and more across redirects:

payload = {'key1': 'value1', 'key2': 'value2'}
headers = {'user-agent': 'my-app/0.0.1'}

with requests.Session() as s:
    s.headers.update(headers)
    response = s.get('http://httpbin.org/redirect/1', params=payload)
    for resp in response.history:
        print(resp.request.headers, resp.url)

Summary

Mastering the Requests module for tracking HTTP redirections enhances the robustness and efficiency of your web interaction scripts. This tutorial provided foundational knowledge and techniques to adeptly navigate and utilize HTTP redirections in Python.