Python asyncio.as_completed() function (with examples)

In this concise and code-centric article, you’ll learn about the asyncio._as_completed() function in Python (Python 3.4 or newer is required).

Overview

The asyncio.as_completed() function is used to run a collection of tasks and coroutines concurrently, and return an iterator of coroutines that can be awaited to get the results in the order that they are completed (this point is important).

Syntax:

asyncio.as_completed(aws, *, timeout=None)

Where:

• aws: An iterable of awaitable objects that will be run concurrently. Any coroutines in this iterable will be wrapped in tasks automatically.
• timeout: An optional timeout value in seconds that specifies how long the function will wait for all awaitables to be done. If the timeout elapses before all awaitables are done, an asyncio.TimeoutError exception will be raised.

The returned value of the function is an iterator of coroutines. Each coroutine returned by the iterator can be awaited to get the next result from the iterable of the remaining awaitables.

asyncio.as_completed() is useful when you want to execute multiple tasks or coroutines concurrently and get the results in the order that they are completed, rather than the order they were issued. This can be useful for optimizing performance or handling tasks with different priorities. The function should not be used when you need to preserve the order of the results or when you don’t need to wait for all tasks or coroutines to finish.

A lot of words can make you get bored and feel sleepy. It’s time to write some code.

Examples

Basic Example

The example below demonstrates how to use asyncio.as_completed() to run three coroutines that sleep for different durations and print their results in the order that they are completed:

# SlingAcademy.com
# This code uses Python 3.11.4

import asyncio
import time


# Define three coroutines that sleep for different durations
async def sleeper(duration, task_name=""):
    await asyncio.sleep(duration)
    return f"{task_name} slept for {duration} seconds"


# Create a list of coroutines
coros = [
    sleeper(4, task_name="task A"),
    sleeper(1, task_name="task B"),
    sleeper(2, task_name="task C"),
]


# Run the coroutines concurrently and get the results in the order they are completed
async def main():
    start = time.perf_counter()
    for coro in asyncio.as_completed(coros):
        # Wait for the result
        result = await coro
        print(f"Result: {result}")
    end = time.perf_counter()
    print(f"Elapsed time: {end - start}")


# Run the main coroutine
asyncio.run(main())

Output:

Result: task B slept for 1 seconds
Result: task C slept for 2 seconds
Result: task A slept for 4 seconds
Elapsed time: 4.001339041999017

Advanced Example

In this example, we’ll use asyncio.as_completed() and the aiohttp library to fetch data from multiple API URLs concurrently and print their status codes in the order that they are received. It also uses a timeout argument to limit the waiting time.

Install aiohttp:

pip install aiohttp
# or
pip3 install aiohttp

The code:

# SlingAcademy.com
# This code uses Python 3.11.4

import asyncio
import aiohttp

# Define a list of API URLs to fetch
urls = [
    "https://api.slingacademy.com",
    "https://api.slingacademy.com/v1/sample-data/photos",
    "https://api.slingacademy.com/v1/sample-data/files/customers.csv"
]


# Define a coroutine that fetches data from a URL and returns its status code
async def fetch(url):
    async with aiohttp.ClientSession() as session:
        async with session.get(url) as response:
            return f"{response.status} - {url}"


# Run the coroutines concurrently and get the results in the order they are received
async def main():
    timeout = 5  # Set a timeout value in seconds
    try:
        for coro in asyncio.as_completed([fetch(url) for url in urls], timeout=timeout):
            status = await coro  # Wait for the next result
            print(f"Status: {status}")
    except asyncio.TimeoutError:
        print(f"Timeout after {timeout} seconds")


# Run the main coroutine
asyncio.run(main())

Output:

Status: 200 - https://api.slingacademy.com
Status: 200 - https://api.slingacademy.com/v1/sample-data/photos
Status: 200 - https://api.slingacademy.com/v1/sample-data/files/customers.csv

The order of the results is not consistent. This is because the asyncio.as_completed() function returns the results in the order that they are received, which depends on the network latency and the server response time of each URL. Therefore, the order of the results may vary each time you re-execute the code.

Afterword

You’ve learned the fundamentals of the asyncio.as_completed() function and gone over some examples of utilizing it in practice. From now on, you’ll have a great option to choose from when dealing with asynchronous jobs in Python. If you find errors or anachronisms in the code examples, please let me know by leaving comments. I will review and update them as soon as possible.