Sling Academy
Home/Python/Python asyncio.as_completed() function (with examples)

Python asyncio.as_completed() function (with examples)

Last updated: August 07, 2023

In this concise and code-centric article, you’ll learn about the asyncio._as_completed() function in Python (Python 3.4 or newer is required).

Overview

The asyncio.as_completed() function is used to run a collection of tasks and coroutines concurrently, and return an iterator of coroutines that can be awaited to get the results in the order that they are completed (this point is important).

Syntax:

asyncio.as_completed(aws, *, timeout=None)

Where:

  • aws: An iterable of awaitable objects that will be run concurrently. Any coroutines in this iterable will be wrapped in tasks automatically.
  • timeout: An optional timeout value in seconds that specifies how long the function will wait for all awaitables to be done. If the timeout elapses before all awaitables are done, an asyncio.TimeoutError exception will be raised.

The returned value of the function is an iterator of coroutines. Each coroutine returned by the iterator can be awaited to get the next result from the iterable of the remaining awaitables.

asyncio.as_completed() is useful when you want to execute multiple tasks or coroutines concurrently and get the results in the order that they are completed, rather than the order they were issued. This can be useful for optimizing performance or handling tasks with different priorities. The function should not be used when you need to preserve the order of the results or when you don’t need to wait for all tasks or coroutines to finish.

A lot of words can make you get bored and feel sleepy. It’s time to write some code.

Examples

Basic Example

The example below demonstrates how to use asyncio.as_completed() to run three coroutines that sleep for different durations and print their results in the order that they are completed:

# SlingAcademy.com
# This code uses Python 3.11.4

import asyncio
import time


# Define three coroutines that sleep for different durations
async def sleeper(duration, task_name=""):
    await asyncio.sleep(duration)
    return f"{task_name} slept for {duration} seconds"


# Create a list of coroutines
coros = [
    sleeper(4, task_name="task A"),
    sleeper(1, task_name="task B"),
    sleeper(2, task_name="task C"),
]


# Run the coroutines concurrently and get the results in the order they are completed
async def main():
    start = time.perf_counter()
    for coro in asyncio.as_completed(coros):
        # Wait for the result
        result = await coro
        print(f"Result: {result}")
    end = time.perf_counter()
    print(f"Elapsed time: {end - start}")


# Run the main coroutine
asyncio.run(main())

Output:

Result: task B slept for 1 seconds
Result: task C slept for 2 seconds
Result: task A slept for 4 seconds
Elapsed time: 4.001339041999017

Advanced Example

In this example, we’ll use asyncio.as_completed() and the aiohttp library to fetch data from multiple API URLs concurrently and print their status codes in the order that they are received. It also uses a timeout argument to limit the waiting time.

Install aiohttp:

pip install aiohttp
# or
pip3 install aiohttp

The code:

# SlingAcademy.com
# This code uses Python 3.11.4

import asyncio
import aiohttp

# Define a list of API URLs to fetch
urls = [
    "https://api.slingacademy.com",
    "https://api.slingacademy.com/v1/sample-data/photos",
    "https://api.slingacademy.com/v1/sample-data/files/customers.csv"
]


# Define a coroutine that fetches data from a URL and returns its status code
async def fetch(url):
    async with aiohttp.ClientSession() as session:
        async with session.get(url) as response:
            return f"{response.status} - {url}"


# Run the coroutines concurrently and get the results in the order they are received
async def main():
    timeout = 5  # Set a timeout value in seconds
    try:
        for coro in asyncio.as_completed([fetch(url) for url in urls], timeout=timeout):
            status = await coro  # Wait for the next result
            print(f"Status: {status}")
    except asyncio.TimeoutError:
        print(f"Timeout after {timeout} seconds")


# Run the main coroutine
asyncio.run(main())

Output:

Status: 200 - https://api.slingacademy.com
Status: 200 - https://api.slingacademy.com/v1/sample-data/photos
Status: 200 - https://api.slingacademy.com/v1/sample-data/files/customers.csv

The order of the results is not consistent. This is because the asyncio.as_completed() function returns the results in the order that they are received, which depends on the network latency and the server response time of each URL. Therefore, the order of the results may vary each time you re-execute the code.

Afterword

You’ve learned the fundamentals of the asyncio.as_completed() function and gone over some examples of utilizing it in practice. From now on, you’ll have a great option to choose from when dealing with asynchronous jobs in Python. If you find errors or anachronisms in the code examples, please let me know by leaving comments. I will review and update them as soon as possible.

Next Article: Python: Using the “yield” keyword with async/await (3 examples)

Previous Article: Python asyncio.wait() function (with examples)

Series: Python Asynchronous Programming Tutorials

Python

You May Also Like

  • Introduction to yfinance: Fetching Historical Stock Data in Python
  • Monitoring Volatility and Daily Averages Using cryptocompare
  • Advanced DOM Interactions: XPath and CSS Selectors in Playwright (Python)
  • Automating Strategy Updates and Version Control in freqtrade
  • Setting Up a freqtrade Dashboard for Real-Time Monitoring
  • Deploying freqtrade on a Cloud Server or Docker Environment
  • Optimizing Strategy Parameters with freqtrade’s Hyperopt
  • Risk Management: Setting Stop Loss, Trailing Stops, and ROI in freqtrade
  • Integrating freqtrade with TA-Lib and pandas-ta Indicators
  • Handling Multiple Pairs and Portfolios with freqtrade
  • Using freqtrade’s Backtesting and Hyperopt Modules
  • Developing Custom Trading Strategies for freqtrade
  • Debugging Common freqtrade Errors: Exchange Connectivity and More
  • Configuring freqtrade Bot Settings and Strategy Parameters
  • Installing freqtrade for Automated Crypto Trading in Python
  • Scaling cryptofeed for High-Frequency Trading Environments
  • Building a Real-Time Market Dashboard Using cryptofeed in Python
  • Customizing cryptofeed Callbacks for Advanced Market Insights
  • Integrating cryptofeed into Automated Trading Bots