Python asyncio.loop.run_in_executor() method (3 examples)

This concise, straight-to-the-point article is about the asyncio.loop.run_in_executor() method in Python.

Quick Overview

The asyncio.loop.run_in_executor() method was added to the programming language in Python 3.4, as part of the asyncio module. It’s useful for running blocking code in a different thread or process without blocking the event loop.

Syntax & parameters

Syntax:

loop.run_in_executor(executor, func, *args)

Where:

• loop is an instance of asyncio.AbstractEventLoop or a subclass.
• executor is an instance of concurrent.futures.Executor or None. If None, the default executor is used.
• func is a callable object that will be executed in the executor.
• *args are positional arguments that will be passed to func when it is executed.

The returned value of the method is an asyncio.Future object that can be awaited or used with callbacks to get the result of the execution. The result will be either the return value of func or an exception if something goes wrong.

When and when NOT to use the loop.run_in_executor() method

The asyncio.loop.run_in_executor() method shines when there is a need to run blocking code (such as I/O operations, CPU-intensive calculations, or subprocesses) in a different thread or process without blocking the event loop. This way, the event loop can continue to run other coroutines and callbacks while waiting for the result of the execution.

The method shouldn’t be used when there is a non-blocking alternative available (such as asyncio.sleep(), asyncio.subprocess, or asyncio.streams), or when the code can be refactored to use coroutines and await expressions.

Examples

Words can be boring and confusing. Let’s take a look at some practical examples to see how asyncio.loop.run_in_executor() works in real-world applications.

Basic example

What we’re going to do here is to run a function that sleeps for 5 seconds in a thread pool and prints the result:

# SlingAcademy.com
# This code uses Python 3.11.4

import asyncio
import time
from datetime import datetime

# This is a blocking function
def blocking_func():
    # Simulate a blocking operation by sleeping for 5 seconds
    # Note that we are using time.sleep() instead of asyncio.sleep()
    time.sleep(5)
    return "This is the result of the blocking function."

# This is the main coroutine
async def main():
    # Print the start time
    print(f"Started at {datetime.now().strftime('%H:%M:%S')}")

    # Get the current event loop
    loop = asyncio.get_running_loop()

    # Run the blocking function in a thread pool and get a Future object
    future = loop.run_in_executor(None, blocking_func)

    # Wait for the Future object to complete and get the result
    result = await future

    # Print the result
    print(result)

    # Print the end time
    print(f"Finished at {datetime.now().strftime('%H:%M:%S')}")


asyncio.run(main())

Output (the printed time will depend on when you execute the code):

Started at 15:45:16
This is the result of the blocking function.
Finished at 15:45:21

Advanced example

This example shows how to run multiple functions concurrently in different threads using an asyncio.gather() call. It also shows how to handle exceptions raised by the functions using a try/except block:

# SlingAcademy.com
# This code uses Python 3.11.4

import asyncio
import random


# Define the lucky_number() function
def lucky_number():
    # Generate a random number between 1 and 10
    n = random.randint(1, 10)

    # If the number is 7, raise an exception
    if n == 7:
        raise ValueError("Unlucky number!")
    
    # Otherwise, perform some CPU-intensive calculations
    x = 0
    for i in range(1000000):
        x += i * n

    # Return the final value of x
    return x


# Define the main() coroutine
async def main():
    # Get the current event loop
    loop = asyncio.get_running_loop()

    # start time
    start = loop.time()

    # Create a list of futures for running lucky_number() five times in different threads
    futures = [loop.run_in_executor(None, lucky_number) for _ in range(5)]

    # Wait for all futures to complete and collect the results
    try:
        results = await asyncio.gather(*futures)
        print(f"The lucky numbers are {results}")
    except ValueError as e:
        print(f"An exception occurred: {e}")

    # end time
    end = loop.time()

    # Print the total execution time
    print(f"Total execution time: {end - start} second(s)")


asyncio.run(main())

The output might vary. Here’s a possible one:

An exception occurred: Unlucky number!
Total execution time: 0.0834342079997441 second(s)

And this is another possible one:

The lucky numbers are [1499998500000, 3999996000000, 4999995000000, 2499997500000, 4999995000000]
Total execution time: 0.16242466699986835 second(s)

Final words

This article covered everything you need to know about the asyncio.loop.run_in_executor() method in Python. From this point, you’ll have one more powerful option to deal with heavy, blocking operations in your programs. If you find something outdated or incorrect, please leave a comment.

The tutorial ends here. Happy coding & have fun with Python programming!