Python: Using the “yield” keyword with async/await (3 examples)

In Python, the yield keyword is used to create a generator, which is a special kind of iterator that can be paused and resumed. In Python 3.6 and above, you can use the yield keyword inside an async function to create an asynchronous generator, which is an iterator that can also perform asynchronous operations.

This succinct, code-centric article will walk you through several examples of using yield with async/await in modern Python. Without any further ado, let’s get started.

A simple async generator that yields numbers (basic)

In this example, we’ll create a simple async generator that yields numbers from 0 to n with a delay of one second:

# SlingAcademy.com
# This code uses Python 3.11.4

import asyncio


async def countdown(n):
    while n >= 0:
        # wait for one second
        await asyncio.sleep(1)  

        # yield the current value of n
        yield n  

        # decrement n by one
        n -= 1  


async def main():
    # iterate over the async generator
    async for i in countdown(5):  
        # print the yielded value
        print(i)  

# run the main coroutine
asyncio.run(main())

The code will print 5, 4, 3, 2, 1, and 0 with a one-second interval between each print:

5
4
3
2
1
0

Using “yield” with an asyncio queue (intermediate)

In this example, we’ll use the yield keyword with an asynchronous queue to create an async generator that produces Fibonacci numbers:

# SlingAcademy.com
# This code uses Python 3.11.4

import asyncio


async def fibonacci():
    # create an async queue
    queue = asyncio.Queue()

    # put the initial pair of numbers in the queue
    await queue.put((0, 1))
    while True:
        # get the next pair of numbers from the queue
        a, b = await queue.get()

        # yield the first number
        yield a

        # put the next pair of numbers in the queue
        await queue.put((b, a + b))


async def main():
     # iterate over the async generator
    async for i in fibonacci(): 
        # stop when the number is greater than 100
        if i > 100:  
            break

        # mimic some delay
        await asyncio.sleep(1)

        # print the Fibonacci number
        print(i)  

# run the main coroutine
asyncio.run(main())  

The code will generate Fibonacci numbers up to 89. Each number will show up after a one-second delay:

0
1
1
2
3
5
8
13
21
34
55
89

You can find more details about async queues in this article: Python asyncio.Queue class.

Async generator producer-consumer pattern with queue and sentinel (advanced)

Note: The word “sentinel” here means a guard or a watchman. It is used to describe a value that acts as a signal or a marker for the end of a data stream or a loop. A sentinel value is usually chosen to be different from any valid data value so that it can be easily recognized and handled.

This example demonstrates the way to use an async generator to implement a producer-consumer pattern, where one coroutine produces data and another coroutine consumes it. The producer coroutine uses an async queue to store the data, and the consumer coroutine uses an async for loop to get the data from the queue. The producer coroutine also uses a sentinel value to signal the end of the data stream:

# SlingAcademy.com
# This code uses Python 3.11.4


import asyncio
import random

# set a random seed for reproducibility
random.seed(2023)

async def produce(queue):
    for i in range(10):
        # simulate some work by sleeping for a random amount of time
        await asyncio.sleep(random.random())

        # generate a random item
        item = random.randint(1, 10)
        print(f"Produced {item}")

        # put the item in the queue
        await queue.put(item)
    # put None in the queue to indicate the end of the stream
    await queue.put(None)

async def consume(queue):
    # create an async generator from the queue
    async def get_items():
        while True:
            # get an item from the queue
            item = await queue.get()

            # if the item is None, break the loop
            if item is None:
                break

            # yield the item
            yield item

    # use an async for loop to iterate over the generator
    async for item in get_items():
        # simulate some work
        await asyncio.sleep(random.random())
        print(f"Consumed {item}")

async def main():
    # create a bounded queue with a capacity of 5
    queue = asyncio.Queue(maxsize=5)
    
    # run the producer and consumer coroutines concurrently
    await asyncio.gather(produce(queue), consume(queue))

asyncio.run(main())

Output (you’ll notice a short delay before each print):

Produced 8
Consumed 8
Produced 10
Consumed 10
Produced 2
Produced 9
Consumed 2
Consumed 9
Produced 4
Produced 1
Consumed 4
Consumed 1
Produced 3
Produced 4
Consumed 3
Produced 8
Consumed 4
Produced 2
Consumed 8

Final words

We’ve examined some practical examples of using async generators, which can produce a stream of values asynchronously. You can make use of them to improve the performance and responsiveness of your programs. This tutorial ends here. Happy coding & have fun with modern Python programming!