In Python, the yield
keyword is used to create a generator, which is a special kind of iterator that can be paused and resumed. In Python 3.6 and above, you can use the yield
keyword inside an async function to create an asynchronous generator, which is an iterator that can also perform asynchronous operations.
This succinct, code-centric article will walk you through several examples of using yield
with async/await
in modern Python. Without any further ado, let’s get started.
A simple async generator that yields numbers (basic)
In this example, we’ll create a simple async generator that yields numbers from 0 to n
with a delay of one second:
# SlingAcademy.com
# This code uses Python 3.11.4
import asyncio
async def countdown(n):
while n >= 0:
# wait for one second
await asyncio.sleep(1)
# yield the current value of n
yield n
# decrement n by one
n -= 1
async def main():
# iterate over the async generator
async for i in countdown(5):
# print the yielded value
print(i)
# run the main coroutine
asyncio.run(main())
The code will print 5, 4, 3, 2, 1, and 0 with a one-second interval between each print:
5
4
3
2
1
0
Using “yield” with an asyncio queue (intermediate)
In this example, we’ll use the yield
keyword with an asynchronous queue to create an async generator that produces Fibonacci numbers:
# SlingAcademy.com
# This code uses Python 3.11.4
import asyncio
async def fibonacci():
# create an async queue
queue = asyncio.Queue()
# put the initial pair of numbers in the queue
await queue.put((0, 1))
while True:
# get the next pair of numbers from the queue
a, b = await queue.get()
# yield the first number
yield a
# put the next pair of numbers in the queue
await queue.put((b, a + b))
async def main():
# iterate over the async generator
async for i in fibonacci():
# stop when the number is greater than 100
if i > 100:
break
# mimic some delay
await asyncio.sleep(1)
# print the Fibonacci number
print(i)
# run the main coroutine
asyncio.run(main())
The code will generate Fibonacci numbers up to 89. Each number will show up after a one-second delay:
0
1
1
2
3
5
8
13
21
34
55
89
You can find more details about async queues in this article: Python asyncio.Queue class.
Async generator producer-consumer pattern with queue and sentinel (advanced)
Note: The word “sentinel” here means a guard or a watchman. It is used to describe a value that acts as a signal or a marker for the end of a data stream or a loop. A sentinel value is usually chosen to be different from any valid data value so that it can be easily recognized and handled.
This example demonstrates the way to use an async generator to implement a producer-consumer pattern, where one coroutine produces data and another coroutine consumes it. The producer coroutine uses an async queue to store the data, and the consumer coroutine uses an async for loop to get the data from the queue. The producer coroutine also uses a sentinel value to signal the end of the data stream:
# SlingAcademy.com
# This code uses Python 3.11.4
import asyncio
import random
# set a random seed for reproducibility
random.seed(2023)
async def produce(queue):
for i in range(10):
# simulate some work by sleeping for a random amount of time
await asyncio.sleep(random.random())
# generate a random item
item = random.randint(1, 10)
print(f"Produced {item}")
# put the item in the queue
await queue.put(item)
# put None in the queue to indicate the end of the stream
await queue.put(None)
async def consume(queue):
# create an async generator from the queue
async def get_items():
while True:
# get an item from the queue
item = await queue.get()
# if the item is None, break the loop
if item is None:
break
# yield the item
yield item
# use an async for loop to iterate over the generator
async for item in get_items():
# simulate some work
await asyncio.sleep(random.random())
print(f"Consumed {item}")
async def main():
# create a bounded queue with a capacity of 5
queue = asyncio.Queue(maxsize=5)
# run the producer and consumer coroutines concurrently
await asyncio.gather(produce(queue), consume(queue))
asyncio.run(main())
Output (you’ll notice a short delay before each print):
Produced 8
Consumed 8
Produced 10
Consumed 10
Produced 2
Produced 9
Consumed 2
Consumed 9
Produced 4
Produced 1
Consumed 4
Consumed 1
Produced 3
Produced 4
Consumed 3
Produced 8
Consumed 4
Produced 2
Consumed 8
Final words
We’ve examined some practical examples of using async generators, which can produce a stream of values asynchronously. You can make use of them to improve the performance and responsiveness of your programs. This tutorial ends here. Happy coding & have fun with modern Python programming!