Python Generators: Tutorial & Examples

Overview
Syntax
Examples

Overview

Generators in Python are a way of creating iterators that can produce a sequence of values lazily without storing them all in memory at once. Their purpose is to simplify the creation of iterators and to enable efficient processing of large or infinite data streams.

Syntax

The keywords used in Python generators are:

def: Defines a generator function.
yield: Produces a value from the generator and pauses the function execution.
next(): Resumes the function execution and returns the next value from the generator.
for: Iterates over the generator object.

Let’s see how we can use them in specific cases.

Creating a generator function

To create a generator function, use the def keyword followed by the function name and parameters. Inside the function body, use the yield keyword to produce a value from the generator. The yield keyword also pauses the function execution until the next value is requested. The syntax is as follows:

def generator_name(arg):
    # statements
    yield something

Creating a generator object

To create a generator object, call the generator function like any other function:

generator = generator_name(arg)

Iterating over a generator object

To iterate over the generator object, use a for loop or the next() function:

for value in generator:
    # do something with value

# or

value = next(generator)
# do something with value

Creating a generator expression

To create a generator expression, use parentheses and a similar syntax to list comprehensions, but without brackets:

generator = (expression for item in iterable)

Examples

Producing a sequence of numbers by using a generator

This basic example shows how to define a generator function using the yield keyword, as well as iterate over the generator object using a for loop:

def my_generator(n):
    # initialize counter
    value = 0
    # loop until counter is less than n
    while value < n:
        # produce the current value of the counter
        yield value
        # increment the counter
        value += 1


# iterate over the generator object produced by my_generator
for value in my_generator(3):
    # print each value produced by generator
    print(value)

Output:

0
1
2

Using a generator function to read a large file line by line

This example shows how to use a generator function to process a large file without loading it all into memory at once. Here’s the sample CSV file used in the example:

https://api.slingacademy.com/v1/sample-data/files/customers.csv

The code:

def csv_reader(file_name):
    # open the file and create a file object
    file = open(file_name)
    # loop over each line in the file object
    for row in file:
        # yield the row as a list of values
        yield row.split(",")

# create the generator object from the csv_reader function
csv_data = csv_reader("customers.csv")
# iterate over the generator object and print each row
for row in csv_data:
    print(row)

You can find more details about the sample data on this page: Customers Sample Data (CSV, JSON, XML, and XLSX).

Using a generator function to generate a sequence of Fibonacci numbers

This advanced example shows how to use a generator function to create an infinite sequence of values and how to use the next() function to get the next value from the generator:

def fibonacci():
    # initialize the first two numbers of the sequence
    a = 0
    b = 1
    # loop indefinitely
    while True:
        # yield the current value of a
        yield a
        # update a and b to the next two numbers in the sequence
        a, b = b, a + b

# create the generator object from the fibonacci function
fib = fibonacci()
# iterate over the generator object and print the first 10 Fibonacci numbers
for i in range(10):
    print(next(fib))

Output: