Using numpy.fromiter() function (5 examples)

Introduction

Numpy is a cornerstone library in the Python ecosystem for numerical computations. Among its versatile set of functions, numpy.fromiter() stands out for its efficiency in converting iterable objects into Numpy arrays. This tutorial aims to guide you through the usage of numpy.fromiter(), showcasing its utility through five progressively complex examples.

The Fundamentals of numpy.fromiter()

The numpy.fromiter() function is used to create a new 1-dimensional array from an iterable object. The syntax is fairly simple:

numpy.fromiter(iterable, dtype, count=-1)

• iterable: the iterable object to be converted.
• dtype: the data type of the resulting array. It’s mandatory.
• count: the number of items to read from the iterable. The default, -1, means all items will be read.

Now, let’s dive into the examples to see numpy.fromiter() in action.

Example 1: Basic Usage

The simplest use case is converting a basic iterator, like a range object, into a Numpy array:

import numpy as np

iterable = range(5)
arr = np.fromiter(iterable, dtype='int')
print(arr)

Output:

[0 1 2 3 4]

This example demonstrates the function’s basic utility, efficiently transforming a range object into a Numpy array of integers.

Example 2: Working with Generators

Generators offer a memory-efficient way to work with large datasets. Here’s how you can use numpy.fromiter() with a generator expression:

import numpy as np

iterable = (x**2 for x in range(5))
arr = np.fromiter(iterable, dtype='int')
print(arr)

Output:

[0 1 4 9 16]

This highlights numpy.fromiter()‘s ability to handle generators, converting a sequence of squared numbers into an array efficiently.

Example 3: Specifying the Count

If you’re working with endless generators, specifying the count parameter helps in limiting the array size:

import numpy as np
from itertools import count

iterable = (x for x in count(0, 2)) # an endless generator of even numbers
arr = np.fromiter(iterable, dtype='int', count=5)
print(arr)

Output:

[0 2 4 6 8]

This example is particularly useful in scenarios where the iterable’s size isn’t naturally bounded.

Example 4: Using with Files

When dealing with file data, numpy.fromiter() can be extremely useful. For instance, reading in numerical values from a file:

import numpy as np

with open('data.txt', 'r') as f:
    iterable = (float(line.strip()) for line in f)
    arr = np.fromiter(iterable, dtype='float')
    print(arr)

Here, data.txt contains one number per line. This approach is memory efficient for large data files.

Example 5: Complex Data Processing

You can combine numpy.fromiter() with more complex data processing steps. For instance, filtering and converting data in a single step:

import numpy as np

iterable = (x for x in range(10) if x % 2 == 0)
arr = np.fromiter(iterable, dtype='int')
print(arr)

Output:

[0 2 4 6 8]

This example illustrates the flexibility of numpy.fromiter() when used in conjunction with list comprehensions or generator expressions for filtering data.

Conclusion

The numpy.fromiter() function is a powerful, versatile tool in the Numpy library, enabling efficient array creation from iterable objects. These examples, ranging from simple to advanced scenarios, highlight this function’s utility in data processing workflows.