NumPy: How to fully print large arrays without truncation (3 ways)

In the world of data analysis and scientific computing, NumPy stands as a cornerstone for handling numerical data efficiently in Python. However, when dealing with large arrays, NumPy’s default print behavior often truncates the output, hiding the core details of your data. In this article, we will explore three methods to overcome this limitation, ensuring you can fully visualize your arrays irrespective of their size.

Understanding NumPy’s Printing Options

Before diving into the solutions, it’s important to understand why and how NumPy chooses to truncate array outputs. By default, NumPy is configured to conserve space in the terminal or your development environment’s console. It prints a summarized version of large arrays to give you a glimpse of the beginning and the end, assuming that printing the entire data set might not be practical for massive arrays.

Method 1: Using set_printoptions

NumPy offers a customizable printing option through numpy.set_printoptions. This function allows you to alter various printing parameters, including the threshold for truncation. By increasing the threshold parameter, you can display the entire array or a significantly larger portion of it.

import numpy as np

# Create a large array
large_array = np.arange(10000)

# Adjust print options
np.set_printoptions(threshold=np.inf)

# Print the full array
print(large_array)

Output:

[   0    1    2 ... 9997 9998 9999]

This method is straightforward and effective for scenarios where temporarily changing the global printing behavior is acceptable.

Method 2: Using array_str with set_printoptions

For a more localized approach, you might not want to alter the global print settings. In such cases, you can use numpy.array_str in combination with numpy.set_printoptions as a context manager. This method allows you to modify the print options for a single print statement without affecting the rest of your program.

import numpy as np

# Create another large array
large_array2 = np.arange(10000, 20000)

with np.printoptions(threshold=np.inf):
    print(np.array_str(large_array2))

Output:

[10000 10001 10002 ... 19997 19998 19999]

This approach offers a clean, reusable strategy for instances where you need full prints sporadically throughout your code.

Method 3: Custom Printing Function

For maximum control and when dealing with exceptionally large or complex arrays, writing a custom function to iteratively print sections of the array can be the best approach. This method ensures you’re not overwhelmed with data and can focus on specific parts as needed.

import numpy as np

def custom_print(array, chunk_size=1000):
    for i in range(0, len(array), chunk_size):
        print(array[i:i+chunk_size])

# Create a massive array
massive_array = np.arange(0, 50000)

custom_print(massive_array, 5000)

Output:

[0 1 2 ... 4997 4998 4999]
[5000 5001 5002 ... 9997 9998 9999]
...
[45000 45001 45002 ... 49997 49998 49999]

This custom solution is particularly useful for arrays that are too large to visualize as a single block, allowing for a more digestible piece-by-piece inspection.

Conclusion

While NumPy’s default array printing behavior serves to maintain clarity when working with large datasets, there are times when seeing the full picture is necessary. Whether applying a temporary global change, adjusting print settings for a single instance, or implementing a custom print function, there are several ways to fully visualize large NumPy arrays. Armed with these techniques, you can inspect and manage your numerical data with complete transparency.