Tensors are a central feature in TensorFlow, but when it comes to deep learning models, you often need mutable storage to handle weights that change over time through training. In TensorFlow, this mutable storage is managed using the Variable objects. Variables persist across multiple executions of a graph, which is essential for implementing and training machine learning models efficiently.
This article aims to introduce you to creating and updating TensorFlow `Variable` objects, which are indispensable when working with TensorFlow. We will provide step-by-step instructions along with code examples that highlight the practical usage of Variables.
Creating TensorFlow Variables
A TensorFlow Variable can be created using the tf.Variable constructor. When one initialises a variable, they provide the initial value; afterwards, it can be modified through operations. Here is a simple example of creating a TensorFlow Variable:
import tensorflow as tf
# Initialize variable with a scalar value
my_var = tf.Variable(0.0)
print(my_var)
In this code snippet, a variable named my_var is initialized with the scalar value 0.0. You can print the content of the variable to see its initial state.
You can also initialize a Variable with a tensor value. For instance, creating a tensor of a specific shape with a random uniform distribution:
# Initialize variable with a tensor
tensor_var = tf.Variable(tf.random.uniform([2, 2], minval=0, maxval=1))
print(tensor_var)
Updating TensorFlow Variables
Once created, the value of a Variable can be changed in place. Unlike tf.Tensor, a Variable object is mutable, so you can perform in-place updates. This feature is useful, particularly for model weights during gradient descent updates.
You can update a TensorFlow Variable using operations like assign, assign_add, or assign_sub. Here's how you can do it:
# Assigning a new value
my_var.assign(5.0)
print("Updated variable to 5.0:", my_var)
# Add a value to the current variable
my_var.assign_add(3.0)
print("Added 3.0 to variable:", my_var)
# Subtract a value from the current variable
my_var.assign_sub(2.0)
print("Subtracted 2.0 from variable:", my_var)
Each of these methods modifies the variable in-place. The ability to modify variables is key to the backpropagation step in training a neural network.
Working with Trainable Variables
Variables are often utilized as parameters of machine learning models, typically in the form of weights or biases. By default, all tf.Variable instances are trainable; this means TensorFlow will automatically calculate the gradients during the backpropagation step.
# Define a trainable variable
w = tf.Variable(tf.random.normal([3, 3]), trainable=True)
# Check if the variable is trainable
print("Is `w` trainable?:", w.trainable)
If you wish to create a non-trainable variable, you can explicitly set the trainable flag to False:
# Define a non-trainable variable
b = tf.Variable(tf.ones([3]), trainable=False)
print("Is `b` trainable?:", b.trainable)
Conclusion
Understanding how to create and update variables in TensorFlow is crucial for their effective use in machine learning models. Variables form the backbone of any trainable model dealing with operations and state changes over successive training iterations.
By mastering TensorFlow Variable objects, developers can better leverage the framework to create sophisticated, stateful machine learning models and gain finer control over their training processes.
