Implementing Multivariate Forecasting Using GRUs in PyTorch

Multivariate time series forecasting is an essential task in various domains such as finance, economics, and weather prediction. In this article, we will explore how to implement a multivariate forecasting model using Gated Recurrent Units (GRUs) in PyTorch. GRUs are a type of recurrent neural network architecture that is effective for sequence prediction tasks.

Understanding GRUs

Gated Recurrent Units (GRUs) are similar to Long Short-Term Memory (LSTM) networks in that they aim to solve the vanishing gradient problem in traditional RNNs. GRUs have gating units that modulate the flow of information within the unit, but they have a simpler architecture with fewer parameters compared to LSTMs. This often leads to faster training and less computational cost while still preserving important sequence learning capabilities.

Getting Started with PyTorch

Before we dive into the code, make sure you have PyTorch installed. If not, you can install it using pip:

pip install torch torchvision

Dataset Preparation

First, we need a multivariate time series dataset. For simplicity, let's consider a dataset with two features: temperature and humidity recorded over time. We assume this data is stored in a CSV file named timeseries.csv.

import pandas as pd

data = pd.read_csv('timeseries.csv')
print(data.head())

Suppose our dataset looks like this:

• timestamp
• temperature
• humidity

Data Preprocessing

Next, we preprocess the data by normalizing it and turning it into sequences that can be fed into our GRU model.

from sklearn.preprocessing import MinMaxScaler
import numpy as np

scaler = MinMaxScaler()
data_scaled = scaler.fit_transform(data[['temperature', 'humidity']])

# Function to create sequences
 def create_sequences(data, sequence_length):
    sequences = []
    for i in range(len(data) - sequence_length):
        sequence = data[i:i+sequence_length]
        label = data[i+sequence_length]
        sequences.append((sequence, label))
    return sequences

sequence_length = 10
sequences = create_sequences(data_scaled, sequence_length)

Defining the GRU Model

Let's define a simple GRU model using PyTorch.

import torch
import torch.nn as nn

class GRUModel(nn.Module):
    def __init__(self, input_dim, hidden_dim, layer_dim, output_dim):
        super(GRUModel, self).__init__()
        self.hidden_dim = hidden_dim
        self.layer_dim = layer_dim
        
        self.gru = nn.GRU(input_dim, hidden_dim, layer_dim, batch_first=True)
        self.fc = nn.Linear(hidden_dim, output_dim)

    def forward(self, x):
        # Initialize hidden state with zeros
        h0 = torch.zeros(self.layer_dim, x.size(0), self.hidden_dim).requires_grad_()

        # Forward propagation by passing in the input and hidden state into the model
        out, hn = self.gru(x, h0.detach())

        # Reshaping the outputs in the shape of (batch_size, seq_length, hidden_dim)
        out = self.fc(out[:, -1, :]) 
        return out

Training the Model

Before training, we split the sequences into features and labels, and then convert them into a PyTorch dataset.

from torch.utils.data import DataLoader, TensorDataset

# Convert sequences to tensors
features = np.array([sequence for sequence, label in sequences])
labels = np.array([label for sequence, label in sequences])

tensor_features = torch.Tensor(features)
tensor_labels = torch.Tensor(labels)

dataset = TensorDataset(tensor_features, tensor_labels)
dataloader = DataLoader(dataset, batch_size=16, shuffle=True)

Now, we define loss and optimizer and start training the GRU model:

model = GRUModel(input_dim=2, hidden_dim=64, layer_dim=2, output_dim=2)
criterion = nn.MSELoss()
optimizer = torch.optim.Adam(model.parameters(), lr=0.01)

def train_model(model, dataloader, criterion, optimizer, epochs=100):
    for epoch in range(epochs):
        for i, (features, labels) in enumerate(dataloader):
            outputs = model(features)
            loss = criterion(outputs, labels)

            optimizer.zero_grad()
            loss.backward()
            optimizer.step()

            if (i+1) % 10 == 0:
                print(f'Epoch [{epoch+1}/{epochs}], Step [{i+1}/{len(dataloader)}], Loss: {loss.item():.4f}')

train_model(model, dataloader, criterion, optimizer)

Conclusion

We have implemented a basic GRU model for multivariate time series forecasting in PyTorch. This setup can be extended by incorporating more features, adding more GRU layers, or altering the hyperparameters to suit your specific dataset and needs. By grasping these core concepts, you'll be better equipped to handle more complex time series forecasting scenarios using deep learning.