TempoKGAT: Improving temporal graph analysis with time-decreasing weights and selective neighbor aggregation

GNNs have excelled in analyzing structured data, but they face challenges with dynamic and temporal graphs. Traditional forecasting, often used in fields such as economics and biology, relied on statistical models for time-series data. Deep learning, particularly GNNs, shifted the focus to non-Euclidean data such as social and biological networks. However, the application of GNNs to dynamic graphs, where relationships are constantly evolving, still needs improvement. Although graph attention networks (GATs) partially address these challenges, further advances are needed, particularly in the use of edge attributes.

Researchers from Sorbonne University and TotalEnergies have developed a graph attention network called TempoKGAT, which integrates time-decaying weights and a selective neighbor aggregation mechanism to uncover latent patterns in spatiotemporal graph data. This approach involves selecting top-k neighbors based on edge weights, which improves the representation of evolving graph features. TempoKGAT was tested on datasets from the traffic, energy, and healthcare sectors, and consistently outperformed existing state-of-the-art methods on multiple metrics. These findings demonstrate TempoKGAT’s ability to improve prediction accuracy and provide deeper insights into temporal graph analysis.

Forecasting has evolved from traditional statistical methods to advanced machine learning, increasingly using graph-based approaches to capture spatial dependencies. This progression has led from CNNs to GCNs and Graph Attention Networks (GATs). While models such as Diffusion Convolutional Recurrent Neural Networks (DCRNNs) and Temporal Graph Convolutional Networks (TGCNs) incorporate temporal dynamics, they often overlook the benefits of weighted edges. Existing advances in edge modeling, particularly for static and multi-relational graphs, have not yet been fully adapted to temporal contexts. TempoKGAT aims to address this gap by improving the utilization of edge weights in temporal graph forecasting, thereby improving prediction accuracy and analysis of complex temporal data.

The TempoKGAT model improves temporal graph analysis by refining node features through time-decomposed weights and selective neighbor aggregation. From node features, temporal decomposition is applied to prioritize recent data, ensuring that dynamic graphs are accurately represented. The model then selects the k most significant neighbors based on edge weights, focusing on the most relevant interactions. An attention mechanism computes attention coefficients, normalized and used to aggregate neighbor features, weighted by attention scores and edge strengths. This approach dynamically integrates temporal and spatial information, improving prediction accuracy and capturing evolving graph patterns.

TempoKGAT demonstrates exceptional performance on multiple datasets by effectively integrating temporal and spatial dynamics in graph data. The model significantly improved over the original GAT, with notable improvements in metrics such as MAE, MSE, and RMSE, particularly on datasets such as PedalMe, ChickenPox, and England Covid. TempoKGAT’s adaptability is highlighted by its optimal neighborhood size parameter (k), which improves prediction accuracy. The consistent success, especially at k = 1, underlines the model’s ability to capture critical features of immediate neighbors, making it a robust and versatile tool for graph-based predictive analytics across different network complexities.

In conclusion, TempoKGAT is a graph attention network designed for temporal graph analysis, which excels at integrating time-decaying weights and selective neighbor aggregation. The model outperforms traditional methods in predicting outcomes on datasets such as PedalMe, ChickenPox, and England Covid, showing significant improvements in RMSE, MAE, and MSE metrics. However, the computational complexity increases with larger neighborhood sizes. Future research will optimize computational efficiency, explore multi-headed attention, and scale the model for larger graphs, paving the way for broader applications in graph-based predictive analytics.

Take a look at the Paper. All credit for this research goes to the researchers of this project. Also, don't forget to follow us on twitter.com/Marktechpost”>twitter and LinkedInJoin our Telegram Channel.

If you like our work, you will love our fact sheet..

Don't forget to join our SubReddit of over 50,000 ml

Sana Hassan, a Consulting Intern at Marktechpost and a dual degree student at IIT Madras, is passionate about applying technology and ai to address real-world challenges. With a keen interest in solving practical problems, she brings a fresh perspective to the intersection of ai and real-life solutions.