2020-IJCAI A New Attention Mechanism to Classify Multivariate Time Series

Motivation

First, the long-range dependencies of the time-series sequences are not well captured. Second, the interactions of multiple variables are generally not represented in features. To address these aforementioned issues, the paper propose a novel Cross Attention Stabilized Fully Convolutional Neural Network (CA-SFCN) to classify MTS data.

Method

The main contributions are summarized as follow:

A “novel” attention mechanism was proposed which actually is just self-attention beyond to transformers, while applied to X and Y axis over feature maps. Consequently, attention across time axis and channel axis was modeled, so the method in this paper was called cross-attention.

The overall pipeline can be seen below:

Results