论文标题
时间序列预测的极端长期记忆
Extreme-Long-short Term Memory for Time-series Prediction
论文作者
论文摘要
长期记忆（LSTM）的出现解决了消失的梯度和爆炸梯度在传统的复发神经网络（RNN）中的问题。 LSTM作为一种新型的RNN，已被广泛用于各个领域，例如文本预测，风速预测，通过EEG信号进行抑郁预测等。结果表明，提高LSTM的效率可以帮助提高其他应用领域的效率。 在本文中，我们提出了一种高级LSTM算法，即极端的长期记忆（E-LSTM），该算法将极端学习机器（ELM）的反基质部分添加为新的“门”，为LSTM的结构。此“门”预处理一部分数据，并涉及LSTM的单元格更新中的处理数据，以获取更准确的数据，从而减少了整体训练时间。 在这项研究中，E-LSTM模型用于文本预测任务。实验结果表明，E-LSTM有时需要更长的时间才能执行单个训练回合，但是在对小数据集进行测试时，新的E-LSTM只需要2个时代才能获得第七届时代传统LSTM的结果。因此，E-LSTM保留了传统LSTM的高精度，同时也提高了LSTM的训练速度和整体效率。
The emergence of Long Short-Term Memory (LSTM) solves the problems of vanishing gradient and exploding gradient in traditional Recurrent Neural Networks (RNN). LSTM, as a new type of RNN, has been widely used in various fields, such as text prediction, Wind Speed Forecast, depression prediction by EEG signals, etc. The results show that improving the efficiency of LSTM can help to improve the efficiency in other application areas. In this paper, we proposed an advanced LSTM algorithm, the Extreme Long Short-Term Memory (E-LSTM), which adds the inverse matrix part of Extreme Learning Machine (ELM) as a new "gate" into the structure of LSTM. This "gate" preprocess a portion of the data and involves the processed data in the cell update of the LSTM to obtain more accurate data with fewer training rounds, thus reducing the overall training time. In this research, the E-LSTM model is used for the text prediction task. Experimental results showed that the E-LSTM sometimes takes longer to perform a single training round, but when tested on a small data set, the new E-LSTM requires only 2 epochs to obtain the results of the 7th epoch traditional LSTM. Therefore, the E-LSTM retains the high accuracy of the traditional LSTM, whilst also improving the training speed and the overall efficiency of the LSTM.
