论文标题
不规则间隔事件及其参与者的变压器嵌入
Transformer Embeddings of Irregularly Spaced Events and Their Participants
论文作者
论文摘要
神经鹰的过程（Mei＆Eisner，2017年）是离散事件不规则间隔序列的生成模型。为了处理具有许多事件类型的复杂域，Mei等人。 （2020a）进一步考虑一个设置，在该设置中，序列中的每个事件都会更新事实的演绎数据库（通过特定于域的模式匹配规则）；然后将未来事件在数据库内容上进行调节。他们展示了如何将这种符号系统转换为神经符号连续的生成模型，在该模型中，每个数据库事实和可能的事件都具有从其符号出处得出的时变嵌入。 在本文中，我们修改了这两种模型，用基于较平坦的基于注意力的架构代替了其经常性的基于LSTM的体系结构（Vaswani等，2017），这更简单，更平行。这似乎并没有损害我们的准确性，这与原始模型以及（如果适用）先前基于注意力的方法相当或更好（Zuo等，2020; Zhang等，2020a）。
The neural Hawkes process (Mei & Eisner, 2017) is a generative model of irregularly spaced sequences of discrete events. To handle complex domains with many event types, Mei et al. (2020a) further consider a setting in which each event in the sequence updates a deductive database of facts (via domain-specific pattern-matching rules); future events are then conditioned on the database contents. They show how to convert such a symbolic system into a neuro-symbolic continuous-time generative model, in which each database fact and the possible event has a time-varying embedding that is derived from its symbolic provenance. In this paper, we modify both models, replacing their recurrent LSTM-based architectures with flatter attention-based architectures (Vaswani et al., 2017), which are simpler and more parallelizable. This does not appear to hurt our accuracy, which is comparable to or better than that of the original models as well as (where applicable) previous attention-based methods (Zuo et al., 2020; Zhang et al., 2020a).
