论文标题
培训有关信息安全任务的变压器：恶意URL预测的案例研究
Training Transformers for Information Security Tasks: A Case Study on Malicious URL Prediction
论文作者
论文摘要
用于信息安全性的机器学习（ML）（INFOSEC）采用了不同的数据类型和格式，这些数据类型和格式在对原始数据的优化/培训期间需要不同的处理。在本文中，我们基于从头开始训练的变压器体系结构实现了恶意/良性URL预测指标。我们表明，与传统的自然语言处理（NLP）变压器相比，该模型需要一种不同的训练方法才能正常工作。具体而言，我们表明1）预先进行自动退缩任务的大量未标记的URL数据进行预培训不会容易转移到恶意/良性预测中，而是2）使用辅助自动回火损失可以在从划痕训练时提高性能。我们引入了一种混合客观优化的方法，该方法可以动态地平衡两个损失项的贡献，以使它们俩都占主导地位。我们表明，这种方法的性能与几个表现最佳基准分类器相当。
Machine Learning (ML) for information security (InfoSec) utilizes distinct data types and formats which require different treatments during optimization/training on raw data. In this paper, we implement a malicious/benign URL predictor based on a transformer architecture that is trained from scratch. We show that in contrast to conventional natural language processing (NLP) transformers, this model requires a different training approach to work well. Specifically, we show that 1) pre-training on a massive corpus of unlabeled URL data for an auto-regressive task does not readily transfer to malicious/benign prediction but 2) that using an auxiliary auto-regressive loss improves performance when training from scratch. We introduce a method for mixed objective optimization, which dynamically balances contributions from both loss terms so that neither one of them dominates. We show that this method yields performance comparable to that of several top-performing benchmark classifiers.
