论文标题
开源可重复性的警笛声
A Siren Song of Open Source Reproducibility
论文作者
论文摘要
随着可重复性成为更大的关注点，会议在很大程度上融合了一种策略，要求审阅者指示是否附加了代码。这是基于假定理想采取行动的更大趋势的一部分，而无需研究这些行动是否会产生预期的结果。我们的论点是，如果我们想改善可重复的研究状态，将重点放在复制代码上。这种重点可能是有害的 - 我们不应强制提交代码。缺乏证据表明会议采取有效行动来鼓励和奖励可重复性。我们认为，现场必须采取更多行动来推进可重复的机器学习研究。
As reproducibility becomes a greater concern, conferences have largely converged to a strategy of asking reviewers to indicate whether code was attached to a submission. This is part of a larger trend of taking action based on assumed ideals, without studying if those actions will yield the desired outcome. Our argument is that this focus on code for replication is misguided if we want to improve the state of reproducible research. This focus can be harmful -- we should not force code to be submitted. There is a lack of evidence for effective actions taken by conferences to encourage and reward reproducibility. We argue that venues must take more action to advance reproducible machine learning research today.
