论文标题
优化基准测试：最佳实践和开放问题
Benchmarking in Optimization: Best Practice and Open Issues
论文作者
论文摘要
这项调查汇编了来自不同背景和世界各地不同机构的十多名研究人员的想法和建议。在基准测试中促进最佳实践是其主要目标。本文讨论了基准测试中的八个基本主题：明确指定的目标，明确的问题，合适的算法，适当的绩效指标，周到的分析，有效而有效的设计，可理解的演示文稿和保证的可重复性。最终目标是提供可能对作者和审阅者有用的良好指南（规则）。由于优化中的基准测试是一个积极且不断发展的研究领域，因此该手稿旨在通过定期更新随着时间的流逝而共同发展。
This survey compiles ideas and recommendations from more than a dozen researchers with different backgrounds and from different institutes around the world. Promoting best practice in benchmarking is its main goal. The article discusses eight essential topics in benchmarking: clearly stated goals, well-specified problems, suitable algorithms, adequate performance measures, thoughtful analysis, effective and efficient designs, comprehensible presentations, and guaranteed reproducibility. The final goal is to provide well-accepted guidelines (rules) that might be useful for authors and reviewers. As benchmarking in optimization is an active and evolving field of research this manuscript is meant to co-evolve over time by means of periodic updates.
