Call for Papers Call for Papers

at the Sixteenth International Conference on Parallel Problem Solving from Nature (PPSN XVI)

September 5-9, 2020 in Leiden, The Netherlands


The Good Benchmarking Practices for Evolutionary Computation Workshop (BENCHMARK@PPSN), a part of the Sixteenth International Conference on Parallel Problem Solving from Nature (PPSN XVI), is cordially inviting the submission of contributions. Here you can download the BB-DOB@PPSN Workshop Call for Papers (CfP) in PDF format and here as plain text file.

PPSN 2020 Conference Logo

In the era of explainable and interpretable AI, it is increasingly necessary to develop a deep understanding of how algorithms work and how new algorithms compare to existing ones, both in terms of strengths and weaknesses. For this reason, benchmarking plays a vital role for understanding algorithms’ behavior. Even though benchmarking is a highly-researched topic within the evolutionary computation community, there are still a number of open questions and challenges that should be explored:

  1. most commonly-used benchmarks are too small and cover only a part of the problem space,
  2. benchmarks lack the complexity of real-world problems, making it difficult to transfer the learned knowledge to work in practice,
  3. we need to develop proper statistical analysis techniques that can be applied depending on the nature of the data, and
  4. we need to develop user-friendly, openly accessible benchmarking software.

This enables a culture of sharing resources to ensure reproducibility, and which helps to avoid common pitfalls in benchmarking optimization techniques. As such, we need to establish new standards for benchmarking in evolutionary computation research so we can objectively compare novel algorithms and fully demonstrate where they excel and where they can be improved.

The topics of interest for this workshop include, but are not limited to:

  • performance measures for comparing algorithms behavior,
  • novel statistical approaches for analyzing empirical data,
  • the selection of meaningful benchmark problems,
  • landscape analysis,
  • data mining approaches for understanding algorithm behavior,
  • transfer learning from benchmark experiences to real-world problems, and
  • benchmarking tools for executing experiments and analysis of experimental results.

Together with our colleagues Markus Wagner, Jörg Lässig, Bin Li, and Xingyi Zhang, we are organizing a Special Issue on "Benchmarking of Computational Intelligence Algorithms" in the Applied Soft Computing (ASOC). We received quite a lot of very cool submissions until the deadline in April 2019. This is a virtual special issue, which means that the papers will immediately published when they have passed the editorial process and may appear in different physical issues and volumes of the journal. This has now happened for the first few of the submissions we have received. As a result, the journal website for our special issue is now online, too, at We still have quite a few very interesting papers in our queue, which will appear step by step on that page as well.

Second Institute Workshop on Applied Optimization


We are happy to announce that the Second Institute Workshop on Applied Optimization will be held in form of the Hefei University Operations Research and Optimization Seminar [合肥学院运筹与优化研讨会] in the conference room of the Institute of Applied Optimization on November 5, 2019. It follows the First Institute Workshop on Applied Optimization, which was still held 2018 in the meeting room near our old offices. Talks will be given by:

  • Prof. Dr. Rolf H. MÖHRING, Hefei University [合肥学院] and Technische Universtiät Berlin [柏林工业大学] ,
  • Prof. Dachuan XU [徐大川教授], Beijing University of Technology [北京工业大学] ,
  • Prof. Xiaoyan ZHANG [张晓岩教授], Nanjing Normal University [南京师范大学] ,
  • Prof. Longkun GUO [郭龙坤教授], Fuzhou University [福州大学] ,
  • Dr. Yong ZHANG, [张涌研究员], Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences [中科院深圳先进技术研究院] , and
  • Dr. Zijun WU [吴自军博士], Hefei University [合肥学院] .

我校人工智能与大数据学院将于2019年11月5日上午在中德应用优化研究所会议室召开第一次“合肥学院运筹与优化研讨会”。届时, Rolf MÖHRING教授(合肥学院,柏林工业大学)、徐大川教授(北京工业大学)、张晓岩教授(南京师范大学)、郭龙坤教授(福州大学)、张涌研究员(中科院深圳先进技术研究院)以及吴自军博士(合肥学院)等国内外运筹与优化领域专家学者将作学术报告,欢迎我校师生踊跃参加!

From October 20 to 25, 2019, our Profs. Möhring and Weise took part in the Dagstuhl Seminar 19431 with the topic Theory of Randomized Optimization Heuristics, organized by Dr. Carola Doerr (Sorbonne University, Paris, France), Carlos M. Fonseca (University of Coimbra, Portugal), Tobias Friedrich, (Hasso-Plattner-Institut, Potsdam, Germany), and Xin Yao (Southern University of Science and Technology, China). Dagstuhl Seminars are invitation-based meetings taking place in Schloss Dagstuhl, a castle located slightly remotely near the city of Wadern, Germany. The remote location is intentional: Here, researchers can meet for, say, five days, to have in-depth discussions without distractions. This is what Dagstuhl Seminars are, and we were very happy and felt honoured to be invited to attend the seminar on theory on randomized optimization heuristics. Not only did the seminar give us the chance to catch up with old friends and plan future collaborations, there we also very interesting and fruitful research discussions. I (Thomas Weise) was particularly happy to see that the topic of benchmarking and algorithm configuration also becomes more and more interesting for theoreticians. I also learned a lot about the progress that theory on optimization has made the past few years. In summary, the seminar was a very very nice event which indeed brought us one or two steps forward in our research. Many thanks to the organizers for doing such a great job!

P.S. This was a very highly productive meeting. Here, the foundations for two works were laid, as shown below. The first work is as a joint efforts of a larger group of international researchers and the second one is by our IAO team:

  • Thomas Bartz-Beielstein, Carola Doerr, Jakob Bossek, Sowmya Chandrasekaran, Tome Eftimov, Andreas Fischbach, Pascal Kerschke, Manuel López-Ibáñez, Katherine M. Malan, Jason H. Moore, Boris Naujoks, Patryk Orzechowski, Vanessa Volz, Markus Wagner, Thomas Weise. Benchmarking in Optimization: Best Practice and Open Issues. Technical report available at arXiv:2007.03488v1 [cs.NE] 7 Jul 2020.
    pdf / pdf@arxiv
  • Thomas Weise, Zhize Wu, Xinlu Li, and Yan Chen. Frequency Fitness Assignment: Making Optimization Algorithms Invariant under Bijective Transformations of the Objective Function Value. Submitted to IEEE Transactions on Evolutionary Computation in January 2020. Preprint available at arXiv:2001.01416v3 [cs.NE] 17 Jun 2020.
    pdf / paper@arxiv / experimental results and source code data set @ doi:10.5281/zenodo.3899474

Today, I attended the Celebration of the 70th Anniversary of the People's Republic of China for People from all Walks of Life [壮丽70年奋斗新时代:合肥市各界人士迎国庆茶话会]. The city hall prepared a wonderful celebration in which people from entirely different careers could participate, ranging from monks and religious leaders over army officers to businessmen and researchers. Accompanied by an exhibition of calligraphy and started with ceremonial speeches, a very nice program of excellent singers and dancers was presented. Indeed, this was a very nice celebration of this important anniversary of China.

feed-image rss feed-image atom