Black Box Discrete Optimization Benchmarking (BB-DOB) Workshop The Genetic and Evolutionary Computation Conference (GECCO 2019) July 13-17, 2019, Prague, Czech Republic http://iao.hfuu.edu.cn/bbdob-gecco19 The Black-Box Discrete Optimization Benchmarking (BB-DOB) Workshop, a part of the Genetic and Evolutionary Computation Conference (GECCO) 2019, is cordially inviting the submission of original and unpublished research papers. The Black-Box-Optimization Benchmarking (BBOB) methodology associated to the BBOB-GECCO workshops has become a well-established standard for benchmarking stochastic and deterministic continuous optimization algorithms. The aim of the BB-DOB workshop series is to set up a process that will allow to achieve a similar standard methodology for the benchmarking of black box optimization algorithms in discrete and combinatorial search spaces. The long term aim of our workshop series is to produce, for the domain of discrete optimization: (1) a well-motivated benchmark function testbed, (2) an experimental set-up, (3) generation of data output for post-processing and (4) presentation of the results in graphs and tables. The aims of this GECCO 2019 BB-DOB workshop are to finalize the benchmarking testbed for discrete optimization and to promote a discussion of which performance measures should be used. The benchmark functions should capture the difficulties of combinatorial optimization problems in practice. They also should be comprehensible so that algorithm behaviors can be understood or interpreted according to the performance on a given benchmark problem. The goal is that a desired search behavior can be pictured and algorithm deficiencies can be understood in depth. This understanding will lead to the design of improved algorithms. Ideally, we would like the benchmark functions to be scalable with the problem size and non-trivial in the black box optimization sense (the function may be shifted such that the global optimum may be any point). Achieving this goal would help greatly in bridging the gap between theoreticians and experimentalists. We also wish to investigate which measures should be used to compare algorithm performance, which statistical tests should be run to compare algorithms, and how to deal with unsuccessful runs. This workshop wants to bring together experts on benchmarking of optimization algorithms. It will provide a common forum for discussions and exchange of opinions. Interested participants are encouraged to submit a paper related to black-box optimization benchmarking of discrete optimizers. The topics of interesting especially include papers that - suggest functions to be included in the benchmark and motivate the reasons for inclusion, - suggest benchmark function properties that allow to capture difficulties which occur in real-world applications (e.g., deception, separability, etc.), - suggest which classes of standard combinatorial optimization problems should be included and how to select significant instances, - suggest which classes of toy problems should be included and motivate why, - suggest which performance measures should be used to analyze and compare algorithms and comment/suggestions on related issues, and/or - tackle any other aspect of benchmarking methodology for discrete optimizers such as design of experiments, presentation methods, benchmarking frameworks, etc. - conduct performance comparisons, landscape analysis, discussion of selected benchmark problems and/or provided statistics of IOHprofiler (https://github.com/IOHprofiler), a ready-to-use software for the empirical analysis of iterative optimization heuristics For more information please contact Pietro S. Oliveto at p.oliveto@sheffield.ac.uk. This workshop is organized as part of ImAppNIO Cost Action 15140. 1. Important Dates Paper Submission Opening: 27 February 2019 Paper Submission Deadline: 10 April 2019 (extended) Decisions Due: 17 April 2019 Camera-Ready Material due: 24 April 2019 Author Registration Deadline: 24 April 2019 Conference Presentation: 13-14 July 2019 2. Instructions for Authors Instructions regarding how to submit papers and the paper format are given at https://gecco-2019.sigevo.org/index.html/tiki-index.php?page=Workshops. Some of the things to consider are: - Submitted papers must be ANONYMIZED, i.e., cannot contain any element that may reveal the identity of their authors, in order to facilitate a double-blind review process. - The maximum paper length is eight pages. - The maximum number of words in the abstract is 200. - In the GECCO submission page, select "Workshop Paper" and in the field "Workshop" of the next form, select "Workshop Black Box Discrete Optimization Benchmarking". - At least one author from each accepted paper must register at the conference by April 24, 2019, pay the conference fee, and be present at the conference to give an oral presentation. 3. Chairs - Carola Doerr, Sorbonne University, Paris, France - Pietro S. Oliveto, University of Sheffield, UK - Thomas Weise, Institute of Applied Optimization, Hefei University, Hefei, China - Borys Wróbel, Adam Mickiewicz University, Poland - Aleš Zamuda, University of Maribor, Slovenia 4. International Program Committee - Abhishek Awasthi, University of Applied Sciences Zittau/Görlitz, Görlitz, Germany - Thomas Bäck, Leiden University, Leiden, The Netherlands - Thomas Bartz-Beielstein, Technical University of Cologne, Köln (Cologne), Germany - Andreas Beham, FH Upper Austria, Hagenberg, Austria - Maxim Buzdalov, ITMO University, St. Petersburg, Russia - Arina Buzdalova, ITMO University, St. Petersburg, Russia - Josu Ceberio Uribe, University of the Basque Country, Bilbao, Spain - Francisco Chicano, University of Málaga, Málaga, Spain - Carola Doerr, Sorbonne University, Paris, France - Johann Dréo, THALES Research & Technology, Massy, Île-de-France, France - Tome Eftimov, Biomedical Data Science, Stanford University, Stanford, California, USA - Aniko Ekart, Aston University, Birmingham, UK - Thomas Jansen, Aberystwyth University, UK - Pascal Kerschke, Westfälische Wilhelms-Universität Münster, Münster, Germany - Algirdas Lančinkas, Vilnius University, Lithuania - Johannes Lengler, ETH Zürich, Zürich, Switzerland - Bin Li, University of Science and Technology of China, Hefei, China - Jinlong Li, University of Science and Technology of China, Hefei, China - Xinlu Li, Institute of Applied Optimization, Hefei University, Hefei, China - Arnaud Liefooghe, Université de Lille Sciences et Technologies, Lille, France - Frank Neumann, University of Adelaide, Adelaide, SA, Australia - Miguel Nicolau, University College Dublin, Ireland - Pietro S. Oliveto, University of Sheffield, UK - Karol Opara, Systems Research Institute, Polish Academy of Sciences, Warsaw, Poland - Chao Qian, University of Science and Technology of China, Hefei, China - Mohammad Ali Raayatpanah, Kharazmi University, Tehran, Iran - Roman Senkerik, Tomas Bata University, Zlin, Czech Republic - Dirk Sudholt, University of Sheffield, UK - Heike Trautmann, Westfälische Wilhelms-Universität Münster, Münster, Germany - Markus Ullrich, University of Applied Sciences Zittau/Görlitz, Görlitz, Germany - Markus Wagner, University of Adelaide, Adelaide, SA, Australia - Stefan Wagner, FH Upper Austria, Hagenberg, Austria - Hao Wang, Leiden University, Leiden, The Netherlands - Thomas Weise, Institute of Applied Optimization, Hefei University, Hefei, China - Carsten Witt, Technical University of Denmark, Denmark - Borys Wróbel, Adam Mickiewicz University, Poland - Yuezhong Wu, University of New South Wales (UNSW), Sydney, Australia - Zhize Wu, Institute of Applied Optimization, Hefei University, Hefei, China - Aleš Zamuda, University of Maribor, Slovenia - Christine Zarges, Aberystwyth University, Aberystwyth, Wales, United Kingdom - Xingyi Zhang, Anhui University, Hefei, China 5. Chair Biographies Carola Doerr is a permanent CNRS researcher at Sorbonne University in Paris, France. She studied Mathematics at Kiel University (Germany, Diplom, 2007) and Computer Science at the Max Planck Institute for Informatics and Saarland University (Germany, PhD, 2011). Before joining the CNRS she was a post-doc at Paris Diderot University (Paris 7) and the Max Planck Institute for Informatics. From 2007 to 2009, she worked as a business consultant for McKinsey & Company, where her interest in evolutionary algorithms (EAs) originates from. Her main research activities are in the mathematical analysis of randomized algorithms, with a strong focus on EAs and other black-box optimizers. She has been very active in the design and analysis of black-box complexity models, a theory-guided approach to explore the limitations of heuristic search algorithms. Most recently, she has used knowledge from these studies to prove superiority of dynamic parameter choices in evolutionary computation, a topic that she believes to carry huge unexplored potential for the community. Carola Doerr has received several awards for her work on evolutionary computation, among them the Otto Hahn Medal of the Max Planck Society and four best paper awards at GECCO. She is chairing the program committee of FOGA 2019 and previously chaired the theory tracks of GECCO 2015 and 2017. She is editor of two special issues in Algorithmica and vice chair of the EU-funded COST action 15140 on "Improving Applicability of Nature-Inspired Optimisation by Joining Theory and Practice (ImAppNIO)". Pietro S. Oliveto is a Senior Lecturer and an EPSRC Early Career Fellow at the University of Sheffield,UK. He re-ceived the Laurea degree in computer science from the University of Catania, Italy in 2005 and the PhD degree from the University of Birmingham,UK in 2009. He has been EPSRC PhD+ Fellow (2009-2010) and EPSRC Postdoctoral Fellow (2010-2013) at Birmingham and Vice-Chancellor's Fellow at Sheffield (2013-2016). His main research interest is the performance analysis of bio-inspired computation techniques including evolutionary algorithms, genetic programming, artificial immune systems and hyperheuristics. He has won best paper awards at GECCO 2008, ICARIS 2011 and GECCO 2014. He is part of the Steering Committee of the annual workshop on Theory of Randomized Search Heuristics (ThRaSH), Associate Editor of the IEEE Transactions on Evolution-ary Computation, Chair of the IEEE CIS Technical Committee on Evolutionary Computation, Leader of the ImAppNIO Cost Action Working Group on Benchmarking and member of the EPSRC Peer Review College. Dr. Oliveto has given tutorials on the runtime complexity analysis of EAs regularly at CEC, GECCO, WCCI, SSCI and PPSN since 2012. Thomas Weise obtained the MSc in Computer Science in 2005 from the Chemnitz University of Technology and his PhD from the University of Kassel in 2009. He then joined the University of Science and Technology of China (USTC) as PostDoc and subsequently became Associate Professor at the USTC-Birmingham Joint Research Institute in Intelligent Computation and Its Applications (UBRI) at USTC. In 2016, he joined Hefei University as Full Professor to found the Institute of Applied Optimization at the Faculty of Computer Science and Technology. Prof. Weise has more than seven years of experience as a full time researcher in China, having contributed significantly both to fundamental as well as applied research. He has more than 80 scientific publications in international peer reviewed journals and conferences. His book "Global Optimization Algorithms – Theory and Application" has been cited more than 730 times. He has acted as reviewer, editor, or program committee member at 70 different venues. Borys Wróbel works at the intersection between biology and computer science, and his current research interests include computational properties of biologically-inspired models of computation (artificial gene regulatory networks and spiking neural networks), which involves building artificial life software platforms that use high-performance computing and neuromorphic hardware. Borys Wróbel received his PhD at the University of Gdańsk (Poland) in 1998, was a Fulbright Visiting Researcher in the Salk Institute for Biological Studies in San Diego, CA, and later FEBS and EMBO Fellow at the Hebrew University of Jerusalem (Israel), Marie Curie Postdoctoral Fellow at the University of Valencia, and Sciex Fellow in the Insitute of Neuroinformatics at the University of Zurich and ETHZ (Switzerland). He is now a professor at the Adam Mickiewicz University in Poznań, Poland. He is one of the vice-chairs of the ImAppNIO working group on Benchmarking. Aleš Zamuda is an Assistant Professor and Researcher at University of Maribor (UM), Slovenia. He received Ph.D. (2012), M.Sc. (2008), and B.Sc. (2006) degrees in computer science from UM. He is management committee member for Slovenia at European Cooperation in Science (COST), actions CA15140 (ImAppNIO - Improving Applicability of Nature-Inspired Optimisation by Joining Theory and Practice) and IC1406 (cHiPSet - High-Performance Modelling and Simulation for Big Data Applications). He is IEEE Senior Member, IEEE Slovenia Section Vice Chairman and Young Professionals Chairman, IEEE CIS member, ACM SIGEVO member, ImAppNIO Benchmarks working group vice-chair, and associate editor for Swarm and Evolutionary Computation (IF3.818). His areas of computer science applications include ecosystems, evolutionary algorithms, multicriterion optimization, artificial life, and computer animation; currently yielding h-index 18, 41 publications, and 883 citations on Scopus. He won IEEE R8 SPC 2007 award, IEEE CEC 2009 ECiDUE, 2016 Danubuius Young Scientist Award, and 1% top reviewer at 2017 and 2018 Publons Peer Review Awards, including reviews for over 40 journals and 85 conferences. 6. Hosting Event The Genetic and Evolutionary Computation Conference (GECCO 2019) July 13-17, 2019, Prague, Czech Republic http://gecco-2019.sigevo.org The Genetic and Evolutionary Computation Conference (GECCO 2019) will present the latest high-quality results in genetic and evolutionary computation. Topics include genetic algorithms, genetic programming, evolution strategies, evolutionary programming, memetic algorithms, hyper-heuristics, real-world applications, evolutionary machine learning, evolvable hardware, artificial life, adaptive behavior, ant colony optimization, swarm intelligence, biological applications, evolutionary robotics, coevolution, artificial immune systems, and more. The full list of tracks is available at: https://gecco-2019.sigevo.org/index.html/Program+Tracks The GECCO 2019 Program Committee invites the submission of technical papers describing your best work in genetic and evolutionary computation. Full papers of at most 8 pages (excluding references) should present original work that meets the high-quality standards of GECCO. Accepted full papers appear in the ACM digital library as part of the Main Proceedings of GECCO. For full papers, a separate abstract needs to be submitted first by January 30, 2019. Full papers are due by the non-extensible deadline of February 6, 2019. Each paper submitted to GECCO will be rigorously evaluated in a double-blind review process. Evaluation is done on a per-track basis, ensuring high interest and high expertise of the reviewers. Review criteria include the significance of the work, technical soundness, novelty, clarity, writing quality, relevance and, if applicable, sufficiency of information to permit replication. Besides full papers, poster-only papers of at most 2 pages may be submitted. Poster-only papers should present original work that has not yet reached the maturity and completeness of research results that are published as full papers at GECCO. The review of poster-only papers follows the same double-blind process described above. Accepted poster-only papers will appear in the ACM digital library as part of the Companion Proceedings of GECCO. Poster-only papers are due by the non-extensible deadline of February 6, 2019, and no abstract needs to be submitted first. By submitting a paper, the author(s) agree that, if their paper is accepted, they will: - Submit a final, revised, camera-ready version to the publisher on or before the camera-ready deadline - Register at least one author to attend the conference on April 17, 2019 - Attend the conference (at least one author) - Present the accepted paper at the conference Each paper accepted needs to have at least one author registered. If an author is presenting more than one paper at the conference, she/he does not pay any additional registration fees. 7. Related Events Special Issue on Benchmarking of Computational Intelligence Algorithms, Applied Soft Computing, Elsevier B.V., http://iao.hfuu.edu.cn/bocia-asoc-si Special Session on Benchmarking of Evolutionary Algorithms for Discrete Optimization (BEADO), 2019 IEEE Congress on Evolutionary Computation (CEC'19), June 10-13, 2019 in Wellington, New Zealand , http://iao.hfuu.edu.cn/beado19