Written by: Thomas Weise

Created: 07 January 2019

• Benchmarking
• Operations Research
• Optimization
• Evolutionary Computation
• Machine Learning
• Statistics
• Datamining
• Computational Intelligence

The paper submission deadline of the 2019 IEEE Congress on Evolutionary Computation has been extended until January 31, 2019. As a result, we are happy to announce that there are also two more weeks for authors to submit their papers to our Special Session on Benchmarking of Evolutionary Algorithms for Discrete Optimization (BEADO).

Evolutionary Computation (EC) is a huge and expanding field, attracting more and more interests from both academia and industry. It includes a wide and ever-growing variety of optimization algorithms, which, in turn, are applied to an even wider and faster growing range of different problem domains, including discrete optimization. Benchmarking is the engine driving research in the fields of Evolutionary Algorithms (EAs) for decades, while its potential has not been fully explored. With our special session, we want to bring together experts on benchmarking, evolutionary computation algorithms, and discrete optimization and provide a platform for them to exchange findings, to explore new paradigms for performance comparison, and to discuss issues such as

• modelling of algorithm behaviors and performance
• visualizations of algorithm behaviors and performance
• statistics for performance comparison (robust statistics, PCA, ANOVA, statistical tests, ROC, …)
• evaluation of real-world goals such as algorithm robustness, and reliability
• theoretical results for algorithm performance comparison
• comparison of theoretical and empirical results
• new benchmark problems
• the comparison of algorithms in “non-traditional” scenarios such as
  • multi- or many-objective domains
  • parallel implementations, e.g., using GGPUs, MPI, CUDA, clusters, or running in clouds
  • large-scale problems or problems where objective function evaluations are costly
  • dynamic problems or where the objective functions involve randomized simulations or noise
• comparative surveys with new ideas on
  • dos and don'ts, i.e., best and worst practices, for algorithm performance comparison
  • tools for experiment execution, result collection, and algorithm comparison
  • benchmark sets for certain problem domains and their mutual advantages and weaknesses

Please visit the special session website for more information. Here you can download the BEADO Special Session Call for Papers (CfP) in PDF format and here as plain text file.

Written by: Thomas Weise

Created: 11 December 2018

• Benchmarking
• Optimization
• Combinatorial Optimization
• Discrete Optimization
• Wu Zhize
• Li Xinlu
• Workshop

at the Genetic and Evolutionary Computation Conference (GECCO 2019)

July 13-17, 2019, Prague, Czech Republic

http://iao.hfuu.edu.cn/bbdob-gecco19

The Black-Box Discrete Optimization Benchmarking (BB-DOB) Workshop, a part of the Genetic and Evolutionary Computation Conference (GECCO) 2019, is cordially inviting the submission of original and unpublished research papers.

The Black-Box-Optimization Benchmarking (BBOB) methodology associated to the BBOB-GECCO workshops has become a well-established standard for benchmarking stochastic and deterministic continuous optimization algorithms. The aim of the BB-DOB workshop series is to set up a process that will allow to achieve a similar standard methodology for the benchmarking of black box optimization algorithms in discrete and combinatorial search spaces.

Here you can download the BBDOB Workshop Call for Papers (CfP) in PDF format and here as plain text file.

The long term aim of our workshop series is to produce, for the domain of discrete optimization:

1. a well-motivated benchmark function testbed,
2. an experimental set-up,
3. methods for the generation of data output for post-processing and
4. proper presentations of the results in graphs and tables.

The aims of this GECCO 2019 BB-DOB workshop are to finalize the benchmarking testbed for discrete optimization and to promote a discussion of which performance measures should be used.

The benchmark functions should capture the difficulties of combinatorial optimization problems in practice. They also should be comprehensible so that algorithm behaviors can be understood or interpreted according to the performance on a given benchmark problem. The goal is that a desired search behavior can be pictured and algorithm deficiencies can be understood in depth. This understanding will lead to the design of improved algorithms. Ideally, we would like the benchmark functions to be scalable with the problem size and non-trivial in the black box optimization sense (the function may be shifted such that the global optimum may be any point). Achieving this goal would help greatly in bridging the gap between theoreticians and experimentalists.

We also wish to investigate which measures should be used to compare algorithm performance, which statistical tests should be run to compare algorithms, and how to deal with unsuccessful runs.

This workshop wants to bring together experts on benchmarking of optimization algorithms. It will provide a common forum for discussions and exchange of opinions. Interested participants are encouraged to submit a paper related to black-box optimization benchmarking of discrete optimizers. The topics of interesting especially include papers that

• suggest functions to be included in the benchmark and motivate the reasons for inclusion,
• suggest benchmark function properties that allow to capture difficulties which occur in real-world applications (e.g., deception, separability, etc.),
• suggest which classes of standard combinatorial optimization problems should be included and how to select significant instances,
• suggest which classes of toy problems should be included and motivate why,
• suggest which performance measures should be used to analyze and compare algorithms and comment/suggestions on related issues, and/or
• tackle any other aspect of benchmarking methodology for discrete optimizers such as design of experiments, presentation methods, benchmarking frameworks, etc.
• conduct performance comparisons, landscape analysis, discussion of selected benchmark problems and/or provided statistics of IOHprofiler, a ready-to-use software for the empirical analysis of iterative optimization heuristics

For more information please contact Pietro S. Oliveto at This email address is being protected from spambots. You need JavaScript enabled to view it..

This workshop is organized as part of ImAppNIO (Cost Action 15140).

Read more: Black Box Discrete Optimization Benchmarking (BB-DOB) Workshop