- Written by Thomas Weise
at the Genetic and Evolutionary Computation Conference (GECCO 2019)
July 13-17, 2019, Prague, Czech Republic
The Black-Box Discrete Optimization Benchmarking (BB-DOB) Workshop, a part of the Genetic and Evolutionary Computation Conference (GECCO) 2019, is cordially inviting the submission of original and unpublished research papers.
The Black-Box-Optimization Benchmarking (BBOB) methodology associated to the BBOB-GECCO workshops has become a well-established standard for benchmarking stochastic and deterministic continuous optimization algorithms. The aim of the BB-DOB workshop series is to set up a process that will allow to achieve a similar standard methodology for the benchmarking of black box optimization algorithms in discrete and combinatorial search spaces.
The long term aim of our workshop series is to produce, for the domain of discrete optimization:
- a well-motivated benchmark function testbed,
- an experimental set-up,
- methods for the generation of data output for post-processing and
- proper presentations of the results in graphs and tables.
The aims of this GECCO 2019 BB-DOB workshop are to finalize the benchmarking testbed for discrete optimization and to promote a discussion of which performance measures should be used.
The benchmark functions should capture the difficulties of combinatorial optimization problems in practice. They also should be comprehensible so that algorithm behaviors can be understood or interpreted according to the performance on a given benchmark problem. The goal is that a desired search behavior can be pictured and algorithm deficiencies can be understood in depth. This understanding will lead to the design of improved algorithms. Ideally, we would like the benchmark functions to be scalable with the problem size and non-trivial in the black box optimization sense (the function may be shifted such that the global optimum may be any point). Achieving this goal would help greatly in bridging the gap between theoreticians and experimentalists.
We also wish to investigate which measures should be used to compare algorithm performance, which statistical tests should be run to compare algorithms, and how to deal with unsuccessful runs.
This workshop wants to bring together experts on benchmarking of optimization algorithms. It will provide a common forum for discussions and exchange of opinions. Interested participants are encouraged to submit a paper related to black-box optimization benchmarking of discrete optimizers. The topics of interesting especially include papers that
- suggest functions to be included in the benchmark and motivate the reasons for inclusion,
- suggest benchmark function properties that allow to capture difficulties which occur in real-world applications (e.g., deception, separability, etc.),
- suggest which classes of standard combinatorial optimization problems should be included and how to select significant instances,
- suggest which classes of toy problems should be included and motivate why,
- suggest which performance measures should be used to analyze and compare algorithms and comment/suggestions on related issues, and/or
- tackle any other aspect of benchmarking methodology for discrete optimizers such as design of experiments, presentation methods, benchmarking frameworks, etc.
- conduct performance comparisons, landscape analysis, discussion of selected benchmark problems and/or provided statistics of IOHprofiler, a ready-to-use software for the empirical analysis of iterative optimization heuristics
This workshop is organized as part of ImAppNIO (Cost Action 15140).
- Written by Thomas Weise
at the 2019 IEEE Congress on Evolutionary Computation (CEC'19)
June 10-13, 2019 in Wellington, New Zealand
The Special Session on Benchmarking of Evolutionary Algorithms for Discrete Optimization (BEADO), a part of the 2019 IEEE Congress on Evolutionary Computation (CEC'19), is cordially inviting the submission of original and unpublished research papers.
Evolutionary Computation (EC) is a huge and expanding field, attracting more and more interests from both academia and industry. It includes a wide and ever-growing variety of optimization algorithms, which, in turn, are applied to an even wider and faster growing range of different problem domains, including discrete optimization. For the discrete domain and application scenarios, we want to pick the best algorithms. Actually, we want to do more, we want to improve upon the best algorithm. This requires a deep understanding of the problem at hand, the performance of the algorithms we have for that problem, the features that make instances of the problem hard for these algorithms, and the parameter settings for which the algorithms perform the best. Such knowledge can only be obtained empirically, by collecting data from experiments, by analyzing this data statistically, and by mining new information from it. Benchmarking is the engine driving research in the fields of EAs for decades, while its potential has not been fully explored.
The goal of this special session is to solicit original works on the research in benchmarking: Works which contribute to the domain of benchmarking of discrete algorithms from the field of Evolutionary Computation, by adding new theoretical or practical knowledge. Papers which only apply benchmarking are not in the scope of the special session.
This special session wants to bring together experts on benchmarking, evolutionary computation algorithms, and discrete optimization. It provides a common forum for them to exchange findings, to explore new paradigms for performance comparison, and to discuss issues such as
- modelling of algorithm behaviors and performance
- visualizations of algorithm behaviors and performance
- statistics for performance comparison (robust statistics, PCA, ANOVA, statistical tests, ROC, …)
- evaluation of real-world goals such as algorithm robustness, and reliability
- theoretical results for algorithm performance comparison
- comparison of theoretical and empirical results
- new benchmark problems
- the comparison of algorithms in “non-traditional” scenarios such as
- multi- or many-objective domains
- parallel implementations, e.g., using GGPUs, MPI, CUDA, clusters, or running in clouds
- large-scale problems or problems where objective function evaluations are costly
- dynamic problems or where the objective functions involve randomized simulations or noise
- comparative surveys with new ideas on
- dos and don'ts, i.e., best and worst practices, for algorithm performance comparison
- tools for experiment execution, result collection, and algorithm comparison
- benchmark sets for certain problem domains and their mutual advantages and weaknesses
- Written by Thomas Weise
On November 26, 2018, our university was visited by a delegation from the German province Saxony [萨克森自由州], which, to a large degree, was composed of professors from the Chemnitz University of Technology [Technische Universität Chemnitz] (TUC) from Chemnitz [开姆尼茨], Germany. This made me personally very happy, since Chemnitz is my hometown, I received my Master's degree from that university, and visited it to give research talks in 2017 and 2018.
The delegation was led by Dr. Peter Homilius, the vice-directory of the Economic Development Corporation (WFS) Saxony and Prof. Dr. Maximilian Eibl, the vice-president of the TUC and chair of Media Informatics in my old faculty there, the Faculty of Computer Science. Further members of the delegation were Prof. Dr. Egon Müller from the Department of Factory Planning and Factory Management and the Chemnitz Automotive Institute (CATI) at TUC, Prof. Dr. Andreas Schubert, chair of Micromanufacturing Technology, Mr. Claus-Peter Held (CATI), Dr. Frank Löschmann, director of the SisTeam company, as well as Mr. Huaidong Wu and Mr. Chao Ying (SisTeam).
Saxony is a province in the eastern part of Germany. Its capital is Dresden city, but the most industrialized city has always been Chemnitz, which, historically, is one of the cradles of industrialization of Germany. This area is also named as one of the top-20 innovative regions of Europe. The TU Chemnitz has more than 180 years of history and is the motor of innovation in that area. Automation, engineering, lightweight material engineering, automotive industries, the constant improvement of existing technologies, the improvement of production efficiency, research on new materials – the TU Chemnitz is highly competitive in all of these fields. For instance, it also holds the MERGE excellence cluster for multifunctional lightweight structure technologies. All of these fields are important to fill concepts such as Industry 4.0 and Made in China 2025 with life. There were many fruitful discussions with the aim to establish collaborations between the TUC and our uni, centered around such important topics as smart production and the education of engineers. The delegation was impressed with the application-oriented education that our university has developed by adopting German approaches to the Chinese environment. As a result of our talks, I am convinced that our unis will establish successful, long-lasting, and highly productive collaborations.
- Written by Thomas Weise
Today and yesterday I attended the meeting "Research and Teaching in China" organized by the DAAD, the [Deutscher Akademischer Austauschdienst], i.e., the German Academic Exchange Service [德国学术交流中心] in the German Embassy in Beijing. This meeting was well-attended by probably over 50 German researchers who live, work, study, and teach in China. The event was organized as two half-day meetings, which stroke a perfect balance between presentations and opportunities for personal discussions. It allowed us to exchange thoughts, experiences, and impressions and to discuss topics such as research funding and career planning in China. The meeting also showed that there is a very thriving and growing community of German researchers in China, many of which live here for a long time and are well-integrated into the society. These scientists contribute research in many diverse areas such as architecture, chemistry, biology, city planning, computer science, operations research, mathematics, medicine, archeology, as well as law studies and history. It was a really nice meeting and I hope that many such meetings will follow. (The second meeting was held in June 2019 in Shanghai.)
- Written by Thomas Weise
On October 30, 2018, I gave my research talk Automating Scientific Research in Optimization at the chair for Artificial Intelligence and Software Technology (KIST) of Prof. Dr. Manfred Schmidt-Schauß at the Computer Science Institute, Computer Science and Mathematics Department of the Goethe University Frankfurt in Frankfurt am Main, Germany.
Prof. Schmidt-Schauß and his group are following several highly interesting strands of research centered around functional programming languages and program analysis. For instance, they work on the static analyses of programs and expressions, including termination analysis, demand analysis, and strictness analysis, which are important topics for compilers of (non-strict) functional programming languages. They work on understanding the observational semantics of functional programming languages with call-by-need evaluation in the contexts of studying extended lambda-calculi and concurrent computation. Another one of their research topics is understanding the side effects of input/output operations in lazy functional programming languages. They also conduct research on unification, i.e., solving equations in a logical form, encompassing unification in equational theories, sorted unification, combination of unification algorithms and unification in higher order logics. Finally, they work on knowledge representation, and investigate the properties of concept description languages and their reasoning behavior and complexity.
It was very nice to meet Prof. Schmidt-Schauß and his group. I am very thankful for their hospitality and for the nice discussion as well as the kind reception of my talk.
Page 6 of 17
- Welcome to Dr. Pei XU
- Prof. Weise Attends the Hefei National Day Tea Party for People from All Walks of Life [奋斗百年路 启航新征程 合肥市各界人士迎国庆茶话会]
- Guest Lesson "Comparing Optimization Algorithm" at the University of Adelaide
- Research Talk by Prof. Xuejun WANG [王学军]: Some convergence results for weighted sums of dependent random variables and applications [2021-06-18]
- Research Talk by Prof. Junlong ZHAO [赵俊龙]: Multiple influential point detection in high dimensional regression spaces [2021-06-18]