Special Issue on Benchmarking of Computational Intelligence Algorithms in the Applied Soft Computing Journal

Call for Papers Call for Papers

Special Issue on Benchmarking of Computational Intelligence Algorithms

Applied Soft Computing by Elsevier B.V.
http://iao.hfuu.edu.cn/bocia-asoc-si

 

Cover of the Applied Soft Computing journal.Computational Intelligence (CI) is a huge and expanding field which is rapidly gaining importance, attracting more and more interests from both academia and industry. It includes a wide and ever-growing variety of optimization and machine learning algorithms, which, in turn, are applied to an even wider and faster growing range of different problem domains. For all of these domains and application scenarios, we want to pick the best algorithms. Actually, we want to do more, we want to improve upon the best algorithm. This requires a deep understanding of the problem at hand, the performance of the algorithms we have for that problem, the features that make instances of the problem hard for these algorithms, and the parameter settings for which the algorithms perform the best. Such knowledge can only be obtained empirically, by collecting data from experiments, by analyzing this data statistically, and by mining new information from it. Benchmarking is the engine driving research in the fields of optimization and machine learning for decades, while its potential has not been fully explored. Benchmarking the algorithms of Computational Intelligence is an application of Computational Intelligence itself! This special issue of the EI/SCIE-indexed Applied Soft Computing journal published by Elsevier B.V. solicited novel contributions from this domain according to the topics of interest listed below.

Here you can download the Call for Papers (CfP) in PDF format and here as plain text file. The journal's website where the papers appear is here. The special issue has now been completed and 14 articles were accepted and published.

Articles in this Special Issue

The special issue is now completed. 14 articles were accepted and in August 2020, the editorial was published.

  1. Thomas Weise, Markus Wagner, Bin Li, Xingyi Zhang, and Jörg Lässig. Special Issue on Benchmarking of Computational Intelligence Algorithms in the Applied Soft Computing Journal (Editorial). Applied Soft Computing 93:106502. August 2020. doi:10.1016/j.asoc.2020.106502
  2. Andreas Fischbach and Thomas Bartz-Beielstein. Improving the Reliability of Test Functions Generators. Applied Soft Computing 92:106315, July 2020. doi:10.1016/j.asoc.2020.106315.
  3. Lucas Augusto Müller de Souza, José Eduardo Henriques da Silva, Luciano Jerez Chaves, and Heder Soares Bernardino. A Benchmark Suite for Designing Combinational Logic Circuits via Metaheuristics. Applied Soft Computing 91:106246, June 2020, doi:10.1016/j.asoc.2020.106246.
  4. Urban Škvorc, Tome Eftimov, and Peter Korošec. Understanding the Problem Space in Single-Objective Numerical Optimization using Exploratory Landscape Analysis. Applied Soft Computing 90:106138, May 2020, doi:10.1016/j.asoc.2020.106138.
  5. Ivan Reinaldo Meneghini, Marcos Antonio Alves, António Gaspar-Cunha, and Frederico Gadelha Guimarães. Scalable and Customizable Benchmark Problems for Many-Objective Optimization. Applied Soft Computing 90:106139, May 2020, doi:10.1016/j.asoc.2020.106139.
  6. Zhi-Ze Wu, Shou-Hong Wan, Xiao-Feng Wang, Ming Tan, Le Zou, Xin-Lu Li, and Yan Chen. A Benchmark Data Set for Aircraft Type Recognition from Remote Sensing Images. Applied Soft Computing 89:106132, April 2020, doi:10.1016/j.asoc.2020.106132.
  7. Mihaela Oprea. A General Framework and Guidelines for Benchmarking Computational Intelligence Algorithms Applied to Forecasting Problems Derived from an Application Domain-Oriented Survey. Applied Soft Computing 89:106103, April 2020, doi:10.1016/j.asoc.2020.106103.
  8. Ryoji Tanabe and Hisao Ishibuchi. An Easy-to-Use Real-World Multi-Objective Optimization Problem Suite. Applied Soft Computing 89:106078, April 2020, doi:10.1016/j.asoc.2020.106078.
  9. Carola Doerr, Furong Ye, Naama Horesh, Hao Wang, Ofer M. Shir, Thomas Bäck. Benchmarking Discrete Optimization Heuristics with IOHprofiler. Applied Soft Computing 88:106027, March 2020, doi:10.1016/j.asoc.2019.106027.
  10. David J. Walker and Matthew J. Craven. Identifying Good Algorithm Parameters in Evolutionary Multi- and Many-Objective Optimisation: A Visualisation Approach. Applied Soft Computing 88:105902, March 2020, doi:10.1016/j.asoc.2019.105902.
  11. Jakob Bossek, Pascal Kerschke, and Heike Trautmann. A Multi-Objective Perspective on Performance Assessment and Automated Selection of Single-Objective Optimization Algorithms Applied Soft Computing 88:105901, March 2020, doi:10.1016/j.asoc.2019.105901.
  12. Tome Eftimov, Gašper Petelin, and Peter Korošec. DSCTool: A Web-Service-based Framework for Statistical Comparison of Stochastic Optimization Algorithms. Applied Soft Computing 87:105977, February 2020, doi:10.1016/j.asoc.2019.105977.
  13. Andrea Tangherloni, Simone Spolaor, Paolo Cazzaniga, Daniela Besozzi, Leonardo Rundo, Giancarlo Mauri, Marco S. Nobile Biochemical Parameter Estimation vs. Benchmark Functions: A Comparative Study of Optimization Performance and Representation Design. Applied Soft Computing 81:105494, August 2019, doi:10.1016/j.asoc.2019.105494.
  14. Muwei Jian, Qiang Qi, Hui Yu, Junyu Dong, Chaoran Cui, Xiushan Nie, Huaxiang Zhang, Yilong Yin, Kin-Man Lam. The Extended Marine Underwater Environment Database and Baseline Evaluations. Applied Soft Computing 80:425-437, July 2019, doi:10.1016/j.asoc.2019.04.025.
  15. Sina Torabi and Mattias Wahde. A Method for Performance Analysis of a Genetic Algorithm Applied to the Problem of Fuel Consumption Minimization for Heavy-Duty Vehicles. Applied Soft Computing 80:735-741, July 2019, doi:10.1016/j.asoc.2019.04.042.

Topics of Interest

  • mining of higher-level information from experimental results
  • modelling of algorithm behaviors and performance
  • visualizations of algorithm behaviors and performance
  • statistics for performance comparison (robust statistics, PCA, ANOVA, statistical tests, ROC, …)
  • evaluation of real-world goals such as algorithm robustness, reliability, and implementation issues
  • theoretical results for algorithm performance comparison
  • comparison of theoretical and empirical results
  • new benchmark problems
  • automatic algorithm configuration and selection
  • the comparison of algorithms in "non-traditional" scenarios such as
    • multi- or many-objective domains
    • parallel implementations, e.g., using GGPUs, MPI, CUDA, clusters, or running in clouds
    • large-scale problems or problems where objective function evaluations are costly
    • dynamic problems or where the objective functions involve randomized simulations or noise
    • deep learning and big data setups
  • comparative surveys with new ideas on
    • dos and don'ts, i.e., best and worst practices, for algorithm performance comparison
    • tools for experiment execution, result collection, and algorithm comparison
    • benchmark sets for certain problem domains and their mutual advantages and weaknesses
  • survey and review of benchmarking, algorithm performance statistics, and performance-related theorems
  • performance indicators, objective (problem) reduction

All accepted papers in this special issue will be included in the Applied Soft Computing journal published by Elsevier B.V., and indexed by EI and SCIE.

Submission Process

The submission for this special issue is closed. No new papers can be accepted.

The Applied Soft Computing journal subscribes to the concept of Virtual Special Issues (VSIs). In a VSI, submissions are made through the editorial system at http://ees.elsevier.com/asoc/ at any time until the final submission deadline. They go through the normal editorial process (managed by the guest editors), and, if accepted, are published immediately after acceptance. Accepted papers can appear at different times, but are bundled to the VSI.

IMPORTANT: Please choose "VSI: Benchmarking CI" when specifying the Article Type.

Please send any inquiries to Thomas Weise at This email address is being protected from spambots. You need JavaScript enabled to view it. with CC to This email address is being protected from spambots. You need JavaScript enabled to view it. and This email address is being protected from spambots. You need JavaScript enabled to view it..

Important Dates

The submission for this special issue is closed. No new papers can be accepted.

Virtual Special Issue Start: July 2018
Final Deadline for Submissions: May 19, 2019
Review notification about 12 weeks after submission..
Up to 2 rounds of revisions.
Publication immediately after receipt of accepted final version.

Guest Editors

  • Thomas Weise, Institute of Applied Optimization, Hefei University, Hefei, China
  • Bin Li, University of Science and Technology of China, Hefei, China
  • Markus Wagner, University of Adelaide, Adelaide, SA, Australia
  • Xingyi Zhang, Anhui University, Hefei, China
  • Jörg Lässig, University of Applied Sciences Zittau/Görlitz, Görlitz, Germany

Biographies of the Guest Editors

Portrait of Prof. Dr. Thomas Weise

Prof. Dr. Thomas Weise obtained the MSc in Computer Science in 2005 from the Chemnitz University of Technology and his PhD from the University of Kassel in 2009. He then joined the University of Science and Technology of China (USTC) as PostDoc and subsequently became Associate Professor at the USTC-Birmingham Joint Research Institute in Intelligent Computation and Its Applications (UBRI) at USTC. In 2016, he joined Hefei University as Full Professor to found the Institute of Applied Optimization at the Faculty of Computer Science and Technology. Prof. Weise has more than seven years of experience as a full time researcher in China, having contributed significantly both to fundamental as well as applied research. He has more than 80 scientific publications in international peer reviewed journals and conferences. His book "Global Optimization Algorithms – Theory and Application" has been cited more than 730 times. He has acted as reviewer, editor, or programme committee member at 70 different venues.

 

Portrait of Prof. Dr. Bin Li

Prof. Dr. Bin Li received the B.S. degree from Hefei University of Technology, Hefei, China, in 1992, the M.Sc. degree from Institute of Plasma Physics, China Academy of Science, Hefei, China, in 1995, and the Ph.D. degree from University of Science and Technology of China (USTC), China in 2001. He is currently a professor with the School of Information Science and Technology, USTC, Hefei, China. He has authored and co-authored more than 40 refereed publications. His major research interests include evolutionary computation, memetic algorithms, pattern recognition, and real-world applications. Dr. Li is the Founding Chair of IEEE CIS Hefei Chapter, Counselor of IEEE USTC Student Branch, senior member of Chinese Institute of Electronics (CIE), member of the Technical Committee of Electronic Circuits and Systems Section of CIE.

 

Portrait of Senior Lecturer Dr. Markus Wagner

Dr. Markus Wagner is a Senior Lecturer at the School of Computer Science, University of Adelaide, Australia. He has done his PhD studies at the Max Planck Institute for Informatics in Saarbrücken, Germany and at the University of Adelaide, Australia. His research topics range from mathematical runtime analysis of heuristic optimization algorithms and theory-guided algorithm design to applications of heuristic methods to renewable energy production, professional team cycling and software engineering. So far, he has been a program committee member 30 times, and he has written over 70 articles with over 70 different co-authors. He has chaired several education-related committees within the IEEE CIS, is Co-Chair of ACALCI 2017 and General Chair of ACALCI 2018.

 

Portrait of Prof. Dr. Xingyi Zhang

Prof. Dr. Xingyi Zhang received the B.Sc. from Fuyang Normal College in 2003, and the M.Sc. in 2006 and Ph.D. in 2009 both from Huazhong University of Science and Technology. Currently, he is a professor in the School of Computer Science and Technology, Anhui University. His main research interests include unconventional models and algorithms of computation, multi-objective evolutionary optimization and membrane computing. He is the chair of 2017 Data Driven Optimization of Complex Systems and Applications and 2015 Asian Conference on Membrane Computing. He also serves as Editorial Board Member of Complex & Intelligent Systems and International Journal of Bio-inspired Computing.

 

Portrait of Prof. Dr. Jörg Lässig

Prof. Dr. Jörg Lässig leads the Enterprise Application Development Group (EAD) of the University of Applied Sciences Zittau/Görlitz (HSZG) in Germany since 2011. He holds degrees in Computer Science, Computational Physics, and received a Ph.D. in Computer Science for his research on efficient algorithms and models for the generation and control of cooperation networks at Chemnitz University of Technology (Germany), which he finished in 2009. Prof. Lässig has PostDocs at the Università della Svizzera italiana, Institute of Computational Sciences (Lugano, Switzerland) and the International Computer Science Institute Berkeley (California, USA). His EAD research group at HSZG and his IT security group with the Fraunhofer Society are focusing on topics concerned with intelligent data driven technologies for state-of-the-art IT infrastructures and services. His research directions include sustainable IT services, energy efficiency benchmarking, regional carbon footprints and energy informatics.

 

Related Event: BOCIA'18 Workshop

BOCIA 2018, the International Workshop on Benchmarking of Computational Intelligence Algorithms takes place as part of the Tenth International Conference on Advanced Computational Intelligence (ICACI 2018) from March 29 to 31, 2018, in Xiamen, China. It's topics of interest are the same as for this Special Issue. The (extended) workshop submission deadline is December 1, 2017. Authors of selected papers from the workshop will be invited to submit extended versions of these papers to this special issue, although the Special Issue CfP is also open to submissions unrelated to the workshop.

Web Links

  1. Special Issue on Benchmarking of Computational Intelligence Algorithms in the Applied Soft Computing Journal Completed
  2. The journal's website for the Special Issue: https://www.sciencedirect.com/journal/applied-soft-computing/special-issue/10LQLZBS7T4
  3. Special Issue Call for Papers (CfP) in PDF format
  4. Special Issue Call for Papers (CfP) as plain text file
  5. Special Issue Call for Papers (CfP) on journal website
  6. Black-Box Discrete Optimization Benchmarking (BB-DOB@GECCO) Workshop (2019)
  7. Special Session on Benchmarking of Evolutionary Algorithms for Discrete Optimization (BEADO) (2019)
  8. Black-Box Discrete Optimization Benchmarking (BB-DOB@PPSN) Workshop (2018)
  9. Black-Box Discrete Optimization Benchmarking (BB-DOB@GECCO) Workshop (2018)
  10. Black Box Discrete Optimization Benchmarking (BB-DOB) Workshop
  11. BOCIA 2018 Workshop Homepage
  12. Submission Deadline Extended to December 1, 2017: International Workshop on Benchmarking of Computational Intelligence Algorithms (BOCIA)
  13. BOCIA 2018 Workshop Call for Papers (CfP) in PDF format
  14. BOCIA 2018 Workshop Call for Papers (CfP) as plain text file
  15. BOCIA 2018 Workshop WikiCfP Entry
  16. BOCIA 2018 Workshop CfP on LinkedIn