We are happy to announce that the submissions for our International Workshop on Benchmarking of Computational Intelligence Algorithms (BOCIA) will be open for two more weeks, until December 1, 2017 (the notification deadline has moved to December 30 accordingly). The hosting event, the Tenth International Conference on Advanced Computational Intelligence 2018 (ICACI 2018), has extended its submission deadline. The ICACI conference is organized by the IEEE and will take place in the beautiful city of Xiamen in China from March 29 to 31, 2018. Besides being a very interesting conference with seven special sessions and workshops, ICACI also features an exciting list of top-level speakers, such as Kay Chen Tan, Jun Wang, and Zhi-Hua Zhou.

All accepted papers of our workshop will be included in the Proceedings of the ICACI 2018 published by IEEE Press and indexed by EI. Authors of selected papers will be invited to submit extended versions of these papers to the Special Issue on Benchmarking of Computational Intelligence Algorithms in the Applied Soft Computing journal by Elsevier B.V., indexed by EI and SCIE. Here you can download the Special Issue Call for Papers (CfP) in PDF format and here as plain text file.

The BOCIA workshop provides a forum for researchers to discuss all issues related to the benchmarking and performance comparison of Computational Intelligence methods, including algorithms for optimization, Machine Learning, data mining, operations research, Big Data, and Deep Learning, as well as Evolutionary Computation and Swarm Intelligence. Most of these fields have in common that the algorithms developed in them need to balance the quality of the solutions they produce with the time they require to discover them. So performance has two dimensions, time and quality. A rule of thumb is that if we have a higher computational budget, we can hopefully attain better solutions - but this very strongly depends both on the algorithm we use and the problem we try to solve. Some algorithms are better, some are worse. Actually, some setups of the same algorithm may be good while others are bad. Some problems are harder, some are easier. Actually, some instances of the same problem may be harder than others (say a Traveling Salesman Problem where all cities are on a circle is easier than one where the cities are randomly distributed). In practice, we want to solve the problems at hand in the most efficient way, to pick the right algorithm setup for the right problem instance. Since we usually cannot determine which way is most efficient using theoretic concerns alone, experiments are needed - benchmarking. Benchmarking is also necessary for rigorous research in the domains, since we can only improve algorithms if we understand their mutual advantages and disadvantages, understand what features make problems hard or easy for them, and which setup parameters have which impact on performance.

There are lots of interesting issues involved in benchmarking, such as how to design experiments, how to extract useful information from large sets of experimental results, how to visualize results, down to what should be measured, how to store and document results, and again up to questions such as whether we can design models that tell us how good an algorithm will likely perform on a new problem based on the features of this problem. While this field was under-rated for a long time, its importance is more and more recognized. The field is widely considered as one of the most important "construction sides" in Computational Intelligence. It is not sufficient to just develop more and more algorithms, we also need to get an exact understanding of their mutual advantages and disadvantages. This is vital for practical applications as well. This development is manifested in the fact that major parts of huge international projects such as the COST Action CA15140: ImAppNIO revolve around it. Also, the best algorithms in many domains now use insights into algorithm performance to automatically select the right strategy for a given problem - algorithm selection, portfolios, and configuration all draw from research on benchmarking (see, e.g., the international COSEAL group).  The field is massively getting traction and offers many challenges, because the analysis and benchmarking of Computational Intelligence algorithms is an application of Computational Intelligence as well!

Our workshop will be one of the first general events for this field, and the only one which brings together experts from all fields of Computational Intelligence who are interested in algorithm comparison, evaluation, benchmarking, analysis, configuration, and performance modeling.

We are looking forward to meeting you in March 2018 in Xiamen.

The workshop is now over, here you can find a short workshop report.