Rigorous Performance Analysis of Algorithms

Optimization is a technology which has the potential to become even more important than data mining and big data are now, as it can improve the efficiency of each and every aspect of an enterprises operation and products. Optimization means to find approximate solutions for hard problems. Optimization algorithms can find short routes in logistic planning scenarios, construction plans for work pieces that require only little amounts of material, or efficient schedules in production planning. They can help to reduce costs and pollution at the same time.

The term "optimization algorithms" implies a plural, and there are already many different families of algorithms, including exact methods, heuristics, metaheuristics sub-families such as evolutionary computation and swarm intelligence, as well as local search and hybrid algorithms. Each of these families contains dozens of general and specialized algorithms. We want to know how we can find out which method is best for which problem and when and why. First, using the best algorithm will give us the best solutions, which is what we want. Knowing why an algorithm is best could help us to create even better ones. Thus, benchmarking of optimization algorithms in a statistically sound, robust, and ideally automatic way seems to be an important topic to me. Yet it is entirely under-represented in literature. We want to emphasize how important it is, as for practical applications of optimization, we simply need to know that we are using the best and most reliable methods.

Projects

  • the optimizationBenchmarking.org, a framework for comparing and researching the performance of any optimization algorithm implemented in any programming language on any optimization problem
  • the TSP Suite, a framework for investigating the performance of solvers for the Traveling Salesman Problem (TSP), with a fast collection of state-of-the-art TSP solvers implemented in Java

Publications

  • Thomas Weise, Zhize Wu, Xinlu Li, Yan Chen, and Jörg Lässig. Frequency Fitness Assignment: Optimization without Bias for Good Solutions can be Efficient. IEEE Transactions on Evolutionary Computation (TEVC). 2022. Early Access.
    doi:10.1109/TEVC.2022.3191698 / preprint@arxiv / data and source code @ doi:10.5281/zenodo.5567725.
    Indexing: SCI, EI, 1区, CCF-B类
  • Zhiyang Liu, Thomas Weise, and Zhize Wu. Detection of Personal Protective Equipment in Factories: A Survey and Benchmark Dataset. In De-Shuang Huang, Kang-Hyun Jo, Junfeng Jing, Prashan Premaratne, Vitoantonio Bevilacqua, and Abir Hussain, editors, Proceedings of the 18th International Conference Intelligent Computing Methodologies (ICIC'22), August 7–11, 2022, Xi'an, China, Part III, pages 448-459. Lecture Notes in Computer Science (LNCS), sub-series LNAI, volume 13395.
    doi:10.1007/978-3-031-13832-4_37
    Indexing: EI
  • Thomas Weise, Zhize Wu, Xinlu Li, and Yan Chen. Frequency Fitness Assignment: Making Optimization Algorithms Invariant under Bijective Transformations of the Objective Function Value. IEEE Transactions on Evolutionary Computation 25(2):307–319. April 2021.
    Preprint available at arXiv:2001.01416v5 [cs.NE] 15 Oct 2020.
    doi:10.1109/TEVC.2020.3032090 / pdf / paper@arxiv / experimental results and source code data set @ doi:10.5281/zenodo.3899474
    Indexing: SCI, EI, 1区, CCF-B类
  • Thomas Weise, Markus Wagner, Bin Li, Xingyi Zhang, and Jörg Lässig. Special Issue on Benchmarking of Computational Intelligence Algorithms in the Applied Soft Computing Journal (Editorial). Applied Soft Computing Journal (ASOC) 93:106502. August 2020.
    doi:10.1016/j.asoc.2020.106502
    Indexing: EI, WOS, ESCI, 1区
  • Thomas Bartz-Beielstein, Carola Doerr, Daan van den Berg, Jakob Bossek, Sowmya Chandrasekaran, Tome Eftimov, Andreas Fischbach, Pascal Kerschke, William La Cava, Manuel López-Ibáñez, Katherine M. Malan, Jason H. Moore, Boris Naujoks, Patryk Orzechowski, Vanessa Volz, Markus Wagner, Thomas Weise. Benchmarking in Optimization: Best Practice and Open Issues. Technical report available at arXiv:2007.03488v2 [cs.NE] 16 Dec 2020.
    pdf / pdf@arxiv
  • Thomas Weise, Yan Chen, Xinlu Li, and Zhize Wu. Selecting a diverse set of benchmark instances from a tunable model problem for black-box discrete optimization algorithms. Applied Soft Computing Journal (ASOC), 92:106269, June 2020.
    doi:10.1016/j.asoc.2020.106269
    Indexing: EI, WOS, ESCI, 1区
  • Zhi-Ze Wu, Shou-Hong Wan, Xiao-Feng Wang, Ming Tan, Le Zou, Xin-Lu Li, and Yan Chen. A Benchmark Data Set for Aircraft Type Recognition from Remote Sensing Images. Applied Soft Computing Journal (ASOC). 89:106132. April 2020.
    doi:10.1016/j.asoc.2020.106132
    Indexing: EI, WOS, ESCI, 1区
  • Thomas Weise, Yan Jiang, Qi Qi, and Weichen Liu. A Branch-and-Bound-Based Crossover Operator for the Traveling Salesman Problem. International Journal of Cognitive Informatics and Natural Intelligence (IJCINI) 13(3):1-18, Fall 2019
    doi:10.4018/IJCINI.2019070101
    Indexing: ESCI
  • Thomas Weise, Zijun Wu, and Markus Wagner. An Improved Generic Bet-and-Run Strategy with Performance Prediction for Stochastic Local Search. Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence (AAAI 2019), January 27 – February 1, 2019, Honolulu, Hawaii, USA, pages 2395–2402. Palo Alto, CA, USA: AAAI Press. ISBN: 978-1-57735-809-1
    doi:10.1609/aaai.v33i01.33012395 / pdf@IAO / pdf@AAAI / slides / poster / blog 1 / blog 2 / early preprint@arxiv
    Indexing: CCF-A类, EI
  • Thomas Weise, Yuezhong Wu, Weichen Liu, and Raymond Chiong. Implementation Issues in Optimization Algorithms: Do they matter? Journal of Experimental & Theoretical Artificial Intelligence (JETAI) 31(4):533–554, 2019.
    doi:10.1080/0952813X.2019.1574908
    Indexing: EI, ESCI, 4区, CCF-C类
  • Thomas Weise, Xiaofeng Wang, Qi Qi, Bin Li, and Ke Tang. Automatically discovering clusters of algorithm and problem instance behaviors as well as their causes from experimental data, algorithm setups, and instance features. Applied Soft Computing Journal (ASOC), 73:366–382, December 2018.
    doi:10.1016/j.asoc.2018.08.030 / blog entry
    Indexing: EI, WOS:000450124900027, ESCI, 1区
  • Thomas Weise and Zijun Wu. Difficult Features of Combinatorial Optimization Problems and the Tunable W-Model Benchmark Problem for Simulating them. In Black Box Discrete Optimization Benchmarking (BB-DOB) Workshop of Companion Material Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2018), July 15th-19th 2018, Kyoto, Japan, pages 1769-1776, ISBN: 978-1-4503-5764-7. ACM.
    doi:10.1145/3205651.3208240 / pdf / slides / source codes / workshop website
    Additional experimental results with the W-Model, which were not used in this paper, can be found at doi:10.5281/zenodo.1256883.
    Indexing: EI, CCF-C类
  • Markus Ullrich, Thomas Weise, Abhishek Awasthi, and Jörg Lässig. A Generic Problem Instance Generator for Discrete Optimization Problems. In Black Box Discrete Optimization Benchmarking (BB-DOB) Workshop of Companion Material Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2018), July 15th-19th 2018, Kyoto, Japan, pages 1761-1768, ISBN: 978-1-4503-5764-7. ACM.
    doi:10.1145/3205651.3208284 / pdf / slides / source codes / workshop website
    Indexing: EI, CCF-C类
  • Qi Qi, Thomas Weise, and Bin Li. Optimization Algorithm Behavior Modeling: A Study on the Traveling Salesman Problem. In Proceedings of the Tenth International Conference on Advanced Computational Intelligence (ICACI 2018), March 29-31, 2018 in Xiamen [厦门], Fujian [福建省], China, IEEE, pages 861–866. ISBN: 978-1-5386-4362-4. Appeared in the International Workshop on Benchmarking of Computational Intelligence Algorithms (BOCIA) at the ICACI 2018.
    doi:10.1109/ICACI.2018.8377576 / pdf / slides / workshop website
  • Zijun Wu, Michael Kolonko, and Rolf H. Möhring. Stochastic Runtime Analysis of the Cross-Entropy Algorithm. IEEE Transactions on Evolutionary Computation (IEEE-EC) 21(4):616-628, August 2017.
    doi:10.1109/TEVC.2017.2667713
    Indexing: SCI, EI, 1区, CCF-B类
  • Qi Qi, Thomas Weise, and Bin Li. Modeling Optimization Algorithm Runtime Behavior and its Applications. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO'17) Companion, July 15-19, 2017, Berlin, Germany, New York, NY, USA: ACM Press, pages 115-116, ISBN: 978-1-4503-4939-0.
    doi:10.1145/3067695.3076042 / paper / poster / blog entry 1 / blog entry 2
    Indexing: EI, CCF-C类
  • Weichen Liu, Thomas Weise, Yuezhong Wu, and Qi Qi. Combining Two Local Searches with Crossover: An Efficient Hybrid Algorithm for the Traveling Salesman Problem. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO'17), July 15-19, 2017, Berlin, Germany, New York, NY, USA: ACM Press, pages 298-305, ISBN: 978-1-4503-4920-8.
    doi:10.1145/3071178.3071201 / paper / slides / blog entry 1 / blog entry 2
    Indexing: EI:20173104005769, CCF-C类
  • Thomas Weise. From Standardized Data Formats to Standardized Tools for Optimization Algorithm Benchmarking. In Newton Howard, Yingxu Wang, Amir Hussain, Freddie Hamdy, Bernard Widrow, and Lotfi A. Zadeh, editors, Proceedings of the 16th IEEE Conference on Cognitive Informatics & Cognitive Computing (ICCI*CC'17), July 26-28, 2017, University of Oxford, Oxford, UK, pages 490-497. Los Alamitos, CA, USA: IEEE Computer Society Press, ISBN: 978-1-5386-0770-1.
    doi:10.1109/ICCI-CC.2017.8109794 / paper / slides / blog entry
    Indexing: EI
  • Thomas Weise, Yuezhong Wu, Raymond Chiong, Ke Tang, and Jörg Lässig. Global versus Local Search: The Impact of Population Sizes on Evolutionary Algorithm Performance. Journal of Global Optimization 66(3):511-534, November 2016.
    doi:10.1007/s10898-016-0417-5 / pdf
    Indexing: EI:20160902030834, WOS:000386373700007, SCI, 2区
  • Yuezhong Wu, Thomas Weise, and Weichen Liu. Hybridizing Different Local Search Algorithms with Each Other and Evolutionary Computation: Better Performance on the Traveling Salesman Problem. In Proceedings of the 18th Genetic and Evolutionary Computation Conference (GECCO'16), Denver, Colorado, USA, July 20–24, 2016, pages 57-58, New York, NY, USA: Association for Computing Machinery (ACM). ISBN: 978-1-4503-4323-7
    doi:10.1145/2908961.2909001 / pdf / poster
    Indexing: EI:20163702804277, WOS:000383741800029, CCF-C类
  • Weichen Liu, Thomas Weise, Yuezhong Wu, Dan Xu, and Raymond Chiong. An Improved Ejection Chain Method and Its Hybrid Versions for Solving the Traveling Salesman Problem. Journal of Computational and Theoretical Nanoscience 13(6):3601-3610, June 2016
    doi:10.1166/jctn.2016.5189
    Indexing: EI:20164002870059
  • Weichen Liu, Thomas Weise, Yuezhong Wu, and Raymond Chiong. Hybrid Ejection Chain Methods for the Traveling Salesman Problem. In Proceedings of the 10th International Conference on Bio-Inspired Computing – Theories and Applications (BIC-TA'15), Maoguo Gong, Linqiang Pan, Tao Song, Ke Tang, and Xingyi Zhang, editors, September 25-28, 2015, Hefei, Anhui, China, volume 562 of Communications in Computer and Information Science. Berlin/Heidelberg: Springer-Verlag, pages 268-282, ISBN 978-3-662-49013-6.
    doi:10.1007/978-3-662-49014-3_25 / pdf
    Indexing: EI:20160300001732, WOS:000369890300025
  • Dan Xu, Thomas Weise, Yuezhong Wu, Jörg Lässig, and Raymond Chiong. An Investigation of Hybrid Tabu Search for the Traveling Salesman Problem. In Proceedings of the 10th International Conference on Bio-Inspired Computing – Theories and Applications (BIC-TA'15), Maoguo Gong, Linqiang Pan, Tao Song, Ke Tang, and Xingyi Zhang, editors, September 25-28, 2015, Hefei, Anhui, China, volume 562 of Communications in Computer and Information Science. Berlin/Heidelberg: Springer-Verlag, pages 523-537, ISBN 978-3-662-49013-6.
    doi:10.1007/978-3-662-49014-3_47 / pdf
    Indexing: EI:20160300001756, WOS:000369890300047
  • Yuezhong Wu, Thomas Weise, and Raymond Chiong. Local Search for the Traveling Salesman Problem: A Comparative Study. In Proceedings of the 14th IEEE Conference on Cognitive Informatics & Cognitive Computing (ICCI*CC'15), July 6-8, 2015, Beijing, China, pages 213-220, Los Alamitos, CA, USA: IEEE Computer Society Press, ISBN: 978-1-4673-7289-3.
    doi:10.1109/ICCI-CC.2015.7259388 / pdf
    Indexing: EI:20161202119039, WOS:000380466100034
  • Thomas Weise and Raymond Chiong. An Alternative Way of Presenting Statistical Test Results when Evaluating the Performance of Stochastic Approaches. Neurocomputing, 147:235-238, January, 2015.
    doi:10.1016/j.neucom.2014.06.071 / pdf
    Indexing: EI:20143600022800, WOS:000343337600020, SCI, 2区, CCF-C类
  • Yan Jiang, Thomas Weise, Jörg Lässig, Raymond Chiong, and Rukshan Athauda. Comparing a Hybrid Branch and Bound Algorithm with Evolutionary Computation Methods, Local Search and their Hybrids on the TSP. In Proceedings of the IEEE Symposium on Computational Intelligence in Production and Logistics Systems (CIPLS'14), Proceedings of the IEEE Symposium Series on Computational Intelligence (SSCI'14), Orlando, FL, USA: Caribe Royale All-Suite Hotel and Convention Center, December 9-12, 2014, pages 148-155. Los Alamitos, CA, USA: IEEE Computer Society Press. ISBN 978-1-4799-5375-2.
    doi:10.1109/CIPLS.2014.7007174 / pdf
    Indexing: EI:20150700522268, WOS:000380487400021
  • Thomas Weise, Raymond Chiong, Ke Tang, Jörg Lässig, Shigeyoshi Tsutsui, Wenxiang Chen, Zbigniew Michalewicz, and Xin Yao. Benchmarking Optimization Algorithms: An Open Source Framework for the Traveling Salesman Problem. IEEE Computational Intelligence Magazine (CIM), 9(3):40-52, August 2014.
    Featured article and selected paper at the website of the IEEE Computational Intelligence Society (http://cis.ieee.org/).
    doi:10.1109/MCI.2014.2326101 / pdf
    Indexing: EI:20143117995681, WOS:000340271800004, SCI, 1区, Google Scholar
  • Thomas Weise, Brian M. Blake, and Steffen Bleul. Semantic Web Service Composition: The Web Service Challenge Perspective. In Athman Bouguettaya, Quan Z. Sheng, and Florian Daniel, editors, Web Services Foundations – Part 1: Foundations of Web Services, chapter 7, pages 161-187. Secaucus, NJ, USA: Springer-Verlag New York, Inc., first online September 2013 / Copyright 2014. ISBN: 978-1-4614-7517-0.
    doi:10.1007/978-1-4614-7518-7_7 / pdf
    Indexing: EI:20172603853294
  • Ke Tang, Zhenyu Yang, and Thomas Weise. Special Session on Evolutionary Computation for Large Scale Global Optimization at 2012 IEEE World Congress on Computational Intelligence (CEC@WCCI-2012). Technical Report, Hefei, Anhui, China: University of Science and Technology of China (USTC), School of Computer Science and Technology, Nature Inspired Computation and Applications Laboratory (NICAL), June 14, 2012.
  • Ajay Bansal, Srividya Bansal, Brian M. Blake, Steffen Bleul, and Thomas Weise. Overview of the Web Services Challenge (WSC): Discovery and Composition of Semantic Web Services. In Brian M. Blake, Liliana Cabral, Birgitta König-Ries, Ulrich Küster, and David Martin, editors, Semantic Web Services – Advancement through Evaluation, chapter 19, pages 297-312. April 2012, Berlin/Heidelberg: Springer-Verlag. ISBN: 978-3-642-28734-3.
    doi:10.1007/978-3-642-28735-0_19
    Indexing: EI:20172803915654
  • Brian M. Blake, Thomas Weise, and Steffen Bleul. WSC-2010: Web Services Composition and Evaluation. In Proceedings of the IEEE International Conference on Service-Oriented Computing and Applications (SOCA'10), pages 1-4, Perth, WA, Australia, December 13–15, 2010. ISBN: 978-1-4244-9802-4, Los Alamitos, CA, USA: IEEE Computer Society Press.
    doi:10.1109/SOCA.2010.5707190
    Indexing: EI:20111113747701
  • Thomas Weise, Li Niu, and Ke Tang. AOAB – Automated Optimization Algorithm Benchmarking. In Proceedings of the 12th Annual Conference Companion on Genetic and Evolutionary Computation (GECCO'10), pages 1479-1486, Portland, OR, USA: Portland Marriott Downtown Waterfront Hotel, July 7-11, 2010. ISBN: 978-1-4503-0073-5, New York, NY, USA: ACM Press.
    doi:10.1145/1830761.1830763
    Indexing: EI:20103513190754, WOS:000322071400001, CCF-C类
  • Ke Tang, Xiaodong Li, Ponnuthurai Nagaratnam Suganthan, Zhenyu Yang, and Thomas Weise. Benchmark Functions for the CEC'2010 Special Session and Competition on Large-Scale Global Optimization. Technical Report, Hefei, Anhui, China: University of Science and Technology of China (USTC), School of Computer Science and Technology, Nature Inspired Computation and Applications Laboratory (NICAL), January 8, 2010.
  • Srividya Kona, Ajay Bansal, Brian M. Blake, Steffen Bleul, and Thomas Weise. WSC-2009: A Quality of Service-Oriented Web Services Challenge. In Birgit Hofreiter, editor, Proceedings of the 11th IEEE Conference on Commerce and Enterprise Computing (CEC'09), pages 487-490, Vienna, Austria: Vienna University of Technology, July 20–23, 2009. ISBN: 978-0-7695-3755-9, Piscataway, NJ, USA: IEEE Computer Society and Red Hook, NY, USA: Curran Associates, Inc.
    doi:10.1109/CEC.2009.80
    Indexing: EI:20094712467206, Google Scholar
  • Steffen Bleul, Thomas Weise, and Kurt Geihs. The Web Service Challenge – A Review on Semantic Web Service Composition. Electronic Communications of the EASST, 17(204):1-13, Special Issue: Michael Wagner, Dieter Hogrefe, Kurt Geihs, and Klaus David, editors, “Kommunikation in Verteilten Systemen 2009”, KiVS'09, Proceedings, Workshops der Wissenschaftlichen Konferenz Kommunikation in Verteilten Systemen 2009 (WowKiVS 2009), Kassel, Hesse, Germany: University of Kassel, Fachbereich 16: Elektrotechnik/Informatik, March 5, 2009. Potsdam, Germany: European Association of Software Science and Technology (EASST; Universität Potsdam, Institute for Informatics).
    doi:10.14279/tuj.eceasst.17.204
  • Ajay Bansal, Brian M. Blake, Srividya Kona, Steffen Bleul, Thomas Weise, and Michael C. Jäger. WSC-08: Continuing the Web Services Challenge. In Proceedings of IEEE Joint Conference on E-Commerce Technology (10th CEC) and Enterprise Computing, E-Commerce and E-Services (5th EEE) (CEC/EEE'08), pages 351-354, Washington, DC, USA, July 21–24, 2008. ISBN: 978-0-7695-3340-7, Piscataway, NJ, USA: IEEE Computer Society.
    doi:10.1109/CECandEEE.2008.146
    Indexing: EI:20091512027446, WOS:000265327500048, Google Scholar
  • Thomas Weise, Stefan Niemczyk, Hendrik Skubch, Roland Reichle, and Kurt Geihs. A Tunable Model for Multi-Objective, Epistatic, Rugged, and Neutral Fitness Landscapes. In Maarten Keijzer, Giuliano Antoniol, Clare Bates Congdon, Kalyanmoy Deb, Benjamin Doerr, Nikolaus Hansen, John H. Holmes, Gregory S. Hornby, Daniel Howard, James Kennedy, Sanjeev P. Kumar, Fernando G. Lobo, Julian Francis Miller, Jason H. Moore, Frank Neumann, Martin Pelikan, Jordan B. Pollack, Kumara Sastry, Kenneth Owen Stanley, Adrian Stoica, El-Ghazali, and Ingo Wegener, editors, Proceedings of the 10th Annual Conference on Genetic and Evolutionary Computation Conference (GECCO'08), pages 795-802, July 12-16, 2008, Renaissance Atlanta Hotel Downtown: Atlanta, GA, USA. ISBN: 978-1-60558-130-9, New York, NY, USA: ACM Press.
    doi:10.1145/1389095.1389252 / pdf / slides / new implementation [Java+Maven+JUnit] / original implementation [Java]
    Indexing: EI:20085111786051, CCF-C类
  • Thomas Weise, Steffen Bleul, and Kurt Geihs. Web Service Composition Systems for the Web Service Challenge – A Detailed Review. Kasseler Informatikschriften (KIS) 2007, 7, Kassel, Hesse, Germany: University of Kassel, Fachbereich 16: Elektrotechnik/Informatik, November 19, 2007.
    urn:nbn:de:hebis:34-2007111919638

Organized Events

  • Anna V. Kononova, Hao Wang, Thomas Weise, Carola Doerr, Thomas Bäck, Fabio Caraffini, and Johann Dreo. Analysing Algorithmic Behaviour of Optimisation Heuristics Workshop (AABOH'21) at the 2021 Genetic and Evolutionary Computation Conference (GECCO'21), July 10-14, 2021 in Lille, France
    website / Call for Papers (CfP)
  • Jürgen Branke, Carola Doerr, Tome Eftimov, Pascal Kerschke, Manuel López-Ibáñez, Boris Naujoks, Luís Paquete, Vanessa Volz, and Thomas Weise. Good Benchmarking Practices for Evolutionary Computation (BENCHMARKING'21) workshop at the 2021 Genetic and Evolutionary Computation Conference (GECCO'21), July 10-14, 2021 in Lille, France.
    website 1, website 2, website 2
  • Thomas Weise, Aleš Zamuda, and Markus Wagner. Special Session on Benchmarking of Computational Intelligence Algorithms (BOCIA'21) at the 2021 IEEE Congress on Evolutionary Computation (CEC 2021), June 28-July 1, 2021 in Kraków, Poland
    website / Call for Papers (CfP)
  • Thomas Bäck, Thomas Bartz-Beielstein, Jakob Bossek, Bilel Derbel, Carola Doerr, Tome Eftimov, Pascal Kerschke, William La Cava, Arnaud Liefooghe, Manuel López-Ibáñez, Boris Naujoks, Pietro S. Oliveto, Patryk Orzechowski, Mike Preuss, Jérémy Rapin, Ofer M. Shir, Olivier Teytaud, Heike Trautmann, Ryan J. Urbanowicz, Vanessa Volz, Markus Wagner, Hao Wang, Thomas Weise, Borys Wróbel, and Aleš Zamuda. Good Benchmarking Practices for Evolutionary Computation (BENCHMARK@PPSN) Workshop at the Sixteenth International Conference on Parallel Problem Solving from Nature (PPSN XVI), September 5-9, 2020 in Leiden, The Netherlands.
    website / Call for Papers (CfP)
  • Thomas Bäck, Carola Doerr, Tome Eftimov, Pascal Kerschke, William La Cava, Manuel López-Ibáñez, Boris Naujoks, Pietro S. Oliveto, Patryk Orzechowski, Mike Preuss, Jérémy Rapin, Ofer M. Shir, Olivier Teytaud, Heike Trautmann, Ryan J. Urbanowicz, Vanessa Volz, Markus Wagner, Hao Wang, Thomas Weise, Borys Wróbel, and Aleš Zamuda. Good Benchmarking Practices for Evolutionary Computation (BENCHMARK@GECCO) Workshop at the Genetic and Evolutionary Computation Conference (GECCO 2020), July 8-12, 2020, Cancún, Quintana Roo, Mexico.
    website / Call for Papers (CfP) / slides@google: 1,2,3 / slides@iao: 1,3
  • Carola Doerr, Pietro S. Oliveto, Thomas Weise, Borys Wróbel, and Aleš Zamuda. Black Box Discrete Optimization Benchmarking (BB-DOB) Workshop at the Genetic and Evolutionary Computation Conference (GECCO 2019), July 13-17, 2019, Prague, Czech Republic.
    website / Call for Papers (CfP)
  • Markus Wagner, Thomas Weise, and Aleš Zamuda. Special Session on Benchmarking of Evolutionary Algorithms for Discrete Optimization (BEADO) at the 2019 IEEE Congress on Evolutionary Computation (CEC'19), June 10-13, 2019 in Wellington, New Zealand.
    website / Call for Papers (CfP)
  • Thomas Weise, Bin Li, Markus Wagner, Xingyi Zhang, and Jörg Lässig, eds. Special Issue on Benchmarking of Computational Intelligence Algorithms published in the Applied Soft Computing journal published by Elsevier B.V. and indexed by EI and SCIE. The special issue was open for submissions from July 2018 and closed in April 14, 2019. It is a virtual special issues, where papers will be published as soon as they are accepted.
    website / CfP-website / Call for Papers (CfP)
  • Pietro S. Oliveto, Markus Wagner, Thomas Weise, Borys Wróbel, and Aleš Zamuda. Black-Box Discrete Optimization Benchmarking (BB-DOB@PPSN) Workshop at the Fifteenth International Conference on Parallel Problem Solving from Nature (PPSN XV), 8-9 September 2018, Coimbra, Portugal
    website / Call for Papers (CfP)
  • Pietro S. Oliveto, Markus Wagner, Thomas Weise, Borys Wróbel, and Aleš Zamuda. Black-Box Discrete Optimization Benchmarking (BB-DOB@GECCO) Workshop at the 2018 Genetic and Evolutionary Computation Conference (GECCO 2018), July 15-19, 2018 in Kyoto, Japan
    website / Call for Papers (CfP)
  • Thomas Weise, Bin Li, Markus Wagner, Xingyi Zhang, and Jörg Lässig. International Workshop on Benchmarking of Computational Intelligence Algorithms (BOCIA) at the Tenth International Conference on Advanced Computational Intelligence (ICACI 2018), March 29-31, 2018 in Xiamen [厦门], Fujian [福建省], China, IEEE, ISBN: 978-1-5386-4362-4.
    website / Call for Papers (CfP)
  • Thomas Weise and Jörg Lässig. SITA-UBRI Joint Workshop on Sustainable Logistics, November 5, 2015, Hefei, Anhui, China
  • Thomas Weise and Jörg Lässig. Special Session on Benchmarking and Testing for Production and Logistics Optimization of the 2014 IEEE Symposium on Computational Intelligence in Production and Logistics at the 2014 IEEE Symposium Series on Computational Intelligence (SSCI 2014), December 9-12, 2014, Orlando, Florida, USA
  • M. Brian Blake, Thomas Weise, and Steffen Bleul. 2010 Web Service Challenge (WSC-2010) on Semantic Web Service Composition with Quality-of-Service at the Conference on Commerce and Enterprise Computing (CEC), November 11, 2010, Fudan University, Shanghai, China

Funding

  • Domaine d’Intérêt Majeur, Réseau Francilien en Sciences Informatiques (DIM-RFSI). AlgoSelect: Automated Algorithm Selection for Discrete Black-Box Optimization, 2019/2020. Principal Investigator: Carola Doerr (LIP6, Sorbonne Université). Project partners: Benjamin Doerr (LIX, École Polytechnique), Johann Dreo (Thales Research & Technology, Palaiseau), Pascal Kerschke (University of Münster, Germany), Olivier Teytaud (Facebook Artificial Intelligence Lab, Paris), and Thomas Weise (Institute of Applied Optimization at Hefei University, China).
  • National Natural Science Foundation of China (NSFC), Regular Project "Research on the Automatic Performance Analysis of Optimization,"  2017 to 2020, Principal Investigator is Dr. Thomas Weise
  • The University of Newcastle in Callaghan, NSW, Australia, Faculty Strategic Initiatives Research Fund (SIRF) Grant "A Framework for Next Generation Algorithm Benchmarking: Performance Testing and Community Building," 2014, Principal Investigators were Dr. Raymond Chiong, Dr. Thomas Weise, and Dr. Rukshan Athauda
  • The University of Newcastle in Callaghan, NSW, Australia, Faculty Strategic Initiatives Research Fund (SIRF) Grant "Intelligent Transportation Planning: Benchmarking of Novel Business Analytics Techniques using the Travelling Salesman Problem as a Test-Bed," 2013, Principal Investigators were Dr. Raymond Chiong, Dr. Thomas Weise, Dr. David Cornforth, and Prof. Zbigniew Michalewicz

Lectures and Courses

News

  1. International Workshop on Benchmarking of Computational Intelligence Algorithms (BOCIA)
  2. Program for Workshop on Benchmarking of Computational Intelligence Algorithms (BOCIA) at ICACI 2018 Released
  3. BB-DOB @ GECCO: Submission Deadline Extended to April 3, 2018
  4. Submission Deadline Extended to December 1, 2017: International Workshop on Benchmarking of Computational Intelligence Algorithms (BOCIA)
  5. Black-Box Discrete Optimization Benchmarking (BB-DOB@PPSN) Workshop Accepted at PPSN XV
  6. Black-Box Discrete Optimization Benchmarking (BB-DOB@PPSN) Workshop
  7. Black-Box Discrete Optimization Benchmarking (BB-DOB@GECCO) Workshop Accepted at GECCO 2018
  8. Black-Box Discrete Optimization Benchmarking (BB-DOB@GECCO) Workshop
  9. International Workshop on Benchmarking of Computational Intelligence Algorithms (BOCIA)
  10. Workshop on Benchmarking of Computational Intelligence Algorithms approved for ICACI 2018
  11. Research Talk at the Ludwig-Maximilians-Universität Munich in Munich (München), Germany
  12. Research Talk at the Johannes Gutenberg University Mainz in Mainz, Germany
  13. Research Talk at the University of Cologne in Köln (Cologne), Germany
  14. Research Talk at the University of Leipzig in Leipzig, Germany
  15. Research Talk at the University of Applied Sciences Zittau/Görlitz in Görlitz, Germany
  16. Research Talk at the Otto von Guericke University Magdeburg in Magdeburg, Germany
  17. Research Talk at the Clausthal University of Technology in Clausthal-Zellerfeld, Germany
  18. Research Talk at the Technische Universität Ilmenau in Ilmenau, Germany
  19. Research Talk at the Technische Universität Ilmenau in Ilmenau, Germany
  20. Research Talk at the University of Applied Sciences Mittweida in Mittweida, Germany
  21. Research Talk at the Chemnitz University of Technology in Chemnitz, Germany
  22. Research Talk at the Chemnitz University of Technology in Chemnitz, Germany
  23. Research Talk at the Friedrich Schiller University in Jena, Germany
  24. Research Talk: Automating Scientific Research in Optimization
  25. From Standardized Data Formats to Standardized Tools for Optimization Algorithm Benchmarking
  26. New Beta Release of our optimizationBenchmarking.org Software for Automating Research in Optimization
  27. Two Papers accepted at GECCO 2017

Popular Tags

Latest Articles