Optimization is a technology which has the potential to become even more important than data mining and big data are now, as it can improve the efficiency of each and every aspect of an enterprises operation and products. Optimization means to find approximate solutions for hard problems. Optimization algorithms can find short routes in logistic planning scenarios, construction plans for work pieces that require only little amounts of material, or efficient schedules in production planning. They can help to reduce costs and pollution at the same time.

The term "optimization algorithms" implies a plural, and there are already many different families of algorithms, including exact methods, heuristics, metaheuristics sub-families such as evolutionary computation and swarm intelligence, as well as local search and hybrid algorithms. Each of these families contains dozens of general and specialized algorithms. We want to know how we can find out which method is best for which problem and when and why. First, using the best algorithm will give us the best solutions, which is what we want. Knowing why an algorithm is best could help us to create even better ones. Thus, benchmarking of optimization algorithms in a statistically sound, robust, and ideally automatic way seems to be an important topic to me. Yet it is entirely under-represented in literature. We want to emphasize how important it is, as for practical applications of optimization, we simply need to know that we are using the best and most reliable methods.

1. Projects

  • the optimizationBenchmarking.org, a framework for comparing and researching the performance of any optimization algorithm implemented in any programming language on any optimization problem
  • the TSP Suite, a framework for investigating the performance of solvers for the Traveling Salesman Problem (TSP), with a fast collection of state-of-the-art TSP solvers implemented in Java

2. Publications

  • Qi Qi, Thomas Weise, and Bin Li. Modeling Optimization Algorithm Runtime Behavior and its Applications. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO'17) Companion, July 15-19, 2017, Berlin, Germany, New York, NY, USA: ACM Press.
    accepted for publication as short paper
    doi:10.1145/3067695.3076042

  • Weichen Liu, Thomas Weise, Yuezhong Wu, and Qi Qi. Combining Two Local Searches with Crossover: An Efficient Hybrid Algorithm for the Traveling Salesman Problem. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO'17), July 15-19, 2017, Berlin, Germany, New York, NY, USA: ACM Press.
    accepted for publication as full paper
    doi:10.1145/3071178.3071201

  • Thomas Weise. From Standardized Data Formats to Standardized Tools for Optimization Algorithm Benchmarking. In Proceedings of the 16th IEEE Conference on Cognitive Informatics & Cognitive Computing (ICCI*CC'17), July 26-28, 2017, University of Oxford, Oxford, UK. Los Alamitos, CA, USA: IEEE Computer Society Press.
    accepted as full paper.

  • Thomas Weise, Yuezhong Wu, Raymond Chiong, Ke Tang, and Jörg Lässig. Global versus Local Search: The Impact of Population Sizes on Evolutionary Algorithm Performance. Journal of Global Optimization 66(3):511-534, November 2016.
    doi:10.1007/s10898-016-0417-5 / pdf

  • Yuezhong Wu, Thomas Weise, and Weichen Liu. Hybridizing Different Local Search Algorithms with Each Other and Evolutionary Computation: Better Performance on the Traveling Salesman Problem. In Proceedings of the 18th Genetic and Evolutionary Computation Conference (GECCO'16), Denver, Colorado, USA, July 20–24, 2016, pages 57-58, New York, NY, USA: Association for Computing Machinery (ACM). ISBN: 978-1-4503-4323-7
    doi:10.1145/2908961.2909001 / pdf / poster

  • Weichen Liu, Thomas Weise, Yuezhong Wu, Dan Xu, and Raymond Chiong. Hybrid Ejection Chain Methods for the Traveling Salesman Problem. Journal of Computational and Theoretical Nanoscience 13(6):3601-3610, June 2016
    doi:10.1166/jctn.2016.5189

  • Weichen Liu, Thomas Weise, Yuezhong Wu, and Raymond Chiong. Hybrid Ejection Chain Methods for the Traveling Salesman Problem. In Proceedings of the 10th International Conference on Bio-Inspired Computing – Theories and Applications (BIC-TA'15), Maoguo Gong, Linqiang Pan, Tao Song, Ke Tang, and Xingyi Zhang, editors, September 25-28, 2015, Hefei, Anhui, China, volume 562 of Communications in Computer and Information Science. Berlin/Heidelberg: Springer-Verlag, pages 268-282, ISBN 978-3-662-49013-6.
    doi:10.1007/978-3-662-49014-3_25 / pdf

  • Dan Xu, Thomas Weise, Yuezhong Wu, Jörg Lässig, and Raymond Chiong. An Investigation of Hybrid Tabu Search for the Traveling Salesman Problem. In Proceedings of the 10th International Conference on Bio-Inspired Computing – Theories and Applications (BIC-TA'15), Maoguo Gong, Linqiang Pan, Tao Song, Ke Tang, and Xingyi Zhang, editors, September 25-28, 2015, Hefei, Anhui, China, volume 562 of Communications in Computer and Information Science. Berlin/Heidelberg: Springer-Verlag, pages 523-537, ISBN 978-3-662-49013-6.
    doi:10.1007/978-3-662-49014-3_47 / pdf

  • Yuezhong Wu, Thomas Weise, and Raymond Chiong. Local Search for the Traveling Salesman Problem: A Comparative Study. In Proceedings of the 14th IEEE Conference on Cognitive Informatics & Cognitive Computing (ICCI*CC'15), July 6-8, 2015, Beijing, China, pages 213-220, Los Alamitos, CA, USA: IEEE Computer Society Press, ISBN: 978-1-4673-7289-3.
    doi:10.1109/ICCI-CC.2015.7259388 / pdf

  • Thomas Weise and Raymond Chiong. An Alternative Way of Presenting Statistical Test Results when Evaluating the Performance of Stochastic Approaches. Neurocomputing, 147:235-238, January, 2015.
    doi:10.1016/j.neucom.2014.06.071 / pdf

  • Yan Jiang, Thomas Weise, Jörg Lässig, Raymond Chiong, and Rukshan Athauda. Comparing a Hybrid Branch and Bound Algorithm with Evolutionary Computation Methods, Local Search and their Hybrids on the TSP. In Proceedings of the IEEE Symposium on Computational Intelligence in Production and Logistics Systems (CIPLS'14), Proceedings of the IEEE Symposium Series on Computational Intelligence (SSCI'14), Orlando, FL, USA: Caribe Royale All-Suite Hotel and Convention Center, December 9-12, 2014, pages 148-155. Los Alamitos, CA, USA: IEEE Computer Society Press. ISBN 978-1-4799-5375-2.
    doi:10.1109/CIPLS.2014.7007174 / pdf

  • Thomas Weise, Raymond Chiong, Ke Tang, Jörg Lässig, Shigeyoshi Tsutsui, Wenxiang Chen, Zbigniew Michalewicz, and Xin Yao. Benchmarking Optimization Algorithms: An Open Source Framework for the Traveling Salesman Problem. IEEE Computational Intelligence Magazine (CIM), 9(3):40-52, August 2014.
    Featured article and selected paper at the website of the IEEE Computational Intelligence Society (http://cis.ieee.org/).
    doi:10.1109/MCI.2014.2326101 / pdf

  • Thomas Weise, Brian M. Blake, and Steffen Bleul. Semantic Web Service Composition: The Web Service Challenge Perspective. In Athman Bouguettaya, Quan Z. Sheng, and Florian Daniel, editors, Web Services Foundations – Part 1: Foundations of Web Services, chapter 7, pages 161-187. ISBN 978-1-4614-7517-0, Secaucus, NJ, USA: Springer-Verlag New York, Inc., September 2013.
    doi:10.1007/978-1-4614-7518-7_7 / pdf

  • Ke Tang, Zhenyu Yang, and Thomas Weise. Special Session on Evolutionary Computation for Large Scale Global Optimization at 2012 IEEE World Congress on Computational Intelligence (CEC@WCCI-2012). Technical Report, Hefei, Anhui, China: University of Science and Technology of China (USTC), School of Computer Science and Technology, Nature Inspired Computation and Applications Laboratory (NICAL), June 14, 2012.

  • Ajay Bansal, Srividya Bansal, Brian M. Blake, Steffen Bleul, and Thomas Weise. Overview of the Web Services Challenge (WSC): Discovery and Composition of Semantic Web Services. In Brian M. Blake, Liliana Cabral, Birgitta König-Ries, Ulrich Küster, and David Martin, editors, Semantic Web Services – Advancement through Evaluation, chapter 19, pages 297-312. April 2012, Berlin/Heidelberg: Springer-Verlag. ISBN: 978-3-642-28734-3.
    doi:10.1007/978-3-642-28735-0_19

  • Brian M. Blake, Thomas Weise, and Steffen Bleul. WSC-2010: Web Services Composition and Evaluation. In Proceedings of the IEEE International Conference on Service-Oriented Computing and Applications (SOCA'10), pages 1-4, Perth, WA, Australia, December 13–15, 2010. ISBN: 978-1-4244-9802-4, Los Alamitos, CA, USA: IEEE Computer Society Press.
    doi:10.1109/SOCA.2010.5707190

  • Thomas Weise, Li Niu, and Ke Tang. AOAB – Automated Optimization Algorithm Benchmarking. In Proceedings of the 12th Annual Conference Companion on Genetic and Evolutionary Computation (GECCO'10), pages 1479-1486, Portland, OR, USA: Portland Marriott Downtown Waterfront Hotel, July 7-11, 2010. ISBN: 978-1-4503-0073-5, New York, NY, USA: ACM Press.
    doi:10.1145/1830761.1830763

  • Ke Tang, Xiaodong Li, Ponnuthurai Nagaratnam Suganthan, Zhenyu Yang, and Thomas Weise. Benchmark Functions for the CEC'2010 Special Session and Competition on Large-Scale Global Optimization. Technical Report, Hefei, Anhui, China: University of Science and Technology of China (USTC), School of Computer Science and Technology, Nature Inspired Computation and Applications Laboratory (NICAL), January 8, 2010.

  • Srividya Kona, Ajay Bansal, Brian M. Blake, Steffen Bleul, and Thomas Weise. WSC-2009: A Quality of Service-Oriented Web Services Challenge. In Birgit Hofreiter, editor, Proceedings of the 11th IEEE Conference on Commerce and Enterprise Computing (CEC'09), pages 487-490, Vienna, Austria: Vienna University of Technology, July 20–23, 2009. ISBN: 978-0-7695-3755-9, Piscataway, NJ, USA: IEEE Computer Society and Red Hook, NY, USA: Curran Associates, Inc..
    details / doi:10.1109/CEC.2009.80

  • Steffen Bleul, Thomas Weise, and Kurt Geihs. The Web Service Challenge – A Review on Semantic Web Service Composition. Electronic Communications of the EASST, 17(204):1-13, Special Issue: Michael Wagner, Dieter Hogrefe, Kurt Geihs, and Klaus David, editors, “Kommunikation in Verteilten Systemen 2009”, KiVS'09, Proceedings, Workshops der Wissenschaftlichen Konferenz Kommunikation in Verteilten Systemen 2009 (WowKiVS 2009), Kassel, Hesse, Germany: University of Kassel, Fachbereich 16: Elektrotechnik/Informatik, March 5, 2009. Potsdam, Germany: European Association of Software Science and Technology (EASST; Universität Potsdam, Institute for Informatics).
    doi:10.14279/tuj.eceasst.17.204

  • Ajay Bansal, Brian M. Blake, Srividya Kona, Steffen Bleul, Thomas Weise, and Michael C. Jäger. WSC-08: Continuing the Web Services Challenge. In Proceedings of IEEE Joint Conference on E-Commerce Technology (10th CEC) and Enterprise Computing, E-Commerce and E-Services (5th EEE) (CEC/EEE'08), pages 351-354, Washington, DC, USA, July 21–24, 2008. ISBN: 978-0-7695-3340-7, Piscataway, NJ, USA: IEEE Computer Society.
    doi:10.1109/CECandEEE.2008.146

  • Thomas Weise, Stefan Niemczyk, Hendrik Skubch, Roland Reichle, and Kurt Geihs. A Tunable Model for Multi-Objective, Epistatic, Rugged, and Neutral Fitness Landscapes. In Maarten Keijzer, Giuliano Antoniol, Clare Bates Congdon, Kalyanmoy Deb, Benjamin Doerr, Nikolaus Hansen, John H. Holmes, Gregory S. Hornby, Daniel Howard, James Kennedy, Sanjeev P. Kumar, Fernando G. Lobo, Julian Francis Miller, Jason H. Moore, Frank Neumann, Martin Pelikan, Jordan B. Pollack, Kumara Sastry, Kenneth Owen Stanley, Adrian Stoica, El-Ghazali, and Ingo Wegener, editors, Proceedings of the 10th Annual Conference on Genetic and Evolutionary Computation Conference (GECCO'08), pages 795-802, July 12-16, 2008, Renaissance Atlanta Hotel Downtown: Atlanta, GA, USA. ISBN: 978-1-60558-130-9, New York, NY, USA: ACM Press.
    doi:10.1145/1389095.1389252 / pdf / slides / implementation [Java]

  • Thomas Weise, Steffen Bleul, and Kurt Geihs. Web Service Composition Systems for the Web Service Challenge – A Detailed Review. Kasseler Informatikschriften (KIS) 2007, 7, Kassel, Hesse, Germany: University of Kassel, Fachbereich 16: Elektrotechnik/Informatik, November 19, 2007.
    urn:nbn:de:hebis:34-2007111919638

3. Organized Events

  • Thomas Weise and Jörg Lässig. 1st SITA-UBRI Joint Workshop on Sustainable Logistics, November 5, 2015, Hefei, Anhui, China

  • Thomas Weise and Jörg Lässig. Special Session on Benchmarking and Testing for Production and Logistics Optimization of the 2014 IEEE Symposium on Computational Intelligence in Production and Logistics at the 2014 IEEE Symposium Series on Computational Intelligence (SSCI 2014), December 9-12, 2014, Orlando, Florida, USA

  • M. Brian Blake, Thomas Weise, and Steffen Bleul. 2010 Web Service Challenge (WSC-2010) on Semantic Web Service Composition with Quality-of-Service at the Conference on Commerce and Enterprise Computing (CEC), November 11, 2010, Fudan University, Shanghai, China

4. News

  1. New Beta Release of our optimizationBenchmarking.org Software for Automating Research in Optimization