The book is divided into two parts: foundations and applications. The first part introduces the foundations of benchmarking as a discipline, covering the three fundamental elements of each benchmarking approach: metrics, workloads, and measurement methodology. The second part focuses on different application areas, presenting contributions in specific fields of benchmark development. These contributions address the unique challenges that arise in the conception and development of benchmarks for specific systems or subsystems, and demonstrate how the foundations and concepts in the first part of the book are being used in existing benchmarks. Further,the book presents a number of concrete applications and case studies based on input from leading benchmark developers from consortia such as the Standard Performance Evaluation Corporation (SPEC) and the Transaction Processing Performance Council (TPC).

Part I Foundations.- 1 Benchmarking Basics.- 2 Review of Basic Probability and Statistics.- 3 Metrics.- 4 Statistical Measurements.- 5 Experimental Design.- 6 Measurement Techniques.- 7 Operational Analysis and Basic Queueing Models.- 8 Workloads.- 9 Standardization.- Part II Applications.- 10 The SPEC CPU Benchmark Suite.- 11 Benchmarking the Energy Efficiency of Servers.- 12 Virtualization Benchmarks.- 13 Storage Benchmarks.- 14 TeaStore: A Micro-Service Reference Application for Research Use.- 15 Elasticity of Cloud Platforms.- 16 Performance Isolation.- 17 Resource Demand Estimation.- 18 Software and System Security.