Benchmarking — measuring the speed of code under controlled conditions

Intermediate5 min read·eng-20-008

interviewperformance

Concept

Benchmarking — measuring the performance of a specific piece of code or operation under controlled conditions. The goal is to compare approaches or verify that performance meets requirements.

Benchmarking vs profiling:

Benchmarking: "Is approach A faster than approach B?" or "Does this function meet our 50ms target?" Isolated measurement.
Profiling: "Where is the application spending its time overall?" Whole-system measurement.

What makes a good benchmark:

Warm up: Run the code a few times before measuring to warm up JIT, OPcache, CPU caches.
Many iterations: Run hundreds or thousands of times, take the average (or median).
Controlled environment: Same hardware, no other processes consuming CPU.
Measure the right thing: Don't include setup code in the timed section.
Statistical significance: Report mean, standard deviation, min/max — not just average.

PHP benchmarking tools:

PHPBench: Dedicated benchmarking framework. Run thousands of iterations, report mean time per iteration.
microtime(true): Quick manual timing for one-off measurements.
Benchmark PHP: php -d opcache.enable_cli=1 -r "..." for micro-benchmarks.

Common pitfalls:

Micro-benchmark fallacy: "Function X is 3x faster!" — but in production, the bottleneck is I/O, not that function.
Optimizing the wrong thing: Always profile first, then benchmark the slow part.
Comparing incomparable things: Benchmark in the same environment, same PHP version, same data size.

Code Example

php

<?php
// PHPBENCH — proper benchmarking framework
// composer require phpbench/phpbench --dev

/**
 * @BeforeMethods({"setUp"})
 * @Iterations(5)
 * @Revs(1000)
 * @Warmup(10)
 */
class StringBench
{
    private array $data;

    public function setUp(): void
    {
        $this->data = range(1, 100);
    }

    /** @Subject */
    public function bench_implode(): void
    {
        implode(',', $this->data);
    }

    /** @Subject */
    public function bench_join(): void
    {
        join(',', $this->data); // alias for implode
    }

    /** @Subject */
    public function bench_sprintf(): void
    {
        $result = '';
        foreach ($this->data as $item) {
            $result .= $item . ',';
        }
        rtrim($result, ',');
    }
}

// Run: ./vendor/bin/phpbench run benchmarks/ --report=aggregate
// Output:
// benchmark           subject          mean      stdev     diff
// StringBench         bench_implode    0.62μs    0.05μs    1.00x
// StringBench         bench_join       0.63μs    0.04μs    1.02x
// StringBench         bench_sprintf    3.84μs    0.12μs    6.19x

// MANUAL TIMING — quick and dirty
function benchmarkApproach(callable $fn, int $iterations = 1000): float
{
    // Warm up
    for ($i = 0; $i < 10; $i++) $fn();

    // Measure
    $start = microtime(true);
    for ($i = 0; $i < $iterations; $i++) $fn();
    $elapsed = (microtime(true) - $start) * 1000;

    return $elapsed / $iterations; // ms per iteration
}

$timeA = benchmarkApproach(fn() => array_map(fn($x) => $x * 2, range(1, 100)));
$timeB = benchmarkApproach(fn() => array_reduce(range(1, 100), fn($c, $x) => [...$c, $x * 2], []));

printf("array_map: %.4fms\n", $timeA);
printf("array_reduce: %.4fms\n", $timeB);
printf("Difference: %.1fx\n", $timeB / $timeA);

PreviousProfiling — measuring where time and memory are actually spent in codeVocabulary — Performance NextBottleneck — the single slowest part of a system that limits total throughputVocabulary — Performance