PerformanceToolStabil

Benchmark Runner

Performance-Benchmarks mit statistischer Analyse, Regressionserkennung und CI-Integration.

Benchmark schreiben

benchmarks/sort_bench.velinvelin
use bench

@Benchmark
fn bench_quicksort() {
    let data = generate_random_array(10000);
    
    bench.iter(|| {
        quicksort(data.clone())
    });
}

@Benchmark
fn bench_mergesort() {
    let data = generate_random_array(10000);
    
    bench.iter(|| {
        mergesort(data.clone())
    });
}

@Benchmark(iterations = 1000)
fn bench_binary_search() {
    let sorted = (0..10000).collect();
    
    bench.iter(|| {
        binary_search(sorted, 5000)
    });
}

Ausführen

# Alle Benchmarks ausführen
velin-bench run

# Bestimmte Benchmarks
velin-bench run --filter "sort"

# Mit Baseline vergleichen
velin-bench run --baseline main

# JSON-Output
velin-bench run --format json > results.json

Beispiel-Output

Benchmark Results
=================

bench_quicksort
  Time:   2.34 ms ± 0.12 ms
  Ops/s:  427.35 ops/sec
  Memory: 1.2 MB peak

bench_mergesort
  Time:   3.12 ms ± 0.08 ms
  Ops/s:  320.51 ops/sec
  Memory: 2.4 MB peak

bench_binary_search
  Time:   0.45 µs ± 0.02 µs
  Ops/s:  2,222,222 ops/sec
  Memory: 0 B allocated

Comparison vs baseline (main):
  bench_quicksort:    -5.2% faster ✓
  bench_mergesort:    +2.1% slower ⚠
  bench_binary_search: unchanged

Features

Warmup

Automatisches Aufwärmen vor Messung

Statistik

Mean, Median, Std Dev, Percentiles

Regression

Erkennt Performance-Verschlechterungen

Vergleich

Vergleich zwischen Git-Branches

CPU Cycles

Hardware Performance Counters

CI Gates

Blockiert Merges bei Regression

Konfiguration

velin.tomltoml
[tools.benchmark]
# Standardanzahl Iterationen
iterations = 100

# Warmup-Durchläufe
warmup = 10

# Regressions-Schwellwert (%)
regression_threshold = 5.0

# Baseline-Branch
baseline_branch = "main"

# Output-Format
format = "pretty" # oder "json", "csv"

# Benchmark-Verzeichnis
bench_dir = "benchmarks/"

CI Integration

.github/workflows/bench.ymlyaml
- name: Run Benchmarks
  run: |
    velin-bench run --baseline origin/main --format json > results.json
    
- name: Check Regression
  run: |
    velin-bench check --threshold 5% --fail-on-regression
    
- name: Upload Results
  uses: actions/upload-artifact@v3
  with:
    name: benchmark-results
    path: results.json
ProfilerSicherheit