Class peagen.plugins.evaluators.simple_time.SimpleTimeEvaluator

peagen.plugins.evaluators.simple_time.SimpleTimeEvaluator dataclass

SimpleTimeEvaluator()

Bases: Evaluator

Median wall-clock time of a command (lower is better).

last_result class-attribute instance-attribute

last_result = field(default_factory=dict, init=False)

run

run(workspace, bench_cmd, runs=1, **kw)

Source code in peagen/plugins/evaluators/simple_time.py

def run(self, workspace: Path, bench_cmd: str, runs: int = 1, **kw: Any) -> float:
    durations: List[float] = []
    for _ in range(max(1, runs)):
        start = time.perf_counter()
        subprocess.run(
            bench_cmd,
            shell=True,
            cwd=workspace,
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE,
        )
        durations.append((time.perf_counter() - start) * 1000)

    median_ms = statistics.median(durations)
    self.last_result = {
        "runs": len(durations),
        "times_ms": [round(t, 3) for t in durations],
        "median_ms": round(median_ms, 3),
    }
    return -median_ms