An experiment console for judging harness changes

Pi · Herdr · metadata only

Black Signal Lab turns trace metadata into a compact Herdr popup. It makes the active experiment, outcomes, cost, tokens, time, and optional scores comparable without exposing prompt or response content.

pi · main @ 9691711production renderer · representative QA data

SurfaceThree panelsOverview · Comparison · Recent

Data boundaryMetadata onlyTrace metrics, outcomes, optional scores

HomePi + HerdrTransient popup, no permanent pane

Overview

orient before comparing

The opening panel establishes the active experiment and the health of the current run set before deeper inspection.

Black Signal Lab Overview panel — Active experiment, run count, pass rate, cost, tokens, duration, and the current Herdr state in one orientation view.

Comparison

baseline against candidate

Arm-level aggregates keep the experiment decision visible: outcome quality alongside the operational cost of producing it.

Black Signal Lab Comparison panel — A side-by-side comparison of experiment arms using trace metadata and optional evaluation scores.

Recent runs

inspect without reading content

The recent view makes individual outcomes selectable and filterable while maintaining the content boundary.

Black Signal Lab Recent panel — Recent experiment runs with metadata-only status and metric context for follow-up.

The privacy boundary is part of the product

Lab does not read prompts, responses, tool arguments, tool results, credentials, or environment values. It operates on trace metrics, outcomes, and optional scores.

Building out

Lab’s core loop now covers labels, outcomes, filters, selection, arm comparison, optional scores, dashboard handoff, and deep links. Around it, Black Signal is becoming a broader control plane: Cockpit for live diagnostics, Review for guarded JJ/GitHub work, Recipes for repeatable layouts, and Signal Links for routing experiments and changes into the right panel.

Evidence boundary: production renderer at Pi commit 9691711. Representative QA data is shown because the publishing host had no local trace corpus.