Benchmark explorer

Read the conditions before the scores

Dispatch-200 runs both policies on the same 200 support tickets under the same limits. One thing changes: the route.

Test card

Dispatch-200

Conditions held equal

Test conditions for the Baseline and Budgeted policies on Dispatch-200
Condition	Baseline	Same	Budgeted
Workload	200 tickets	same	200 tickets
Input limit	2,048 tokens	same	2,048 tokens
Output limit	256 tokens	same	256 tokens
Rubric	fixed, /100	same	fixed, /100
Latency	p95, gateway	same	p95, gateway
Energy	est. accelerator	same	est. accelerator

Routethe one variable

200 Gannet-L

differs

150 Gannet-S, 50 Gannet-L

Switch the policy, keep the workload

Each cell is one request of 200. Each figure sits against its policy limit, with the other policy as a faint marker.

policy support-summariesDispatch-200n = 200

Baseline

All 200 requests to Gannet-L

Over the energy ceiling

Dispatch ledger25 requests per row

Per-route breakdown

Baseline requests and estimated energy per route
Route	Requests	Wh est. each	Wh est.
Gannet-S	0	0.20	0
Gannet-L	200	0.40	80
Batch	200		80

200 × 0.40=80 Wh est.

Every request takes the larger route, so the batch estimate lands 20 Wh above the 60 Wh ceiling.

Rubric scorescored

92/100

Floor ≥ 89

Baseline scores 92 out of 100, above the floor of 89. Budgeted scores 90.
p95 latencymeasured

1.6s

Ceiling ≤ 1.8 s

Baseline p95 latency 1.6 seconds, under the 1.8 second ceiling. Budgeted 1.4 seconds.
Batch energyest.

80Wh

20 Wh over ≤ 60

Baseline estimated batch energy 80 watt hours, 20 above the 60 watt hour ceiling. Budgeted 50.

Budgeted

150 to Gannet-S, 50 to Gannet-L

Meets all three limits

Dispatch ledger25 requests per row, grouped by route

Per-route breakdown

Budgeted requests and estimated energy per route
Route	Requests	Wh est. each	Wh est.
Gannet-S	150	0.20	30
Gannet-L	50	0.40	20
Batch	200		50

150 × 0.20 + 50 × 0.40=50 Wh est.

Three in four requests take the smaller route. Gannet-L carries the other 50, at twice the estimate each.

Rubric scorescored

90/100

Floor ≥ 89

Budgeted scores 90 out of 100, above the floor of 89. Baseline scores 92.
p95 latencymeasured

1.4s

Ceiling ≤ 1.8 s

Budgeted p95 latency 1.4 seconds, under the 1.8 second ceiling. Baseline 1.6 seconds.
Batch energyest.

50Wh

Ceiling ≤ 60 Wh

Budgeted estimated batch energy 50 watt hours: 30 on Gannet-S and 20 on Gannet-L, under the 60 watt hour ceiling. Baseline 80.

Gannet-S
Gannet-L
Other policy
est.Estimated, accelerator energy only

What was held equal, and what was not measured

Sample size first, then scope. The result below belongs to this test and these conditions only.

Requests per policy: 200
Requests in all: 400
Sorted time read as p95: 190of 200

Held equal for both policies

The same 200 tickets, sent in the same order to both policies.
The same limits: 2,048 tokens in and 256 tokens out per request.
The same fixed rubric, scored out of 100 for every summary.
The same measuring point: p95 latency at the gateway.
The same energy method: estimated accelerator energy per request.

The result, with its scope

Dispatch-200 only

37.5%

lower estimated batch energy for this test, with both policies above the 89 floor.

(80 − 50) ÷ 80=0.375

Estimated batch energy on a 0 to 120 watt hour axis: Baseline 80, Budgeted 50, ceiling 60. Facility overhead is not measured and would add an unknown amount to both.

Not measured in either run

Facility overheadCooling and power conversion sit outside the energy estimate.
Network beyond the gatewayClient time on either side of the gateway is not in the p95.
Other workloadsThe 37.5% does not carry over without a new run.

Five terms, each defined once

Every figure on this page uses these definitions. Open one to see what it counts, its values here, and what it leaves out.

01 Rubric score scored points /100

What it counts

Each summary is scored out of 100 against a fixed rubric that does not change between policies. The batch score is the mean of the 200 scores.

Baseline: 92
Budgeted: 90
Floor: ≥ 89

Leaves outQualities the rubric does not name. A score on another rubric is not comparable.

02 p95 latency measured seconds, p95

What it counts

Time from a request reaching the gateway to its full response leaving it, measured for every request. The p95 is the 190th of the 200 times, sorted fastest first.

Baseline: 1.6 s
Budgeted: 1.4 s
Ceiling: ≤ 1.8 s

Leaves outClient network time on either side of the gateway.

03 Energy estimate estimated Wh est. per batch

What it counts

Estimated accelerator energy per request, from the hardware, its utilization and the run time, summed over the batch. Gannet-S is estimated at 0.20 Wh a request, Gannet-L at 0.40 Wh.

Baseline: 80 Wh est.
Budgeted: 50 Wh est.
Ceiling: ≤ 60 Wh

Leaves outFacility overhead, such as cooling and power conversion. It is not measured, so it is not in any figure.

04 Routes configured model per request

What it counts

The models a policy may send a request to, here Gannet-S and Gannet-L. The route is the one thing that differs between the two policies in this test.

Baseline: 200 Gannet-L
Budgeted: 150 S, 50 L
Fallback: named route

When none fitsThe policy uses the fallback the team named, or reports the exception before the request runs.

05 Workload fixed set 200 requests

What it counts

Dispatch-200: 200 support ticket summaries, run once under each policy with the same input and output limits.

Tickets: 200
Input: 2,048 tokens
Output: 256 tokens

Leaves outOther workloads. A new workload needs its own run before any figure applies to it.

A still highland tarn at first light, a moss-green ridge mirrored in the water

Every figure here has a written method

How the sample was drawn, where latency is timed and how each energy estimate is built, in one document a review can check.

scoredrubric, fixed for both policies
measuredp95 latency, at the gateway
estimatedaccelerator energy, overhead excluded

Read the reporting methodology