API integration

Route the requests you already send

Point your client at one endpoint and name a policy. Every response returns its route, measured latency and an energy estimate.

Request access How each figure is produced

Endpoint: POST /v1/route
Body: Your model request
Returns: Output and metadata

POST/v1/route

Request

{ "policy": "support-summaries", "messages": [ { "role": "user", "content": "…" } ], "max_tokens": 300}

200 routed to gannet-s

latency_ms: 1180; measured
p95_budget: within; ≤ 1.8 s
energy_wh_est: 0.20; est.

Two lines change in your client

Keep the body, the retries and the timeouts. Send the call to the gateway and name a policy.

The endpoint
Call /v1/route on the gateway instead of your provider's URL.
The policy id
Name the policy where the model name was. It picks the route for each request.
Everything else stays
Messages, token limits and sampling settings pass to the route as you wrote them.

How routing policies are written

+2−2

−Removed: curl "$PROVIDER_URL/v1/chat" \+Added: curl "$JOULIS_GATEWAY/v1/route" \1 -H "Authorization: Bearer $API_KEY" \ -H "Content-Type: application/json" \ -d '{−Removed:  "model": "gannet-l",+Added:  "policy": "support-summaries",2 "messages": [ { "role": "system", "content": "Summarize in three sentences." }, { "role": "user", "content": "Ticket 4812: export stops at 500 rows" } ], "max_tokens": 300, "temperature": 0.2 }'

import os, requests−Removed: resp = requests.post(f"{PROVIDER_URL}/v1/chat",+Added: resp = requests.post(f"{JOULIS_GATEWAY}/v1/route",1 headers={"Authorization": f"Bearer {os.environ['API_KEY']}"}, json={−Removed:  "model": "gannet-l",+Added:  "policy": "support-summaries",2 "messages": [ {"role": "system", "content": "Summarize in three sentences."}, {"role": "user", "content": ticket_text}, ], "max_tokens": 300, }, timeout=10,)meta = resp.json()["joulis"]

const res = await fetch(−Removed:  `${PROVIDER_URL}/v1/chat`,+Added:  `${JOULIS_GATEWAY}/v1/route`,1 { method: "POST", headers: { Authorization: `Bearer ${process.env.API_KEY}`, "Content-Type": "application/json", }, body: JSON.stringify({−Removed:  model: "gannet-l",+Added:  policy: "support-summaries",2 messages: [ { role: "system", content: "Summarize in three sentences." }, { role: "user", content: ticketText }, ], max_tokens: 300, }), },);const { joulis } = await res.json();

Responsethe model output as before, plus one joulis object

What comes back with every response

One joulis object beside the model output. Each field says where it comes from: your policy, the gateway or an estimate.

Route and policy

from your policy

routestring"gannet-s": The route that ran this request.
policystring"support-summaries": The policy id you sent, returned as is.
rubric_floorinteger89: The quality floor the route qualified against, on the fixed rubric.
fallback_usedbooleanfalse: True only when your named fallback ran.

Measured at the gateway

measured

latency_msinteger1180: Gateway time for this request, from receipt to last token.
p95_msinteger1400: The policy’s rolling p95 over its last 1,000 requests.
p95_budgetstring"within": "within" or "over" the latency ceiling.

Estimated energy

est.

energy_wh_estnumber0.20: Accelerator energy for this request, in watt hours.
energy_estimatebooleantrue: Always true. Energy is estimated, never metered.
energy_scopestring"accelerator": What the estimate covers. Facility overhead is left out.

exceeded_budgets: []string[]

Every budget this request went over: "quality_floor", "latency_p95" or "energy". Empty when it fit. Overruns are reported, never dropped.

How each figure is produced

Errors

Refused before anything runs

A policy exception is decided before any model call. The body names every candidate route and the budget it misses.

executed: falseNo model was called, so no inference was spent.
candidatesEach eligible route, with every budget it misses and by how much.
fallback: nullNo fallback named, so the request returns to your service.

422policy_exception

Before execution

{ "error": { "type": "policy_exception", "policy": "support-summaries-fast", "executed": false,1 "candidates": [2 { "route": "gannet-l", "misses": [ { "budget": "p95_ms", "value": 1600, "limit": 1200 }, { "budget": "energy_wh", "value": 80, "limit": 60, "est": true } ] }, { "route": "gannet-s", "misses": [ { "budget": "p95_ms", "value": 1300, "limit": 1200 } ] } ], "fallback": null3 }}

404Nothing ran

unknown_policy

The policy id is not defined in this workspace. Check the spelling or publish the policy first.
503Nothing ran

provider_unavailable

No qualifying route’s provider answered and the policy names no fallback. Retry after the stated delay.
200Fallback ran

notice: provider_unavailable

The provider did not answer and your named fallback ran. The response says so with fallback_used: true.

With a fallback, or without one

When no route qualifies, the policy decides: run the fallback you named, or return the request unexecuted.

No route qualifies for support-summaries-fast

Policy sets"fallback": "gannet-l"

200The fallback runs

"joulis": { "route": "gannet-l", "policy": "support-summaries-fast", "fallback_used": true, "latency_ms": 1540, "p95_budget": "over", "energy_wh_est": 0.40, "exceeded_budgets": ["latency_p95"]}

Your service gets an answer and a record of the overrun, ready for your own reporting.

Policy sets"fallback": null

422The request returns unexecuted

"error": { "type": "policy_exception", "policy": "support-summaries-fast", "message": "No route meets this policy", "executed": false, "candidates": [ … ], "fallback": null}

Your service decides what happens next: relax a budget, retry later or hold the rollout.

Route the requests you already send

The endpoint

The policy id

Everything else stays