Open Switch

A deployment can look cheap and healthy until a run of failures or a latency spike changes the tradeoff. The routing core is deterministic and separate from HTTP, so the browser demo can exercise the same decision model without contacting a provider.

The axum gateway accepts OpenAI-compatible chat requests, lists configured models, emits SSE for streaming responses, and exposes plain-text request and latency metrics.

A deployment breaker starts closed, opens after consecutive failures, and later admits a limited half-open probe set. Successful probes return it to closed. The simulation exposes that state for every deployment after a run.

Latency is an exponentially weighted moving average. Unlike a fixed window, the alpha makes the reaction speed explicit. Cost-aware routing selects the cheapest healthy choice within a configured latency budget.

When the first request crosses the hedge delay, the gateway starts the next deployment and uses the first completed answer. Tokio cancels the losing attempt when the request future is dropped. The simulation reports hedge count and modeled tail latency saved.

Bearer keys have token ceilings per minute. Model groups and prices stay in deployment configuration, and mock, OpenAI-compatible, and Anthropic adapters share the request shape at the gateway boundary.