feat: retry LLM API calls once after 401 by force-refreshing the OAuth token

The Client trait's default chat_completions, chat_completions_streaming,
and embeddings methods now classify failures via ApiStatusError: on a
401 with a cached OAuth token, the token is distrusted (identity-aware
marker) and the call retried exactly once — the retry's prepare step
sees the marker and force-refreshes. Streaming retries only while the
SSE handler has received no content, preventing duplicate rendering.
A second 401 propagates the original error; other retry errors
propagate as-is. API-key clients never retry. No backoff by design:
cost is bounded to one refresh + one retry per failing request.
This commit is contained in:
2026-08-14 13:17:56 -06:00
parent 684f19250a
commit 2596194417
3 changed files with 202 additions and 9 deletions
-1
View File
@@ -63,7 +63,6 @@ pub fn set_access_token(
/// not it matched `rejected`) — i.e. the client is token-authed and a retry
/// after refresh is worthwhile. Returns false when there is no entry
/// (API-key clients).
#[allow(dead_code)] // Called by the 401-retry path once it lands.
pub fn distrust_access_token(client_name: &str, rejected: &str) -> bool {
let mut access_tokens = ACCESS_TOKENS.write();
let (token, _, _) = match access_tokens.get(client_name) {