diff --git a/spec/router-openapi.yaml b/spec/router-openapi.yaml index fca146e..55dcffe 100644 --- a/spec/router-openapi.yaml +++ b/spec/router-openapi.yaml @@ -105,6 +105,12 @@ paths: $ref: '#/components/headers/RouterRequestIdHeader' Idempotent-Replayed: $ref: '#/components/headers/RouterIdempotentReplayedHeader' + X-Committed-Spend-Limit: + $ref: '#/components/headers/CommittedSpendLimitHeader' + X-Committed-Spend-Current: + $ref: '#/components/headers/CommittedSpendCurrentHeader' + X-Committed-Spend-Remaining: + $ref: '#/components/headers/CommittedSpendRemainingHeader' content: application/json: schema: @@ -496,7 +502,7 @@ components: name: Idempotency-Key in: header required: false - description: 'Caller-generated key that makes retrying ONE logical call safe. A call that reached the caller with an answer is recorded against its key for 24 hours, and a retry carrying the same key is answered from that record instead of dispatching - and charging - the provider a second time, marked `Idempotent-Replayed: true`. Keys are scoped to the workspace your credential carries, or to your user when it carries none - so the keyspace is SHARED by every member of a workspace rather than private to one caller. Make a key unique across the whole workspace, not just within your own client: a second member who reuses a key string is answered from the first member''s record, or refused `409` if the request differs. Because the scope follows the CREDENTIAL and not the person, a credential that carries no workspace at all scopes to your user id instead - so retrying one logical call under a different credential can land in a different namespace, where it is dispatched and charged again. Retry with the credential you started with. A keyed request with no authenticated caller is refused `401`. The guarantee is a BILLING one: a key is charged at most once. It is not a promise that a key is dispatched at most once, and it does not make a lost call resumable. Some answers are RECORDED but not replayable for the full 24 hours, and the billing guarantee is the half that always holds: the key stays consumed - the retry never re-runs and never re-charges - but it is answered `409 invalid_input` instead of being served the original body. That happens whenever Comfy does not hold a copy of the response it can still stand behind; a response past the replay size cap and a result addressed by an asset URL Comfy does not host are the two you are most likely to meet. The second is the one worth planning for, because it looks like an ordinary success. Models whose results are assets are re-hosted onto Comfy storage before they are returned, and where a model''s output schema says that re-hosting is per entry - the xAI Grok Imagine image family says so explicitly - an entry Comfy could not re-host carries the PARTNER''s own short-lived URL, which carries no promise of outliving the record. On a model that returns its result on the original call - the xAI Grok Imagine image family again - that answer is still replayed for a few minutes, which is where a dropped connection puts an SDK''s automatic same-key re-send and while the partner''s link is certainly still alive; after that it is refused rather than replayed dead. So a prompt retry of a partially re-hosted result behaves exactly like any other replay, and only a later one meets the `409`. That short window is deliberately NOT offered on a model that submits and is polled, because there the partner may have minted the URL long before your call collected it and its remaining life is unknowable - and those models do not need it: a call cut off mid-generation keeps its key holding the generation, so the same-key retry collects the ORIGINAL result rather than a recorded copy of it. A response past the size cap has no window either and is refused from the start. The action on any of these `409 invalid_input` refusals is the same: use a new key. Only an answer a provider actually produced is recorded, though. A refusal Router raises on its own BEFORE dispatching anything - not enabled for you yet (`403`), unknown model (`404`), not entitled to the model (`403`), a body the model''s schema rejects or that names a different model than the path (`422`), a malformed request (`400 invalid_input`) - dispatched nothing and charged nothing, so it RELEASES the key: re-send the SAME key once you are on the rollout ramp or have corrected the request and it runs for real, rather than replaying the refusal or colliding with it as a `409`. That turns on whether a provider was reached, NEVER on the status, so a `400 content_policy_violation` - the partner''s own answer to a call that ran, which some models meter - is recorded and replayed like any other answer. Releasing a refusal that dispatched nothing frees nothing chargeable, so it does not weaken the at-most-once billing guarantee above.' + description: 'Caller-generated key that makes retrying ONE logical call safe. A call that reached the caller with an answer is recorded against its key for 24 hours, and a retry carrying the same key is answered from that record instead of dispatching - and charging - the provider a second time, marked `Idempotent-Replayed: true`. Keys are scoped to the workspace your credential carries, or to your user when it carries none - so the keyspace is SHARED by every member of a workspace rather than private to one caller. Make a key unique across the whole workspace, not just within your own client: a second member who reuses a key string is answered from the first member''s record, or refused `409` if the request differs. Because the scope follows the CREDENTIAL and not the person, a credential that carries no workspace at all scopes to your user id instead - so retrying one logical call under a different credential can land in a different namespace, where it is dispatched and charged again. Retry with the credential you started with. A keyed request with no authenticated caller is refused `401`. The guarantee is a BILLING one: a key is charged at most once. It is not a promise that a key is dispatched at most once, and it does not make a lost call resumable. Some answers are RECORDED but not replayable for the full 24 hours, and the billing guarantee is the half that always holds: the key stays consumed - the retry never re-runs and never re-charges - but it is answered `409 invalid_input` instead of being served the original body. That happens whenever Comfy does not hold a copy of the response it can still stand behind; a response past the replay size cap and a result addressed by an asset URL Comfy does not host are the two you are most likely to meet. The second is the one worth planning for, because it looks like an ordinary success: which models answer with a Comfy-hosted asset link, how long one stays valid, and what a result carries when an individual asset could not be copied are stated in one place, under Result assets in the API reference, and this paragraph does not restate them. On a model that returns its result on the original call, an answer still holding a partner''s own asset link is replayed for a few minutes - which is where a dropped connection puts an SDK''s automatic same-key re-send, and while the partner''s link is certainly still alive - and refused after that rather than replayed dead. So a prompt retry of a partially re-hosted result behaves exactly like any other replay, and only a later one meets the `409`. That short window is deliberately NOT offered on a model that submits and is polled, because there the partner may have minted the URL long before your call collected it and its remaining life is unknowable - and those models do not need it: a call cut off mid-generation keeps its key holding the generation, so the same-key retry collects the ORIGINAL result rather than a recorded copy of it. A response past the size cap has no window either and is refused from the start. The action on any of these `409 invalid_input` refusals is the same: use a new key. Only an answer a provider actually produced is recorded, though. A refusal Router raises on its own BEFORE dispatching anything - not enabled for you yet (`403`), unknown model (`404`), not entitled to the model (`403`), a body the model''s schema rejects or that names a different model than the path (`422`), a malformed request (`400 invalid_input`) - dispatched nothing and charged nothing, so it RELEASES the key: re-send the SAME key once you are on the rollout ramp or have corrected the request and it runs for real, rather than replaying the refusal or colliding with it as a `409`. That turns on whether a provider was reached, NEVER on the status, so a `400 content_policy_violation` - the partner''s own answer to a call that ran, which some models meter - is recorded and replayed like any other answer. Releasing a refusal that dispatched nothing frees nothing chargeable, so it does not weaken the at-most-once billing guarantee above.' schema: type: string minLength: 1 @@ -518,7 +524,7 @@ components: $ref: '#/components/schemas/RouterProviderSegment' headers: CommittedSpendCurrentHeader: - description: The USD cents the caller currently has committed to calls still in flight, not counting the refused call. Present alongside `X-Committed-Spend-Limit`. + description: The USD cents the caller currently has committed to calls still in flight. On a `429` this EXCLUDES the refused call, whose commitment was rolled back before the refusal was sent; on an admitted response it INCLUDES the call being answered. Present alongside `X-Committed-Spend-Limit`. required: false schema: type: integer @@ -526,7 +532,7 @@ components: minimum: 0 example: 9600 CommittedSpendLimitHeader: - description: The ceiling, in USD cents, on the partner spend the caller may have committed to calls still in flight. Present when the refusal was the committed-spend ceiling rather than the concurrent-call pool. + description: 'The ceiling, in USD cents, on the partner spend the caller may have committed to calls still in flight - money held from the moment a call is admitted and released when that call finishes. It is not a budget, a balance, or any running total of what the caller has spent to date: settling an invoice frees no room under it, and letting an in-flight call finish does. Contrast `X-Concurrency-Limit`, which bounds those same in-flight calls counted as a NUMBER OF CALLS rather than priced. How the ceiling is SIZED is a separate question from what it measures, and it is not tier-independent: the ceiling moves with the account''s lifetime paid spend, off the same thresholds the concurrent-call tier uses, so paying more raises it - see [partner-node concurrency limits](https://docs.comfy.org/tutorials/partner-nodes/concurrency-limits) for that ladder and for the concurrent-call bound that shares this `429`. Present on BOTH outcomes of an enforcing committed-spend gate - the `429` it raises and the success it admits - and absent while the gate is not enforcing, when it declines to decide and lets the call through, or on a `429` raised by the concurrent-call pool instead (a committed-spend `429` carries this trio and drops `X-Concurrency-*`).' required: false schema: type: integer