Credit and billing

What you pay

Each request pays for its tokens at the model's prices, with a small minimum. Waiting in the queue is free, and failed requests cost nothing. Two more charges cover the GPU when it isn't running your tokens:

Start fee: the request that starts a worker pays this once.
Idle fee: a request that reuses a worker which has been idle pays for that idle time by the second, up to a limit. Gaps under a second are free.

See pricing for the amounts.

Reservations

Credit is reserved before a request runs: the most its tokens could cost, a start fee and a full idle fee. You pay what it actually used, and the rest returns immediately. Setting max_tokens lowers the reservation. If a client hangs up mid-stream, the token count never arrives, so that request pays for the GPU time it used instead, by the second, and never more than its tokens could have cost.

Auto-reload

Auto-reload charges your saved card when credit drops below a threshold you choose, up to a monthly limit you choose. If a request is short of credit and auto-reload is on, it waits a few seconds while your card is charged, then runs. Turn it off and requests simply stop with a 402 when credit runs out.

Spend limits

A monthly spend limit caps usage whatever your credit, on the account, a project or a key. Reaching one stops requests with 402 spend_limit_reached.

Monthly plans

Each month's fee is added to your credit. Once a calendar month's usage has passed the fee, token prices are discounted, down to the model's minimum per request. Start and idle fees aren't discounted. Changing plan applies from the next renewal. Cancelling takes effect at the end of the month paid for, and your credit stays usable.

With the API

All of this is in the console. With an admin key:

POST /v1/billing/top-up adds credit
POST /v1/billing/settings sets auto_reload, reload_threshold_usd, reload_amount_usd and monthly_limit_usd
POST /v1/billing/spend-limit caps monthly usage
POST /v1/billing/plan and DELETE /v1/billing/plan start and cancel a plan
POST /v1/billing/portal manages your card and receipts