The short answer: a ₹/GPU-hour rate and a ₹/million-token price measure different things, and converting between them depends on a number that no pricing page states for you, which is sustained throughput on your own model. Two GPUs with the same hourly rate can differ several-fold in cost per token, and two very different hourly rates can land at a similar cost per token. This guide shows the arithmetic and the inputs that drive it.
The conversion, step by step
Cost per million tokens equals the hourly rate divided by the tokens served per hour, scaled to a million. Tokens per hour is throughput in tokens per second multiplied by 3,600, multiplied again by utilisation, meaning the fraction of billed hours in which the GPU is actually serving requests. Three inputs, then: the hourly rate, measured throughput, and utilisation.
The example below uses illustrative round numbers. They are not any provider's price and not a benchmark of any GPU or model; they exist to show how the formula behaves. Substitute live rates from the compare table and your own measured throughput.
- Baseline. A hypothetical ₹100/GPU-hour rate and 1,000 tokens per second sustained. That is 3.6 million tokens per hour at full utilisation, or about ₹28 per million tokens.
- Same GPU, 25% utilisation. Billed hours are the same but only a quarter carry traffic, so the effective cost rises fourfold to about ₹111 per million tokens.
- Double the throughput. Through batching or a more efficient serving stack, 2,000 tokens per second at the same rate halves the cost to about ₹14 per million tokens.
Utilisation and throughput move the result by more than the difference between most providers' hourly rates. This is why comparing rates alone can mislead, a point developed in what a ₹/GPU-hour figure hides.
What determines throughput
Throughput is not a property of the GPU model alone. Model size and precision set how much memory must be read for every generated token, so a quantised model typically serves faster on the same card. Batch size matters because serving several requests together spreads each memory read across more tokens. Context length and output length change how much work each request needs. Prompt processing and token generation also behave differently, so a workload with long prompts and short answers does not match one with short prompts and long answers. Because of these interactions, a throughput figure from a public benchmark is a starting hypothesis, not a substitute for a test on your own traffic.
Why utilisation is the quiet variable
A dedicated instance bills whether or not requests arrive. Traffic that peaks during business hours and idles overnight leaves a dedicated GPU well below full utilisation, and that idle share is paid for at the full hourly rate. Metering also matters for bursty serving; see the notes on per-second and per-hour billing. Where request volume is low or unpredictable, a per-token API can cost less than a dedicated GPU even if its nominal per-token price is higher, and the reverse tends to hold at steady, high volume. The crossover point depends on your own numbers, which is why a break-even calculation is more useful than a general rule. The comparison is worked through more fully in the guide to self-hosting an LLM versus using an API.
Costs outside the per-token figure
A cost-per-token estimate covers compute only. Storage for model weights, data transfer, and engineering time to run and monitor a serving stack sit outside it; egress structures are covered in the data-transfer cost notes. For Indian buyers, GST on the invoice adds to the base rate, and whether input tax credit is available depends on the invoice, as set out in the GST invoicing guide. If prompts or outputs contain personal data, DPDP obligations attach to how and where that data is processed, independent of the pricing model.
A short method for your own estimate
- Pick the model and precision you would actually deploy.
- Load-test it on a rented instance with realistic prompt and output lengths, and record sustained tokens per second at your target concurrency.
- Estimate the share of billed hours that will carry traffic, using a realistic daily pattern rather than a peak.
- Take the live hourly rate for that GPU from its page, for example L4 or H100, and compute cost per million tokens with the formula above.
- Repeat for two or three GPU options and for a per-token API, and compare at low, expected and high traffic.
Bottom line
Cost per token is the hourly rate divided by real, utilised throughput. Rate is the input you can read off a page; throughput and utilisation are the inputs you have to measure. The cost calculator can be used to test different hourly rates and usage patterns against your own numbers.