The most common mistake Indian ML teams make when renting GPU compute is treating all GPU workloads as the same. They're not. Running a RAG pipeline, fine-tuning a model, and training from scratch each have different memory requirements, different GPU characteristics that matter, and correspondingly different price points in the Indian market. Getting this wrong typically costs 3–10x more per useful output than getting it right.
RAG inference: the VRAM-bound workload
A retrieval-augmented generation pipeline has two GPU-intensive stages: embedding documents into vectors, and generating answers using retrieved context with the LLM. Embedding models are lightweight — under 1 GB of VRAM for popular choices like all-MiniLM-L6-v2. The LLM inference stage dominates the memory budget.
As a rule of thumb, an LLM needs roughly 2 GB of VRAM per billion parameters at FP16, or about 0.5 GB per billion at INT4, plus 15–20% on top for KV cache and framework overhead. A typical RAG pipeline with a quantised 7–8B model uses 6–8 GB total — fitting on any GPU with 16 GB VRAM. For a production setup running a 13B model with 16K context, 24–40 GB is the comfortable range.
What matters for RAG is VRAM capacity first, then memory bandwidth. RAG inference is memory-bandwidth-bound at generation time, not compute-bound. You don't need the highest-FLOPS GPU — you need one that moves model weights from VRAM to GPU cores efficiently. This is why the L40S often outperforms raw TFLOPS comparisons would suggest for inference.
In India's market right now: E2E Networks' L4 at ₹49/hr handles 7–13B quantised models well for single-user or low-concurrency RAG. For multi-user RAG serving a 13–30B model, the L40S at ₹75–102/hr from RunPod or E2E Networks is the practical minimum. An H100 is overkill for pure RAG unless you're serving at meaningful concurrency.
Fine-tuning: the memory-and-time workload
Fine-tuning simultaneously stores model weights, gradients, and optimizer states — roughly 4–6× the inference memory requirement for the same model. A 7B model that runs comfortably in 14 GB for inference needs 40–50 GB for LoRA fine-tuning, and 80–100 GB for full fine-tuning.
For fine-tuning, VRAM capacity matters more than bandwidth, and NVLink becomes relevant if you're spanning multiple GPUs. For most Indian teams fine-tuning a 7–13B model with LoRA, an A100 80GB is the practical minimum for a comfortable run. At ₹113–189/hr from RunPod or E2E Networks, an A100 80GB is the price benchmark for LoRA fine-tuning.
For 70B LoRA fine-tuning you need minimum 2× A100 80GB (160 GB total) — roughly ₹226–360/hr. Full fine-tuning of a 70B model needs 4–8× H100s. Fine-tuning is also time-bounded, not concurrency-bounded: a 7B LoRA run on a single A100 might finish in 2–8 hours. You rent the GPU for that duration and stop, making per-hour pricing the critical number.
Full pre-training: a different category entirely
Pre-training from scratch requires storing weights (~140 GB for a 70B model at FP16), gradients (~140 GB), Adam optimizer states (~280 GB), and activations. The total for a 70B model exceeds 570 GB — requiring 8× H100s minimum with tensor and pipeline parallelism. At ₹362/hr per H100 from E2E Networks, an 8-GPU cluster costs roughly ₹2,900/hr. For most Indian teams, continued pre-training (starting from an existing checkpoint and running on domain data) is far more practical at ₹15,000–50,000 for a meaningful run.
The India pricing decision matrix
- RAG inference, 7–13B model: L4 at ₹28–49/hr (Leapswitch or E2E Networks). For multi-user: L40S at ₹75–102/hr.
- RAG inference, 30–70B model: A100 80GB at ₹113–178/hr. The memory bandwidth justifies the step-up.
- LoRA fine-tuning, 7–13B: A100 80GB (₹113/hr RunPod or ₹189/hr E2E). Extra VRAM allows larger batches and faster convergence than the 40GB variant.
- LoRA fine-tuning, 70B: Minimum 2× A100 80GB. Expect ₹226–360/hr depending on provider.
- Full pre-training / continued pre-training: H100 at ₹191–362/hr depending on provider. Multi-node only.
Use the cost calculator to model total spend for a specific training duration, and the GPU recommender to filter by VRAM and workload type.
Frequently asked questions
Can I run a RAG pipeline on an L4 in India?
Yes. An L4 (24 GB) handles quantised 13B models plus an embedding model with room for 8–16K context. E2E Networks offers the L4 at ₹49/hr, making it one of the most cost-effective RAG inference options in India. If you need longer context or higher concurrency, step up to an L40S or A100 80GB.
Which Indian provider is best for fine-tuning?
RunPod offers the lowest per-hour A100 80GB rate at ~₹113/hr, but bills in USD and doesn't provide a GST invoice. E2E Networks at ₹189/hr provides INR billing with a GST-compliant invoice and MeitY empanelment, which matters for government and regulated-sector workloads. Use the compare tool to model the total cost including effective GST for both.
Is it cheaper to use a hosted API or run my own GPU for RAG?
For low-volume RAG (under 10M tokens/month), a hosted API like DeepSeek V4 Flash at $0.14/1M input tokens is almost always cheaper than renting a GPU. The break-even shifts around 50–100M tokens/month for a 7B model, where GPU rental costs drop below API costs per token at reasonable concurrency. See the LLM API pricing table for current rates.