Colab vs Kaggle vs Paid GPU for Fine-Tuning Small Models: What Fits

Free notebooks can fine-tune a small model if the job is a QLoRA run on a 7B or smaller base, with short sequences, checkpointed often. They cannot do bf16 LoRA on a 7B, or anything that must finish without interruption. Below is a way to decide before you waste an evening: estimate the memory, find out what the session really gives you, make the run resumable, and know the point where renting is cheaper than fighting limits.

I checked Google's documentation for Colab on 2026-10-09. Kaggle's documentation pages would not load for me (they render in the browser), so everything about Kaggle below is secondary and says so.

What does Colab officially promise?

Almost nothing, and it says so. These points come from Colab's FAQ (research.google.com/colaboratory/faq.html), which I read as plain text on 2026-10-09:

• Colab "does not publish" its usage limits, because they "can vary over time". Overall limits, idle timeouts, maximum VM lifetime and GPU types all vary.
• The types of GPU and TPU available "vary over time". Premium GPUs are available "subject to availability" on paid plans.
• Free notebooks can run for at most 12 hours, "depending on availability and your usage patterns". Colab Pro+ supports continuous execution for up to 24 hours "if you have sufficient compute units".
• Runtimes time out if idle. Virtual machines are deleted when idle and have a maximum lifetime.
• Paid users whose compute unit balance runs out revert to the free tier's restrictions.
• Colab recommends closing tabs and not selecting a GPU unless you need one, to reduce the chance of hitting limits.
• Colab Pro for Education is "no longer available for new signups".

The FAQ sends you to a sign-up page for prices, and that page rendered only a title for me (a second pricing URL returned 403). So I cannot give you a verified Colab price list. Third-party listings I found by search on the same day agree on a pay-as-you-go rate of $9.99 for 100 compute units, but they disagree about how many units Pro+ includes and I could not confirm any of it on a Google page. Check the plan page yourself before budgeting.

The practical meaning: you cannot plan a free-tier job around a guaranteed GPU, hour count or session length. You can plan around "may be interrupted".

What about Kaggle?

From secondary sources only (search results on 2026-10-09, including reviews that cite Kaggle's notebook docs): sessions are reported as 12 hours for CPU and GPU, 9 for TPU. The weekly GPU quota is reported by users at around 30 hours, but one review says that figure is not in the docs and the real number is shown in your account settings. Hardware reported: a single P100 (16 GB) or two T4s, listed in the notebook's accelerator menu.

The part to remember about two T4s: with a data-parallel setup each card holds its own copy of the model, so you do not get a 32 GB card. You would need to shard the model (for example with device_map="auto" or FSDP) to use both, and I did not test either on Kaggle.

Which fine-tuning jobs fit a free 16 GB card?

Use the VRAM calculator with a safety margin. I ran its estimates for the longest sequence (batch size 1, rank 16, all linear layers, gradient checkpointing) that fits 90 percent of a card's memory. Real output:

qwen2.5-3b qlora
   T4 / P100 class (16 GB)            budget  14.4 GB  max seq 6400
   L4 / A10 / 3090 / 4090 (24 GB)     budget  21.6 GB  max seq 10752
   A100 40 GB                         budget  36.0 GB  max seq 16384

qwen2.5-7b qlora
   T4 / P100 class (16 GB)            budget  14.4 GB  max seq 3584
   L4 / A10 / 3090 / 4090 (24 GB)     budget  21.6 GB  max seq 7680
   A100 40 GB                         budget  36.0 GB  max seq 15616

qwen2.5-7b lora
   T4 / P100 class (16 GB)            budget  14.4 GB  max seq does not fit (256 tokens needs 18.9 GB)
   L4 / A10 / 3090 / 4090 (24 GB)     budget  21.6 GB  max seq 1536
   A100 40 GB                         budget  36.0 GB  max seq 9728

qwen2.5-14b qlora
   T4 / P100 class (16 GB)            budget  14.4 GB  max seq 512
   L4 / A10 / 3090 / 4090 (24 GB)     budget  21.6 GB  max seq 3840
   A100 40 GB                         budget  36.0 GB  max seq 10752

These are estimates from arithmetic, not runs on a notebook GPU. Read them as a map:

If your training examples are short (classification, extraction, short chat turns), a free notebook is a fair place to learn. For an end-to-end example of what a small fine-tune produces and where it breaks, see the LoRA walkthrough. To decide whether you need fine-tuning at all, read fine-tuning vs prompting first.

How do I find out what my session actually has?

Run this as your first cell, every time. It prints the GPU, memory, disk and a start time, so you can see how long the session really lasted when it dies. I ran it in this session's container, which has no GPU, to show the output format:

"""session_probe.py: print what this notebook session actually gives you. Run it first, every session."""
import os, shutil, subprocess, time

def sh(cmd):
    try:
        return subprocess.run(cmd, capture_output=True, text=True, timeout=20).stdout.strip()
    except (FileNotFoundError, subprocess.TimeoutExpired):
        return ""

gpu = sh(["nvidia-smi", "--query-gpu=name,memory.total,memory.used", "--format=csv,noheader"])
print("GPU:", gpu or "none visible (check Runtime / Accelerator settings)")
with open("/proc/meminfo") as f:
    kb = int(next(l for l in f if l.startswith("MemTotal")).split()[1])
print(f"System RAM: {kb / 1024 / 1024:.1f} GiB, CPU cores: {os.cpu_count()}")
for path in ("/content", "/kaggle/working", "/tmp"):
    if os.path.isdir(path):
        d = shutil.disk_usage(path)
        print(f"Disk at {path}: {d.free / 2**30:.0f} GiB free of {d.total / 2**30:.0f} GiB")
print("Started:", time.strftime("%Y-%m-%d %H:%M:%S UTC", time.gmtime()), "(note it; compare when the session dies)")

GPU: none visible (check Runtime / Accelerator settings)
System RAM: 15.7 GiB, CPU cores: 4
Disk at /tmp: 13 GiB free of 252 GiB
Started: 2026-10-09 05:57:33 UTC (note it; compare when the session dies)

On a GPU runtime the first line shows the card name and its total memory in MiB. Compare that against the "budget" column above. If you were given less than you planned for, change the sequence length or the base model before you start, not after the first out-of-memory error.

I have not run this on Colab or Kaggle. It uses only the standard library and nvidia-smi.

How do I make a run survive a disconnect?

Assume the session will end. Save the adapter to storage that outlives the VM, often, and resume from the newest checkpoint. LoRA adapters are small, so saving every 50 steps is cheap. This is the setup, with the Hugging Face Trainer arguments (I constructed the TrainingArguments with transformers 5.19.0 and it ran; I did not train anything with it):

import os
from transformers import TrainingArguments

OUT = "/content/drive/MyDrive/runs/qlora-7b"   # on Kaggle use /kaggle/working/runs/qlora-7b and save outputs as a dataset

args = TrainingArguments(
    output_dir=OUT,
    per_device_train_batch_size=1,
    gradient_accumulation_steps=16,
    gradient_checkpointing=True,
    save_strategy="steps",
    save_steps=50,            # adapters are small, so saving often is cheap
    save_total_limit=2,
    logging_steps=10,
    max_steps=600,
)

has_ckpt = os.path.isdir(OUT) and any(d.startswith("checkpoint-") for d in os.listdir(OUT))
# trainer = Trainer(model=model, args=args, train_dataset=ds)
# trainer.train(resume_from_checkpoint=True if has_ckpt else None)
print("resume:", has_ckpt)

Colab's FAQ warns that Drive operations can be slow or fail with very large folders and that an interrupted file move can lose data in transit, so write checkpoints into a fresh sub-folder with few files, and do not move them during a run. Keep the data, the script and the evaluation set in one place so a fresh session can start from the top in two cells.

When does paying beat free?

Three signals:

1. The memory estimate says the card is too small. More than about 14 GB estimated means free tiers are out.
2. A lost run costs you more than the rental. Rented 24 GB cards were listed at $0.27 to $0.89 per hour on RunPod's public pricing page (updated September 27, 2026 per the page, read 2026-10-09). A three-hour QLoRA run at those listed rates is roughly $0.81 to $2.67, before storage and idle time. The rental cost post has the method and a dated table across providers.
3. You need more than one session. If the job takes days of accumulated time, a paid or rented machine removes the resume dance.

Colab's own paid route is compute units. Their pricing page was not readable by my tools, so use their sign-up page for the current numbers. What the FAQ does tell you is the catch: units are consumed at a rate that depends on the GPU, and when they run out you are back on free-tier rules.

What would I do first?

1. Estimate memory for your model and longest realistic sequence.
2. If it fits a 16 GB budget, try a free notebook with the probe and checkpointing above. Time the first 20 steps and multiply to get hours.
3. If hours exceed one session or the card is too small, rent for that run only.
4. If you are not sure fine-tuning is the right move, run the cheaper test first: a good prompt and a few examples on a hosted model. The fine-tuning vs RAG vs prompting guide has the decision tree.

What I could not verify

• Colab's current plan prices, compute unit counts and unit burn rates. The pages did not render or returned 403.
• Kaggle's current GPU quota, accelerator list and session limits from Kaggle's own pages. Figures above are from secondary sources.
• That any specific GPU will be assigned to you. Neither service promises one.
• Real out-of-memory behaviour on a T4 or P100: the table is arithmetic, with overhead and fragmentation terms I could not measure without a GPU.
• Training speed. I have no step-time measurements, so I give none.