Series idea: fine-tuning small local models on domain-specific data — worth exploring?

@HumbleBee14·Jul 17, 2026

Pitch: a hands-on series about fine-tuning small local models (1-8B) on your own domain-specific dataset instead of paying per-token for a frontier API.

Rough shape:

  1. When does this actually make sense? Honest cost/quality math — API calls vs. a fine-tuned Qwen/Llama/Phi running on a single consumer GPU or a MacBook.
  2. Building the dataset — turning internal docs, support tickets, or logs into instruction pairs; how much data is actually enough (spoiler: less than people think with LoRA).
  3. Training — LoRA/QLoRA on a single 24GB card, what breaks, real loss curves.
  4. Serving it locally — llama.cpp / Ollama / vLLM, quantization choices, latency numbers on real hardware.
  5. Evals — how to know your 3B fine-tune actually beats the base model on your domain and not just vibes.

Would you read this? Anything you would add or cut — e.g. is the dataset-building part the interesting piece, or the serving part?

0 replies