Xrticles │ How To Fine-Tune a Small LLM on Your Own Data (Full Guide)

You do not need a 70B model.

You do not need $100,000 in compute.

You do not need a machine learning team.

A 1.5B parameter model fine-tuned on 200-500 good examples can outperform a frontier model prompted generically at your specific task.

This is the complete guide.

Data collection. Cleaning. Dataset creation. QLoRA training. Google Colab. Evaluation. Deployment.

Working code at every step.

Save this. Feed it to AI and fine tune your data.

Actual goal of fine-tuning

Most people think fine-tuning makes a model smarter.

It does not.

Fine-tuning makes a model consistently better at one narrow task.

That is the whole value proposition.

Here is the example we will use throughout this guide:

A model that accepts a startup brief like this:

And produces funding-pitch copy like this:

Clean. Grounded. No invented metrics. No forced slide structure.

A 1.5B model doing this reliably is more valuable than a 70B model doing it inconsistently.

That is the bet we are making.

Fine-tuning is not your first step

Before you train anything, build three baselines:

Fine-tuning can improve: tone, structure, consistency, domain vocabulary, instruction compliance.

Fine-tuning cannot reliably give the model current market data, competitor intelligence, or real-time facts.

If your model needs to know current market sizes or funding trends, use retrieval. Not training.

The architecture that works:

Only start fine-tuning after your prompt-only baseline fails at something specific.

Step 1 — Pick a small base model

Start between 0.5B and 3B parameters.

Good starting points:

For this guide: Qwen/Qwen2.5-1.5B-Instruct

Small enough for QLoRA on a T4 GPU. Large enough for useful business writing.

Do not start with 7B or 14B. Larger models mean:

The smaller model first. Upgrade only if it fails evaluation.

Step 2 — Data rights before scraping anything

This is the step everyone skips.

And it is the most dangerous one.

Public availability does not mean training permission.

For every data source, record this before downloading a single file:

What goes in your public GitHub repo:

What stays private:

Get this right first. The rest is just engineering.

Step 3 — Download data reproducibly

Never blindly crawl. Use an explicit allowlist of approved URLs.

For every file:

Every file gets: a hash, a timestamp, a manifest entry.

Duplicates skipped by content hash. No silent re-downloads.

Step 4 — Extract text from PDFs and OCR

Pitch decks are hard documents.

Large text. Tiny footnotes. Charts. Tables. Rotated text. Numbers embedded in images. Multiple columns.

Use a cascade:

Record everything — including failures:

Critical rule: do not silently drop pages where OCR returns nothing.

Empty results are your hardest cases. They tell you where the pipeline breaks.

Machine draft ≠ ground truth. Never conflate them.

Step 5 — Clean the text without destroying it

Cleaning means removing extraction noise. Not making it "nicer."

Safe operations:

Never auto-correct numbers.

If OCR returns $12.SM instead of $12.5M — send it to review. Do not fix it with a language model.

A model trained on wrong numbers will produce fluent but financially incorrect output.

Track every numeric string separately:

Numbers get evaluated independently from text quality. Always.

Step 6 — Build the training dataset

Raw pitch deck pages are not training data.

A deck page might contain a slogan, a chart, a legal disclaimer, or a logo wall.

Your training task is:

startup brief → funding-pitch copy

Not:

PDF page → extracted text

You have to build the bridge.

Every training example needs:

Here is the JSONL format:

The output is deliberately plain text.

No forced slide count. No JSON. No title field. No bullets field.

The downstream app converts output into whatever format it needs.

Do not auto-generate training examples casually.

Safe order:

If the source does not verify a metric, do not include the metric.

Use: "The company reports strong early traction." Only if the source actually says that.

Otherwise: "The company is focused on expanding early customer adoption."

Step 7 — Split by company, not by page

This is the data mistake that makes evaluation meaningless.

If the same company appears in both training and validation:

Company A pitch deck, page 1 → training

Company A pitch deck, page 2 → validation

The model memorizes company-specific language and appears to generalize. It does not.

Split by company:

Company A → training

Company B → training Company C → validation Company D → test

Target split:

Training: 80%

Validation: 10% Test: 10%

Keep a split manifest:

Never change the test set after looking at results.

Once you evaluate against the test set, it is burned. It becomes your training signal, not your true evaluation.

Step 8 — Set up the environment

Your 2-vCPU, 4GB VPS is not a training machine.

It is perfect for:

✓ API gateway

✓ Authentication and rate limiting ✓ Request queuing ✓ Redis ✓ Logging service ✓ Health checks ✓ Deployment controller

It is wrong for:

✗ Training a 1.5B QLoRA model

✗ Running GPU inference ✗ Serving real traffic to a language model

Use the right machine for each job.

Step 9 — Train with QLoRA

QLoRA loads the base model in 4-bit precision and trains a small adapter.

Dramatically less memory than full fine-tuning. Almost the same results.

Run it:

Key hyperparameters explained:

Do not increase epochs just because training loss drops.

The model can memorize your examples while getting worse on unseen companies.

Lower training loss ≠ better generalization.

Step 10 — Training on Google Colab

Colab is great for first experiments. Free GPU. No server to maintain.

The tradeoffs to know:

✓ Free T4 GPU for experiments

✓ No infrastructure maintenance ✓ Great for notebooks

✗ Sessions can disconnect at any time ✗ GPU availability varies by time of day ✗ Storage resets between sessions ✗ Not a production training service

Start a GPU notebook. Check your runtime:

Mount Drive to save checkpoints:

Install dependencies:

Set your paths to Drive:

Save to Drive every 50 steps, not just at the end.

If Colab disconnects at 95%, you want to resume, not restart.

Record your environment before every run:

Reproducibility requires knowing exactly what version ran.

Step 11 — GPU provider strategy

Use different infrastructure for different stages.

Google Colab:

Use for: first experiments, debugging, small QLoRA jobs

Avoid for: production, long runs, reliable scheduling

Spot GPU providers (RunPod, Vast.ai, Modal, Lambda, Paperspace):

Use for: longer training jobs, per-second billing, reproducible Docker environments

Avoid for: production serving, stable latency requirements

Check live pricing before every run. GPU rates change frequently.

Dedicated GPU hosting:

Use for: public serving, stable latency, production deployment

Avoid for: experimentation (you pay while idle)

Estimated training cost:

GPU rate: $0.40/hour

Active time: 3 hours Retries: 1 hour Storage: $0.10

Total: ($0.40 × 4) + $0.10 = $1.70

The expensive part is not the first run.

It is repeated experimentation:

Changing prompts

Rebuilding datasets after bugs Testing multiple ranks (r=8, r=16, r=32) Trying different base models Running evaluation Serving GPU continuously for testing

Budget for 5-10x your first estimate.

Step 12 — Evaluate against a prompt-only baseline

Do not compare your fine-tuned model against training loss.

Compare it against the model you were using before.

Fixed evaluation metrics:

Plain-text compliance (no JSON, no slide structure)

Output usefulness (human rating 1-10) Unsupported claim rate Numeric preservation accuracy Repetition rate Readability Latency Token usage and cost per request

Numeric consistency check:

A generated number not in the source evidence is a hallucination.

Catch it before your users do.

Example scorecard:

Only claim improvement if the test set was isolated before you started evaluating.

If you peeked at the test set, those results are meaningless.

Step 13 — Serve the adapter

Two options depending on your traffic.

Option A: vLLM (high throughput)

Check the current vLLM LoRA docs before deploying. Options change between versions.

Option B: FastAPI + Transformers (low traffic)

Never expose a raw model server to the internet.

Your production setup:

Client → HTTPS + auth → VPS rate limiter → request queue → GPU model server → plain-text response

The VPS handles: TLS, auth, rate limits, queueing.

The GPU handles: inference.

Step 14 — Monitor quality after deployment

Track these metrics from day one:

Request latency

Queue time GPU memory and utilization Error rate Empty outputs Token counts per request User retry rate Unsupported-claim reports Model version + adapter version

Do not auto-collect user data for retraining.

Use an explicit opt-in:

This request may be used to improve the model.

[ ] Yes, I consent [ ] No

Even with consent, strip before storing:

✗ Personal phone numbers

✗ Email addresses ✗ Customer names ✗ Private revenue figures ✗ Confidential fundraising terms ✗ Investor contact details

Keep production training data separate from evaluation data. Always.

When to retrain

Not every user submission triggers a retrain.

Retrain when you have:

✓ Meaningful number of reviewed examples (not raw submissions) ✓ A clear documented failure pattern ✓ A stable evaluation set ✓ A documented dataset version ✓ A rollback plan

A sensible cadence:

v0: prompt-only baseline v1: 200 reviewed examples v2: 1,000 reviewed examples v3: larger dataset + retrieval

Every version records:

Never deploy without a documented rollback path.

Sequence that actually works

This is what I would do for this exact project:

The highest-leverage improvement is almost never the learning rate.

It is the examples.

That is what turns 1.5B parameters into something genuinely useful.

━━━━━━━━━━━━━━━━━━

Project structure

Raw data private. Processing code public. Evaluation code public.