Vizuara AI Labs · closed-book QA fine-tune

SLM‑500M QA

Our 518M base, supervised-fine-tuned on 7,141 closed-book legal/financial question-answer pairs. It answers from its own weights, with no passage supplied. Ask a question and it responds; it says so when it does not know.

517.8M
parameters
32,768
vocab
7,141
QA pairs
1.69
SFT val loss
10%
refusals
Validation metrics along the 500M lineage
Each perplexity is measured on that stage's own validation set, so read the trend as 'how well the model fits its own stage's data', not as one curve on one dataset. DPO and RLAIF optimize preferences rather than likelihood, so they log preference margin and reward instead of perplexity. Click a stage to open that model.
Base
ppl 7.91
pretrain val
QA SFT
ppl 5.41
QA val
Instruct
ppl 6.69
instruction val
DPO
margin 88.0%
preference val, no ppl
/
RLAIF
reward 6.7→10.1
RM reward, no ppl
RAFT on DPO
ppl 1.95
RAFT val
/
RAFT on RLAIF
ppl 2.01
RAFT val
ask a closed-book question
ready
The model's answer will appear here.

What this is closed-book QA

A base completer turned into a question answerer by supervised fine-tuning on our own QA dataset (every question generated from the same legal corpus the base was pretrained on, then gated for faithfulness and self-containment). Loss was masked to the answer tokens only.

Because it is closed-book, it answers from memory, and at 518M parameters it holds limited knowledge, so it can be confidently wrong on specifics. About 10% of its training was refusals, so it will sometimes decline rather than guess. First call may take ~20–40s while the model wakes from idle.

what this model cost to build
$118.45 total to assemble, on real Modal billing
Data & tokenization $1.35Pretraining $115.69Fine-tuning (SFT) $1.41
StageCostShare
Data pipeline32K tokenizer + re-tokenize (CPU); corpus cleaning is shared with the 125M$1.351.1%
Pretraining5 epochs on 8x B200, 10.4B tokens seen, incl. smoke runs$115.6997.7%
QA dataset (shared)1/9 share of the 7,141-pair closed-book QA set$0.400.3%
Fine-tuning (QA SFT)full fine-tune on H100$1.010.9%
Total to assemble$118.45100%

Pretraining, with the data pipeline that fed it, is 98.8% of the bill. Everything after it (fine-tuning) came to $1.41. Teaching a model to behave is cheap; teaching it to know is not.

Figures are actual invoiced Modal usage (GPU + CPU + memory) for this model’s lineage, not estimates. Shared inputs (the QA and preference datasets) are charged at this model’s share, and their generation also used ~$1–2 of external LLM API calls, included above. Serving is billed separately and scales to zero.