Our 518M base, supervised-fine-tuned on 7,141 closed-book legal/financial question-answer pairs. It answers from its own weights, with no passage supplied. Ask a question and it responds; it says so when it does not know.
A base completer turned into a question answerer by supervised fine-tuning on our own QA dataset (every question generated from the same legal corpus the base was pretrained on, then gated for faithfulness and self-containment). Loss was masked to the answer tokens only.
Because it is closed-book, it answers from memory, and at 518M parameters it holds limited knowledge, so it can be confidently wrong on specifics. About 10% of its training was refusals, so it will sometimes decline rather than guess. First call may take ~20–40s while the model wakes from idle.
| Stage | Cost | Share |
|---|---|---|
| Data pipeline32K tokenizer + re-tokenize (CPU); corpus cleaning is shared with the 125M | $1.35 | 1.1% |
| Pretraining5 epochs on 8x B200, 10.4B tokens seen, incl. smoke runs | $115.69 | 97.7% |
| QA dataset (shared)1/9 share of the 7,141-pair closed-book QA set | $0.40 | 0.3% |
| Fine-tuning (QA SFT)full fine-tune on H100 | $1.01 | 0.9% |
| Total to assemble | $118.45 | 100% |
Pretraining, with the data pipeline that fed it, is 98.8% of the bill. Everything after it (fine-tuning) came to $1.41. Teaching a model to behave is cheap; teaching it to know is not.
Figures are actual invoiced Modal usage (GPU + CPU + memory) for this model’s lineage, not estimates. Shared inputs (the QA and preference datasets) are charged at this model’s share, and their generation also used ~$1–2 of external LLM API calls, included above. Serving is billed separately and scales to zero.