Model Customization & Fine-Tuning · Build
Fine-Tuning Dataset Curation
Every accepted record carries permitted provenance, labels hold across critical slices, and training stays isolated from validation and test cases. A fine-tuning dataset is ready once those three hold, and not before.
A folder of good examples is not yet training data. We trace each record, confirm how it may be used, calibrate the labeling rules, remove duplicates, and keep evaluation cases out of the training set. This engagement ends with the dataset. It does not train the model.
The folder of good examples turns into a versioned training dataset. Stable source trails, calibrated labels, protected splits, and visible leakage findings come with it, under limits your data owner has accepted.
Some of the 500+ brands we've worked with
See all references
Steps, gates, and who decides
How we work
The difficult calls happen while the sample is small enough for your reviewers to inspect the accepted, rejected, and ambiguous records themselves.
Set the rules before collection
We define the target behavior and baseline, agree which sources are permitted, specify formats and labels, name critical slices, record rejection reasons, and reserve the held-out boundary.
- AI assist
- The target behavior and calibration sample give the model material for a first rubric draft, which the data owner reviews.
- Human gate
- Has your owner approved the labeling rubric and permitted sources? Your data owner approves the rubric and permitted sources before collection starts.
Bring each example in with a trace
Candidate records arrive with provenance and rights information attached. We normalize their formats without erasing the identity of the source.
- AI assist
- The model flags candidates without a provenance entry or usage-rights record for the data owner's decision.
- Human gate
- Can every accepted example be traced to a permitted source? Your data owner resolves examples with missing or unclear rights.
Build a representative working set
Reviewers filter and label the examples, then examine the balance across critical slices. Ambiguous and rejected records remain visible until a subject-matter decision settles them.
- AI assist
- The model compares reviewer labels with the rubric and surfaces inconsistent calls.
- Human gate
- Are labels consistent enough to apply the rubric beyond the calibration sample? A subject-matter reviewer resolves every flagged ambiguous label.
Isolate the splits and freeze the version
We remove duplicates, scan for leakage, assign train, validation, and test membership, freeze the result, and write the dataset card.
- AI assist
- The model scans the three splits for duplicates and leakage candidates, which a person reviews before any record moves.
- Human gate
- Are held-out cases isolated and the remaining defects accepted by the owner? Your data owner accepts the dataset version and its documented limits.
Named artifacts you keep
What you get
The handover answers which records belong, where they came from, what reviewers checked, and the uses the owner accepted.
Playbook
Target-behavior and permitted-source curation plan
The target behavior, permitted-source rules, labeling rubric, formats, critical slices, rejection criteria, and held-out boundary.
Dataset
Versioned fine-tuning dataset with source trail
Accepted examples with stable identifiers, labels, source references, normalization history, and dataset version.
Risk register
Usage-rights, slice-coverage, and leakage report
Source and rights coverage, critical-slice coverage, duplicate and leakage findings, label issues, and open exceptions.
Model card
Train-validation-test split manifest and limits file
Train, validation, and test membership beside intended use, limitations, known defects, review status, and the responsible owner.
Scope and honest limits
When to bring us in
You know the behavior the model should learn. The available examples still need a defensible route into a training dataset.
A good fit when
- Examples arrive from several systems, but uneven formats, labels, and permission records leave reviewers unable to decide what may enter training.
- The dataset looks large, yet critical users, tasks, and error slices remain thin enough to distort what the training sample teaches.
- Baseline and held-out cases sit near the training pool, so duplicate or leaked records can make the later experiment look better than it is.
- The target behavior is understood, but no approved data recipe, labeling rubric, format rule, or critical-slice list guides collection.
- Sources, normalization decisions, balance checks, provenance, and usage rights live in separate records, so an accepted example cannot be traced cleanly.
- The duplicate and leakage candidates remain, while train, validation, and test membership has not been isolated.
- A dataset version is ready to hand over, but its quality review, split manifest, known defects, and responsible owner are not documented.
Better handled as other work when
- Usage rights, privacy, or regulated-data use have to be legally approved before tuning. Curation records the provenance evidence your counsel needs for that call.
- You want a coverage gap filled with unverified or unauthorized examples. Representation matters, but every accepted record still needs a permitted source.
- You need the model trained, deployed, or operated. This engagement ends with an accepted dataset, while those stages require a separate scope.
If one of these is closer to your situation, start here instead: See model customization services
Engineers who ship production AI
This is the part of Zeo that writes and ships code. Our senior engineers build agents, chatbots, and RAG pipelines, along with the automation and data work around them, and they keep operating those systems once they're live. We've worked with more than 500 brands since 2011.
Tools we use
Tools behind this work
Hugging Facethe format the curated dataset ships in, ready for a training run to consume
DVCversions the dataset through every curation pass, record by record
Tonicfills coverage gaps in the real record pool with synthetic examples
Jupyterruns deduplication and rubric-consistency checks as re-runnable, inspectable code
Labelboxruns the labeling workflow with reviewer agreement tracked against the rubric
Adjacent work
Where this leads
Build
LLM Fine-Tuning & Model Customization
Curated and validated instruction datasets provide the formatted training tokens executed in parameter-efficient fine-tuning runs.
Build
Domain Adaptation
Dataset curation cleans specialized domain documents to build the high-quality corpus required for continuous language adaptation.
Assurance
Model Selection & Baseline Benchmarking
Held-out evaluation datasets created during curation serve as the gold-standard benchmark for testing base and fine-tuned model variants.
Advisory
Fine-Tune vs RAG vs Prompting Assessment
Data availability and curation costs provide empirical inputs to assessments deciding between model training and retrieval architectures.
Next step
Prepare the data before you train
Before you decide