Most first robot data programmes are scoped backwards: a volume target is agreed, a budget is attached, and the specification gets written afterwards. Custom egocentric data collection services only work when the sequence runs the other way.
This guide covers why you scope from the model rather than the data, the eight decisions a specification has to settle, how to run a pilot that actually measures something, and what to have ready before you approach a partner.
Start from the model, not the data
The most common opening question is “how many hours do we need?” It is unanswerable, and it is the wrong end of the problem.
Work backwards instead. What behaviour must the robot perform? On what hardware? In what environments? Against what success criteria? Each answer narrows the data specification, and by the time you have all four the volume question mostly answers itself.
Programmes that start from a volume target end up with large datasets that are narrow in exactly the dimension that mattered.
The eight decisions a scope needs
| Decision | Why it comes first |
|---|---|
| Target embodiment | Action labels are hardware-specific and rarely transfer |
| Task list | Capability groups, not a vague domain, per task taxonomy |
| Capture method | Headset, leader-follower rig, or wearable human capture |
| Sensor stack | What you cannot add later without re-collecting |
| Variation dimensions | Objects, scenes, lighting, operators, phrasing |
| Log schema | Action convention, units, metadata, per log design |
| Quality gates | What gets rejected, and measured how |
| Delivery model | Dedicated, crowdsource, or hybrid |
Only the last one is really a commercial decision. The other seven are technical, and getting them wrong is what produces the expensive re-collections.
Pilot first, and treat it as a measurement
A pilot is not a small version of the programme. It is an experiment to determine the numbers the full programme depends on.
- Cycle time per episode, including scene reset and overhead, which is usually longer than anyone estimates.
- Yield. What share of collected episodes survives QA. This one number drives your whole budget, per cost per trajectory.
- The learning floor. Episodes needed on one fixed configuration before performance stops improving. Everything above that goes on variety.
- Operator variance. Run several operators on the same script and measure divergence. High divergence means the script is ambiguous, not that the operators are.
- Failure taxonomy. The real failure modes, which are never the ones predicted in planning.
A pilot that produces only a dataset has failed. A pilot that produces five numbers and a revised specification has succeeded, even if every episode is discarded.
What to have ready before approaching a partner
- The robot, or its specification. Degrees of freedom, gripper, sensors, control interface.
- A task list with success criteria, graded rather than binary.
- Object sets, at instance level, and ideally physical samples.
- Environment description, including how much it varies in deployment.
- Your log schema, or an admission that you do not have one yet.
- The evaluation you will judge success by, per policy evaluation.
A partner who does not ask for most of these is quoting on volume rather than on outcome. That is a useful filter in itself, and it is one of the questions in what to ask a data partner.
The three scoping mistakes that recur
- Deferring sensors to phase two. Force, wrist cameras, and pose are cheap at setup and require re-collection later. See force-torque capture.
- Collecting only successes. Recovery behaviour is the most valuable and most commonly discarded data in any programme.
- Fixing the scene to reduce variance. Repeatability feels like quality and produces a model that memorises one staging.
Phasing that works
Three phases, with a decision gate between each.
Phase one, pilot. A few hundred episodes on a narrow slice. Output: measured cycle time, yield, learning floor, revised schema. Decide whether to proceed.
Phase two, breadth. Expand across variation dimensions rather than repetitions. Output: a dataset broad enough to train a first credible policy, and a real failure taxonomy.
Phase three, targeted. Collect against specific weaknesses the evaluation exposed. This is where deployed-fleet signal starts to guide collection, per the intervention loop.
Most programmes try to run phase two first, then discover in phase one’s absence that the schema was wrong. For a programme that moved through these stages quickly, our mobile manipulation case study covers a 90-day path from cold-start to production.
Frequently asked questions
How long does a pilot take?
Weeks rather than months for most manipulation tasks, and the constraint is usually hardware availability and site access rather than collection time itself.
Can we scope without having the robot yet?
Partly. Task lists, environments, and object sets can be settled early. Action conventions and sensor placement cannot be finalised without the embodiment, so expect to revisit those.
Should the first programme be in-house or outsourced?
Outsourcing tends to win when the task definition is still moving, because you are buying the ability to change direction without stranded capital. See our build vs buy comparison.
What if our task list changes mid-programme?
Expect it to. Design the schema so new tasks are additions rather than migrations, and keep task identifiers versioned so old batches stay interpretable.
The specification is the cheapest part of a data programme and it determines what everything else buys. If you want yours pressure-tested before you commit budget, send us the task list.
