Selecting a robot data partner usually turns into a comparison of hourly rates and promised volumes. Those are the least predictive numbers available. Assessing an enterprise physical AI data engine properly means asking about process, schema, and evidence.
This guide sets out the questions worth asking across quality, interoperability, operations, commercial terms, and compliance, plus the answers that should end a conversation early.
Volume questions are the wrong questions
Most vendor conversations start with hours, episodes, and price per unit. Those numbers are easy to compare and predict almost nothing about whether you end up with a usable dataset.
The questions that predict outcomes are about process: how quality is measured, what happens when something is ambiguous, and whether the data can be pooled with what you already have. A vendor who is comfortable with those questions is a different proposition from one who wants to talk about capacity.
Data quality
- What is your yield rate and how is it measured? A vendor who does not know is not running QA at capture. This is the single most revealing question, and it feeds directly into cost per usable trajectory.
- Show me one complete raw episode, including metadata. Not a reel. This artifact answers more than any deck.
- What automated checks run at collection time? Sync, calibration drift, force spikes, latency breaches, dropped frames.
- What fraction of delivered episodes are failures or recoveries? Zero means the set was cleaned and the recovery behaviour is gone.
- How is per-operator quality tracked? If the answer is throughput, variance is not being managed.
Schema and interoperability
- What action convention do you deliver in, and can you match ours? Mismatched conventions split a corpus permanently.
- Are units, coordinate frames, and control rates stated per episode or assumed? Assumed is the quiet corruption in pooled data.
- Is measured control rate recorded, or only the configured value?
- How are annotations versioned relative to episodes, per dataset structure?
- What happens when the schema changes mid-programme? Forward migration, or history rewritten?
Operations
- Who owns this programme by name? A programme manager and a QA lead, not a shared inbox.
- What is your escalation path for ambiguity? Silence means operators are guessing differently.
- How are task scripts written and versioned, and can we see one, per task script design?
- What happens to quality when an operator leaves? This tests whether knowledge lives in a system.
- How quickly do we get feedback on a bad batch? Same day is achievable; end of month means five weeks of a repeated defect.
Commercial terms worth pinning down
- Who owns the data, and may the vendor reuse or resell it? Ownership and exclusivity are separate questions.
- What is the acceptance test? Define what “delivered” means before collection, not after.
- Who pays for re-collection when data fails acceptance, and on whose determination?
- Can volume be changed mid-programme, in both directions?
- What happens at contract end? Data handover format, and deletion evidence.
- Are subcontractors used, and do they inherit the same obligations?
Acceptance testing is the term most often left vague and most often disputed. It should reference measurable properties: sync tolerance, calibration currency, metadata completeness, and yield against a defined standard.
Compliance
- Where is data captured, stored, and accessed from, per data residency
- What consent framework applies, and can we see the template, per consent and privacy
- What redaction runs, at what stage, and how is it verified
- What are the retention schedules, and how is deletion evidenced
- Are operators vetted, and to what standard for regulated environments
- What audit rights do we have
These come up in every serious procurement, as covered in what enterprise procurement actually asks. Vendors with documented answers move through the process considerably faster.
Answers that should end the conversation
- “We deliver whatever format you need” without asking what yours is. That is a sales answer to a technical question.
- “Our quality is 99 percent” with no definition of the denominator.
- “We can start next week” before seeing your task list or embodiment.
- “We only deliver successful episodes.” Sold as quality; it removes the recovery data you need most.
- Reluctance to show one complete raw episode. There is no good reason for this.
The fourth is the most common and the most costly, because it sounds like a feature.
What good looks like from the other side
A capable partner asks you harder questions than you ask them: what the robot is, what the gripper is, what success means, what your evaluation set looks like, and what your log schema is. A vendor quoting on hours without that context is quoting on volume rather than outcome.
For a programme built on that kind of specification-first engagement, our humanoid foundation model case study documents a 3x sample-efficiency result. The scoping side is covered in scoping your first programme.
Frequently asked questions
What is the single most revealing question?
Ask for one complete raw episode with full metadata. Schema quality, QA rigour, and whether failures are retained are all visible in that one artifact.
Should we run a paid pilot before committing?
Almost always. A pilot measures cycle time, yield, and operator variance on your actual task, and those numbers should set the terms of the full programme.
How do we compare vendors with different pricing models?
Normalise to cost per usable trajectory, including your own QA and integration effort. Per-hour comparisons systematically favour whoever counts hours most generously.
Is a large operator network a good signal?
Only alongside coverage in the geographies, languages, and credentials you need. Headcount without relevant coverage solves nothing, per managed workforce.
The cheapest quote and the cheapest dataset are rarely the same thing. If you want to put these questions to us, or want help assessing another vendor’s answers, tell us what you are building.
