A tabletop arm only has to answer one question: where does the gripper go next? A robot that drives to the counter, opens a cabinet and carries a pan has to answer three at once: where the base goes, where the arms go, and how the two fit together. Mobile manipulation teleoperation is how most teams capture those answers, and it is where many datasets quietly fall apart.
This guide explains what whole-body teleoperation must record, how to structure base and arm control, and what separates usable mobile manipulator data from a pile of video. It is written for teams building household robot data programs and for anyone scaling past the tabletop.
Why mobile manipulation is a different data problem
The authors of Mobile ALOHA point out that most imitation learning results focus on tabletop work, “lacking the mobility and dexterity necessary for generally useful tasks.” Adding a base does more than add degrees of freedom. It couples them. Arm reach depends on where the base stopped, and the base path depends on what the arms are carrying.
That coupling makes failures compositional. A flawless arm trajectory from a poor base position is still a failed grasp, and a dataset that records only the arm cannot tell you why. Whole-body teleoperation exists to capture both halves of the decision together.
What whole-body teleoperation must record
Mobile ALOHA recorded base velocity and arm puppeteering at the same time, and that principle is the core of mobile manipulation teleoperation: one session, one clock, every channel. A usable episode includes:
- Base commands and state: velocity or pose commands, odometry and localization output.
- Arm and gripper data: commands and measured joint state for each arm, plus gripper width.
- Body extras: torso or lift height, head pan and tilt, where the platform has them.
- Multi-view video: head camera, wrist cameras (see wrist vs head camera data) and a wide view that shows the base in the room.
- Range sensing: LiDAR or depth where available, as covered in our guide to LiDAR and point cloud collection for mobile robots.
- One shared clock: timestamps that let you align every stream after the fact.
Base and arm control: three interface patterns
| Pattern | How it works | Strength | Weakness |
| Single-operator whole-body rig | One person drives the base and both arms | Natural coordination between base and arms | Physically demanding; fatigue builds quickly |
| Split control | Joystick or pedals for the base, leader arms or VR for manipulation | Easy to learn; lower strain | Coordination is learned, so style varies by operator |
| Two operators | One drives, one manipulates | Specialist skill on each half | Intent is split across people; hard to label and replay |
No pattern wins everywhere. Pick the one whose weakness your quality process can absorb.
Five pitfalls in mobile manipulator data
- Stream drift. Base and arm logs fall out of sync by a few hundred milliseconds, and the policy learns the wrong cause and effect.
- Inconsistent standoff. Operators park at different distances from the counter, so the arm’s workspace varies by who drove.
- Site-specific maps. Localization that works in one kitchen breaks in the next, silently corrupting base labels.
- Success-only episodes. Repositioning after a missed approach is exactly what a robot must learn; trimming it out removes the recovery.
- Unlabeled mixing. Pooling static and mobile data without tags hides which behaviors came from which embodiment.
Co-training: let static data carry the breadth
Mobile data is expensive, so use it for what only mobile data can show. Mobile ALOHA’s authors found that co-training with existing static bimanual datasets boosted performance, reporting success-rate gains of up to 90% with just 50 demonstrations per task. The lesson for data programs: collect bimanual manipulation datasets for breadth and reserve whole-body sessions for base-arm coordination, door and drawer interactions, and long-horizon routines.
Household robot data: where it gets hard
Homes are the hardest test of mobile manipulator data. Thresholds, narrow doorways, clutter and moving people change every session, and tasks such as kitchen work and activities of daily living chain navigation and manipulation without clean boundaries. Diversity across real homes matters more than repetitions in one, which is why many programs pair a dedicated team for controlled collection with broader capture for environmental variety.
How Roborax runs whole-body programs
Roborax treats mobile manipulation teleoperation as a pipeline, not a rig. On our mobile manipulation platforms, we scope each program in four moves:
- Define the task chain and the base-arm handoffs inside it.
- Fix the schema first: shared clock, action convention and metadata.
- Train operators on one control pattern and verify standoff consistency.
- Run every episode through our QA pipeline before it reaches your training set.
Frequently asked questions
How many demonstrations does a mobile manipulation task need?
It depends on task length and variation. Co-training with static data lowers the count. Start with a pilot of tens of episodes per task and scale with measured success.
Can one operator control the base and both arms?
Yes, on whole-body rigs, but fatigue limits session length. Plan shorter sessions and rotate operators.
Should base and arm actions share one action space?
Keep them as separate, labeled channels in the log. You can concatenate them for training, but you cannot easily separate a merged label later.
The bottom line
Robots that move and manipulate need data that moves and manipulates together. Capture base and arm on one clock, keep the channels separate, let static data supply breadth, and save whole-body sessions for the coordination only they can show.
Planning a household or warehouse mobile manipulation program? Tell us what you are training and Roborax will scope the whole-body collection plan with you.
Related reading
- Mobile manipulation data platforms
- Mobile manipulation solutions
- Teleoperation data collection
- Bimanual manipulation datasets
- Wrist camera vs head camera robot data
- LiDAR and point cloud data collection for mobile robots
- Kitchen task robotics data
- Activities of daily living robot tasks
