GUIDER—Global User Intent Dual-phase Estimation for Robots—tries to recognise what a teleoperator is working towards without requiring a predefined list of goals. It separates the journey to a work area from the subsequent manipulation, using scene information and human motion to maintain a changing belief about the intended target.
Why Intent Needs Two Levels
When a person drives a mobile manipulator towards a table, the immediate goal is a useful area, not yet necessarily a particular grasp. Once the robot is nearby, arm movement can distinguish between neighbouring objects. Treating both phases as one fixed target-selection problem loses that structure.
GUIDER first estimates likely navigation regions, then uses a local scene reconstruction to infer manipulation intent. This does not mean the robot reads a person’s thoughts: it ranks hypotheses from observable commands and geometry. Ambiguous movement or a deliberate change of goal should change that ranking.
From Navigation Evidence to Object Candidates
The navigation phase combines controller velocities with an occupancy-grid representation. A synergy map helps identify plausible work regions from the environment and commanded motion. Candidate areas are derived from the scene rather than supplied as a fixed operator-selected goal list.
After reaching the selected area, an autonomous multi-view scan constructs a local point cloud. Plane extraction and clustering organise the geometry, and cluster centroids provide prompts for instance segmentation. This is a bridge between three-dimensional structure and image-based object separation.
During manipulation, U2Net saliency supplies information about visually prominent regions, while FastSAM separates candidate objects. Geometric and grasp-related constraints help reject implausible interpretations. An object attracting visual attention is not necessarily reachable, graspable or intended; the movement evidence has to refine the initial scene-based ranking.
This explains the value of combining perception with motion. Geometry restricts what is physically plausible, while the operator’s trajectory provides evidence about which plausible target is currently relevant.
Understanding the Reported Metrics
The final journal study, published on 5 March 2026, evaluates recorded teleoperation data from 100 Isaac Sim trials: 20 participants performing five task variants. These results supersede the smaller preprint evaluation previously described here. BOIR is the navigation baseline, while Robot Trajectron is the manipulation baseline; neither is a unified comparison system for both phases.
| Comparison | GUIDER | Baseline | Interpretation |
|---|---|---|---|
| Navigation median stability, across tasks | 100% | 89.85% pooled median for BOIR | Persistent navigation intent |
| Navigation median stability, redirection T2 / T5 | 100% / 100% | 59.67% / 63.49% for BOIR | Correct intent after goal changes |
| Manipulation median stability, tool grasp T4 | 100% | 62.68% for Robot Trajectron | Object-level intent during tool selection |
| Median remaining time at confident prediction, geometry-constrained T5 | 20.31 s | 3.89 s for Robot Trajectron | Earlier sustained correct object prediction |
Stability concerns maintaining the correct interpretation over time, not merely producing a consistent answer. A consistently wrong target is not a successful stable prediction.
Remaining time at confident prediction measures the interval from when the correct target first stays highest-probability for at least 0.5 seconds until target contact during navigation or the operator’s command during manipulation. Higher values mean earlier sustained intent recognition, not faster computation or robot movement. The 20.31-second versus 3.89-second comparison concerns the geometry-constrained manipulation task, not every task in the study.
What This Enables—and What Was Not Tested
Earlier, persistent identification of an intended target could give an assistive controller time to prepare a useful movement or ask for confirmation. The inference system is therefore a potential input to shared autonomy, rather than proof that every subsequent assistance action will be helpful.
The reported trials use post-hoc replay: GUIDER did not provide live intent-based assistance to the participants. Demonstrating better prediction and demonstrating better assisted task completion require different experiments. Twenty participants in simulation also cannot establish general performance across users, sensor conditions and physical workcells; no physical-robot evaluation is reported in this journal study.
Numerical reproduction would need the same scenes, recordings, perception models, calibration, manually chosen parameters and baseline information. Replaying identical time-stamped commands through competing estimators would help separate algorithm differences from differences in the input, while retaining participant-level structure in the analysis.
Testing and Real-World Use
A prospective hardware study could test online predictions with operator override, measuring false or missed intent handoffs and assisted task outcomes. In remote handling, the estimate could help an assistant prepare target-specific support, but the published simulation-replay results do not yet establish improved assisted task completion.
Paper Versions
Read the journal publication for the final evaluation, especially Table 2 and the replay protocol in Section 4. The earlier preprint remains available for version history and the pipeline illustration above; its smaller study should not be substituted for the journal results.