Erik Mischiatti

Index

Robotic cable manipulation from demonstration

A master's thesis research prototype for manipulating rigid and deformable objects: one kinesthetic demonstration, encoded as DHB invariant trajectories and combined with vision-based perception, is reused to insert Ethernet cables and interact with modular components.

Thesis: Leveraging Demonstration Learning and Invariant Representations for Modular Assembly and Cable Manipulation.

My contribution: author of the thesis.

Camera view of the board with ArUco markers and detected poses overlaid.

Fig. 1 Perception input: ArUco markers on the modular board, with the estimated poses overlaid on the camera view.

Red cable routed through printed clips on the perforated board.

Fig. 2 The task setting: a cable routed through 3D-printed clips on the board.

1 Overview

The thesis develops a documented robotic research prototype for manipulating rigid and deformable objects. The approach combines kinesthetic demonstration, invariant trajectory representations (DHB) and vision-based perception to insert Ethernet cables and interact with modular components under real-world constraints.

The public summary should be read as partially reproducible: results depend on local hardware, calibration and recorded data.

2 Approach and system architecture

Table 1 Key elements of the approach
ElementDescription
Invariant representation (DHB) Encodes motion in a bidirectional Denavit–Hartenberg space, so it can be reproduced across spatial configurations
Vision-based perception Intel RealSense depth sensing and ArUco markers provide 6D pose estimation and 3D awareness of the scene
Modular environment A board inspired by NIST benchmarks, to evaluate task generalization across reconfigurable layouts
Learning from demonstration Kinesthetic teaching to program cable-manipulation skills

The robotic platform integrates a Franka Emika Panda arm, an Intel RealSense D435i camera and a ROS-based control system. A perception module estimates the 6D pose of target objects, while DHB encodes the trajectory in a space-invariant form. Execution is coordinated through Behavior Trees and ROS action nodes.

System block architecture in three phases: teaching (operator guides the robot), encoding (demonstration trajectory, time and velocity, pre-processing, DHB) and reproducing (camera, detection, Behavior Trees, planner, Cartesian impedance controller, robot).
Fig. 3 System architecture: teaching, encoding and reproducing phases.
Experimental setup: the robotic arm system with the RealSense camera, and the assembly board system with ArUco markers, the camera view and the shared base frame.
Fig. 4 Experimental setup configuration: robotic arm system, camera view and assembly board system, linked through the base frame.

3 Experimental scenarios and results

Table 2 Three experimental scenarios, as reported
ScenarioResult
Clip insertion High-precision cable insertion into modular clips using learned trajectories
Full modular path Partial success in executing complex sequences across several modules, limited by joint constraints and cable slack
Board reconfiguration The system adapted to new board layouts, confirming the DHB invariance properties
Clip insertion scenario: forward sequence (Look, Get_cable, Create_pick_goal, Approach_cable, Insert_clip, Grab_clip, Close_clip, Lock_clip up and down) and inverse sequence (Alignment, Grab_clip, Unlock_clip, Look, Get_cable, Create_pick_goal, Approach_cable, Open_clip), beside the board with one clip.
Fig. 5 Clip insertion scenario: forward and inverse action sequences, and the board layout.
Full modular path scenario: a longer action sequence (Look, Get_cable, Create_pick_goal, Approach_cable, Approach_tube, Insert_tube, Step_back, Push, Step_back, Push, Approach_cable, continuing), beside the board with a cable routed through several modules.
Fig. 6 Full modular path scenario: the action sequence across several modules, and the board layout.

The repository is a research prototype: results, behavior and replayability depend on local hardware, calibration and recorded data.

Process

  1. Record demonstrations of the task through kinesthetic teaching.
  2. Filter the trajectory data to reduce noise and improve stability.
  3. Encode the demonstrations with the DHB invariant representation.
  4. Generalize the trajectories to new configurations.

Advantage

  1. A single demonstration can be reused in different spatial setups, thanks to the invariance properties of DHB.

What Figs. 7–11 show. The generalization step of the process above, applied to five motion steps of the clip task. Each plot shows a demonstrated trajectory (blue) and the trajectory generated from it for a new start and goal (red); points 1–2 mark the original start and goal, points 3–4 the new ones. The plots illustrate how a demonstration is re-targeted; they are not measurements of task success. 640 × 480, no sound.

Fig. 7 Looking for the clip.
Fig. 8 Grabbing the clip.
Fig. 9 Inserting the cable.
Fig. 10 Closing the clip.
Fig. 11 Unlocking the clip.

4 Technologies

Table 3 Technologies used
AreaTechnology
Robotics Franka Emika Panda arm for motion execution and kinesthetic teaching
Vision Intel RealSense D435i depth camera and ArUco markers for 6D pose estimation
Software ROS, Python, OpenCV and custom DHB libraries for planning and control
Modular hardware 3D-printed boards and inserts for flexible, reconfigurable testing

5 Conclusions and future work

The thesis explores whether invariant representations can support tasks such as cable insertion and modular assembly. DHB supports trajectory generalization across configurations; future work would need stronger robustness for deformable objects, occlusions and calibration changes.

  • Bimanual coordination for advanced wire routing.
  • Better visual tracking and tactile feedback.
  • Real-time pose refinement and learning for new parts.

6 Resources

  1. [1] Master-Thesis-Project repository (GitHub)